2024-05-15T13:38:09.582 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-15T13:38:09.582 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_leonidus@teuthology --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} --name leonidus-2024-05-15_11:41:37-fs-wip-lusov-quiesce-distro-default-smithi --archive /home/teuthworker/archive/leonidus-2024-05-15_11:41:37-fs-wip-lusov-quiesce-distro-default-smithi/7707053 -- /home/teuthworker/archive/leonidus-2024-05-15_11:41:37-fs-wip-lusov-quiesce-distro-default-smithi/7707053/orig.config.yaml 2024-05-15T13:38:09.646 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-15T13:38:09.738 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-05-15_11:41:37-fs-wip-lusov-quiesce-distro-default-smithi/7707053 branch: wip-lusov-quiesce description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} email: null first_in_suite: false job_id: '7707053' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: leonidus-2024-05-15_11:41:37-fs-wip-lusov-quiesce-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiesce ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 9b63866860f0b71216b2c0a73aedd2e0a8a5c98b subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 9b63866860f0b71216b2c0a73aedd2e0a8a5c98b kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiesce sha1: 9b63866860f0b71216b2c0a73aedd2e0a8a5c98b owner: scheduled_leonidus@teuthology priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 413 sha1: 9b63866860f0b71216b2c0a73aedd2e0a8a5c98b sleep_before_teardown: 0 subset: 200/111111 suite: fs suite_branch: wip-lusov-quiesce suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 9b63866860f0b71216b2c0a73aedd2e0a8a5c98b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi033.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDpGRjSdfgSUankN5XPJOkJiIA0VxcoD0zz+dYpyFodcyJ9HESornrG6dSqjjRMOFcL0ol70eJjW9yHQNOvhLew= smithi052.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIC9ClE4WGx0zqnxVqBs7s8RIJeNuykeOAvQLxVRq8oBQLOslIX9ebXR0a58y5Xt8Y2THLDUEP3/bpeR3LwTAcU= smithi081.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJpX0yPXiwAKdwnzHU3PD0i6JTvy/2hx17oCvLzOKWBgZgrGd3iPLtP/out24nBWNzZ+5/GtvB+6xlYXyWU0qKo= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-15_11:41:37 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-15T13:38:09.739 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/qa; will attempt to use it 2024-05-15T13:38:09.739 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/qa/tasks 2024-05-15T13:38:09.740 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-15T13:38:09.741 INFO:teuthology.task.internal:Checking packages... 2024-05-15T13:38:09.764 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '9b63866860f0b71216b2c0a73aedd2e0a8a5c98b' 2024-05-15T13:38:09.764 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-15T13:38:09.764 INFO:teuthology.packaging:ref: None 2024-05-15T13:38:09.764 INFO:teuthology.packaging:tag: None 2024-05-15T13:38:09.764 INFO:teuthology.packaging:branch: wip-lusov-quiesce 2024-05-15T13:38:09.764 INFO:teuthology.packaging:sha1: 9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-15T13:38:09.765 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiesce 2024-05-15T13:38:10.005 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3687.g9b638668 2024-05-15T13:38:10.007 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-15T13:38:10.013 INFO:teuthology.task.internal:no buildpackages task found 2024-05-15T13:38:10.013 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-15T13:38:10.020 INFO:teuthology.task.internal:Saving configuration 2024-05-15T13:38:10.042 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-15T13:38:10.048 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-15T13:38:10.071 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-15_11:41:37-fs-wip-lusov-quiesce-distro-default-smithi/7707053', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-15 13:33:06.182275', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDpGRjSdfgSUankN5XPJOkJiIA0VxcoD0zz+dYpyFodcyJ9HESornrG6dSqjjRMOFcL0ol70eJjW9yHQNOvhLew='} 2024-05-15T13:38:10.094 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi052.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-15_11:41:37-fs-wip-lusov-quiesce-distro-default-smithi/7707053', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-15 13:33:06.179570', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIC9ClE4WGx0zqnxVqBs7s8RIJeNuykeOAvQLxVRq8oBQLOslIX9ebXR0a58y5Xt8Y2THLDUEP3/bpeR3LwTAcU='} 2024-05-15T13:38:10.113 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi081.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-15_11:41:37-fs-wip-lusov-quiesce-distro-default-smithi/7707053', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-15 13:33:06.180855', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJpX0yPXiwAKdwnzHU3PD0i6JTvy/2hx17oCvLzOKWBgZgrGd3iPLtP/out24nBWNzZ+5/GtvB+6xlYXyWU0qKo='} 2024-05-15T13:38:10.113 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-15T13:38:10.119 INFO:teuthology.task.internal:roles: ubuntu@smithi033.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-15T13:38:10.119 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-15T13:38:10.119 INFO:teuthology.task.internal:roles: ubuntu@smithi081.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-15T13:38:10.119 INFO:teuthology.run_tasks:Running task console_log... 2024-05-15T13:38:10.225 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2e104e50d0>, signals=[15]) 2024-05-15T13:38:10.226 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-15T13:38:10.231 INFO:teuthology.task.internal:Opening connections... 2024-05-15T13:38:10.232 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi033.front.sepia.ceph.com 2024-05-15T13:38:10.233 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-15T13:38:10.312 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi052.front.sepia.ceph.com 2024-05-15T13:38:10.313 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-15T13:38:10.387 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi081.front.sepia.ceph.com 2024-05-15T13:38:10.387 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-15T13:38:10.461 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-15T13:38:10.467 DEBUG:teuthology.orchestra.run.smithi033:> uname -m 2024-05-15T13:38:10.486 INFO:teuthology.orchestra.run.smithi033.stdout:x86_64 2024-05-15T13:38:10.487 DEBUG:teuthology.orchestra.run.smithi033:> cat /etc/os-release 2024-05-15T13:38:10.544 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="CentOS Stream" 2024-05-15T13:38:10.545 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="9" 2024-05-15T13:38:10.545 INFO:teuthology.orchestra.run.smithi033.stdout:ID="centos" 2024-05-15T13:38:10.545 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE="rhel fedora" 2024-05-15T13:38:10.545 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="9" 2024-05-15T13:38:10.545 INFO:teuthology.orchestra.run.smithi033.stdout:PLATFORM_ID="platform:el9" 2024-05-15T13:38:10.545 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-15T13:38:10.545 INFO:teuthology.orchestra.run.smithi033.stdout:ANSI_COLOR="0;31" 2024-05-15T13:38:10.545 INFO:teuthology.orchestra.run.smithi033.stdout:LOGO="fedora-logo-icon" 2024-05-15T13:38:10.545 INFO:teuthology.orchestra.run.smithi033.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-15T13:38:10.545 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://centos.org/" 2024-05-15T13:38:10.545 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-15T13:38:10.545 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-15T13:38:10.546 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-15T13:38:10.546 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server 2024-05-15T13:38:10.571 DEBUG:teuthology.orchestra.run.smithi052:> uname -m 2024-05-15T13:38:10.590 INFO:teuthology.orchestra.run.smithi052.stdout:x86_64 2024-05-15T13:38:10.591 DEBUG:teuthology.orchestra.run.smithi052:> cat /etc/os-release 2024-05-15T13:38:10.649 INFO:teuthology.orchestra.run.smithi052.stdout:NAME="CentOS Stream" 2024-05-15T13:38:10.650 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION="9" 2024-05-15T13:38:10.650 INFO:teuthology.orchestra.run.smithi052.stdout:ID="centos" 2024-05-15T13:38:10.650 INFO:teuthology.orchestra.run.smithi052.stdout:ID_LIKE="rhel fedora" 2024-05-15T13:38:10.650 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION_ID="9" 2024-05-15T13:38:10.650 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM_ID="platform:el9" 2024-05-15T13:38:10.650 INFO:teuthology.orchestra.run.smithi052.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-15T13:38:10.650 INFO:teuthology.orchestra.run.smithi052.stdout:ANSI_COLOR="0;31" 2024-05-15T13:38:10.650 INFO:teuthology.orchestra.run.smithi052.stdout:LOGO="fedora-logo-icon" 2024-05-15T13:38:10.650 INFO:teuthology.orchestra.run.smithi052.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-15T13:38:10.650 INFO:teuthology.orchestra.run.smithi052.stdout:HOME_URL="https://centos.org/" 2024-05-15T13:38:10.651 INFO:teuthology.orchestra.run.smithi052.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-15T13:38:10.651 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-15T13:38:10.651 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-15T13:38:10.651 INFO:teuthology.lock.ops:Updating smithi052.front.sepia.ceph.com on lock server 2024-05-15T13:38:10.671 DEBUG:teuthology.orchestra.run.smithi081:> uname -m 2024-05-15T13:38:10.689 INFO:teuthology.orchestra.run.smithi081.stdout:x86_64 2024-05-15T13:38:10.689 DEBUG:teuthology.orchestra.run.smithi081:> cat /etc/os-release 2024-05-15T13:38:10.746 INFO:teuthology.orchestra.run.smithi081.stdout:NAME="CentOS Stream" 2024-05-15T13:38:10.746 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION="9" 2024-05-15T13:38:10.746 INFO:teuthology.orchestra.run.smithi081.stdout:ID="centos" 2024-05-15T13:38:10.746 INFO:teuthology.orchestra.run.smithi081.stdout:ID_LIKE="rhel fedora" 2024-05-15T13:38:10.746 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION_ID="9" 2024-05-15T13:38:10.746 INFO:teuthology.orchestra.run.smithi081.stdout:PLATFORM_ID="platform:el9" 2024-05-15T13:38:10.747 INFO:teuthology.orchestra.run.smithi081.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-15T13:38:10.747 INFO:teuthology.orchestra.run.smithi081.stdout:ANSI_COLOR="0;31" 2024-05-15T13:38:10.747 INFO:teuthology.orchestra.run.smithi081.stdout:LOGO="fedora-logo-icon" 2024-05-15T13:38:10.747 INFO:teuthology.orchestra.run.smithi081.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-15T13:38:10.747 INFO:teuthology.orchestra.run.smithi081.stdout:HOME_URL="https://centos.org/" 2024-05-15T13:38:10.747 INFO:teuthology.orchestra.run.smithi081.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-15T13:38:10.747 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-15T13:38:10.747 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-15T13:38:10.747 INFO:teuthology.lock.ops:Updating smithi081.front.sepia.ceph.com on lock server 2024-05-15T13:38:10.771 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-15T13:38:10.781 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-15T13:38:10.792 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-15T13:38:10.792 DEBUG:teuthology.orchestra.run.smithi033:> test '!' -e /home/ubuntu/cephtest 2024-05-15T13:38:10.795 DEBUG:teuthology.orchestra.run.smithi052:> test '!' -e /home/ubuntu/cephtest 2024-05-15T13:38:10.797 DEBUG:teuthology.orchestra.run.smithi081:> test '!' -e /home/ubuntu/cephtest 2024-05-15T13:38:10.814 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-15T13:38:10.819 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-15T13:38:10.819 DEBUG:teuthology.orchestra.run.smithi033:> test -z $(ls -A /var/lib/ceph) 2024-05-15T13:38:10.852 DEBUG:teuthology.orchestra.run.smithi052:> test -z $(ls -A /var/lib/ceph) 2024-05-15T13:38:10.854 DEBUG:teuthology.orchestra.run.smithi081:> test -z $(ls -A /var/lib/ceph) 2024-05-15T13:38:10.878 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-15T13:38:10.955 INFO:teuthology.run_tasks:Running task kernel... 2024-05-15T13:38:10.970 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-05-15T13:38:10.970 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-05-15T13:38:10.970 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-05-15T13:38:10.971 DEBUG:teuthology.orchestra.run.smithi033:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-15T13:38:10.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-15T13:38:10.988 DEBUG:teuthology.orchestra.run.smithi033:> uname -r 2024-05-15T13:38:11.047 INFO:teuthology.orchestra.run.smithi033.stdout:5.14.0-437.el9.x86_64 2024-05-15T13:38:11.047 INFO:teuthology.task.kernel:Running kernel on smithi033: 5.14.0-437.el9.x86_64 2024-05-15T13:38:11.047 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum install -y kernel 2024-05-15T13:38:17.968 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - BaseOS 1.5 MB/s | 8.1 MB 00:05 2024-05-15T13:38:19.686 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - AppStream 33 MB/s | 19 MB 00:00 2024-05-15T13:38:24.011 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - CRB 9.8 MB/s | 6.2 MB 00:00 2024-05-15T13:38:25.487 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 16 kB 00:00 2024-05-15T13:38:27.099 INFO:teuthology.orchestra.run.smithi033.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-05-15T13:38:32.405 INFO:teuthology.orchestra.run.smithi033.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-05-15T13:38:34.454 INFO:teuthology.orchestra.run.smithi033.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-15T13:38:34.454 INFO:teuthology.orchestra.run.smithi033.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-15T13:38:34.545 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-15T13:38:34.550 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-15T13:38:34.550 INFO:teuthology.orchestra.run.smithi033.stdout: Package Architecture Version Repository Size 2024-05-15T13:38:34.551 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-15T13:38:34.551 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-05-15T13:38:34.551 INFO:teuthology.orchestra.run.smithi033.stdout: kernel x86_64 5.14.0-446.el9 baseos 550 k 2024-05-15T13:38:34.551 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-05-15T13:38:34.551 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-core x86_64 5.14.0-446.el9 baseos 15 M 2024-05-15T13:38:34.551 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-modules x86_64 5.14.0-446.el9 baseos 34 M 2024-05-15T13:38:34.551 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-modules-core x86_64 5.14.0-446.el9 baseos 28 M 2024-05-15T13:38:34.551 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:38:34.551 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-05-15T13:38:34.551 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-15T13:38:34.551 INFO:teuthology.orchestra.run.smithi033.stdout:Install 4 Packages 2024-05-15T13:38:34.551 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:38:34.551 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 78 M 2024-05-15T13:38:34.551 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 123 M 2024-05-15T13:38:34.551 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-05-15T13:38:35.339 INFO:teuthology.orchestra.run.smithi033.stdout:(1/4): kernel-5.14.0-446.el9.x86_64.rpm 818 kB/s | 550 kB 00:00 2024-05-15T13:38:49.980 INFO:teuthology.orchestra.run.smithi033.stdout:(2/4): kernel-modules-5.14.0-446.el9.x86_64.rpm 2.2 MB/s | 34 MB 00:15 2024-05-15T13:38:50.396 INFO:teuthology.orchestra.run.smithi033.stdout:(3/4): kernel-core-5.14.0-446.el9.x86_64.rpm 1.0 MB/s | 15 MB 00:15 2024-05-15T13:38:55.829 INFO:teuthology.orchestra.run.smithi033.stdout:(4/4): kernel-modules-core-5.14.0-446.el9.x86_6 1.4 MB/s | 28 MB 00:20 2024-05-15T13:38:55.829 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-05-15T13:38:55.830 INFO:teuthology.orchestra.run.smithi033.stdout:Total 3.7 MB/s | 78 MB 00:21 2024-05-15T13:38:56.544 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-05-15T13:38:56.956 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-05-15T13:38:56.956 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-05-15T13:38:57.899 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-05-15T13:38:57.899 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-05-15T13:39:00.416 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-05-15T13:39:01.043 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : kernel-modules-core-5.14.0-446.el9.x86_64 1/4 2024-05-15T13:39:01.129 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-15T13:39:04.008 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-15T13:39:04.147 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-15T13:39:09.232 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-15T13:39:09.284 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : kernel-5.14.0-446.el9.x86_64 4/4 2024-05-15T13:39:14.377 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-15T13:39:45.358 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-core-5.14.0-446.el9.x86_64 4/4 2024-05-15T13:39:45.398 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-modules-5.14.0-446.el9.x86_64 4/4 2024-05-15T13:39:45.682 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-5.14.0-446.el9.x86_64 4/4 2024-05-15T13:39:45.683 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : kernel-5.14.0-446.el9.x86_64 1/4 2024-05-15T13:39:45.683 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : kernel-core-5.14.0-446.el9.x86_64 2/4 2024-05-15T13:39:45.683 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : kernel-modules-5.14.0-446.el9.x86_64 3/4 2024-05-15T13:39:46.431 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : kernel-modules-core-5.14.0-446.el9.x86_64 4/4 2024-05-15T13:39:46.431 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:39:46.431 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-05-15T13:39:46.431 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-5.14.0-446.el9.x86_64 2024-05-15T13:39:46.431 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-core-5.14.0-446.el9.x86_64 2024-05-15T13:39:46.431 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-modules-5.14.0-446.el9.x86_64 2024-05-15T13:39:46.432 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-modules-core-5.14.0-446.el9.x86_64 2024-05-15T13:39:46.432 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:39:46.432 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-15T13:39:46.777 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q kernel | sort -rV | head -n 1 2024-05-15T13:39:46.860 INFO:teuthology.orchestra.run.smithi033.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-15T13:39:46.860 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-15T13:39:46.860 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-446.el9.x86_64 2024-05-15T13:39:46.860 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-15T13:39:46.861 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-15T13:39:46.861 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-05-15T13:39:46.861 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi033.front.sepia.ceph.com, path=None, version=distro) 2024-05-15T13:39:46.861 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q kernel | sort -rV | head -n 1 2024-05-15T13:39:46.912 INFO:teuthology.orchestra.run.smithi033.stdout:kernel-5.14.0-446.el9.x86_64 2024-05-15T13:39:46.912 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-446.el9.x86_64 2024-05-15T13:39:46.912 DEBUG:teuthology.orchestra.run.smithi033:> sudo rpm -qi grub2-tools 2024-05-15T13:39:46.954 INFO:teuthology.orchestra.run.smithi033.stdout:Name : grub2-tools 2024-05-15T13:39:46.954 INFO:teuthology.orchestra.run.smithi033.stdout:Epoch : 1 2024-05-15T13:39:46.954 INFO:teuthology.orchestra.run.smithi033.stdout:Version : 2.06 2024-05-15T13:39:46.954 INFO:teuthology.orchestra.run.smithi033.stdout:Release : 68.el9 2024-05-15T13:39:46.954 INFO:teuthology.orchestra.run.smithi033.stdout:Architecture: x86_64 2024-05-15T13:39:46.954 INFO:teuthology.orchestra.run.smithi033.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-15T13:39:46.954 INFO:teuthology.orchestra.run.smithi033.stdout:Group : Unspecified 2024-05-15T13:39:46.954 INFO:teuthology.orchestra.run.smithi033.stdout:Size : 8264254 2024-05-15T13:39:46.954 INFO:teuthology.orchestra.run.smithi033.stdout:License : GPLv3+ 2024-05-15T13:39:46.955 INFO:teuthology.orchestra.run.smithi033.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-15T13:39:46.955 INFO:teuthology.orchestra.run.smithi033.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-15T13:39:46.955 INFO:teuthology.orchestra.run.smithi033.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-15T13:39:46.955 INFO:teuthology.orchestra.run.smithi033.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-15T13:39:46.955 INFO:teuthology.orchestra.run.smithi033.stdout:Packager : builder@centos.org 2024-05-15T13:39:46.955 INFO:teuthology.orchestra.run.smithi033.stdout:Vendor : CentOS 2024-05-15T13:39:46.955 INFO:teuthology.orchestra.run.smithi033.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-15T13:39:46.955 INFO:teuthology.orchestra.run.smithi033.stdout:Summary : Support tools for GRUB. 2024-05-15T13:39:46.955 INFO:teuthology.orchestra.run.smithi033.stdout:Description : 2024-05-15T13:39:46.955 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:39:46.955 INFO:teuthology.orchestra.run.smithi033.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-15T13:39:46.955 INFO:teuthology.orchestra.run.smithi033.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-15T13:39:46.955 INFO:teuthology.orchestra.run.smithi033.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-15T13:39:46.955 INFO:teuthology.orchestra.run.smithi033.stdout:hardware devices. 2024-05-15T13:39:46.956 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:39:46.956 INFO:teuthology.orchestra.run.smithi033.stdout:This subpackage provides tools for support of all platforms. 2024-05-15T13:39:46.957 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-15T13:39:46.957 INFO:teuthology.task.kernel:Updating grub on smithi033 to boot 5.14.0-446.el9.x86_64 2024-05-15T13:39:46.957 DEBUG:teuthology.orchestra.run.smithi033:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-15T13:39:47.620 INFO:teuthology.orchestra.run.smithi033.stderr:Generating grub configuration file ... 2024-05-15T13:39:49.068 INFO:teuthology.orchestra.run.smithi033.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-15T13:39:49.093 INFO:teuthology.orchestra.run.smithi033.stderr:done 2024-05-15T13:39:49.096 DEBUG:teuthology.orchestra.run.smithi033:> mktemp 2024-05-15T13:39:49.151 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.JZFIUa4fLr 2024-05-15T13:39:49.152 DEBUG:teuthology.orchestra.run.smithi033:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.JZFIUa4fLr 2024-05-15T13:39:49.217 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0666 /tmp/tmp.JZFIUa4fLr 2024-05-15T13:39:49.355 DEBUG:teuthology.orchestra.remote:smithi033:/tmp/tmp.JZFIUa4fLr is 6KB 2024-05-15T13:39:49.404 DEBUG:teuthology.orchestra.run.smithi033:> rm -fr /tmp/tmp.JZFIUa4fLr 2024-05-15T13:39:49.418 DEBUG:teuthology.orchestra.run.smithi033:> sudo /bin/ls /boot/loader/entries || true 2024-05-15T13:39:49.480 INFO:teuthology.orchestra.run.smithi033.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-15T13:39:49.481 INFO:teuthology.orchestra.run.smithi033.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-15T13:39:49.481 INFO:teuthology.orchestra.run.smithi033.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-15T13:39:49.481 INFO:teuthology.orchestra.run.smithi033.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64.conf 2024-05-15T13:39:49.482 DEBUG:teuthology.orchestra.run.smithi033:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-446.el9.x86_64 2024-05-15T13:39:49.627 DEBUG:teuthology.orchestra.run.smithi033:> sudo shutdown -r now 2024-05-15T13:40:19.630 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-15T13:40:19.630 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-05-15T13:40:19.631 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-15T13:40:38.050 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.33 2024-05-15T13:40:47.051 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-05-15T13:40:47.052 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-15T13:41:05.438 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.33 2024-05-15T13:41:17.439 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-05-15T13:41:17.440 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-15T13:41:18.102 DEBUG:teuthology.orchestra.run.smithi033:> true 2024-05-15T13:41:18.517 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-05-15T13:41:18.517 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-446.el9.x86_64"... 2024-05-15T13:41:18.517 DEBUG:teuthology.orchestra.run.smithi033:> uname -r 2024-05-15T13:41:18.533 INFO:teuthology.orchestra.run.smithi033.stdout:5.14.0-446.el9.x86_64 2024-05-15T13:41:18.533 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-446.el9.x86_64 vs 5.14.0-446.el9.x86_64 2024-05-15T13:41:18.533 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-15T13:41:18.533 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-15T13:41:19.534 DEBUG:teuthology.parallel:result is None 2024-05-15T13:41:19.535 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-15T13:41:19.542 INFO:teuthology.task.internal:Creating test directory... 2024-05-15T13:41:19.543 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-15T13:41:19.545 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-15T13:41:19.548 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-15T13:41:19.566 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-15T13:41:19.573 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-15T13:41:19.583 INFO:teuthology.task.internal:Creating archive directory... 2024-05-15T13:41:19.583 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-15T13:41:19.601 DEBUG:teuthology.orchestra.run.smithi052:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-15T13:41:19.606 DEBUG:teuthology.orchestra.run.smithi081:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-15T13:41:19.723 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-15T13:41:19.730 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-15T13:41:19.730 DEBUG:teuthology.orchestra.run.smithi052:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-15T13:41:19.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-15T13:41:19.747 DEBUG:teuthology.orchestra.run.smithi081:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-15T13:41:19.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-15T13:41:19.763 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-15T13:41:19.765 DEBUG:teuthology.orchestra.run.smithi052:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-15T13:41:19.789 DEBUG:teuthology.orchestra.run.smithi081:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-15T13:41:19.827 INFO:teuthology.orchestra.run.smithi052.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-15T13:41:19.841 INFO:teuthology.orchestra.run.smithi052.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-15T13:41:19.843 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-15T13:41:19.850 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-15T13:41:19.885 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-15T13:41:19.903 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-15T13:41:19.905 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-15T13:41:19.911 INFO:teuthology.task.internal:Configuring sudo... 2024-05-15T13:41:19.912 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-15T13:41:19.914 DEBUG:teuthology.orchestra.run.smithi052:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-15T13:41:19.916 DEBUG:teuthology.orchestra.run.smithi081:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-15T13:41:19.941 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-15T13:41:19.950 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-15T13:41:19.950 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-15T13:41:19.980 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-15T13:41:19.983 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-15T13:41:19.999 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-15T13:41:20.077 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-15T13:41:20.125 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-15T13:41:20.207 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-15T13:41:20.249 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-15T13:41:20.249 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-15T13:41:20.329 DEBUG:teuthology.orchestra.run.smithi052:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-15T13:41:20.371 DEBUG:teuthology.orchestra.run.smithi052:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-15T13:41:20.425 DEBUG:teuthology.orchestra.run.smithi052:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-15T13:41:20.510 DEBUG:teuthology.orchestra.run.smithi052:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-15T13:41:20.552 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-15T13:41:20.552 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-15T13:41:20.619 DEBUG:teuthology.orchestra.run.smithi081:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-15T13:41:20.663 DEBUG:teuthology.orchestra.run.smithi081:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-15T13:41:20.724 DEBUG:teuthology.orchestra.run.smithi081:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-15T13:41:20.808 DEBUG:teuthology.orchestra.run.smithi081:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-15T13:41:20.849 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-15T13:41:20.849 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-15T13:41:20.914 DEBUG:teuthology.orchestra.run.smithi033:> sudo service rsyslog restart 2024-05-15T13:41:20.917 DEBUG:teuthology.orchestra.run.smithi052:> sudo service rsyslog restart 2024-05-15T13:41:20.919 DEBUG:teuthology.orchestra.run.smithi081:> sudo service rsyslog restart 2024-05-15T13:41:20.955 INFO:teuthology.orchestra.run.smithi052.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-15T13:41:20.956 INFO:teuthology.orchestra.run.smithi033.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-15T13:41:20.990 INFO:teuthology.orchestra.run.smithi081.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-15T13:41:21.397 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-15T13:41:21.404 INFO:teuthology.task.internal:Starting timer... 2024-05-15T13:41:21.404 INFO:teuthology.run_tasks:Running task pcp... 2024-05-15T13:41:21.423 INFO:teuthology.run_tasks:Running task selinux... 2024-05-15T13:41:21.432 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-15T13:41:21.432 DEBUG:teuthology.orchestra.run.smithi033:> sudo service auditd rotate 2024-05-15T13:41:21.503 INFO:teuthology.orchestra.run.smithi033.stdout:Rotating logs: 2024-05-15T13:41:21.505 DEBUG:teuthology.orchestra.run.smithi052:> sudo service auditd rotate 2024-05-15T13:41:21.607 INFO:teuthology.orchestra.run.smithi052.stdout:Rotating logs: 2024-05-15T13:41:21.609 DEBUG:teuthology.orchestra.run.smithi081:> sudo service auditd rotate 2024-05-15T13:41:21.669 INFO:teuthology.orchestra.run.smithi081.stdout:Rotating logs: 2024-05-15T13:41:21.671 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-15T13:41:21.671 DEBUG:teuthology.orchestra.run.smithi033:> /usr/sbin/getenforce 2024-05-15T13:41:21.691 INFO:teuthology.orchestra.run.smithi033.stdout:Permissive 2024-05-15T13:41:21.691 DEBUG:teuthology.orchestra.run.smithi052:> /usr/sbin/getenforce 2024-05-15T13:41:21.716 INFO:teuthology.orchestra.run.smithi052.stdout:Permissive 2024-05-15T13:41:21.717 DEBUG:teuthology.orchestra.run.smithi081:> /usr/sbin/getenforce 2024-05-15T13:41:21.743 INFO:teuthology.orchestra.run.smithi081.stdout:Permissive 2024-05-15T13:41:21.743 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi033.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi052.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi081.front.sepia.ceph.com': 'permissive'} 2024-05-15T13:41:21.744 DEBUG:teuthology.orchestra.run.smithi033:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-15T13:41:21.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-15T13:41:21.770 DEBUG:teuthology.orchestra.run.smithi052:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-15T13:41:21.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-15T13:41:21.797 DEBUG:teuthology.orchestra.run.smithi081:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-15T13:41:21.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-15T13:41:21.824 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-15T13:41:21.824 DEBUG:teuthology.orchestra.run.smithi033:> sudo /usr/sbin/setenforce permissive 2024-05-15T13:41:21.850 DEBUG:teuthology.orchestra.run.smithi052:> sudo /usr/sbin/setenforce permissive 2024-05-15T13:41:21.876 DEBUG:teuthology.orchestra.run.smithi081:> sudo /usr/sbin/setenforce permissive 2024-05-15T13:41:21.902 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-15T13:41:21.911 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-15T13:41:21.916 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-15T13:41:22.091 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-15T13:41:22.105 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-15T13:41:22.107 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi033.front.sepia.ceph.com,smithi052.front.sepia.ceph.com,smithi081.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-15T13:46:04.648 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi033.front.sepia.ceph.com'), Remote(name='ubuntu@smithi052.front.sepia.ceph.com'), Remote(name='ubuntu@smithi081.front.sepia.ceph.com')] 2024-05-15T13:46:04.649 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-05-15T13:46:04.651 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-15T13:46:04.742 DEBUG:teuthology.orchestra.run.smithi033:> true 2024-05-15T13:46:04.826 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-05-15T13:46:04.827 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-05-15T13:46:04.827 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-15T13:46:04.946 DEBUG:teuthology.orchestra.run.smithi052:> true 2024-05-15T13:46:05.022 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-05-15T13:46:05.022 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-05-15T13:46:05.022 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-15T13:46:05.104 DEBUG:teuthology.orchestra.run.smithi081:> true 2024-05-15T13:46:05.184 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-05-15T13:46:05.184 INFO:teuthology.run_tasks:Running task clock... 2024-05-15T13:46:05.194 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-15T13:46:05.194 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-15T13:46:05.195 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-15T13:46:05.197 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-15T13:46:05.198 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-15T13:46:05.200 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-15T13:46:05.200 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-15T13:46:05.227 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-15T13:46:05.227 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-15T13:46:05.241 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-15T13:46:05.242 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-15T13:46:05.255 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-15T13:46:05.265 INFO:teuthology.orchestra.run.smithi052.stderr:sudo: ntpd: command not found 2024-05-15T13:46:05.267 INFO:teuthology.orchestra.run.smithi033.stderr:sudo: ntpd: command not found 2024-05-15T13:46:05.270 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-15T13:46:05.276 INFO:teuthology.orchestra.run.smithi052.stdout:506 Cannot talk to daemon 2024-05-15T13:46:05.279 INFO:teuthology.orchestra.run.smithi033.stdout:506 Cannot talk to daemon 2024-05-15T13:46:05.289 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-15T13:46:05.293 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-15T13:46:05.295 INFO:teuthology.orchestra.run.smithi081.stderr:sudo: ntpd: command not found 2024-05-15T13:46:05.303 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-15T13:46:05.306 INFO:teuthology.orchestra.run.smithi081.stdout:506 Cannot talk to daemon 2024-05-15T13:46:05.308 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-15T13:46:05.320 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-15T13:46:05.334 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-15T13:46:05.358 INFO:teuthology.orchestra.run.smithi052.stderr:bash: line 1: ntpq: command not found 2024-05-15T13:46:05.362 INFO:teuthology.orchestra.run.smithi052.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-15T13:46:05.362 INFO:teuthology.orchestra.run.smithi052.stdout:=============================================================================== 2024-05-15T13:46:05.363 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-15T13:46:05.363 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-15T13:46:05.363 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-15T13:46:05.363 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-15T13:46:05.364 INFO:teuthology.orchestra.run.smithi033.stderr:bash: line 1: ntpq: command not found 2024-05-15T13:46:05.367 INFO:teuthology.orchestra.run.smithi033.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-15T13:46:05.368 INFO:teuthology.orchestra.run.smithi033.stdout:=============================================================================== 2024-05-15T13:46:05.368 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-15T13:46:05.368 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-15T13:46:05.368 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-15T13:46:05.368 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-15T13:46:05.400 INFO:teuthology.orchestra.run.smithi081.stderr:bash: line 1: ntpq: command not found 2024-05-15T13:46:05.403 INFO:teuthology.orchestra.run.smithi081.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-15T13:46:05.403 INFO:teuthology.orchestra.run.smithi081.stdout:=============================================================================== 2024-05-15T13:46:05.403 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-15T13:46:05.403 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-15T13:46:05.403 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-15T13:46:05.403 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-15T13:46:05.404 INFO:teuthology.run_tasks:Running task install... 2024-05-15T13:46:05.413 DEBUG:teuthology.task.install:project ceph 2024-05-15T13:46:05.413 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '9b63866860f0b71216b2c0a73aedd2e0a8a5c98b'}} 2024-05-15T13:46:05.413 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '9b63866860f0b71216b2c0a73aedd2e0a8a5c98b'} 2024-05-15T13:46:05.413 INFO:teuthology.task.install:Using flavor: default 2024-05-15T13:46:05.419 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-15T13:46:05.419 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-15T13:46:05.419 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9b63866860f0b71216b2c0a73aedd2e0a8a5c98b', 'tag': None, 'wait_for_package': False} 2024-05-15T13:46:05.420 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-15T13:46:05.422 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9b63866860f0b71216b2c0a73aedd2e0a8a5c98b', 'tag': None, 'wait_for_package': False} 2024-05-15T13:46:05.422 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-15T13:46:05.423 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9b63866860f0b71216b2c0a73aedd2e0a8a5c98b', 'tag': None, 'wait_for_package': False} 2024-05-15T13:46:05.423 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-15T13:46:05.574 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/ 2024-05-15T13:46:05.574 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3687.g9b638668 2024-05-15T13:46:05.578 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/ 2024-05-15T13:46:05.578 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3687.g9b638668 2024-05-15T13:46:05.673 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/ 2024-05-15T13:46:05.673 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3687.g9b638668 2024-05-15T13:46:05.688 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-15T13:46:05.688 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-15T13:46:05.688 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-15T13:46:05.694 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-15T13:46:05.695 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-15T13:46:05.695 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-15T13:46:05.714 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-15T13:46:05.714 DEBUG:teuthology.orchestra.run.smithi081:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-15T13:46:05.721 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-15T13:46:05.721 DEBUG:teuthology.orchestra.run.smithi052:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-15T13:46:05.778 DEBUG:teuthology.orchestra.run.smithi081:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-15T13:46:05.781 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-15T13:46:05.781 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-15T13:46:05.781 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-15T13:46:05.786 DEBUG:teuthology.orchestra.run.smithi052:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-15T13:46:05.808 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-15T13:46:05.809 DEBUG:teuthology.orchestra.run.smithi033:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-15T13:46:05.850 DEBUG:teuthology.orchestra.run.smithi081:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-15T13:46:05.862 DEBUG:teuthology.orchestra.run.smithi052:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-15T13:46:05.876 DEBUG:teuthology.orchestra.run.smithi033:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-15T13:46:05.913 INFO:teuthology.orchestra.run.smithi081.stdout:check_obsoletes = 1 2024-05-15T13:46:05.915 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum clean all 2024-05-15T13:46:05.926 INFO:teuthology.orchestra.run.smithi052.stdout:check_obsoletes = 1 2024-05-15T13:46:05.927 DEBUG:teuthology.orchestra.run.smithi052:> sudo yum clean all 2024-05-15T13:46:05.953 DEBUG:teuthology.orchestra.run.smithi033:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-15T13:46:06.019 INFO:teuthology.orchestra.run.smithi033.stdout:check_obsoletes = 1 2024-05-15T13:46:06.020 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum clean all 2024-05-15T13:46:06.164 INFO:teuthology.orchestra.run.smithi081.stdout:57 files removed 2024-05-15T13:46:06.164 INFO:teuthology.orchestra.run.smithi052.stdout:57 files removed 2024-05-15T13:46:06.190 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-15T13:46:06.192 DEBUG:teuthology.orchestra.run.smithi052:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-15T13:46:06.270 INFO:teuthology.orchestra.run.smithi033.stdout:57 files removed 2024-05-15T13:46:06.294 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-15T13:46:07.044 INFO:teuthology.orchestra.run.smithi081.stdout:ceph packages for x86_64 141 kB/s | 84 kB 00:00 2024-05-15T13:46:07.130 INFO:teuthology.orchestra.run.smithi033.stdout:ceph packages for x86_64 153 kB/s | 84 kB 00:00 2024-05-15T13:46:07.139 INFO:teuthology.orchestra.run.smithi052.stdout:ceph packages for x86_64 124 kB/s | 84 kB 00:00 2024-05-15T13:46:07.494 INFO:teuthology.orchestra.run.smithi081.stdout:ceph noarch packages 29 kB/s | 12 kB 00:00 2024-05-15T13:46:07.590 INFO:teuthology.orchestra.run.smithi052.stdout:ceph noarch packages 29 kB/s | 12 kB 00:00 2024-05-15T13:46:07.723 INFO:teuthology.orchestra.run.smithi033.stdout:ceph noarch packages 22 kB/s | 12 kB 00:00 2024-05-15T13:46:08.003 INFO:teuthology.orchestra.run.smithi081.stdout:ceph source packages 4.0 kB/s | 1.9 kB 00:00 2024-05-15T13:46:08.099 INFO:teuthology.orchestra.run.smithi052.stdout:ceph source packages 4.0 kB/s | 1.9 kB 00:00 2024-05-15T13:46:08.241 INFO:teuthology.orchestra.run.smithi033.stdout:ceph source packages 4.0 kB/s | 1.9 kB 00:00 2024-05-15T13:46:08.620 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.1 MB 00:00 2024-05-15T13:46:08.934 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.1 MB 00:00 2024-05-15T13:46:09.899 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.1 MB 00:01 2024-05-15T13:46:10.629 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-05-15T13:46:11.825 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-05-15T13:46:12.344 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - AppStream 8.5 MB/s | 19 MB 00:02 2024-05-15T13:46:16.151 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-05-15T13:46:16.205 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - CRB 3.1 MB/s | 6.2 MB 00:01 2024-05-15T13:46:16.853 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - CRB 8.2 MB/s | 6.2 MB 00:00 2024-05-15T13:46:17.564 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - Extras packages 78 kB/s | 16 kB 00:00 2024-05-15T13:46:17.593 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - Extras packages 69 kB/s | 16 kB 00:00 2024-05-15T13:46:18.412 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - Extras packages 67 kB/s | 16 kB 00:00 2024-05-15T13:46:18.635 INFO:teuthology.orchestra.run.smithi081.stdout:Extra Packages for Enterprise Linux 23 MB/s | 21 MB 00:00 2024-05-15T13:46:18.954 INFO:teuthology.orchestra.run.smithi052.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-05-15T13:46:19.730 INFO:teuthology.orchestra.run.smithi033.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-05-15T13:46:23.708 INFO:teuthology.orchestra.run.smithi081.stdout:lab-extras 40 kB/s | 1.7 kB 00:00 2024-05-15T13:46:24.295 INFO:teuthology.orchestra.run.smithi052.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-15T13:46:25.046 INFO:teuthology.orchestra.run.smithi033.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-05-15T13:46:25.959 INFO:teuthology.orchestra.run.smithi081.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-15T13:46:25.960 INFO:teuthology.orchestra.run.smithi081.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-15T13:46:25.970 INFO:teuthology.orchestra.run.smithi081.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-15T13:46:25.971 INFO:teuthology.orchestra.run.smithi081.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-15T13:46:25.972 INFO:teuthology.orchestra.run.smithi081.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-15T13:46:25.973 INFO:teuthology.orchestra.run.smithi081.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-15T13:46:25.974 INFO:teuthology.orchestra.run.smithi081.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-15T13:46:25.977 INFO:teuthology.orchestra.run.smithi081.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-15T13:46:25.977 INFO:teuthology.orchestra.run.smithi081.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-15T13:46:26.038 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-05-15T13:46:26.055 INFO:teuthology.orchestra.run.smithi081.stdout:======================================================================================= 2024-05-15T13:46:26.055 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-05-15T13:46:26.055 INFO:teuthology.orchestra.run.smithi081.stdout:======================================================================================= 2024-05-15T13:46:26.055 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-05-15T13:46:26.055 INFO:teuthology.orchestra.run.smithi081.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-15T13:46:26.055 INFO:teuthology.orchestra.run.smithi081.stdout: ceph x86_64 2:19.0.0-3687.g9b638668.el9 ceph 6.6 k 2024-05-15T13:46:26.055 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-base x86_64 2:19.0.0-3687.g9b638668.el9 ceph 5.4 M 2024-05-15T13:46:26.055 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-fuse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 891 k 2024-05-15T13:46:26.056 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3687.g9b638668.el9 ceph 145 k 2024-05-15T13:46:26.056 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr x86_64 2:19.0.0-3687.g9b638668.el9 ceph 1.5 M 2024-05-15T13:46:26.056 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 143 k 2024-05-15T13:46:26.056 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 3.5 M 2024-05-15T13:46:26.056 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 7.4 M 2024-05-15T13:46:26.056 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-rook noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 49 k 2024-05-15T13:46:26.056 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-radosgw x86_64 2:19.0.0-3687.g9b638668.el9 ceph 11 M 2024-05-15T13:46:26.056 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-test x86_64 2:19.0.0-3687.g9b638668.el9 ceph 49 M 2024-05-15T13:46:26.056 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-volume noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 273 k 2024-05-15T13:46:26.056 INFO:teuthology.orchestra.run.smithi081.stdout: cephadm noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 763 k 2024-05-15T13:46:26.056 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-mirror x86_64 2:19.0.0-3687.g9b638668.el9 ceph 225 k 2024-05-15T13:46:26.056 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-shell x86_64 2:19.0.0-3687.g9b638668.el9 ceph 27 k 2024-05-15T13:46:26.056 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-top noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 18 k 2024-05-15T13:46:26.056 INFO:teuthology.orchestra.run.smithi081.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-15T13:46:26.057 INFO:teuthology.orchestra.run.smithi081.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-15T13:46:26.057 INFO:teuthology.orchestra.run.smithi081.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-15T13:46:26.057 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs-devel x86_64 2:19.0.0-3687.g9b638668.el9 ceph 32 k 2024-05-15T13:46:26.057 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 748 k 2024-05-15T13:46:26.057 INFO:teuthology.orchestra.run.smithi081.stdout: librados-devel x86_64 2:19.0.0-3687.g9b638668.el9 ceph 127 k 2024-05-15T13:46:26.057 INFO:teuthology.orchestra.run.smithi081.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-15T13:46:26.057 INFO:teuthology.orchestra.run.smithi081.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-15T13:46:26.057 INFO:teuthology.orchestra.run.smithi081.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-15T13:46:26.057 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-15T13:46:26.057 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-15T13:46:26.057 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-15T13:46:26.057 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cephfs x86_64 2:19.0.0-3687.g9b638668.el9 ceph 161 k 2024-05-15T13:46:26.057 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rados x86_64 2:19.0.0-3687.g9b638668.el9 ceph 321 k 2024-05-15T13:46:26.058 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rbd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 299 k 2024-05-15T13:46:26.058 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rgw x86_64 2:19.0.0-3687.g9b638668.el9 ceph 100 k 2024-05-15T13:46:26.058 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-fuse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 85 k 2024-05-15T13:46:26.058 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-mirror x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.1 M 2024-05-15T13:46:26.058 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-nbd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 170 k 2024-05-15T13:46:26.058 INFO:teuthology.orchestra.run.smithi081.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-15T13:46:26.058 INFO:teuthology.orchestra.run.smithi081.stdout:Upgrading: 2024-05-15T13:46:26.058 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-15T13:46:26.058 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-15T13:46:26.058 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-15T13:46:26.058 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-15T13:46:26.058 INFO:teuthology.orchestra.run.smithi081.stdout: librados2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.4 M 2024-05-15T13:46:26.058 INFO:teuthology.orchestra.run.smithi081.stdout: librbd1 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.1 M 2024-05-15T13:46:26.059 INFO:teuthology.orchestra.run.smithi081.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-15T13:46:26.059 INFO:teuthology.orchestra.run.smithi081.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-15T13:46:26.059 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-05-15T13:46:26.059 INFO:teuthology.orchestra.run.smithi081.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-15T13:46:26.059 INFO:teuthology.orchestra.run.smithi081.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-15T13:46:26.059 INFO:teuthology.orchestra.run.smithi081.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-15T13:46:26.059 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-common x86_64 2:19.0.0-3687.g9b638668.el9 ceph 21 M 2024-05-15T13:46:26.059 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 34 k 2024-05-15T13:46:26.059 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mds x86_64 2:19.0.0-3687.g9b638668.el9 ceph 2.4 M 2024-05-15T13:46:26.059 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 274 k 2024-05-15T13:46:26.059 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mon x86_64 2:19.0.0-3687.g9b638668.el9 ceph 4.7 M 2024-05-15T13:46:26.059 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-osd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 17 M 2024-05-15T13:46:26.059 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 16 k 2024-05-15T13:46:26.059 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-selinux x86_64 2:19.0.0-3687.g9b638668.el9 ceph 25 k 2024-05-15T13:46:26.060 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-15T13:46:26.060 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-15T13:46:26.060 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-15T13:46:26.060 INFO:teuthology.orchestra.run.smithi081.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-15T13:46:26.060 INFO:teuthology.orchestra.run.smithi081.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-15T13:46:26.060 INFO:teuthology.orchestra.run.smithi081.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-15T13:46:26.060 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-15T13:46:26.060 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-15T13:46:26.060 INFO:teuthology.orchestra.run.smithi081.stdout: libcephsqlite x86_64 2:19.0.0-3687.g9b638668.el9 ceph 163 k 2024-05-15T13:46:26.060 INFO:teuthology.orchestra.run.smithi081.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-15T13:46:26.060 INFO:teuthology.orchestra.run.smithi081.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-15T13:46:26.060 INFO:teuthology.orchestra.run.smithi081.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-15T13:46:26.060 INFO:teuthology.orchestra.run.smithi081.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-15T13:46:26.061 INFO:teuthology.orchestra.run.smithi081.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-15T13:46:26.061 INFO:teuthology.orchestra.run.smithi081.stdout: libradosstriper1 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 501 k 2024-05-15T13:46:26.061 INFO:teuthology.orchestra.run.smithi081.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-15T13:46:26.061 INFO:teuthology.orchestra.run.smithi081.stdout: librgw2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 5.4 M 2024-05-15T13:46:26.061 INFO:teuthology.orchestra.run.smithi081.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-15T13:46:26.061 INFO:teuthology.orchestra.run.smithi081.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-15T13:46:26.061 INFO:teuthology.orchestra.run.smithi081.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-15T13:46:26.061 INFO:teuthology.orchestra.run.smithi081.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-15T13:46:26.061 INFO:teuthology.orchestra.run.smithi081.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-15T13:46:26.061 INFO:teuthology.orchestra.run.smithi081.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-15T13:46:26.061 INFO:teuthology.orchestra.run.smithi081.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-15T13:46:26.061 INFO:teuthology.orchestra.run.smithi081.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-15T13:46:26.061 INFO:teuthology.orchestra.run.smithi081.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-15T13:46:26.062 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-15T13:46:26.062 INFO:teuthology.orchestra.run.smithi081.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-15T13:46:26.062 INFO:teuthology.orchestra.run.smithi081.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-15T13:46:26.062 INFO:teuthology.orchestra.run.smithi081.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-15T13:46:26.062 INFO:teuthology.orchestra.run.smithi081.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-15T13:46:26.062 INFO:teuthology.orchestra.run.smithi081.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-15T13:46:26.062 INFO:teuthology.orchestra.run.smithi081.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-15T13:46:26.062 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-15T13:46:26.062 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-argparse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 45 k 2024-05-15T13:46:26.062 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-common x86_64 2:19.0.0-3687.g9b638668.el9 ceph 136 k 2024-05-15T13:46:26.062 INFO:teuthology.orchestra.run.smithi081.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-15T13:46:26.062 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-15T13:46:26.062 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-15T13:46:26.062 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-15T13:46:26.063 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-15T13:46:26.063 INFO:teuthology.orchestra.run.smithi081.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-15T13:46:26.063 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-15T13:46:26.063 INFO:teuthology.orchestra.run.smithi081.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-15T13:46:26.063 INFO:teuthology.orchestra.run.smithi081.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-15T13:46:26.063 INFO:teuthology.orchestra.run.smithi081.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-15T13:46:26.063 INFO:teuthology.orchestra.run.smithi081.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-15T13:46:26.063 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-15T13:46:26.063 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-15T13:46:26.063 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-15T13:46:26.063 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-15T13:46:26.063 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-15T13:46:26.063 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-15T13:46:26.064 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-15T13:46:26.064 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-15T13:46:26.064 INFO:teuthology.orchestra.run.smithi081.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-15T13:46:26.064 INFO:teuthology.orchestra.run.smithi081.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-15T13:46:26.064 INFO:teuthology.orchestra.run.smithi081.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-15T13:46:26.064 INFO:teuthology.orchestra.run.smithi081.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-15T13:46:26.064 INFO:teuthology.orchestra.run.smithi081.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-15T13:46:26.064 INFO:teuthology.orchestra.run.smithi081.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-15T13:46:26.064 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-15T13:46:26.064 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-15T13:46:26.064 INFO:teuthology.orchestra.run.smithi081.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-15T13:46:26.064 INFO:teuthology.orchestra.run.smithi081.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-15T13:46:26.064 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-15T13:46:26.064 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-15T13:46:26.064 INFO:teuthology.orchestra.run.smithi081.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-15T13:46:26.065 INFO:teuthology.orchestra.run.smithi081.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-15T13:46:26.065 INFO:teuthology.orchestra.run.smithi081.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-15T13:46:26.065 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-15T13:46:26.065 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-15T13:46:26.065 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-15T13:46:26.065 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-15T13:46:26.065 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-15T13:46:26.065 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-15T13:46:26.065 INFO:teuthology.orchestra.run.smithi081.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-15T13:46:26.065 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-15T13:46:26.065 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-15T13:46:26.065 INFO:teuthology.orchestra.run.smithi081.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-15T13:46:26.065 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-15T13:46:26.065 INFO:teuthology.orchestra.run.smithi081.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-15T13:46:26.066 INFO:teuthology.orchestra.run.smithi081.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-15T13:46:26.066 INFO:teuthology.orchestra.run.smithi081.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-15T13:46:26.066 INFO:teuthology.orchestra.run.smithi081.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-15T13:46:26.066 INFO:teuthology.orchestra.run.smithi081.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-15T13:46:26.066 INFO:teuthology.orchestra.run.smithi081.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-15T13:46:26.066 INFO:teuthology.orchestra.run.smithi081.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-15T13:46:26.066 INFO:teuthology.orchestra.run.smithi081.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-15T13:46:26.066 INFO:teuthology.orchestra.run.smithi081.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-15T13:46:26.066 INFO:teuthology.orchestra.run.smithi081.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-15T13:46:26.066 INFO:teuthology.orchestra.run.smithi081.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-15T13:46:26.066 INFO:teuthology.orchestra.run.smithi081.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-15T13:46:26.066 INFO:teuthology.orchestra.run.smithi081.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-15T13:46:26.067 INFO:teuthology.orchestra.run.smithi081.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-15T13:46:26.067 INFO:teuthology.orchestra.run.smithi081.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-15T13:46:26.067 INFO:teuthology.orchestra.run.smithi081.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-15T13:46:26.067 INFO:teuthology.orchestra.run.smithi081.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-15T13:46:26.067 INFO:teuthology.orchestra.run.smithi081.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-15T13:46:26.067 INFO:teuthology.orchestra.run.smithi081.stdout:Installing weak dependencies: 2024-05-15T13:46:26.067 INFO:teuthology.orchestra.run.smithi081.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-15T13:46:26.067 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-15T13:46:26.067 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-05-15T13:46:26.067 INFO:teuthology.orchestra.run.smithi081.stdout:======================================================================================= 2024-05-15T13:46:26.067 INFO:teuthology.orchestra.run.smithi081.stdout:Install 148 Packages 2024-05-15T13:46:26.067 INFO:teuthology.orchestra.run.smithi081.stdout:Upgrade 8 Packages 2024-05-15T13:46:26.067 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-15T13:46:26.068 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 226 M 2024-05-15T13:46:26.068 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-05-15T13:46:26.618 INFO:teuthology.orchestra.run.smithi081.stdout:(1/156): ceph-19.0.0-3687.g9b638668.el9.x86_64. 37 kB/s | 6.6 kB 00:00 2024-05-15T13:46:26.664 INFO:teuthology.orchestra.run.smithi052.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-15T13:46:26.665 INFO:teuthology.orchestra.run.smithi052.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-15T13:46:26.677 INFO:teuthology.orchestra.run.smithi052.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-15T13:46:26.678 INFO:teuthology.orchestra.run.smithi052.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-15T13:46:26.678 INFO:teuthology.orchestra.run.smithi052.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-15T13:46:26.679 INFO:teuthology.orchestra.run.smithi052.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-15T13:46:26.681 INFO:teuthology.orchestra.run.smithi052.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-15T13:46:26.683 INFO:teuthology.orchestra.run.smithi052.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-15T13:46:26.684 INFO:teuthology.orchestra.run.smithi052.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-15T13:46:26.750 INFO:teuthology.orchestra.run.smithi052.stdout:Dependencies resolved. 2024-05-15T13:46:26.767 INFO:teuthology.orchestra.run.smithi052.stdout:======================================================================================= 2024-05-15T13:46:26.767 INFO:teuthology.orchestra.run.smithi052.stdout: Package Arch Version Repository Size 2024-05-15T13:46:26.767 INFO:teuthology.orchestra.run.smithi052.stdout:======================================================================================= 2024-05-15T13:46:26.768 INFO:teuthology.orchestra.run.smithi052.stdout:Installing: 2024-05-15T13:46:26.768 INFO:teuthology.orchestra.run.smithi052.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-15T13:46:26.768 INFO:teuthology.orchestra.run.smithi052.stdout: ceph x86_64 2:19.0.0-3687.g9b638668.el9 ceph 6.6 k 2024-05-15T13:46:26.768 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-base x86_64 2:19.0.0-3687.g9b638668.el9 ceph 5.4 M 2024-05-15T13:46:26.768 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-fuse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 891 k 2024-05-15T13:46:26.768 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3687.g9b638668.el9 ceph 145 k 2024-05-15T13:46:26.768 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr x86_64 2:19.0.0-3687.g9b638668.el9 ceph 1.5 M 2024-05-15T13:46:26.768 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 143 k 2024-05-15T13:46:26.768 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 3.5 M 2024-05-15T13:46:26.768 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 7.4 M 2024-05-15T13:46:26.768 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-rook noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 49 k 2024-05-15T13:46:26.768 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-radosgw x86_64 2:19.0.0-3687.g9b638668.el9 ceph 11 M 2024-05-15T13:46:26.769 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-test x86_64 2:19.0.0-3687.g9b638668.el9 ceph 49 M 2024-05-15T13:46:26.769 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-volume noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 273 k 2024-05-15T13:46:26.769 INFO:teuthology.orchestra.run.smithi052.stdout: cephadm noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 763 k 2024-05-15T13:46:26.769 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-mirror x86_64 2:19.0.0-3687.g9b638668.el9 ceph 225 k 2024-05-15T13:46:26.769 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-shell x86_64 2:19.0.0-3687.g9b638668.el9 ceph 27 k 2024-05-15T13:46:26.769 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-top noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 18 k 2024-05-15T13:46:26.769 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-15T13:46:26.769 INFO:teuthology.orchestra.run.smithi052.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-15T13:46:26.769 INFO:teuthology.orchestra.run.smithi052.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-15T13:46:26.769 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs-devel x86_64 2:19.0.0-3687.g9b638668.el9 ceph 32 k 2024-05-15T13:46:26.770 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 748 k 2024-05-15T13:46:26.770 INFO:teuthology.orchestra.run.smithi052.stdout: librados-devel x86_64 2:19.0.0-3687.g9b638668.el9 ceph 127 k 2024-05-15T13:46:26.770 INFO:teuthology.orchestra.run.smithi052.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-15T13:46:26.770 INFO:teuthology.orchestra.run.smithi052.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-15T13:46:26.770 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-15T13:46:26.770 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-15T13:46:26.770 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-15T13:46:26.770 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-15T13:46:26.770 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cephfs x86_64 2:19.0.0-3687.g9b638668.el9 ceph 161 k 2024-05-15T13:46:26.770 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rados x86_64 2:19.0.0-3687.g9b638668.el9 ceph 321 k 2024-05-15T13:46:26.771 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rbd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 299 k 2024-05-15T13:46:26.771 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rgw x86_64 2:19.0.0-3687.g9b638668.el9 ceph 100 k 2024-05-15T13:46:26.771 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-fuse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 85 k 2024-05-15T13:46:26.771 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-mirror x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.1 M 2024-05-15T13:46:26.771 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-nbd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 170 k 2024-05-15T13:46:26.771 INFO:teuthology.orchestra.run.smithi052.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-15T13:46:26.771 INFO:teuthology.orchestra.run.smithi052.stdout:Upgrading: 2024-05-15T13:46:26.771 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-15T13:46:26.771 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-15T13:46:26.771 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-15T13:46:26.771 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-15T13:46:26.771 INFO:teuthology.orchestra.run.smithi052.stdout: librados2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.4 M 2024-05-15T13:46:26.771 INFO:teuthology.orchestra.run.smithi052.stdout: librbd1 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.1 M 2024-05-15T13:46:26.771 INFO:teuthology.orchestra.run.smithi052.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-15T13:46:26.772 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-15T13:46:26.772 INFO:teuthology.orchestra.run.smithi052.stdout:Installing dependencies: 2024-05-15T13:46:26.772 INFO:teuthology.orchestra.run.smithi052.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-15T13:46:26.772 INFO:teuthology.orchestra.run.smithi052.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-15T13:46:26.772 INFO:teuthology.orchestra.run.smithi052.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-15T13:46:26.772 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-common x86_64 2:19.0.0-3687.g9b638668.el9 ceph 21 M 2024-05-15T13:46:26.772 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 34 k 2024-05-15T13:46:26.772 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mds x86_64 2:19.0.0-3687.g9b638668.el9 ceph 2.4 M 2024-05-15T13:46:26.772 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 274 k 2024-05-15T13:46:26.772 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mon x86_64 2:19.0.0-3687.g9b638668.el9 ceph 4.7 M 2024-05-15T13:46:26.772 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-osd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 17 M 2024-05-15T13:46:26.772 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 16 k 2024-05-15T13:46:26.772 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-selinux x86_64 2:19.0.0-3687.g9b638668.el9 ceph 25 k 2024-05-15T13:46:26.772 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-15T13:46:26.773 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-15T13:46:26.773 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-15T13:46:26.773 INFO:teuthology.orchestra.run.smithi052.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-15T13:46:26.773 INFO:teuthology.orchestra.run.smithi052.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-15T13:46:26.773 INFO:teuthology.orchestra.run.smithi052.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-15T13:46:26.773 INFO:teuthology.orchestra.run.smithi052.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-15T13:46:26.773 INFO:teuthology.orchestra.run.smithi052.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-15T13:46:26.773 INFO:teuthology.orchestra.run.smithi052.stdout: libcephsqlite x86_64 2:19.0.0-3687.g9b638668.el9 ceph 163 k 2024-05-15T13:46:26.773 INFO:teuthology.orchestra.run.smithi052.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-15T13:46:26.773 INFO:teuthology.orchestra.run.smithi052.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-15T13:46:26.773 INFO:teuthology.orchestra.run.smithi052.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-15T13:46:26.773 INFO:teuthology.orchestra.run.smithi052.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-15T13:46:26.773 INFO:teuthology.orchestra.run.smithi052.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-15T13:46:26.773 INFO:teuthology.orchestra.run.smithi052.stdout: libradosstriper1 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 501 k 2024-05-15T13:46:26.774 INFO:teuthology.orchestra.run.smithi052.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-15T13:46:26.774 INFO:teuthology.orchestra.run.smithi052.stdout: librgw2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 5.4 M 2024-05-15T13:46:26.774 INFO:teuthology.orchestra.run.smithi052.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-15T13:46:26.774 INFO:teuthology.orchestra.run.smithi052.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-15T13:46:26.774 INFO:teuthology.orchestra.run.smithi052.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-15T13:46:26.774 INFO:teuthology.orchestra.run.smithi052.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-15T13:46:26.774 INFO:teuthology.orchestra.run.smithi052.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-15T13:46:26.774 INFO:teuthology.orchestra.run.smithi052.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-15T13:46:26.774 INFO:teuthology.orchestra.run.smithi052.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-15T13:46:26.774 INFO:teuthology.orchestra.run.smithi052.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-15T13:46:26.774 INFO:teuthology.orchestra.run.smithi052.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-15T13:46:26.774 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-15T13:46:26.774 INFO:teuthology.orchestra.run.smithi052.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-15T13:46:26.774 INFO:teuthology.orchestra.run.smithi052.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-15T13:46:26.775 INFO:teuthology.orchestra.run.smithi052.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-15T13:46:26.775 INFO:teuthology.orchestra.run.smithi052.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-15T13:46:26.775 INFO:teuthology.orchestra.run.smithi052.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-15T13:46:26.775 INFO:teuthology.orchestra.run.smithi052.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-15T13:46:26.775 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-15T13:46:26.775 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-argparse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 45 k 2024-05-15T13:46:26.775 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-common x86_64 2:19.0.0-3687.g9b638668.el9 ceph 136 k 2024-05-15T13:46:26.775 INFO:teuthology.orchestra.run.smithi052.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-15T13:46:26.775 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-15T13:46:26.775 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-15T13:46:26.775 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-15T13:46:26.775 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-15T13:46:26.775 INFO:teuthology.orchestra.run.smithi052.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-15T13:46:26.775 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-15T13:46:26.775 INFO:teuthology.orchestra.run.smithi052.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-15T13:46:26.776 INFO:teuthology.orchestra.run.smithi052.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-15T13:46:26.776 INFO:teuthology.orchestra.run.smithi052.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-15T13:46:26.776 INFO:teuthology.orchestra.run.smithi052.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-15T13:46:26.776 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-15T13:46:26.776 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-15T13:46:26.776 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-15T13:46:26.776 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-15T13:46:26.776 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-15T13:46:26.776 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-15T13:46:26.776 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-15T13:46:26.776 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-15T13:46:26.777 INFO:teuthology.orchestra.run.smithi052.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-15T13:46:26.777 INFO:teuthology.orchestra.run.smithi052.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-15T13:46:26.777 INFO:teuthology.orchestra.run.smithi052.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-15T13:46:26.777 INFO:teuthology.orchestra.run.smithi052.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-15T13:46:26.777 INFO:teuthology.orchestra.run.smithi052.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-15T13:46:26.777 INFO:teuthology.orchestra.run.smithi052.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-15T13:46:26.777 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-15T13:46:26.777 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-15T13:46:26.777 INFO:teuthology.orchestra.run.smithi052.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-15T13:46:26.777 INFO:teuthology.orchestra.run.smithi052.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-15T13:46:26.777 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-15T13:46:26.777 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-15T13:46:26.777 INFO:teuthology.orchestra.run.smithi052.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-15T13:46:26.777 INFO:teuthology.orchestra.run.smithi052.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-15T13:46:26.778 INFO:teuthology.orchestra.run.smithi052.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-15T13:46:26.778 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-15T13:46:26.778 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-15T13:46:26.778 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-15T13:46:26.778 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-15T13:46:26.778 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-15T13:46:26.778 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-15T13:46:26.778 INFO:teuthology.orchestra.run.smithi052.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-15T13:46:26.778 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-15T13:46:26.778 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-15T13:46:26.778 INFO:teuthology.orchestra.run.smithi052.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-15T13:46:26.778 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-15T13:46:26.778 INFO:teuthology.orchestra.run.smithi052.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-15T13:46:26.778 INFO:teuthology.orchestra.run.smithi052.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-15T13:46:26.779 INFO:teuthology.orchestra.run.smithi052.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-15T13:46:26.779 INFO:teuthology.orchestra.run.smithi052.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-15T13:46:26.779 INFO:teuthology.orchestra.run.smithi052.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-15T13:46:26.779 INFO:teuthology.orchestra.run.smithi052.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-15T13:46:26.779 INFO:teuthology.orchestra.run.smithi052.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-15T13:46:26.779 INFO:teuthology.orchestra.run.smithi052.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-15T13:46:26.779 INFO:teuthology.orchestra.run.smithi052.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-15T13:46:26.779 INFO:teuthology.orchestra.run.smithi052.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-15T13:46:26.779 INFO:teuthology.orchestra.run.smithi052.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-15T13:46:26.779 INFO:teuthology.orchestra.run.smithi052.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-15T13:46:26.779 INFO:teuthology.orchestra.run.smithi052.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-15T13:46:26.779 INFO:teuthology.orchestra.run.smithi052.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-15T13:46:26.779 INFO:teuthology.orchestra.run.smithi052.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-15T13:46:26.779 INFO:teuthology.orchestra.run.smithi052.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-15T13:46:26.779 INFO:teuthology.orchestra.run.smithi052.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-15T13:46:26.780 INFO:teuthology.orchestra.run.smithi052.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-15T13:46:26.780 INFO:teuthology.orchestra.run.smithi052.stdout:Installing weak dependencies: 2024-05-15T13:46:26.780 INFO:teuthology.orchestra.run.smithi052.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-15T13:46:26.780 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-15T13:46:26.780 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction Summary 2024-05-15T13:46:26.780 INFO:teuthology.orchestra.run.smithi052.stdout:======================================================================================= 2024-05-15T13:46:26.780 INFO:teuthology.orchestra.run.smithi052.stdout:Install 148 Packages 2024-05-15T13:46:26.780 INFO:teuthology.orchestra.run.smithi052.stdout:Upgrade 8 Packages 2024-05-15T13:46:26.780 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-15T13:46:26.781 INFO:teuthology.orchestra.run.smithi052.stdout:Total download size: 226 M 2024-05-15T13:46:26.781 INFO:teuthology.orchestra.run.smithi052.stdout:Downloading Packages: 2024-05-15T13:46:26.827 INFO:teuthology.orchestra.run.smithi081.stdout:(2/156): ceph-fuse-19.0.0-3687.g9b638668.el9.x8 4.2 MB/s | 891 kB 00:00 2024-05-15T13:46:26.877 INFO:teuthology.orchestra.run.smithi081.stdout:(3/156): ceph-immutable-object-cache-19.0.0-368 2.8 MB/s | 145 kB 00:00 2024-05-15T13:46:27.177 INFO:teuthology.orchestra.run.smithi081.stdout:(4/156): ceph-mds-19.0.0-3687.g9b638668.el9.x86 7.9 MB/s | 2.4 MB 00:00 2024-05-15T13:46:27.361 INFO:teuthology.orchestra.run.smithi081.stdout:(5/156): ceph-base-19.0.0-3687.g9b638668.el9.x8 5.8 MB/s | 5.4 MB 00:00 2024-05-15T13:46:27.403 INFO:teuthology.orchestra.run.smithi081.stdout:(6/156): ceph-mgr-19.0.0-3687.g9b638668.el9.x86 6.7 MB/s | 1.5 MB 00:00 2024-05-15T13:46:27.474 INFO:teuthology.orchestra.run.smithi033.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-15T13:46:27.475 INFO:teuthology.orchestra.run.smithi033.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-15T13:46:27.487 INFO:teuthology.orchestra.run.smithi033.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-15T13:46:27.488 INFO:teuthology.orchestra.run.smithi033.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-15T13:46:27.488 INFO:teuthology.orchestra.run.smithi033.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-15T13:46:27.489 INFO:teuthology.orchestra.run.smithi033.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-15T13:46:27.491 INFO:teuthology.orchestra.run.smithi033.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-15T13:46:27.494 INFO:teuthology.orchestra.run.smithi033.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-15T13:46:27.495 INFO:teuthology.orchestra.run.smithi033.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-15T13:46:27.561 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-15T13:46:27.572 INFO:teuthology.orchestra.run.smithi033.stdout:======================================================================================= 2024-05-15T13:46:27.572 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-05-15T13:46:27.572 INFO:teuthology.orchestra.run.smithi033.stdout:======================================================================================= 2024-05-15T13:46:27.572 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-05-15T13:46:27.572 INFO:teuthology.orchestra.run.smithi033.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-15T13:46:27.572 INFO:teuthology.orchestra.run.smithi033.stdout: ceph x86_64 2:19.0.0-3687.g9b638668.el9 ceph 6.6 k 2024-05-15T13:46:27.572 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base x86_64 2:19.0.0-3687.g9b638668.el9 ceph 5.4 M 2024-05-15T13:46:27.572 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 891 k 2024-05-15T13:46:27.572 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3687.g9b638668.el9 ceph 145 k 2024-05-15T13:46:27.572 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr x86_64 2:19.0.0-3687.g9b638668.el9 ceph 1.5 M 2024-05-15T13:46:27.572 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 143 k 2024-05-15T13:46:27.572 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 3.5 M 2024-05-15T13:46:27.573 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 7.4 M 2024-05-15T13:46:27.573 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 49 k 2024-05-15T13:46:27.573 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw x86_64 2:19.0.0-3687.g9b638668.el9 ceph 11 M 2024-05-15T13:46:27.573 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test x86_64 2:19.0.0-3687.g9b638668.el9 ceph 49 M 2024-05-15T13:46:27.573 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-volume noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 273 k 2024-05-15T13:46:27.573 INFO:teuthology.orchestra.run.smithi033.stdout: cephadm noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 763 k 2024-05-15T13:46:27.573 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-mirror x86_64 2:19.0.0-3687.g9b638668.el9 ceph 225 k 2024-05-15T13:46:27.573 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-shell x86_64 2:19.0.0-3687.g9b638668.el9 ceph 27 k 2024-05-15T13:46:27.573 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-top noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 18 k 2024-05-15T13:46:27.573 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-15T13:46:27.574 INFO:teuthology.orchestra.run.smithi033.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-15T13:46:27.574 INFO:teuthology.orchestra.run.smithi033.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-15T13:46:27.574 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel x86_64 2:19.0.0-3687.g9b638668.el9 ceph 32 k 2024-05-15T13:46:27.574 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 748 k 2024-05-15T13:46:27.574 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel x86_64 2:19.0.0-3687.g9b638668.el9 ceph 127 k 2024-05-15T13:46:27.574 INFO:teuthology.orchestra.run.smithi033.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-15T13:46:27.574 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-15T13:46:27.574 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-15T13:46:27.574 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-15T13:46:27.575 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-15T13:46:27.575 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-15T13:46:27.575 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cephfs x86_64 2:19.0.0-3687.g9b638668.el9 ceph 161 k 2024-05-15T13:46:27.575 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados x86_64 2:19.0.0-3687.g9b638668.el9 ceph 321 k 2024-05-15T13:46:27.575 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rbd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 299 k 2024-05-15T13:46:27.575 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rgw x86_64 2:19.0.0-3687.g9b638668.el9 ceph 100 k 2024-05-15T13:46:27.575 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 85 k 2024-05-15T13:46:27.575 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-mirror x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.1 M 2024-05-15T13:46:27.575 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-nbd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 170 k 2024-05-15T13:46:27.575 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-15T13:46:27.575 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrading: 2024-05-15T13:46:27.575 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-15T13:46:27.576 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-15T13:46:27.576 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-15T13:46:27.576 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-15T13:46:27.576 INFO:teuthology.orchestra.run.smithi033.stdout: librados2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.4 M 2024-05-15T13:46:27.576 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 3.1 M 2024-05-15T13:46:27.576 INFO:teuthology.orchestra.run.smithi033.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-15T13:46:27.576 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-15T13:46:27.576 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-05-15T13:46:27.576 INFO:teuthology.orchestra.run.smithi033.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-15T13:46:27.576 INFO:teuthology.orchestra.run.smithi033.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-15T13:46:27.576 INFO:teuthology.orchestra.run.smithi033.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-15T13:46:27.576 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common x86_64 2:19.0.0-3687.g9b638668.el9 ceph 21 M 2024-05-15T13:46:27.576 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 34 k 2024-05-15T13:46:27.577 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds x86_64 2:19.0.0-3687.g9b638668.el9 ceph 2.4 M 2024-05-15T13:46:27.577 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 274 k 2024-05-15T13:46:27.577 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon x86_64 2:19.0.0-3687.g9b638668.el9 ceph 4.7 M 2024-05-15T13:46:27.577 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd x86_64 2:19.0.0-3687.g9b638668.el9 ceph 17 M 2024-05-15T13:46:27.577 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3687.g9b638668.el9 ceph-noarch 16 k 2024-05-15T13:46:27.577 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux x86_64 2:19.0.0-3687.g9b638668.el9 ceph 25 k 2024-05-15T13:46:27.577 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-15T13:46:27.577 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-15T13:46:27.577 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-15T13:46:27.577 INFO:teuthology.orchestra.run.smithi033.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-15T13:46:27.577 INFO:teuthology.orchestra.run.smithi033.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-15T13:46:27.577 INFO:teuthology.orchestra.run.smithi033.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-15T13:46:27.577 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-15T13:46:27.577 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-15T13:46:27.578 INFO:teuthology.orchestra.run.smithi033.stdout: libcephsqlite x86_64 2:19.0.0-3687.g9b638668.el9 ceph 163 k 2024-05-15T13:46:27.578 INFO:teuthology.orchestra.run.smithi033.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-15T13:46:27.578 INFO:teuthology.orchestra.run.smithi033.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-15T13:46:27.578 INFO:teuthology.orchestra.run.smithi033.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-15T13:46:27.578 INFO:teuthology.orchestra.run.smithi033.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-15T13:46:27.578 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-15T13:46:27.578 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 501 k 2024-05-15T13:46:27.578 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-15T13:46:27.578 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2 x86_64 2:19.0.0-3687.g9b638668.el9 ceph 5.4 M 2024-05-15T13:46:27.578 INFO:teuthology.orchestra.run.smithi033.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-15T13:46:27.578 INFO:teuthology.orchestra.run.smithi033.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-15T13:46:27.578 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-15T13:46:27.578 INFO:teuthology.orchestra.run.smithi033.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-15T13:46:27.579 INFO:teuthology.orchestra.run.smithi033.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-15T13:46:27.579 INFO:teuthology.orchestra.run.smithi033.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-15T13:46:27.579 INFO:teuthology.orchestra.run.smithi033.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-15T13:46:27.579 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-15T13:46:27.579 INFO:teuthology.orchestra.run.smithi033.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-15T13:46:27.579 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-15T13:46:27.579 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-15T13:46:27.579 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-15T13:46:27.579 INFO:teuthology.orchestra.run.smithi033.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-15T13:46:27.579 INFO:teuthology.orchestra.run.smithi033.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-15T13:46:27.579 INFO:teuthology.orchestra.run.smithi033.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-15T13:46:27.579 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-15T13:46:27.579 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-15T13:46:27.579 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse x86_64 2:19.0.0-3687.g9b638668.el9 ceph 45 k 2024-05-15T13:46:27.580 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common x86_64 2:19.0.0-3687.g9b638668.el9 ceph 136 k 2024-05-15T13:46:27.580 INFO:teuthology.orchestra.run.smithi033.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-15T13:46:27.580 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-15T13:46:27.580 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-15T13:46:27.580 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-15T13:46:27.580 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-15T13:46:27.580 INFO:teuthology.orchestra.run.smithi033.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-15T13:46:27.580 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-15T13:46:27.580 INFO:teuthology.orchestra.run.smithi033.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-15T13:46:27.580 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-15T13:46:27.580 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-15T13:46:27.580 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-15T13:46:27.580 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-15T13:46:27.580 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-15T13:46:27.581 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-15T13:46:27.581 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-15T13:46:27.581 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-15T13:46:27.581 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-15T13:46:27.581 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-15T13:46:27.581 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-15T13:46:27.581 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-15T13:46:27.581 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-15T13:46:27.581 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-15T13:46:27.581 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-15T13:46:27.581 INFO:teuthology.orchestra.run.smithi033.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-15T13:46:27.581 INFO:teuthology.orchestra.run.smithi033.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-15T13:46:27.581 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-15T13:46:27.582 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-15T13:46:27.582 INFO:teuthology.orchestra.run.smithi033.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-15T13:46:27.582 INFO:teuthology.orchestra.run.smithi033.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-15T13:46:27.582 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-15T13:46:27.582 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-15T13:46:27.582 INFO:teuthology.orchestra.run.smithi033.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-15T13:46:27.582 INFO:teuthology.orchestra.run.smithi033.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-15T13:46:27.582 INFO:teuthology.orchestra.run.smithi033.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-15T13:46:27.582 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-15T13:46:27.582 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-15T13:46:27.583 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-15T13:46:27.583 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-15T13:46:27.583 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-15T13:46:27.583 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-15T13:46:27.583 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-15T13:46:27.583 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-15T13:46:27.583 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-15T13:46:27.583 INFO:teuthology.orchestra.run.smithi033.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-15T13:46:27.583 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-15T13:46:27.583 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-15T13:46:27.583 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-15T13:46:27.583 INFO:teuthology.orchestra.run.smithi033.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-15T13:46:27.584 INFO:teuthology.orchestra.run.smithi033.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-15T13:46:27.584 INFO:teuthology.orchestra.run.smithi033.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-15T13:46:27.584 INFO:teuthology.orchestra.run.smithi033.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-15T13:46:27.584 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-15T13:46:27.584 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-15T13:46:27.584 INFO:teuthology.orchestra.run.smithi033.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-15T13:46:27.584 INFO:teuthology.orchestra.run.smithi033.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-15T13:46:27.584 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-15T13:46:27.584 INFO:teuthology.orchestra.run.smithi033.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-15T13:46:27.584 INFO:teuthology.orchestra.run.smithi033.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-15T13:46:27.585 INFO:teuthology.orchestra.run.smithi033.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-15T13:46:27.585 INFO:teuthology.orchestra.run.smithi033.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-15T13:46:27.585 INFO:teuthology.orchestra.run.smithi033.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-15T13:46:27.585 INFO:teuthology.orchestra.run.smithi033.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-15T13:46:27.585 INFO:teuthology.orchestra.run.smithi033.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-15T13:46:27.585 INFO:teuthology.orchestra.run.smithi033.stdout:Installing weak dependencies: 2024-05-15T13:46:27.585 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-15T13:46:27.585 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:46:27.585 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-05-15T13:46:27.585 INFO:teuthology.orchestra.run.smithi033.stdout:======================================================================================= 2024-05-15T13:46:27.585 INFO:teuthology.orchestra.run.smithi033.stdout:Install 148 Packages 2024-05-15T13:46:27.585 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrade 8 Packages 2024-05-15T13:46:27.585 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:46:27.586 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 226 M 2024-05-15T13:46:27.586 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-05-15T13:46:27.705 INFO:teuthology.orchestra.run.smithi052.stdout:(1/156): ceph-19.0.0-3687.g9b638668.el9.x86_64. 18 kB/s | 6.6 kB 00:00 2024-05-15T13:46:27.795 INFO:teuthology.orchestra.run.smithi081.stdout:(7/156): ceph-mon-19.0.0-3687.g9b638668.el9.x86 11 MB/s | 4.7 MB 00:00 2024-05-15T13:46:28.248 INFO:teuthology.orchestra.run.smithi033.stdout:(1/156): ceph-19.0.0-3687.g9b638668.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-05-15T13:46:28.905 INFO:teuthology.orchestra.run.smithi052.stdout:(2/156): ceph-fuse-19.0.0-3687.g9b638668.el9.x8 742 kB/s | 891 kB 00:01 2024-05-15T13:46:29.089 INFO:teuthology.orchestra.run.smithi052.stdout:(3/156): ceph-immutable-object-cache-19.0.0-368 790 kB/s | 145 kB 00:00 2024-05-15T13:46:29.120 INFO:teuthology.orchestra.run.smithi081.stdout:(8/156): ceph-radosgw-19.0.0-3687.g9b638668.el9 8.0 MB/s | 11 MB 00:01 2024-05-15T13:46:29.148 INFO:teuthology.orchestra.run.smithi033.stdout:(2/156): ceph-fuse-19.0.0-3687.g9b638668.el9.x8 990 kB/s | 891 kB 00:00 2024-05-15T13:46:29.220 INFO:teuthology.orchestra.run.smithi081.stdout:(9/156): ceph-selinux-19.0.0-3687.g9b638668.el9 252 kB/s | 25 kB 00:00 2024-05-15T13:46:29.273 INFO:teuthology.orchestra.run.smithi033.stdout:(3/156): ceph-immutable-object-cache-19.0.0-368 1.1 MB/s | 145 kB 00:00 2024-05-15T13:46:30.373 INFO:teuthology.orchestra.run.smithi033.stdout:(4/156): ceph-mds-19.0.0-3687.g9b638668.el9.x86 2.2 MB/s | 2.4 MB 00:01 2024-05-15T13:46:30.414 INFO:teuthology.orchestra.run.smithi052.stdout:(4/156): ceph-base-19.0.0-3687.g9b638668.el9.x8 1.7 MB/s | 5.4 MB 00:03 2024-05-15T13:46:30.790 INFO:teuthology.orchestra.run.smithi033.stdout:(5/156): ceph-mgr-19.0.0-3687.g9b638668.el9.x86 3.6 MB/s | 1.5 MB 00:00 2024-05-15T13:46:30.954 INFO:teuthology.orchestra.run.smithi081.stdout:(10/156): ceph-common-19.0.0-3687.g9b638668.el9 4.7 MB/s | 21 MB 00:04 2024-05-15T13:46:31.154 INFO:teuthology.orchestra.run.smithi081.stdout:(11/156): cephfs-mirror-19.0.0-3687.g9b638668.e 1.1 MB/s | 225 kB 00:00 2024-05-15T13:46:31.247 INFO:teuthology.orchestra.run.smithi052.stdout:(5/156): ceph-mgr-19.0.0-3687.g9b638668.el9.x86 1.8 MB/s | 1.5 MB 00:00 2024-05-15T13:46:31.271 INFO:teuthology.orchestra.run.smithi081.stdout:(12/156): cephfs-shell-19.0.0-3687.g9b638668.el 232 kB/s | 27 kB 00:00 2024-05-15T13:46:31.388 INFO:teuthology.orchestra.run.smithi081.stdout:(13/156): libcephfs-devel-19.0.0-3687.g9b638668 272 kB/s | 32 kB 00:00 2024-05-15T13:46:31.588 INFO:teuthology.orchestra.run.smithi081.stdout:(14/156): ceph-osd-19.0.0-3687.g9b638668.el9.x8 4.0 MB/s | 17 MB 00:04 2024-05-15T13:46:31.680 INFO:teuthology.orchestra.run.smithi081.stdout:(15/156): libcephsqlite-19.0.0-3687.g9b638668.e 1.7 MB/s | 163 kB 00:00 2024-05-15T13:46:31.682 INFO:teuthology.orchestra.run.smithi033.stdout:(6/156): ceph-mon-19.0.0-3687.g9b638668.el9.x86 5.3 MB/s | 4.7 MB 00:00 2024-05-15T13:46:31.697 INFO:teuthology.orchestra.run.smithi052.stdout:(6/156): ceph-mds-19.0.0-3687.g9b638668.el9.x86 929 kB/s | 2.4 MB 00:02 2024-05-15T13:46:31.705 INFO:teuthology.orchestra.run.smithi081.stdout:(16/156): libcephfs2-19.0.0-3687.g9b638668.el9. 2.3 MB/s | 748 kB 00:00 2024-05-15T13:46:31.747 INFO:teuthology.orchestra.run.smithi081.stdout:(17/156): librados-devel-19.0.0-3687.g9b638668. 1.9 MB/s | 127 kB 00:00 2024-05-15T13:46:31.922 INFO:teuthology.orchestra.run.smithi081.stdout:(18/156): libradosstriper1-19.0.0-3687.g9b63866 2.3 MB/s | 501 kB 00:00 2024-05-15T13:46:32.039 INFO:teuthology.orchestra.run.smithi081.stdout:(19/156): python3-ceph-argparse-19.0.0-3687.g9b 386 kB/s | 45 kB 00:00 2024-05-15T13:46:32.165 INFO:teuthology.orchestra.run.smithi081.stdout:(20/156): python3-ceph-common-19.0.0-3687.g9b63 1.1 MB/s | 136 kB 00:00 2024-05-15T13:46:32.273 INFO:teuthology.orchestra.run.smithi081.stdout:(21/156): python3-cephfs-19.0.0-3687.g9b638668. 1.5 MB/s | 161 kB 00:00 2024-05-15T13:46:33.057 INFO:teuthology.orchestra.run.smithi081.stdout:(22/156): python3-rados-19.0.0-3687.g9b638668.e 410 kB/s | 321 kB 00:00 2024-05-15T13:46:33.241 INFO:teuthology.orchestra.run.smithi081.stdout:(23/156): python3-rbd-19.0.0-3687.g9b638668.el9 1.6 MB/s | 299 kB 00:00 2024-05-15T13:46:33.349 INFO:teuthology.orchestra.run.smithi081.stdout:(24/156): python3-rgw-19.0.0-3687.g9b638668.el9 920 kB/s | 100 kB 00:00 2024-05-15T13:46:33.466 INFO:teuthology.orchestra.run.smithi081.stdout:(25/156): rbd-fuse-19.0.0-3687.g9b638668.el9.x8 731 kB/s | 85 kB 00:00 2024-05-15T13:46:33.858 INFO:teuthology.orchestra.run.smithi081.stdout:(26/156): librgw2-19.0.0-3687.g9b638668.el9.x86 2.5 MB/s | 5.4 MB 00:02 2024-05-15T13:46:33.959 INFO:teuthology.orchestra.run.smithi081.stdout:(27/156): rbd-nbd-19.0.0-3687.g9b638668.el9.x86 1.7 MB/s | 170 kB 00:00 2024-05-15T13:46:34.051 INFO:teuthology.orchestra.run.smithi081.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3687.g 374 kB/s | 34 kB 00:00 2024-05-15T13:46:34.151 INFO:teuthology.orchestra.run.smithi081.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3687.g9b63866 1.4 MB/s | 143 kB 00:00 2024-05-15T13:46:34.618 INFO:teuthology.orchestra.run.smithi081.stdout:(30/156): rbd-mirror-19.0.0-3687.g9b638668.el9. 2.7 MB/s | 3.1 MB 00:01 2024-05-15T13:46:34.782 INFO:teuthology.orchestra.run.smithi033.stdout:(7/156): ceph-base-19.0.0-3687.g9b638668.el9.x8 798 kB/s | 5.4 MB 00:06 2024-05-15T13:46:35.139 INFO:teuthology.orchestra.run.smithi052.stdout:(7/156): ceph-mon-19.0.0-3687.g9b638668.el9.x86 1.2 MB/s | 4.7 MB 00:03 2024-05-15T13:46:35.593 INFO:teuthology.orchestra.run.smithi081.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3687.g9b638 2.5 MB/s | 3.5 MB 00:01 2024-05-15T13:46:35.707 INFO:teuthology.orchestra.run.smithi033.stdout:(8/156): ceph-osd-19.0.0-3687.g9b638668.el9.x86 4.2 MB/s | 17 MB 00:04 2024-05-15T13:46:35.726 INFO:teuthology.orchestra.run.smithi081.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3687.g9b 2.0 MB/s | 274 kB 00:00 2024-05-15T13:46:35.765 INFO:teuthology.orchestra.run.smithi033.stdout:(9/156): ceph-selinux-19.0.0-3687.g9b638668.el9 433 kB/s | 25 kB 00:00 2024-05-15T13:46:35.802 INFO:teuthology.orchestra.run.smithi081.stdout:(33/156): ceph-mgr-rook-19.0.0-3687.g9b638668.e 658 kB/s | 49 kB 00:00 2024-05-15T13:46:35.902 INFO:teuthology.orchestra.run.smithi081.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3687.g9 164 kB/s | 16 kB 00:00 2024-05-15T13:46:36.069 INFO:teuthology.orchestra.run.smithi081.stdout:(35/156): ceph-volume-19.0.0-3687.g9b638668.el9 1.6 MB/s | 273 kB 00:00 2024-05-15T13:46:36.386 INFO:teuthology.orchestra.run.smithi081.stdout:(36/156): cephadm-19.0.0-3687.g9b638668.el9.noa 2.4 MB/s | 763 kB 00:00 2024-05-15T13:46:36.461 INFO:teuthology.orchestra.run.smithi081.stdout:(37/156): cephfs-top-19.0.0-3687.g9b638668.el9. 243 kB/s | 18 kB 00:00 2024-05-15T13:46:36.670 INFO:teuthology.orchestra.run.smithi081.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 386 kB/s | 80 kB 00:00 2024-05-15T13:46:36.862 INFO:teuthology.orchestra.run.smithi081.stdout:(39/156): libgfortran-11.4.1-3.el9.x86_64.rpm 4.1 MB/s | 803 kB 00:00 2024-05-15T13:46:36.921 INFO:teuthology.orchestra.run.smithi081.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2024-05-15T13:46:36.979 INFO:teuthology.orchestra.run.smithi081.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-05-15T13:46:37.063 INFO:teuthology.orchestra.run.smithi081.stdout:(42/156): ceph-mgr-diskprediction-local-19.0.0- 3.0 MB/s | 7.4 MB 00:02 2024-05-15T13:46:37.113 INFO:teuthology.orchestra.run.smithi081.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 9.2 MB/s | 1.2 MB 00:00 2024-05-15T13:46:37.172 INFO:teuthology.orchestra.run.smithi081.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-05-15T13:46:37.230 INFO:teuthology.orchestra.run.smithi081.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-05-15T13:46:37.256 INFO:teuthology.orchestra.run.smithi081.stdout:(46/156): python3-ply-3.11-14.el9.noarch.rpm 554 kB/s | 106 kB 00:00 2024-05-15T13:46:37.289 INFO:teuthology.orchestra.run.smithi081.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-05-15T13:46:37.431 INFO:teuthology.orchestra.run.smithi081.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.8 MB/s | 689 kB 00:00 2024-05-15T13:46:37.473 INFO:teuthology.orchestra.run.smithi081.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 5.0 MB/s | 943 kB 00:00 2024-05-15T13:46:37.499 INFO:teuthology.orchestra.run.smithi081.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-05-15T13:46:37.524 INFO:teuthology.orchestra.run.smithi081.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 428 kB/s | 21 kB 00:00 2024-05-15T13:46:37.549 INFO:teuthology.orchestra.run.smithi081.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-05-15T13:46:37.575 INFO:teuthology.orchestra.run.smithi081.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 654 kB/s | 33 kB 00:00 2024-05-15T13:46:37.617 INFO:teuthology.orchestra.run.smithi081.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 435 kB/s | 18 kB 00:00 2024-05-15T13:46:37.667 INFO:teuthology.orchestra.run.smithi081.stdout:(55/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 25 MB/s | 3.0 MB 00:00 2024-05-15T13:46:37.692 INFO:teuthology.orchestra.run.smithi081.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 148 kB/s | 11 kB 00:00 2024-05-15T13:46:37.718 INFO:teuthology.orchestra.run.smithi081.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-05-15T13:46:37.743 INFO:teuthology.orchestra.run.smithi081.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 901 kB/s | 45 kB 00:00 2024-05-15T13:46:37.778 INFO:teuthology.orchestra.run.smithi081.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-05-15T13:46:37.802 INFO:teuthology.orchestra.run.smithi081.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 9.7 MB/s | 582 kB 00:00 2024-05-15T13:46:37.827 INFO:teuthology.orchestra.run.smithi081.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 506 kB/s | 25 kB 00:00 2024-05-15T13:46:37.853 INFO:teuthology.orchestra.run.smithi081.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-05-15T13:46:37.878 INFO:teuthology.orchestra.run.smithi081.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-05-15T13:46:37.903 INFO:teuthology.orchestra.run.smithi081.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-05-15T13:46:37.928 INFO:teuthology.orchestra.run.smithi081.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 705 kB/s | 35 kB 00:00 2024-05-15T13:46:38.021 INFO:teuthology.orchestra.run.smithi081.stdout:(66/156): openblas-openmp-0.3.21-2.el9.x86_64.r 41 MB/s | 4.7 MB 00:00 2024-05-15T13:46:38.104 INFO:teuthology.orchestra.run.smithi081.stdout:(67/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-05-15T13:46:38.146 INFO:teuthology.orchestra.run.smithi081.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-05-15T13:46:38.180 INFO:teuthology.orchestra.run.smithi081.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-05-15T13:46:38.205 INFO:teuthology.orchestra.run.smithi081.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-05-15T13:46:38.256 INFO:teuthology.orchestra.run.smithi081.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 20 MB/s | 1.0 MB 00:00 2024-05-15T13:46:38.582 INFO:teuthology.orchestra.run.smithi033.stdout:(10/156): ceph-common-19.0.0-3687.g9b638668.el9 2.0 MB/s | 21 MB 00:10 2024-05-15T13:46:38.698 INFO:teuthology.orchestra.run.smithi081.stdout:(72/156): ceph-test-19.0.0-3687.g9b638668.el9.x 5.2 MB/s | 49 MB 00:09 2024-05-15T13:46:38.714 INFO:teuthology.orchestra.run.smithi081.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 191 kB/s | 87 kB 00:00 2024-05-15T13:46:38.724 INFO:teuthology.orchestra.run.smithi033.stdout:(11/156): cephfs-mirror-19.0.0-3687.g9b638668.e 1.6 MB/s | 225 kB 00:00 2024-05-15T13:46:38.790 INFO:teuthology.orchestra.run.smithi081.stdout:(74/156): postgresql-server-13.14-1.el9.x86_64. 9.5 MB/s | 5.8 MB 00:00 2024-05-15T13:46:38.791 INFO:teuthology.orchestra.run.smithi033.stdout:(12/156): cephfs-shell-19.0.0-3687.g9b638668.el 406 kB/s | 27 kB 00:00 2024-05-15T13:46:38.815 INFO:teuthology.orchestra.run.smithi081.stdout:(75/156): python3-devel-3.9.18-3.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-05-15T13:46:38.840 INFO:teuthology.orchestra.run.smithi081.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-05-15T13:46:38.866 INFO:teuthology.orchestra.run.smithi081.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-05-15T13:46:38.875 INFO:teuthology.orchestra.run.smithi033.stdout:(13/156): libcephfs-devel-19.0.0-3687.g9b638668 381 kB/s | 32 kB 00:00 2024-05-15T13:46:38.891 INFO:teuthology.orchestra.run.smithi081.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-05-15T13:46:38.916 INFO:teuthology.orchestra.run.smithi081.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-05-15T13:46:38.941 INFO:teuthology.orchestra.run.smithi081.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 692 kB/s | 35 kB 00:00 2024-05-15T13:46:38.992 INFO:teuthology.orchestra.run.smithi081.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-05-15T13:46:39.068 INFO:teuthology.orchestra.run.smithi081.stdout:(82/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 5.1 MB 00:00 2024-05-15T13:46:39.093 INFO:teuthology.orchestra.run.smithi081.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-05-15T13:46:39.118 INFO:teuthology.orchestra.run.smithi081.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-15T13:46:39.143 INFO:teuthology.orchestra.run.smithi081.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 845 kB/s | 42 kB 00:00 2024-05-15T13:46:39.169 INFO:teuthology.orchestra.run.smithi081.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-05-15T13:46:39.194 INFO:teuthology.orchestra.run.smithi081.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-05-15T13:46:39.200 INFO:teuthology.orchestra.run.smithi033.stdout:(14/156): libcephfs2-19.0.0-3687.g9b638668.el9. 2.2 MB/s | 748 kB 00:00 2024-05-15T13:46:39.219 INFO:teuthology.orchestra.run.smithi081.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-05-15T13:46:39.244 INFO:teuthology.orchestra.run.smithi081.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-15T13:46:39.270 INFO:teuthology.orchestra.run.smithi081.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-15T13:46:39.292 INFO:teuthology.orchestra.run.smithi033.stdout:(15/156): libcephsqlite-19.0.0-3687.g9b638668.e 1.7 MB/s | 163 kB 00:00 2024-05-15T13:46:39.345 INFO:teuthology.orchestra.run.smithi081.stdout:(91/156): python3-babel-2.9.1-2.el9.noarch.rpm 9.2 MB/s | 6.0 MB 00:00 2024-05-15T13:46:39.370 INFO:teuthology.orchestra.run.smithi081.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-05-15T13:46:39.375 INFO:teuthology.orchestra.run.smithi033.stdout:(16/156): librados-devel-19.0.0-3687.g9b638668. 1.5 MB/s | 127 kB 00:00 2024-05-15T13:46:39.395 INFO:teuthology.orchestra.run.smithi081.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 883 kB/s | 44 kB 00:00 2024-05-15T13:46:39.421 INFO:teuthology.orchestra.run.smithi081.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-15T13:46:39.446 INFO:teuthology.orchestra.run.smithi081.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 713 kB/s | 36 kB 00:00 2024-05-15T13:46:39.472 INFO:teuthology.orchestra.run.smithi081.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-05-15T13:46:39.497 INFO:teuthology.orchestra.run.smithi081.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-05-15T13:46:39.522 INFO:teuthology.orchestra.run.smithi081.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-05-15T13:46:39.548 INFO:teuthology.orchestra.run.smithi081.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-05-15T13:46:39.573 INFO:teuthology.orchestra.run.smithi081.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-15T13:46:39.592 INFO:teuthology.orchestra.run.smithi033.stdout:(17/156): libradosstriper1-19.0.0-3687.g9b63866 2.3 MB/s | 501 kB 00:00 2024-05-15T13:46:39.607 INFO:teuthology.orchestra.run.smithi081.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 767 kB/s | 45 kB 00:00 2024-05-15T13:46:39.698 INFO:teuthology.orchestra.run.smithi081.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 178 kB/s | 22 kB 00:00 2024-05-15T13:46:39.857 INFO:teuthology.orchestra.run.smithi081.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-05-15T13:46:39.982 INFO:teuthology.orchestra.run.smithi081.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.2 MB/s | 863 kB 00:00 2024-05-15T13:46:40.005 INFO:teuthology.orchestra.run.smithi052.stdout:(8/156): ceph-common-19.0.0-3687.g9b638668.el9. 1.7 MB/s | 21 MB 00:12 2024-05-15T13:46:40.114 INFO:teuthology.orchestra.run.smithi052.stdout:(9/156): ceph-selinux-19.0.0-3687.g9b638668.el9 232 kB/s | 25 kB 00:00 2024-05-15T13:46:40.283 INFO:teuthology.orchestra.run.smithi081.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 65 kB/s | 19 kB 00:00 2024-05-15T13:46:40.308 INFO:teuthology.orchestra.run.smithi081.stdout:(106/156): abseil-cpp-20211102.0-3.el9.x86_64.r 900 kB/s | 548 kB 00:00 2024-05-15T13:46:40.374 INFO:teuthology.orchestra.run.smithi081.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 597 kB/s | 309 kB 00:00 2024-05-15T13:46:40.400 INFO:teuthology.orchestra.run.smithi081.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 281 kB/s | 26 kB 00:00 2024-05-15T13:46:40.484 INFO:teuthology.orchestra.run.smithi081.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 446 kB/s | 49 kB 00:00 2024-05-15T13:46:40.651 INFO:teuthology.orchestra.run.smithi081.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 269 kB/s | 67 kB 00:00 2024-05-15T13:46:40.673 INFO:teuthology.orchestra.run.smithi052.stdout:(10/156): ceph-osd-19.0.0-3687.g9b638668.el9.x8 1.9 MB/s | 17 MB 00:08 2024-05-15T13:46:40.717 INFO:teuthology.orchestra.run.smithi081.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 647 kB/s | 151 kB 00:00 2024-05-15T13:46:40.781 INFO:teuthology.orchestra.run.smithi052.stdout:(11/156): ceph-radosgw-19.0.0-3687.g9b638668.el 1.9 MB/s | 11 MB 00:05 2024-05-15T13:46:40.798 INFO:teuthology.orchestra.run.smithi052.stdout:(12/156): cephfs-mirror-19.0.0-3687.g9b638668.e 1.8 MB/s | 225 kB 00:00 2024-05-15T13:46:40.801 INFO:teuthology.orchestra.run.smithi081.stdout:(112/156): libarrow-9.0.0-11.el9.x86_64.rpm 8.5 MB/s | 4.4 MB 00:00 2024-05-15T13:46:40.857 INFO:teuthology.orchestra.run.smithi052.stdout:(13/156): cephfs-shell-19.0.0-3687.g9b638668.el 360 kB/s | 27 kB 00:00 2024-05-15T13:46:40.868 INFO:teuthology.orchestra.run.smithi081.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 652 kB/s | 43 kB 00:00 2024-05-15T13:46:40.890 INFO:teuthology.orchestra.run.smithi052.stdout:(14/156): libcephfs-devel-19.0.0-3687.g9b638668 345 kB/s | 32 kB 00:00 2024-05-15T13:46:40.968 INFO:teuthology.orchestra.run.smithi081.stdout:(114/156): python3-cachetools-4.2.4-1.el9.noarc 322 kB/s | 32 kB 00:00 2024-05-15T13:46:40.994 INFO:teuthology.orchestra.run.smithi081.stdout:(115/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.4 MB/s | 839 kB 00:00 2024-05-15T13:46:41.024 INFO:teuthology.orchestra.run.smithi052.stdout:(15/156): libcephsqlite-19.0.0-3687.g9b638668.e 1.2 MB/s | 163 kB 00:00 2024-05-15T13:46:41.052 INFO:teuthology.orchestra.run.smithi081.stdout:(116/156): python3-asyncssh-2.13.2-5.el9.noarch 1.6 MB/s | 548 kB 00:00 2024-05-15T13:46:41.078 INFO:teuthology.orchestra.run.smithi081.stdout:(117/156): python3-certifi-2021.10.8-2.el9.noar 137 kB/s | 15 kB 00:00 2024-05-15T13:46:41.103 INFO:teuthology.orchestra.run.smithi081.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-05-15T13:46:41.124 INFO:teuthology.orchestra.run.smithi052.stdout:(16/156): librados-devel-19.0.0-3687.g9b638668. 1.2 MB/s | 127 kB 00:00 2024-05-15T13:46:41.170 INFO:teuthology.orchestra.run.smithi081.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-05-15T13:46:41.195 INFO:teuthology.orchestra.run.smithi081.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 384 kB/s | 35 kB 00:00 2024-05-15T13:46:41.220 INFO:teuthology.orchestra.run.smithi081.stdout:(121/156): python3-cherrypy-18.6.1-2.el9.noarch 2.1 MB/s | 358 kB 00:00 2024-05-15T13:46:41.291 INFO:teuthology.orchestra.run.smithi052.stdout:(17/156): libcephfs2-19.0.0-3687.g9b638668.el9. 1.7 MB/s | 748 kB 00:00 2024-05-15T13:46:41.304 INFO:teuthology.orchestra.run.smithi081.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 1.5 MB/s | 206 kB 00:00 2024-05-15T13:46:41.350 INFO:teuthology.orchestra.run.smithi052.stdout:(18/156): libradosstriper1-19.0.0-3687.g9b63866 2.2 MB/s | 501 kB 00:00 2024-05-15T13:46:41.392 INFO:teuthology.orchestra.run.smithi033.stdout:(18/156): ceph-radosgw-19.0.0-3687.g9b638668.el 1.6 MB/s | 11 MB 00:06 2024-05-15T13:46:41.396 INFO:teuthology.orchestra.run.smithi081.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 117 kB/s | 11 kB 00:00 2024-05-15T13:46:41.421 INFO:teuthology.orchestra.run.smithi081.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 720 kB/s | 144 kB 00:00 2024-05-15T13:46:41.476 INFO:teuthology.orchestra.run.smithi033.stdout:(19/156): librgw2-19.0.0-3687.g9b638668.el9.x86 2.9 MB/s | 5.4 MB 00:01 2024-05-15T13:46:41.501 INFO:teuthology.orchestra.run.smithi033.stdout:(20/156): python3-ceph-argparse-19.0.0-3687.g9b 417 kB/s | 45 kB 00:00 2024-05-15T13:46:41.505 INFO:teuthology.orchestra.run.smithi081.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-05-15T13:46:41.530 INFO:teuthology.orchestra.run.smithi081.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 213 kB/s | 23 kB 00:00 2024-05-15T13:46:41.543 INFO:teuthology.orchestra.run.smithi033.stdout:(21/156): python3-ceph-common-19.0.0-3687.g9b63 2.0 MB/s | 136 kB 00:00 2024-05-15T13:46:41.544 INFO:teuthology.orchestra.run.smithi052.stdout:(19/156): python3-ceph-argparse-19.0.0-3687.g9b 233 kB/s | 45 kB 00:00 2024-05-15T13:46:41.572 INFO:teuthology.orchestra.run.smithi081.stdout:(127/156): python3-grpcio-1.46.7-10.el9.x86_64. 5.4 MB/s | 2.0 MB 00:00 2024-05-15T13:46:41.598 INFO:teuthology.orchestra.run.smithi081.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-05-15T13:46:41.623 INFO:teuthology.orchestra.run.smithi081.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 213 kB/s | 20 kB 00:00 2024-05-15T13:46:41.626 INFO:teuthology.orchestra.run.smithi033.stdout:(22/156): python3-cephfs-19.0.0-3687.g9b638668. 1.3 MB/s | 161 kB 00:00 2024-05-15T13:46:41.650 INFO:teuthology.orchestra.run.smithi052.stdout:(20/156): python3-ceph-common-19.0.0-3687.g9b63 1.2 MB/s | 136 kB 00:00 2024-05-15T13:46:41.673 INFO:teuthology.orchestra.run.smithi081.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 616 kB/s | 46 kB 00:00 2024-05-15T13:46:41.685 INFO:teuthology.orchestra.run.smithi033.stdout:(23/156): python3-rados-19.0.0-3687.g9b638668.e 2.2 MB/s | 321 kB 00:00 2024-05-15T13:46:41.690 INFO:teuthology.orchestra.run.smithi081.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-05-15T13:46:41.724 INFO:teuthology.orchestra.run.smithi081.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-05-15T13:46:41.768 INFO:teuthology.orchestra.run.smithi052.stdout:(21/156): python3-cephfs-19.0.0-3687.g9b638668. 1.4 MB/s | 161 kB 00:00 2024-05-15T13:46:41.777 INFO:teuthology.orchestra.run.smithi033.stdout:(24/156): python3-rgw-19.0.0-3687.g9b638668.el9 1.1 MB/s | 100 kB 00:00 2024-05-15T13:46:41.782 INFO:teuthology.orchestra.run.smithi081.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 530 kB/s | 58 kB 00:00 2024-05-15T13:46:41.808 INFO:teuthology.orchestra.run.smithi081.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-05-15T13:46:41.827 INFO:teuthology.orchestra.run.smithi033.stdout:(25/156): python3-rbd-19.0.0-3687.g9b638668.el9 1.5 MB/s | 299 kB 00:00 2024-05-15T13:46:41.841 INFO:teuthology.orchestra.run.smithi081.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-05-15T13:46:41.871 INFO:teuthology.orchestra.run.smithi033.stdout:(26/156): rbd-fuse-19.0.0-3687.g9b638668.el9.x8 928 kB/s | 85 kB 00:00 2024-05-15T13:46:41.876 INFO:teuthology.orchestra.run.smithi052.stdout:(22/156): python3-rados-19.0.0-3687.g9b638668.e 2.9 MB/s | 321 kB 00:00 2024-05-15T13:46:41.883 INFO:teuthology.orchestra.run.smithi081.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 893 kB/s | 90 kB 00:00 2024-05-15T13:46:41.917 INFO:teuthology.orchestra.run.smithi081.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 259 kB/s | 28 kB 00:00 2024-05-15T13:46:41.942 INFO:teuthology.orchestra.run.smithi081.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 305 kB/s | 31 kB 00:00 2024-05-15T13:46:41.984 INFO:teuthology.orchestra.run.smithi081.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-05-15T13:46:42.001 INFO:teuthology.orchestra.run.smithi052.stdout:(23/156): python3-rbd-19.0.0-3687.g9b638668.el9 2.3 MB/s | 299 kB 00:00 2024-05-15T13:46:42.003 INFO:teuthology.orchestra.run.smithi033.stdout:(27/156): rbd-nbd-19.0.0-3687.g9b638668.el9.x86 1.2 MB/s | 170 kB 00:00 2024-05-15T13:46:42.010 INFO:teuthology.orchestra.run.smithi081.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 642 kB/s | 59 kB 00:00 2024-05-15T13:46:42.035 INFO:teuthology.orchestra.run.smithi081.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 387 kB/s | 36 kB 00:00 2024-05-15T13:46:42.086 INFO:teuthology.orchestra.run.smithi081.stdout:(142/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-05-15T13:46:42.095 INFO:teuthology.orchestra.run.smithi033.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3687.g 373 kB/s | 34 kB 00:00 2024-05-15T13:46:42.102 INFO:teuthology.orchestra.run.smithi052.stdout:(24/156): python3-rgw-19.0.0-3687.g9b638668.el9 996 kB/s | 100 kB 00:00 2024-05-15T13:46:42.110 INFO:teuthology.orchestra.run.smithi081.stdout:(143/156): python3-typing-extensions-4.4.0-2.el 405 kB/s | 51 kB 00:00 2024-05-15T13:46:42.136 INFO:teuthology.orchestra.run.smithi081.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 892 kB/s | 90 kB 00:00 2024-05-15T13:46:42.170 INFO:teuthology.orchestra.run.smithi081.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-05-15T13:46:42.187 INFO:teuthology.orchestra.run.smithi033.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3687.g9b63866 1.5 MB/s | 143 kB 00:00 2024-05-15T13:46:42.194 INFO:teuthology.orchestra.run.smithi081.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 238 kB/s | 20 kB 00:00 2024-05-15T13:46:42.202 INFO:teuthology.orchestra.run.smithi052.stdout:(25/156): rbd-fuse-19.0.0-3687.g9b638668.el9.x8 853 kB/s | 85 kB 00:00 2024-05-15T13:46:42.262 INFO:teuthology.orchestra.run.smithi081.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 1.5 MB/s | 192 kB 00:00 2024-05-15T13:46:42.495 INFO:teuthology.orchestra.run.smithi081.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 4.9 MB/s | 1.6 MB 00:00 2024-05-15T13:46:42.837 INFO:teuthology.orchestra.run.smithi081.stdout:(149/156): NetworkManager-1.47.90-1.el9.x86_64. 6.8 MB/s | 2.3 MB 00:00 2024-05-15T13:46:42.988 INFO:teuthology.orchestra.run.smithi081.stdout:(150/156): NetworkManager-libnm-1.47.90-1.el9.x 12 MB/s | 1.8 MB 00:00 2024-05-15T13:46:43.046 INFO:teuthology.orchestra.run.smithi081.stdout:(151/156): NetworkManager-team-1.47.90-1.el9.x8 714 kB/s | 41 kB 00:00 2024-05-15T13:46:43.106 INFO:teuthology.orchestra.run.smithi081.stdout:(152/156): NetworkManager-tui-1.47.90-1.el9.x86 4.2 MB/s | 248 kB 00:00 2024-05-15T13:46:43.211 INFO:teuthology.orchestra.run.smithi052.stdout:(26/156): librgw2-19.0.0-3687.g9b638668.el9.x86 2.8 MB/s | 5.4 MB 00:01 2024-05-15T13:46:43.272 INFO:teuthology.orchestra.run.smithi081.stdout:(153/156): openssl-3.2.1-1.el9.x86_64.rpm 8.1 MB/s | 1.3 MB 00:00 2024-05-15T13:46:43.314 INFO:teuthology.orchestra.run.smithi052.stdout:(27/156): rbd-mirror-19.0.0-3687.g9b638668.el9. 2.8 MB/s | 3.1 MB 00:01 2024-05-15T13:46:43.336 INFO:teuthology.orchestra.run.smithi052.stdout:(28/156): rbd-nbd-19.0.0-3687.g9b638668.el9.x86 1.3 MB/s | 170 kB 00:00 2024-05-15T13:46:43.420 INFO:teuthology.orchestra.run.smithi052.stdout:(29/156): ceph-grafana-dashboards-19.0.0-3687.g 315 kB/s | 34 kB 00:00 2024-05-15T13:46:43.431 INFO:teuthology.orchestra.run.smithi081.stdout:(154/156): librados2-19.0.0-3687.g9b638668.el9. 2.7 MB/s | 3.4 MB 00:01 2024-05-15T13:46:43.445 INFO:teuthology.orchestra.run.smithi052.stdout:(30/156): ceph-mgr-cephadm-19.0.0-3687.g9b63866 1.3 MB/s | 143 kB 00:00 2024-05-15T13:46:43.480 INFO:teuthology.orchestra.run.smithi081.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-05-15T13:46:43.504 INFO:teuthology.orchestra.run.smithi033.stdout:(30/156): rbd-mirror-19.0.0-3687.g9b638668.el9. 1.8 MB/s | 3.1 MB 00:01 2024-05-15T13:46:43.723 INFO:teuthology.orchestra.run.smithi081.stdout:(156/156): librbd1-19.0.0-3687.g9b638668.el9.x8 2.2 MB/s | 3.1 MB 00:01 2024-05-15T13:46:43.728 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-05-15T13:46:43.728 INFO:teuthology.orchestra.run.smithi081.stdout:Total 13 MB/s | 226 MB 00:17 2024-05-15T13:46:43.729 INFO:teuthology.orchestra.run.smithi033.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3687.g9b638 2.3 MB/s | 3.5 MB 00:01 2024-05-15T13:46:43.880 INFO:teuthology.orchestra.run.smithi033.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3687.g9b 1.8 MB/s | 274 kB 00:00 2024-05-15T13:46:43.963 INFO:teuthology.orchestra.run.smithi033.stdout:(33/156): ceph-mgr-rook-19.0.0-3687.g9b638668.e 594 kB/s | 49 kB 00:00 2024-05-15T13:46:44.047 INFO:teuthology.orchestra.run.smithi033.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3687.g9 197 kB/s | 16 kB 00:00 2024-05-15T13:46:44.189 INFO:teuthology.orchestra.run.smithi033.stdout:(35/156): ceph-volume-19.0.0-3687.g9b638668.el9 1.9 MB/s | 273 kB 00:00 2024-05-15T13:46:44.541 INFO:teuthology.orchestra.run.smithi033.stdout:(36/156): cephadm-19.0.0-3687.g9b638668.el9.noa 2.1 MB/s | 763 kB 00:00 2024-05-15T13:46:44.623 INFO:teuthology.orchestra.run.smithi033.stdout:(37/156): cephfs-top-19.0.0-3687.g9b638668.el9. 219 kB/s | 18 kB 00:00 2024-05-15T13:46:44.829 INFO:teuthology.orchestra.run.smithi052.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3687.g9b638 2.5 MB/s | 3.5 MB 00:01 2024-05-15T13:46:44.866 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-05-15T13:46:45.016 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-05-15T13:46:45.016 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-05-15T13:46:45.029 INFO:teuthology.orchestra.run.smithi052.stdout:(32/156): ceph-mgr-diskprediction-local-19.0.0- 4.7 MB/s | 7.4 MB 00:01 2024-05-15T13:46:45.040 INFO:teuthology.orchestra.run.smithi033.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 193 kB/s | 80 kB 00:00 2024-05-15T13:46:45.054 INFO:teuthology.orchestra.run.smithi052.stdout:(33/156): ceph-mgr-modules-core-19.0.0-3687.g9b 1.2 MB/s | 274 kB 00:00 2024-05-15T13:46:45.088 INFO:teuthology.orchestra.run.smithi052.stdout:(34/156): ceph-mgr-rook-19.0.0-3687.g9b638668.e 846 kB/s | 49 kB 00:00 2024-05-15T13:46:45.121 INFO:teuthology.orchestra.run.smithi052.stdout:(35/156): ceph-prometheus-alerts-19.0.0-3687.g9 246 kB/s | 16 kB 00:00 2024-05-15T13:46:45.163 INFO:teuthology.orchestra.run.smithi052.stdout:(36/156): ceph-volume-19.0.0-3687.g9b638668.el9 3.5 MB/s | 273 kB 00:00 2024-05-15T13:46:45.230 INFO:teuthology.orchestra.run.smithi052.stdout:(37/156): cephfs-top-19.0.0-3687.g9b638668.el9. 274 kB/s | 18 kB 00:00 2024-05-15T13:46:45.358 INFO:teuthology.orchestra.run.smithi052.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 643 kB/s | 80 kB 00:00 2024-05-15T13:46:45.406 INFO:teuthology.orchestra.run.smithi052.stdout:(39/156): cephadm-19.0.0-3687.g9b638668.el9.noa 2.6 MB/s | 763 kB 00:00 2024-05-15T13:46:45.705 INFO:teuthology.orchestra.run.smithi033.stdout:(39/156): libgfortran-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 803 kB 00:00 2024-05-15T13:46:45.705 INFO:teuthology.orchestra.run.smithi033.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-05-15T13:46:45.706 INFO:teuthology.orchestra.run.smithi033.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-05-15T13:46:45.706 INFO:teuthology.orchestra.run.smithi052.stdout:(40/156): libgfortran-11.4.1-3.el9.x86_64.rpm 4.5 MB/s | 803 kB 00:00 2024-05-15T13:46:45.706 INFO:teuthology.orchestra.run.smithi052.stdout:(41/156): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-05-15T13:46:45.706 INFO:teuthology.orchestra.run.smithi052.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-05-15T13:46:45.706 INFO:teuthology.orchestra.run.smithi052.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-05-15T13:46:45.706 INFO:teuthology.orchestra.run.smithi052.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 8.6 MB/s | 1.2 MB 00:00 2024-05-15T13:46:45.724 INFO:teuthology.orchestra.run.smithi052.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-05-15T13:46:45.750 INFO:teuthology.orchestra.run.smithi052.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-05-15T13:46:45.775 INFO:teuthology.orchestra.run.smithi052.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-05-15T13:46:45.934 INFO:teuthology.orchestra.run.smithi052.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 689 kB 00:00 2024-05-15T13:46:45.957 INFO:teuthology.orchestra.run.smithi033.stdout:(42/156): python3-cryptography-36.0.1-4.el9.x86 4.5 MB/s | 1.2 MB 00:00 2024-05-15T13:46:45.984 INFO:teuthology.orchestra.run.smithi052.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-05-15T13:46:46.027 INFO:teuthology.orchestra.run.smithi052.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 3.7 MB/s | 943 kB 00:00 2024-05-15T13:46:46.051 INFO:teuthology.orchestra.run.smithi052.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 321 kB/s | 21 kB 00:00 2024-05-15T13:46:46.058 INFO:teuthology.orchestra.run.smithi033.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-05-15T13:46:46.077 INFO:teuthology.orchestra.run.smithi052.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-05-15T13:46:46.102 INFO:teuthology.orchestra.run.smithi052.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 650 kB/s | 33 kB 00:00 2024-05-15T13:46:46.141 INFO:teuthology.orchestra.run.smithi033.stdout:(44/156): ceph-mgr-diskprediction-local-19.0.0- 2.8 MB/s | 7.4 MB 00:02 2024-05-15T13:46:46.152 INFO:teuthology.orchestra.run.smithi052.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-05-15T13:46:46.166 INFO:teuthology.orchestra.run.smithi033.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-05-15T13:46:46.211 INFO:teuthology.orchestra.run.smithi052.stdout:(55/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 22 MB/s | 3.0 MB 00:00 2024-05-15T13:46:46.236 INFO:teuthology.orchestra.run.smithi052.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-05-15T13:46:46.261 INFO:teuthology.orchestra.run.smithi052.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-05-15T13:46:46.275 INFO:teuthology.orchestra.run.smithi033.stdout:(46/156): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-05-15T13:46:46.287 INFO:teuthology.orchestra.run.smithi052.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 903 kB/s | 45 kB 00:00 2024-05-15T13:46:46.320 INFO:teuthology.orchestra.run.smithi052.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-05-15T13:46:46.362 INFO:teuthology.orchestra.run.smithi052.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 582 kB 00:00 2024-05-15T13:46:46.367 INFO:teuthology.orchestra.run.smithi033.stdout:(47/156): automake-1.16.2-8.el9.noarch.rpm 7.4 MB/s | 689 kB 00:00 2024-05-15T13:46:46.388 INFO:teuthology.orchestra.run.smithi052.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 380 kB/s | 25 kB 00:00 2024-05-15T13:46:46.413 INFO:teuthology.orchestra.run.smithi052.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-05-15T13:46:46.438 INFO:teuthology.orchestra.run.smithi052.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-05-15T13:46:46.464 INFO:teuthology.orchestra.run.smithi052.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-05-15T13:46:46.476 INFO:teuthology.orchestra.run.smithi033.stdout:(48/156): bison-3.7.4-5.el9.x86_64.rpm 8.5 MB/s | 943 kB 00:00 2024-05-15T13:46:46.489 INFO:teuthology.orchestra.run.smithi052.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 702 kB/s | 35 kB 00:00 2024-05-15T13:46:46.543 INFO:teuthology.orchestra.run.smithi033.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-05-15T13:46:46.593 INFO:teuthology.orchestra.run.smithi033.stdout:(50/156): python3-requests-2.25.1-8.el9.noarch. 277 kB/s | 125 kB 00:00 2024-05-15T13:46:46.618 INFO:teuthology.orchestra.run.smithi033.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 286 kB/s | 21 kB 00:00 2024-05-15T13:46:46.669 INFO:teuthology.orchestra.run.smithi033.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 654 kB/s | 33 kB 00:00 2024-05-15T13:46:46.702 INFO:teuthology.orchestra.run.smithi033.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-05-15T13:46:46.752 INFO:teuthology.orchestra.run.smithi033.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-05-15T13:46:46.794 INFO:teuthology.orchestra.run.smithi033.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 268 kB/s | 11 kB 00:00 2024-05-15T13:46:46.809 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-05-15T13:46:46.810 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-05-15T13:46:46.833 INFO:teuthology.orchestra.run.smithi052.stdout:(66/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-05-15T13:46:46.898 INFO:teuthology.orchestra.run.smithi052.stdout:(67/156): openblas-openmp-0.3.21-2.el9.x86_64.r 11 MB/s | 4.7 MB 00:00 2024-05-15T13:46:46.903 INFO:teuthology.orchestra.run.smithi033.stdout:(56/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-05-15T13:46:46.928 INFO:teuthology.orchestra.run.smithi033.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-05-15T13:46:46.956 INFO:teuthology.orchestra.run.smithi052.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-05-15T13:46:46.979 INFO:teuthology.orchestra.run.smithi033.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 602 kB/s | 45 kB 00:00 2024-05-15T13:46:46.998 INFO:teuthology.orchestra.run.smithi052.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 8.9 MB/s | 911 kB 00:00 2024-05-15T13:46:47.023 INFO:teuthology.orchestra.run.smithi052.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-05-15T13:46:47.037 INFO:teuthology.orchestra.run.smithi033.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 662 kB 00:00 2024-05-15T13:46:47.107 INFO:teuthology.orchestra.run.smithi052.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-05-15T13:46:47.157 INFO:teuthology.orchestra.run.smithi052.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-05-15T13:46:47.205 INFO:teuthology.orchestra.run.smithi033.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 2.5 MB/s | 582 kB 00:00 2024-05-15T13:46:47.224 INFO:teuthology.orchestra.run.smithi052.stdout:(73/156): postgresql-server-13.14-1.el9.x86_64. 26 MB/s | 5.8 MB 00:00 2024-05-15T13:46:47.230 INFO:teuthology.orchestra.run.smithi033.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 132 kB/s | 25 kB 00:00 2024-05-15T13:46:47.281 INFO:teuthology.orchestra.run.smithi033.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 632 kB/s | 47 kB 00:00 2024-05-15T13:46:47.292 INFO:teuthology.orchestra.run.smithi052.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-05-15T13:46:47.331 INFO:teuthology.orchestra.run.smithi033.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.9 MB/s | 292 kB 00:00 2024-05-15T13:46:47.352 INFO:teuthology.orchestra.run.smithi052.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-05-15T13:46:47.372 INFO:teuthology.orchestra.run.smithi033.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-05-15T13:46:47.398 INFO:teuthology.orchestra.run.smithi033.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 527 kB/s | 35 kB 00:00 2024-05-15T13:46:47.417 INFO:teuthology.orchestra.run.smithi052.stdout:(76/156): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 6.0 MB 00:00 2024-05-15T13:46:47.434 INFO:teuthology.orchestra.run.smithi052.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 306 kB/s | 26 kB 00:00 2024-05-15T13:46:47.451 INFO:teuthology.orchestra.run.smithi052.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 570 kB/s | 19 kB 00:00 2024-05-15T13:46:47.485 INFO:teuthology.orchestra.run.smithi052.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-05-15T13:46:47.502 INFO:teuthology.orchestra.run.smithi052.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 692 kB/s | 35 kB 00:00 2024-05-15T13:46:47.561 INFO:teuthology.orchestra.run.smithi052.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-05-15T13:46:47.598 INFO:teuthology.orchestra.run.smithi033.stdout:(66/156): openblas-openmp-0.3.21-2.el9.x86_64.r 21 MB/s | 4.7 MB 00:00 2024-05-15T13:46:47.653 INFO:teuthology.orchestra.run.smithi052.stdout:(82/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 30 MB/s | 5.1 MB 00:00 2024-05-15T13:46:47.678 INFO:teuthology.orchestra.run.smithi052.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.9 MB/s | 222 kB 00:00 2024-05-15T13:46:47.681 INFO:teuthology.orchestra.run.smithi033.stdout:(67/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-05-15T13:46:47.703 INFO:teuthology.orchestra.run.smithi052.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-15T13:46:47.728 INFO:teuthology.orchestra.run.smithi052.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 841 kB/s | 42 kB 00:00 2024-05-15T13:46:47.754 INFO:teuthology.orchestra.run.smithi052.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-05-15T13:46:47.779 INFO:teuthology.orchestra.run.smithi052.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-05-15T13:46:47.804 INFO:teuthology.orchestra.run.smithi052.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-05-15T13:46:47.829 INFO:teuthology.orchestra.run.smithi052.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-15T13:46:47.855 INFO:teuthology.orchestra.run.smithi052.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-15T13:46:47.905 INFO:teuthology.orchestra.run.smithi052.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 837 kB/s | 42 kB 00:00 2024-05-15T13:46:47.955 INFO:teuthology.orchestra.run.smithi052.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 883 kB/s | 44 kB 00:00 2024-05-15T13:46:48.014 INFO:teuthology.orchestra.run.smithi052.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-05-15T13:46:48.064 INFO:teuthology.orchestra.run.smithi052.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-05-15T13:46:48.114 INFO:teuthology.orchestra.run.smithi052.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 950 kB/s | 47 kB 00:00 2024-05-15T13:46:48.140 INFO:teuthology.orchestra.run.smithi033.stdout:(68/156): ceph-test-19.0.0-3687.g9b638668.el9.x 4.0 MB/s | 49 MB 00:12 2024-05-15T13:46:48.207 INFO:teuthology.orchestra.run.smithi033.stdout:(69/156): postgresql-13.14-1.el9.x86_64.rpm 2.6 MB/s | 1.6 MB 00:00 2024-05-15T13:46:48.215 INFO:teuthology.orchestra.run.smithi052.stdout:(96/156): socat-1.7.4.1-5.el9.x86_64.rpm 3.0 MB/s | 305 kB 00:00 2024-05-15T13:46:48.232 INFO:teuthology.orchestra.run.smithi033.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-05-15T13:46:48.274 INFO:teuthology.orchestra.run.smithi033.stdout:(71/156): postgresql-contrib-13.14-1.el9.x86_64 1.5 MB/s | 911 kB 00:00 2024-05-15T13:46:48.341 INFO:teuthology.orchestra.run.smithi033.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-05-15T13:46:48.375 INFO:teuthology.orchestra.run.smithi033.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 7.1 MB/s | 1.0 MB 00:00 2024-05-15T13:46:48.382 INFO:teuthology.orchestra.run.smithi052.stdout:(97/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-05-15T13:46:48.407 INFO:teuthology.orchestra.run.smithi052.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 301 kB/s | 58 kB 00:00 2024-05-15T13:46:48.433 INFO:teuthology.orchestra.run.smithi052.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-05-15T13:46:48.458 INFO:teuthology.orchestra.run.smithi052.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-15T13:46:48.458 INFO:teuthology.orchestra.run.smithi033.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-05-15T13:46:48.475 INFO:teuthology.orchestra.run.smithi052.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.0 MB/s | 45 kB 00:00 2024-05-15T13:46:48.500 INFO:teuthology.orchestra.run.smithi052.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 528 kB/s | 22 kB 00:00 2024-05-15T13:46:48.542 INFO:teuthology.orchestra.run.smithi052.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 13 MB/s | 863 kB 00:00 2024-05-15T13:46:48.559 INFO:teuthology.orchestra.run.smithi033.stdout:(75/156): postgresql-server-13.14-1.el9.x86_64. 16 MB/s | 5.8 MB 00:00 2024-05-15T13:46:48.609 INFO:teuthology.orchestra.run.smithi033.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-05-15T13:46:48.634 INFO:teuthology.orchestra.run.smithi033.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 341 kB/s | 26 kB 00:00 2024-05-15T13:46:48.676 INFO:teuthology.orchestra.run.smithi033.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 287 kB/s | 19 kB 00:00 2024-05-15T13:46:48.768 INFO:teuthology.orchestra.run.smithi033.stdout:(79/156): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-05-15T13:46:48.793 INFO:teuthology.orchestra.run.smithi033.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 297 kB/s | 35 kB 00:00 2024-05-15T13:46:48.819 INFO:teuthology.orchestra.run.smithi033.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 935 kB/s | 172 kB 00:00 2024-05-15T13:46:48.886 INFO:teuthology.orchestra.run.smithi033.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-05-15T13:46:48.919 INFO:teuthology.orchestra.run.smithi033.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-05-15T13:46:48.926 INFO:teuthology.orchestra.run.smithi052.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.3 MB/s | 548 kB 00:00 2024-05-15T13:46:48.951 INFO:teuthology.orchestra.run.smithi052.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 756 kB/s | 309 kB 00:00 2024-05-15T13:46:48.978 INFO:teuthology.orchestra.run.smithi033.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 840 kB/s | 77 kB 00:00 2024-05-15T13:46:49.001 INFO:teuthology.orchestra.run.smithi052.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 259 kB/s | 19 kB 00:00 2024-05-15T13:46:49.003 INFO:teuthology.orchestra.run.smithi033.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 505 kB/s | 42 kB 00:00 2024-05-15T13:46:49.068 INFO:teuthology.orchestra.run.smithi052.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 389 kB/s | 26 kB 00:00 2024-05-15T13:46:49.087 INFO:teuthology.orchestra.run.smithi033.stdout:(86/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-05-15T13:46:49.135 INFO:teuthology.orchestra.run.smithi052.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 728 kB/s | 49 kB 00:00 2024-05-15T13:46:49.137 INFO:teuthology.orchestra.run.smithi033.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 1.7 MB/s | 269 kB 00:00 2024-05-15T13:46:49.171 INFO:teuthology.orchestra.run.smithi033.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 949 kB/s | 159 kB 00:00 2024-05-15T13:46:49.196 INFO:teuthology.orchestra.run.smithi033.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-05-15T13:46:49.202 INFO:teuthology.orchestra.run.smithi052.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-05-15T13:46:49.222 INFO:teuthology.orchestra.run.smithi033.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 605 kB/s | 51 kB 00:00 2024-05-15T13:46:49.247 INFO:teuthology.orchestra.run.smithi033.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 711 kB/s | 54 kB 00:00 2024-05-15T13:46:49.286 INFO:teuthology.orchestra.run.smithi052.stdout:(110/156): libarrow-9.0.0-11.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-05-15T13:46:49.297 INFO:teuthology.orchestra.run.smithi033.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 555 kB/s | 42 kB 00:00 2024-05-15T13:46:49.311 INFO:teuthology.orchestra.run.smithi052.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-05-15T13:46:49.322 INFO:teuthology.orchestra.run.smithi033.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 586 kB/s | 44 kB 00:00 2024-05-15T13:46:49.373 INFO:teuthology.orchestra.run.smithi033.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 719 kB/s | 36 kB 00:00 2024-05-15T13:46:49.378 INFO:teuthology.orchestra.run.smithi052.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-05-15T13:46:49.398 INFO:teuthology.orchestra.run.smithi033.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-05-15T13:46:49.412 INFO:teuthology.orchestra.run.smithi052.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 5.3 MB/s | 548 kB 00:00 2024-05-15T13:46:49.445 INFO:teuthology.orchestra.run.smithi052.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 648 kB/s | 43 kB 00:00 2024-05-15T13:46:49.450 INFO:teuthology.orchestra.run.smithi033.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 630 kB/s | 47 kB 00:00 2024-05-15T13:46:49.487 INFO:teuthology.orchestra.run.smithi052.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 428 kB/s | 32 kB 00:00 2024-05-15T13:46:49.499 INFO:teuthology.orchestra.run.smithi033.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 3.0 MB/s | 305 kB 00:00 2024-05-15T13:46:49.521 INFO:teuthology.orchestra.run.smithi052.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-05-15T13:46:49.524 INFO:teuthology.orchestra.run.smithi033.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 770 kB/s | 58 kB 00:00 2024-05-15T13:46:49.563 INFO:teuthology.orchestra.run.smithi052.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-05-15T13:46:49.574 INFO:teuthology.orchestra.run.smithi033.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.2 MB/s | 94 kB 00:00 2024-05-15T13:46:49.596 INFO:teuthology.orchestra.run.smithi052.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-05-15T13:46:49.599 INFO:teuthology.orchestra.run.smithi033.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 846 kB/s | 64 kB 00:00 2024-05-15T13:46:49.647 INFO:teuthology.orchestra.run.smithi052.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-05-15T13:46:49.658 INFO:teuthology.orchestra.run.smithi033.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 537 kB/s | 45 kB 00:00 2024-05-15T13:46:49.672 INFO:teuthology.orchestra.run.smithi052.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 469 kB/s | 35 kB 00:00 2024-05-15T13:46:49.692 INFO:teuthology.orchestra.run.smithi033.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 242 kB/s | 22 kB 00:00 2024-05-15T13:46:49.722 INFO:teuthology.orchestra.run.smithi052.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-05-15T13:46:49.789 INFO:teuthology.orchestra.run.smithi052.stdout:(122/156): python3-grpcio-1.46.7-10.el9.x86_64. 17 MB/s | 2.0 MB 00:00 2024-05-15T13:46:49.814 INFO:teuthology.orchestra.run.smithi052.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-05-15T13:46:49.848 INFO:teuthology.orchestra.run.smithi052.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 183 kB/s | 11 kB 00:00 2024-05-15T13:46:49.860 INFO:teuthology.orchestra.run.smithi033.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-05-15T13:46:49.873 INFO:teuthology.orchestra.run.smithi052.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 301 kB/s | 18 kB 00:00 2024-05-15T13:46:49.907 INFO:teuthology.orchestra.run.smithi052.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 395 kB/s | 23 kB 00:00 2024-05-15T13:46:49.941 INFO:teuthology.orchestra.run.smithi052.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 290 kB/s | 19 kB 00:00 2024-05-15T13:46:49.974 INFO:teuthology.orchestra.run.smithi052.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 293 kB/s | 20 kB 00:00 2024-05-15T13:46:50.017 INFO:teuthology.orchestra.run.smithi033.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.3 MB/s | 863 kB 00:00 2024-05-15T13:46:50.041 INFO:teuthology.orchestra.run.smithi052.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 695 kB/s | 46 kB 00:00 2024-05-15T13:46:50.043 INFO:teuthology.orchestra.run.smithi033.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 1.6 MB/s | 309 kB 00:00 2024-05-15T13:46:50.076 INFO:teuthology.orchestra.run.smithi033.stdout:(106/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.4 MB/s | 548 kB 00:00 2024-05-15T13:46:50.092 INFO:teuthology.orchestra.run.smithi052.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-05-15T13:46:50.104 INFO:teuthology.orchestra.run.smithi033.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 232 kB/s | 19 kB 00:00 2024-05-15T13:46:50.118 INFO:teuthology.orchestra.run.smithi052.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-05-15T13:46:50.152 INFO:teuthology.orchestra.run.smithi033.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 346 kB/s | 26 kB 00:00 2024-05-15T13:46:50.160 INFO:teuthology.orchestra.run.smithi052.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 863 kB/s | 58 kB 00:00 2024-05-15T13:46:50.178 INFO:teuthology.orchestra.run.smithi033.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 644 kB/s | 49 kB 00:00 2024-05-15T13:46:50.193 INFO:teuthology.orchestra.run.smithi052.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-05-15T13:46:50.203 INFO:teuthology.orchestra.run.smithi033.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-05-15T13:46:50.227 INFO:teuthology.orchestra.run.smithi052.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 245 kB/s | 16 kB 00:00 2024-05-15T13:46:50.260 INFO:teuthology.orchestra.run.smithi052.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-05-15T13:46:50.261 INFO:teuthology.orchestra.run.smithi033.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-05-15T13:46:50.294 INFO:teuthology.orchestra.run.smithi052.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 422 kB/s | 28 kB 00:00 2024-05-15T13:46:50.329 INFO:teuthology.orchestra.run.smithi052.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 458 kB/s | 31 kB 00:00 2024-05-15T13:46:50.370 INFO:teuthology.orchestra.run.smithi052.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-05-15T13:46:50.371 INFO:teuthology.orchestra.run.smithi033.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.9 MB/s | 839 kB 00:00 2024-05-15T13:46:50.396 INFO:teuthology.orchestra.run.smithi052.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 884 kB/s | 59 kB 00:00 2024-05-15T13:46:50.404 INFO:teuthology.orchestra.run.smithi033.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.3 MB/s | 43 kB 00:00 2024-05-15T13:46:50.438 INFO:teuthology.orchestra.run.smithi052.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 535 kB/s | 36 kB 00:00 2024-05-15T13:46:50.440 INFO:teuthology.orchestra.run.smithi033.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 3.0 MB/s | 548 kB 00:00 2024-05-15T13:46:50.454 INFO:teuthology.orchestra.run.smithi033.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 640 kB/s | 32 kB 00:00 2024-05-15T13:46:50.462 INFO:teuthology.orchestra.run.smithi052.stdout:(141/156): python3-typing-extensions-4.4.0-2.el 763 kB/s | 51 kB 00:00 2024-05-15T13:46:50.471 INFO:teuthology.orchestra.run.smithi033.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 445 kB/s | 15 kB 00:00 2024-05-15T13:46:50.513 INFO:teuthology.orchestra.run.smithi052.stdout:(142/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-05-15T13:46:50.537 INFO:teuthology.orchestra.run.smithi052.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-05-15T13:46:50.554 INFO:teuthology.orchestra.run.smithi033.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-05-15T13:46:50.589 INFO:teuthology.orchestra.run.smithi052.stdout:(144/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.6 MB/s | 427 kB 00:00 2024-05-15T13:46:50.597 INFO:teuthology.orchestra.run.smithi033.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-05-15T13:46:50.613 INFO:teuthology.orchestra.run.smithi052.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-05-15T13:46:50.640 INFO:teuthology.orchestra.run.smithi033.stdout:(119/156): python3-colorama-0.4.4-12.el9.noarch 849 kB/s | 35 kB 00:00 2024-05-15T13:46:50.664 INFO:teuthology.orchestra.run.smithi052.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-05-15T13:46:50.681 INFO:teuthology.orchestra.run.smithi033.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-05-15T13:46:50.714 INFO:teuthology.orchestra.run.smithi052.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-05-15T13:46:50.740 INFO:teuthology.orchestra.run.smithi033.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-05-15T13:46:50.799 INFO:teuthology.orchestra.run.smithi033.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-05-15T13:46:50.840 INFO:teuthology.orchestra.run.smithi033.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 258 kB/s | 11 kB 00:00 2024-05-15T13:46:50.881 INFO:teuthology.orchestra.run.smithi033.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 425 kB/s | 18 kB 00:00 2024-05-15T13:46:50.957 INFO:teuthology.orchestra.run.smithi033.stdout:(125/156): libarrow-9.0.0-11.el9.x86_64.rpm 4.8 MB/s | 4.4 MB 00:00 2024-05-15T13:46:50.982 INFO:teuthology.orchestra.run.smithi033.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-05-15T13:46:51.008 INFO:teuthology.orchestra.run.smithi033.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-05-15T13:46:51.033 INFO:teuthology.orchestra.run.smithi033.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-05-15T13:46:51.075 INFO:teuthology.orchestra.run.smithi033.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-05-15T13:46:51.119 INFO:teuthology.orchestra.run.smithi033.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 1.9 MB/s | 79 kB 00:00 2024-05-15T13:46:51.160 INFO:teuthology.orchestra.run.smithi033.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 1.4 MB/s | 58 kB 00:00 2024-05-15T13:46:51.190 INFO:teuthology.orchestra.run.smithi052.stdout:(148/156): ceph-test-19.0.0-3687.g9b638668.el9. 4.4 MB/s | 49 MB 00:11 2024-05-15T13:46:51.242 INFO:teuthology.orchestra.run.smithi033.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-05-15T13:46:51.298 INFO:teuthology.orchestra.run.smithi052.stdout:(149/156): librbd1-19.0.0-3687.g9b638668.el9.x8 5.4 MB/s | 3.1 MB 00:00 2024-05-15T13:46:51.301 INFO:teuthology.orchestra.run.smithi033.stdout:(133/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.3 MB/s | 2.0 MB 00:00 2024-05-15T13:46:51.327 INFO:teuthology.orchestra.run.smithi033.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-05-15T13:46:51.360 INFO:teuthology.orchestra.run.smithi033.stdout:(135/156): python3-kubernetes-26.1.0-2.el9.noar 2.9 MB/s | 1.0 MB 00:00 2024-05-15T13:46:51.385 INFO:teuthology.orchestra.run.smithi033.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 481 kB/s | 28 kB 00:00 2024-05-15T13:46:51.407 INFO:teuthology.orchestra.run.smithi052.stdout:(150/156): NetworkManager-1.47.90-1.el9.x86_64. 11 MB/s | 2.3 MB 00:00 2024-05-15T13:46:51.410 INFO:teuthology.orchestra.run.smithi033.stdout:(137/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 825 kB/s | 90 kB 00:00 2024-05-15T13:46:51.436 INFO:teuthology.orchestra.run.smithi033.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-05-15T13:46:51.448 INFO:teuthology.orchestra.run.smithi052.stdout:(151/156): NetworkManager-team-1.47.90-1.el9.x8 1.0 MB/s | 41 kB 00:00 2024-05-15T13:46:51.461 INFO:teuthology.orchestra.run.smithi033.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-05-15T13:46:51.486 INFO:teuthology.orchestra.run.smithi033.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-05-15T13:46:51.499 INFO:teuthology.orchestra.run.smithi052.stdout:(152/156): NetworkManager-tui-1.47.90-1.el9.x86 4.8 MB/s | 248 kB 00:00 2024-05-15T13:46:51.512 INFO:teuthology.orchestra.run.smithi033.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-05-15T13:46:51.537 INFO:teuthology.orchestra.run.smithi033.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 677 kB/s | 51 kB 00:00 2024-05-15T13:46:51.578 INFO:teuthology.orchestra.run.smithi033.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-05-15T13:46:51.604 INFO:teuthology.orchestra.run.smithi033.stdout:(144/156): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-05-15T13:46:51.615 INFO:teuthology.orchestra.run.smithi052.stdout:(153/156): NetworkManager-libnm-1.47.90-1.el9.x 5.7 MB/s | 1.8 MB 00:00 2024-05-15T13:46:51.630 INFO:teuthology.orchestra.run.smithi033.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 395 kB/s | 20 kB 00:00 2024-05-15T13:46:51.666 INFO:teuthology.orchestra.run.smithi052.stdout:(154/156): librados2-19.0.0-3687.g9b638668.el9. 3.4 MB/s | 3.4 MB 00:01 2024-05-15T13:46:51.681 INFO:teuthology.orchestra.run.smithi033.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-05-15T13:46:51.696 INFO:teuthology.orchestra.run.smithi033.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-05-15T13:46:51.750 INFO:teuthology.orchestra.run.smithi052.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 18 MB/s | 2.4 MB 00:00 2024-05-15T13:46:51.783 INFO:teuthology.orchestra.run.smithi052.stdout:(156/156): openssl-3.2.1-1.el9.x86_64.rpm 4.8 MB/s | 1.3 MB 00:00 2024-05-15T13:46:51.788 INFO:teuthology.orchestra.run.smithi052.stdout:-------------------------------------------------------------------------------- 2024-05-15T13:46:51.788 INFO:teuthology.orchestra.run.smithi052.stdout:Total 9.0 MB/s | 226 MB 00:25 2024-05-15T13:46:51.905 INFO:teuthology.orchestra.run.smithi033.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 5.7 MB/s | 1.6 MB 00:00 2024-05-15T13:46:52.123 INFO:teuthology.orchestra.run.smithi033.stdout:(149/156): librados2-19.0.0-3687.g9b638668.el9. 7.5 MB/s | 3.4 MB 00:00 2024-05-15T13:46:52.297 INFO:teuthology.orchestra.run.smithi033.stdout:(150/156): librbd1-19.0.0-3687.g9b638668.el9.x8 5.2 MB/s | 3.1 MB 00:00 2024-05-15T13:46:52.648 INFO:teuthology.orchestra.run.smithi033.stdout:(151/156): NetworkManager-team-1.47.90-1.el9.x8 119 kB/s | 41 kB 00:00 2024-05-15T13:46:52.864 INFO:teuthology.orchestra.run.smithi033.stdout:(152/156): NetworkManager-1.47.90-1.el9.x86_64. 2.4 MB/s | 2.3 MB 00:00 2024-05-15T13:46:52.972 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction check 2024-05-15T13:46:52.998 INFO:teuthology.orchestra.run.smithi033.stdout:(153/156): NetworkManager-tui-1.47.90-1.el9.x86 708 kB/s | 248 kB 00:00 2024-05-15T13:46:53.056 INFO:teuthology.orchestra.run.smithi033.stdout:(154/156): openssl-3.2.1-1.el9.x86_64.rpm 7.0 MB/s | 1.3 MB 00:00 2024-05-15T13:46:53.079 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-05-15T13:46:53.124 INFO:teuthology.orchestra.run.smithi033.stdout:(155/156): NetworkManager-libnm-1.47.90-1.el9.x 1.8 MB/s | 1.8 MB 00:01 2024-05-15T13:46:53.142 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction check succeeded. 2024-05-15T13:46:53.142 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction test 2024-05-15T13:46:53.144 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-15T13:46:53.182 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-15T13:46:53.235 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-15T13:46:53.269 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-15T13:46:53.295 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-15T13:46:53.337 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-15T13:46:53.465 INFO:teuthology.orchestra.run.smithi033.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 5.1 MB/s | 2.4 MB 00:00 2024-05-15T13:46:53.468 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-05-15T13:46:53.469 INFO:teuthology.orchestra.run.smithi033.stdout:Total 8.7 MB/s | 226 MB 00:25 2024-05-15T13:46:53.666 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-15T13:46:53.788 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : librados2-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-15T13:46:53.910 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librados2-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-15T13:46:53.939 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-15T13:46:54.030 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-15T13:46:54.060 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 10/164 2024-05-15T13:46:54.205 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-15T13:46:54.257 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-15T13:46:54.285 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-15T13:46:54.388 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-15T13:46:54.411 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 14/164 2024-05-15T13:46:54.474 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 14/164 2024-05-15T13:46:54.496 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-15T13:46:54.521 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-15T13:46:54.672 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-05-15T13:46:54.704 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-15T13:46:54.835 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-15T13:46:54.841 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-05-15T13:46:54.841 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-05-15T13:46:54.977 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-15T13:46:55.059 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction test succeeded. 2024-05-15T13:46:55.059 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction 2024-05-15T13:46:55.071 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-15T13:46:55.154 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-15T13:46:55.264 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-15T13:46:55.455 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-15T13:46:55.560 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-15T13:46:55.585 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-15T13:46:55.620 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-15T13:46:55.684 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-15T13:46:55.728 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-15T13:46:55.772 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-15T13:46:55.805 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-15T13:46:55.833 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-15T13:46:55.901 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-15T13:46:55.955 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-15T13:46:55.987 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-15T13:46:56.024 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-15T13:46:56.099 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-15T13:46:56.135 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-ceph-common-2:19.0.0-3687.g9b638668.el9. 32/164 2024-05-15T13:46:56.178 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-ceph-argparse-2:19.0.0-3687.g9b638668.el 33/164 2024-05-15T13:46:56.207 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_6 34/164 2024-05-15T13:46:56.275 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-15T13:46:56.330 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-15T13:46:56.403 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-15T13:46:56.536 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-15T13:46:56.590 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-15T13:46:57.105 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-15T13:46:57.147 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-05-15T13:46:57.147 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-05-15T13:46:57.186 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-15T13:46:57.216 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-15T13:46:57.259 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-15T13:46:57.297 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librados-devel-2:19.0.0-3687.g9b638668.el9.x86_6 44/164 2024-05-15T13:46:57.338 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-15T13:46:57.480 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-15T13:46:57.541 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-15T13:46:57.631 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-15T13:46:57.696 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-15T13:46:57.769 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-15T13:46:57.801 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-15T13:46:57.839 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-15T13:46:57.884 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-15T13:46:57.920 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-15T13:46:57.963 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-15T13:46:58.007 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-15T13:46:58.139 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-15T13:46:58.177 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-15T13:46:58.208 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-15T13:46:58.248 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-15T13:46:58.342 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-15T13:46:58.383 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-15T13:46:58.423 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-15T13:46:58.450 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-15T13:46:58.473 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-15T13:46:58.546 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-15T13:46:58.872 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-15T13:46:58.906 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-15T13:46:58.934 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-15T13:46:58.968 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-15T13:46:59.007 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-15T13:46:59.039 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-15T13:46:59.069 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-15T13:46:59.095 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-15T13:46:59.129 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-15T13:46:59.170 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-15T13:46:59.205 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-15T13:46:59.290 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-15T13:46:59.334 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-15T13:46:59.366 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-15T13:46:59.412 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-15T13:46:59.511 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-15T13:46:59.568 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-15T13:47:00.273 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-15T13:47:00.323 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-15T13:47:00.357 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-15T13:47:00.457 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-15T13:47:00.515 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-15T13:47:00.593 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-15T13:47:00.626 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-15T13:47:00.802 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-15T13:47:00.832 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-15T13:47:00.909 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-15T13:47:01.379 INFO:teuthology.orchestra.run.smithi052.stdout: Preparing : 1/1 2024-05-15T13:47:01.454 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-15T13:47:01.493 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-15T13:47:01.545 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-15T13:47:01.546 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-15T13:47:01.580 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-15T13:47:01.605 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-15T13:47:01.649 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-15T13:47:01.672 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-15T13:47:02.003 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-15T13:47:02.114 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : librados2-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-15T13:47:02.223 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librados2-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-15T13:47:02.249 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-15T13:47:02.342 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-15T13:47:02.370 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 10/164 2024-05-15T13:47:02.534 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-15T13:47:02.585 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-15T13:47:02.612 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-15T13:47:02.717 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-15T13:47:02.738 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 14/164 2024-05-15T13:47:02.801 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 14/164 2024-05-15T13:47:02.823 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-15T13:47:02.845 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-15T13:47:02.863 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-15T13:47:02.953 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-15T13:47:02.981 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-15T13:47:03.007 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-15T13:47:03.015 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-15T13:47:03.135 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-15T13:47:03.295 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-15T13:47:03.325 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-15T13:47:03.355 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-15T13:47:03.370 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 101/164 2024-05-15T13:47:03.449 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-15T13:47:03.460 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 101/164 2024-05-15T13:47:03.491 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 102/164 2024-05-15T13:47:03.515 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-15T13:47:03.527 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-15T13:47:03.731 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-15T13:47:03.789 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-05-15T13:47:03.843 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-15T13:47:03.869 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-15T13:47:03.878 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-15T13:47:03.905 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-15T13:47:03.917 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-15T13:47:03.981 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-15T13:47:04.006 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-15T13:47:04.012 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-15T13:47:04.029 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-15T13:47:04.054 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-15T13:47:04.064 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 105/164 2024-05-15T13:47:04.073 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-15T13:47:04.078 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-15T13:47:04.105 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-15T13:47:04.119 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 105/164 2024-05-15T13:47:04.131 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-15T13:47:04.133 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-15T13:47:04.158 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 106/164 2024-05-15T13:47:04.205 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-15T13:47:04.256 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-15T13:47:04.286 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-15T13:47:04.338 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-15T13:47:04.416 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-15T13:47:04.449 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-ceph-common-2:19.0.0-3687.g9b638668.el9. 32/164 2024-05-15T13:47:04.492 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-ceph-argparse-2:19.0.0-3687.g9b638668.el 33/164 2024-05-15T13:47:04.521 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_6 34/164 2024-05-15T13:47:04.543 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-15T13:47:04.594 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-15T13:47:04.596 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : librados2-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-15T13:47:04.645 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-15T13:47:04.722 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados2-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-15T13:47:04.722 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-15T13:47:04.747 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-15T13:47:04.832 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-15T13:47:04.860 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 10/164 2024-05-15T13:47:04.870 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-15T13:47:04.923 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-15T13:47:05.046 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-15T13:47:05.100 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-15T13:47:05.119 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-15T13:47:05.224 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-15T13:47:05.253 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 14/164 2024-05-15T13:47:05.316 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 14/164 2024-05-15T13:47:05.337 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-15T13:47:05.373 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-15T13:47:05.492 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-15T13:47:05.556 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-15T13:47:05.572 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-15T13:47:05.607 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-15T13:47:05.650 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-15T13:47:05.696 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librados-devel-2:19.0.0-3687.g9b638668.el9.x86_6 44/164 2024-05-15T13:47:05.716 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-15T13:47:05.738 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-15T13:47:05.850 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-15T13:47:05.900 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-15T13:47:05.911 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 17/164 2024-05-15T13:47:05.966 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-15T13:47:06.015 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-15T13:47:06.062 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-15T13:47:06.087 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-15T13:47:06.123 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-15T13:47:06.197 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-15T13:47:06.234 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-15T13:47:06.270 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-15T13:47:06.294 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 18/164 2024-05-15T13:47:06.318 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-15T13:47:06.353 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-15T13:47:06.397 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-15T13:47:06.410 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-15T13:47:06.439 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-15T13:47:06.442 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-15T13:47:06.471 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-15T13:47:06.486 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-15T13:47:06.504 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-15T13:47:06.562 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-15T13:47:06.584 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-15T13:47:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-15T13:47:06.627 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-15T13:47:06.646 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-15T13:47:06.657 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-15T13:47:06.686 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-15T13:47:06.699 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-15T13:47:06.713 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-15T13:47:06.786 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-15T13:47:06.797 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-15T13:47:06.832 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-15T13:47:06.837 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-15T13:47:06.866 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-15T13:47:06.881 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-15T13:47:06.902 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-15T13:47:06.914 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-15T13:47:06.939 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-15T13:47:06.981 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-15T13:47:07.010 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-15T13:47:07.012 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ceph-common-2:19.0.0-3687.g9b638668.el9. 32/164 2024-05-15T13:47:07.044 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-15T13:47:07.061 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ceph-argparse-2:19.0.0-3687.g9b638668.el 33/164 2024-05-15T13:47:07.091 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 108/164 2024-05-15T13:47:07.092 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_6 34/164 2024-05-15T13:47:07.139 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-15T13:47:07.168 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-15T13:47:07.180 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-15T13:47:07.181 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 108/164 2024-05-15T13:47:07.181 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-15T13:47:07.181 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-15T13:47:07.181 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-15T13:47:07.201 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 109/164 2024-05-15T13:47:07.207 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-15T13:47:07.217 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-15T13:47:07.250 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-15T13:47:07.295 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-15T13:47:07.298 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-15T13:47:07.338 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-15T13:47:07.368 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-15T13:47:07.403 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-15T13:47:07.436 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-15T13:47:07.451 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-15T13:47:07.485 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-15T13:47:07.512 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-15T13:47:07.520 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-15T13:47:07.602 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-15T13:47:07.651 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-15T13:47:07.682 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-15T13:47:07.729 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-15T13:47:07.841 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-15T13:47:07.893 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-15T13:47:08.079 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-15T13:47:08.168 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-15T13:47:08.202 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-15T13:47:08.245 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-15T13:47:08.283 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librados-devel-2:19.0.0-3687.g9b638668.el9.x86_6 44/164 2024-05-15T13:47:08.326 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-15T13:47:08.491 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-15T13:47:08.583 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-15T13:47:08.680 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-15T13:47:08.682 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-15T13:47:08.730 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-15T13:47:08.752 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-15T13:47:08.764 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-15T13:47:08.827 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-15T13:47:08.862 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-15T13:47:08.869 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-15T13:47:08.899 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-15T13:47:08.923 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-15T13:47:08.947 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-15T13:47:08.981 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-15T13:47:09.014 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-15T13:47:09.025 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-15T13:47:09.050 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-15T13:47:09.078 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-15T13:47:09.221 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-15T13:47:09.234 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-15T13:47:09.264 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-15T13:47:09.287 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-15T13:47:09.294 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-15T13:47:09.345 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-15T13:47:09.369 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-15T13:47:09.444 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-15T13:47:09.477 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-15T13:47:09.519 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-15T13:47:09.551 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-15T13:47:09.575 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-15T13:47:09.685 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-15T13:47:09.817 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-15T13:47:09.850 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-15T13:47:09.886 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-15T13:47:09.921 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-15T13:47:09.960 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-15T13:47:10.000 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-15T13:47:10.037 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-15T13:47:10.071 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-15T13:47:10.072 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-15T13:47:10.113 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-15T13:47:10.156 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-15T13:47:10.189 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-15T13:47:10.191 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-15T13:47:10.281 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-15T13:47:10.329 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-15T13:47:10.359 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-15T13:47:10.410 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-15T13:47:10.522 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-15T13:47:10.581 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-15T13:47:11.365 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-15T13:47:11.410 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-15T13:47:11.444 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-15T13:47:11.521 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-15T13:47:11.552 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-15T13:47:11.601 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-15T13:47:11.612 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-15T13:47:11.630 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-15T13:47:11.662 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-15T13:47:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-15T13:47:11.738 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-15T13:47:11.926 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-15T13:47:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-15T13:47:11.991 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-15T13:47:12.033 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 101/164 2024-05-15T13:47:12.034 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-15T13:47:12.125 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 101/164 2024-05-15T13:47:12.155 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 102/164 2024-05-15T13:47:12.200 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-15T13:47:12.705 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-15T13:47:12.732 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-15T13:47:12.736 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 105/164 2024-05-15T13:47:12.800 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 105/164 2024-05-15T13:47:12.830 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 106/164 2024-05-15T13:47:12.848 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-15T13:47:14.204 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-15T13:47:14.282 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-15T13:47:14.311 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-15T13:47:14.344 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-15T13:47:14.674 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-15T13:47:14.723 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 101/164 2024-05-15T13:47:14.815 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 101/164 2024-05-15T13:47:14.845 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 102/164 2024-05-15T13:47:14.882 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-15T13:47:15.194 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-15T13:47:15.226 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-15T13:47:15.389 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-15T13:47:15.426 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 105/164 2024-05-15T13:47:15.498 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 105/164 2024-05-15T13:47:15.528 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 106/164 2024-05-15T13:47:15.796 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-15T13:47:15.830 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 108/164 2024-05-15T13:47:15.929 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 108/164 2024-05-15T13:47:15.929 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-15T13:47:15.929 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-15T13:47:15.929 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-15T13:47:15.956 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 109/164 2024-05-15T13:47:17.968 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-15T13:47:18.000 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-15T13:47:18.575 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 107/164 2024-05-15T13:47:18.612 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 108/164 2024-05-15T13:47:18.712 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 108/164 2024-05-15T13:47:18.712 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-15T13:47:18.712 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-15T13:47:18.712 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:47:18.749 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 109/164 2024-05-15T13:47:18.853 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 109/164 2024-05-15T13:47:18.854 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /sys 2024-05-15T13:47:18.854 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /proc 2024-05-15T13:47:18.854 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /mnt 2024-05-15T13:47:18.854 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /var/tmp 2024-05-15T13:47:18.854 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /home 2024-05-15T13:47:18.854 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /root 2024-05-15T13:47:18.854 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /tmp 2024-05-15T13:47:18.854 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-15T13:47:19.025 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 110/164 2024-05-15T13:47:19.040 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 110/164 2024-05-15T13:47:19.041 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T13:47:19.041 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-15T13:47:19.041 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-15T13:47:20.546 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 111/164 2024-05-15T13:47:20.571 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 111/164 2024-05-15T13:47:20.571 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T13:47:20.571 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-15T13:47:20.571 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-15T13:47:20.571 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-15T13:47:20.572 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-15T13:47:20.809 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 112/164 2024-05-15T13:47:20.830 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 112/164 2024-05-15T13:47:20.831 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T13:47:20.831 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-15T13:47:20.831 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-15T13:47:20.831 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-15T13:47:20.831 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-15T13:47:21.268 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 113/164 2024-05-15T13:47:21.290 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 113/164 2024-05-15T13:47:21.290 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T13:47:21.290 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-15T13:47:21.290 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-15T13:47:21.290 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-15T13:47:21.290 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-15T13:47:21.401 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-15T13:47:21.489 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-15T13:47:21.555 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-15T13:47:21.739 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-15T13:47:21.788 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-15T13:47:21.865 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-15T13:47:21.962 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-15T13:47:22.101 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-15T13:47:22.138 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-15T13:47:22.178 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-15T13:47:22.206 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-15T13:47:22.634 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: cephadm-2:19.0.0-3687.g9b638668.el9.noarch 125/164 2024-05-15T13:47:22.663 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : cephadm-2:19.0.0-3687.g9b638668.el9.noarch 125/164 2024-05-15T13:47:22.695 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.e 126/164 2024-05-15T13:47:22.757 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3687.g9b638668. 127/164 2024-05-15T13:47:22.787 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 128/164 2024-05-15T13:47:23.817 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 128/164 2024-05-15T13:47:23.847 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 129/164 2024-05-15T13:47:24.804 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 129/164 2024-05-15T13:47:24.839 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 130/164 2024-05-15T13:47:24.948 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 130/164 2024-05-15T13:47:25.109 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el 131/164 2024-05-15T13:47:25.141 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 132/164 2024-05-15T13:47:25.164 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 132/164 2024-05-15T13:47:25.164 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T13:47:25.164 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-15T13:47:25.164 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-15T13:47:25.165 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-15T13:47:25.165 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-15T13:47:25.208 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 133/164 2024-05-15T13:47:25.219 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 133/164 2024-05-15T13:47:25.318 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-2:19.0.0-3687.g9b638668.el9.x86_64 134/164 2024-05-15T13:47:25.365 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 135/164 2024-05-15T13:47:26.284 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-15T13:47:26.319 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 137/164 2024-05-15T13:47:26.342 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 137/164 2024-05-15T13:47:26.343 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T13:47:26.343 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-15T13:47:26.343 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-15T13:47:26.343 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-15T13:47:26.343 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-15T13:47:26.387 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3687.g9b638 138/164 2024-05-15T13:47:26.409 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3687.g9b638 138/164 2024-05-15T13:47:26.409 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T13:47:26.409 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-15T13:47:26.409 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-15T13:47:26.454 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 139/164 2024-05-15T13:47:26.476 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 139/164 2024-05-15T13:47:26.476 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T13:47:26.477 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-15T13:47:26.477 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-15T13:47:26.477 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-15T13:47:26.477 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-15T13:47:26.788 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 140/164 2024-05-15T13:47:26.811 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 140/164 2024-05-15T13:47:26.811 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T13:47:26.811 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-15T13:47:26.811 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-15T13:47:26.811 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-15T13:47:26.811 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-15T13:47:28.004 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 109/164 2024-05-15T13:47:28.004 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /sys 2024-05-15T13:47:28.004 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /proc 2024-05-15T13:47:28.005 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /mnt 2024-05-15T13:47:28.005 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /var/tmp 2024-05-15T13:47:28.005 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /home 2024-05-15T13:47:28.005 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /root 2024-05-15T13:47:28.005 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /tmp 2024-05-15T13:47:28.005 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-15T13:47:28.180 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 110/164 2024-05-15T13:47:28.196 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 110/164 2024-05-15T13:47:28.196 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T13:47:28.196 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-15T13:47:28.196 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-15T13:47:29.735 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 111/164 2024-05-15T13:47:29.762 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 111/164 2024-05-15T13:47:29.762 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T13:47:29.762 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-15T13:47:29.762 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-15T13:47:29.762 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-15T13:47:29.762 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-15T13:47:29.993 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 112/164 2024-05-15T13:47:30.017 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 112/164 2024-05-15T13:47:30.017 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T13:47:30.017 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-15T13:47:30.017 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-15T13:47:30.017 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-15T13:47:30.017 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-15T13:47:30.484 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 113/164 2024-05-15T13:47:30.508 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 113/164 2024-05-15T13:47:30.508 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T13:47:30.508 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-15T13:47:30.508 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-15T13:47:30.509 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-15T13:47:30.509 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-15T13:47:30.624 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-15T13:47:30.682 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-15T13:47:30.746 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-15T13:47:30.942 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-15T13:47:30.961 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 109/164 2024-05-15T13:47:30.961 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /sys 2024-05-15T13:47:30.961 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /proc 2024-05-15T13:47:30.961 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /mnt 2024-05-15T13:47:30.961 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /var/tmp 2024-05-15T13:47:30.961 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /home 2024-05-15T13:47:30.961 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /root 2024-05-15T13:47:30.961 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /tmp 2024-05-15T13:47:30.961 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:47:31.002 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-15T13:47:31.084 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-15T13:47:31.138 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 110/164 2024-05-15T13:47:31.154 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 110/164 2024-05-15T13:47:31.155 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T13:47:31.155 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-15T13:47:31.155 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:47:31.182 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-15T13:47:31.327 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-15T13:47:31.366 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 141/164 2024-05-15T13:47:31.377 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-15T13:47:31.408 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 142/164 2024-05-15T13:47:31.408 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-15T13:47:31.435 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-15T13:47:31.449 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 143/164 2024-05-15T13:47:31.481 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-15T13:47:31.508 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-15T13:47:31.688 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 146/164 2024-05-15T13:47:31.729 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-15T13:47:31.760 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_ 148/164 2024-05-15T13:47:31.789 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 149/164 2024-05-15T13:47:31.843 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 150/164 2024-05-15T13:47:31.881 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: cephadm-2:19.0.0-3687.g9b638668.el9.noarch 125/164 2024-05-15T13:47:31.910 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : cephadm-2:19.0.0-3687.g9b638668.el9.noarch 125/164 2024-05-15T13:47:31.942 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.e 126/164 2024-05-15T13:47:32.006 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3687.g9b638668. 127/164 2024-05-15T13:47:32.032 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 128/164 2024-05-15T13:47:32.404 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-15T13:47:32.441 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-15T13:47:32.461 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-15T13:47:32.700 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 111/164 2024-05-15T13:47:32.728 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 111/164 2024-05-15T13:47:32.728 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T13:47:32.728 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-15T13:47:32.728 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-15T13:47:32.729 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-15T13:47:32.729 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:47:32.967 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 112/164 2024-05-15T13:47:32.992 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 112/164 2024-05-15T13:47:32.992 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T13:47:32.992 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-15T13:47:32.992 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-15T13:47:32.992 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-15T13:47:32.992 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:47:33.134 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 128/164 2024-05-15T13:47:33.167 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 129/164 2024-05-15T13:47:33.282 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 152/164 2024-05-15T13:47:33.371 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-15T13:47:33.412 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-15T13:47:33.459 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 113/164 2024-05-15T13:47:33.484 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 113/164 2024-05-15T13:47:33.484 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T13:47:33.484 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-15T13:47:33.484 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-15T13:47:33.484 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-15T13:47:33.484 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:47:33.492 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-15T13:47:33.524 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-15T13:47:33.550 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-15T13:47:33.550 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-15T13:47:33.590 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-15T13:47:33.618 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-15T13:47:33.649 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-15T13:47:33.651 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-15T13:47:33.651 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-15T13:47:33.717 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-15T13:47:33.735 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-15T13:47:33.760 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-15T13:47:33.766 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-15T13:47:33.766 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-15T13:47:33.918 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-15T13:47:33.945 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-15T13:47:33.946 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-15T13:47:33.961 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-15T13:47:34.039 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-15T13:47:34.042 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-15T13:47:34.101 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-15T13:47:34.141 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-15T13:47:34.182 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 129/164 2024-05-15T13:47:34.209 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 130/164 2024-05-15T13:47:34.292 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-15T13:47:34.325 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 130/164 2024-05-15T13:47:34.344 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-15T13:47:34.385 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-15T13:47:34.412 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-15T13:47:34.495 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el 131/164 2024-05-15T13:47:34.519 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 132/164 2024-05-15T13:47:34.544 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 132/164 2024-05-15T13:47:34.544 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T13:47:34.544 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-15T13:47:34.544 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-15T13:47:34.544 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-15T13:47:34.544 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-15T13:47:34.595 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 133/164 2024-05-15T13:47:34.606 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 133/164 2024-05-15T13:47:34.708 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-2:19.0.0-3687.g9b638668.el9.x86_64 134/164 2024-05-15T13:47:34.761 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 135/164 2024-05-15T13:47:34.800 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cephadm-2:19.0.0-3687.g9b638668.el9.noarch 125/164 2024-05-15T13:47:34.827 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephadm-2:19.0.0-3687.g9b638668.el9.noarch 125/164 2024-05-15T13:47:34.860 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.e 126/164 2024-05-15T13:47:34.924 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3687.g9b638668. 127/164 2024-05-15T13:47:34.950 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 128/164 2024-05-15T13:47:35.738 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-15T13:47:35.764 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 137/164 2024-05-15T13:47:35.787 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 137/164 2024-05-15T13:47:35.788 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T13:47:35.788 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-15T13:47:35.788 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-15T13:47:35.788 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-15T13:47:35.788 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-15T13:47:35.831 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3687.g9b638 138/164 2024-05-15T13:47:35.854 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3687.g9b638 138/164 2024-05-15T13:47:35.854 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T13:47:35.855 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-15T13:47:35.855 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-15T13:47:35.899 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 139/164 2024-05-15T13:47:35.922 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 139/164 2024-05-15T13:47:35.922 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T13:47:35.922 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-15T13:47:35.923 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-15T13:47:35.923 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-15T13:47:35.923 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-15T13:47:36.057 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 128/164 2024-05-15T13:47:36.094 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 129/164 2024-05-15T13:47:36.249 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 140/164 2024-05-15T13:47:36.273 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 140/164 2024-05-15T13:47:36.273 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T13:47:36.273 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-15T13:47:36.273 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-15T13:47:36.273 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-15T13:47:36.273 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-15T13:47:37.113 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 129/164 2024-05-15T13:47:37.136 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 130/164 2024-05-15T13:47:37.256 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 130/164 2024-05-15T13:47:37.431 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el 131/164 2024-05-15T13:47:37.455 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 132/164 2024-05-15T13:47:37.481 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 132/164 2024-05-15T13:47:37.481 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T13:47:37.481 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-15T13:47:37.481 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-15T13:47:37.481 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-15T13:47:37.481 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:47:37.531 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 133/164 2024-05-15T13:47:37.543 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 133/164 2024-05-15T13:47:37.644 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-2:19.0.0-3687.g9b638668.el9.x86_64 134/164 2024-05-15T13:47:37.711 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 135/164 2024-05-15T13:47:38.705 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-15T13:47:38.733 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 137/164 2024-05-15T13:47:38.758 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 137/164 2024-05-15T13:47:38.758 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T13:47:38.758 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-15T13:47:38.758 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-15T13:47:38.758 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-15T13:47:38.758 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:47:38.801 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3687.g9b638 138/164 2024-05-15T13:47:38.825 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3687.g9b638 138/164 2024-05-15T13:47:38.825 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T13:47:38.825 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-15T13:47:38.826 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:47:38.893 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 139/164 2024-05-15T13:47:38.918 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 139/164 2024-05-15T13:47:38.918 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T13:47:38.918 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-15T13:47:38.918 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-15T13:47:38.918 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-15T13:47:38.918 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:47:39.235 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 140/164 2024-05-15T13:47:39.260 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 140/164 2024-05-15T13:47:39.260 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-15T13:47:39.260 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-15T13:47:39.260 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-15T13:47:39.260 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-15T13:47:39.260 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:47:39.724 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-15T13:47:39.724 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-2:19.0.0-3687.g9b638668.el9.x86_64 1/164 2024-05-15T13:47:39.724 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 2/164 2024-05-15T13:47:39.724 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 3/164 2024-05-15T13:47:39.724 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 4/164 2024-05-15T13:47:39.724 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3687.g9b638 5/164 2024-05-15T13:47:39.724 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 6/164 2024-05-15T13:47:39.725 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 7/164 2024-05-15T13:47:39.725 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-15T13:47:39.725 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-15T13:47:39.725 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 10/164 2024-05-15T13:47:39.725 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 11/164 2024-05-15T13:47:39.725 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 12/164 2024-05-15T13:47:39.725 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-15T13:47:39.725 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 14/164 2024-05-15T13:47:39.725 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_ 15/164 2024-05-15T13:47:39.725 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 16/164 2024-05-15T13:47:39.725 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 17/164 2024-05-15T13:47:39.725 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librados-devel-2:19.0.0-3687.g9b638668.el9.x86_6 18/164 2024-05-15T13:47:39.725 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 19/164 2024-05-15T13:47:39.725 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 20/164 2024-05-15T13:47:39.726 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3687.g9b638668.el 21/164 2024-05-15T13:47:39.726 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-ceph-common-2:19.0.0-3687.g9b638668.el9. 22/164 2024-05-15T13:47:39.726 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_6 23/164 2024-05-15T13:47:39.726 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 24/164 2024-05-15T13:47:39.726 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 25/164 2024-05-15T13:47:39.726 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 26/164 2024-05-15T13:47:39.726 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 27/164 2024-05-15T13:47:39.726 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 28/164 2024-05-15T13:47:39.726 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 29/164 2024-05-15T13:47:39.726 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3687.g9b638668. 30/164 2024-05-15T13:47:39.726 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 31/164 2024-05-15T13:47:39.726 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 32/164 2024-05-15T13:47:39.727 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 33/164 2024-05-15T13:47:39.727 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el 34/164 2024-05-15T13:47:39.727 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 35/164 2024-05-15T13:47:39.727 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.e 36/164 2024-05-15T13:47:39.727 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 37/164 2024-05-15T13:47:39.727 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : cephadm-2:19.0.0-3687.g9b638668.el9.noarch 38/164 2024-05-15T13:47:39.727 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 39/164 2024-05-15T13:47:39.727 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-15T13:47:39.727 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-15T13:47:39.727 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-15T13:47:39.727 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-15T13:47:39.727 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-15T13:47:39.727 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-15T13:47:39.727 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-15T13:47:39.728 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-15T13:47:39.728 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-15T13:47:39.728 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-15T13:47:39.728 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-15T13:47:39.728 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-15T13:47:39.728 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-15T13:47:39.728 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-15T13:47:39.728 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-15T13:47:39.728 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-15T13:47:39.728 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-15T13:47:39.728 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-15T13:47:39.728 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-15T13:47:39.728 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-15T13:47:39.728 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-15T13:47:39.729 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-15T13:47:39.729 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-15T13:47:39.729 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-15T13:47:39.729 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-15T13:47:39.729 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-15T13:47:39.729 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-15T13:47:39.729 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-15T13:47:39.729 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-15T13:47:39.729 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-15T13:47:39.729 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-15T13:47:39.729 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-15T13:47:39.729 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-15T13:47:39.729 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-15T13:47:39.730 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-15T13:47:39.730 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-15T13:47:39.730 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-15T13:47:39.730 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-15T13:47:39.730 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-15T13:47:39.730 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-15T13:47:39.730 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-15T13:47:39.730 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-15T13:47:39.730 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-15T13:47:39.730 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-15T13:47:39.730 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-15T13:47:39.730 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-15T13:47:39.731 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-15T13:47:39.731 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-15T13:47:39.731 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-15T13:47:39.731 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-15T13:47:39.731 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-15T13:47:39.731 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-15T13:47:39.731 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-15T13:47:39.731 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-15T13:47:39.731 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-15T13:47:39.731 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-15T13:47:39.731 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-15T13:47:39.732 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-15T13:47:39.732 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-15T13:47:39.732 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-15T13:47:39.732 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-15T13:47:39.732 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-15T13:47:39.732 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-15T13:47:39.732 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-15T13:47:39.732 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-15T13:47:39.732 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-15T13:47:39.733 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-15T13:47:39.733 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-15T13:47:39.733 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-15T13:47:39.733 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-15T13:47:39.733 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-15T13:47:39.733 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-15T13:47:39.733 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-15T13:47:39.733 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-15T13:47:39.733 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-15T13:47:39.733 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-15T13:47:39.733 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-15T13:47:39.733 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-15T13:47:39.733 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-15T13:47:39.734 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-15T13:47:39.734 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-15T13:47:39.734 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-15T13:47:39.734 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-15T13:47:39.734 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-15T13:47:39.734 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-15T13:47:39.734 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-15T13:47:39.734 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-15T13:47:39.734 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-15T13:47:39.734 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-15T13:47:39.734 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-15T13:47:39.734 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-15T13:47:39.734 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-15T13:47:39.734 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-15T13:47:39.735 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-15T13:47:39.735 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-15T13:47:39.735 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-15T13:47:39.735 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-15T13:47:39.735 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-15T13:47:39.735 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-15T13:47:39.735 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-15T13:47:39.735 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-15T13:47:39.735 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-15T13:47:39.735 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-15T13:47:39.735 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-15T13:47:39.735 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-15T13:47:39.735 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-15T13:47:39.735 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-15T13:47:39.736 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-15T13:47:39.736 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-15T13:47:39.736 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librados2-2:19.0.0-3687.g9b638668.el9.x86_64 149/164 2024-05-15T13:47:39.736 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-15T13:47:39.736 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 151/164 2024-05-15T13:47:39.736 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-15T13:47:39.736 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/164 2024-05-15T13:47:39.736 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-15T13:47:39.736 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/164 2024-05-15T13:47:39.736 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-15T13:47:39.736 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/164 2024-05-15T13:47:39.736 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-15T13:47:39.736 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/164 2024-05-15T13:47:39.736 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-15T13:47:39.737 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-15T13:47:39.737 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-15T13:47:39.737 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-15T13:47:41.128 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 141/164 2024-05-15T13:47:41.170 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 142/164 2024-05-15T13:47:41.211 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 143/164 2024-05-15T13:47:41.242 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-15T13:47:41.269 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-15T13:47:41.468 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 146/164 2024-05-15T13:47:41.515 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-15T13:47:41.545 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_ 148/164 2024-05-15T13:47:41.575 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 149/164 2024-05-15T13:47:41.613 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 150/164 2024-05-15T13:47:42.260 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-15T13:47:42.310 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-15T13:47:42.331 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-15T13:47:42.384 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-15T13:47:42.384 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-15T13:47:42.384 INFO:teuthology.orchestra.run.smithi081.stdout:Upgraded: 2024-05-15T13:47:42.384 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-15T13:47:42.385 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-15T13:47:42.385 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-15T13:47:42.385 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-15T13:47:42.385 INFO:teuthology.orchestra.run.smithi081.stdout: librados2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:42.385 INFO:teuthology.orchestra.run.smithi081.stdout: librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:42.385 INFO:teuthology.orchestra.run.smithi081.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-15T13:47:42.385 INFO:teuthology.orchestra.run.smithi081.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-15T13:47:42.385 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-05-15T13:47:42.385 INFO:teuthology.orchestra.run.smithi081.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-15T13:47:42.385 INFO:teuthology.orchestra.run.smithi081.stdout: automake-1.16.2-8.el9.noarch 2024-05-15T13:47:42.385 INFO:teuthology.orchestra.run.smithi081.stdout: bison-3.7.4-5.el9.x86_64 2024-05-15T13:47:42.385 INFO:teuthology.orchestra.run.smithi081.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-15T13:47:42.385 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:42.385 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:42.385 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:42.386 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:42.386 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-grafana-dashboards-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-15T13:47:42.386 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-immutable-object-cache-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:42.386 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:42.386 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:42.386 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-15T13:47:42.386 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-15T13:47:42.386 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-15T13:47:42.386 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-15T13:47:42.386 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-15T13:47:42.386 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:42.386 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:42.386 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-15T13:47:42.387 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:42.387 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:42.387 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:42.387 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-15T13:47:42.387 INFO:teuthology.orchestra.run.smithi081.stdout: cephadm-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-15T13:47:42.387 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:42.387 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:42.387 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-15T13:47:42.387 INFO:teuthology.orchestra.run.smithi081.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-15T13:47:42.387 INFO:teuthology.orchestra.run.smithi081.stdout: flex-2.6.4-9.el9.x86_64 2024-05-15T13:47:42.387 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-15T13:47:42.387 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-15T13:47:42.387 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-15T13:47:42.387 INFO:teuthology.orchestra.run.smithi081.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-15T13:47:42.387 INFO:teuthology.orchestra.run.smithi081.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-15T13:47:42.388 INFO:teuthology.orchestra.run.smithi081.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-15T13:47:42.388 INFO:teuthology.orchestra.run.smithi081.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-15T13:47:42.388 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-15T13:47:42.388 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-15T13:47:42.388 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:42.388 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:42.388 INFO:teuthology.orchestra.run.smithi081.stdout: libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:42.388 INFO:teuthology.orchestra.run.smithi081.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-15T13:47:42.388 INFO:teuthology.orchestra.run.smithi081.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-15T13:47:42.388 INFO:teuthology.orchestra.run.smithi081.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-15T13:47:42.388 INFO:teuthology.orchestra.run.smithi081.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-15T13:47:42.388 INFO:teuthology.orchestra.run.smithi081.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-15T13:47:42.388 INFO:teuthology.orchestra.run.smithi081.stdout: librados-devel-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:42.389 INFO:teuthology.orchestra.run.smithi081.stdout: libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:42.389 INFO:teuthology.orchestra.run.smithi081.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-15T13:47:42.389 INFO:teuthology.orchestra.run.smithi081.stdout: librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:42.389 INFO:teuthology.orchestra.run.smithi081.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-15T13:47:42.389 INFO:teuthology.orchestra.run.smithi081.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-15T13:47:42.389 INFO:teuthology.orchestra.run.smithi081.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-15T13:47:42.389 INFO:teuthology.orchestra.run.smithi081.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-15T13:47:42.389 INFO:teuthology.orchestra.run.smithi081.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-15T13:47:42.389 INFO:teuthology.orchestra.run.smithi081.stdout: lua-5.4.4-4.el9.x86_64 2024-05-15T13:47:42.389 INFO:teuthology.orchestra.run.smithi081.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-15T13:47:42.389 INFO:teuthology.orchestra.run.smithi081.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-15T13:47:42.389 INFO:teuthology.orchestra.run.smithi081.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-15T13:47:42.389 INFO:teuthology.orchestra.run.smithi081.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-15T13:47:42.390 INFO:teuthology.orchestra.run.smithi081.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-15T13:47:42.390 INFO:teuthology.orchestra.run.smithi081.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-15T13:47:42.390 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-13.14-1.el9.x86_64 2024-05-15T13:47:42.390 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-15T13:47:42.390 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-15T13:47:42.390 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-15T13:47:42.390 INFO:teuthology.orchestra.run.smithi081.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-15T13:47:42.390 INFO:teuthology.orchestra.run.smithi081.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-15T13:47:42.390 INFO:teuthology.orchestra.run.smithi081.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-15T13:47:42.391 INFO:teuthology.orchestra.run.smithi081.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-15T13:47:42.391 INFO:teuthology.orchestra.run.smithi081.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-15T13:47:42.391 INFO:teuthology.orchestra.run.smithi081.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-15T13:47:42.391 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-15T13:47:42.391 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-argparse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:42.391 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:42.391 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:42.391 INFO:teuthology.orchestra.run.smithi081.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-15T13:47:42.391 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-15T13:47:42.391 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-15T13:47:42.391 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-15T13:47:42.391 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-15T13:47:42.391 INFO:teuthology.orchestra.run.smithi081.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-15T13:47:42.391 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-15T13:47:42.391 INFO:teuthology.orchestra.run.smithi081.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-15T13:47:42.392 INFO:teuthology.orchestra.run.smithi081.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-15T13:47:42.392 INFO:teuthology.orchestra.run.smithi081.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-15T13:47:42.392 INFO:teuthology.orchestra.run.smithi081.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-15T13:47:42.392 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-15T13:47:42.392 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-15T13:47:42.392 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-15T13:47:42.392 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-15T13:47:42.392 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-15T13:47:42.392 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-15T13:47:42.392 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-15T13:47:42.393 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-15T13:47:42.393 INFO:teuthology.orchestra.run.smithi081.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-15T13:47:42.393 INFO:teuthology.orchestra.run.smithi081.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-15T13:47:42.393 INFO:teuthology.orchestra.run.smithi081.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-15T13:47:42.393 INFO:teuthology.orchestra.run.smithi081.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-15T13:47:42.393 INFO:teuthology.orchestra.run.smithi081.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-15T13:47:42.393 INFO:teuthology.orchestra.run.smithi081.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-15T13:47:42.393 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-15T13:47:42.393 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-15T13:47:42.393 INFO:teuthology.orchestra.run.smithi081.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-15T13:47:42.393 INFO:teuthology.orchestra.run.smithi081.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-15T13:47:42.393 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-15T13:47:42.393 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ply-3.11-14.el9.noarch 2024-05-15T13:47:42.394 INFO:teuthology.orchestra.run.smithi081.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-15T13:47:42.394 INFO:teuthology.orchestra.run.smithi081.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-15T13:47:42.394 INFO:teuthology.orchestra.run.smithi081.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-15T13:47:42.394 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-15T13:47:42.394 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-15T13:47:42.394 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-15T13:47:42.394 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-15T13:47:42.394 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-15T13:47:42.394 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-15T13:47:42.394 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:42.394 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:42.395 INFO:teuthology.orchestra.run.smithi081.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-15T13:47:42.395 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-15T13:47:42.395 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-15T13:47:42.395 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:42.395 INFO:teuthology.orchestra.run.smithi081.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-15T13:47:42.395 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-15T13:47:42.395 INFO:teuthology.orchestra.run.smithi081.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-15T13:47:42.395 INFO:teuthology.orchestra.run.smithi081.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-15T13:47:42.395 INFO:teuthology.orchestra.run.smithi081.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-15T13:47:42.395 INFO:teuthology.orchestra.run.smithi081.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-15T13:47:42.395 INFO:teuthology.orchestra.run.smithi081.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-15T13:47:42.395 INFO:teuthology.orchestra.run.smithi081.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-15T13:47:42.395 INFO:teuthology.orchestra.run.smithi081.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-15T13:47:42.396 INFO:teuthology.orchestra.run.smithi081.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-15T13:47:42.396 INFO:teuthology.orchestra.run.smithi081.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-15T13:47:42.396 INFO:teuthology.orchestra.run.smithi081.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-15T13:47:42.396 INFO:teuthology.orchestra.run.smithi081.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-15T13:47:42.396 INFO:teuthology.orchestra.run.smithi081.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-15T13:47:42.396 INFO:teuthology.orchestra.run.smithi081.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-15T13:47:42.396 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:42.396 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:42.396 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:42.396 INFO:teuthology.orchestra.run.smithi081.stdout: re2-1:20211101-3.el9.x86_64 2024-05-15T13:47:42.396 INFO:teuthology.orchestra.run.smithi081.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-15T13:47:42.397 INFO:teuthology.orchestra.run.smithi081.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-15T13:47:42.397 INFO:teuthology.orchestra.run.smithi081.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-15T13:47:42.397 INFO:teuthology.orchestra.run.smithi081.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-15T13:47:42.397 INFO:teuthology.orchestra.run.smithi081.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-15T13:47:42.397 INFO:teuthology.orchestra.run.smithi081.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-15T13:47:42.397 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-15T13:47:42.398 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-05-15T13:47:42.816 DEBUG:teuthology.parallel:result is None 2024-05-15T13:47:43.226 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 152/164 2024-05-15T13:47:43.322 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-15T13:47:43.365 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-15T13:47:43.449 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-15T13:47:43.484 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-15T13:47:43.509 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-15T13:47:43.510 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-15T13:47:43.594 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-15T13:47:43.627 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-15T13:47:43.627 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-15T13:47:43.694 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-15T13:47:43.720 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-15T13:47:43.725 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-15T13:47:43.725 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-15T13:47:43.913 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-15T13:47:43.914 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-15T13:47:44.065 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-15T13:47:44.130 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-15T13:47:44.148 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 141/164 2024-05-15T13:47:44.193 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 142/164 2024-05-15T13:47:44.236 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 143/164 2024-05-15T13:47:44.268 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-15T13:47:44.298 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-15T13:47:44.502 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 146/164 2024-05-15T13:47:44.552 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-15T13:47:44.574 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_ 148/164 2024-05-15T13:47:44.603 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 149/164 2024-05-15T13:47:44.641 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 150/164 2024-05-15T13:47:45.308 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-15T13:47:45.347 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-15T13:47:45.368 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-15T13:47:46.296 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 152/164 2024-05-15T13:47:46.385 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-15T13:47:46.438 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-15T13:47:46.546 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-15T13:47:46.579 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-15T13:47:46.605 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-15T13:47:46.605 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-15T13:47:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-15T13:47:46.723 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-15T13:47:46.723 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-15T13:47:46.807 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-15T13:47:46.832 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-15T13:47:46.837 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-15T13:47:46.837 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-15T13:47:47.034 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-15T13:47:47.034 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-15T13:47:47.119 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-15T13:47:47.204 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-15T13:47:49.773 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-15T13:47:49.773 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-2:19.0.0-3687.g9b638668.el9.x86_64 1/164 2024-05-15T13:47:49.773 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 2/164 2024-05-15T13:47:49.773 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 3/164 2024-05-15T13:47:49.774 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 4/164 2024-05-15T13:47:49.774 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3687.g9b638 5/164 2024-05-15T13:47:49.774 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 6/164 2024-05-15T13:47:49.774 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 7/164 2024-05-15T13:47:49.774 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-15T13:47:49.774 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-15T13:47:49.774 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 10/164 2024-05-15T13:47:49.774 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 11/164 2024-05-15T13:47:49.774 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 12/164 2024-05-15T13:47:49.774 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-15T13:47:49.792 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 14/164 2024-05-15T13:47:49.792 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_ 15/164 2024-05-15T13:47:49.792 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 16/164 2024-05-15T13:47:49.792 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 17/164 2024-05-15T13:47:49.792 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librados-devel-2:19.0.0-3687.g9b638668.el9.x86_6 18/164 2024-05-15T13:47:49.792 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 19/164 2024-05-15T13:47:49.792 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 20/164 2024-05-15T13:47:49.792 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3687.g9b638668.el 21/164 2024-05-15T13:47:49.793 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-ceph-common-2:19.0.0-3687.g9b638668.el9. 22/164 2024-05-15T13:47:49.793 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_6 23/164 2024-05-15T13:47:49.793 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 24/164 2024-05-15T13:47:49.793 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 25/164 2024-05-15T13:47:49.793 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 26/164 2024-05-15T13:47:49.793 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 27/164 2024-05-15T13:47:49.793 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 28/164 2024-05-15T13:47:49.793 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 29/164 2024-05-15T13:47:49.793 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3687.g9b638668. 30/164 2024-05-15T13:47:49.793 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 31/164 2024-05-15T13:47:49.793 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 32/164 2024-05-15T13:47:49.793 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 33/164 2024-05-15T13:47:49.794 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el 34/164 2024-05-15T13:47:49.794 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 35/164 2024-05-15T13:47:49.794 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.e 36/164 2024-05-15T13:47:49.794 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 37/164 2024-05-15T13:47:49.794 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cephadm-2:19.0.0-3687.g9b638668.el9.noarch 38/164 2024-05-15T13:47:49.794 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 39/164 2024-05-15T13:47:49.794 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-15T13:47:49.794 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-15T13:47:49.794 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-15T13:47:49.794 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-15T13:47:49.794 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-15T13:47:49.794 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-15T13:47:49.795 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-15T13:47:49.795 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-15T13:47:49.795 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-15T13:47:49.795 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-15T13:47:49.795 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-15T13:47:49.795 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-15T13:47:49.795 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-15T13:47:49.795 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-15T13:47:49.796 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-15T13:47:49.796 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-15T13:47:49.796 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-15T13:47:49.796 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-15T13:47:49.796 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-15T13:47:49.796 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-15T13:47:49.796 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-15T13:47:49.796 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-15T13:47:49.796 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-15T13:47:49.796 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-15T13:47:49.796 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-15T13:47:49.796 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-15T13:47:49.796 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-15T13:47:49.797 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-15T13:47:49.797 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-15T13:47:49.797 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-15T13:47:49.797 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-15T13:47:49.797 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-15T13:47:49.797 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-15T13:47:49.797 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-15T13:47:49.797 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-15T13:47:49.797 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-15T13:47:49.797 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-15T13:47:49.797 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-15T13:47:49.797 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-15T13:47:49.798 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-15T13:47:49.798 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-15T13:47:49.798 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-15T13:47:49.798 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-15T13:47:49.798 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-15T13:47:49.798 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-15T13:47:49.798 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-15T13:47:49.798 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-15T13:47:49.798 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-15T13:47:49.798 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-15T13:47:49.798 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-15T13:47:49.798 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-15T13:47:49.799 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-15T13:47:49.799 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-15T13:47:49.799 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-15T13:47:49.799 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-15T13:47:49.799 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-15T13:47:49.799 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-15T13:47:49.799 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-15T13:47:49.799 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-15T13:47:49.799 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-15T13:47:49.799 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-15T13:47:49.799 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-15T13:47:49.799 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-15T13:47:49.799 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-15T13:47:49.799 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-15T13:47:49.800 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-15T13:47:49.800 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-15T13:47:49.800 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-15T13:47:49.800 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-15T13:47:49.800 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-15T13:47:49.800 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-15T13:47:49.800 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-15T13:47:49.800 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-15T13:47:49.800 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-15T13:47:49.800 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-15T13:47:49.800 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-15T13:47:49.800 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-15T13:47:49.800 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-15T13:47:49.801 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-15T13:47:49.801 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-15T13:47:49.801 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-15T13:47:49.801 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-15T13:47:49.801 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-15T13:47:49.801 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-15T13:47:49.801 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-15T13:47:49.801 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-15T13:47:49.801 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-15T13:47:49.801 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-15T13:47:49.801 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-15T13:47:49.801 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-15T13:47:49.801 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-15T13:47:49.801 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-15T13:47:49.802 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-15T13:47:49.802 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-15T13:47:49.802 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-15T13:47:49.802 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-15T13:47:49.802 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-15T13:47:49.802 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-15T13:47:49.802 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-15T13:47:49.802 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-15T13:47:49.802 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-15T13:47:49.802 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-15T13:47:49.802 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-15T13:47:49.802 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-15T13:47:49.802 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-15T13:47:49.802 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-15T13:47:49.803 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-15T13:47:49.803 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-15T13:47:49.803 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-15T13:47:49.803 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librados2-2:19.0.0-3687.g9b638668.el9.x86_64 149/164 2024-05-15T13:47:49.803 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-15T13:47:49.803 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 151/164 2024-05-15T13:47:49.803 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-15T13:47:49.803 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/164 2024-05-15T13:47:49.803 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-15T13:47:49.803 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/164 2024-05-15T13:47:49.803 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-15T13:47:49.803 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/164 2024-05-15T13:47:49.803 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-15T13:47:49.803 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/164 2024-05-15T13:47:49.803 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-15T13:47:49.804 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-15T13:47:49.804 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-15T13:47:49.804 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-15T13:47:52.613 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-15T13:47:52.614 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-15T13:47:52.614 INFO:teuthology.orchestra.run.smithi052.stdout:Upgraded: 2024-05-15T13:47:52.614 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-15T13:47:52.614 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-15T13:47:52.614 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-15T13:47:52.614 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-15T13:47:52.614 INFO:teuthology.orchestra.run.smithi052.stdout: librados2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:52.614 INFO:teuthology.orchestra.run.smithi052.stdout: librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:52.614 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-15T13:47:52.614 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-15T13:47:52.614 INFO:teuthology.orchestra.run.smithi052.stdout:Installed: 2024-05-15T13:47:52.615 INFO:teuthology.orchestra.run.smithi052.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-15T13:47:52.615 INFO:teuthology.orchestra.run.smithi052.stdout: automake-1.16.2-8.el9.noarch 2024-05-15T13:47:52.615 INFO:teuthology.orchestra.run.smithi052.stdout: bison-3.7.4-5.el9.x86_64 2024-05-15T13:47:52.615 INFO:teuthology.orchestra.run.smithi052.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-15T13:47:52.615 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:52.615 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:52.615 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:52.615 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:52.615 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-grafana-dashboards-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-15T13:47:52.615 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-immutable-object-cache-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:52.615 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:52.615 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:52.615 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-15T13:47:52.616 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-15T13:47:52.616 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-15T13:47:52.616 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-15T13:47:52.616 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-15T13:47:52.616 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:52.616 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:52.616 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-15T13:47:52.616 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:52.616 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:52.616 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:52.616 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-15T13:47:52.616 INFO:teuthology.orchestra.run.smithi052.stdout: cephadm-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-15T13:47:52.616 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:52.616 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:52.616 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-15T13:47:52.617 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-15T13:47:52.617 INFO:teuthology.orchestra.run.smithi052.stdout: flex-2.6.4-9.el9.x86_64 2024-05-15T13:47:52.617 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-15T13:47:52.617 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-15T13:47:52.617 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-15T13:47:52.617 INFO:teuthology.orchestra.run.smithi052.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-15T13:47:52.617 INFO:teuthology.orchestra.run.smithi052.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-15T13:47:52.617 INFO:teuthology.orchestra.run.smithi052.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-15T13:47:52.617 INFO:teuthology.orchestra.run.smithi052.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-15T13:47:52.617 INFO:teuthology.orchestra.run.smithi052.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-15T13:47:52.617 INFO:teuthology.orchestra.run.smithi052.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-15T13:47:52.617 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:52.617 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:52.617 INFO:teuthology.orchestra.run.smithi052.stdout: libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:52.618 INFO:teuthology.orchestra.run.smithi052.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-15T13:47:52.618 INFO:teuthology.orchestra.run.smithi052.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-15T13:47:52.618 INFO:teuthology.orchestra.run.smithi052.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-15T13:47:52.618 INFO:teuthology.orchestra.run.smithi052.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-15T13:47:52.618 INFO:teuthology.orchestra.run.smithi052.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-15T13:47:52.618 INFO:teuthology.orchestra.run.smithi052.stdout: librados-devel-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:52.618 INFO:teuthology.orchestra.run.smithi052.stdout: libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:52.618 INFO:teuthology.orchestra.run.smithi052.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-15T13:47:52.618 INFO:teuthology.orchestra.run.smithi052.stdout: librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:52.618 INFO:teuthology.orchestra.run.smithi052.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-15T13:47:52.618 INFO:teuthology.orchestra.run.smithi052.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-15T13:47:52.618 INFO:teuthology.orchestra.run.smithi052.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-15T13:47:52.619 INFO:teuthology.orchestra.run.smithi052.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-15T13:47:52.619 INFO:teuthology.orchestra.run.smithi052.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-15T13:47:52.619 INFO:teuthology.orchestra.run.smithi052.stdout: lua-5.4.4-4.el9.x86_64 2024-05-15T13:47:52.619 INFO:teuthology.orchestra.run.smithi052.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-15T13:47:52.619 INFO:teuthology.orchestra.run.smithi052.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-15T13:47:52.619 INFO:teuthology.orchestra.run.smithi052.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-15T13:47:52.619 INFO:teuthology.orchestra.run.smithi052.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-15T13:47:52.619 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-15T13:47:52.619 INFO:teuthology.orchestra.run.smithi052.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-15T13:47:52.619 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-13.14-1.el9.x86_64 2024-05-15T13:47:52.619 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-15T13:47:52.619 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-15T13:47:52.619 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-15T13:47:52.620 INFO:teuthology.orchestra.run.smithi052.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-15T13:47:52.620 INFO:teuthology.orchestra.run.smithi052.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-15T13:47:52.620 INFO:teuthology.orchestra.run.smithi052.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-15T13:47:52.620 INFO:teuthology.orchestra.run.smithi052.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-15T13:47:52.620 INFO:teuthology.orchestra.run.smithi052.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-15T13:47:52.620 INFO:teuthology.orchestra.run.smithi052.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-15T13:47:52.620 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-15T13:47:52.620 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-argparse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:52.620 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:52.620 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:52.620 INFO:teuthology.orchestra.run.smithi052.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-15T13:47:52.620 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-15T13:47:52.620 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-15T13:47:52.620 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-15T13:47:52.620 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-15T13:47:52.621 INFO:teuthology.orchestra.run.smithi052.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-15T13:47:52.621 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-15T13:47:52.621 INFO:teuthology.orchestra.run.smithi052.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-15T13:47:52.621 INFO:teuthology.orchestra.run.smithi052.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-15T13:47:52.621 INFO:teuthology.orchestra.run.smithi052.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-15T13:47:52.621 INFO:teuthology.orchestra.run.smithi052.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-15T13:47:52.621 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-15T13:47:52.621 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-15T13:47:52.621 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-15T13:47:52.621 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-15T13:47:52.621 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-15T13:47:52.621 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-15T13:47:52.621 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-15T13:47:52.622 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-15T13:47:52.622 INFO:teuthology.orchestra.run.smithi052.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-15T13:47:52.622 INFO:teuthology.orchestra.run.smithi052.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-15T13:47:52.622 INFO:teuthology.orchestra.run.smithi052.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-15T13:47:52.622 INFO:teuthology.orchestra.run.smithi052.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-15T13:47:52.622 INFO:teuthology.orchestra.run.smithi052.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-15T13:47:52.622 INFO:teuthology.orchestra.run.smithi052.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-15T13:47:52.622 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-15T13:47:52.622 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-15T13:47:52.622 INFO:teuthology.orchestra.run.smithi052.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-15T13:47:52.622 INFO:teuthology.orchestra.run.smithi052.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-15T13:47:52.622 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-15T13:47:52.622 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ply-3.11-14.el9.noarch 2024-05-15T13:47:52.622 INFO:teuthology.orchestra.run.smithi052.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-15T13:47:52.623 INFO:teuthology.orchestra.run.smithi052.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-15T13:47:52.623 INFO:teuthology.orchestra.run.smithi052.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-15T13:47:52.623 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-15T13:47:52.623 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-15T13:47:52.623 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-15T13:47:52.623 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-15T13:47:52.623 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-15T13:47:52.623 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-15T13:47:52.623 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:52.623 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:52.623 INFO:teuthology.orchestra.run.smithi052.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-15T13:47:52.623 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-15T13:47:52.623 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-15T13:47:52.623 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:52.624 INFO:teuthology.orchestra.run.smithi052.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-15T13:47:52.626 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-15T13:47:52.626 INFO:teuthology.orchestra.run.smithi052.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-15T13:47:52.626 INFO:teuthology.orchestra.run.smithi052.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-15T13:47:52.626 INFO:teuthology.orchestra.run.smithi052.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-15T13:47:52.626 INFO:teuthology.orchestra.run.smithi052.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-15T13:47:52.626 INFO:teuthology.orchestra.run.smithi052.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-15T13:47:52.626 INFO:teuthology.orchestra.run.smithi052.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-15T13:47:52.626 INFO:teuthology.orchestra.run.smithi052.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-15T13:47:52.626 INFO:teuthology.orchestra.run.smithi052.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-15T13:47:52.626 INFO:teuthology.orchestra.run.smithi052.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-15T13:47:52.627 INFO:teuthology.orchestra.run.smithi052.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-15T13:47:52.627 INFO:teuthology.orchestra.run.smithi052.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-15T13:47:52.627 INFO:teuthology.orchestra.run.smithi052.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-15T13:47:52.627 INFO:teuthology.orchestra.run.smithi052.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-15T13:47:52.627 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:52.627 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:52.627 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:52.627 INFO:teuthology.orchestra.run.smithi052.stdout: re2-1:20211101-3.el9.x86_64 2024-05-15T13:47:52.627 INFO:teuthology.orchestra.run.smithi052.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-15T13:47:52.627 INFO:teuthology.orchestra.run.smithi052.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-15T13:47:52.627 INFO:teuthology.orchestra.run.smithi052.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-15T13:47:52.627 INFO:teuthology.orchestra.run.smithi052.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-15T13:47:52.627 INFO:teuthology.orchestra.run.smithi052.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-15T13:47:52.628 INFO:teuthology.orchestra.run.smithi052.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-15T13:47:52.628 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-05-15T13:47:52.628 INFO:teuthology.orchestra.run.smithi052.stdout:Complete! 2024-05-15T13:47:52.644 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-15T13:47:52.644 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-2:19.0.0-3687.g9b638668.el9.x86_64 1/164 2024-05-15T13:47:52.644 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 2/164 2024-05-15T13:47:52.644 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 3/164 2024-05-15T13:47:52.644 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 4/164 2024-05-15T13:47:52.644 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3687.g9b638 5/164 2024-05-15T13:47:52.644 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 6/164 2024-05-15T13:47:52.644 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 7/164 2024-05-15T13:47:52.645 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 8/164 2024-05-15T13:47:52.645 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 9/164 2024-05-15T13:47:52.645 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 10/164 2024-05-15T13:47:52.645 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 11/164 2024-05-15T13:47:52.645 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 12/164 2024-05-15T13:47:52.645 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 13/164 2024-05-15T13:47:52.645 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 14/164 2024-05-15T13:47:52.645 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_ 15/164 2024-05-15T13:47:52.645 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 16/164 2024-05-15T13:47:52.645 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 17/164 2024-05-15T13:47:52.645 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados-devel-2:19.0.0-3687.g9b638668.el9.x86_6 18/164 2024-05-15T13:47:52.645 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86 19/164 2024-05-15T13:47:52.645 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 20/164 2024-05-15T13:47:52.645 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3687.g9b638668.el 21/164 2024-05-15T13:47:52.646 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ceph-common-2:19.0.0-3687.g9b638668.el9. 22/164 2024-05-15T13:47:52.646 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_6 23/164 2024-05-15T13:47:52.646 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 24/164 2024-05-15T13:47:52.646 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 25/164 2024-05-15T13:47:52.646 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 26/164 2024-05-15T13:47:52.646 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 27/164 2024-05-15T13:47:52.646 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 28/164 2024-05-15T13:47:52.646 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 29/164 2024-05-15T13:47:52.646 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3687.g9b638668. 30/164 2024-05-15T13:47:52.646 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noa 31/164 2024-05-15T13:47:52.646 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.n 32/164 2024-05-15T13:47:52.646 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b6 33/164 2024-05-15T13:47:52.646 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el 34/164 2024-05-15T13:47:52.647 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 35/164 2024-05-15T13:47:52.647 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.e 36/164 2024-05-15T13:47:52.647 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 37/164 2024-05-15T13:47:52.647 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephadm-2:19.0.0-3687.g9b638668.el9.noarch 38/164 2024-05-15T13:47:52.647 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 39/164 2024-05-15T13:47:52.647 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-15T13:47:52.647 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-15T13:47:52.647 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-15T13:47:52.647 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-15T13:47:52.647 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-15T13:47:52.647 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-15T13:47:52.647 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-15T13:47:52.647 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-15T13:47:52.648 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-15T13:47:52.648 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-15T13:47:52.648 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-15T13:47:52.648 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-15T13:47:52.648 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-15T13:47:52.648 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-15T13:47:52.648 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-15T13:47:52.648 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-15T13:47:52.648 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-15T13:47:52.648 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-15T13:47:52.648 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-15T13:47:52.648 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-15T13:47:52.648 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-15T13:47:52.648 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-15T13:47:52.649 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-15T13:47:52.649 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-15T13:47:52.649 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-15T13:47:52.649 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-15T13:47:52.649 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-15T13:47:52.649 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-15T13:47:52.649 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-15T13:47:52.649 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-15T13:47:52.649 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-15T13:47:52.649 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-15T13:47:52.649 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-15T13:47:52.649 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-15T13:47:52.649 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-15T13:47:52.650 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-15T13:47:52.650 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-15T13:47:52.650 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-15T13:47:52.650 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-15T13:47:52.650 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-15T13:47:52.650 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-15T13:47:52.650 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-15T13:47:52.650 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-15T13:47:52.650 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-15T13:47:52.650 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-15T13:47:52.650 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-15T13:47:52.650 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-15T13:47:52.650 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-15T13:47:52.651 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-15T13:47:52.651 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-15T13:47:52.651 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-15T13:47:52.651 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-15T13:47:52.651 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-15T13:47:52.651 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-15T13:47:52.651 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-15T13:47:52.651 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-15T13:47:52.651 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-15T13:47:52.651 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-15T13:47:52.651 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-15T13:47:52.651 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-15T13:47:52.651 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-15T13:47:52.651 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-15T13:47:52.651 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-15T13:47:52.652 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-15T13:47:52.652 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-15T13:47:52.652 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-15T13:47:52.652 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-15T13:47:52.652 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-15T13:47:52.652 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-15T13:47:52.652 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-15T13:47:52.652 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-15T13:47:52.652 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-15T13:47:52.652 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-15T13:47:52.652 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-15T13:47:52.652 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-15T13:47:52.652 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-15T13:47:52.653 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-15T13:47:52.653 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-15T13:47:52.653 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-15T13:47:52.653 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-15T13:47:52.653 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-15T13:47:52.653 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-15T13:47:52.653 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-15T13:47:52.653 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-15T13:47:52.653 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-15T13:47:52.653 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-15T13:47:52.653 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-15T13:47:52.653 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-15T13:47:52.653 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-15T13:47:52.653 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-15T13:47:52.653 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-15T13:47:52.654 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-15T13:47:52.654 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-15T13:47:52.654 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-15T13:47:52.654 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-15T13:47:52.654 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-15T13:47:52.654 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-15T13:47:52.654 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-15T13:47:52.654 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-15T13:47:52.654 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-15T13:47:52.654 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-15T13:47:52.654 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-15T13:47:52.654 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-15T13:47:52.654 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-15T13:47:52.655 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-15T13:47:52.655 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-15T13:47:52.655 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-15T13:47:52.655 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-15T13:47:52.655 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-15T13:47:52.655 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados2-2:19.0.0-3687.g9b638668.el9.x86_64 149/164 2024-05-15T13:47:52.655 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-15T13:47:52.655 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 151/164 2024-05-15T13:47:52.655 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-15T13:47:52.655 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/164 2024-05-15T13:47:52.655 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-15T13:47:52.655 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/164 2024-05-15T13:47:52.655 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-15T13:47:52.655 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/164 2024-05-15T13:47:52.655 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-15T13:47:52.656 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/164 2024-05-15T13:47:52.656 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-15T13:47:52.656 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-15T13:47:52.656 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-15T13:47:52.656 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-15T13:47:53.102 DEBUG:teuthology.parallel:result is None 2024-05-15T13:47:55.475 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-15T13:47:55.475 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:47:55.475 INFO:teuthology.orchestra.run.smithi033.stdout:Upgraded: 2024-05-15T13:47:55.475 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-15T13:47:55.475 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-15T13:47:55.475 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-15T13:47:55.475 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-15T13:47:55.475 INFO:teuthology.orchestra.run.smithi033.stdout: librados2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:55.475 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:55.476 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-15T13:47:55.476 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-15T13:47:55.476 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-05-15T13:47:55.476 INFO:teuthology.orchestra.run.smithi033.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-15T13:47:55.476 INFO:teuthology.orchestra.run.smithi033.stdout: automake-1.16.2-8.el9.noarch 2024-05-15T13:47:55.476 INFO:teuthology.orchestra.run.smithi033.stdout: bison-3.7.4-5.el9.x86_64 2024-05-15T13:47:55.476 INFO:teuthology.orchestra.run.smithi033.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-15T13:47:55.476 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:55.476 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:55.476 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:55.476 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:55.476 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-15T13:47:55.476 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-immutable-object-cache-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:55.476 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:55.477 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:55.477 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-cephadm-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-15T13:47:55.477 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-15T13:47:55.477 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-15T13:47:55.477 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-15T13:47:55.477 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-15T13:47:55.477 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:55.477 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:55.477 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-prometheus-alerts-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-15T13:47:55.477 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:55.477 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:55.477 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:55.477 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-volume-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-15T13:47:55.477 INFO:teuthology.orchestra.run.smithi033.stdout: cephadm-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-15T13:47:55.478 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:55.478 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-shell-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:55.478 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-top-2:19.0.0-3687.g9b638668.el9.noarch 2024-05-15T13:47:55.478 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-15T13:47:55.478 INFO:teuthology.orchestra.run.smithi033.stdout: flex-2.6.4-9.el9.x86_64 2024-05-15T13:47:55.478 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-15T13:47:55.478 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-15T13:47:55.478 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-15T13:47:55.478 INFO:teuthology.orchestra.run.smithi033.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-15T13:47:55.478 INFO:teuthology.orchestra.run.smithi033.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-15T13:47:55.478 INFO:teuthology.orchestra.run.smithi033.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-15T13:47:55.478 INFO:teuthology.orchestra.run.smithi033.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-15T13:47:55.482 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-15T13:47:55.482 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-15T13:47:55.482 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:55.482 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:55.482 INFO:teuthology.orchestra.run.smithi033.stdout: libcephsqlite-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:55.483 INFO:teuthology.orchestra.run.smithi033.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-15T13:47:55.483 INFO:teuthology.orchestra.run.smithi033.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-15T13:47:55.483 INFO:teuthology.orchestra.run.smithi033.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-15T13:47:55.483 INFO:teuthology.orchestra.run.smithi033.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-15T13:47:55.483 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-15T13:47:55.483 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:55.483 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:55.483 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-15T13:47:55.483 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:55.483 INFO:teuthology.orchestra.run.smithi033.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-15T13:47:55.483 INFO:teuthology.orchestra.run.smithi033.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-15T13:47:55.483 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-15T13:47:55.483 INFO:teuthology.orchestra.run.smithi033.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-15T13:47:55.483 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-15T13:47:55.484 INFO:teuthology.orchestra.run.smithi033.stdout: lua-5.4.4-4.el9.x86_64 2024-05-15T13:47:55.484 INFO:teuthology.orchestra.run.smithi033.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-15T13:47:55.484 INFO:teuthology.orchestra.run.smithi033.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-15T13:47:55.484 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-15T13:47:55.484 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-15T13:47:55.484 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-15T13:47:55.484 INFO:teuthology.orchestra.run.smithi033.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-15T13:47:55.484 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-13.14-1.el9.x86_64 2024-05-15T13:47:55.484 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-15T13:47:55.484 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-15T13:47:55.484 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-15T13:47:55.484 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-15T13:47:55.484 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-15T13:47:55.485 INFO:teuthology.orchestra.run.smithi033.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-15T13:47:55.485 INFO:teuthology.orchestra.run.smithi033.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-15T13:47:55.485 INFO:teuthology.orchestra.run.smithi033.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-15T13:47:55.485 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-15T13:47:55.485 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-15T13:47:55.485 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:55.485 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:55.485 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cephfs-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:55.485 INFO:teuthology.orchestra.run.smithi033.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-15T13:47:55.485 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-15T13:47:55.485 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-15T13:47:55.485 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-15T13:47:55.485 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-15T13:47:55.485 INFO:teuthology.orchestra.run.smithi033.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-15T13:47:55.486 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-15T13:47:55.486 INFO:teuthology.orchestra.run.smithi033.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-15T13:47:55.486 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-15T13:47:55.486 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-15T13:47:55.486 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-15T13:47:55.486 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-15T13:47:55.486 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-15T13:47:55.486 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-15T13:47:55.486 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-15T13:47:55.486 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-15T13:47:55.486 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-15T13:47:55.486 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-15T13:47:55.486 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-15T13:47:55.487 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-15T13:47:55.487 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-15T13:47:55.487 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-15T13:47:55.487 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-15T13:47:55.487 INFO:teuthology.orchestra.run.smithi033.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-15T13:47:55.487 INFO:teuthology.orchestra.run.smithi033.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-15T13:47:55.487 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-15T13:47:55.487 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-15T13:47:55.487 INFO:teuthology.orchestra.run.smithi033.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-15T13:47:55.487 INFO:teuthology.orchestra.run.smithi033.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-15T13:47:55.487 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-15T13:47:55.487 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ply-3.11-14.el9.noarch 2024-05-15T13:47:55.487 INFO:teuthology.orchestra.run.smithi033.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-15T13:47:55.487 INFO:teuthology.orchestra.run.smithi033.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-15T13:47:55.488 INFO:teuthology.orchestra.run.smithi033.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-15T13:47:55.488 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-15T13:47:55.488 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-15T13:47:55.488 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-15T13:47:55.488 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-15T13:47:55.488 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-15T13:47:55.488 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-15T13:47:55.488 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:55.488 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rbd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:55.488 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-15T13:47:55.488 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-15T13:47:55.488 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-15T13:47:55.488 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rgw-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:55.488 INFO:teuthology.orchestra.run.smithi033.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-15T13:47:55.488 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-15T13:47:55.489 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-15T13:47:55.489 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-15T13:47:55.489 INFO:teuthology.orchestra.run.smithi033.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-15T13:47:55.489 INFO:teuthology.orchestra.run.smithi033.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-15T13:47:55.489 INFO:teuthology.orchestra.run.smithi033.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-15T13:47:55.489 INFO:teuthology.orchestra.run.smithi033.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-15T13:47:55.489 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-15T13:47:55.489 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-15T13:47:55.489 INFO:teuthology.orchestra.run.smithi033.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-15T13:47:55.489 INFO:teuthology.orchestra.run.smithi033.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-15T13:47:55.489 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-15T13:47:55.489 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-15T13:47:55.489 INFO:teuthology.orchestra.run.smithi033.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-15T13:47:55.489 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:55.489 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-mirror-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:55.490 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-nbd-2:19.0.0-3687.g9b638668.el9.x86_64 2024-05-15T13:47:55.490 INFO:teuthology.orchestra.run.smithi033.stdout: re2-1:20211101-3.el9.x86_64 2024-05-15T13:47:55.490 INFO:teuthology.orchestra.run.smithi033.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-15T13:47:55.490 INFO:teuthology.orchestra.run.smithi033.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-15T13:47:55.490 INFO:teuthology.orchestra.run.smithi033.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-15T13:47:55.490 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-15T13:47:55.490 INFO:teuthology.orchestra.run.smithi033.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-15T13:47:55.490 INFO:teuthology.orchestra.run.smithi033.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-15T13:47:55.490 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:47:55.491 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-15T13:47:55.920 DEBUG:teuthology.parallel:result is None 2024-05-15T13:47:55.920 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-15T13:47:56.072 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-15T13:47:56.095 INFO:teuthology.orchestra.run.smithi033.stdout:19.0.0-3687.g9b638668.el9 2024-05-15T13:47:56.095 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3687.g9b638668.el9 2024-05-15T13:47:56.096 INFO:teuthology.task.install:The correct ceph version 19.0.0-3687.g9b638668 is installed. 2024-05-15T13:47:56.098 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-15T13:47:56.238 DEBUG:teuthology.orchestra.run.smithi052:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-15T13:47:56.264 INFO:teuthology.orchestra.run.smithi052.stdout:19.0.0-3687.g9b638668.el9 2024-05-15T13:47:56.264 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3687.g9b638668.el9 2024-05-15T13:47:56.264 INFO:teuthology.task.install:The correct ceph version 19.0.0-3687.g9b638668 is installed. 2024-05-15T13:47:56.266 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-15T13:47:56.409 DEBUG:teuthology.orchestra.run.smithi081:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-15T13:47:56.434 INFO:teuthology.orchestra.run.smithi081.stdout:19.0.0-3687.g9b638668.el9 2024-05-15T13:47:56.434 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3687.g9b638668.el9 2024-05-15T13:47:56.434 INFO:teuthology.task.install:The correct ceph version 19.0.0-3687.g9b638668 is installed. 2024-05-15T13:47:56.437 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-15T13:47:56.437 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-15T13:47:56.437 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-15T13:47:56.468 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-15T13:47:56.468 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-15T13:47:56.500 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-15T13:47:56.500 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-15T13:47:56.531 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-15T13:47:56.531 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-15T13:47:56.531 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/daemon-helper 2024-05-15T13:47:56.559 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-15T13:47:56.626 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-15T13:47:56.626 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/usr/bin/daemon-helper 2024-05-15T13:47:56.655 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-15T13:47:56.725 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-15T13:47:56.725 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/daemon-helper 2024-05-15T13:47:56.750 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-15T13:47:56.815 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-15T13:47:56.815 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-15T13:47:56.815 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-15T13:47:56.843 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-15T13:47:56.910 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-15T13:47:56.910 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-15T13:47:56.939 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-15T13:47:57.008 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-15T13:47:57.008 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-15T13:47:57.032 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-15T13:47:57.095 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-15T13:47:57.096 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-15T13:47:57.096 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/stdin-killer 2024-05-15T13:47:57.124 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-15T13:47:57.191 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-15T13:47:57.191 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/usr/bin/stdin-killer 2024-05-15T13:47:57.221 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-15T13:47:57.288 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-15T13:47:57.288 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/stdin-killer 2024-05-15T13:47:57.312 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-15T13:47:57.375 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-15T13:47:57.485 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '9b63866860f0b71216b2c0a73aedd2e0a8a5c98b', 'subvols': {'create': 2}} 2024-05-15T13:47:57.485 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-15T13:47:57.486 INFO:tasks.cephadm:Cluster fsid is bca78ca4-12c1-11ef-bc98-c7b262605968 2024-05-15T13:47:57.486 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-15T13:47:57.486 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.33', 'mon.b': '172.21.15.52', 'mon.c': '172.21.15.81'} 2024-05-15T13:47:57.486 INFO:tasks.cephadm:First mon is mon.a on smithi033 2024-05-15T13:47:57.486 INFO:tasks.cephadm:First mgr is x 2024-05-15T13:47:57.487 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-15T13:47:57.487 DEBUG:teuthology.orchestra.run.smithi033:> sudo hostname $(hostname -s) 2024-05-15T13:47:57.518 DEBUG:teuthology.orchestra.run.smithi052:> sudo hostname $(hostname -s) 2024-05-15T13:47:57.548 DEBUG:teuthology.orchestra.run.smithi081:> sudo hostname $(hostname -s) 2024-05-15T13:47:57.571 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-15T13:47:57.571 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-15T13:47:57.714 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '9b63866860f0b71216b2c0a73aedd2e0a8a5c98b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79371/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3687-g9b638668', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3687.g9b638668'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-14 15:14:57.349842', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiesce', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-05-15T13:47:58.045 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-lusov-quiesce, sha1 9b63866860f0b71216b2c0a73aedd2e0a8a5c98b from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-15T13:47:58.047 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/x86_64/flavors/default/cephadm 2024-05-15T13:47:58.047 INFO:tasks.cephadm:Downloading cephadm from url: https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/x86_64/flavors/default/cephadm 2024-05-15T13:47:58.047 DEBUG:teuthology.orchestra.run.smithi033:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-15T13:47:59.062 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-r--r--. 1 ubuntu ubuntu 782726 May 15 13:47 /home/ubuntu/cephtest/cephadm 2024-05-15T13:47:59.062 DEBUG:teuthology.orchestra.run.smithi052:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-15T13:47:59.763 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r--. 1 ubuntu ubuntu 782726 May 15 13:47 /home/ubuntu/cephtest/cephadm 2024-05-15T13:47:59.763 DEBUG:teuthology.orchestra.run.smithi081:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce/9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-15T13:48:01.167 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 782726 May 15 13:48 /home/ubuntu/cephtest/cephadm 2024-05-15T13:48:01.167 DEBUG:teuthology.orchestra.run.smithi033:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-15T13:48:01.188 DEBUG:teuthology.orchestra.run.smithi052:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-15T13:48:01.207 DEBUG:teuthology.orchestra.run.smithi081:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-15T13:48:01.243 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b on all hosts... 2024-05-15T13:48:01.244 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b pull 2024-05-15T13:48:01.246 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b pull 2024-05-15T13:48:01.250 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b pull 2024-05-15T13:48:02.059 INFO:teuthology.orchestra.run.smithi052.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b... 2024-05-15T13:48:02.094 INFO:teuthology.orchestra.run.smithi033.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b... 2024-05-15T13:48:02.127 INFO:teuthology.orchestra.run.smithi081.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b... 2024-05-15T13:48:30.617 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T13:48:30.617 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_version": "ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev)", 2024-05-15T13:48:30.617 INFO:teuthology.orchestra.run.smithi033.stdout: "image_id": "c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a", 2024-05-15T13:48:30.617 INFO:teuthology.orchestra.run.smithi033.stdout: "repo_digests": [ 2024-05-15T13:48:30.617 INFO:teuthology.orchestra.run.smithi033.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d" 2024-05-15T13:48:30.617 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-15T13:48:30.618 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T13:48:55.517 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-05-15T13:48:55.517 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_version": "ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev)", 2024-05-15T13:48:55.518 INFO:teuthology.orchestra.run.smithi081.stdout: "image_id": "c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a", 2024-05-15T13:48:55.518 INFO:teuthology.orchestra.run.smithi081.stdout: "repo_digests": [ 2024-05-15T13:48:55.518 INFO:teuthology.orchestra.run.smithi081.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d" 2024-05-15T13:48:55.518 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-05-15T13:48:55.518 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-05-15T13:49:01.453 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-15T13:49:01.453 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_version": "ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev)", 2024-05-15T13:49:01.453 INFO:teuthology.orchestra.run.smithi052.stdout: "image_id": "c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a", 2024-05-15T13:49:01.453 INFO:teuthology.orchestra.run.smithi052.stdout: "repo_digests": [ 2024-05-15T13:49:01.453 INFO:teuthology.orchestra.run.smithi052.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d" 2024-05-15T13:49:01.453 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-05-15T13:49:01.454 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-15T13:49:01.479 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /etc/ceph 2024-05-15T13:49:01.514 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /etc/ceph 2024-05-15T13:49:01.542 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /etc/ceph 2024-05-15T13:49:01.574 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 777 /etc/ceph 2024-05-15T13:49:01.604 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 777 /etc/ceph 2024-05-15T13:49:01.628 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 777 /etc/ceph 2024-05-15T13:49:01.656 INFO:tasks.cephadm:Writing seed config... 2024-05-15T13:49:01.657 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-15T13:49:01.657 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-15T13:49:01.657 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-15T13:49:01.658 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-15T13:49:01.658 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-15T13:49:01.658 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-15T13:49:01.658 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-15T13:49:01.658 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-15T13:49:01.658 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-15T13:49:01.658 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-15T13:49:01.658 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-15T13:49:01.658 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-15T13:49:01.658 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-15T13:49:01.658 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-15T13:49:01.658 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-15T13:49:01.658 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-15T13:49:01.658 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-15T13:49:01.659 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-05-15T13:49:01.659 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-15T13:49:01.659 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-15T13:49:01.659 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-15T13:49:01.659 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-15T13:49:01.659 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-05-15T13:49:01.659 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-15T13:49:01.659 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-15T13:49:01.659 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-15T13:49:01.659 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-15T13:49:01.659 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-15T13:49:01.659 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-15T13:49:01.659 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-15T13:49:01.659 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-15T13:49:01.660 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-05-15T13:49:01.660 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-05-15T13:49:01.660 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-05-15T13:49:01.660 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-15T13:49:01.660 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-15T13:49:01.660 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-05-15T13:49:01.660 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-05-15T13:49:01.660 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-15T13:49:01.660 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-15T13:49:01.660 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-05-15T13:49:01.660 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-15T13:49:01.660 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-15T13:49:01.660 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-15T13:49:01.661 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-15T13:49:01.661 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-15T13:49:01.661 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-15T13:49:01.661 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-15T13:49:01.661 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-05-15T13:49:01.662 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-15T13:49:01.662 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-15T13:49:01.679 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = bca78ca4-12c1-11ef-bc98-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-15T13:49:01.679 DEBUG:teuthology.orchestra.run.smithi033:mon.a> sudo journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mon.a.service 2024-05-15T13:49:01.722 DEBUG:teuthology.orchestra.run.smithi033:mgr.x> sudo journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mgr.x.service 2024-05-15T13:49:01.765 INFO:tasks.cephadm:Bootstrapping... 2024-05-15T13:49:01.765 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b -v bootstrap --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.33 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-15T13:49:01.966 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-05-15T13:49:01.966 INFO:teuthology.orchestra.run.smithi033.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b', '-v', 'bootstrap', '--fsid', 'bca78ca4-12c1-11ef-bc98-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.33', '--skip-admin-label'] 2024-05-15T13:49:01.967 INFO:teuthology.orchestra.run.smithi033.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-15T13:49:01.967 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying podman|docker is present... 2024-05-15T13:49:01.994 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stdout 5.0.2 2024-05-15T13:49:01.995 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying lvm2 is present... 2024-05-15T13:49:01.995 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying time synchronization is in place... 2024-05-15T13:49:02.004 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-15T13:49:02.004 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-15T13:49:02.011 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-15T13:49:02.011 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-05-15T13:49:02.019 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout enabled 2024-05-15T13:49:02.026 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout active 2024-05-15T13:49:02.026 INFO:teuthology.orchestra.run.smithi033.stdout:Unit chronyd.service is enabled and running 2024-05-15T13:49:02.026 INFO:teuthology.orchestra.run.smithi033.stdout:Repeating the final host check... 2024-05-15T13:49:02.053 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stdout 5.0.2 2024-05-15T13:49:02.053 INFO:teuthology.orchestra.run.smithi033.stdout:podman (/bin/podman) version 5.0.2 is present 2024-05-15T13:49:02.053 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl is present 2024-05-15T13:49:02.053 INFO:teuthology.orchestra.run.smithi033.stdout:lvcreate is present 2024-05-15T13:49:02.060 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-15T13:49:02.060 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-15T13:49:02.067 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-15T13:49:02.067 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-05-15T13:49:02.074 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout enabled 2024-05-15T13:49:02.081 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout active 2024-05-15T13:49:02.081 INFO:teuthology.orchestra.run.smithi033.stdout:Unit chronyd.service is enabled and running 2024-05-15T13:49:02.081 INFO:teuthology.orchestra.run.smithi033.stdout:Host looks OK 2024-05-15T13:49:02.081 INFO:teuthology.orchestra.run.smithi033.stdout:Cluster fsid: bca78ca4-12c1-11ef-bc98-c7b262605968 2024-05-15T13:49:02.082 INFO:teuthology.orchestra.run.smithi033.stdout:Acquiring lock 140705453055520 on /run/cephadm/bca78ca4-12c1-11ef-bc98-c7b262605968.lock 2024-05-15T13:49:02.082 INFO:teuthology.orchestra.run.smithi033.stdout:Lock 140705453055520 acquired on /run/cephadm/bca78ca4-12c1-11ef-bc98-c7b262605968.lock 2024-05-15T13:49:02.082 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying IP 172.21.15.33 port 3300 ... 2024-05-15T13:49:02.082 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying IP 172.21.15.33 port 6789 ... 2024-05-15T13:49:02.083 INFO:teuthology.orchestra.run.smithi033.stdout:Base mon IP(s) is [172.21.15.33:3300, 172.21.15.33:6789], mon addrv is [v2:172.21.15.33:3300,v1:172.21.15.33:6789] 2024-05-15T13:49:02.086 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.33 metric 100 2024-05-15T13:49:02.086 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.33 metric 100 2024-05-15T13:49:02.089 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-15T13:49:02.089 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-05-15T13:49:02.089 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-05-15T13:49:02.091 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-15T13:49:02.092 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-15T13:49:02.092 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-15T13:49:02.092 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-05-15T13:49:02.092 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fea5:d294/64 scope link noprefixroute 2024-05-15T13:49:02.092 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-15T13:49:02.092 INFO:teuthology.orchestra.run.smithi033.stdout:Mon IP `172.21.15.33` is in CIDR network `172.21.0.0/20` 2024-05-15T13:49:02.093 INFO:teuthology.orchestra.run.smithi033.stdout:Mon IP `172.21.15.33` is in CIDR network `172.21.0.0/20` 2024-05-15T13:49:02.093 INFO:teuthology.orchestra.run.smithi033.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-15T13:49:02.093 INFO:teuthology.orchestra.run.smithi033.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-15T13:49:02.094 INFO:teuthology.orchestra.run.smithi033.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b... 2024-05-15T13:49:02.602 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stdout c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a 2024-05-15T13:49:02.602 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b... 2024-05-15T13:49:02.602 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Getting image source signatures 2024-05-15T13:49:02.602 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Copying blob sha256:959e8422c821254139eba7e305bc0e5ccc9179a9c74388bbc27d9cf0e33e5891 2024-05-15T13:49:02.602 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Copying blob sha256:d0e808cd455827ca3108db1b0125dcee9ffc148daa1d7dde963f550bc831d501 2024-05-15T13:49:02.602 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Copying config sha256:c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a 2024-05-15T13:49:02.603 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-15T13:49:03.490 INFO:teuthology.orchestra.run.smithi033.stdout:ceph: stdout ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev) 2024-05-15T13:49:03.490 INFO:teuthology.orchestra.run.smithi033.stdout:Ceph version: ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev) 2024-05-15T13:49:03.490 INFO:teuthology.orchestra.run.smithi033.stdout:Extracting ceph user uid/gid from container image... 2024-05-15T13:49:04.363 INFO:teuthology.orchestra.run.smithi033.stdout:stat: stdout 167 167 2024-05-15T13:49:04.363 INFO:teuthology.orchestra.run.smithi033.stdout:Creating initial keys... 2024-05-15T13:49:05.302 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph-authtool: stdout AQBQvURmFMycKhAAOydeugDkC9v39ugEGO9pgA== 2024-05-15T13:49:06.141 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph-authtool: stdout AQBRvURm+kjrJBAA8Im93ih3FP9pgW7pfDCf8A== 2024-05-15T13:49:07.097 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph-authtool: stdout AQBSvURmquhPHRAARDlqsQ3YORrcL7kfEcs3Aw== 2024-05-15T13:49:07.098 INFO:teuthology.orchestra.run.smithi033.stdout:Creating initial monmap... 2024-05-15T13:49:07.986 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-15T13:49:07.986 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-15T13:49:07.987 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to bca78ca4-12c1-11ef-bc98-c7b262605968 2024-05-15T13:49:07.987 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-15T13:49:07.987 INFO:teuthology.orchestra.run.smithi033.stdout:monmaptool for a [v2:172.21.15.33:3300,v1:172.21.15.33:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-15T13:49:07.987 INFO:teuthology.orchestra.run.smithi033.stdout:setting min_mon_release = quincy 2024-05-15T13:49:07.987 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: set fsid to bca78ca4-12c1-11ef-bc98-c7b262605968 2024-05-15T13:49:07.987 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-15T13:49:07.987 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:49:07.987 INFO:teuthology.orchestra.run.smithi033.stdout:Creating mon... 2024-05-15T13:49:08.917 INFO:teuthology.orchestra.run.smithi033.stdout:create mon.a on 2024-05-15T13:49:09.131 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-15T13:49:09.277 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-15T13:49:09.454 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-bca78ca4-12c1-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-bca78ca4-12c1-11ef-bc98-c7b262605968.target. 2024-05-15T13:49:09.454 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-bca78ca4-12c1-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-bca78ca4-12c1-11ef-bc98-c7b262605968.target. 2024-05-15T13:49:09.691 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mon.a 2024-05-15T13:49:09.691 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to reset failed state of unit ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mon.a.service: Unit ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mon.a.service not loaded. 2024-05-15T13:49:09.867 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-bca78ca4-12c1-11ef-bc98-c7b262605968.target.wants/ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mon.a.service → /etc/systemd/system/ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@.service. 2024-05-15T13:49:10.369 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:10 smithi033 ceph-mon[31275]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-15T13:49:10.383 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-15T13:49:10.383 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-05-15T13:49:10.390 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-15T13:49:10.390 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-05-15T13:49:10.390 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-05-15T13:49:10.390 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to enable service . firewalld.service is not available 2024-05-15T13:49:10.391 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mon to start... 2024-05-15T13:49:10.391 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mon... 2024-05-15T13:49:10.620 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:10 smithi033 bash[31256]: e60973ca96dc3d5c472dca2f5d1d03e25d323af4329b2f396e4845392c313b15 2024-05-15T13:49:10.620 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:10 smithi033 systemd[1]: Started Ceph mon.a for bca78ca4-12c1-11ef-bc98-c7b262605968. 2024-05-15T13:49:11.592 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:11 smithi033 ceph-mon[31275]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-15T13:49:11.592 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:11 smithi033 ceph-mon[31275]: monmap epoch 1 2024-05-15T13:49:11.592 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:11 smithi033 ceph-mon[31275]: fsid bca78ca4-12c1-11ef-bc98-c7b262605968 2024-05-15T13:49:11.592 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:11 smithi033 ceph-mon[31275]: last_changed 2024-05-15T13:49:07.404471+0000 2024-05-15T13:49:11.592 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:11 smithi033 ceph-mon[31275]: created 2024-05-15T13:49:07.404471+0000 2024-05-15T13:49:11.593 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:11 smithi033 ceph-mon[31275]: min_mon_release 19 (squid) 2024-05-15T13:49:11.593 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:11 smithi033 ceph-mon[31275]: election_strategy: 1 2024-05-15T13:49:11.593 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:11 smithi033 ceph-mon[31275]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-05-15T13:49:11.593 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:11 smithi033 ceph-mon[31275]: fsmap 2024-05-15T13:49:11.593 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:11 smithi033 ceph-mon[31275]: osdmap e1: 0 total, 0 up, 0 in 2024-05-15T13:49:11.593 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:11 smithi033 ceph-mon[31275]: mgrmap e1: no daemons active 2024-05-15T13:49:11.593 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:11 smithi033 ceph-mon[31275]: from='client.? 172.21.15.33:0/290389130' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-15T13:49:11.595 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout cluster: 2024-05-15T13:49:11.601 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout id: bca78ca4-12c1-11ef-bc98-c7b262605968 2024-05-15T13:49:11.601 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-15T13:49:11.602 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-15T13:49:11.602 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout services: 2024-05-15T13:49:11.602 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.584657s) 2024-05-15T13:49:11.602 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-15T13:49:11.602 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-15T13:49:11.602 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-15T13:49:11.602 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout data: 2024-05-15T13:49:11.602 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-15T13:49:11.602 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-15T13:49:11.602 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-15T13:49:11.602 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout pgs: 2024-05-15T13:49:11.602 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-15T13:49:11.603 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.830+0000 7f313a5fb700 1 Processor -- start 2024-05-15T13:49:11.603 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.830+0000 7f313a5fb700 1 -- start start 2024-05-15T13:49:11.603 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.830+0000 7f313a5fb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3134109080 0x7f31341094a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:11.603 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.830+0000 7f313a5fb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f31341099e0 con 0x7f3134109080 2024-05-15T13:49:11.603 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.830+0000 7f3133fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3134109080 0x7f31341094a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:11.603 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.830+0000 7f3133fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3134109080 0x7f31341094a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43618/0 (socket says 172.21.15.33:43618) 2024-05-15T13:49:11.603 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.830+0000 7f3133fff700 1 -- 172.21.15.33:0/4225034277 learned_addr learned my addr 172.21.15.33:0/4225034277 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:11.603 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.831+0000 7f3133fff700 1 -- 172.21.15.33:0/4225034277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3134109b60 con 0x7f3134109080 2024-05-15T13:49:11.603 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.831+0000 7f3133fff700 1 --2- 172.21.15.33:0/4225034277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3134109080 0x7f31341094a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f31240098a0 tx=0x7f3124009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae46dac3a27006f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:11.603 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.831+0000 7f3132ffd700 1 -- 172.21.15.33:0/4225034277 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3124004160 con 0x7f3134109080 2024-05-15T13:49:11.603 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.831+0000 7f3132ffd700 1 -- 172.21.15.33:0/4225034277 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3124004300 con 0x7f3134109080 2024-05-15T13:49:11.604 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.831+0000 7f3132ffd700 1 -- 172.21.15.33:0/4225034277 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31240036a0 con 0x7f3134109080 2024-05-15T13:49:11.604 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.832+0000 7f313a5fb700 1 -- 172.21.15.33:0/4225034277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3134109080 msgr2=0x7f31341094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:11.604 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.832+0000 7f313a5fb700 1 --2- 172.21.15.33:0/4225034277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3134109080 0x7f31341094a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f31240098a0 tx=0x7f3124009bb0 comp rx=0 tx=0).stop 2024-05-15T13:49:11.604 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.832+0000 7f313a5fb700 1 -- 172.21.15.33:0/4225034277 shutdown_connections 2024-05-15T13:49:11.604 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.832+0000 7f313a5fb700 1 --2- 172.21.15.33:0/4225034277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3134109080 0x7f31341094a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:11.604 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.832+0000 7f313a5fb700 1 -- 172.21.15.33:0/4225034277 >> 172.21.15.33:0/4225034277 conn(0x7f3134104720 msgr2=0x7f3134106b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:11.604 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.832+0000 7f313a5fb700 1 -- 172.21.15.33:0/4225034277 shutdown_connections 2024-05-15T13:49:11.604 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.832+0000 7f313a5fb700 1 -- 172.21.15.33:0/4225034277 wait complete. 2024-05-15T13:49:11.604 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.832+0000 7f313a5fb700 1 Processor -- start 2024-05-15T13:49:11.604 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.832+0000 7f313a5fb700 1 -- start start 2024-05-15T13:49:11.604 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.833+0000 7f313a5fb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3134109080 0x7f313419c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:11.605 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.833+0000 7f313a5fb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f313410a530 con 0x7f3134109080 2024-05-15T13:49:11.605 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.833+0000 7f3133fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3134109080 0x7f313419c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:11.605 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.833+0000 7f3133fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3134109080 0x7f313419c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43624/0 (socket says 172.21.15.33:43624) 2024-05-15T13:49:11.605 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.833+0000 7f3133fff700 1 -- 172.21.15.33:0/290389130 learned_addr learned my addr 172.21.15.33:0/290389130 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:11.605 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.833+0000 7f3133fff700 1 -- 172.21.15.33:0/290389130 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313419cc80 con 0x7f3134109080 2024-05-15T13:49:11.605 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.833+0000 7f3133fff700 1 --2- 172.21.15.33:0/290389130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3134109080 0x7f313419c740 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3124000c00 tx=0x7f3124004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:11.605 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.833+0000 7f31317fa700 1 -- 172.21.15.33:0/290389130 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3124003db0 con 0x7f3134109080 2024-05-15T13:49:11.605 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.834+0000 7f31317fa700 1 -- 172.21.15.33:0/290389130 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3124011630 con 0x7f3134109080 2024-05-15T13:49:11.605 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.834+0000 7f31317fa700 1 -- 172.21.15.33:0/290389130 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3124011950 con 0x7f3134109080 2024-05-15T13:49:11.605 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.834+0000 7f313a5fb700 1 -- 172.21.15.33:0/290389130 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f313419cf10 con 0x7f3134109080 2024-05-15T13:49:11.605 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.834+0000 7f313a5fb700 1 -- 172.21.15.33:0/290389130 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f313419fc00 con 0x7f3134109080 2024-05-15T13:49:11.606 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.834+0000 7f31317fa700 1 -- 172.21.15.33:0/290389130 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3124011af0 con 0x7f3134109080 2024-05-15T13:49:11.606 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.834+0000 7f31317fa700 1 -- 172.21.15.33:0/290389130 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f312401c450 con 0x7f3134109080 2024-05-15T13:49:11.606 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.834+0000 7f313a5fb700 1 -- 172.21.15.33:0/290389130 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3134064ba0 con 0x7f3134109080 2024-05-15T13:49:11.606 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.837+0000 7f31317fa700 1 -- 172.21.15.33:0/290389130 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f3124004260 con 0x7f3134109080 2024-05-15T13:49:11.606 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.910+0000 7f313a5fb700 1 -- 172.21.15.33:0/290389130 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f313402e6e0 con 0x7f3134109080 2024-05-15T13:49:11.606 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.911+0000 7f31317fa700 1 -- 172.21.15.33:0/290389130 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f312401b450 con 0x7f3134109080 2024-05-15T13:49:11.606 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.911+0000 7f313a5fb700 1 -- 172.21.15.33:0/290389130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3134109080 msgr2=0x7f313419c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:11.606 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.911+0000 7f313a5fb700 1 --2- 172.21.15.33:0/290389130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3134109080 0x7f313419c740 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3124000c00 tx=0x7f3124004750 comp rx=0 tx=0).stop 2024-05-15T13:49:11.606 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.911+0000 7f313a5fb700 1 -- 172.21.15.33:0/290389130 shutdown_connections 2024-05-15T13:49:11.606 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.911+0000 7f313a5fb700 1 --2- 172.21.15.33:0/290389130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3134109080 0x7f313419c740 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:11.606 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.911+0000 7f313a5fb700 1 -- 172.21.15.33:0/290389130 >> 172.21.15.33:0/290389130 conn(0x7f3134104720 msgr2=0x7f3134193330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:11.606 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.911+0000 7f313a5fb700 1 -- 172.21.15.33:0/290389130 shutdown_connections 2024-05-15T13:49:11.606 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:10.911+0000 7f313a5fb700 1 -- 172.21.15.33:0/290389130 wait complete. 2024-05-15T13:49:11.607 INFO:teuthology.orchestra.run.smithi033.stdout:mon is available 2024-05-15T13:49:11.607 INFO:teuthology.orchestra.run.smithi033.stdout:Assimilating anything we can from ceph.conf... 2024-05-15T13:49:12.615 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:12 smithi033 ceph-mon[31275]: from='client.? 172.21.15.33:0/3425152030' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-15T13:49:12.615 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:12 smithi033 ceph-mon[31275]: from='client.? 172.21.15.33:0/3425152030' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-15T13:49:12.625 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-15T13:49:12.625 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [global] 2024-05-15T13:49:12.625 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout fsid = bca78ca4-12c1-11ef-bc98-c7b262605968 2024-05-15T13:49:12.625 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-15T13:49:12.625 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.33:3300,v1:172.21.15.33:6789] 2024-05-15T13:49:12.626 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-15T13:49:12.626 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-15T13:49:12.626 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-15T13:49:12.626 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-15T13:49:12.626 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-15T13:49:12.626 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-15T13:49:12.626 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-15T13:49:12.626 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-15T13:49:12.626 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [osd] 2024-05-15T13:49:12.626 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-05-15T13:49:12.626 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-15T13:49:12.626 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-15T13:49:12.626 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-15T13:49:12.626 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-15T13:49:12.627 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-15T13:49:12.627 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.993+0000 7febef50b700 1 Processor -- start 2024-05-15T13:49:12.627 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.994+0000 7febef50b700 1 -- start start 2024-05-15T13:49:12.627 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.994+0000 7febef50b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febe807ed60 0x7febe807d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:12.627 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.994+0000 7febef50b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7febe807d810 con 0x7febe807ed60 2024-05-15T13:49:12.627 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.994+0000 7febed2a7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febe807ed60 0x7febe807d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:12.627 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.994+0000 7febed2a7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febe807ed60 0x7febe807d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43626/0 (socket says 172.21.15.33:43626) 2024-05-15T13:49:12.627 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.994+0000 7febed2a7700 1 -- 172.21.15.33:0/2270355251 learned_addr learned my addr 172.21.15.33:0/2270355251 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:12.627 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.994+0000 7febed2a7700 1 -- 172.21.15.33:0/2270355251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe807d990 con 0x7febe807ed60 2024-05-15T13:49:12.628 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.994+0000 7febed2a7700 1 --2- 172.21.15.33:0/2270355251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febe807ed60 0x7febe807d2d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7febe40098a0 tx=0x7febe4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a9df0dfc444c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:12.628 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.995+0000 7febdffff700 1 -- 172.21.15.33:0/2270355251 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febe4004160 con 0x7febe807ed60 2024-05-15T13:49:12.628 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.995+0000 7febdffff700 1 -- 172.21.15.33:0/2270355251 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febe4004300 con 0x7febe807ed60 2024-05-15T13:49:12.628 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.995+0000 7febdffff700 1 -- 172.21.15.33:0/2270355251 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febe40036a0 con 0x7febe807ed60 2024-05-15T13:49:12.628 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.995+0000 7febef50b700 1 -- 172.21.15.33:0/2270355251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febe807ed60 msgr2=0x7febe807d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:12.628 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.995+0000 7febef50b700 1 --2- 172.21.15.33:0/2270355251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febe807ed60 0x7febe807d2d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7febe40098a0 tx=0x7febe4009bb0 comp rx=0 tx=0).stop 2024-05-15T13:49:12.628 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.995+0000 7febef50b700 1 -- 172.21.15.33:0/2270355251 shutdown_connections 2024-05-15T13:49:12.628 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.995+0000 7febef50b700 1 --2- 172.21.15.33:0/2270355251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febe807ed60 0x7febe807d2d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:12.628 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.995+0000 7febef50b700 1 -- 172.21.15.33:0/2270355251 >> 172.21.15.33:0/2270355251 conn(0x7febe8104720 msgr2=0x7febe8106b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:12.628 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.995+0000 7febef50b700 1 -- 172.21.15.33:0/2270355251 shutdown_connections 2024-05-15T13:49:12.628 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.995+0000 7febef50b700 1 -- 172.21.15.33:0/2270355251 wait complete. 2024-05-15T13:49:12.628 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.996+0000 7febef50b700 1 Processor -- start 2024-05-15T13:49:12.628 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.996+0000 7febef50b700 1 -- start start 2024-05-15T13:49:12.628 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.996+0000 7febef50b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febe807ed60 0x7febe819e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:12.629 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.996+0000 7febef50b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7febe807de60 con 0x7febe807ed60 2024-05-15T13:49:12.629 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.996+0000 7febed2a7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febe807ed60 0x7febe819e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:12.629 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.996+0000 7febed2a7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febe807ed60 0x7febe819e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43634/0 (socket says 172.21.15.33:43634) 2024-05-15T13:49:12.629 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.996+0000 7febed2a7700 1 -- 172.21.15.33:0/3425152030 learned_addr learned my addr 172.21.15.33:0/3425152030 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:12.629 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.997+0000 7febed2a7700 1 -- 172.21.15.33:0/3425152030 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe819eef0 con 0x7febe807ed60 2024-05-15T13:49:12.629 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.997+0000 7febed2a7700 1 --2- 172.21.15.33:0/3425152030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febe807ed60 0x7febe819e9b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7febe4000c00 tx=0x7febe4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:12.629 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.997+0000 7febde7fc700 1 -- 172.21.15.33:0/3425152030 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febe4003db0 con 0x7febe807ed60 2024-05-15T13:49:12.629 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.997+0000 7febde7fc700 1 -- 172.21.15.33:0/3425152030 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febe4011630 con 0x7febe807ed60 2024-05-15T13:49:12.629 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.997+0000 7febef50b700 1 -- 172.21.15.33:0/3425152030 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febe819f180 con 0x7febe807ed60 2024-05-15T13:49:12.629 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.997+0000 7febde7fc700 1 -- 172.21.15.33:0/3425152030 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febe4011950 con 0x7febe807ed60 2024-05-15T13:49:12.629 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.997+0000 7febef50b700 1 -- 172.21.15.33:0/3425152030 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febe81a1e20 con 0x7febe807ed60 2024-05-15T13:49:12.630 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.997+0000 7febde7fc700 1 -- 172.21.15.33:0/3425152030 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7febe4011af0 con 0x7febe807ed60 2024-05-15T13:49:12.630 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.997+0000 7febde7fc700 1 -- 172.21.15.33:0/3425152030 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7febe401c450 con 0x7febe807ed60 2024-05-15T13:49:12.630 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:11.998+0000 7febef50b700 1 -- 172.21.15.33:0/3425152030 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febe8064ba0 con 0x7febe807ed60 2024-05-15T13:49:12.630 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:12.000+0000 7febde7fc700 1 -- 172.21.15.33:0/3425152030 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7febe4032020 con 0x7febe807ed60 2024-05-15T13:49:12.630 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:12.071+0000 7febef50b700 1 -- 172.21.15.33:0/3425152030 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7febe802e790 con 0x7febe807ed60 2024-05-15T13:49:12.630 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:12.074+0000 7febde7fc700 1 -- 172.21.15.33:0/3425152030 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+492 (secure 0 0 0) 0x7febe4004260 con 0x7febe807ed60 2024-05-15T13:49:12.630 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:12.075+0000 7febde7fc700 1 -- 172.21.15.33:0/3425152030 <== mon.0 v2:172.21.15.33:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7febe401c6a0 con 0x7febe807ed60 2024-05-15T13:49:12.630 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:12.076+0000 7febef50b700 1 -- 172.21.15.33:0/3425152030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febe807ed60 msgr2=0x7febe819e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:12.630 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:12.076+0000 7febef50b700 1 --2- 172.21.15.33:0/3425152030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febe807ed60 0x7febe819e9b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7febe4000c00 tx=0x7febe4004750 comp rx=0 tx=0).stop 2024-05-15T13:49:12.630 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:12.076+0000 7febef50b700 1 -- 172.21.15.33:0/3425152030 shutdown_connections 2024-05-15T13:49:12.630 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:12.076+0000 7febef50b700 1 --2- 172.21.15.33:0/3425152030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febe807ed60 0x7febe819e9b0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:12.630 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:12.076+0000 7febef50b700 1 -- 172.21.15.33:0/3425152030 >> 172.21.15.33:0/3425152030 conn(0x7febe8104720 msgr2=0x7febe8106aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:12.630 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:12.076+0000 7febef50b700 1 -- 172.21.15.33:0/3425152030 shutdown_connections 2024-05-15T13:49:12.631 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:12.076+0000 7febef50b700 1 -- 172.21.15.33:0/3425152030 wait complete. 2024-05-15T13:49:12.631 INFO:teuthology.orchestra.run.smithi033.stdout:Generating new minimal ceph.conf... 2024-05-15T13:49:13.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:13 smithi033 ceph-mon[31275]: from='client.? 172.21.15.33:0/1068501560' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:49:13.737 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.035+0000 7f83ab067700 1 Processor -- start 2024-05-15T13:49:13.737 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.035+0000 7f83ab067700 1 -- start start 2024-05-15T13:49:13.737 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.035+0000 7f83ab067700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83a4109080 0x7f83a41094a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:13.737 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.035+0000 7f83ab067700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f83a41099e0 con 0x7f83a4109080 2024-05-15T13:49:13.737 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.036+0000 7f83a8e03700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83a4109080 0x7f83a41094a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:13.738 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.036+0000 7f83a8e03700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83a4109080 0x7f83a41094a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55838/0 (socket says 172.21.15.33:55838) 2024-05-15T13:49:13.738 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.036+0000 7f83a8e03700 1 -- 172.21.15.33:0/3864264847 learned_addr learned my addr 172.21.15.33:0/3864264847 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:13.738 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.036+0000 7f83a8e03700 1 -- 172.21.15.33:0/3864264847 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a4109b60 con 0x7f83a4109080 2024-05-15T13:49:13.738 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.036+0000 7f83a8e03700 1 --2- 172.21.15.33:0/3864264847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83a4109080 0x7f83a41094a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8394009e80 tx=0x7f839400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c66b557647de93b server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:13.738 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.036+0000 7f83a37fe700 1 -- 172.21.15.33:0/3864264847 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8394017070 con 0x7f83a4109080 2024-05-15T13:49:13.738 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.036+0000 7f83a37fe700 1 -- 172.21.15.33:0/3864264847 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f839400b8c0 con 0x7f83a4109080 2024-05-15T13:49:13.738 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.036+0000 7f83a37fe700 1 -- 172.21.15.33:0/3864264847 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8394004490 con 0x7f83a4109080 2024-05-15T13:49:13.738 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.037+0000 7f83ab067700 1 -- 172.21.15.33:0/3864264847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83a4109080 msgr2=0x7f83a41094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:13.738 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.037+0000 7f83ab067700 1 --2- 172.21.15.33:0/3864264847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83a4109080 0x7f83a41094a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8394009e80 tx=0x7f839400b330 comp rx=0 tx=0).stop 2024-05-15T13:49:13.738 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.037+0000 7f83ab067700 1 -- 172.21.15.33:0/3864264847 shutdown_connections 2024-05-15T13:49:13.738 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.037+0000 7f83ab067700 1 --2- 172.21.15.33:0/3864264847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83a4109080 0x7f83a41094a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:13.738 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.037+0000 7f83ab067700 1 -- 172.21.15.33:0/3864264847 >> 172.21.15.33:0/3864264847 conn(0x7f83a4104720 msgr2=0x7f83a4106b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:13.738 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.037+0000 7f83ab067700 1 -- 172.21.15.33:0/3864264847 shutdown_connections 2024-05-15T13:49:13.738 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.037+0000 7f83ab067700 1 -- 172.21.15.33:0/3864264847 wait complete. 2024-05-15T13:49:13.739 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.038+0000 7f83ab067700 1 Processor -- start 2024-05-15T13:49:13.739 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.038+0000 7f83ab067700 1 -- start start 2024-05-15T13:49:13.739 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.038+0000 7f83ab067700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83a4109080 0x7f83a407eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:13.739 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.038+0000 7f83ab067700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f83a410b080 con 0x7f83a4109080 2024-05-15T13:49:13.739 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.038+0000 7f83a8e03700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83a4109080 0x7f83a407eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:13.739 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.038+0000 7f83a8e03700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83a4109080 0x7f83a407eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55844/0 (socket says 172.21.15.33:55844) 2024-05-15T13:49:13.739 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.038+0000 7f83a8e03700 1 -- 172.21.15.33:0/1068501560 learned_addr learned my addr 172.21.15.33:0/1068501560 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:13.739 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.038+0000 7f83a8e03700 1 -- 172.21.15.33:0/1068501560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a407f0a0 con 0x7f83a4109080 2024-05-15T13:49:13.739 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.038+0000 7f83a8e03700 1 --2- 172.21.15.33:0/1068501560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83a4109080 0x7f83a407eb60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f839400b7b0 tx=0x7f839400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:13.739 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.039+0000 7f83a1ffb700 1 -- 172.21.15.33:0/1068501560 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8394020020 con 0x7f83a4109080 2024-05-15T13:49:13.739 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.039+0000 7f83a1ffb700 1 -- 172.21.15.33:0/1068501560 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8394012430 con 0x7f83a4109080 2024-05-15T13:49:13.740 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.039+0000 7f83ab067700 1 -- 172.21.15.33:0/1068501560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a407d290 con 0x7f83a4109080 2024-05-15T13:49:13.740 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.039+0000 7f83a1ffb700 1 -- 172.21.15.33:0/1068501560 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8394017040 con 0x7f83a4109080 2024-05-15T13:49:13.740 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.039+0000 7f83ab067700 1 -- 172.21.15.33:0/1068501560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a407d7d0 con 0x7f83a4109080 2024-05-15T13:49:13.740 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.039+0000 7f83a1ffb700 1 -- 172.21.15.33:0/1068501560 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8394023020 con 0x7f83a4109080 2024-05-15T13:49:13.740 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.039+0000 7f83a1ffb700 1 -- 172.21.15.33:0/1068501560 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f839400f050 con 0x7f83a4109080 2024-05-15T13:49:13.740 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.040+0000 7f83ab067700 1 -- 172.21.15.33:0/1068501560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83a4064ba0 con 0x7f83a4109080 2024-05-15T13:49:13.740 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.042+0000 7f83a1ffb700 1 -- 172.21.15.33:0/1068501560 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f839402a020 con 0x7f83a4109080 2024-05-15T13:49:13.740 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.113+0000 7f83ab067700 1 -- 172.21.15.33:0/1068501560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f83a407da20 con 0x7f83a4109080 2024-05-15T13:49:13.740 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.113+0000 7f83a1ffb700 1 -- 172.21.15.33:0/1068501560 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f839400b860 con 0x7f83a4109080 2024-05-15T13:49:13.740 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.114+0000 7f83ab067700 1 -- 172.21.15.33:0/1068501560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83a4109080 msgr2=0x7f83a407eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:13.740 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.114+0000 7f83ab067700 1 --2- 172.21.15.33:0/1068501560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83a4109080 0x7f83a407eb60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f839400b7b0 tx=0x7f839400be20 comp rx=0 tx=0).stop 2024-05-15T13:49:13.740 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.114+0000 7f83ab067700 1 -- 172.21.15.33:0/1068501560 shutdown_connections 2024-05-15T13:49:13.741 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.114+0000 7f83ab067700 1 --2- 172.21.15.33:0/1068501560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83a4109080 0x7f83a407eb60 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:13.741 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.114+0000 7f83ab067700 1 -- 172.21.15.33:0/1068501560 >> 172.21.15.33:0/1068501560 conn(0x7f83a4104720 msgr2=0x7f83a4105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:13.741 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.114+0000 7f83ab067700 1 -- 172.21.15.33:0/1068501560 shutdown_connections 2024-05-15T13:49:13.741 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:13.114+0000 7f83ab067700 1 -- 172.21.15.33:0/1068501560 wait complete. 2024-05-15T13:49:13.741 INFO:teuthology.orchestra.run.smithi033.stdout:Restarting the monitor... 2024-05-15T13:49:14.001 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:13 smithi033 systemd[1]: Stopping Ceph mon.a for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T13:49:14.367 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:13 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mon-a[31271]: 2024-05-15T13:49:13.997+0000 7f3db70c6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T13:49:14.367 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:13 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mon-a[31271]: 2024-05-15T13:49:13.997+0000 7f3db70c6700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-15T13:49:14.367 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:14 smithi033 podman[31523]: 2024-05-15 13:49:14.110710879 +0000 UTC m=+0.207510060 container died e60973ca96dc3d5c472dca2f5d1d03e25d323af4329b2f396e4845392c313b15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0) 2024-05-15T13:49:14.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:14 smithi033 podman[31523]: 2024-05-15 13:49:14.472377606 +0000 UTC m=+0.569176784 container cleanup e60973ca96dc3d5c472dca2f5d1d03e25d323af4329b2f396e4845392c313b15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mon-a, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-15T13:49:14.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:14 smithi033 bash[31523]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mon-a 2024-05-15T13:49:15.051 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:14 smithi033 podman[31534]: 2024-05-15 13:49:14.715044999 +0000 UTC m=+0.607637745 container remove e60973ca96dc3d5c472dca2f5d1d03e25d323af4329b2f396e4845392c313b15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866) 2024-05-15T13:49:15.051 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:14 smithi033 systemd[1]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-15T13:49:15.051 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:14 smithi033 systemd[1]: Stopped Ceph mon.a for bca78ca4-12c1-11ef-bc98-c7b262605968. 2024-05-15T13:49:15.051 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:14 smithi033 systemd[1]: Starting Ceph mon.a for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T13:49:15.304 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 podman[31631]: 2024-05-15 13:49:15.049154962 +0000 UTC m=+0.104158465 container create 40d78556a6be0d1051a52d5c5ad4f7b3dcb7b9fc62a70f200f75e93206213a0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mon-a, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-15T13:49:15.304 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 podman[31631]: 2024-05-15 13:49:14.962558705 +0000 UTC m=+0.017562177 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-15T13:49:15.304 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 podman[31631]: 2024-05-15 13:49:15.226753498 +0000 UTC m=+0.281756970 container init 40d78556a6be0d1051a52d5c5ad4f7b3dcb7b9fc62a70f200f75e93206213a0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mon-a, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5) 2024-05-15T13:49:15.304 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 podman[31631]: 2024-05-15 13:49:15.229968719 +0000 UTC m=+0.284972180 container start 40d78556a6be0d1051a52d5c5ad4f7b3dcb7b9fc62a70f200f75e93206213a0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mon-a, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-15T13:49:15.304 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: set uid:gid to 167:167 (ceph:ceph) 2024-05-15T13:49:15.304 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev), process ceph-mon, pid 2 2024-05-15T13:49:15.305 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: pidfile_write: ignore empty --pid-file 2024-05-15T13:49:15.305 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: load: jerasure load: lrc 2024-05-15T13:49:15.305 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: RocksDB version: 7.9.2 2024-05-15T13:49:15.305 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Git sha 0 2024-05-15T13:49:15.305 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Compile date 2024-05-14 14:11:45 2024-05-15T13:49:15.305 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: DB SUMMARY 2024-05-15T13:49:15.305 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: DB Session ID: XAAUA4Z0VLPTSHKAX8TQ 2024-05-15T13:49:15.305 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: CURRENT file: CURRENT 2024-05-15T13:49:15.305 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: IDENTITY file: IDENTITY 2024-05-15T13:49:15.305 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-15T13:49:15.305 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-15T13:49:15.305 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105438 ; 2024-05-15T13:49:15.305 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.error_if_exists: 0 2024-05-15T13:49:15.307 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.create_if_missing: 0 2024-05-15T13:49:15.307 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.paranoid_checks: 1 2024-05-15T13:49:15.307 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-15T13:49:15.307 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-15T13:49:15.307 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-15T13:49:15.307 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.env: 0x55f1d8b2dcc0 2024-05-15T13:49:15.307 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.fs: PosixFileSystem 2024-05-15T13:49:15.308 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.info_log: 0x55f1dac59da0 2024-05-15T13:49:15.308 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.max_file_opening_threads: 16 2024-05-15T13:49:15.308 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.statistics: (nil) 2024-05-15T13:49:15.308 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.use_fsync: 0 2024-05-15T13:49:15.308 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.max_log_file_size: 0 2024-05-15T13:49:15.308 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-15T13:49:15.308 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-15T13:49:15.308 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.keep_log_file_num: 1000 2024-05-15T13:49:15.308 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.recycle_log_file_num: 0 2024-05-15T13:49:15.308 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.allow_fallocate: 1 2024-05-15T13:49:15.309 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.allow_mmap_reads: 0 2024-05-15T13:49:15.309 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.allow_mmap_writes: 0 2024-05-15T13:49:15.309 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.use_direct_reads: 0 2024-05-15T13:49:15.309 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-15T13:49:15.309 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.create_missing_column_families: 0 2024-05-15T13:49:15.309 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.db_log_dir: 2024-05-15T13:49:15.309 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.wal_dir: 2024-05-15T13:49:15.309 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-15T13:49:15.309 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-15T13:49:15.309 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-15T13:49:15.309 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-15T13:49:15.310 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-15T13:49:15.310 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-15T13:49:15.310 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.advise_random_on_open: 1 2024-05-15T13:49:15.310 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.db_write_buffer_size: 0 2024-05-15T13:49:15.310 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.write_buffer_manager: 0x55f1dac2b0e0 2024-05-15T13:49:15.310 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-15T13:49:15.310 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-15T13:49:15.310 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-15T13:49:15.310 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.rate_limiter: (nil) 2024-05-15T13:49:15.310 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-15T13:49:15.311 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.wal_recovery_mode: 2 2024-05-15T13:49:15.311 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.enable_thread_tracking: 0 2024-05-15T13:49:15.311 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.enable_pipelined_write: 0 2024-05-15T13:49:15.311 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.unordered_write: 0 2024-05-15T13:49:15.311 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-15T13:49:15.311 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-15T13:49:15.311 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-15T13:49:15.311 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-15T13:49:15.311 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.row_cache: None 2024-05-15T13:49:15.311 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.wal_filter: None 2024-05-15T13:49:15.311 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-15T13:49:15.312 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.allow_ingest_behind: 0 2024-05-15T13:49:15.312 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.two_write_queues: 0 2024-05-15T13:49:15.312 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.manual_wal_flush: 0 2024-05-15T13:49:15.312 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.wal_compression: 0 2024-05-15T13:49:15.312 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.atomic_flush: 0 2024-05-15T13:49:15.312 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-15T13:49:15.312 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-15T13:49:15.312 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-15T13:49:15.312 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.log_readahead_size: 0 2024-05-15T13:49:15.312 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-15T13:49:15.312 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.best_efforts_recovery: 0 2024-05-15T13:49:15.313 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-15T13:49:15.313 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-15T13:49:15.313 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.allow_data_in_errors: 0 2024-05-15T13:49:15.313 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.db_host_id: __hostname__ 2024-05-15T13:49:15.313 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-15T13:49:15.313 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.max_background_jobs: 2 2024-05-15T13:49:15.313 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.max_background_compactions: -1 2024-05-15T13:49:15.313 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.max_subcompactions: 1 2024-05-15T13:49:15.313 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-15T13:49:15.313 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-15T13:49:15.313 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-15T13:49:15.313 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.max_total_wal_size: 0 2024-05-15T13:49:15.313 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-15T13:49:15.313 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-15T13:49:15.313 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-15T13:49:15.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-15T13:49:15.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.max_open_files: -1 2024-05-15T13:49:15.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.bytes_per_sync: 0 2024-05-15T13:49:15.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-15T13:49:15.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-15T13:49:15.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.compaction_readahead_size: 0 2024-05-15T13:49:15.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.max_background_flushes: -1 2024-05-15T13:49:15.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Compression algorithms supported: 2024-05-15T13:49:15.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-15T13:49:15.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: kZSTD supported: 0 2024-05-15T13:49:15.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: kXpressCompression supported: 0 2024-05-15T13:49:15.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: kLZ4HCCompression supported: 1 2024-05-15T13:49:15.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: kZlibCompression supported: 1 2024-05-15T13:49:15.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: kSnappyCompression supported: 1 2024-05-15T13:49:15.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: kLZ4Compression supported: 1 2024-05-15T13:49:15.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: kBZip2Compression supported: 0 2024-05-15T13:49:15.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-15T13:49:15.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-15T13:49:15.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-15T13:49:15.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-15T13:49:15.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-15T13:49:15.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.merge_operator: 2024-05-15T13:49:15.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.compaction_filter: None 2024-05-15T13:49:15.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.compaction_filter_factory: None 2024-05-15T13:49:15.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.sst_partitioner_factory: None 2024-05-15T13:49:15.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-15T13:49:15.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-15T13:49:15.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f1dac59f60) 2024-05-15T13:49:15.315 INFO:journalctl@ceph.mon.a.smithi033.stdout: cache_index_and_filter_blocks: 1 2024-05-15T13:49:15.316 INFO:journalctl@ceph.mon.a.smithi033.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-15T13:49:15.316 INFO:journalctl@ceph.mon.a.smithi033.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-15T13:49:15.316 INFO:journalctl@ceph.mon.a.smithi033.stdout: pin_top_level_index_and_filter: 1 2024-05-15T13:49:15.316 INFO:journalctl@ceph.mon.a.smithi033.stdout: index_type: 0 2024-05-15T13:49:15.316 INFO:journalctl@ceph.mon.a.smithi033.stdout: data_block_index_type: 0 2024-05-15T13:49:15.316 INFO:journalctl@ceph.mon.a.smithi033.stdout: index_shortening: 1 2024-05-15T13:49:15.316 INFO:journalctl@ceph.mon.a.smithi033.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-15T13:49:15.316 INFO:journalctl@ceph.mon.a.smithi033.stdout: checksum: 4 2024-05-15T13:49:15.316 INFO:journalctl@ceph.mon.a.smithi033.stdout: no_block_cache: 0 2024-05-15T13:49:15.316 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_cache: 0x55f1dac5d1f0 2024-05-15T13:49:15.316 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_cache_name: BinnedLRUCache 2024-05-15T13:49:15.316 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_cache_options: 2024-05-15T13:49:15.316 INFO:journalctl@ceph.mon.a.smithi033.stdout: capacity : 536870912 2024-05-15T13:49:15.317 INFO:journalctl@ceph.mon.a.smithi033.stdout: num_shard_bits : 4 2024-05-15T13:49:15.317 INFO:journalctl@ceph.mon.a.smithi033.stdout: strict_capacity_limit : 0 2024-05-15T13:49:15.317 INFO:journalctl@ceph.mon.a.smithi033.stdout: high_pri_pool_ratio: 0.000 2024-05-15T13:49:15.317 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_cache_compressed: (nil) 2024-05-15T13:49:15.317 INFO:journalctl@ceph.mon.a.smithi033.stdout: persistent_cache: (nil) 2024-05-15T13:49:15.317 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_size: 4096 2024-05-15T13:49:15.317 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_size_deviation: 10 2024-05-15T13:49:15.317 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_restart_interval: 16 2024-05-15T13:49:15.317 INFO:journalctl@ceph.mon.a.smithi033.stdout: index_block_restart_interval: 1 2024-05-15T13:49:15.317 INFO:journalctl@ceph.mon.a.smithi033.stdout: metadata_block_size: 4096 2024-05-15T13:49:15.317 INFO:journalctl@ceph.mon.a.smithi033.stdout: partition_filters: 0 2024-05-15T13:49:15.317 INFO:journalctl@ceph.mon.a.smithi033.stdout: use_delta_encoding: 1 2024-05-15T13:49:15.317 INFO:journalctl@ceph.mon.a.smithi033.stdout: filter_policy: bloomfilter 2024-05-15T13:49:15.317 INFO:journalctl@ceph.mon.a.smithi033.stdout: whole_key_filtering: 1 2024-05-15T13:49:15.317 INFO:journalctl@ceph.mon.a.smithi033.stdout: verify_compression: 0 2024-05-15T13:49:15.318 INFO:journalctl@ceph.mon.a.smithi033.stdout: read_amp_bytes_per_bit: 0 2024-05-15T13:49:15.318 INFO:journalctl@ceph.mon.a.smithi033.stdout: format_version: 5 2024-05-15T13:49:15.318 INFO:journalctl@ceph.mon.a.smithi033.stdout: enable_index_compression: 1 2024-05-15T13:49:15.318 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_align: 0 2024-05-15T13:49:15.318 INFO:journalctl@ceph.mon.a.smithi033.stdout: max_auto_readahead_size: 262144 2024-05-15T13:49:15.318 INFO:journalctl@ceph.mon.a.smithi033.stdout: prepopulate_block_cache: 0 2024-05-15T13:49:15.318 INFO:journalctl@ceph.mon.a.smithi033.stdout: initial_auto_readahead_size: 8192 2024-05-15T13:49:15.318 INFO:journalctl@ceph.mon.a.smithi033.stdout: num_file_reads_for_auto_readahead: 2 2024-05-15T13:49:15.318 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.write_buffer_size: 33554432 2024-05-15T13:49:15.318 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.max_write_buffer_number: 2 2024-05-15T13:49:15.318 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.compression: NoCompression 2024-05-15T13:49:15.318 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.bottommost_compression: Disabled 2024-05-15T13:49:15.318 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.prefix_extractor: nullptr 2024-05-15T13:49:15.318 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-15T13:49:15.318 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.num_levels: 7 2024-05-15T13:49:15.319 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-15T13:49:15.319 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-15T13:49:15.319 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-15T13:49:15.319 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-15T13:49:15.319 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-15T13:49:15.319 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-15T13:49:15.319 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-15T13:49:15.319 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-15T13:49:15.319 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-15T13:49:15.319 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-15T13:49:15.319 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-15T13:49:15.319 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-15T13:49:15.319 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-15T13:49:15.319 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.compression_opts.level: 32767 2024-05-15T13:49:15.319 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.compression_opts.strategy: 0 2024-05-15T13:49:15.320 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-15T13:49:15.320 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-15T13:49:15.320 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-15T13:49:15.320 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-15T13:49:15.320 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.compression_opts.enabled: false 2024-05-15T13:49:15.320 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-15T13:49:15.320 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-15T13:49:15.320 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-15T13:49:15.320 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-15T13:49:15.320 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.target_file_size_base: 67108864 2024-05-15T13:49:15.320 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-15T13:49:15.320 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-15T13:49:15.320 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-15T13:49:15.321 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-15T13:49:15.321 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-15T13:49:15.321 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-15T13:49:15.321 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-15T13:49:15.321 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-15T13:49:15.321 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-15T13:49:15.321 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-15T13:49:15.321 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-15T13:49:15.321 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-15T13:49:15.321 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-15T13:49:15.321 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-15T13:49:15.321 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.arena_block_size: 1048576 2024-05-15T13:49:15.321 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-15T13:49:15.321 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-15T13:49:15.322 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.disable_auto_compactions: 0 2024-05-15T13:49:15.322 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-15T13:49:15.322 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-15T13:49:15.322 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-15T13:49:15.322 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-15T13:49:15.322 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-15T13:49:15.322 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-15T13:49:15.322 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-15T13:49:15.322 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-15T13:49:15.322 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-15T13:49:15.322 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-15T13:49:15.322 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-15T13:49:15.322 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.inplace_update_support: 0 2024-05-15T13:49:15.323 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-15T13:49:15.323 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-15T13:49:15.323 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-15T13:49:15.323 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-15T13:49:15.323 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.bloom_locality: 0 2024-05-15T13:49:15.323 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.max_successive_merges: 0 2024-05-15T13:49:15.323 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-15T13:49:15.323 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.paranoid_file_checks: 0 2024-05-15T13:49:15.323 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.force_consistency_checks: 1 2024-05-15T13:49:15.323 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.report_bg_io_stats: 0 2024-05-15T13:49:15.323 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.ttl: 2592000 2024-05-15T13:49:15.323 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-15T13:49:15.323 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-15T13:49:15.324 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-15T13:49:15.324 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.enable_blob_files: false 2024-05-15T13:49:15.324 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.min_blob_size: 0 2024-05-15T13:49:15.324 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.blob_file_size: 268435456 2024-05-15T13:49:15.324 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-15T13:49:15.324 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-15T13:49:15.324 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-15T13:49:15.324 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-15T13:49:15.324 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-15T13:49:15.324 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.blob_file_starting_level: 0 2024-05-15T13:49:15.324 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-15T13:49:15.324 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-15T13:49:15.324 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-15T13:49:15.324 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: bc68baf9-61c2-4e1f-8c5f-ec6b376b7bb0 2024-05-15T13:49:15.324 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715780955260148, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-15T13:49:15.325 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-15T13:49:15.325 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715780955261283, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100550, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98435, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14264, "raw_average_key_size": 49, "raw_value_size": 90689, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715780955, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "bc68baf9-61c2-4e1f-8c5f-ec6b376b7bb0", "db_session_id": "XAAUA4Z0VLPTSHKAX8TQ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-15T13:49:15.325 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715780955261369, "job": 1, "event": "recovery_finished"} 2024-05-15T13:49:15.325 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-15T13:49:15.325 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-15T13:49:15.325 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f1dad7a000 2024-05-15T13:49:15.325 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: DB pointer 0x55f1dad66000 2024-05-15T13:49:15.325 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-15T13:49:15.325 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-15T13:49:15.325 INFO:journalctl@ceph.mon.a.smithi033.stdout: ** DB Stats ** 2024-05-15T13:49:15.325 INFO:journalctl@ceph.mon.a.smithi033.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-15T13:49:15.325 INFO:journalctl@ceph.mon.a.smithi033.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-15T13:49:15.325 INFO:journalctl@ceph.mon.a.smithi033.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-15T13:49:15.326 INFO:journalctl@ceph.mon.a.smithi033.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-15T13:49:15.326 INFO:journalctl@ceph.mon.a.smithi033.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-15T13:49:15.326 INFO:journalctl@ceph.mon.a.smithi033.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-15T13:49:15.326 INFO:journalctl@ceph.mon.a.smithi033.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-15T13:49:15.326 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-05-15T13:49:15.326 INFO:journalctl@ceph.mon.a.smithi033.stdout: ** Compaction Stats [default] ** 2024-05-15T13:49:15.326 INFO:journalctl@ceph.mon.a.smithi033.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-15T13:49:15.326 INFO:journalctl@ceph.mon.a.smithi033.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-15T13:49:15.326 INFO:journalctl@ceph.mon.a.smithi033.stdout: L0 2/0 100.05 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 135.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-15T13:49:15.326 INFO:journalctl@ceph.mon.a.smithi033.stdout: Sum 2/0 100.05 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 135.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-15T13:49:15.326 INFO:journalctl@ceph.mon.a.smithi033.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 135.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-15T13:49:15.326 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-05-15T13:49:15.326 INFO:journalctl@ceph.mon.a.smithi033.stdout: ** Compaction Stats [default] ** 2024-05-15T13:49:15.327 INFO:journalctl@ceph.mon.a.smithi033.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-15T13:49:15.327 INFO:journalctl@ceph.mon.a.smithi033.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-15T13:49:15.327 INFO:journalctl@ceph.mon.a.smithi033.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 135.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-15T13:49:15.327 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-05-15T13:49:15.327 INFO:journalctl@ceph.mon.a.smithi033.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-15T13:49:15.327 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-05-15T13:49:15.327 INFO:journalctl@ceph.mon.a.smithi033.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-15T13:49:15.327 INFO:journalctl@ceph.mon.a.smithi033.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-15T13:49:15.327 INFO:journalctl@ceph.mon.a.smithi033.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-15T13:49:15.327 INFO:journalctl@ceph.mon.a.smithi033.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-15T13:49:15.327 INFO:journalctl@ceph.mon.a.smithi033.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-15T13:49:15.327 INFO:journalctl@ceph.mon.a.smithi033.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-15T13:49:15.327 INFO:journalctl@ceph.mon.a.smithi033.stdout: Cumulative compaction: 0.00 GB write, 29.98 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-15T13:49:15.327 INFO:journalctl@ceph.mon.a.smithi033.stdout: Interval compaction: 0.00 GB write, 29.98 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-15T13:49:15.327 INFO:journalctl@ceph.mon.a.smithi033.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-15T13:49:15.328 INFO:journalctl@ceph.mon.a.smithi033.stdout: Block cache BinnedLRUCache@0x55f1dac5d1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-15T13:49:15.328 INFO:journalctl@ceph.mon.a.smithi033.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-15T13:49:15.328 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-05-15T13:49:15.328 INFO:journalctl@ceph.mon.a.smithi033.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-15T13:49:15.328 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: starting mon.a rank 0 at public addrs [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] at bind addrs [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid bca78ca4-12c1-11ef-bc98-c7b262605968 2024-05-15T13:49:15.328 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: mon.a@-1(???) e1 preinit fsid bca78ca4-12c1-11ef-bc98-c7b262605968 2024-05-15T13:49:15.328 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: mon.a@-1(???).mds e1 new map 2024-05-15T13:49:15.328 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: mon.a@-1(???).mds e1 print_map 2024-05-15T13:49:15.328 INFO:journalctl@ceph.mon.a.smithi033.stdout: e1 2024-05-15T13:49:15.328 INFO:journalctl@ceph.mon.a.smithi033.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T13:49:15.329 INFO:journalctl@ceph.mon.a.smithi033.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-15T13:49:15.329 INFO:journalctl@ceph.mon.a.smithi033.stdout: legacy client fscid: -1 2024-05-15T13:49:15.329 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-05-15T13:49:15.329 INFO:journalctl@ceph.mon.a.smithi033.stdout: No filesystems configured 2024-05-15T13:49:15.329 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-15T13:49:15.329 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-15T13:49:15.329 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-15T13:49:15.329 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-15T13:49:15.329 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-15T13:49:15.329 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 bash[31631]: 40d78556a6be0d1051a52d5c5ad4f7b3dcb7b9fc62a70f200f75e93206213a0b 2024-05-15T13:49:15.329 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 systemd[1]: Started Ceph mon.a for bca78ca4-12c1-11ef-bc98-c7b262605968. 2024-05-15T13:49:15.330 INFO:teuthology.orchestra.run.smithi033.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-15T13:49:15.572 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-15T13:49:15.572 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: monmap epoch 1 2024-05-15T13:49:15.573 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: fsid bca78ca4-12c1-11ef-bc98-c7b262605968 2024-05-15T13:49:15.573 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: last_changed 2024-05-15T13:49:07.404471+0000 2024-05-15T13:49:15.573 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: created 2024-05-15T13:49:07.404471+0000 2024-05-15T13:49:15.573 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: min_mon_release 19 (squid) 2024-05-15T13:49:15.573 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: election_strategy: 1 2024-05-15T13:49:15.573 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-05-15T13:49:15.573 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: fsmap 2024-05-15T13:49:15.573 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: osdmap e1: 0 total, 0 up, 0 in 2024-05-15T13:49:15.573 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:15 smithi033 ceph-mon[31650]: mgrmap e1: no daemons active 2024-05-15T13:49:16.442 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.786+0000 7fa237eae700 1 Processor -- start 2024-05-15T13:49:16.442 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.787+0000 7fa237eae700 1 -- start start 2024-05-15T13:49:16.443 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.787+0000 7fa237eae700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa230109080 0x7fa2301094a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:16.443 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.787+0000 7fa237eae700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa2301099e0 con 0x7fa230109080 2024-05-15T13:49:16.443 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.787+0000 7fa235c4a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa230109080 0x7fa2301094a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:16.443 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.787+0000 7fa235c4a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa230109080 0x7fa2301094a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55848/0 (socket says 172.21.15.33:55848) 2024-05-15T13:49:16.443 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.787+0000 7fa235c4a700 1 -- 172.21.15.33:0/99059152 learned_addr learned my addr 172.21.15.33:0/99059152 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:16.443 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.788+0000 7fa235c4a700 1 -- 172.21.15.33:0/99059152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa230109b60 con 0x7fa230109080 2024-05-15T13:49:16.443 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.788+0000 7fa235c4a700 1 --2- 172.21.15.33:0/99059152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa230109080 0x7fa2301094a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa22c0098a0 tx=0x7fa22c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f89939fe484f32 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:16.443 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.788+0000 7fa234c48700 1 -- 172.21.15.33:0/99059152 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa22c004160 con 0x7fa230109080 2024-05-15T13:49:16.443 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.788+0000 7fa234c48700 1 -- 172.21.15.33:0/99059152 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa22c004300 con 0x7fa230109080 2024-05-15T13:49:16.443 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.788+0000 7fa234c48700 1 -- 172.21.15.33:0/99059152 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa22c011af0 con 0x7fa230109080 2024-05-15T13:49:16.443 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.789+0000 7fa237eae700 1 -- 172.21.15.33:0/99059152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa230109080 msgr2=0x7fa2301094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:16.443 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.789+0000 7fa237eae700 1 --2- 172.21.15.33:0/99059152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa230109080 0x7fa2301094a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa22c0098a0 tx=0x7fa22c009bb0 comp rx=0 tx=0).stop 2024-05-15T13:49:16.443 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.789+0000 7fa237eae700 1 -- 172.21.15.33:0/99059152 shutdown_connections 2024-05-15T13:49:16.443 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.789+0000 7fa237eae700 1 --2- 172.21.15.33:0/99059152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa230109080 0x7fa2301094a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:16.444 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.789+0000 7fa237eae700 1 -- 172.21.15.33:0/99059152 >> 172.21.15.33:0/99059152 conn(0x7fa230104720 msgr2=0x7fa230106b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:16.444 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.789+0000 7fa237eae700 1 -- 172.21.15.33:0/99059152 shutdown_connections 2024-05-15T13:49:16.444 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.789+0000 7fa237eae700 1 -- 172.21.15.33:0/99059152 wait complete. 2024-05-15T13:49:16.444 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.789+0000 7fa237eae700 1 Processor -- start 2024-05-15T13:49:16.444 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.789+0000 7fa237eae700 1 -- start start 2024-05-15T13:49:16.444 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.790+0000 7fa237eae700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa230109080 0x7fa23019cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:16.444 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.790+0000 7fa237eae700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa23010b080 con 0x7fa230109080 2024-05-15T13:49:16.444 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.790+0000 7fa235c4a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa230109080 0x7fa23019cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:16.444 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.790+0000 7fa235c4a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa230109080 0x7fa23019cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55864/0 (socket says 172.21.15.33:55864) 2024-05-15T13:49:16.444 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.790+0000 7fa235c4a700 1 -- 172.21.15.33:0/4143988110 learned_addr learned my addr 172.21.15.33:0/4143988110 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:16.444 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.790+0000 7fa235c4a700 1 -- 172.21.15.33:0/4143988110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa23019d4a0 con 0x7fa230109080 2024-05-15T13:49:16.444 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.790+0000 7fa235c4a700 1 --2- 172.21.15.33:0/4143988110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa230109080 0x7fa23019cf60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa22c003f40 tx=0x7fa22c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:16.444 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.790+0000 7fa226ffd700 1 -- 172.21.15.33:0/4143988110 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa22c0038b0 con 0x7fa230109080 2024-05-15T13:49:16.444 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.790+0000 7fa226ffd700 1 -- 172.21.15.33:0/4143988110 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa22c003a50 con 0x7fa230109080 2024-05-15T13:49:16.445 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.790+0000 7fa226ffd700 1 -- 172.21.15.33:0/4143988110 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa22c01a5c0 con 0x7fa230109080 2024-05-15T13:49:16.445 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.790+0000 7fa237eae700 1 -- 172.21.15.33:0/4143988110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa23019d730 con 0x7fa230109080 2024-05-15T13:49:16.445 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.790+0000 7fa237eae700 1 -- 172.21.15.33:0/4143988110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2301a0420 con 0x7fa230109080 2024-05-15T13:49:16.445 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.791+0000 7fa226ffd700 1 -- 172.21.15.33:0/4143988110 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa22c01ac50 con 0x7fa230109080 2024-05-15T13:49:16.445 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.791+0000 7fa226ffd700 1 -- 172.21.15.33:0/4143988110 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa22c019700 con 0x7fa230109080 2024-05-15T13:49:16.445 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.791+0000 7fa237eae700 1 -- 172.21.15.33:0/4143988110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa230064ba0 con 0x7fa230109080 2024-05-15T13:49:16.445 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.794+0000 7fa226ffd700 1 -- 172.21.15.33:0/4143988110 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fa22c019900 con 0x7fa230109080 2024-05-15T13:49:16.445 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.864+0000 7fa237eae700 1 -- 172.21.15.33:0/4143988110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fa2301a05f0 con 0x7fa230109080 2024-05-15T13:49:16.445 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.866+0000 7fa226ffd700 1 -- 172.21.15.33:0/4143988110 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fa22c019ae0 con 0x7fa230109080 2024-05-15T13:49:16.445 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.867+0000 7fa237eae700 1 -- 172.21.15.33:0/4143988110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa230109080 msgr2=0x7fa23019cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:16.445 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.867+0000 7fa237eae700 1 --2- 172.21.15.33:0/4143988110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa230109080 0x7fa23019cf60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa22c003f40 tx=0x7fa22c011890 comp rx=0 tx=0).stop 2024-05-15T13:49:16.445 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.867+0000 7fa237eae700 1 -- 172.21.15.33:0/4143988110 shutdown_connections 2024-05-15T13:49:16.445 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.867+0000 7fa237eae700 1 --2- 172.21.15.33:0/4143988110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa230109080 0x7fa23019cf60 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:16.445 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.867+0000 7fa237eae700 1 -- 172.21.15.33:0/4143988110 >> 172.21.15.33:0/4143988110 conn(0x7fa230104720 msgr2=0x7fa230105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:16.446 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.867+0000 7fa237eae700 1 -- 172.21.15.33:0/4143988110 shutdown_connections 2024-05-15T13:49:16.446 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:15.867+0000 7fa237eae700 1 -- 172.21.15.33:0/4143988110 wait complete. 2024-05-15T13:49:16.446 INFO:teuthology.orchestra.run.smithi033.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-15T13:49:16.468 INFO:teuthology.orchestra.run.smithi033.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-15T13:49:16.468 INFO:teuthology.orchestra.run.smithi033.stdout:Creating mgr... 2024-05-15T13:49:16.468 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-15T13:49:16.468 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-15T13:49:16.732 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mgr.x 2024-05-15T13:49:16.732 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to reset failed state of unit ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mgr.x.service: Unit ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mgr.x.service not loaded. 2024-05-15T13:49:16.893 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-bca78ca4-12c1-11ef-bc98-c7b262605968.target.wants/ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@.service. 2024-05-15T13:49:17.130 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:16 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/4143988110' entity='client.admin' 2024-05-15T13:49:17.131 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:17 smithi033 podman[31861]: 2024-05-15 13:49:17.031305689 +0000 UTC m=+0.017374420 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-15T13:49:17.398 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:17 smithi033 podman[31861]: 2024-05-15 13:49:17.328849986 +0000 UTC m=+0.314918690 container init cfcd834cab7f62715ead13e0c4ccc57ede5e0c0fdedf5a146a9e0a6e5413d350 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-15T13:49:17.398 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:17 smithi033 podman[31861]: 2024-05-15 13:49:17.331731649 +0000 UTC m=+0.317800368 container start cfcd834cab7f62715ead13e0c4ccc57ede5e0c0fdedf5a146a9e0a6e5413d350 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x, org.label-schema.build-date=20240514, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-15T13:49:17.412 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-15T13:49:17.412 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-05-15T13:49:17.420 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-15T13:49:17.420 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-05-15T13:49:17.420 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-05-15T13:49:17.420 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to enable service . firewalld.service is not available 2024-05-15T13:49:17.429 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-15T13:49:17.429 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-05-15T13:49:17.436 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-15T13:49:17.436 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-05-15T13:49:17.437 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-05-15T13:49:17.437 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-15T13:49:17.437 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr to start... 2024-05-15T13:49:17.437 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr... 2024-05-15T13:49:17.689 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:17 smithi033 bash[31861]: cfcd834cab7f62715ead13e0c4ccc57ede5e0c0fdedf5a146a9e0a6e5413d350 2024-05-15T13:49:17.689 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:17 smithi033 systemd[1]: Started Ceph mgr.x for bca78ca4-12c1-11ef-bc98-c7b262605968. 2024-05-15T13:49:17.689 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:17 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:17.491+0000 7fd03d8a9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-15T13:49:17.689 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:17 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:17.686+0000 7fd03d8a9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-15T13:49:17.942 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:17 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:17.794+0000 7fd03d8a9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-15T13:49:18.333 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:18 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/1685928326' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-15T13:49:18.333 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:18 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:18.128+0000 7fd03d8a9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-15T13:49:18.333 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:18 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:18.218+0000 7fd03d8a9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-15T13:49:18.573 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-15T13:49:18.576 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-05-15T13:49:18.576 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsid": "bca78ca4-12c1-11ef-bc98-c7b262605968", 2024-05-15T13:49:18.576 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "health": { 2024-05-15T13:49:18.576 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-15T13:49:18.577 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-15T13:49:18.577 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-15T13:49:18.577 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-15T13:49:18.577 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-15T13:49:18.577 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-15T13:49:18.577 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 0 2024-05-15T13:49:18.577 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-15T13:49:18.577 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-15T13:49:18.577 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "a" 2024-05-15T13:49:18.577 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-15T13:49:18.577 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-15T13:49:18.577 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-15T13:49:18.577 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-15T13:49:18.578 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-15T13:49:18.578 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-15T13:49:18.578 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-15T13:49:18.578 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-15T13:49:18.578 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-15T13:49:18.578 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-15T13:49:18.578 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-15T13:49:18.578 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-15T13:49:18.586 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-15T13:49:18.586 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-15T13:49:18.586 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-15T13:49:18.586 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-15T13:49:18.586 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-15T13:49:18.586 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-15T13:49:18.586 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-15T13:49:18.586 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-15T13:49:18.586 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-15T13:49:18.586 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-15T13:49:18.587 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-15T13:49:18.587 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-15T13:49:18.587 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-15T13:49:18.587 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-15T13:49:18.587 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-15T13:49:18.587 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-15T13:49:18.587 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-15T13:49:18.587 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-15T13:49:18.587 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-15T13:49:18.587 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-15T13:49:18.587 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-15T13:49:18.587 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-15T13:49:18.587 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-15T13:49:18.587 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-15T13:49:18.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-15T13:49:18.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "restful" 2024-05-15T13:49:18.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-15T13:49:18.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-15T13:49:18.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-15T13:49:18.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-15T13:49:18.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-15T13:49:18.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modified": "2024-05-15T13:49:10.328089+0000", 2024-05-15T13:49:18.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-15T13:49:18.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-15T13:49:18.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-15T13:49:18.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-05-15T13:49:18.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.902+0000 7f71a51c1700 1 Processor -- start 2024-05-15T13:49:18.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.902+0000 7f71a51c1700 1 -- start start 2024-05-15T13:49:18.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.902+0000 7f71a51c1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71a0109ad0 0x7f71a010bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:18.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.902+0000 7f71a51c1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f71a0076e40 con 0x7f71a0109ad0 2024-05-15T13:49:18.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.902+0000 7f719ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71a0109ad0 0x7f71a010bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:18.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.902+0000 7f719ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71a0109ad0 0x7f71a010bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55900/0 (socket says 172.21.15.33:55900) 2024-05-15T13:49:18.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.902+0000 7f719ed9d700 1 -- 172.21.15.33:0/3135993428 learned_addr learned my addr 172.21.15.33:0/3135993428 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:18.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.903+0000 7f719ed9d700 1 -- 172.21.15.33:0/3135993428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a010c400 con 0x7f71a0109ad0 2024-05-15T13:49:18.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.903+0000 7f719ed9d700 1 --2- 172.21.15.33:0/3135993428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71a0109ad0 0x7f71a010bec0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f71900098a0 tx=0x7f7190009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896ac9b65e190ac1 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:18.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.903+0000 7f719dd9b700 1 -- 172.21.15.33:0/3135993428 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7190004160 con 0x7f71a0109ad0 2024-05-15T13:49:18.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.903+0000 7f719dd9b700 1 -- 172.21.15.33:0/3135993428 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7190004300 con 0x7f71a0109ad0 2024-05-15T13:49:18.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.903+0000 7f719dd9b700 1 -- 172.21.15.33:0/3135993428 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7190011af0 con 0x7f71a0109ad0 2024-05-15T13:49:18.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.904+0000 7f71a51c1700 1 -- 172.21.15.33:0/3135993428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71a0109ad0 msgr2=0x7f71a010bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:18.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.904+0000 7f71a51c1700 1 --2- 172.21.15.33:0/3135993428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71a0109ad0 0x7f71a010bec0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f71900098a0 tx=0x7f7190009bb0 comp rx=0 tx=0).stop 2024-05-15T13:49:18.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.904+0000 7f71a51c1700 1 -- 172.21.15.33:0/3135993428 shutdown_connections 2024-05-15T13:49:18.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.904+0000 7f71a51c1700 1 --2- 172.21.15.33:0/3135993428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71a0109ad0 0x7f71a010bec0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:18.590 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.904+0000 7f71a51c1700 1 -- 172.21.15.33:0/3135993428 >> 172.21.15.33:0/3135993428 conn(0x7f71a01034e0 msgr2=0x7f71a0105940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:18.590 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.904+0000 7f71a51c1700 1 -- 172.21.15.33:0/3135993428 shutdown_connections 2024-05-15T13:49:18.590 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.904+0000 7f71a51c1700 1 -- 172.21.15.33:0/3135993428 wait complete. 2024-05-15T13:49:18.590 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.905+0000 7f71a51c1700 1 Processor -- start 2024-05-15T13:49:18.590 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.905+0000 7f71a51c1700 1 -- start start 2024-05-15T13:49:18.590 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.905+0000 7f71a51c1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71a0109ad0 0x7f71a0109350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:18.590 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.905+0000 7f71a51c1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f71a010d3d0 con 0x7f71a0109ad0 2024-05-15T13:49:18.590 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.905+0000 7f719ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71a0109ad0 0x7f71a0109350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:18.590 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.905+0000 7f719ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71a0109ad0 0x7f71a0109350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55904/0 (socket says 172.21.15.33:55904) 2024-05-15T13:49:18.590 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.905+0000 7f719ed9d700 1 -- 172.21.15.33:0/1685928326 learned_addr learned my addr 172.21.15.33:0/1685928326 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:18.590 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.906+0000 7f719ed9d700 1 -- 172.21.15.33:0/1685928326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a0109890 con 0x7f71a0109ad0 2024-05-15T13:49:18.590 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.906+0000 7f719ed9d700 1 --2- 172.21.15.33:0/1685928326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71a0109ad0 0x7f71a0109350 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7190009fb0 tx=0x7f7190011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:18.590 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.906+0000 7f7187fff700 1 -- 172.21.15.33:0/1685928326 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71900036a0 con 0x7f71a0109ad0 2024-05-15T13:49:18.591 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.906+0000 7f71a51c1700 1 -- 172.21.15.33:0/1685928326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71a0107a80 con 0x7f71a0109ad0 2024-05-15T13:49:18.591 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.906+0000 7f7187fff700 1 -- 172.21.15.33:0/1685928326 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f719001a430 con 0x7f71a0109ad0 2024-05-15T13:49:18.591 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.906+0000 7f7187fff700 1 -- 172.21.15.33:0/1685928326 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7190019550 con 0x7f71a0109ad0 2024-05-15T13:49:18.591 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.906+0000 7f71a51c1700 1 -- 172.21.15.33:0/1685928326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71a0107fc0 con 0x7f71a0109ad0 2024-05-15T13:49:18.591 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.906+0000 7f7187fff700 1 -- 172.21.15.33:0/1685928326 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7190019be0 con 0x7f71a0109ad0 2024-05-15T13:49:18.591 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.907+0000 7f7187fff700 1 -- 172.21.15.33:0/1685928326 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7190024730 con 0x7f71a0109ad0 2024-05-15T13:49:18.591 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.907+0000 7f71a51c1700 1 -- 172.21.15.33:0/1685928326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71a0064ba0 con 0x7f71a0109ad0 2024-05-15T13:49:18.591 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.909+0000 7f7187fff700 1 -- 172.21.15.33:0/1685928326 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f7190043ec0 con 0x7f71a0109ad0 2024-05-15T13:49:18.591 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.985+0000 7f71a51c1700 1 -- 172.21.15.33:0/1685928326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f71a01084f0 con 0x7f71a0109ad0 2024-05-15T13:49:18.591 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.985+0000 7f7187fff700 1 -- 172.21.15.33:0/1685928326 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f7190032030 con 0x7f71a0109ad0 2024-05-15T13:49:18.591 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.987+0000 7f71a51c1700 1 -- 172.21.15.33:0/1685928326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71a0109ad0 msgr2=0x7f71a0109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:18.591 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.987+0000 7f71a51c1700 1 --2- 172.21.15.33:0/1685928326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71a0109ad0 0x7f71a0109350 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7190009fb0 tx=0x7f7190011940 comp rx=0 tx=0).stop 2024-05-15T13:49:18.591 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.987+0000 7f71a51c1700 1 -- 172.21.15.33:0/1685928326 shutdown_connections 2024-05-15T13:49:18.591 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.987+0000 7f71a51c1700 1 --2- 172.21.15.33:0/1685928326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71a0109ad0 0x7f71a0109350 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:18.591 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.987+0000 7f71a51c1700 1 -- 172.21.15.33:0/1685928326 >> 172.21.15.33:0/1685928326 conn(0x7f71a01034e0 msgr2=0x7f71a0104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:18.592 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.987+0000 7f71a51c1700 1 -- 172.21.15.33:0/1685928326 shutdown_connections 2024-05-15T13:49:18.592 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:17.987+0000 7f71a51c1700 1 -- 172.21.15.33:0/1685928326 wait complete. 2024-05-15T13:49:18.592 INFO:teuthology.orchestra.run.smithi033.stdout:mgr not available, waiting (1/15)... 2024-05-15T13:49:19.848 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:19 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:19.491+0000 7fd03d8a9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-15T13:49:19.848 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:19 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:19.668+0000 7fd03d8a9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-15T13:49:19.848 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:19 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:19.845+0000 7fd03d8a9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-15T13:49:20.442 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:20 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:20.147+0000 7fd03d8a9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-15T13:49:20.442 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:20 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:20.244+0000 7fd03d8a9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-15T13:49:20.730 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:20 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:20.454+0000 7fd03d8a9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-15T13:49:20.730 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:20 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:20.658+0000 7fd03d8a9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-15T13:49:21.306 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:21 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/4242253877' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-15T13:49:21.660 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:21 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:21.303+0000 7fd03d8a9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-15T13:49:21.660 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:21 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:21.536+0000 7fd03d8a9200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-15T13:49:21.672 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-15T13:49:21.672 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-05-15T13:49:21.672 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsid": "bca78ca4-12c1-11ef-bc98-c7b262605968", 2024-05-15T13:49:21.672 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "health": { 2024-05-15T13:49:21.672 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-15T13:49:21.672 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-15T13:49:21.672 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-15T13:49:21.672 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-15T13:49:21.673 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-15T13:49:21.673 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-15T13:49:21.673 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 0 2024-05-15T13:49:21.673 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-15T13:49:21.673 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-15T13:49:21.673 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "a" 2024-05-15T13:49:21.673 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-15T13:49:21.673 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-05-15T13:49:21.673 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-15T13:49:21.673 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-15T13:49:21.673 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-15T13:49:21.673 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-15T13:49:21.673 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-15T13:49:21.673 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-15T13:49:21.674 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-15T13:49:21.674 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-15T13:49:21.674 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-15T13:49:21.674 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-15T13:49:21.674 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-15T13:49:21.674 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-15T13:49:21.674 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-15T13:49:21.674 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-15T13:49:21.674 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-15T13:49:21.674 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-15T13:49:21.674 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-15T13:49:21.674 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-15T13:49:21.674 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-15T13:49:21.674 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-15T13:49:21.674 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-15T13:49:21.675 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-15T13:49:21.675 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-15T13:49:21.675 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-15T13:49:21.675 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-15T13:49:21.675 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-15T13:49:21.675 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-15T13:49:21.675 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-15T13:49:21.675 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-15T13:49:21.675 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-15T13:49:21.675 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-15T13:49:21.675 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-15T13:49:21.675 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-15T13:49:21.675 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-15T13:49:21.675 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-15T13:49:21.676 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "restful" 2024-05-15T13:49:21.676 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-15T13:49:21.676 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-15T13:49:21.676 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-15T13:49:21.676 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-15T13:49:21.676 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-15T13:49:21.676 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modified": "2024-05-15T13:49:10.328089+0000", 2024-05-15T13:49:21.676 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-15T13:49:21.676 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-15T13:49:21.676 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-15T13:49:21.676 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-05-15T13:49:21.676 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.025+0000 7fca066c2700 1 Processor -- start 2024-05-15T13:49:21.676 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.025+0000 7fca066c2700 1 -- start start 2024-05-15T13:49:21.676 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.025+0000 7fca066c2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca001078e0 0x7fca00107d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:21.676 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.025+0000 7fca066c2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fca00076e20 con 0x7fca001078e0 2024-05-15T13:49:21.677 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.025+0000 7fc9fffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca001078e0 0x7fca00107d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:21.677 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.025+0000 7fc9fffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca001078e0 0x7fca00107d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55912/0 (socket says 172.21.15.33:55912) 2024-05-15T13:49:21.677 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.025+0000 7fc9fffff700 1 -- 172.21.15.33:0/2773769281 learned_addr learned my addr 172.21.15.33:0/2773769281 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:21.677 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.026+0000 7fc9fffff700 1 -- 172.21.15.33:0/2773769281 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca00108240 con 0x7fca001078e0 2024-05-15T13:49:21.677 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.026+0000 7fc9fffff700 1 --2- 172.21.15.33:0/2773769281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca001078e0 0x7fca00107d00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc9e80098a0 tx=0x7fc9e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90415f7026be45e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:21.677 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.026+0000 7fc9feffd700 1 -- 172.21.15.33:0/2773769281 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9e8004160 con 0x7fca001078e0 2024-05-15T13:49:21.677 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.026+0000 7fc9feffd700 1 -- 172.21.15.33:0/2773769281 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc9e8004300 con 0x7fca001078e0 2024-05-15T13:49:21.677 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.026+0000 7fc9feffd700 1 -- 172.21.15.33:0/2773769281 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9e8011af0 con 0x7fca001078e0 2024-05-15T13:49:21.677 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.027+0000 7fca066c2700 1 -- 172.21.15.33:0/2773769281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca001078e0 msgr2=0x7fca00107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:21.677 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.027+0000 7fca066c2700 1 --2- 172.21.15.33:0/2773769281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca001078e0 0x7fca00107d00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc9e80098a0 tx=0x7fc9e8009bb0 comp rx=0 tx=0).stop 2024-05-15T13:49:21.677 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.027+0000 7fca066c2700 1 -- 172.21.15.33:0/2773769281 shutdown_connections 2024-05-15T13:49:21.677 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.027+0000 7fca066c2700 1 --2- 172.21.15.33:0/2773769281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca001078e0 0x7fca00107d00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:21.677 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.027+0000 7fca066c2700 1 -- 172.21.15.33:0/2773769281 >> 172.21.15.33:0/2773769281 conn(0x7fca001034e0 msgr2=0x7fca00105940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:21.677 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.027+0000 7fca066c2700 1 -- 172.21.15.33:0/2773769281 shutdown_connections 2024-05-15T13:49:21.678 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.027+0000 7fca066c2700 1 -- 172.21.15.33:0/2773769281 wait complete. 2024-05-15T13:49:21.678 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.027+0000 7fca066c2700 1 Processor -- start 2024-05-15T13:49:21.678 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.028+0000 7fca066c2700 1 -- start start 2024-05-15T13:49:21.678 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.028+0000 7fca066c2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca001078e0 0x7fca0019f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:21.678 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.028+0000 7fca066c2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fca00109270 con 0x7fca001078e0 2024-05-15T13:49:21.678 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.028+0000 7fc9fffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca001078e0 0x7fca0019f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:21.678 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.028+0000 7fc9fffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca001078e0 0x7fca0019f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55920/0 (socket says 172.21.15.33:55920) 2024-05-15T13:49:21.678 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.028+0000 7fc9fffff700 1 -- 172.21.15.33:0/4242253877 learned_addr learned my addr 172.21.15.33:0/4242253877 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:21.678 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.028+0000 7fc9fffff700 1 -- 172.21.15.33:0/4242253877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca0019f760 con 0x7fca001078e0 2024-05-15T13:49:21.678 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.028+0000 7fc9fffff700 1 --2- 172.21.15.33:0/4242253877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca001078e0 0x7fca0019f220 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8011ac0 tx=0x7fc9e8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:21.678 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.029+0000 7fc9fd7fa700 1 -- 172.21.15.33:0/4242253877 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9e8003970 con 0x7fca001078e0 2024-05-15T13:49:21.678 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.029+0000 7fca066c2700 1 -- 172.21.15.33:0/4242253877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca0019f9f0 con 0x7fca001078e0 2024-05-15T13:49:21.678 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.029+0000 7fc9fd7fa700 1 -- 172.21.15.33:0/4242253877 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc9e8003b10 con 0x7fca001078e0 2024-05-15T13:49:21.678 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.029+0000 7fc9fd7fa700 1 -- 172.21.15.33:0/4242253877 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9e8024610 con 0x7fca001078e0 2024-05-15T13:49:21.679 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.029+0000 7fca066c2700 1 -- 172.21.15.33:0/4242253877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca001a26e0 con 0x7fca001078e0 2024-05-15T13:49:21.679 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.029+0000 7fc9fd7fa700 1 -- 172.21.15.33:0/4242253877 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc9e8024bd0 con 0x7fca001078e0 2024-05-15T13:49:21.679 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.029+0000 7fc9fd7fa700 1 -- 172.21.15.33:0/4242253877 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc9e8019700 con 0x7fca001078e0 2024-05-15T13:49:21.679 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.030+0000 7fca066c2700 1 -- 172.21.15.33:0/4242253877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca00064ba0 con 0x7fca001078e0 2024-05-15T13:49:21.679 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.034+0000 7fc9fd7fa700 1 -- 172.21.15.33:0/4242253877 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fc9e8043ec0 con 0x7fca001078e0 2024-05-15T13:49:21.679 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.111+0000 7fca066c2700 1 -- 172.21.15.33:0/4242253877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fca000008d0 con 0x7fca001078e0 2024-05-15T13:49:21.679 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.111+0000 7fc9fd7fa700 1 -- 172.21.15.33:0/4242253877 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fc9e8032030 con 0x7fca001078e0 2024-05-15T13:49:21.679 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.113+0000 7fca066c2700 1 -- 172.21.15.33:0/4242253877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca001078e0 msgr2=0x7fca0019f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:21.679 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.113+0000 7fca066c2700 1 --2- 172.21.15.33:0/4242253877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca001078e0 0x7fca0019f220 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8011ac0 tx=0x7fc9e8011ba0 comp rx=0 tx=0).stop 2024-05-15T13:49:21.679 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.113+0000 7fca066c2700 1 -- 172.21.15.33:0/4242253877 shutdown_connections 2024-05-15T13:49:21.679 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.113+0000 7fca066c2700 1 --2- 172.21.15.33:0/4242253877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca001078e0 0x7fca0019f220 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:21.679 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.113+0000 7fca066c2700 1 -- 172.21.15.33:0/4242253877 >> 172.21.15.33:0/4242253877 conn(0x7fca001034e0 msgr2=0x7fca00195e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:21.679 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.113+0000 7fca066c2700 1 -- 172.21.15.33:0/4242253877 shutdown_connections 2024-05-15T13:49:21.679 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:21.113+0000 7fca066c2700 1 -- 172.21.15.33:0/4242253877 wait complete. 2024-05-15T13:49:21.680 INFO:teuthology.orchestra.run.smithi033.stdout:mgr not available, waiting (2/15)... 2024-05-15T13:49:22.442 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:21 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:21.982+0000 7fd03d8a9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-15T13:49:22.442 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:22 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:22.082+0000 7fd03d8a9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-15T13:49:22.843 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:22 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:22.467+0000 7fd03d8a9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-15T13:49:22.843 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:22 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:22.841+0000 7fd03d8a9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-15T13:49:23.191 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:22 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:22.938+0000 7fd03d8a9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-15T13:49:23.692 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:23 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:23.196+0000 7fd03d8a9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-15T13:49:23.692 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:23 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:23.336+0000 7fd03d8a9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-15T13:49:23.692 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:23 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:23.437+0000 7fd03d8a9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-15T13:49:24.096 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:23 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:23.787+0000 7fd03d8a9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-15T13:49:24.096 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:23 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:23.885+0000 7fd03d8a9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-15T13:49:24.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:24 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/1299418818' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-15T13:49:24.771 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-15T13:49:24.771 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-05-15T13:49:24.771 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsid": "bca78ca4-12c1-11ef-bc98-c7b262605968", 2024-05-15T13:49:24.771 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "health": { 2024-05-15T13:49:24.771 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-15T13:49:24.771 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-15T13:49:24.771 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-15T13:49:24.771 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-15T13:49:24.771 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-15T13:49:24.772 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-15T13:49:24.772 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 0 2024-05-15T13:49:24.772 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-15T13:49:24.772 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-15T13:49:24.772 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "a" 2024-05-15T13:49:24.772 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-15T13:49:24.772 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-05-15T13:49:24.772 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-15T13:49:24.772 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-15T13:49:24.772 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-15T13:49:24.772 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-15T13:49:24.772 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-15T13:49:24.772 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-15T13:49:24.772 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-15T13:49:24.772 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-15T13:49:24.773 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-15T13:49:24.773 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-15T13:49:24.773 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-15T13:49:24.773 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-15T13:49:24.773 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-15T13:49:24.773 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-15T13:49:24.773 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-15T13:49:24.773 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-15T13:49:24.773 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-15T13:49:24.773 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-15T13:49:24.773 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-15T13:49:24.773 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-15T13:49:24.773 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-15T13:49:24.773 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-15T13:49:24.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-15T13:49:24.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-15T13:49:24.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-15T13:49:24.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-15T13:49:24.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-15T13:49:24.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-15T13:49:24.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-15T13:49:24.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-15T13:49:24.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-15T13:49:24.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-15T13:49:24.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-15T13:49:24.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-15T13:49:24.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-15T13:49:24.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "restful" 2024-05-15T13:49:24.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-15T13:49:24.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-15T13:49:24.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-15T13:49:24.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-15T13:49:24.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-15T13:49:24.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modified": "2024-05-15T13:49:10.328089+0000", 2024-05-15T13:49:24.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-15T13:49:24.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-15T13:49:24.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-15T13:49:24.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-05-15T13:49:24.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.112+0000 7ff4b7227700 1 Processor -- start 2024-05-15T13:49:24.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.112+0000 7ff4b7227700 1 -- start start 2024-05-15T13:49:24.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.113+0000 7ff4b7227700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b0107f00 0x7ff4b0108340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:24.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.113+0000 7ff4b7227700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b0108880 con 0x7ff4b0107f00 2024-05-15T13:49:24.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.113+0000 7ff4b4fc3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b0107f00 0x7ff4b0108340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:24.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.113+0000 7ff4b4fc3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b0107f00 0x7ff4b0108340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46116/0 (socket says 172.21.15.33:46116) 2024-05-15T13:49:24.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.113+0000 7ff4b4fc3700 1 -- 172.21.15.33:0/225471538 learned_addr learned my addr 172.21.15.33:0/225471538 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:24.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.113+0000 7ff4b4fc3700 1 -- 172.21.15.33:0/225471538 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b01090e0 con 0x7ff4b0107f00 2024-05-15T13:49:24.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.113+0000 7ff4b4fc3700 1 --2- 172.21.15.33:0/225471538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b0107f00 0x7ff4b0108340 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400bb40 tx=0x7ff4a400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181c71708645b667 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:24.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.114+0000 7ff4af7fe700 1 -- 172.21.15.33:0/225471538 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4a400dbe0 con 0x7ff4b0107f00 2024-05-15T13:49:24.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.114+0000 7ff4af7fe700 1 -- 172.21.15.33:0/225471538 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff4a40044c0 con 0x7ff4b0107f00 2024-05-15T13:49:24.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.114+0000 7ff4af7fe700 1 -- 172.21.15.33:0/225471538 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4a4010470 con 0x7ff4b0107f00 2024-05-15T13:49:24.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.114+0000 7ff4b7227700 1 -- 172.21.15.33:0/225471538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b0107f00 msgr2=0x7ff4b0108340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:24.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.114+0000 7ff4b7227700 1 --2- 172.21.15.33:0/225471538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b0107f00 0x7ff4b0108340 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400bb40 tx=0x7ff4a400be50 comp rx=0 tx=0).stop 2024-05-15T13:49:24.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.114+0000 7ff4b7227700 1 -- 172.21.15.33:0/225471538 shutdown_connections 2024-05-15T13:49:24.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.114+0000 7ff4b7227700 1 --2- 172.21.15.33:0/225471538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b0107f00 0x7ff4b0108340 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:24.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.114+0000 7ff4b7227700 1 -- 172.21.15.33:0/225471538 >> 172.21.15.33:0/225471538 conn(0x7ff4b0103510 msgr2=0x7ff4b0105930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:24.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.114+0000 7ff4b7227700 1 -- 172.21.15.33:0/225471538 shutdown_connections 2024-05-15T13:49:24.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.114+0000 7ff4b7227700 1 -- 172.21.15.33:0/225471538 wait complete. 2024-05-15T13:49:24.777 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.115+0000 7ff4b7227700 1 Processor -- start 2024-05-15T13:49:24.777 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.115+0000 7ff4b7227700 1 -- start start 2024-05-15T13:49:24.777 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.115+0000 7ff4b7227700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b0107f00 0x7ff4b019adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:24.777 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.115+0000 7ff4b7227700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b007bfc0 con 0x7ff4b0107f00 2024-05-15T13:49:24.777 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.116+0000 7ff4b4fc3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b0107f00 0x7ff4b019adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:24.777 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.116+0000 7ff4b4fc3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b0107f00 0x7ff4b019adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46124/0 (socket says 172.21.15.33:46124) 2024-05-15T13:49:24.777 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.116+0000 7ff4b4fc3700 1 -- 172.21.15.33:0/1299418818 learned_addr learned my addr 172.21.15.33:0/1299418818 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:24.777 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.116+0000 7ff4b4fc3700 1 -- 172.21.15.33:0/1299418818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b019b2f0 con 0x7ff4b0107f00 2024-05-15T13:49:24.777 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.116+0000 7ff4b4fc3700 1 --2- 172.21.15.33:0/1299418818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b0107f00 0x7ff4b019adb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400d650 tx=0x7ff4a4003ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:24.777 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.116+0000 7ff4adffb700 1 -- 172.21.15.33:0/1299418818 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4a4027070 con 0x7ff4b0107f00 2024-05-15T13:49:24.777 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.116+0000 7ff4b7227700 1 -- 172.21.15.33:0/1299418818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4b019b580 con 0x7ff4b0107f00 2024-05-15T13:49:24.777 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.116+0000 7ff4adffb700 1 -- 172.21.15.33:0/1299418818 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff4a4010c00 con 0x7ff4b0107f00 2024-05-15T13:49:24.777 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.116+0000 7ff4adffb700 1 -- 172.21.15.33:0/1299418818 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4a4022ba0 con 0x7ff4b0107f00 2024-05-15T13:49:24.777 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.117+0000 7ff4b7227700 1 -- 172.21.15.33:0/1299418818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4b019e230 con 0x7ff4b0107f00 2024-05-15T13:49:24.778 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.117+0000 7ff4adffb700 1 -- 172.21.15.33:0/1299418818 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff4a402a070 con 0x7ff4b0107f00 2024-05-15T13:49:24.778 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.117+0000 7ff4adffb700 1 -- 172.21.15.33:0/1299418818 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff4a401f070 con 0x7ff4b0107f00 2024-05-15T13:49:24.778 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.118+0000 7ff4b7227700 1 -- 172.21.15.33:0/1299418818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4b0064ba0 con 0x7ff4b0107f00 2024-05-15T13:49:24.778 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.120+0000 7ff4adffb700 1 -- 172.21.15.33:0/1299418818 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7ff4a4031030 con 0x7ff4b0107f00 2024-05-15T13:49:24.778 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.196+0000 7ff4b7227700 1 -- 172.21.15.33:0/1299418818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff4b0194250 con 0x7ff4b0107f00 2024-05-15T13:49:24.778 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.196+0000 7ff4adffb700 1 -- 172.21.15.33:0/1299418818 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7ff4b0194250 con 0x7ff4b0107f00 2024-05-15T13:49:24.778 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.197+0000 7ff4b7227700 1 -- 172.21.15.33:0/1299418818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b0107f00 msgr2=0x7ff4b019adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:24.778 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.197+0000 7ff4b7227700 1 --2- 172.21.15.33:0/1299418818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b0107f00 0x7ff4b019adb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400d650 tx=0x7ff4a4003ff0 comp rx=0 tx=0).stop 2024-05-15T13:49:24.778 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.198+0000 7ff4b7227700 1 -- 172.21.15.33:0/1299418818 shutdown_connections 2024-05-15T13:49:24.778 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.198+0000 7ff4b7227700 1 --2- 172.21.15.33:0/1299418818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4b0107f00 0x7ff4b019adb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:24.778 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.198+0000 7ff4b7227700 1 -- 172.21.15.33:0/1299418818 >> 172.21.15.33:0/1299418818 conn(0x7ff4b0103510 msgr2=0x7ff4b0191330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:24.778 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.198+0000 7ff4b7227700 1 -- 172.21.15.33:0/1299418818 shutdown_connections 2024-05-15T13:49:24.778 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:24.198+0000 7ff4b7227700 1 -- 172.21.15.33:0/1299418818 wait complete. 2024-05-15T13:49:24.779 INFO:teuthology.orchestra.run.smithi033.stdout:mgr not available, waiting (3/15)... 2024-05-15T13:49:24.822 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:24 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:24.537+0000 7fd03d8a9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-15T13:49:24.822 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:24 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:24.772+0000 7fd03d8a9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-15T13:49:25.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:25 smithi033 ceph-mon[31650]: Activating manager daemon x 2024-05-15T13:49:25.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:25 smithi033 ceph-mon[31650]: mgrmap e2: x(active, starting, since 0.00338674s) 2024-05-15T13:49:25.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:25 smithi033 ceph-mon[31650]: from='mgr.14100 172.21.15.33:0/659458173' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-15T13:49:25.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:25 smithi033 ceph-mon[31650]: from='mgr.14100 172.21.15.33:0/659458173' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-15T13:49:25.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:25 smithi033 ceph-mon[31650]: from='mgr.14100 172.21.15.33:0/659458173' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-15T13:49:25.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:25 smithi033 ceph-mon[31650]: from='mgr.14100 172.21.15.33:0/659458173' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-15T13:49:25.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:25 smithi033 ceph-mon[31650]: from='mgr.14100 172.21.15.33:0/659458173' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-15T13:49:25.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:25 smithi033 ceph-mon[31650]: Manager daemon x is now available 2024-05-15T13:49:25.693 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:25 smithi033 ceph-mon[31650]: from='mgr.14100 172.21.15.33:0/659458173' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-15T13:49:25.693 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:25 smithi033 ceph-mon[31650]: from='mgr.14100 172.21.15.33:0/659458173' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-15T13:49:25.693 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:25 smithi033 ceph-mon[31650]: from='mgr.14100 172.21.15.33:0/659458173' entity='mgr.x' 2024-05-15T13:49:25.693 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:25 smithi033 ceph-mon[31650]: from='mgr.14100 172.21.15.33:0/659458173' entity='mgr.x' 2024-05-15T13:49:25.693 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:25 smithi033 ceph-mon[31650]: from='mgr.14100 172.21.15.33:0/659458173' entity='mgr.x' 2024-05-15T13:49:27.061 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:26 smithi033 ceph-mon[31650]: mgrmap e3: x(active, since 1.00719s) 2024-05-15T13:49:28.056 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:27 smithi033 ceph-mon[31650]: mgrmap e4: x(active, since 2s) 2024-05-15T13:49:28.056 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:27 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/3972549473' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-15T13:49:28.061 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-15T13:49:28.061 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-05-15T13:49:28.061 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsid": "bca78ca4-12c1-11ef-bc98-c7b262605968", 2024-05-15T13:49:28.061 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "health": { 2024-05-15T13:49:28.061 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-15T13:49:28.061 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-15T13:49:28.061 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-15T13:49:28.061 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-15T13:49:28.061 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-15T13:49:28.062 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-15T13:49:28.062 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 0 2024-05-15T13:49:28.062 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-15T13:49:28.062 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-15T13:49:28.062 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "a" 2024-05-15T13:49:28.062 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-15T13:49:28.062 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-05-15T13:49:28.062 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-15T13:49:28.062 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-15T13:49:28.062 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-15T13:49:28.062 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-15T13:49:28.062 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-15T13:49:28.062 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-15T13:49:28.062 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-15T13:49:28.063 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-15T13:49:28.063 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-15T13:49:28.072 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-15T13:49:28.072 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-15T13:49:28.072 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-15T13:49:28.072 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-15T13:49:28.072 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-15T13:49:28.072 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-15T13:49:28.072 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-15T13:49:28.072 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-15T13:49:28.072 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-15T13:49:28.072 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-15T13:49:28.072 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-15T13:49:28.072 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-15T13:49:28.072 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-15T13:49:28.072 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-15T13:49:28.073 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-15T13:49:28.073 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-15T13:49:28.073 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-15T13:49:28.073 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-15T13:49:28.073 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-15T13:49:28.073 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-15T13:49:28.073 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-15T13:49:28.073 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-15T13:49:28.073 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-15T13:49:28.073 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-15T13:49:28.073 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-15T13:49:28.073 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-15T13:49:28.073 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "restful" 2024-05-15T13:49:28.073 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-15T13:49:28.073 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-15T13:49:28.074 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-15T13:49:28.074 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-15T13:49:28.074 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-15T13:49:28.074 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modified": "2024-05-15T13:49:10.328089+0000", 2024-05-15T13:49:28.074 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-15T13:49:28.074 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-15T13:49:28.074 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-15T13:49:28.074 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-05-15T13:49:28.074 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.167+0000 7fb193daa700 1 Processor -- start 2024-05-15T13:49:28.074 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.167+0000 7fb193daa700 1 -- start start 2024-05-15T13:49:28.074 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.167+0000 7fb193daa700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb18c109080 0x7fb18c1094a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:28.074 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.167+0000 7fb193daa700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb18c1099e0 con 0x7fb18c109080 2024-05-15T13:49:28.074 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.167+0000 7fb191b46700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb18c109080 0x7fb18c1094a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:28.075 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.167+0000 7fb191b46700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb18c109080 0x7fb18c1094a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46194/0 (socket says 172.21.15.33:46194) 2024-05-15T13:49:28.075 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.167+0000 7fb191b46700 1 -- 172.21.15.33:0/543779823 learned_addr learned my addr 172.21.15.33:0/543779823 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:28.075 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.168+0000 7fb191b46700 1 -- 172.21.15.33:0/543779823 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb18c109b60 con 0x7fb18c109080 2024-05-15T13:49:28.075 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.168+0000 7fb191b46700 1 --2- 172.21.15.33:0/543779823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb18c109080 0x7fb18c1094a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb1880098a0 tx=0x7fb188009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b0b0fde7a838f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:28.075 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.168+0000 7fb190b44700 1 -- 172.21.15.33:0/543779823 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb188004160 con 0x7fb18c109080 2024-05-15T13:49:28.075 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.168+0000 7fb190b44700 1 -- 172.21.15.33:0/543779823 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb188004300 con 0x7fb18c109080 2024-05-15T13:49:28.075 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.168+0000 7fb190b44700 1 -- 172.21.15.33:0/543779823 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb188011af0 con 0x7fb18c109080 2024-05-15T13:49:28.075 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.169+0000 7fb193daa700 1 -- 172.21.15.33:0/543779823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb18c109080 msgr2=0x7fb18c1094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:28.075 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.169+0000 7fb193daa700 1 --2- 172.21.15.33:0/543779823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb18c109080 0x7fb18c1094a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb1880098a0 tx=0x7fb188009bb0 comp rx=0 tx=0).stop 2024-05-15T13:49:28.075 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.169+0000 7fb193daa700 1 -- 172.21.15.33:0/543779823 shutdown_connections 2024-05-15T13:49:28.075 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.169+0000 7fb193daa700 1 --2- 172.21.15.33:0/543779823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb18c109080 0x7fb18c1094a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:28.075 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.169+0000 7fb193daa700 1 -- 172.21.15.33:0/543779823 >> 172.21.15.33:0/543779823 conn(0x7fb18c104720 msgr2=0x7fb18c106b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:28.075 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.169+0000 7fb193daa700 1 -- 172.21.15.33:0/543779823 shutdown_connections 2024-05-15T13:49:28.075 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.169+0000 7fb193daa700 1 -- 172.21.15.33:0/543779823 wait complete. 2024-05-15T13:49:28.075 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.169+0000 7fb193daa700 1 Processor -- start 2024-05-15T13:49:28.076 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.169+0000 7fb193daa700 1 -- start start 2024-05-15T13:49:28.076 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.170+0000 7fb193daa700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb18c109080 0x7fb18c19cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:28.076 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.170+0000 7fb193daa700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb18c10b080 con 0x7fb18c109080 2024-05-15T13:49:28.076 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.170+0000 7fb191b46700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb18c109080 0x7fb18c19cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:28.076 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.170+0000 7fb191b46700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb18c109080 0x7fb18c19cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46200/0 (socket says 172.21.15.33:46200) 2024-05-15T13:49:28.076 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.170+0000 7fb191b46700 1 -- 172.21.15.33:0/3972549473 learned_addr learned my addr 172.21.15.33:0/3972549473 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:28.076 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.170+0000 7fb191b46700 1 -- 172.21.15.33:0/3972549473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb18c19d4a0 con 0x7fb18c109080 2024-05-15T13:49:28.076 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.170+0000 7fb191b46700 1 --2- 172.21.15.33:0/3972549473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb18c109080 0x7fb18c19cf60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb188003f40 tx=0x7fb188011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:28.076 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.170+0000 7fb182ffd700 1 -- 172.21.15.33:0/3972549473 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1880038b0 con 0x7fb18c109080 2024-05-15T13:49:28.076 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.170+0000 7fb182ffd700 1 -- 172.21.15.33:0/3972549473 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb188003a50 con 0x7fb18c109080 2024-05-15T13:49:28.076 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.170+0000 7fb193daa700 1 -- 172.21.15.33:0/3972549473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb18c19d730 con 0x7fb18c109080 2024-05-15T13:49:28.076 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.170+0000 7fb182ffd700 1 -- 172.21.15.33:0/3972549473 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb18801a5c0 con 0x7fb18c109080 2024-05-15T13:49:28.076 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.171+0000 7fb193daa700 1 -- 172.21.15.33:0/3972549473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb18c1a0420 con 0x7fb18c109080 2024-05-15T13:49:28.077 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.171+0000 7fb182ffd700 1 -- 172.21.15.33:0/3972549473 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fb188024430 con 0x7fb18c109080 2024-05-15T13:49:28.077 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.172+0000 7fb193daa700 1 -- 172.21.15.33:0/3972549473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb18c064ba0 con 0x7fb18c109080 2024-05-15T13:49:28.077 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.172+0000 7fb182ffd700 1 --2- 172.21.15.33:0/3972549473 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7fb17803e3d0 0x7fb178040890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:28.077 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.172+0000 7fb182ffd700 1 -- 172.21.15.33:0/3972549473 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb188053a00 con 0x7fb18c109080 2024-05-15T13:49:28.077 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.172+0000 7fb191345700 1 --2- 172.21.15.33:0/3972549473 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7fb17803e3d0 0x7fb178040890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:28.077 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.172+0000 7fb191345700 1 --2- 172.21.15.33:0/3972549473 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7fb17803e3d0 0x7fb178040890 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb17c006fd0 tx=0x7fb17c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:28.077 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.177+0000 7fb182ffd700 1 -- 172.21.15.33:0/3972549473 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb18800b3e0 con 0x7fb18c109080 2024-05-15T13:49:28.077 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.490+0000 7fb193daa700 1 -- 172.21.15.33:0/3972549473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb18c1a05f0 con 0x7fb18c109080 2024-05-15T13:49:28.077 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.490+0000 7fb182ffd700 1 -- 172.21.15.33:0/3972549473 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fb18c1a05f0 con 0x7fb18c109080 2024-05-15T13:49:28.077 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.492+0000 7fb193daa700 1 -- 172.21.15.33:0/3972549473 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7fb17803e3d0 msgr2=0x7fb178040890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:28.077 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.492+0000 7fb193daa700 1 --2- 172.21.15.33:0/3972549473 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7fb17803e3d0 0x7fb178040890 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb17c006fd0 tx=0x7fb17c006e40 comp rx=0 tx=0).stop 2024-05-15T13:49:28.077 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.492+0000 7fb193daa700 1 -- 172.21.15.33:0/3972549473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb18c109080 msgr2=0x7fb18c19cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:28.077 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.492+0000 7fb193daa700 1 --2- 172.21.15.33:0/3972549473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb18c109080 0x7fb18c19cf60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb188003f40 tx=0x7fb188011890 comp rx=0 tx=0).stop 2024-05-15T13:49:28.077 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.492+0000 7fb193daa700 1 -- 172.21.15.33:0/3972549473 shutdown_connections 2024-05-15T13:49:28.077 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.492+0000 7fb193daa700 1 --2- 172.21.15.33:0/3972549473 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7fb17803e3d0 0x7fb178040890 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:28.078 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.492+0000 7fb193daa700 1 --2- 172.21.15.33:0/3972549473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb18c109080 0x7fb18c19cf60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:28.078 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.492+0000 7fb193daa700 1 -- 172.21.15.33:0/3972549473 >> 172.21.15.33:0/3972549473 conn(0x7fb18c104720 msgr2=0x7fb18c105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:28.078 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.492+0000 7fb193daa700 1 -- 172.21.15.33:0/3972549473 shutdown_connections 2024-05-15T13:49:28.078 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:27.492+0000 7fb193daa700 1 -- 172.21.15.33:0/3972549473 wait complete. 2024-05-15T13:49:28.078 INFO:teuthology.orchestra.run.smithi033.stdout:mgr is available 2024-05-15T13:49:29.014 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:28 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/2716061377' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-15T13:49:29.333 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-15T13:49:29.334 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [global] 2024-05-15T13:49:29.334 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout fsid = bca78ca4-12c1-11ef-bc98-c7b262605968 2024-05-15T13:49:29.334 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-15T13:49:29.334 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-15T13:49:29.334 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-15T13:49:29.334 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-15T13:49:29.334 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-15T13:49:29.334 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-15T13:49:29.334 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-15T13:49:29.334 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-15T13:49:29.334 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-15T13:49:29.334 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [osd] 2024-05-15T13:49:29.334 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-05-15T13:49:29.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-15T13:49:29.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-15T13:49:29.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-15T13:49:29.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-15T13:49:29.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-15T13:49:29.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.489+0000 7fd0554f5700 1 Processor -- start 2024-05-15T13:49:29.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.489+0000 7fd0554f5700 1 -- start start 2024-05-15T13:49:29.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.490+0000 7fd0554f5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd050109b60 0x7fd05010bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:29.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.490+0000 7fd0554f5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd05007ce40 con 0x7fd050109b60 2024-05-15T13:49:29.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.490+0000 7fd04effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd050109b60 0x7fd05010bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:29.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.490+0000 7fd04effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd050109b60 0x7fd05010bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46216/0 (socket says 172.21.15.33:46216) 2024-05-15T13:49:29.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.490+0000 7fd04effd700 1 -- 172.21.15.33:0/2055506781 learned_addr learned my addr 172.21.15.33:0/2055506781 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:29.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.490+0000 7fd04effd700 1 -- 172.21.15.33:0/2055506781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05010c490 con 0x7fd050109b60 2024-05-15T13:49:29.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.490+0000 7fd04effd700 1 --2- 172.21.15.33:0/2055506781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd050109b60 0x7fd05010bf50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd0400098a0 tx=0x7fd040009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e76c4f9b5214f94 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:29.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.490+0000 7fd04dffb700 1 -- 172.21.15.33:0/2055506781 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd040004160 con 0x7fd050109b60 2024-05-15T13:49:29.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.491+0000 7fd04dffb700 1 -- 172.21.15.33:0/2055506781 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd040004300 con 0x7fd050109b60 2024-05-15T13:49:29.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.491+0000 7fd04dffb700 1 -- 172.21.15.33:0/2055506781 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd040011af0 con 0x7fd050109b60 2024-05-15T13:49:29.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.491+0000 7fd0554f5700 1 -- 172.21.15.33:0/2055506781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd050109b60 msgr2=0x7fd05010bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:29.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.491+0000 7fd0554f5700 1 --2- 172.21.15.33:0/2055506781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd050109b60 0x7fd05010bf50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd0400098a0 tx=0x7fd040009bb0 comp rx=0 tx=0).stop 2024-05-15T13:49:29.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.491+0000 7fd0554f5700 1 -- 172.21.15.33:0/2055506781 shutdown_connections 2024-05-15T13:49:29.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.491+0000 7fd0554f5700 1 --2- 172.21.15.33:0/2055506781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd050109b60 0x7fd05010bf50 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:29.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.491+0000 7fd0554f5700 1 -- 172.21.15.33:0/2055506781 >> 172.21.15.33:0/2055506781 conn(0x7fd050103570 msgr2=0x7fd050105990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:29.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.491+0000 7fd0554f5700 1 -- 172.21.15.33:0/2055506781 shutdown_connections 2024-05-15T13:49:29.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.491+0000 7fd0554f5700 1 -- 172.21.15.33:0/2055506781 wait complete. 2024-05-15T13:49:29.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.492+0000 7fd0554f5700 1 Processor -- start 2024-05-15T13:49:29.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.492+0000 7fd0554f5700 1 -- start start 2024-05-15T13:49:29.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.492+0000 7fd0554f5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd050109b60 0x7fd05019cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:29.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.492+0000 7fd0554f5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd05010d400 con 0x7fd050109b60 2024-05-15T13:49:29.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.492+0000 7fd04effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd050109b60 0x7fd05019cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:29.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.492+0000 7fd04effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd050109b60 0x7fd05019cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46224/0 (socket says 172.21.15.33:46224) 2024-05-15T13:49:29.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.492+0000 7fd04effd700 1 -- 172.21.15.33:0/2716061377 learned_addr learned my addr 172.21.15.33:0/2716061377 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:29.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.493+0000 7fd04effd700 1 -- 172.21.15.33:0/2716061377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05019d520 con 0x7fd050109b60 2024-05-15T13:49:29.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.493+0000 7fd04effd700 1 --2- 172.21.15.33:0/2716061377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd050109b60 0x7fd05019cfe0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd040011ac0 tx=0x7fd040011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:29.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.493+0000 7fd047fff700 1 -- 172.21.15.33:0/2716061377 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd040003970 con 0x7fd050109b60 2024-05-15T13:49:29.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.493+0000 7fd047fff700 1 -- 172.21.15.33:0/2716061377 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd040003b10 con 0x7fd050109b60 2024-05-15T13:49:29.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.493+0000 7fd0554f5700 1 -- 172.21.15.33:0/2716061377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd05019d7b0 con 0x7fd050109b60 2024-05-15T13:49:29.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.493+0000 7fd047fff700 1 -- 172.21.15.33:0/2716061377 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd040024610 con 0x7fd050109b60 2024-05-15T13:49:29.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.493+0000 7fd0554f5700 1 -- 172.21.15.33:0/2716061377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0501a04a0 con 0x7fd050109b60 2024-05-15T13:49:29.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.494+0000 7fd047fff700 1 -- 172.21.15.33:0/2716061377 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fd040024840 con 0x7fd050109b60 2024-05-15T13:49:29.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.494+0000 7fd0554f5700 1 -- 172.21.15.33:0/2716061377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd050064ba0 con 0x7fd050109b60 2024-05-15T13:49:29.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.495+0000 7fd047fff700 1 --2- 172.21.15.33:0/2716061377 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7fd03c03e470 0x7fd03c040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:29.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.495+0000 7fd047fff700 1 -- 172.21.15.33:0/2716061377 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd040053910 con 0x7fd050109b60 2024-05-15T13:49:29.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.495+0000 7fd04e7fc700 1 --2- 172.21.15.33:0/2716061377 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7fd03c03e470 0x7fd03c040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:29.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.495+0000 7fd04e7fc700 1 --2- 172.21.15.33:0/2716061377 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7fd03c03e470 0x7fd03c040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd038006fd0 tx=0x7fd038006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:29.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.500+0000 7fd047fff700 1 -- 172.21.15.33:0/2716061377 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd04000a0d0 con 0x7fd050109b60 2024-05-15T13:49:29.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.713+0000 7fd0554f5700 1 -- 172.21.15.33:0/2716061377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fd0501a0670 con 0x7fd050109b60 2024-05-15T13:49:29.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.714+0000 7fd047fff700 1 -- 172.21.15.33:0/2716061377 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7fd0501a0670 con 0x7fd050109b60 2024-05-15T13:49:29.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.716+0000 7fd0554f5700 1 -- 172.21.15.33:0/2716061377 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7fd03c03e470 msgr2=0x7fd03c040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:29.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.716+0000 7fd0554f5700 1 --2- 172.21.15.33:0/2716061377 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7fd03c03e470 0x7fd03c040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd038006fd0 tx=0x7fd038006e40 comp rx=0 tx=0).stop 2024-05-15T13:49:29.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.716+0000 7fd0554f5700 1 -- 172.21.15.33:0/2716061377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd050109b60 msgr2=0x7fd05019cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:29.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.716+0000 7fd0554f5700 1 --2- 172.21.15.33:0/2716061377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd050109b60 0x7fd05019cfe0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd040011ac0 tx=0x7fd040011ba0 comp rx=0 tx=0).stop 2024-05-15T13:49:29.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.716+0000 7fd0554f5700 1 -- 172.21.15.33:0/2716061377 shutdown_connections 2024-05-15T13:49:29.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.716+0000 7fd0554f5700 1 --2- 172.21.15.33:0/2716061377 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7fd03c03e470 0x7fd03c040930 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:29.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.716+0000 7fd0554f5700 1 --2- 172.21.15.33:0/2716061377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd050109b60 0x7fd05019cfe0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:29.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.716+0000 7fd0554f5700 1 -- 172.21.15.33:0/2716061377 >> 172.21.15.33:0/2716061377 conn(0x7fd050103570 msgr2=0x7fd0501040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:29.339 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.716+0000 7fd0554f5700 1 -- 172.21.15.33:0/2716061377 shutdown_connections 2024-05-15T13:49:29.339 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:28.716+0000 7fd0554f5700 1 -- 172.21.15.33:0/2716061377 wait complete. 2024-05-15T13:49:29.339 INFO:teuthology.orchestra.run.smithi033.stdout:Enabling cephadm module... 2024-05-15T13:49:30.437 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:30 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/3884962501' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-15T13:49:30.438 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:30 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: ignoring --setuser ceph since I am not root 2024-05-15T13:49:30.438 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:30 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: ignoring --setgroup ceph since I am not root 2024-05-15T13:49:30.438 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:30 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:30.293+0000 7fa00e53d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-15T13:49:30.691 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:30 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:30.486+0000 7fa00e53d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-15T13:49:30.692 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:30 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:30.593+0000 7fa00e53d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-15T13:49:30.705 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.779+0000 7f9c22d93700 1 Processor -- start 2024-05-15T13:49:30.705 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.779+0000 7f9c22d93700 1 -- start start 2024-05-15T13:49:30.705 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.779+0000 7f9c22d93700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c1c109080 0x7f9c1c1094a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:30.705 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.779+0000 7f9c22d93700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c1099e0 con 0x7f9c1c109080 2024-05-15T13:49:30.706 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.779+0000 7f9c20b2f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c1c109080 0x7f9c1c1094a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:30.706 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.779+0000 7f9c20b2f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c1c109080 0x7f9c1c1094a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46238/0 (socket says 172.21.15.33:46238) 2024-05-15T13:49:30.706 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.779+0000 7f9c20b2f700 1 -- 172.21.15.33:0/4133306305 learned_addr learned my addr 172.21.15.33:0/4133306305 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:30.706 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.780+0000 7f9c20b2f700 1 -- 172.21.15.33:0/4133306305 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c1c109b60 con 0x7f9c1c109080 2024-05-15T13:49:30.706 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.780+0000 7f9c20b2f700 1 --2- 172.21.15.33:0/4133306305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c1c109080 0x7f9c1c1094a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9c100098a0 tx=0x7f9c10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8657f59dd81eed5f server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:30.707 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.780+0000 7f9c1b7fe700 1 -- 172.21.15.33:0/4133306305 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c10004160 con 0x7f9c1c109080 2024-05-15T13:49:30.707 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.780+0000 7f9c1b7fe700 1 -- 172.21.15.33:0/4133306305 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9c10004300 con 0x7f9c1c109080 2024-05-15T13:49:30.707 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.780+0000 7f9c1b7fe700 1 -- 172.21.15.33:0/4133306305 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c10011af0 con 0x7f9c1c109080 2024-05-15T13:49:30.707 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.780+0000 7f9c22d93700 1 -- 172.21.15.33:0/4133306305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c1c109080 msgr2=0x7f9c1c1094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:30.707 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.781+0000 7f9c22d93700 1 --2- 172.21.15.33:0/4133306305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c1c109080 0x7f9c1c1094a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9c100098a0 tx=0x7f9c10009bb0 comp rx=0 tx=0).stop 2024-05-15T13:49:30.707 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.781+0000 7f9c22d93700 1 -- 172.21.15.33:0/4133306305 shutdown_connections 2024-05-15T13:49:30.707 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.781+0000 7f9c22d93700 1 --2- 172.21.15.33:0/4133306305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c1c109080 0x7f9c1c1094a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:30.707 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.781+0000 7f9c22d93700 1 -- 172.21.15.33:0/4133306305 >> 172.21.15.33:0/4133306305 conn(0x7f9c1c104720 msgr2=0x7f9c1c106b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:30.707 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.781+0000 7f9c22d93700 1 -- 172.21.15.33:0/4133306305 shutdown_connections 2024-05-15T13:49:30.707 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.781+0000 7f9c22d93700 1 -- 172.21.15.33:0/4133306305 wait complete. 2024-05-15T13:49:30.707 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.781+0000 7f9c22d93700 1 Processor -- start 2024-05-15T13:49:30.708 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.781+0000 7f9c22d93700 1 -- start start 2024-05-15T13:49:30.708 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.782+0000 7f9c22d93700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c1c109080 0x7f9c1c07eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:30.708 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.782+0000 7f9c22d93700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c10b080 con 0x7f9c1c109080 2024-05-15T13:49:30.708 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.782+0000 7f9c20b2f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c1c109080 0x7f9c1c07eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:30.708 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.782+0000 7f9c20b2f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c1c109080 0x7f9c1c07eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46252/0 (socket says 172.21.15.33:46252) 2024-05-15T13:49:30.708 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.782+0000 7f9c20b2f700 1 -- 172.21.15.33:0/3884962501 learned_addr learned my addr 172.21.15.33:0/3884962501 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:30.708 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.782+0000 7f9c20b2f700 1 -- 172.21.15.33:0/3884962501 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c1c07f0a0 con 0x7f9c1c109080 2024-05-15T13:49:30.708 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.782+0000 7f9c20b2f700 1 --2- 172.21.15.33:0/3884962501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c1c109080 0x7f9c1c07eb60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9c10003f40 tx=0x7f9c10011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:30.708 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.782+0000 7f9c19ffb700 1 -- 172.21.15.33:0/3884962501 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c100038b0 con 0x7f9c1c109080 2024-05-15T13:49:30.708 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.782+0000 7f9c19ffb700 1 -- 172.21.15.33:0/3884962501 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9c10003a50 con 0x7f9c1c109080 2024-05-15T13:49:30.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.782+0000 7f9c22d93700 1 -- 172.21.15.33:0/3884962501 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c1c07d290 con 0x7f9c1c109080 2024-05-15T13:49:30.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.782+0000 7f9c19ffb700 1 -- 172.21.15.33:0/3884962501 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c1001a5c0 con 0x7f9c1c109080 2024-05-15T13:49:30.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.783+0000 7f9c22d93700 1 -- 172.21.15.33:0/3884962501 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c1c07d7d0 con 0x7f9c1c109080 2024-05-15T13:49:30.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.783+0000 7f9c19ffb700 1 -- 172.21.15.33:0/3884962501 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f9c10024430 con 0x7f9c1c109080 2024-05-15T13:49:30.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.784+0000 7f9c22d93700 1 -- 172.21.15.33:0/3884962501 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c1c064ba0 con 0x7f9c1c109080 2024-05-15T13:49:30.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.784+0000 7f9c19ffb700 1 --2- 172.21.15.33:0/3884962501 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7f9c0c03e420 0x7f9c0c0408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:30.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.784+0000 7f9c19ffb700 1 -- 172.21.15.33:0/3884962501 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9c100539b0 con 0x7f9c1c109080 2024-05-15T13:49:30.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.784+0000 7f9c1bfff700 1 --2- 172.21.15.33:0/3884962501 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7f9c0c03e420 0x7f9c0c0408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:30.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.784+0000 7f9c1bfff700 1 --2- 172.21.15.33:0/3884962501 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7f9c0c03e420 0x7f9c0c0408e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9c08006fd0 tx=0x7f9c08006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:30.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:29.789+0000 7f9c19ffb700 1 -- 172.21.15.33:0/3884962501 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9c1000b3e0 con 0x7f9c1c109080 2024-05-15T13:49:30.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:30.066+0000 7f9c22d93700 1 -- 172.21.15.33:0/3884962501 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f9c1c07dfc0 con 0x7f9c1c109080 2024-05-15T13:49:30.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:30.117+0000 7f9c19ffb700 1 -- 172.21.15.33:0/3884962501 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f9c100533d0 con 0x7f9c1c109080 2024-05-15T13:49:30.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:30.118+0000 7f9c19ffb700 1 -- 172.21.15.33:0/3884962501 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f9c1002d3f0 con 0x7f9c1c109080 2024-05-15T13:49:30.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:30.120+0000 7f9c22d93700 1 -- 172.21.15.33:0/3884962501 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7f9c0c03e420 msgr2=0x7f9c0c0408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:30.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:30.120+0000 7f9c22d93700 1 --2- 172.21.15.33:0/3884962501 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7f9c0c03e420 0x7f9c0c0408e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9c08006fd0 tx=0x7f9c08006e40 comp rx=0 tx=0).stop 2024-05-15T13:49:30.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:30.120+0000 7f9c22d93700 1 -- 172.21.15.33:0/3884962501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c1c109080 msgr2=0x7f9c1c07eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:30.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:30.120+0000 7f9c22d93700 1 --2- 172.21.15.33:0/3884962501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c1c109080 0x7f9c1c07eb60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9c10003f40 tx=0x7f9c10011890 comp rx=0 tx=0).stop 2024-05-15T13:49:30.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:30.120+0000 7f9c22d93700 1 -- 172.21.15.33:0/3884962501 shutdown_connections 2024-05-15T13:49:30.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:30.120+0000 7f9c22d93700 1 --2- 172.21.15.33:0/3884962501 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7f9c0c03e420 0x7f9c0c0408e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:30.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:30.120+0000 7f9c22d93700 1 --2- 172.21.15.33:0/3884962501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c1c109080 0x7f9c1c07eb60 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:30.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:30.120+0000 7f9c22d93700 1 -- 172.21.15.33:0/3884962501 >> 172.21.15.33:0/3884962501 conn(0x7f9c1c104720 msgr2=0x7f9c1c105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:30.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:30.120+0000 7f9c22d93700 1 -- 172.21.15.33:0/3884962501 shutdown_connections 2024-05-15T13:49:30.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:30.120+0000 7f9c22d93700 1 -- 172.21.15.33:0/3884962501 wait complete. 2024-05-15T13:49:30.952 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:30 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:30.920+0000 7fa00e53d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-15T13:49:31.366 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:31 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/3884962501' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-15T13:49:31.366 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:31 smithi033 ceph-mon[31650]: mgrmap e5: x(active, since 5s) 2024-05-15T13:49:31.367 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:31 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:31.012+0000 7fa00e53d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-15T13:49:32.036 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-05-15T13:49:32.036 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-15T13:49:32.036 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-15T13:49:32.036 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-15T13:49:32.037 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-15T13:49:32.037 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-05-15T13:49:32.037 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.158+0000 7f49e19d8700 1 Processor -- start 2024-05-15T13:49:32.037 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.158+0000 7f49e19d8700 1 -- start start 2024-05-15T13:49:32.037 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.159+0000 7f49e19d8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49dc109ab0 0x7f49dc10bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:32.037 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.159+0000 7f49e19d8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc10c3e0 con 0x7f49dc109ab0 2024-05-15T13:49:32.037 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.159+0000 7f49daffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49dc109ab0 0x7f49dc10bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:32.037 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.159+0000 7f49daffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49dc109ab0 0x7f49dc10bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46288/0 (socket says 172.21.15.33:46288) 2024-05-15T13:49:32.037 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.159+0000 7f49daffd700 1 -- 172.21.15.33:0/967628870 learned_addr learned my addr 172.21.15.33:0/967628870 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:32.037 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.159+0000 7f49daffd700 1 -- 172.21.15.33:0/967628870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49dc10c560 con 0x7f49dc109ab0 2024-05-15T13:49:32.037 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.159+0000 7f49daffd700 1 --2- 172.21.15.33:0/967628870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49dc109ab0 0x7f49dc10bea0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f49cc0098a0 tx=0x7f49cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ee190023a2075f server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:32.037 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.159+0000 7f49d9ffb700 1 -- 172.21.15.33:0/967628870 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49cc004160 con 0x7f49dc109ab0 2024-05-15T13:49:32.037 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.159+0000 7f49d9ffb700 1 -- 172.21.15.33:0/967628870 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f49cc004300 con 0x7f49dc109ab0 2024-05-15T13:49:32.037 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.159+0000 7f49d9ffb700 1 -- 172.21.15.33:0/967628870 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49cc011af0 con 0x7f49dc109ab0 2024-05-15T13:49:32.037 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.160+0000 7f49e19d8700 1 -- 172.21.15.33:0/967628870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49dc109ab0 msgr2=0x7f49dc10bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:32.038 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.160+0000 7f49e19d8700 1 --2- 172.21.15.33:0/967628870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49dc109ab0 0x7f49dc10bea0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f49cc0098a0 tx=0x7f49cc009bb0 comp rx=0 tx=0).stop 2024-05-15T13:49:32.038 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.160+0000 7f49e19d8700 1 -- 172.21.15.33:0/967628870 shutdown_connections 2024-05-15T13:49:32.038 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.160+0000 7f49e19d8700 1 --2- 172.21.15.33:0/967628870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49dc109ab0 0x7f49dc10bea0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:32.038 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.160+0000 7f49e19d8700 1 -- 172.21.15.33:0/967628870 >> 172.21.15.33:0/967628870 conn(0x7f49dc103500 msgr2=0x7f49dc105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:32.038 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.160+0000 7f49e19d8700 1 -- 172.21.15.33:0/967628870 shutdown_connections 2024-05-15T13:49:32.038 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.160+0000 7f49e19d8700 1 -- 172.21.15.33:0/967628870 wait complete. 2024-05-15T13:49:32.038 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.161+0000 7f49e19d8700 1 Processor -- start 2024-05-15T13:49:32.038 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.161+0000 7f49e19d8700 1 -- start start 2024-05-15T13:49:32.038 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.161+0000 7f49e19d8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49dc109ab0 0x7f49dc109330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:32.038 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.161+0000 7f49e19d8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc10d4c0 con 0x7f49dc109ab0 2024-05-15T13:49:32.038 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.161+0000 7f49daffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49dc109ab0 0x7f49dc109330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:32.038 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.161+0000 7f49daffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49dc109ab0 0x7f49dc109330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46296/0 (socket says 172.21.15.33:46296) 2024-05-15T13:49:32.038 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.161+0000 7f49daffd700 1 -- 172.21.15.33:0/1128529937 learned_addr learned my addr 172.21.15.33:0/1128529937 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:32.038 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.162+0000 7f49daffd700 1 -- 172.21.15.33:0/1128529937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49dc109870 con 0x7f49dc109ab0 2024-05-15T13:49:32.039 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.162+0000 7f49daffd700 1 --2- 172.21.15.33:0/1128529937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49dc109ab0 0x7f49dc109330 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f49cc003f40 tx=0x7f49cc011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:32.039 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.162+0000 7f49e09d6700 1 -- 172.21.15.33:0/1128529937 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49cc0038b0 con 0x7f49dc109ab0 2024-05-15T13:49:32.039 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.162+0000 7f49e09d6700 1 -- 172.21.15.33:0/1128529937 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f49cc003a50 con 0x7f49dc109ab0 2024-05-15T13:49:32.039 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.162+0000 7f49e19d8700 1 -- 172.21.15.33:0/1128529937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49dc107a60 con 0x7f49dc109ab0 2024-05-15T13:49:32.039 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.162+0000 7f49e09d6700 1 -- 172.21.15.33:0/1128529937 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49cc01a5c0 con 0x7f49dc109ab0 2024-05-15T13:49:32.039 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.162+0000 7f49e19d8700 1 -- 172.21.15.33:0/1128529937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49dc107fa0 con 0x7f49dc109ab0 2024-05-15T13:49:32.039 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.163+0000 7f49e09d6700 1 -- 172.21.15.33:0/1128529937 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f49cc01a870 con 0x7f49dc109ab0 2024-05-15T13:49:32.039 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.163+0000 7f49e09d6700 1 --2- 172.21.15.33:0/1128529937 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7f49c403e470 0x7f49c4040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:32.039 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.163+0000 7f49e09d6700 1 -- 172.21.15.33:0/1128529937 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f49cc053a40 con 0x7f49dc109ab0 2024-05-15T13:49:32.039 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.163+0000 7f49da7fc700 1 -- 172.21.15.33:0/1128529937 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7f49c403e470 msgr2=0x7f49c4040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3303495364 2024-05-15T13:49:32.039 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.163+0000 7f49da7fc700 1 --2- 172.21.15.33:0/1128529937 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7f49c403e470 0x7f49c4040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-15T13:49:32.039 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.164+0000 7f49e19d8700 1 -- 172.21.15.33:0/1128529937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49dc064ba0 con 0x7f49dc109ab0 2024-05-15T13:49:32.039 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.169+0000 7f49e09d6700 1 -- 172.21.15.33:0/1128529937 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f49cc00b3e0 con 0x7f49dc109ab0 2024-05-15T13:49:32.039 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.364+0000 7f49da7fc700 1 -- 172.21.15.33:0/1128529937 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7f49c403e470 msgr2=0x7f49c4040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3303495364 2024-05-15T13:49:32.039 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.364+0000 7f49da7fc700 1 --2- 172.21.15.33:0/1128529937 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7f49c403e470 0x7f49c4040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-15T13:49:32.040 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.500+0000 7f49e19d8700 1 -- 172.21.15.33:0/1128529937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f49dc108540 con 0x7f49dc109ab0 2024-05-15T13:49:32.040 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.500+0000 7f49e09d6700 1 -- 172.21.15.33:0/1128529937 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f49cc00e260 con 0x7f49dc109ab0 2024-05-15T13:49:32.040 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.503+0000 7f49e19d8700 1 -- 172.21.15.33:0/1128529937 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7f49c403e470 msgr2=0x7f49c4040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-15T13:49:32.040 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.503+0000 7f49e19d8700 1 --2- 172.21.15.33:0/1128529937 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7f49c403e470 0x7f49c4040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:32.040 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.503+0000 7f49e19d8700 1 -- 172.21.15.33:0/1128529937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49dc109ab0 msgr2=0x7f49dc109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:32.040 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.503+0000 7f49e19d8700 1 --2- 172.21.15.33:0/1128529937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49dc109ab0 0x7f49dc109330 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f49cc003f40 tx=0x7f49cc011890 comp rx=0 tx=0).stop 2024-05-15T13:49:32.040 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.503+0000 7f49e19d8700 1 -- 172.21.15.33:0/1128529937 shutdown_connections 2024-05-15T13:49:32.040 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.503+0000 7f49e19d8700 1 --2- 172.21.15.33:0/1128529937 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7f49c403e470 0x7f49c4040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:32.040 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.503+0000 7f49e19d8700 1 --2- 172.21.15.33:0/1128529937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49dc109ab0 0x7f49dc109330 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:32.040 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.503+0000 7f49e19d8700 1 -- 172.21.15.33:0/1128529937 >> 172.21.15.33:0/1128529937 conn(0x7f49dc103500 msgr2=0x7f49dc104010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:32.040 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.503+0000 7f49e19d8700 1 -- 172.21.15.33:0/1128529937 shutdown_connections 2024-05-15T13:49:32.040 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:31.503+0000 7f49e19d8700 1 -- 172.21.15.33:0/1128529937 wait complete. 2024-05-15T13:49:32.040 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for the mgr to restart... 2024-05-15T13:49:32.040 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr epoch 5... 2024-05-15T13:49:32.407 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:32 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/1128529937' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-15T13:49:32.407 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:32 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:32.307+0000 7fa00e53d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-15T13:49:32.681 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:32 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:32.495+0000 7fa00e53d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-15T13:49:32.681 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:32 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:32.678+0000 7fa00e53d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-15T13:49:33.302 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:32 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:32.989+0000 7fa00e53d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-15T13:49:33.302 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:33 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:33.086+0000 7fa00e53d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-15T13:49:33.302 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:33 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:33.300+0000 7fa00e53d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-15T13:49:33.692 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:33 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:33.502+0000 7fa00e53d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-15T13:49:34.442 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:34 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:34.137+0000 7fa00e53d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-15T13:49:34.442 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:34 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:34.368+0000 7fa00e53d200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-15T13:49:35.191 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:34 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:34.811+0000 7fa00e53d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-15T13:49:35.192 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:34 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:34.911+0000 7fa00e53d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-15T13:49:35.676 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:35 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:35.299+0000 7fa00e53d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-15T13:49:35.677 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:35 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:35.674+0000 7fa00e53d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-15T13:49:35.942 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:35 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:35.771+0000 7fa00e53d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-15T13:49:36.442 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:36 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:36.026+0000 7fa00e53d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-15T13:49:36.442 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:36 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:36.165+0000 7fa00e53d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-15T13:49:36.442 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:36 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:36.266+0000 7fa00e53d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-15T13:49:36.941 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:36 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:36.606+0000 7fa00e53d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-15T13:49:36.942 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:36 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:36.703+0000 7fa00e53d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-15T13:49:37.619 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:37 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:37.335+0000 7fa00e53d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-15T13:49:37.619 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:37 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:37.562+0000 7fa00e53d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-15T13:49:37.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:37 smithi033 ceph-mon[31650]: Active manager daemon x restarted 2024-05-15T13:49:37.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:37 smithi033 ceph-mon[31650]: Activating manager daemon x 2024-05-15T13:49:37.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:37 smithi033 ceph-mon[31650]: osdmap e2: 0 total, 0 up, 0 in 2024-05-15T13:49:37.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:37 smithi033 ceph-mon[31650]: mgrmap e6: x(active, starting, since 0.00403305s) 2024-05-15T13:49:37.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:37 smithi033 ceph-mon[31650]: from='mgr.14120 172.21.15.33:0/2424021838' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-15T13:49:37.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:37 smithi033 ceph-mon[31650]: from='mgr.14120 172.21.15.33:0/2424021838' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-15T13:49:37.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:37 smithi033 ceph-mon[31650]: from='mgr.14120 172.21.15.33:0/2424021838' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-15T13:49:37.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:37 smithi033 ceph-mon[31650]: from='mgr.14120 172.21.15.33:0/2424021838' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-15T13:49:37.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:37 smithi033 ceph-mon[31650]: from='mgr.14120 172.21.15.33:0/2424021838' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-15T13:49:37.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:37 smithi033 ceph-mon[31650]: Manager daemon x is now available 2024-05-15T13:49:37.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:37 smithi033 ceph-mon[31650]: from='mgr.14120 172.21.15.33:0/2424021838' entity='mgr.x' 2024-05-15T13:49:37.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:37 smithi033 ceph-mon[31650]: from='mgr.14120 172.21.15.33:0/2424021838' entity='mgr.x' 2024-05-15T13:49:37.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:37 smithi033 ceph-mon[31650]: from='mgr.14120 172.21.15.33:0/2424021838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:49:37.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:37 smithi033 ceph-mon[31650]: from='mgr.14120 172.21.15.33:0/2424021838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:49:38.851 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:38 smithi033 ceph-mon[31650]: Found migration_current of "None". Setting to last migration. 2024-05-15T13:49:38.851 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:38 smithi033 ceph-mon[31650]: from='mgr.14120 172.21.15.33:0/2424021838' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-15T13:49:38.851 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:38 smithi033 ceph-mon[31650]: from='mgr.14120 172.21.15.33:0/2424021838' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-15T13:49:38.851 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:38 smithi033 ceph-mon[31650]: from='mgr.14120 172.21.15.33:0/2424021838' entity='mgr.x' 2024-05-15T13:49:38.851 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:38 smithi033 ceph-mon[31650]: from='mgr.14120 172.21.15.33:0/2424021838' entity='mgr.x' 2024-05-15T13:49:38.851 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:38 smithi033 ceph-mon[31650]: mgrmap e7: x(active, since 1.00616s) 2024-05-15T13:49:39.104 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-05-15T13:49:39.111 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-15T13:49:39.112 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-15T13:49:39.112 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-05-15T13:49:39.112 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.417+0000 7fe371b0f700 1 Processor -- start 2024-05-15T13:49:39.112 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.417+0000 7fe371b0f700 1 -- start start 2024-05-15T13:49:39.112 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.418+0000 7fe371b0f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe36c109a90 0x7fe36c10be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:39.112 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.418+0000 7fe371b0f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe36c076e40 con 0x7fe36c109a90 2024-05-15T13:49:39.112 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.418+0000 7fe36b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe36c109a90 0x7fe36c10be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:39.112 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.418+0000 7fe36b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe36c109a90 0x7fe36c10be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46308/0 (socket says 172.21.15.33:46308) 2024-05-15T13:49:39.112 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.418+0000 7fe36b7fe700 1 -- 172.21.15.33:0/1805980370 learned_addr learned my addr 172.21.15.33:0/1805980370 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:39.112 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.418+0000 7fe36b7fe700 1 -- 172.21.15.33:0/1805980370 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe36c10c3c0 con 0x7fe36c109a90 2024-05-15T13:49:39.112 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.419+0000 7fe36b7fe700 1 --2- 172.21.15.33:0/1805980370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe36c109a90 0x7fe36c10be80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe35c0098a0 tx=0x7fe35c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc614f975fdeea6d server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:39.112 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.419+0000 7fe36a7fc700 1 -- 172.21.15.33:0/1805980370 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe35c004160 con 0x7fe36c109a90 2024-05-15T13:49:39.112 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.419+0000 7fe36a7fc700 1 -- 172.21.15.33:0/1805980370 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe35c004300 con 0x7fe36c109a90 2024-05-15T13:49:39.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.419+0000 7fe36a7fc700 1 -- 172.21.15.33:0/1805980370 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe35c011af0 con 0x7fe36c109a90 2024-05-15T13:49:39.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.419+0000 7fe371b0f700 1 -- 172.21.15.33:0/1805980370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe36c109a90 msgr2=0x7fe36c10be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:39.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.420+0000 7fe371b0f700 1 --2- 172.21.15.33:0/1805980370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe36c109a90 0x7fe36c10be80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe35c0098a0 tx=0x7fe35c009bb0 comp rx=0 tx=0).stop 2024-05-15T13:49:39.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.420+0000 7fe371b0f700 1 -- 172.21.15.33:0/1805980370 shutdown_connections 2024-05-15T13:49:39.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.420+0000 7fe371b0f700 1 --2- 172.21.15.33:0/1805980370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe36c109a90 0x7fe36c10be80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:39.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.420+0000 7fe371b0f700 1 -- 172.21.15.33:0/1805980370 >> 172.21.15.33:0/1805980370 conn(0x7fe36c103540 msgr2=0x7fe36c105960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:39.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.420+0000 7fe371b0f700 1 -- 172.21.15.33:0/1805980370 shutdown_connections 2024-05-15T13:49:39.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.420+0000 7fe371b0f700 1 -- 172.21.15.33:0/1805980370 wait complete. 2024-05-15T13:49:39.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.420+0000 7fe371b0f700 1 Processor -- start 2024-05-15T13:49:39.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.421+0000 7fe371b0f700 1 -- start start 2024-05-15T13:49:39.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.421+0000 7fe371b0f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe36c109a90 0x7fe36c19cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:39.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.421+0000 7fe371b0f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe36c10d390 con 0x7fe36c109a90 2024-05-15T13:49:39.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.421+0000 7fe36b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe36c109a90 0x7fe36c19cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:39.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.421+0000 7fe36b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe36c109a90 0x7fe36c19cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46320/0 (socket says 172.21.15.33:46320) 2024-05-15T13:49:39.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.421+0000 7fe36b7fe700 1 -- 172.21.15.33:0/2233590831 learned_addr learned my addr 172.21.15.33:0/2233590831 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:39.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.421+0000 7fe36b7fe700 1 -- 172.21.15.33:0/2233590831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe36c19d4d0 con 0x7fe36c109a90 2024-05-15T13:49:39.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.421+0000 7fe36b7fe700 1 --2- 172.21.15.33:0/2233590831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe36c109a90 0x7fe36c19cf90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe35c003f40 tx=0x7fe35c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:39.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.422+0000 7fe368ff9700 1 -- 172.21.15.33:0/2233590831 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe35c0038b0 con 0x7fe36c109a90 2024-05-15T13:49:39.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.422+0000 7fe368ff9700 1 -- 172.21.15.33:0/2233590831 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe35c003a50 con 0x7fe36c109a90 2024-05-15T13:49:39.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.422+0000 7fe371b0f700 1 -- 172.21.15.33:0/2233590831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe36c19d760 con 0x7fe36c109a90 2024-05-15T13:49:39.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.422+0000 7fe368ff9700 1 -- 172.21.15.33:0/2233590831 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe35c01a5c0 con 0x7fe36c109a90 2024-05-15T13:49:39.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.422+0000 7fe371b0f700 1 -- 172.21.15.33:0/2233590831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe36c1a0450 con 0x7fe36c109a90 2024-05-15T13:49:39.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.423+0000 7fe368ff9700 1 -- 172.21.15.33:0/2233590831 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fe35c024430 con 0x7fe36c109a90 2024-05-15T13:49:39.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.423+0000 7fe368ff9700 1 --2- 172.21.15.33:0/2233590831 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7fe35403e420 0x7fe3540408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:39.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.423+0000 7fe368ff9700 1 -- 172.21.15.33:0/2233590831 --> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe354040ff0 con 0x7fe35403e420 2024-05-15T13:49:39.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.423+0000 7fe368ff9700 1 -- 172.21.15.33:0/2233590831 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe35c053ac0 con 0x7fe36c109a90 2024-05-15T13:49:39.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.423+0000 7fe36affd700 1 -- 172.21.15.33:0/2233590831 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7fe35403e420 msgr2=0x7fe3540408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3303495364 2024-05-15T13:49:39.115 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.424+0000 7fe36affd700 1 --2- 172.21.15.33:0/2233590831 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7fe35403e420 0x7fe3540408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-15T13:49:39.115 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.625+0000 7fe36affd700 1 -- 172.21.15.33:0/2233590831 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7fe35403e420 msgr2=0x7fe3540408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3303495364 2024-05-15T13:49:39.115 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:32.625+0000 7fe36affd700 1 --2- 172.21.15.33:0/2233590831 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7fe35403e420 0x7fe3540408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-15T13:49:39.115 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:33.026+0000 7fe36affd700 1 -- 172.21.15.33:0/2233590831 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7fe35403e420 msgr2=0x7fe3540408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3303495364 2024-05-15T13:49:39.115 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:33.026+0000 7fe36affd700 1 --2- 172.21.15.33:0/2233590831 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7fe35403e420 0x7fe3540408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-15T13:49:39.115 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:33.827+0000 7fe36affd700 1 -- 172.21.15.33:0/2233590831 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7fe35403e420 msgr2=0x7fe3540408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3303495364 2024-05-15T13:49:39.115 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:33.827+0000 7fe36affd700 1 --2- 172.21.15.33:0/2233590831 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7fe35403e420 0x7fe3540408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-15T13:49:39.115 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:35.428+0000 7fe36affd700 1 -- 172.21.15.33:0/2233590831 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7fe35403e420 msgr2=0x7fe3540408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3303495364 2024-05-15T13:49:39.115 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:35.428+0000 7fe36affd700 1 --2- 172.21.15.33:0/2233590831 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7fe35403e420 0x7fe3540408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-15T13:49:39.115 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:37.566+0000 7fe368ff9700 1 -- 172.21.15.33:0/2233590831 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7fe35c024770 con 0x7fe36c109a90 2024-05-15T13:49:39.115 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:37.566+0000 7fe368ff9700 1 -- 172.21.15.33:0/2233590831 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7fe35403e420 msgr2=0x7fe3540408e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-15T13:49:39.115 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:37.566+0000 7fe368ff9700 1 --2- 172.21.15.33:0/2233590831 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7fe35403e420 0x7fe3540408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:39.115 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:38.568+0000 7fe368ff9700 1 -- 172.21.15.33:0/2233590831 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7fe35c02d3f0 con 0x7fe36c109a90 2024-05-15T13:49:39.115 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:38.568+0000 7fe368ff9700 1 --2- 172.21.15.33:0/2233590831 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7fe354041d10 0x7fe354044100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:39.115 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:38.568+0000 7fe368ff9700 1 -- 172.21.15.33:0/2233590831 --> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe35c02be50 con 0x7fe354041d10 2024-05-15T13:49:39.116 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:38.568+0000 7fe36affd700 1 --2- 172.21.15.33:0/2233590831 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7fe354041d10 0x7fe354044100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:39.116 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:38.569+0000 7fe36affd700 1 --2- 172.21.15.33:0/2233590831 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7fe354041d10 0x7fe354044100 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe360003a10 tx=0x7fe3600092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:39.116 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:38.570+0000 7fe368ff9700 1 -- 172.21.15.33:0/2233590831 <== mgr.14120 v2:172.21.15.33:6800/2680375465 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fe35c02be50 con 0x7fe354041d10 2024-05-15T13:49:39.116 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:38.578+0000 7fe371b0f700 1 -- 172.21.15.33:0/2233590831 --> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fe36c06a6a0 con 0x7fe354041d10 2024-05-15T13:49:39.116 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:38.578+0000 7fe368ff9700 1 -- 172.21.15.33:0/2233590831 <== mgr.14120 v2:172.21.15.33:6800/2680375465 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fe36c06a6a0 con 0x7fe354041d10 2024-05-15T13:49:39.116 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:38.579+0000 7fe371b0f700 1 -- 172.21.15.33:0/2233590831 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7fe354041d10 msgr2=0x7fe354044100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:39.116 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:38.579+0000 7fe371b0f700 1 --2- 172.21.15.33:0/2233590831 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7fe354041d10 0x7fe354044100 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe360003a10 tx=0x7fe3600092b0 comp rx=0 tx=0).stop 2024-05-15T13:49:39.116 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:38.579+0000 7fe371b0f700 1 -- 172.21.15.33:0/2233590831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe36c109a90 msgr2=0x7fe36c19cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:39.116 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:38.579+0000 7fe371b0f700 1 --2- 172.21.15.33:0/2233590831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe36c109a90 0x7fe36c19cf90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe35c003f40 tx=0x7fe35c011890 comp rx=0 tx=0).stop 2024-05-15T13:49:39.116 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:38.579+0000 7fe371b0f700 1 -- 172.21.15.33:0/2233590831 shutdown_connections 2024-05-15T13:49:39.116 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:38.579+0000 7fe371b0f700 1 --2- 172.21.15.33:0/2233590831 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7fe354041d10 0x7fe354044100 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:39.116 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:38.579+0000 7fe371b0f700 1 --2- 172.21.15.33:0/2233590831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe36c109a90 0x7fe36c19cf90 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:39.116 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:38.579+0000 7fe371b0f700 1 --2- 172.21.15.33:0/2233590831 >> [v2:172.21.15.33:6800/3303495364,v1:172.21.15.33:6801/3303495364] conn(0x7fe35403e420 0x7fe3540408e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:39.116 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:38.579+0000 7fe371b0f700 1 -- 172.21.15.33:0/2233590831 >> 172.21.15.33:0/2233590831 conn(0x7fe36c103540 msgr2=0x7fe36c104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:39.117 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:38.579+0000 7fe371b0f700 1 -- 172.21.15.33:0/2233590831 shutdown_connections 2024-05-15T13:49:39.117 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:38.579+0000 7fe371b0f700 1 -- 172.21.15.33:0/2233590831 wait complete. 2024-05-15T13:49:39.117 INFO:teuthology.orchestra.run.smithi033.stdout:mgr epoch 5 is available 2024-05-15T13:49:39.117 INFO:teuthology.orchestra.run.smithi033.stdout:Setting orchestrator backend to cephadm... 2024-05-15T13:49:39.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:39 smithi033 ceph-mon[31650]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-15T13:49:39.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:39 smithi033 ceph-mon[31650]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-15T13:49:40.282 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.490+0000 7f0ef9261700 1 Processor -- start 2024-05-15T13:49:40.282 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.490+0000 7f0ef9261700 1 -- start start 2024-05-15T13:49:40.282 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.491+0000 7f0ef9261700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ef407ee20 0x7f0ef407d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:40.282 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.491+0000 7f0ef9261700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef407d810 con 0x7f0ef407ee20 2024-05-15T13:49:40.282 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.491+0000 7f0ef2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ef407ee20 0x7f0ef407d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:40.282 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.491+0000 7f0ef2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ef407ee20 0x7f0ef407d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34402/0 (socket says 172.21.15.33:34402) 2024-05-15T13:49:40.283 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.491+0000 7f0ef2ffd700 1 -- 172.21.15.33:0/3911299813 learned_addr learned my addr 172.21.15.33:0/3911299813 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:40.283 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.491+0000 7f0ef2ffd700 1 -- 172.21.15.33:0/3911299813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef407d990 con 0x7f0ef407ee20 2024-05-15T13:49:40.283 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.491+0000 7f0ef2ffd700 1 --2- 172.21.15.33:0/3911299813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ef407ee20 0x7f0ef407d2d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0edc0098a0 tx=0x7f0edc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698db7f5081b4056 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:40.283 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.491+0000 7f0ef1ffb700 1 -- 172.21.15.33:0/3911299813 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0edc004160 con 0x7f0ef407ee20 2024-05-15T13:49:40.283 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.492+0000 7f0ef1ffb700 1 -- 172.21.15.33:0/3911299813 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0edc004300 con 0x7f0ef407ee20 2024-05-15T13:49:40.283 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.492+0000 7f0ef1ffb700 1 -- 172.21.15.33:0/3911299813 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0edc011af0 con 0x7f0ef407ee20 2024-05-15T13:49:40.283 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.492+0000 7f0ef9261700 1 -- 172.21.15.33:0/3911299813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ef407ee20 msgr2=0x7f0ef407d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:40.283 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.492+0000 7f0ef9261700 1 --2- 172.21.15.33:0/3911299813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ef407ee20 0x7f0ef407d2d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0edc0098a0 tx=0x7f0edc009bb0 comp rx=0 tx=0).stop 2024-05-15T13:49:40.283 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.492+0000 7f0ef9261700 1 -- 172.21.15.33:0/3911299813 shutdown_connections 2024-05-15T13:49:40.283 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.492+0000 7f0ef9261700 1 --2- 172.21.15.33:0/3911299813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ef407ee20 0x7f0ef407d2d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:40.283 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.492+0000 7f0ef9261700 1 -- 172.21.15.33:0/3911299813 >> 172.21.15.33:0/3911299813 conn(0x7f0ef4104660 msgr2=0x7f0ef4106a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:40.283 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.492+0000 7f0ef9261700 1 -- 172.21.15.33:0/3911299813 shutdown_connections 2024-05-15T13:49:40.283 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.492+0000 7f0ef9261700 1 -- 172.21.15.33:0/3911299813 wait complete. 2024-05-15T13:49:40.283 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.493+0000 7f0ef9261700 1 Processor -- start 2024-05-15T13:49:40.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.493+0000 7f0ef9261700 1 -- start start 2024-05-15T13:49:40.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.493+0000 7f0ef9261700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ef407ee20 0x7f0ef419f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:40.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.493+0000 7f0ef9261700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef410f5d0 con 0x7f0ef407ee20 2024-05-15T13:49:40.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.494+0000 7f0ef2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ef407ee20 0x7f0ef419f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:40.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.494+0000 7f0ef2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ef407ee20 0x7f0ef419f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34412/0 (socket says 172.21.15.33:34412) 2024-05-15T13:49:40.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.494+0000 7f0ef2ffd700 1 -- 172.21.15.33:0/4073147503 learned_addr learned my addr 172.21.15.33:0/4073147503 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:40.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.494+0000 7f0ef2ffd700 1 -- 172.21.15.33:0/4073147503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef419f740 con 0x7f0ef407ee20 2024-05-15T13:49:40.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.494+0000 7f0ef2ffd700 1 --2- 172.21.15.33:0/4073147503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ef407ee20 0x7f0ef419f200 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0edc009fb0 tx=0x7f0edc011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:40.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.494+0000 7f0eebfff700 1 -- 172.21.15.33:0/4073147503 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0edc0036a0 con 0x7f0ef407ee20 2024-05-15T13:49:40.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.494+0000 7f0eebfff700 1 -- 172.21.15.33:0/4073147503 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0edc01a430 con 0x7f0ef407ee20 2024-05-15T13:49:40.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.494+0000 7f0eebfff700 1 -- 172.21.15.33:0/4073147503 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0edc019550 con 0x7f0ef407ee20 2024-05-15T13:49:40.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.494+0000 7f0ef9261700 1 -- 172.21.15.33:0/4073147503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef419f9d0 con 0x7f0ef407ee20 2024-05-15T13:49:40.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.494+0000 7f0ef9261700 1 -- 172.21.15.33:0/4073147503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef41a26c0 con 0x7f0ef407ee20 2024-05-15T13:49:40.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.495+0000 7f0eebfff700 1 -- 172.21.15.33:0/4073147503 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f0edc003840 con 0x7f0ef407ee20 2024-05-15T13:49:40.285 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.495+0000 7f0eebfff700 1 --2- 172.21.15.33:0/4073147503 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f0ee003e340 0x7f0ee0040800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:40.285 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.495+0000 7f0eebfff700 1 -- 172.21.15.33:0/4073147503 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0edc053aa0 con 0x7f0ef407ee20 2024-05-15T13:49:40.285 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.495+0000 7f0ef9261700 1 -- 172.21.15.33:0/4073147503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ef4064ba0 con 0x7f0ef407ee20 2024-05-15T13:49:40.285 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.496+0000 7f0ef27fc700 1 --2- 172.21.15.33:0/4073147503 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f0ee003e340 0x7f0ee0040800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:40.285 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.496+0000 7f0ef27fc700 1 --2- 172.21.15.33:0/4073147503 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f0ee003e340 0x7f0ee0040800 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4006fd0 tx=0x7f0ee4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:40.285 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.501+0000 7f0eebfff700 1 -- 172.21.15.33:0/4073147503 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0edc00b3e0 con 0x7f0ef407ee20 2024-05-15T13:49:40.285 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.619+0000 7f0eebfff700 1 -- 172.21.15.33:0/4073147503 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f0edc01a8e0 con 0x7f0ef407ee20 2024-05-15T13:49:40.285 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.750+0000 7f0ef9261700 1 -- 172.21.15.33:0/4073147503 --> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f0ef4105950 con 0x7f0ee003e340 2024-05-15T13:49:40.285 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.756+0000 7f0eebfff700 1 -- 172.21.15.33:0/4073147503 <== mgr.14120 v2:172.21.15.33:6800/2680375465 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0ef4105950 con 0x7f0ee003e340 2024-05-15T13:49:40.285 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.758+0000 7f0ef9261700 1 -- 172.21.15.33:0/4073147503 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f0ee003e340 msgr2=0x7f0ee0040800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:40.285 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.758+0000 7f0ef9261700 1 --2- 172.21.15.33:0/4073147503 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f0ee003e340 0x7f0ee0040800 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4006fd0 tx=0x7f0ee4006e40 comp rx=0 tx=0).stop 2024-05-15T13:49:40.285 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.758+0000 7f0ef9261700 1 -- 172.21.15.33:0/4073147503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ef407ee20 msgr2=0x7f0ef419f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:40.285 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.758+0000 7f0ef9261700 1 --2- 172.21.15.33:0/4073147503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ef407ee20 0x7f0ef419f200 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0edc009fb0 tx=0x7f0edc011940 comp rx=0 tx=0).stop 2024-05-15T13:49:40.285 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.758+0000 7f0ef9261700 1 -- 172.21.15.33:0/4073147503 shutdown_connections 2024-05-15T13:49:40.286 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.758+0000 7f0ef9261700 1 --2- 172.21.15.33:0/4073147503 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f0ee003e340 0x7f0ee0040800 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:40.286 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.758+0000 7f0ef9261700 1 --2- 172.21.15.33:0/4073147503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ef407ee20 0x7f0ef419f200 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:40.286 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.758+0000 7f0ef9261700 1 -- 172.21.15.33:0/4073147503 >> 172.21.15.33:0/4073147503 conn(0x7f0ef4104660 msgr2=0x7f0ef4105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:40.286 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.758+0000 7f0ef9261700 1 -- 172.21.15.33:0/4073147503 shutdown_connections 2024-05-15T13:49:40.286 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:39.758+0000 7f0ef9261700 1 -- 172.21.15.33:0/4073147503 wait complete. 2024-05-15T13:49:40.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:40 smithi033 ceph-mon[31650]: mgrmap e8: x(active, since 2s) 2024-05-15T13:49:40.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:40 smithi033 ceph-mon[31650]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:49:40.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:40 smithi033 ceph-mon[31650]: from='mgr.14120 172.21.15.33:0/2424021838' entity='mgr.x' 2024-05-15T13:49:40.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:40 smithi033 ceph-mon[31650]: from='mgr.14120 172.21.15.33:0/2424021838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:49:40.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:40 smithi033 ceph-mon[31650]: [15/May/2024:13:49:40] ENGINE Bus STARTING 2024-05-15T13:49:40.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:40 smithi033 ceph-mon[31650]: from='mgr.14120 172.21.15.33:0/2424021838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:49:41.573 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-15T13:49:41.573 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.698+0000 7fea8d374700 1 Processor -- start 2024-05-15T13:49:41.573 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.698+0000 7fea8d374700 1 -- start start 2024-05-15T13:49:41.573 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.698+0000 7fea8d374700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea881090e0 0x7fea88109500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:41.573 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.698+0000 7fea8d374700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fea88109a40 con 0x7fea881090e0 2024-05-15T13:49:41.573 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.698+0000 7fea86ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea881090e0 0x7fea88109500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:41.573 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.698+0000 7fea86ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea881090e0 0x7fea88109500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34420/0 (socket says 172.21.15.33:34420) 2024-05-15T13:49:41.574 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.698+0000 7fea86ffd700 1 -- 172.21.15.33:0/969110558 learned_addr learned my addr 172.21.15.33:0/969110558 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:41.574 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.699+0000 7fea86ffd700 1 -- 172.21.15.33:0/969110558 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea88109bc0 con 0x7fea881090e0 2024-05-15T13:49:41.574 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.699+0000 7fea86ffd700 1 --2- 172.21.15.33:0/969110558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea881090e0 0x7fea88109500 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fea78009e80 tx=0x7fea7800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc0eca3fedc90b2f server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:41.574 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.699+0000 7fea85ffb700 1 -- 172.21.15.33:0/969110558 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea78017070 con 0x7fea881090e0 2024-05-15T13:49:41.574 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.699+0000 7fea85ffb700 1 -- 172.21.15.33:0/969110558 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fea7800b8c0 con 0x7fea881090e0 2024-05-15T13:49:41.574 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.699+0000 7fea85ffb700 1 -- 172.21.15.33:0/969110558 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea78004490 con 0x7fea881090e0 2024-05-15T13:49:41.574 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.700+0000 7fea8d374700 1 -- 172.21.15.33:0/969110558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea881090e0 msgr2=0x7fea88109500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:41.574 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.700+0000 7fea8d374700 1 --2- 172.21.15.33:0/969110558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea881090e0 0x7fea88109500 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fea78009e80 tx=0x7fea7800b330 comp rx=0 tx=0).stop 2024-05-15T13:49:41.574 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.700+0000 7fea8d374700 1 -- 172.21.15.33:0/969110558 shutdown_connections 2024-05-15T13:49:41.574 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.700+0000 7fea8d374700 1 --2- 172.21.15.33:0/969110558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea881090e0 0x7fea88109500 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:41.574 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.700+0000 7fea8d374700 1 -- 172.21.15.33:0/969110558 >> 172.21.15.33:0/969110558 conn(0x7fea88104740 msgr2=0x7fea88106b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:41.574 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.700+0000 7fea8d374700 1 -- 172.21.15.33:0/969110558 shutdown_connections 2024-05-15T13:49:41.574 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.700+0000 7fea8d374700 1 -- 172.21.15.33:0/969110558 wait complete. 2024-05-15T13:49:41.575 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.700+0000 7fea8d374700 1 Processor -- start 2024-05-15T13:49:41.575 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.700+0000 7fea8d374700 1 -- start start 2024-05-15T13:49:41.575 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.701+0000 7fea8d374700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea881090e0 0x7fea8807eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:41.575 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.701+0000 7fea8d374700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fea8810b070 con 0x7fea881090e0 2024-05-15T13:49:41.575 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.701+0000 7fea86ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea881090e0 0x7fea8807eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:41.575 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.701+0000 7fea86ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea881090e0 0x7fea8807eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34428/0 (socket says 172.21.15.33:34428) 2024-05-15T13:49:41.575 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.701+0000 7fea86ffd700 1 -- 172.21.15.33:0/4003221377 learned_addr learned my addr 172.21.15.33:0/4003221377 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:41.575 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.701+0000 7fea86ffd700 1 -- 172.21.15.33:0/4003221377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8807f0a0 con 0x7fea881090e0 2024-05-15T13:49:41.575 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.701+0000 7fea86ffd700 1 --2- 172.21.15.33:0/4003221377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea881090e0 0x7fea8807eb60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fea7800b7b0 tx=0x7fea7800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:41.575 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.701+0000 7fea7ffff700 1 -- 172.21.15.33:0/4003221377 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea78020020 con 0x7fea881090e0 2024-05-15T13:49:41.575 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.701+0000 7fea7ffff700 1 -- 172.21.15.33:0/4003221377 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fea78012430 con 0x7fea881090e0 2024-05-15T13:49:41.575 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.701+0000 7fea8d374700 1 -- 172.21.15.33:0/4003221377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea8807d290 con 0x7fea881090e0 2024-05-15T13:49:41.575 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.701+0000 7fea7ffff700 1 -- 172.21.15.33:0/4003221377 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea78017040 con 0x7fea881090e0 2024-05-15T13:49:41.575 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.701+0000 7fea8d374700 1 -- 172.21.15.33:0/4003221377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea8807d7d0 con 0x7fea881090e0 2024-05-15T13:49:41.575 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.702+0000 7fea7ffff700 1 -- 172.21.15.33:0/4003221377 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fea78023020 con 0x7fea881090e0 2024-05-15T13:49:41.576 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.702+0000 7fea8d374700 1 -- 172.21.15.33:0/4003221377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea8806a6a0 con 0x7fea881090e0 2024-05-15T13:49:41.576 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.702+0000 7fea7ffff700 1 --2- 172.21.15.33:0/4003221377 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7fea7403e420 0x7fea740408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:41.576 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.703+0000 7fea7ffff700 1 -- 172.21.15.33:0/4003221377 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fea7800f050 con 0x7fea881090e0 2024-05-15T13:49:41.576 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.703+0000 7fea867fc700 1 --2- 172.21.15.33:0/4003221377 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7fea7403e420 0x7fea740408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:41.576 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.703+0000 7fea867fc700 1 --2- 172.21.15.33:0/4003221377 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7fea7403e420 0x7fea740408e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fea70006fd0 tx=0x7fea70006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:41.576 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.708+0000 7fea7ffff700 1 -- 172.21.15.33:0/4003221377 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fea7802a020 con 0x7fea881090e0 2024-05-15T13:49:41.576 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.928+0000 7fea8d374700 1 -- 172.21.15.33:0/4003221377 --> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fea881058a0 con 0x7fea7403e420 2024-05-15T13:49:41.576 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.929+0000 7fea7ffff700 1 -- 172.21.15.33:0/4003221377 <== mgr.14120 v2:172.21.15.33:6800/2680375465 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fea881058a0 con 0x7fea7403e420 2024-05-15T13:49:41.576 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.931+0000 7fea8d374700 1 -- 172.21.15.33:0/4003221377 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7fea7403e420 msgr2=0x7fea740408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:41.576 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.931+0000 7fea8d374700 1 --2- 172.21.15.33:0/4003221377 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7fea7403e420 0x7fea740408e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fea70006fd0 tx=0x7fea70006e40 comp rx=0 tx=0).stop 2024-05-15T13:49:41.576 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.931+0000 7fea8d374700 1 -- 172.21.15.33:0/4003221377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea881090e0 msgr2=0x7fea8807eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:41.576 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.931+0000 7fea8d374700 1 --2- 172.21.15.33:0/4003221377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea881090e0 0x7fea8807eb60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fea7800b7b0 tx=0x7fea7800be20 comp rx=0 tx=0).stop 2024-05-15T13:49:41.576 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.931+0000 7fea8d374700 1 -- 172.21.15.33:0/4003221377 shutdown_connections 2024-05-15T13:49:41.577 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.931+0000 7fea8d374700 1 --2- 172.21.15.33:0/4003221377 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7fea7403e420 0x7fea740408e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:41.577 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.931+0000 7fea8d374700 1 --2- 172.21.15.33:0/4003221377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea881090e0 0x7fea8807eb60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:41.577 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.931+0000 7fea8d374700 1 -- 172.21.15.33:0/4003221377 >> 172.21.15.33:0/4003221377 conn(0x7fea88104740 msgr2=0x7fea88105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:41.577 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.931+0000 7fea8d374700 1 -- 172.21.15.33:0/4003221377 shutdown_connections 2024-05-15T13:49:41.577 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:40.931+0000 7fea8d374700 1 -- 172.21.15.33:0/4003221377 wait complete. 2024-05-15T13:49:41.577 INFO:teuthology.orchestra.run.smithi033.stdout:Generating ssh key... 2024-05-15T13:49:41.859 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:41 smithi033 ceph-mon[31650]: [15/May/2024:13:49:40] ENGINE Serving on http://172.21.15.33:8765 2024-05-15T13:49:41.859 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:41 smithi033 ceph-mon[31650]: [15/May/2024:13:49:40] ENGINE Serving on https://172.21.15.33:7150 2024-05-15T13:49:41.860 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:41 smithi033 ceph-mon[31650]: [15/May/2024:13:49:40] ENGINE Bus STARTED 2024-05-15T13:49:41.860 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:41 smithi033 ceph-mon[31650]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:49:42.848 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:42 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: Generating public/private rsa key pair. 2024-05-15T13:49:42.848 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:42 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: Your identification has been saved in /tmp/tmpmcwkvvir/key. 2024-05-15T13:49:42.848 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:42 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: Your public key has been saved in /tmp/tmpmcwkvvir/key.pub. 2024-05-15T13:49:42.848 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:42 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: The key fingerprint is: 2024-05-15T13:49:42.848 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:42 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: SHA256:tDGJF01qWJJWr1CE6Dl45TwCMZnKcFZAAe/VtztPowE ceph-bca78ca4-12c1-11ef-bc98-c7b262605968 2024-05-15T13:49:42.849 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:42 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: The key's randomart image is: 2024-05-15T13:49:42.849 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:42 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: +---[RSA 3072]----+ 2024-05-15T13:49:42.849 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:42 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: |.o*B...=*o. | 2024-05-15T13:49:42.849 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:42 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: |..*...=* =. | 2024-05-15T13:49:42.849 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:42 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: |o+.+.*= X . | 2024-05-15T13:49:42.849 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:42 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: |.o..* +* * | 2024-05-15T13:49:42.849 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:42 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: | .. oE.S | 2024-05-15T13:49:42.849 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:42 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: | . . | 2024-05-15T13:49:42.849 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:42 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: | + o | 2024-05-15T13:49:42.849 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:42 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: | * . | 2024-05-15T13:49:42.849 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:42 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: | . . | 2024-05-15T13:49:42.850 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:42 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: +----[SHA256]-----+ 2024-05-15T13:49:43.112 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.969+0000 7f1aafde1700 1 Processor -- start 2024-05-15T13:49:43.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.969+0000 7f1aafde1700 1 -- start start 2024-05-15T13:49:43.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.969+0000 7f1aafde1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1aa8109080 0x7f1aa81094a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:43.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.969+0000 7f1aafde1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa81099e0 con 0x7f1aa8109080 2024-05-15T13:49:43.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.970+0000 7f1aadb7d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1aa8109080 0x7f1aa81094a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:43.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.970+0000 7f1aadb7d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1aa8109080 0x7f1aa81094a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34440/0 (socket says 172.21.15.33:34440) 2024-05-15T13:49:43.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.970+0000 7f1aadb7d700 1 -- 172.21.15.33:0/2990151652 learned_addr learned my addr 172.21.15.33:0/2990151652 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:43.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.970+0000 7f1aadb7d700 1 -- 172.21.15.33:0/2990151652 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa8109b60 con 0x7f1aa8109080 2024-05-15T13:49:43.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.970+0000 7f1aadb7d700 1 --2- 172.21.15.33:0/2990151652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1aa8109080 0x7f1aa81094a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1aa40098a0 tx=0x7f1aa4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a50f7f27a48eae6b server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:43.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.970+0000 7f1aacb7b700 1 -- 172.21.15.33:0/2990151652 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1aa4004160 con 0x7f1aa8109080 2024-05-15T13:49:43.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.970+0000 7f1aacb7b700 1 -- 172.21.15.33:0/2990151652 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1aa4004300 con 0x7f1aa8109080 2024-05-15T13:49:43.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.970+0000 7f1aacb7b700 1 -- 172.21.15.33:0/2990151652 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1aa4011af0 con 0x7f1aa8109080 2024-05-15T13:49:43.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.971+0000 7f1aafde1700 1 -- 172.21.15.33:0/2990151652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1aa8109080 msgr2=0x7f1aa81094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:43.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.971+0000 7f1aafde1700 1 --2- 172.21.15.33:0/2990151652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1aa8109080 0x7f1aa81094a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1aa40098a0 tx=0x7f1aa4009bb0 comp rx=0 tx=0).stop 2024-05-15T13:49:43.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.971+0000 7f1aafde1700 1 -- 172.21.15.33:0/2990151652 shutdown_connections 2024-05-15T13:49:43.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.971+0000 7f1aafde1700 1 --2- 172.21.15.33:0/2990151652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1aa8109080 0x7f1aa81094a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:43.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.971+0000 7f1aafde1700 1 -- 172.21.15.33:0/2990151652 >> 172.21.15.33:0/2990151652 conn(0x7f1aa8104720 msgr2=0x7f1aa8106b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:43.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.971+0000 7f1aafde1700 1 -- 172.21.15.33:0/2990151652 shutdown_connections 2024-05-15T13:49:43.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.971+0000 7f1aafde1700 1 -- 172.21.15.33:0/2990151652 wait complete. 2024-05-15T13:49:43.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.972+0000 7f1aafde1700 1 Processor -- start 2024-05-15T13:49:43.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.972+0000 7f1aafde1700 1 -- start start 2024-05-15T13:49:43.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.972+0000 7f1aafde1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1aa8109080 0x7f1aa819f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:43.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.972+0000 7f1aafde1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa810b080 con 0x7f1aa8109080 2024-05-15T13:49:43.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.972+0000 7f1aadb7d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1aa8109080 0x7f1aa819f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:43.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.972+0000 7f1aadb7d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1aa8109080 0x7f1aa819f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34454/0 (socket says 172.21.15.33:34454) 2024-05-15T13:49:43.115 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.972+0000 7f1aadb7d700 1 -- 172.21.15.33:0/2335154474 learned_addr learned my addr 172.21.15.33:0/2335154474 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:43.115 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.972+0000 7f1aadb7d700 1 -- 172.21.15.33:0/2335154474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa819f6e0 con 0x7f1aa8109080 2024-05-15T13:49:43.115 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.972+0000 7f1aadb7d700 1 --2- 172.21.15.33:0/2335154474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1aa8109080 0x7f1aa819f1a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4011ac0 tx=0x7f1aa4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:43.115 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.973+0000 7f1a9effd700 1 -- 172.21.15.33:0/2335154474 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1aa4003b20 con 0x7f1aa8109080 2024-05-15T13:49:43.115 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.973+0000 7f1aafde1700 1 -- 172.21.15.33:0/2335154474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aa819f970 con 0x7f1aa8109080 2024-05-15T13:49:43.115 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.973+0000 7f1a9effd700 1 -- 172.21.15.33:0/2335154474 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1aa401a8c0 con 0x7f1aa8109080 2024-05-15T13:49:43.115 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.973+0000 7f1a9effd700 1 -- 172.21.15.33:0/2335154474 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1aa40248d0 con 0x7f1aa8109080 2024-05-15T13:49:43.115 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.973+0000 7f1aafde1700 1 -- 172.21.15.33:0/2335154474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aa81a2660 con 0x7f1aa8109080 2024-05-15T13:49:43.115 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.974+0000 7f1a9effd700 1 -- 172.21.15.33:0/2335154474 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f1aa4022020 con 0x7f1aa8109080 2024-05-15T13:49:43.115 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.974+0000 7f1aafde1700 1 -- 172.21.15.33:0/2335154474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aa8064ba0 con 0x7f1aa8109080 2024-05-15T13:49:43.115 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.974+0000 7f1a9effd700 1 --2- 172.21.15.33:0/2335154474 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f1a9403e470 0x7f1a94040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:43.115 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.974+0000 7f1a9effd700 1 -- 172.21.15.33:0/2335154474 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1aa4053870 con 0x7f1aa8109080 2024-05-15T13:49:43.115 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.974+0000 7f1aad37c700 1 --2- 172.21.15.33:0/2335154474 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f1a9403e470 0x7f1a94040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:43.115 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.975+0000 7f1aad37c700 1 --2- 172.21.15.33:0/2335154474 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f1a9403e470 0x7f1a94040930 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1a98006fd0 tx=0x7f1a98006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:43.116 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:41.979+0000 7f1a9effd700 1 -- 172.21.15.33:0/2335154474 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1aa40172b0 con 0x7f1aa8109080 2024-05-15T13:49:43.116 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:42.202+0000 7f1aafde1700 1 -- 172.21.15.33:0/2335154474 --> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f1aa81058c0 con 0x7f1a9403e470 2024-05-15T13:49:43.116 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:42.494+0000 7f1a9effd700 1 -- 172.21.15.33:0/2335154474 <== mgr.14120 v2:172.21.15.33:6800/2680375465 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1aa81058c0 con 0x7f1a9403e470 2024-05-15T13:49:43.116 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:42.496+0000 7f1aafde1700 1 -- 172.21.15.33:0/2335154474 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f1a9403e470 msgr2=0x7f1a94040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:43.116 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:42.496+0000 7f1aafde1700 1 --2- 172.21.15.33:0/2335154474 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f1a9403e470 0x7f1a94040930 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1a98006fd0 tx=0x7f1a98006e40 comp rx=0 tx=0).stop 2024-05-15T13:49:43.116 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:42.496+0000 7f1aafde1700 1 -- 172.21.15.33:0/2335154474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1aa8109080 msgr2=0x7f1aa819f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:43.116 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:42.496+0000 7f1aafde1700 1 --2- 172.21.15.33:0/2335154474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1aa8109080 0x7f1aa819f1a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4011ac0 tx=0x7f1aa4011ba0 comp rx=0 tx=0).stop 2024-05-15T13:49:43.116 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:42.497+0000 7f1aafde1700 1 -- 172.21.15.33:0/2335154474 shutdown_connections 2024-05-15T13:49:43.116 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:42.497+0000 7f1aafde1700 1 --2- 172.21.15.33:0/2335154474 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f1a9403e470 0x7f1a94040930 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:43.116 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:42.497+0000 7f1aafde1700 1 --2- 172.21.15.33:0/2335154474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1aa8109080 0x7f1aa819f1a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:43.116 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:42.497+0000 7f1aafde1700 1 -- 172.21.15.33:0/2335154474 >> 172.21.15.33:0/2335154474 conn(0x7f1aa8104720 msgr2=0x7f1aa81051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:43.116 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:42.497+0000 7f1aafde1700 1 -- 172.21.15.33:0/2335154474 shutdown_connections 2024-05-15T13:49:43.116 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:42.497+0000 7f1aafde1700 1 -- 172.21.15.33:0/2335154474 wait complete. 2024-05-15T13:49:43.789 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:43 smithi033 ceph-mon[31650]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:49:43.789 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:43 smithi033 ceph-mon[31650]: Generating ssh key... 2024-05-15T13:49:43.789 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:43 smithi033 ceph-mon[31650]: from='mgr.14120 172.21.15.33:0/2424021838' entity='mgr.x' 2024-05-15T13:49:43.789 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:43 smithi033 ceph-mon[31650]: from='mgr.14120 172.21.15.33:0/2424021838' entity='mgr.x' 2024-05-15T13:49:44.346 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCqjB+aTRhJp3xaxbjRkjYKgziFcZNRQIV8T0Plg01lMcLUr4hsC9nik2NM6iUyDmyvz2O4kDw0M2p96Uj9DDMoH68LKcZEVwklbmnaELTl8hazoOBabH219US+CfyP9jhdqE2JX3JOeBAGAQi5oyhHsGrTSJ9HZ2tVGH3+rbbnwz2cLHjmhuyLarwkAabZyuPOjLhGzLQT6ta+pd6PzbHcBpMYKRJN6flIZjBHoEAz5OTTMP8qokDEuwzVyFQpMyfn/AzNB6wu2RYcqQMdHS2l2KQ0fgxzQeyISiF3fRocYqsQ92Om/bFPlHoy/m8Yv2Kll3xnNCRH9ApB6DHkiEix292eAFE5FFMwt96DTUavx2BcRK7FSt5H1R8dniFG7llRuxeCw8oWpZxjNKWgjM+qwTlWKdeco4bllesW2t/xk2WnwRujORiBq5xIBaQMr0beHnB5eNETiw0vliw1WdcOpOSsip7rmY8KZ/EG9kqOYWmNDI8IOQYFWAmCZRZPxHM= ceph-bca78ca4-12c1-11ef-bc98-c7b262605968 2024-05-15T13:49:44.346 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.558+0000 7fcb81daf700 1 Processor -- start 2024-05-15T13:49:44.346 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.559+0000 7fcb81daf700 1 -- start start 2024-05-15T13:49:44.346 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.559+0000 7fcb81daf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb7c109ad0 0x7fcb7c10bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:44.346 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.559+0000 7fcb81daf700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7c076e40 con 0x7fcb7c109ad0 2024-05-15T13:49:44.346 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.559+0000 7fcb7b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb7c109ad0 0x7fcb7c10bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:44.346 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.559+0000 7fcb7b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb7c109ad0 0x7fcb7c10bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46600/0 (socket says 172.21.15.33:46600) 2024-05-15T13:49:44.347 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.559+0000 7fcb7b7fe700 1 -- 172.21.15.33:0/873128010 learned_addr learned my addr 172.21.15.33:0/873128010 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:44.347 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.559+0000 7fcb7b7fe700 1 -- 172.21.15.33:0/873128010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb7c10c400 con 0x7fcb7c109ad0 2024-05-15T13:49:44.347 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.559+0000 7fcb7b7fe700 1 --2- 172.21.15.33:0/873128010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb7c109ad0 0x7fcb7c10bec0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcb640098a0 tx=0x7fcb64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5c53a1af7f365f server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:44.347 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.560+0000 7fcb7a7fc700 1 -- 172.21.15.33:0/873128010 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb64004160 con 0x7fcb7c109ad0 2024-05-15T13:49:44.347 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.560+0000 7fcb7a7fc700 1 -- 172.21.15.33:0/873128010 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcb64004300 con 0x7fcb7c109ad0 2024-05-15T13:49:44.347 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.560+0000 7fcb7a7fc700 1 -- 172.21.15.33:0/873128010 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb64011af0 con 0x7fcb7c109ad0 2024-05-15T13:49:44.347 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.560+0000 7fcb81daf700 1 -- 172.21.15.33:0/873128010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb7c109ad0 msgr2=0x7fcb7c10bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:44.347 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.560+0000 7fcb81daf700 1 --2- 172.21.15.33:0/873128010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb7c109ad0 0x7fcb7c10bec0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcb640098a0 tx=0x7fcb64009bb0 comp rx=0 tx=0).stop 2024-05-15T13:49:44.347 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.561+0000 7fcb81daf700 1 -- 172.21.15.33:0/873128010 shutdown_connections 2024-05-15T13:49:44.347 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.561+0000 7fcb81daf700 1 --2- 172.21.15.33:0/873128010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb7c109ad0 0x7fcb7c10bec0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:44.347 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.561+0000 7fcb81daf700 1 -- 172.21.15.33:0/873128010 >> 172.21.15.33:0/873128010 conn(0x7fcb7c1034e0 msgr2=0x7fcb7c105940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:44.347 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.561+0000 7fcb81daf700 1 -- 172.21.15.33:0/873128010 shutdown_connections 2024-05-15T13:49:44.347 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.561+0000 7fcb81daf700 1 -- 172.21.15.33:0/873128010 wait complete. 2024-05-15T13:49:44.348 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.561+0000 7fcb81daf700 1 Processor -- start 2024-05-15T13:49:44.348 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.561+0000 7fcb81daf700 1 -- start start 2024-05-15T13:49:44.348 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.562+0000 7fcb81daf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb7c109ad0 0x7fcb7c109350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:44.348 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.562+0000 7fcb81daf700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7c10d3d0 con 0x7fcb7c109ad0 2024-05-15T13:49:44.348 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.562+0000 7fcb7b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb7c109ad0 0x7fcb7c109350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:44.348 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.562+0000 7fcb7b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb7c109ad0 0x7fcb7c109350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46614/0 (socket says 172.21.15.33:46614) 2024-05-15T13:49:44.348 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.562+0000 7fcb7b7fe700 1 -- 172.21.15.33:0/3774213201 learned_addr learned my addr 172.21.15.33:0/3774213201 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:44.348 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.562+0000 7fcb7b7fe700 1 -- 172.21.15.33:0/3774213201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb7c109890 con 0x7fcb7c109ad0 2024-05-15T13:49:44.348 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.562+0000 7fcb7b7fe700 1 --2- 172.21.15.33:0/3774213201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb7c109ad0 0x7fcb7c109350 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcb64011ac0 tx=0x7fcb64011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:44.348 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.562+0000 7fcb78ff9700 1 -- 172.21.15.33:0/3774213201 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb64003970 con 0x7fcb7c109ad0 2024-05-15T13:49:44.348 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.563+0000 7fcb78ff9700 1 -- 172.21.15.33:0/3774213201 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcb64003b10 con 0x7fcb7c109ad0 2024-05-15T13:49:44.348 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.563+0000 7fcb81daf700 1 -- 172.21.15.33:0/3774213201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb7c107a80 con 0x7fcb7c109ad0 2024-05-15T13:49:44.348 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.563+0000 7fcb78ff9700 1 -- 172.21.15.33:0/3774213201 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb64024610 con 0x7fcb7c109ad0 2024-05-15T13:49:44.348 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.563+0000 7fcb81daf700 1 -- 172.21.15.33:0/3774213201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb7c107fc0 con 0x7fcb7c109ad0 2024-05-15T13:49:44.349 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.564+0000 7fcb81daf700 1 -- 172.21.15.33:0/3774213201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb7c064ba0 con 0x7fcb7c109ad0 2024-05-15T13:49:44.349 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.564+0000 7fcb78ff9700 1 -- 172.21.15.33:0/3774213201 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fcb640248b0 con 0x7fcb7c109ad0 2024-05-15T13:49:44.349 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.564+0000 7fcb78ff9700 1 --2- 172.21.15.33:0/3774213201 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7fcb6803e470 0x7fcb68040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:44.349 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.564+0000 7fcb78ff9700 1 -- 172.21.15.33:0/3774213201 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcb64053860 con 0x7fcb7c109ad0 2024-05-15T13:49:44.349 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.564+0000 7fcb7affd700 1 --2- 172.21.15.33:0/3774213201 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7fcb6803e470 0x7fcb68040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:44.349 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.565+0000 7fcb7affd700 1 --2- 172.21.15.33:0/3774213201 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7fcb6803e470 0x7fcb68040930 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c006fd0 tx=0x7fcb6c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:44.349 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.570+0000 7fcb78ff9700 1 -- 172.21.15.33:0/3774213201 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcb6400a0d0 con 0x7fcb7c109ad0 2024-05-15T13:49:44.349 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.786+0000 7fcb81daf700 1 -- 172.21.15.33:0/3774213201 --> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fcb7c1047a0 con 0x7fcb6803e470 2024-05-15T13:49:44.349 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.786+0000 7fcb78ff9700 1 -- 172.21.15.33:0/3774213201 <== mgr.14120 v2:172.21.15.33:6800/2680375465 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fcb7c1047a0 con 0x7fcb6803e470 2024-05-15T13:49:44.349 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.788+0000 7fcb81daf700 1 -- 172.21.15.33:0/3774213201 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7fcb6803e470 msgr2=0x7fcb68040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:44.349 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.788+0000 7fcb81daf700 1 --2- 172.21.15.33:0/3774213201 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7fcb6803e470 0x7fcb68040930 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c006fd0 tx=0x7fcb6c006e40 comp rx=0 tx=0).stop 2024-05-15T13:49:44.349 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.788+0000 7fcb81daf700 1 -- 172.21.15.33:0/3774213201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb7c109ad0 msgr2=0x7fcb7c109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:44.349 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.788+0000 7fcb81daf700 1 --2- 172.21.15.33:0/3774213201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb7c109ad0 0x7fcb7c109350 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcb64011ac0 tx=0x7fcb64011ba0 comp rx=0 tx=0).stop 2024-05-15T13:49:44.350 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.789+0000 7fcb81daf700 1 -- 172.21.15.33:0/3774213201 shutdown_connections 2024-05-15T13:49:44.350 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.789+0000 7fcb81daf700 1 --2- 172.21.15.33:0/3774213201 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7fcb6803e470 0x7fcb68040930 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:44.350 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.789+0000 7fcb81daf700 1 --2- 172.21.15.33:0/3774213201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb7c109ad0 0x7fcb7c109350 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:44.350 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.789+0000 7fcb81daf700 1 -- 172.21.15.33:0/3774213201 >> 172.21.15.33:0/3774213201 conn(0x7fcb7c1034e0 msgr2=0x7fcb7c104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:44.350 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.789+0000 7fcb81daf700 1 -- 172.21.15.33:0/3774213201 shutdown_connections 2024-05-15T13:49:44.350 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:43.789+0000 7fcb81daf700 1 -- 172.21.15.33:0/3774213201 wait complete. 2024-05-15T13:49:44.350 INFO:teuthology.orchestra.run.smithi033.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-15T13:49:44.350 INFO:teuthology.orchestra.run.smithi033.stdout:Adding key to root@localhost authorized_keys... 2024-05-15T13:49:44.350 INFO:teuthology.orchestra.run.smithi033.stdout:Adding host smithi033... 2024-05-15T13:49:44.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:44 smithi033 ceph-mon[31650]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:49:45.611 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:45 smithi033 ceph-mon[31650]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi033", "addr": "172.21.15.33", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:49:46.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:46 smithi033 ceph-mon[31650]: Deploying cephadm binary to smithi033 2024-05-15T13:49:49.913 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Added host 'smithi033' with addr '172.21.15.33' 2024-05-15T13:49:49.913 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.747+0000 7ffa5477e700 1 Processor -- start 2024-05-15T13:49:49.913 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.747+0000 7ffa5477e700 1 -- start start 2024-05-15T13:49:49.913 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.748+0000 7ffa5477e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa4c107310 0x7ffa4c107730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:49.913 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.748+0000 7ffa5477e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c107d00 con 0x7ffa4c107310 2024-05-15T13:49:49.913 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.748+0000 7ffa5251a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa4c107310 0x7ffa4c107730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:49.913 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.748+0000 7ffa5251a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa4c107310 0x7ffa4c107730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46624/0 (socket says 172.21.15.33:46624) 2024-05-15T13:49:49.913 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.748+0000 7ffa5251a700 1 -- 172.21.15.33:0/4181284886 learned_addr learned my addr 172.21.15.33:0/4181284886 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:49.913 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.748+0000 7ffa5251a700 1 -- 172.21.15.33:0/4181284886 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa4c107e80 con 0x7ffa4c107310 2024-05-15T13:49:49.913 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.748+0000 7ffa5251a700 1 --2- 172.21.15.33:0/4181284886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa4c107310 0x7ffa4c107730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ffa480098a0 tx=0x7ffa48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb4075069c883ce2 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:49.913 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.748+0000 7ffa51518700 1 -- 172.21.15.33:0/4181284886 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa48004160 con 0x7ffa4c107310 2024-05-15T13:49:49.914 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.748+0000 7ffa51518700 1 -- 172.21.15.33:0/4181284886 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ffa48004300 con 0x7ffa4c107310 2024-05-15T13:49:49.914 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.748+0000 7ffa51518700 1 -- 172.21.15.33:0/4181284886 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa48011af0 con 0x7ffa4c107310 2024-05-15T13:49:49.914 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.749+0000 7ffa5477e700 1 -- 172.21.15.33:0/4181284886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa4c107310 msgr2=0x7ffa4c107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:49.914 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.749+0000 7ffa5477e700 1 --2- 172.21.15.33:0/4181284886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa4c107310 0x7ffa4c107730 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ffa480098a0 tx=0x7ffa48009bb0 comp rx=0 tx=0).stop 2024-05-15T13:49:49.914 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.749+0000 7ffa5477e700 1 -- 172.21.15.33:0/4181284886 shutdown_connections 2024-05-15T13:49:49.914 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.749+0000 7ffa5477e700 1 --2- 172.21.15.33:0/4181284886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa4c107310 0x7ffa4c107730 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:49.914 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.749+0000 7ffa5477e700 1 -- 172.21.15.33:0/4181284886 >> 172.21.15.33:0/4181284886 conn(0x7ffa4c102cb0 msgr2=0x7ffa4c1050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:49.914 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.749+0000 7ffa5477e700 1 -- 172.21.15.33:0/4181284886 shutdown_connections 2024-05-15T13:49:49.914 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.749+0000 7ffa5477e700 1 -- 172.21.15.33:0/4181284886 wait complete. 2024-05-15T13:49:49.914 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.750+0000 7ffa5477e700 1 Processor -- start 2024-05-15T13:49:49.914 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.750+0000 7ffa5477e700 1 -- start start 2024-05-15T13:49:49.914 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.750+0000 7ffa5477e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa4c107310 0x7ffa4c19d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:49.915 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.750+0000 7ffa5477e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c10b1f0 con 0x7ffa4c107310 2024-05-15T13:49:49.915 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.750+0000 7ffa5251a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa4c107310 0x7ffa4c19d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:49.915 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.750+0000 7ffa5251a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa4c107310 0x7ffa4c19d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46628/0 (socket says 172.21.15.33:46628) 2024-05-15T13:49:49.915 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.750+0000 7ffa5251a700 1 -- 172.21.15.33:0/2450650250 learned_addr learned my addr 172.21.15.33:0/2450650250 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:49.915 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.750+0000 7ffa5251a700 1 -- 172.21.15.33:0/2450650250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa4c19d5c0 con 0x7ffa4c107310 2024-05-15T13:49:49.915 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.751+0000 7ffa5251a700 1 --2- 172.21.15.33:0/2450650250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa4c107310 0x7ffa4c19d080 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ffa480117b0 tx=0x7ffa48011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:49.915 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.751+0000 7ffa437fe700 1 -- 172.21.15.33:0/2450650250 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa48003af0 con 0x7ffa4c107310 2024-05-15T13:49:49.915 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.751+0000 7ffa5477e700 1 -- 172.21.15.33:0/2450650250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa4c19d850 con 0x7ffa4c107310 2024-05-15T13:49:49.915 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.751+0000 7ffa5477e700 1 -- 172.21.15.33:0/2450650250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa4c1a0540 con 0x7ffa4c107310 2024-05-15T13:49:49.915 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.751+0000 7ffa437fe700 1 -- 172.21.15.33:0/2450650250 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ffa48020070 con 0x7ffa4c107310 2024-05-15T13:49:49.915 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.751+0000 7ffa437fe700 1 -- 172.21.15.33:0/2450650250 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa4802a060 con 0x7ffa4c107310 2024-05-15T13:49:49.915 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.752+0000 7ffa437fe700 1 -- 172.21.15.33:0/2450650250 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7ffa4801ac50 con 0x7ffa4c107310 2024-05-15T13:49:49.915 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.752+0000 7ffa5477e700 1 -- 172.21.15.33:0/2450650250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa30005360 con 0x7ffa4c107310 2024-05-15T13:49:49.915 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.752+0000 7ffa437fe700 1 --2- 172.21.15.33:0/2450650250 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7ffa3803e4c0 0x7ffa38040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:49.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.752+0000 7ffa437fe700 1 -- 172.21.15.33:0/2450650250 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ffa48053960 con 0x7ffa4c107310 2024-05-15T13:49:49.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.752+0000 7ffa51d19700 1 --2- 172.21.15.33:0/2450650250 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7ffa3803e4c0 0x7ffa38040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:49.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.753+0000 7ffa51d19700 1 --2- 172.21.15.33:0/2450650250 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7ffa3803e4c0 0x7ffa38040980 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c006fd0 tx=0x7ffa3c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:49.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.757+0000 7ffa437fe700 1 -- 172.21.15.33:0/2450650250 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffa480042a0 con 0x7ffa4c107310 2024-05-15T13:49:49.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:44.974+0000 7ffa5477e700 1 -- 172.21.15.33:0/2450650250 --> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi033", "addr": "172.21.15.33", "target": ["mon-mgr", ""]}) -- 0x7ffa30000bf0 con 0x7ffa3803e4c0 2024-05-15T13:49:49.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:49.204+0000 7ffa437fe700 1 -- 172.21.15.33:0/2450650250 <== mgr.14120 v2:172.21.15.33:6800/2680375465 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7ffa30000bf0 con 0x7ffa3803e4c0 2024-05-15T13:49:49.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:49.207+0000 7ffa5477e700 1 -- 172.21.15.33:0/2450650250 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7ffa3803e4c0 msgr2=0x7ffa38040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:49.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:49.207+0000 7ffa5477e700 1 --2- 172.21.15.33:0/2450650250 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7ffa3803e4c0 0x7ffa38040980 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c006fd0 tx=0x7ffa3c006e40 comp rx=0 tx=0).stop 2024-05-15T13:49:49.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:49.207+0000 7ffa5477e700 1 -- 172.21.15.33:0/2450650250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa4c107310 msgr2=0x7ffa4c19d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:49.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:49.207+0000 7ffa5477e700 1 --2- 172.21.15.33:0/2450650250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa4c107310 0x7ffa4c19d080 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ffa480117b0 tx=0x7ffa48011b70 comp rx=0 tx=0).stop 2024-05-15T13:49:49.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:49.207+0000 7ffa5477e700 1 -- 172.21.15.33:0/2450650250 shutdown_connections 2024-05-15T13:49:49.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:49.207+0000 7ffa5477e700 1 --2- 172.21.15.33:0/2450650250 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7ffa3803e4c0 0x7ffa38040980 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:49.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:49.207+0000 7ffa5477e700 1 --2- 172.21.15.33:0/2450650250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa4c107310 0x7ffa4c19d080 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:49.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:49.207+0000 7ffa5477e700 1 -- 172.21.15.33:0/2450650250 >> 172.21.15.33:0/2450650250 conn(0x7ffa4c102cb0 msgr2=0x7ffa4c1038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:49.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:49.207+0000 7ffa5477e700 1 -- 172.21.15.33:0/2450650250 shutdown_connections 2024-05-15T13:49:49.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:49.207+0000 7ffa5477e700 1 -- 172.21.15.33:0/2450650250 wait complete. 2024-05-15T13:49:49.917 INFO:teuthology.orchestra.run.smithi033.stdout:Deploying unmanaged mon service... 2024-05-15T13:49:50.603 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:50 smithi033 ceph-mon[31650]: from='mgr.14120 172.21.15.33:0/2424021838' entity='mgr.x' 2024-05-15T13:49:50.603 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:50 smithi033 ceph-mon[31650]: Added host smithi033 2024-05-15T13:49:50.603 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:50 smithi033 ceph-mon[31650]: from='mgr.14120 172.21.15.33:0/2424021838' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:49:51.363 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-15T13:49:51.363 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.337+0000 7f707c802700 1 Processor -- start 2024-05-15T13:49:51.364 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.337+0000 7f707c802700 1 -- start start 2024-05-15T13:49:51.364 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.338+0000 7f707c802700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70781098e0 0x7f707810bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:51.364 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.338+0000 7f707c802700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7078076e20 con 0x7f70781098e0 2024-05-15T13:49:51.364 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.338+0000 7f7076d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70781098e0 0x7f707810bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:51.364 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.338+0000 7f7076d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70781098e0 0x7f707810bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46630/0 (socket says 172.21.15.33:46630) 2024-05-15T13:49:51.364 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.338+0000 7f7076d9d700 1 -- 172.21.15.33:0/508214497 learned_addr learned my addr 172.21.15.33:0/508214497 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:51.364 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.338+0000 7f7076d9d700 1 -- 172.21.15.33:0/508214497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f707810c210 con 0x7f70781098e0 2024-05-15T13:49:51.364 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.338+0000 7f7076d9d700 1 --2- 172.21.15.33:0/508214497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70781098e0 0x7f707810bcd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f70680098a0 tx=0x7f7068009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=addbffefd5928ae7 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:51.364 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.338+0000 7f7075d9b700 1 -- 172.21.15.33:0/508214497 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7068004160 con 0x7f70781098e0 2024-05-15T13:49:51.365 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.338+0000 7f7075d9b700 1 -- 172.21.15.33:0/508214497 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7068004300 con 0x7f70781098e0 2024-05-15T13:49:51.365 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.339+0000 7f7075d9b700 1 -- 172.21.15.33:0/508214497 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7068011af0 con 0x7f70781098e0 2024-05-15T13:49:51.365 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.339+0000 7f707c802700 1 -- 172.21.15.33:0/508214497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70781098e0 msgr2=0x7f707810bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:51.365 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.339+0000 7f707c802700 1 --2- 172.21.15.33:0/508214497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70781098e0 0x7f707810bcd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f70680098a0 tx=0x7f7068009bb0 comp rx=0 tx=0).stop 2024-05-15T13:49:51.365 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.339+0000 7f707c802700 1 -- 172.21.15.33:0/508214497 shutdown_connections 2024-05-15T13:49:51.365 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.339+0000 7f707c802700 1 --2- 172.21.15.33:0/508214497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70781098e0 0x7f707810bcd0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:51.365 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.339+0000 7f707c802700 1 -- 172.21.15.33:0/508214497 >> 172.21.15.33:0/508214497 conn(0x7f7078103310 msgr2=0x7f7078105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:51.365 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.339+0000 7f707c802700 1 -- 172.21.15.33:0/508214497 shutdown_connections 2024-05-15T13:49:51.365 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.339+0000 7f707c802700 1 -- 172.21.15.33:0/508214497 wait complete. 2024-05-15T13:49:51.365 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.340+0000 7f707c802700 1 Processor -- start 2024-05-15T13:49:51.365 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.340+0000 7f707c802700 1 -- start start 2024-05-15T13:49:51.365 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.340+0000 7f707c802700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70781098e0 0x7f707819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:51.365 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.340+0000 7f707c802700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f707810d160 con 0x7f70781098e0 2024-05-15T13:49:51.366 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.340+0000 7f7076d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70781098e0 0x7f707819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:51.366 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.340+0000 7f7076d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70781098e0 0x7f707819efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46644/0 (socket says 172.21.15.33:46644) 2024-05-15T13:49:51.366 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.340+0000 7f7076d9d700 1 -- 172.21.15.33:0/2515556422 learned_addr learned my addr 172.21.15.33:0/2515556422 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:51.366 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.341+0000 7f7076d9d700 1 -- 172.21.15.33:0/2515556422 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f707819f510 con 0x7f70781098e0 2024-05-15T13:49:51.366 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.341+0000 7f7076d9d700 1 --2- 172.21.15.33:0/2515556422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70781098e0 0x7f707819efd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7068011ac0 tx=0x7f7068011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:51.366 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.341+0000 7f705ffff700 1 -- 172.21.15.33:0/2515556422 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7068003970 con 0x7f70781098e0 2024-05-15T13:49:51.366 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.341+0000 7f705ffff700 1 -- 172.21.15.33:0/2515556422 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7068003b10 con 0x7f70781098e0 2024-05-15T13:49:51.366 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.341+0000 7f705ffff700 1 -- 172.21.15.33:0/2515556422 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7068024610 con 0x7f70781098e0 2024-05-15T13:49:51.366 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.341+0000 7f707c802700 1 -- 172.21.15.33:0/2515556422 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f707819f800 con 0x7f70781098e0 2024-05-15T13:49:51.366 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.341+0000 7f707c802700 1 -- 172.21.15.33:0/2515556422 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70781a2490 con 0x7f70781098e0 2024-05-15T13:49:51.366 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.342+0000 7f705ffff700 1 -- 172.21.15.33:0/2515556422 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f7068024840 con 0x7f70781098e0 2024-05-15T13:49:51.366 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.342+0000 7f707c802700 1 -- 172.21.15.33:0/2515556422 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f707806a6a0 con 0x7f70781098e0 2024-05-15T13:49:51.367 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.342+0000 7f705ffff700 1 --2- 172.21.15.33:0/2515556422 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f706003e4c0 0x7f7060040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:51.367 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.342+0000 7f705ffff700 1 -- 172.21.15.33:0/2515556422 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7068053910 con 0x7f70781098e0 2024-05-15T13:49:51.367 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.342+0000 7f707659c700 1 --2- 172.21.15.33:0/2515556422 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f706003e4c0 0x7f7060040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:51.367 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.343+0000 7f707659c700 1 --2- 172.21.15.33:0/2515556422 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f706003e4c0 0x7f7060040980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f706c006fd0 tx=0x7f706c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:51.367 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.348+0000 7f705ffff700 1 -- 172.21.15.33:0/2515556422 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f706802c350 con 0x7f70781098e0 2024-05-15T13:49:51.367 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.599+0000 7f707c802700 1 -- 172.21.15.33:0/2515556422 --> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f70781a2660 con 0x7f706003e4c0 2024-05-15T13:49:51.367 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.604+0000 7f705ffff700 1 -- 172.21.15.33:0/2515556422 <== mgr.14120 v2:172.21.15.33:6800/2680375465 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f70781a2660 con 0x7f706003e4c0 2024-05-15T13:49:51.367 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.607+0000 7f707c802700 1 -- 172.21.15.33:0/2515556422 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f706003e4c0 msgr2=0x7f7060040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:51.367 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.607+0000 7f707c802700 1 --2- 172.21.15.33:0/2515556422 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f706003e4c0 0x7f7060040980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f706c006fd0 tx=0x7f706c006e40 comp rx=0 tx=0).stop 2024-05-15T13:49:51.367 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.607+0000 7f707c802700 1 -- 172.21.15.33:0/2515556422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70781098e0 msgr2=0x7f707819efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:51.367 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.607+0000 7f707c802700 1 --2- 172.21.15.33:0/2515556422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70781098e0 0x7f707819efd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7068011ac0 tx=0x7f7068011ba0 comp rx=0 tx=0).stop 2024-05-15T13:49:51.367 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.607+0000 7f707c802700 1 -- 172.21.15.33:0/2515556422 shutdown_connections 2024-05-15T13:49:51.368 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.607+0000 7f707c802700 1 --2- 172.21.15.33:0/2515556422 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f706003e4c0 0x7f7060040980 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:51.368 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.607+0000 7f707c802700 1 --2- 172.21.15.33:0/2515556422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70781098e0 0x7f707819efd0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:51.368 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.607+0000 7f707c802700 1 -- 172.21.15.33:0/2515556422 >> 172.21.15.33:0/2515556422 conn(0x7f7078103310 msgr2=0x7f7078103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:51.368 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.607+0000 7f707c802700 1 -- 172.21.15.33:0/2515556422 shutdown_connections 2024-05-15T13:49:51.368 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:50.607+0000 7f707c802700 1 -- 172.21.15.33:0/2515556422 wait complete. 2024-05-15T13:49:51.368 INFO:teuthology.orchestra.run.smithi033.stdout:Deploying unmanaged mgr service... 2024-05-15T13:49:51.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:51 smithi033 ceph-mon[31650]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:49:51.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:51 smithi033 ceph-mon[31650]: Saving service mon spec with placement count:5 2024-05-15T13:49:51.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:51 smithi033 ceph-mon[31650]: from='mgr.14120 172.21.15.33:0/2424021838' entity='mgr.x' 2024-05-15T13:49:53.327 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:53 smithi033 ceph-mon[31650]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:49:53.327 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:53 smithi033 ceph-mon[31650]: Saving service mgr spec with placement count:2 2024-05-15T13:49:53.327 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:53 smithi033 ceph-mon[31650]: from='mgr.14120 172.21.15.33:0/2424021838' entity='mgr.x' 2024-05-15T13:49:53.328 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:53 smithi033 ceph-mon[31650]: from='mgr.14120 172.21.15.33:0/2424021838' entity='mgr.x' 2024-05-15T13:49:53.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-15T13:49:53.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.790+0000 7f6b1b90a700 1 Processor -- start 2024-05-15T13:49:53.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.790+0000 7f6b1b90a700 1 -- start start 2024-05-15T13:49:53.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.791+0000 7f6b1b90a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b1407ee60 0x7f6b1407d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:53.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.791+0000 7f6b1b90a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1407d8e0 con 0x7f6b1407ee60 2024-05-15T13:49:53.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.791+0000 7f6b196a6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b1407ee60 0x7f6b1407d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:53.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.791+0000 7f6b196a6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b1407ee60 0x7f6b1407d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46646/0 (socket says 172.21.15.33:46646) 2024-05-15T13:49:53.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.791+0000 7f6b196a6700 1 -- 172.21.15.33:0/306937541 learned_addr learned my addr 172.21.15.33:0/306937541 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:53.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.791+0000 7f6b196a6700 1 -- 172.21.15.33:0/306937541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1407da60 con 0x7f6b1407ee60 2024-05-15T13:49:53.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.791+0000 7f6b196a6700 1 --2- 172.21.15.33:0/306937541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b1407ee60 0x7f6b1407d310 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6b100098a0 tx=0x7f6b10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad307d9f604fad72 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:53.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.792+0000 7f6b0bfff700 1 -- 172.21.15.33:0/306937541 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b10004160 con 0x7f6b1407ee60 2024-05-15T13:49:53.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.792+0000 7f6b0bfff700 1 -- 172.21.15.33:0/306937541 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6b10004300 con 0x7f6b1407ee60 2024-05-15T13:49:53.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.792+0000 7f6b0bfff700 1 -- 172.21.15.33:0/306937541 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b10011af0 con 0x7f6b1407ee60 2024-05-15T13:49:53.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.792+0000 7f6b1b90a700 1 -- 172.21.15.33:0/306937541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b1407ee60 msgr2=0x7f6b1407d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:53.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.792+0000 7f6b1b90a700 1 --2- 172.21.15.33:0/306937541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b1407ee60 0x7f6b1407d310 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6b100098a0 tx=0x7f6b10009bb0 comp rx=0 tx=0).stop 2024-05-15T13:49:53.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.792+0000 7f6b1b90a700 1 -- 172.21.15.33:0/306937541 shutdown_connections 2024-05-15T13:49:53.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.792+0000 7f6b1b90a700 1 --2- 172.21.15.33:0/306937541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b1407ee60 0x7f6b1407d310 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:53.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.792+0000 7f6b1b90a700 1 -- 172.21.15.33:0/306937541 >> 172.21.15.33:0/306937541 conn(0x7f6b14104660 msgr2=0x7f6b14106a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:53.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.792+0000 7f6b1b90a700 1 -- 172.21.15.33:0/306937541 shutdown_connections 2024-05-15T13:49:53.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.792+0000 7f6b1b90a700 1 -- 172.21.15.33:0/306937541 wait complete. 2024-05-15T13:49:53.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.793+0000 7f6b1b90a700 1 Processor -- start 2024-05-15T13:49:53.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.793+0000 7f6b1b90a700 1 -- start start 2024-05-15T13:49:53.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.793+0000 7f6b1b90a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b1407ee60 0x7f6b1419f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:53.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.793+0000 7f6b1b90a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1410f590 con 0x7f6b1407ee60 2024-05-15T13:49:53.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.793+0000 7f6b196a6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b1407ee60 0x7f6b1419f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:53.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.793+0000 7f6b196a6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b1407ee60 0x7f6b1419f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46654/0 (socket says 172.21.15.33:46654) 2024-05-15T13:49:53.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.793+0000 7f6b196a6700 1 -- 172.21.15.33:0/2108290547 learned_addr learned my addr 172.21.15.33:0/2108290547 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:53.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.794+0000 7f6b196a6700 1 -- 172.21.15.33:0/2108290547 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1419f740 con 0x7f6b1407ee60 2024-05-15T13:49:53.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.794+0000 7f6b196a6700 1 --2- 172.21.15.33:0/2108290547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b1407ee60 0x7f6b1419f200 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6b10011ac0 tx=0x7f6b10011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:53.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.794+0000 7f6b0a7fc700 1 -- 172.21.15.33:0/2108290547 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b10003970 con 0x7f6b1407ee60 2024-05-15T13:49:53.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.794+0000 7f6b0a7fc700 1 -- 172.21.15.33:0/2108290547 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6b10003b10 con 0x7f6b1407ee60 2024-05-15T13:49:53.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.794+0000 7f6b1b90a700 1 -- 172.21.15.33:0/2108290547 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b1419f9d0 con 0x7f6b1407ee60 2024-05-15T13:49:53.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.794+0000 7f6b0a7fc700 1 -- 172.21.15.33:0/2108290547 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b10024610 con 0x7f6b1407ee60 2024-05-15T13:49:53.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.794+0000 7f6b1b90a700 1 -- 172.21.15.33:0/2108290547 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b141a26c0 con 0x7f6b1407ee60 2024-05-15T13:49:53.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.795+0000 7f6b0a7fc700 1 -- 172.21.15.33:0/2108290547 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f6b1001a430 con 0x7f6b1407ee60 2024-05-15T13:49:53.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.795+0000 7f6b1b90a700 1 -- 172.21.15.33:0/2108290547 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b14064ba0 con 0x7f6b1407ee60 2024-05-15T13:49:53.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.795+0000 7f6b0a7fc700 1 --2- 172.21.15.33:0/2108290547 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f6b0003e4c0 0x7f6b00040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:53.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.795+0000 7f6b0a7fc700 1 -- 172.21.15.33:0/2108290547 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6b10053800 con 0x7f6b1407ee60 2024-05-15T13:49:53.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.795+0000 7f6b18ea5700 1 --2- 172.21.15.33:0/2108290547 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f6b0003e4c0 0x7f6b00040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:53.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.796+0000 7f6b18ea5700 1 --2- 172.21.15.33:0/2108290547 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f6b0003e4c0 0x7f6b00040980 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6b04006fd0 tx=0x7f6b04006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:53.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:51.801+0000 7f6b0a7fc700 1 -- 172.21.15.33:0/2108290547 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6b10016260 con 0x7f6b1407ee60 2024-05-15T13:49:53.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:52.034+0000 7f6b1b90a700 1 -- 172.21.15.33:0/2108290547 --> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f6b14105960 con 0x7f6b0003e4c0 2024-05-15T13:49:53.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:52.042+0000 7f6b0a7fc700 1 -- 172.21.15.33:0/2108290547 <== mgr.14120 v2:172.21.15.33:6800/2680375465 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6b14105960 con 0x7f6b0003e4c0 2024-05-15T13:49:53.339 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:52.044+0000 7f6b1b90a700 1 -- 172.21.15.33:0/2108290547 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f6b0003e4c0 msgr2=0x7f6b00040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:53.339 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:52.044+0000 7f6b1b90a700 1 --2- 172.21.15.33:0/2108290547 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f6b0003e4c0 0x7f6b00040980 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6b04006fd0 tx=0x7f6b04006e40 comp rx=0 tx=0).stop 2024-05-15T13:49:53.339 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:52.044+0000 7f6b1b90a700 1 -- 172.21.15.33:0/2108290547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b1407ee60 msgr2=0x7f6b1419f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:53.339 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:52.044+0000 7f6b1b90a700 1 --2- 172.21.15.33:0/2108290547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b1407ee60 0x7f6b1419f200 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6b10011ac0 tx=0x7f6b10011ba0 comp rx=0 tx=0).stop 2024-05-15T13:49:53.339 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:52.044+0000 7f6b1b90a700 1 -- 172.21.15.33:0/2108290547 shutdown_connections 2024-05-15T13:49:53.339 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:52.044+0000 7f6b1b90a700 1 --2- 172.21.15.33:0/2108290547 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f6b0003e4c0 0x7f6b00040980 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:53.339 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:52.044+0000 7f6b1b90a700 1 --2- 172.21.15.33:0/2108290547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b1407ee60 0x7f6b1419f200 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:53.339 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:52.044+0000 7f6b1b90a700 1 -- 172.21.15.33:0/2108290547 >> 172.21.15.33:0/2108290547 conn(0x7f6b14104660 msgr2=0x7f6b14105250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:53.339 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:52.044+0000 7f6b1b90a700 1 -- 172.21.15.33:0/2108290547 shutdown_connections 2024-05-15T13:49:53.339 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:52.044+0000 7f6b1b90a700 1 -- 172.21.15.33:0/2108290547 wait complete. 2024-05-15T13:49:54.567 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.796+0000 7f789f500700 1 Processor -- start 2024-05-15T13:49:54.567 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.796+0000 7f789f500700 1 -- start start 2024-05-15T13:49:54.567 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.796+0000 7f789f500700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7898109b20 0x7f789810bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:54.567 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.796+0000 7f789f500700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f789807ce40 con 0x7f7898109b20 2024-05-15T13:49:54.568 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.796+0000 7f789d29c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7898109b20 0x7f789810bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:54.568 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.796+0000 7f789d29c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7898109b20 0x7f789810bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59882/0 (socket says 172.21.15.33:59882) 2024-05-15T13:49:54.568 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.796+0000 7f789d29c700 1 -- 172.21.15.33:0/776396358 learned_addr learned my addr 172.21.15.33:0/776396358 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:54.568 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.796+0000 7f789d29c700 1 -- 172.21.15.33:0/776396358 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f789810c450 con 0x7f7898109b20 2024-05-15T13:49:54.568 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.797+0000 7f789d29c700 1 --2- 172.21.15.33:0/776396358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7898109b20 0x7f789810bf10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f788c0098a0 tx=0x7f788c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb242a0539f8154e server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:54.568 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.797+0000 7f788bfff700 1 -- 172.21.15.33:0/776396358 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f788c004160 con 0x7f7898109b20 2024-05-15T13:49:54.568 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.797+0000 7f788bfff700 1 -- 172.21.15.33:0/776396358 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f788c004300 con 0x7f7898109b20 2024-05-15T13:49:54.568 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.797+0000 7f788bfff700 1 -- 172.21.15.33:0/776396358 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f788c011a20 con 0x7f7898109b20 2024-05-15T13:49:54.568 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.797+0000 7f789f500700 1 -- 172.21.15.33:0/776396358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7898109b20 msgr2=0x7f789810bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:54.568 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.797+0000 7f789f500700 1 --2- 172.21.15.33:0/776396358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7898109b20 0x7f789810bf10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f788c0098a0 tx=0x7f788c009bb0 comp rx=0 tx=0).stop 2024-05-15T13:49:54.568 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.797+0000 7f789f500700 1 -- 172.21.15.33:0/776396358 shutdown_connections 2024-05-15T13:49:54.568 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.798+0000 7f789f500700 1 --2- 172.21.15.33:0/776396358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7898109b20 0x7f789810bf10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:54.568 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.798+0000 7f789f500700 1 -- 172.21.15.33:0/776396358 >> 172.21.15.33:0/776396358 conn(0x7f7898103510 msgr2=0x7f7898105930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:54.569 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.798+0000 7f789f500700 1 -- 172.21.15.33:0/776396358 shutdown_connections 2024-05-15T13:49:54.569 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.798+0000 7f789f500700 1 -- 172.21.15.33:0/776396358 wait complete. 2024-05-15T13:49:54.569 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.798+0000 7f789f500700 1 Processor -- start 2024-05-15T13:49:54.569 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.798+0000 7f789f500700 1 -- start start 2024-05-15T13:49:54.569 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.798+0000 7f789f500700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7898109b20 0x7f789819cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:54.569 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.799+0000 7f789f500700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f789810d400 con 0x7f7898109b20 2024-05-15T13:49:54.569 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.799+0000 7f789d29c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7898109b20 0x7f789819cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:54.569 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.799+0000 7f789d29c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7898109b20 0x7f789819cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59890/0 (socket says 172.21.15.33:59890) 2024-05-15T13:49:54.569 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.799+0000 7f789d29c700 1 -- 172.21.15.33:0/408511128 learned_addr learned my addr 172.21.15.33:0/408511128 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:54.569 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.799+0000 7f789d29c700 1 -- 172.21.15.33:0/408511128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f789819d4a0 con 0x7f7898109b20 2024-05-15T13:49:54.569 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.799+0000 7f789d29c700 1 --2- 172.21.15.33:0/408511128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7898109b20 0x7f789819cf60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f788c004660 tx=0x7f788c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:54.569 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.799+0000 7f788a7fc700 1 -- 172.21.15.33:0/408511128 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f788c003aa0 con 0x7f7898109b20 2024-05-15T13:49:54.569 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.799+0000 7f788a7fc700 1 -- 172.21.15.33:0/408511128 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f788c01b8f0 con 0x7f7898109b20 2024-05-15T13:49:54.569 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.799+0000 7f788a7fc700 1 -- 172.21.15.33:0/408511128 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f788c023900 con 0x7f7898109b20 2024-05-15T13:49:54.570 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.799+0000 7f789f500700 1 -- 172.21.15.33:0/408511128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f789819d730 con 0x7f7898109b20 2024-05-15T13:49:54.570 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.799+0000 7f789f500700 1 -- 172.21.15.33:0/408511128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78981a0420 con 0x7f7898109b20 2024-05-15T13:49:54.570 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.800+0000 7f788a7fc700 1 -- 172.21.15.33:0/408511128 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f788c023bc0 con 0x7f7898109b20 2024-05-15T13:49:54.570 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.800+0000 7f789f500700 1 -- 172.21.15.33:0/408511128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7898064ba0 con 0x7f7898109b20 2024-05-15T13:49:54.570 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.801+0000 7f788a7fc700 1 --2- 172.21.15.33:0/408511128 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f788403e470 0x7f7884040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:54.570 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.801+0000 7f788a7fc700 1 -- 172.21.15.33:0/408511128 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f788c053ff0 con 0x7f7898109b20 2024-05-15T13:49:54.570 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.801+0000 7f789ca9b700 1 --2- 172.21.15.33:0/408511128 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f788403e470 0x7f7884040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:54.570 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.801+0000 7f789ca9b700 1 --2- 172.21.15.33:0/408511128 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f788403e470 0x7f7884040930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7894006fd0 tx=0x7f7894006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:54.570 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:53.806+0000 7f788a7fc700 1 -- 172.21.15.33:0/408511128 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f788c01f030 con 0x7f7898109b20 2024-05-15T13:49:54.570 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:54.021+0000 7f789f500700 1 -- 172.21.15.33:0/408511128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f78981a0a20 con 0x7f7898109b20 2024-05-15T13:49:54.570 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:54.024+0000 7f788a7fc700 1 -- 172.21.15.33:0/408511128 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f788c01b440 con 0x7f7898109b20 2024-05-15T13:49:54.570 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:54.026+0000 7f789f500700 1 -- 172.21.15.33:0/408511128 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f788403e470 msgr2=0x7f7884040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:54.570 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:54.026+0000 7f789f500700 1 --2- 172.21.15.33:0/408511128 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f788403e470 0x7f7884040930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7894006fd0 tx=0x7f7894006e40 comp rx=0 tx=0).stop 2024-05-15T13:49:54.570 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:54.026+0000 7f789f500700 1 -- 172.21.15.33:0/408511128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7898109b20 msgr2=0x7f789819cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:54.571 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:54.026+0000 7f789f500700 1 --2- 172.21.15.33:0/408511128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7898109b20 0x7f789819cf60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f788c004660 tx=0x7f788c011aa0 comp rx=0 tx=0).stop 2024-05-15T13:49:54.571 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:54.027+0000 7f789f500700 1 -- 172.21.15.33:0/408511128 shutdown_connections 2024-05-15T13:49:54.571 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:54.027+0000 7f789f500700 1 --2- 172.21.15.33:0/408511128 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f788403e470 0x7f7884040930 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:54.571 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:54.027+0000 7f789f500700 1 --2- 172.21.15.33:0/408511128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7898109b20 0x7f789819cf60 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:54.571 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:54.027+0000 7f789f500700 1 -- 172.21.15.33:0/408511128 >> 172.21.15.33:0/408511128 conn(0x7f7898103510 msgr2=0x7f7898104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:54.571 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:54.027+0000 7f789f500700 1 -- 172.21.15.33:0/408511128 shutdown_connections 2024-05-15T13:49:54.571 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:54.027+0000 7f789f500700 1 -- 172.21.15.33:0/408511128 wait complete. 2024-05-15T13:49:54.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:54 smithi033 ceph-mon[31650]: from='mgr.14120 172.21.15.33:0/2424021838' entity='mgr.x' 2024-05-15T13:49:54.810 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:54 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/408511128' entity='client.admin' 2024-05-15T13:49:56.295 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.004+0000 7fdbf60a8700 1 Processor -- start 2024-05-15T13:49:56.295 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.004+0000 7fdbf60a8700 1 -- start start 2024-05-15T13:49:56.295 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.004+0000 7fdbf60a8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbf0109550 0x7fdbf0109970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:56.295 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.004+0000 7fdbf60a8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf0109f40 con 0x7fdbf0109550 2024-05-15T13:49:56.295 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.004+0000 7fdbef7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbf0109550 0x7fdbf0109970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:56.295 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.004+0000 7fdbef7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbf0109550 0x7fdbf0109970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59894/0 (socket says 172.21.15.33:59894) 2024-05-15T13:49:56.295 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.004+0000 7fdbef7fe700 1 -- 172.21.15.33:0/1780481336 learned_addr learned my addr 172.21.15.33:0/1780481336 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:56.296 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.004+0000 7fdbef7fe700 1 -- 172.21.15.33:0/1780481336 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf0108200 con 0x7fdbf0109550 2024-05-15T13:49:56.296 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.005+0000 7fdbef7fe700 1 --2- 172.21.15.33:0/1780481336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbf0109550 0x7fdbf0109970 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdbe00098a0 tx=0x7fdbe0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e95de88e832bf37 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:56.296 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.005+0000 7fdbeeffd700 1 -- 172.21.15.33:0/1780481336 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbe0004160 con 0x7fdbf0109550 2024-05-15T13:49:56.296 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.005+0000 7fdbeeffd700 1 -- 172.21.15.33:0/1780481336 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbe0004300 con 0x7fdbf0109550 2024-05-15T13:49:56.296 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.005+0000 7fdbeeffd700 1 -- 172.21.15.33:0/1780481336 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbe0011a20 con 0x7fdbf0109550 2024-05-15T13:49:56.296 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.005+0000 7fdbf60a8700 1 -- 172.21.15.33:0/1780481336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbf0109550 msgr2=0x7fdbf0109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:56.296 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.005+0000 7fdbf60a8700 1 --2- 172.21.15.33:0/1780481336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbf0109550 0x7fdbf0109970 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdbe00098a0 tx=0x7fdbe0009bb0 comp rx=0 tx=0).stop 2024-05-15T13:49:56.296 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.005+0000 7fdbf60a8700 1 -- 172.21.15.33:0/1780481336 shutdown_connections 2024-05-15T13:49:56.296 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.005+0000 7fdbf60a8700 1 --2- 172.21.15.33:0/1780481336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbf0109550 0x7fdbf0109970 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:56.296 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.005+0000 7fdbf60a8700 1 -- 172.21.15.33:0/1780481336 >> 172.21.15.33:0/1780481336 conn(0x7fdbf0102cb0 msgr2=0x7fdbf01050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:56.296 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.005+0000 7fdbf60a8700 1 -- 172.21.15.33:0/1780481336 shutdown_connections 2024-05-15T13:49:56.296 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.005+0000 7fdbf60a8700 1 -- 172.21.15.33:0/1780481336 wait complete. 2024-05-15T13:49:56.296 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.006+0000 7fdbf60a8700 1 Processor -- start 2024-05-15T13:49:56.296 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.006+0000 7fdbf60a8700 1 -- start start 2024-05-15T13:49:56.297 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.006+0000 7fdbf60a8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbf0109550 0x7fdbf0107b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:56.297 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.006+0000 7fdbf60a8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf010c5c0 con 0x7fdbf0109550 2024-05-15T13:49:56.297 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.006+0000 7fdbef7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbf0109550 0x7fdbf0107b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:56.297 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.007+0000 7fdbef7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbf0109550 0x7fdbf0107b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59910/0 (socket says 172.21.15.33:59910) 2024-05-15T13:49:56.297 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.007+0000 7fdbef7fe700 1 -- 172.21.15.33:0/4234942342 learned_addr learned my addr 172.21.15.33:0/4234942342 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:56.297 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.007+0000 7fdbef7fe700 1 -- 172.21.15.33:0/4234942342 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf019fd90 con 0x7fdbf0109550 2024-05-15T13:49:56.297 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.007+0000 7fdbef7fe700 1 --2- 172.21.15.33:0/4234942342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbf0109550 0x7fdbf0107b70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0000c00 tx=0x7fdbe0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:56.297 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.007+0000 7fdbed7fa700 1 -- 172.21.15.33:0/4234942342 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbe0003aa0 con 0x7fdbf0109550 2024-05-15T13:49:56.297 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.007+0000 7fdbed7fa700 1 -- 172.21.15.33:0/4234942342 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbe001b8f0 con 0x7fdbf0109550 2024-05-15T13:49:56.297 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.007+0000 7fdbf60a8700 1 -- 172.21.15.33:0/4234942342 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf019ff60 con 0x7fdbf0109550 2024-05-15T13:49:56.297 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.007+0000 7fdbed7fa700 1 -- 172.21.15.33:0/4234942342 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbe0023890 con 0x7fdbf0109550 2024-05-15T13:49:56.297 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.007+0000 7fdbf60a8700 1 -- 172.21.15.33:0/4234942342 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf01a0300 con 0x7fdbf0109550 2024-05-15T13:49:56.297 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.008+0000 7fdbed7fa700 1 -- 172.21.15.33:0/4234942342 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fdbe002c3f0 con 0x7fdbf0109550 2024-05-15T13:49:56.298 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.008+0000 7fdbf60a8700 1 -- 172.21.15.33:0/4234942342 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbf0064ba0 con 0x7fdbf0109550 2024-05-15T13:49:56.298 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.008+0000 7fdbed7fa700 1 --2- 172.21.15.33:0/4234942342 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7fdbdc03e4c0 0x7fdbdc040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:56.298 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.008+0000 7fdbed7fa700 1 -- 172.21.15.33:0/4234942342 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdbe0053f00 con 0x7fdbf0109550 2024-05-15T13:49:56.298 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.009+0000 7fdbe6ffd700 1 --2- 172.21.15.33:0/4234942342 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7fdbdc03e4c0 0x7fdbdc040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:56.298 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.009+0000 7fdbe6ffd700 1 --2- 172.21.15.33:0/4234942342 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7fdbdc03e4c0 0x7fdbdc040980 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8006fd0 tx=0x7fdbd8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:56.298 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.013+0000 7fdbed7fa700 1 -- 172.21.15.33:0/4234942342 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdbe001f030 con 0x7fdbf0109550 2024-05-15T13:49:56.298 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.239+0000 7fdbf60a8700 1 -- 172.21.15.33:0/4234942342 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fdbf01a0640 con 0x7fdbf0109550 2024-05-15T13:49:56.298 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.241+0000 7fdbed7fa700 1 -- 172.21.15.33:0/4234942342 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fdbf01a0640 con 0x7fdbf0109550 2024-05-15T13:49:56.298 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.244+0000 7fdbf60a8700 1 -- 172.21.15.33:0/4234942342 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7fdbdc03e4c0 msgr2=0x7fdbdc040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:56.298 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.244+0000 7fdbf60a8700 1 --2- 172.21.15.33:0/4234942342 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7fdbdc03e4c0 0x7fdbdc040980 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8006fd0 tx=0x7fdbd8006e40 comp rx=0 tx=0).stop 2024-05-15T13:49:56.298 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.244+0000 7fdbf60a8700 1 -- 172.21.15.33:0/4234942342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbf0109550 msgr2=0x7fdbf0107b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:56.298 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.244+0000 7fdbf60a8700 1 --2- 172.21.15.33:0/4234942342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbf0109550 0x7fdbf0107b70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0000c00 tx=0x7fdbe0011aa0 comp rx=0 tx=0).stop 2024-05-15T13:49:56.298 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.244+0000 7fdbf60a8700 1 -- 172.21.15.33:0/4234942342 shutdown_connections 2024-05-15T13:49:56.299 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.244+0000 7fdbf60a8700 1 --2- 172.21.15.33:0/4234942342 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7fdbdc03e4c0 0x7fdbdc040980 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:56.299 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.244+0000 7fdbf60a8700 1 --2- 172.21.15.33:0/4234942342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbf0109550 0x7fdbf0107b70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:56.299 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.244+0000 7fdbf60a8700 1 -- 172.21.15.33:0/4234942342 >> 172.21.15.33:0/4234942342 conn(0x7fdbf0102cb0 msgr2=0x7fdbf0103810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:56.299 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.244+0000 7fdbf60a8700 1 -- 172.21.15.33:0/4234942342 shutdown_connections 2024-05-15T13:49:56.299 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:55.244+0000 7fdbf60a8700 1 -- 172.21.15.33:0/4234942342 wait complete. 2024-05-15T13:49:56.299 INFO:teuthology.orchestra.run.smithi033.stdout:Enabling the dashboard module... 2024-05-15T13:49:56.534 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:56 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/4234942342' entity='client.admin' 2024-05-15T13:49:57.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:57 smithi033 ceph-mon[31650]: from='mgr.14120 172.21.15.33:0/2424021838' entity='mgr.x' 2024-05-15T13:49:57.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:57 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/1887098231' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-15T13:49:57.692 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:57 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: ignoring --setuser ceph since I am not root 2024-05-15T13:49:57.692 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:57 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: ignoring --setgroup ceph since I am not root 2024-05-15T13:49:57.692 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:57 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:57.590+0000 7fbe91ccc200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-15T13:49:58.010 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:57 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:57.792+0000 7fbe91ccc200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-15T13:49:58.011 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:57 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:57.898+0000 7fbe91ccc200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-15T13:49:58.018 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.741+0000 7f1e0d02d700 1 Processor -- start 2024-05-15T13:49:58.018 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.741+0000 7f1e0d02d700 1 -- start start 2024-05-15T13:49:58.018 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.742+0000 7f1e0d02d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e08109060 0x7f1e08109480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:58.018 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.742+0000 7f1e0d02d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1e08109a50 con 0x7f1e08109060 2024-05-15T13:49:58.018 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.742+0000 7f1e06d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e08109060 0x7f1e08109480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:58.018 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.742+0000 7f1e06d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e08109060 0x7f1e08109480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59922/0 (socket says 172.21.15.33:59922) 2024-05-15T13:49:58.018 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.742+0000 7f1e06d9d700 1 -- 172.21.15.33:0/159464505 learned_addr learned my addr 172.21.15.33:0/159464505 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:58.019 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.742+0000 7f1e06d9d700 1 -- 172.21.15.33:0/159464505 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e0810a210 con 0x7f1e08109060 2024-05-15T13:49:58.019 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.742+0000 7f1e06d9d700 1 --2- 172.21.15.33:0/159464505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e08109060 0x7f1e08109480 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1df80098a0 tx=0x7f1df8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f9a0b6bf6620079 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:58.019 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.742+0000 7f1e05d9b700 1 -- 172.21.15.33:0/159464505 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1df8004160 con 0x7f1e08109060 2024-05-15T13:49:58.019 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.743+0000 7f1e05d9b700 1 -- 172.21.15.33:0/159464505 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1df8004300 con 0x7f1e08109060 2024-05-15T13:49:58.019 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.743+0000 7f1e05d9b700 1 -- 172.21.15.33:0/159464505 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1df8011a20 con 0x7f1e08109060 2024-05-15T13:49:58.019 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.743+0000 7f1e0d02d700 1 -- 172.21.15.33:0/159464505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e08109060 msgr2=0x7f1e08109480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:58.019 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.743+0000 7f1e0d02d700 1 --2- 172.21.15.33:0/159464505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e08109060 0x7f1e08109480 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1df80098a0 tx=0x7f1df8009bb0 comp rx=0 tx=0).stop 2024-05-15T13:49:58.019 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.743+0000 7f1e0d02d700 1 -- 172.21.15.33:0/159464505 shutdown_connections 2024-05-15T13:49:58.019 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.743+0000 7f1e0d02d700 1 --2- 172.21.15.33:0/159464505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e08109060 0x7f1e08109480 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:58.019 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.743+0000 7f1e0d02d700 1 -- 172.21.15.33:0/159464505 >> 172.21.15.33:0/159464505 conn(0x7f1e08104660 msgr2=0x7f1e08106a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:58.019 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.743+0000 7f1e0d02d700 1 -- 172.21.15.33:0/159464505 shutdown_connections 2024-05-15T13:49:58.019 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.743+0000 7f1e0d02d700 1 -- 172.21.15.33:0/159464505 wait complete. 2024-05-15T13:49:58.019 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.744+0000 7f1e0d02d700 1 Processor -- start 2024-05-15T13:49:58.020 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.744+0000 7f1e0d02d700 1 -- start start 2024-05-15T13:49:58.020 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.745+0000 7f1e0d02d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e08109060 0x7f1e0819cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:58.020 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.745+0000 7f1e0d02d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0810d5c0 con 0x7f1e08109060 2024-05-15T13:49:58.020 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.745+0000 7f1e06d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e08109060 0x7f1e0819cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:58.020 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.745+0000 7f1e06d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e08109060 0x7f1e0819cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59938/0 (socket says 172.21.15.33:59938) 2024-05-15T13:49:58.020 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.745+0000 7f1e06d9d700 1 -- 172.21.15.33:0/1887098231 learned_addr learned my addr 172.21.15.33:0/1887098231 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:58.020 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.745+0000 7f1e06d9d700 1 -- 172.21.15.33:0/1887098231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e0819d4e0 con 0x7f1e08109060 2024-05-15T13:49:58.020 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.745+0000 7f1e06d9d700 1 --2- 172.21.15.33:0/1887098231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e08109060 0x7f1e0819cfa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1df80119f0 tx=0x7f1df8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:58.020 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.745+0000 7f1dfffff700 1 -- 172.21.15.33:0/1887098231 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1df80039e0 con 0x7f1e08109060 2024-05-15T13:49:58.020 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.745+0000 7f1dfffff700 1 -- 172.21.15.33:0/1887098231 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1df801b8f0 con 0x7f1e08109060 2024-05-15T13:49:58.020 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.745+0000 7f1e0d02d700 1 -- 172.21.15.33:0/1887098231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e0819d770 con 0x7f1e08109060 2024-05-15T13:49:58.020 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.745+0000 7f1dfffff700 1 -- 172.21.15.33:0/1887098231 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1df80237a0 con 0x7f1e08109060 2024-05-15T13:49:58.020 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.746+0000 7f1e0d02d700 1 -- 172.21.15.33:0/1887098231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e081a0460 con 0x7f1e08109060 2024-05-15T13:49:58.020 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.746+0000 7f1dfffff700 1 -- 172.21.15.33:0/1887098231 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f1df8023a50 con 0x7f1e08109060 2024-05-15T13:49:58.021 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.747+0000 7f1dfffff700 1 --2- 172.21.15.33:0/1887098231 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f1df403e4d0 0x7f1df4040990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:58.021 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.747+0000 7f1dfffff700 1 -- 172.21.15.33:0/1887098231 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1df8054e00 con 0x7f1e08109060 2024-05-15T13:49:58.021 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.747+0000 7f1e0d02d700 1 -- 172.21.15.33:0/1887098231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e08064ba0 con 0x7f1e08109060 2024-05-15T13:49:58.021 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.747+0000 7f1e0659c700 1 --2- 172.21.15.33:0/1887098231 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f1df403e4d0 0x7f1df4040990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:58.021 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.747+0000 7f1e0659c700 1 --2- 172.21.15.33:0/1887098231 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f1df403e4d0 0x7f1df4040990 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1df0006fd0 tx=0x7f1df0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:58.021 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:56.752+0000 7f1dfffff700 1 -- 172.21.15.33:0/1887098231 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1df801f030 con 0x7f1e08109060 2024-05-15T13:49:58.021 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:57.046+0000 7f1e0d02d700 1 -- 172.21.15.33:0/1887098231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f1e081a0980 con 0x7f1e08109060 2024-05-15T13:49:58.021 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:57.383+0000 7f1dfffff700 1 -- 172.21.15.33:0/1887098231 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f1df800b3e0 con 0x7f1e08109060 2024-05-15T13:49:58.021 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:57.385+0000 7f1dfffff700 1 -- 172.21.15.33:0/1887098231 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f1df802c3f0 con 0x7f1e08109060 2024-05-15T13:49:58.021 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:57.387+0000 7f1e0d02d700 1 -- 172.21.15.33:0/1887098231 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f1df403e4d0 msgr2=0x7f1df4040990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:58.021 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:57.387+0000 7f1e0d02d700 1 --2- 172.21.15.33:0/1887098231 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f1df403e4d0 0x7f1df4040990 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1df0006fd0 tx=0x7f1df0006e40 comp rx=0 tx=0).stop 2024-05-15T13:49:58.021 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:57.387+0000 7f1e0d02d700 1 -- 172.21.15.33:0/1887098231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e08109060 msgr2=0x7f1e0819cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:58.021 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:57.387+0000 7f1e0d02d700 1 --2- 172.21.15.33:0/1887098231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e08109060 0x7f1e0819cfa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1df80119f0 tx=0x7f1df8011ad0 comp rx=0 tx=0).stop 2024-05-15T13:49:58.022 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:57.387+0000 7f1e0d02d700 1 -- 172.21.15.33:0/1887098231 shutdown_connections 2024-05-15T13:49:58.022 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:57.387+0000 7f1e0d02d700 1 --2- 172.21.15.33:0/1887098231 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f1df403e4d0 0x7f1df4040990 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:58.022 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:57.387+0000 7f1e0d02d700 1 --2- 172.21.15.33:0/1887098231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e08109060 0x7f1e0819cfa0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:58.022 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:57.387+0000 7f1e0d02d700 1 -- 172.21.15.33:0/1887098231 >> 172.21.15.33:0/1887098231 conn(0x7f1e08104660 msgr2=0x7f1e081051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:58.022 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:57.387+0000 7f1e0d02d700 1 -- 172.21.15.33:0/1887098231 shutdown_connections 2024-05-15T13:49:58.022 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:57.387+0000 7f1e0d02d700 1 -- 172.21.15.33:0/1887098231 wait complete. 2024-05-15T13:49:58.260 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:58 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:58.227+0000 7fbe91ccc200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-15T13:49:58.667 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:58 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:58.320+0000 7fbe91ccc200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-15T13:49:58.668 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:58 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/1887098231' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-15T13:49:58.668 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:58 smithi033 ceph-mon[31650]: mgrmap e9: x(active, since 19s) 2024-05-15T13:49:59.334 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-05-15T13:49:59.334 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-15T13:49:59.334 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-15T13:49:59.334 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-15T13:49:59.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-15T13:49:59.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-05-15T13:49:59.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.458+0000 7f3beaf24700 1 Processor -- start 2024-05-15T13:49:59.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.458+0000 7f3beaf24700 1 -- start start 2024-05-15T13:49:59.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.458+0000 7f3beaf24700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3be4107960 0x7f3be4107d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:59.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.458+0000 7f3beaf24700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3be4076da0 con 0x7f3be4107960 2024-05-15T13:49:59.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.458+0000 7f3be8cc0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3be4107960 0x7f3be4107d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:59.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.458+0000 7f3be8cc0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3be4107960 0x7f3be4107d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59964/0 (socket says 172.21.15.33:59964) 2024-05-15T13:49:59.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.458+0000 7f3be8cc0700 1 -- 172.21.15.33:0/1326154176 learned_addr learned my addr 172.21.15.33:0/1326154176 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:59.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.458+0000 7f3be8cc0700 1 -- 172.21.15.33:0/1326154176 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be41082c0 con 0x7f3be4107960 2024-05-15T13:49:59.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.459+0000 7f3be8cc0700 1 --2- 172.21.15.33:0/1326154176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3be4107960 0x7f3be4107d80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3bd80098a0 tx=0x7f3bd8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67dcf1081570ad5 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:59.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.459+0000 7f3be37fe700 1 -- 172.21.15.33:0/1326154176 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bd8004160 con 0x7f3be4107960 2024-05-15T13:49:59.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.459+0000 7f3be37fe700 1 -- 172.21.15.33:0/1326154176 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bd8004300 con 0x7f3be4107960 2024-05-15T13:49:59.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.459+0000 7f3be37fe700 1 -- 172.21.15.33:0/1326154176 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bd8011a20 con 0x7f3be4107960 2024-05-15T13:49:59.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.460+0000 7f3beaf24700 1 -- 172.21.15.33:0/1326154176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3be4107960 msgr2=0x7f3be4107d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:59.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.460+0000 7f3beaf24700 1 --2- 172.21.15.33:0/1326154176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3be4107960 0x7f3be4107d80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3bd80098a0 tx=0x7f3bd8009bb0 comp rx=0 tx=0).stop 2024-05-15T13:49:59.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.460+0000 7f3beaf24700 1 -- 172.21.15.33:0/1326154176 shutdown_connections 2024-05-15T13:49:59.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.460+0000 7f3beaf24700 1 --2- 172.21.15.33:0/1326154176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3be4107960 0x7f3be4107d80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:59.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.460+0000 7f3beaf24700 1 -- 172.21.15.33:0/1326154176 >> 172.21.15.33:0/1326154176 conn(0x7f3be41034e0 msgr2=0x7f3be4105940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:59.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.460+0000 7f3beaf24700 1 -- 172.21.15.33:0/1326154176 shutdown_connections 2024-05-15T13:49:59.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.460+0000 7f3beaf24700 1 -- 172.21.15.33:0/1326154176 wait complete. 2024-05-15T13:49:59.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.460+0000 7f3beaf24700 1 Processor -- start 2024-05-15T13:49:59.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.460+0000 7f3beaf24700 1 -- start start 2024-05-15T13:49:59.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.461+0000 7f3beaf24700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3be4107960 0x7f3be419f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:59.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.461+0000 7f3beaf24700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3be4109330 con 0x7f3be4107960 2024-05-15T13:49:59.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.461+0000 7f3be8cc0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3be4107960 0x7f3be419f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:49:59.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.461+0000 7f3be8cc0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3be4107960 0x7f3be419f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59968/0 (socket says 172.21.15.33:59968) 2024-05-15T13:49:59.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.461+0000 7f3be8cc0700 1 -- 172.21.15.33:0/283102627 learned_addr learned my addr 172.21.15.33:0/283102627 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:49:59.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.461+0000 7f3be8cc0700 1 -- 172.21.15.33:0/283102627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be419f6d0 con 0x7f3be4107960 2024-05-15T13:49:59.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.461+0000 7f3be8cc0700 1 --2- 172.21.15.33:0/283102627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3be4107960 0x7f3be419f190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3bd80116e0 tx=0x7f3bd8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:49:59.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.461+0000 7f3be1ffb700 1 -- 172.21.15.33:0/283102627 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bd8003b20 con 0x7f3be4107960 2024-05-15T13:49:59.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.461+0000 7f3be1ffb700 1 -- 172.21.15.33:0/283102627 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bd801b8f0 con 0x7f3be4107960 2024-05-15T13:49:59.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.461+0000 7f3beaf24700 1 -- 172.21.15.33:0/283102627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3be419f960 con 0x7f3be4107960 2024-05-15T13:49:59.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.462+0000 7f3be1ffb700 1 -- 172.21.15.33:0/283102627 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bd80238c0 con 0x7f3be4107960 2024-05-15T13:49:59.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.462+0000 7f3beaf24700 1 -- 172.21.15.33:0/283102627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3be41a2650 con 0x7f3be4107960 2024-05-15T13:49:59.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.463+0000 7f3be1ffb700 1 -- 172.21.15.33:0/283102627 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f3bd8023b70 con 0x7f3be4107960 2024-05-15T13:49:59.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.463+0000 7f3be1ffb700 1 --2- 172.21.15.33:0/283102627 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f3bd403e470 0x7f3bd4040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:49:59.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.463+0000 7f3be1ffb700 1 -- 172.21.15.33:0/283102627 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3bd8054020 con 0x7f3be4107960 2024-05-15T13:49:59.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.463+0000 7f3be3fff700 1 -- 172.21.15.33:0/283102627 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f3bd403e470 msgr2=0x7f3bd4040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2680375465 2024-05-15T13:49:59.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.463+0000 7f3be3fff700 1 --2- 172.21.15.33:0/283102627 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f3bd403e470 0x7f3bd4040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-15T13:49:59.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.463+0000 7f3beaf24700 1 -- 172.21.15.33:0/283102627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3be4064ba0 con 0x7f3be4107960 2024-05-15T13:49:59.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.468+0000 7f3be1ffb700 1 -- 172.21.15.33:0/283102627 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3bd801f030 con 0x7f3be4107960 2024-05-15T13:49:59.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.664+0000 7f3be3fff700 1 -- 172.21.15.33:0/283102627 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f3bd403e470 msgr2=0x7f3bd4040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2680375465 2024-05-15T13:49:59.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.664+0000 7f3be3fff700 1 --2- 172.21.15.33:0/283102627 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f3bd403e470 0x7f3bd4040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-15T13:49:59.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.792+0000 7f3beaf24700 1 -- 172.21.15.33:0/283102627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f3be402e6a0 con 0x7f3be4107960 2024-05-15T13:49:59.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.793+0000 7f3be1ffb700 1 -- 172.21.15.33:0/283102627 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f3bd801baa0 con 0x7f3be4107960 2024-05-15T13:49:59.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.795+0000 7f3beaf24700 1 -- 172.21.15.33:0/283102627 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f3bd403e470 msgr2=0x7f3bd4040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-15T13:49:59.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.795+0000 7f3beaf24700 1 --2- 172.21.15.33:0/283102627 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f3bd403e470 0x7f3bd4040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:59.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.795+0000 7f3beaf24700 1 -- 172.21.15.33:0/283102627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3be4107960 msgr2=0x7f3be419f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:49:59.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.795+0000 7f3beaf24700 1 --2- 172.21.15.33:0/283102627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3be4107960 0x7f3be419f190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3bd80116e0 tx=0x7f3bd8011aa0 comp rx=0 tx=0).stop 2024-05-15T13:49:59.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.795+0000 7f3beaf24700 1 -- 172.21.15.33:0/283102627 shutdown_connections 2024-05-15T13:49:59.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.795+0000 7f3beaf24700 1 --2- 172.21.15.33:0/283102627 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f3bd403e470 0x7f3bd4040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:59.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.795+0000 7f3beaf24700 1 --2- 172.21.15.33:0/283102627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3be4107960 0x7f3be419f190 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:49:59.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.795+0000 7f3beaf24700 1 -- 172.21.15.33:0/283102627 >> 172.21.15.33:0/283102627 conn(0x7f3be41034e0 msgr2=0x7f3be4195dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:49:59.339 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.796+0000 7f3beaf24700 1 -- 172.21.15.33:0/283102627 shutdown_connections 2024-05-15T13:49:59.339 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:58.796+0000 7f3beaf24700 1 -- 172.21.15.33:0/283102627 wait complete. 2024-05-15T13:49:59.339 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for the mgr to restart... 2024-05-15T13:49:59.339 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr epoch 9... 2024-05-15T13:49:59.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:49:59 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/283102627' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-15T13:49:59.780 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:59 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:59.594+0000 7fbe91ccc200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-15T13:49:59.780 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:59 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:59.776+0000 7fbe91ccc200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-15T13:50:00.152 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:49:59 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:49:59.955+0000 7fbe91ccc200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-15T13:50:00.407 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:50:00 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:50:00.262+0000 7fbe91ccc200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-15T13:50:00.407 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:50:00 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:50:00.359+0000 7fbe91ccc200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-15T13:50:00.691 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:50:00 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:50:00.572+0000 7fbe91ccc200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-15T13:50:01.191 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:50:00 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:50:00.774+0000 7fbe91ccc200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-15T13:50:01.476 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:50:01 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:50:01.411+0000 7fbe91ccc200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-15T13:50:01.942 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:50:01 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:50:01.653+0000 7fbe91ccc200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-15T13:50:02.441 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:50:02 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:50:02.110+0000 7fbe91ccc200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-15T13:50:02.442 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:50:02 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:50:02.213+0000 7fbe91ccc200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-15T13:50:02.809 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:50:02 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:50:02.604+0000 7fbe91ccc200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-15T13:50:03.082 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:50:02 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:50:02.981+0000 7fbe91ccc200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-15T13:50:03.082 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:50:03 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:50:03.079+0000 7fbe91ccc200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-15T13:50:03.691 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:50:03 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:50:03.335+0000 7fbe91ccc200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-15T13:50:03.692 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:50:03 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:50:03.476+0000 7fbe91ccc200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-15T13:50:03.692 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:50:03 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:50:03.577+0000 7fbe91ccc200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-15T13:50:04.191 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:50:03 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:50:03.917+0000 7fbe91ccc200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-15T13:50:04.192 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:50:04 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:50:04.014+0000 7fbe91ccc200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-15T13:50:04.936 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:50:04 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:50:04.649+0000 7fbe91ccc200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-15T13:50:04.936 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:50:04 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:50:04.878+0000 7fbe91ccc200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-15T13:50:05.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:04 smithi033 ceph-mon[31650]: Active manager daemon x restarted 2024-05-15T13:50:05.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:04 smithi033 ceph-mon[31650]: Activating manager daemon x 2024-05-15T13:50:05.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:04 smithi033 ceph-mon[31650]: osdmap e3: 0 total, 0 up, 0 in 2024-05-15T13:50:05.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:04 smithi033 ceph-mon[31650]: mgrmap e10: x(active, starting, since 0.00421611s) 2024-05-15T13:50:05.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:04 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-15T13:50:05.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:04 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-15T13:50:05.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:04 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-15T13:50:05.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:04 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-15T13:50:05.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:04 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-15T13:50:05.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:04 smithi033 ceph-mon[31650]: Manager daemon x is now available 2024-05-15T13:50:05.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:04 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:50:06.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:05 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-15T13:50:06.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:05 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-15T13:50:06.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:05 smithi033 ceph-mon[31650]: mgrmap e11: x(active, since 1.00663s) 2024-05-15T13:50:06.879 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-05-15T13:50:06.880 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-15T13:50:06.880 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-15T13:50:06.880 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-05-15T13:50:06.880 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.796+0000 7f5d01468700 1 Processor -- start 2024-05-15T13:50:06.880 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.796+0000 7f5d01468700 1 -- start start 2024-05-15T13:50:06.880 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.796+0000 7f5d01468700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cfc10a680 0x7f5cfc10aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:06.880 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.796+0000 7f5d01468700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5cfc10b070 con 0x7f5cfc10a680 2024-05-15T13:50:06.880 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.796+0000 7f5cfaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cfc10a680 0x7f5cfc10aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:06.880 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.796+0000 7f5cfaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cfc10a680 0x7f5cfc10aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59984/0 (socket says 172.21.15.33:59984) 2024-05-15T13:50:06.880 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.796+0000 7f5cfaffd700 1 -- 172.21.15.33:0/1192090832 learned_addr learned my addr 172.21.15.33:0/1192090832 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:50:06.880 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.796+0000 7f5cfaffd700 1 -- 172.21.15.33:0/1192090832 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cfc10b1f0 con 0x7f5cfc10a680 2024-05-15T13:50:06.880 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.797+0000 7f5cfaffd700 1 --2- 172.21.15.33:0/1192090832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cfc10a680 0x7f5cfc10aaa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0098a0 tx=0x7f5cec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be89bb770ec127ae server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:06.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.797+0000 7f5cf9ffb700 1 -- 172.21.15.33:0/1192090832 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cec004160 con 0x7f5cfc10a680 2024-05-15T13:50:06.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.797+0000 7f5cf9ffb700 1 -- 172.21.15.33:0/1192090832 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cec004300 con 0x7f5cfc10a680 2024-05-15T13:50:06.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.797+0000 7f5cf9ffb700 1 -- 172.21.15.33:0/1192090832 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cec011a20 con 0x7f5cfc10a680 2024-05-15T13:50:06.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.797+0000 7f5d01468700 1 -- 172.21.15.33:0/1192090832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cfc10a680 msgr2=0x7f5cfc10aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:06.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.797+0000 7f5d01468700 1 --2- 172.21.15.33:0/1192090832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cfc10a680 0x7f5cfc10aaa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0098a0 tx=0x7f5cec009bb0 comp rx=0 tx=0).stop 2024-05-15T13:50:06.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.797+0000 7f5d01468700 1 -- 172.21.15.33:0/1192090832 shutdown_connections 2024-05-15T13:50:06.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.798+0000 7f5d01468700 1 --2- 172.21.15.33:0/1192090832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cfc10a680 0x7f5cfc10aaa0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:06.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.798+0000 7f5d01468700 1 -- 172.21.15.33:0/1192090832 >> 172.21.15.33:0/1192090832 conn(0x7f5cfc102d60 msgr2=0x7f5cfc105180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:50:06.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.798+0000 7f5d01468700 1 -- 172.21.15.33:0/1192090832 shutdown_connections 2024-05-15T13:50:06.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.798+0000 7f5d01468700 1 -- 172.21.15.33:0/1192090832 wait complete. 2024-05-15T13:50:06.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.798+0000 7f5d01468700 1 Processor -- start 2024-05-15T13:50:06.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.798+0000 7f5d01468700 1 -- start start 2024-05-15T13:50:06.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.799+0000 7f5d01468700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cfc10a680 0x7f5cfc19cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:06.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.799+0000 7f5d01468700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5cfc107f40 con 0x7f5cfc10a680 2024-05-15T13:50:06.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.799+0000 7f5cfaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cfc10a680 0x7f5cfc19cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:06.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.799+0000 7f5cfaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cfc10a680 0x7f5cfc19cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60000/0 (socket says 172.21.15.33:60000) 2024-05-15T13:50:06.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.799+0000 7f5cfaffd700 1 -- 172.21.15.33:0/1429093029 learned_addr learned my addr 172.21.15.33:0/1429093029 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:50:06.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.799+0000 7f5cfaffd700 1 -- 172.21.15.33:0/1429093029 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cfc19d470 con 0x7f5cfc10a680 2024-05-15T13:50:06.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.799+0000 7f5cfaffd700 1 --2- 172.21.15.33:0/1429093029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cfc10a680 0x7f5cfc19cf30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0119f0 tx=0x7f5cec011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:06.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.799+0000 7f5ce3fff700 1 -- 172.21.15.33:0/1429093029 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cec0039e0 con 0x7f5cfc10a680 2024-05-15T13:50:06.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.799+0000 7f5d01468700 1 -- 172.21.15.33:0/1429093029 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cfc19d760 con 0x7f5cfc10a680 2024-05-15T13:50:06.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.799+0000 7f5ce3fff700 1 -- 172.21.15.33:0/1429093029 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cec01b8f0 con 0x7f5cfc10a680 2024-05-15T13:50:06.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.799+0000 7f5ce3fff700 1 -- 172.21.15.33:0/1429093029 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cec0237a0 con 0x7f5cfc10a680 2024-05-15T13:50:06.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.799+0000 7f5d01468700 1 -- 172.21.15.33:0/1429093029 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cfc1a03f0 con 0x7f5cfc10a680 2024-05-15T13:50:06.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.800+0000 7f5ce3fff700 1 -- 172.21.15.33:0/1429093029 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f5cec023aa0 con 0x7f5cfc10a680 2024-05-15T13:50:06.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.801+0000 7f5ce3fff700 1 --2- 172.21.15.33:0/1429093029 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f5ce403e470 0x7f5ce4040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:06.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.801+0000 7f5ce3fff700 1 -- 172.21.15.33:0/1429093029 --> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5ce4041040 con 0x7f5ce403e470 2024-05-15T13:50:06.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.801+0000 7f5ce3fff700 1 -- 172.21.15.33:0/1429093029 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5cec054080 con 0x7f5cfc10a680 2024-05-15T13:50:06.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.801+0000 7f5cfa7fc700 1 -- 172.21.15.33:0/1429093029 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f5ce403e470 msgr2=0x7f5ce4040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2680375465 2024-05-15T13:50:06.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:49:59.801+0000 7f5cfa7fc700 1 --2- 172.21.15.33:0/1429093029 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f5ce403e470 0x7f5ce4040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-15T13:50:06.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:00.002+0000 7f5cfa7fc700 1 -- 172.21.15.33:0/1429093029 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f5ce403e470 msgr2=0x7f5ce4040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2680375465 2024-05-15T13:50:06.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:00.002+0000 7f5cfa7fc700 1 --2- 172.21.15.33:0/1429093029 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f5ce403e470 0x7f5ce4040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-15T13:50:06.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:00.403+0000 7f5cfa7fc700 1 -- 172.21.15.33:0/1429093029 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f5ce403e470 msgr2=0x7f5ce4040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2680375465 2024-05-15T13:50:06.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:00.403+0000 7f5cfa7fc700 1 --2- 172.21.15.33:0/1429093029 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f5ce403e470 0x7f5ce4040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-15T13:50:06.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:01.204+0000 7f5cfa7fc700 1 -- 172.21.15.33:0/1429093029 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f5ce403e470 msgr2=0x7f5ce4040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2680375465 2024-05-15T13:50:06.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:01.204+0000 7f5cfa7fc700 1 --2- 172.21.15.33:0/1429093029 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f5ce403e470 0x7f5ce4040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-15T13:50:06.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:02.805+0000 7f5cfa7fc700 1 -- 172.21.15.33:0/1429093029 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f5ce403e470 msgr2=0x7f5ce4040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2680375465 2024-05-15T13:50:06.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:02.805+0000 7f5cfa7fc700 1 --2- 172.21.15.33:0/1429093029 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f5ce403e470 0x7f5ce4040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-15T13:50:06.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:04.883+0000 7f5ce3fff700 1 -- 172.21.15.33:0/1429093029 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7f5cec02b350 con 0x7f5cfc10a680 2024-05-15T13:50:06.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:04.883+0000 7f5ce3fff700 1 -- 172.21.15.33:0/1429093029 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f5ce403e470 msgr2=0x7f5ce4040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-15T13:50:06.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:04.883+0000 7f5ce3fff700 1 --2- 172.21.15.33:0/1429093029 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f5ce403e470 0x7f5ce4040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:06.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:05.884+0000 7f5ce3fff700 1 -- 172.21.15.33:0/1429093029 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f5cec024d80 con 0x7f5cfc10a680 2024-05-15T13:50:06.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:05.884+0000 7f5ce3fff700 1 --2- 172.21.15.33:0/1429093029 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f5ce4041ef0 0x7f5ce40442e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:06.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:05.885+0000 7f5ce3fff700 1 -- 172.21.15.33:0/1429093029 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5cec008e50 con 0x7f5ce4041ef0 2024-05-15T13:50:06.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:05.885+0000 7f5cfa7fc700 1 --2- 172.21.15.33:0/1429093029 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f5ce4041ef0 0x7f5ce40442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:06.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:05.886+0000 7f5cfa7fc700 1 --2- 172.21.15.33:0/1429093029 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f5ce4041ef0 0x7f5ce40442e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0003a10 tx=0x7f5cf00092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:06.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:05.886+0000 7f5ce3fff700 1 -- 172.21.15.33:0/1429093029 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f5cec008e50 con 0x7f5ce4041ef0 2024-05-15T13:50:06.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:05.895+0000 7f5d01468700 1 -- 172.21.15.33:0/1429093029 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f5cfc10d650 con 0x7f5ce4041ef0 2024-05-15T13:50:06.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:05.895+0000 7f5ce3fff700 1 -- 172.21.15.33:0/1429093029 <== mgr.14152 v2:172.21.15.33:6800/1797827152 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f5cfc10d650 con 0x7f5ce4041ef0 2024-05-15T13:50:06.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:05.895+0000 7f5d01468700 1 -- 172.21.15.33:0/1429093029 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f5ce4041ef0 msgr2=0x7f5ce40442e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:06.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:05.895+0000 7f5d01468700 1 --2- 172.21.15.33:0/1429093029 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f5ce4041ef0 0x7f5ce40442e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0003a10 tx=0x7f5cf00092b0 comp rx=0 tx=0).stop 2024-05-15T13:50:06.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:05.895+0000 7f5d01468700 1 -- 172.21.15.33:0/1429093029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cfc10a680 msgr2=0x7f5cfc19cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:06.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:05.895+0000 7f5d01468700 1 --2- 172.21.15.33:0/1429093029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cfc10a680 0x7f5cfc19cf30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0119f0 tx=0x7f5cec011ad0 comp rx=0 tx=0).stop 2024-05-15T13:50:06.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:05.895+0000 7f5d01468700 1 -- 172.21.15.33:0/1429093029 shutdown_connections 2024-05-15T13:50:06.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:05.895+0000 7f5d01468700 1 --2- 172.21.15.33:0/1429093029 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f5ce4041ef0 0x7f5ce40442e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:06.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:05.895+0000 7f5d01468700 1 --2- 172.21.15.33:0/1429093029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cfc10a680 0x7f5cfc19cf30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:06.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:05.895+0000 7f5d01468700 1 --2- 172.21.15.33:0/1429093029 >> [v2:172.21.15.33:6800/2680375465,v1:172.21.15.33:6801/2680375465] conn(0x7f5ce403e470 0x7f5ce4040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:06.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:05.895+0000 7f5d01468700 1 -- 172.21.15.33:0/1429093029 >> 172.21.15.33:0/1429093029 conn(0x7f5cfc102d60 msgr2=0x7f5cfc103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:50:06.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:05.895+0000 7f5d01468700 1 -- 172.21.15.33:0/1429093029 shutdown_connections 2024-05-15T13:50:06.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:05.895+0000 7f5d01468700 1 -- 172.21.15.33:0/1429093029 wait complete. 2024-05-15T13:50:06.885 INFO:teuthology.orchestra.run.smithi033.stdout:mgr epoch 9 is available 2024-05-15T13:50:06.885 INFO:teuthology.orchestra.run.smithi033.stdout:Generating a dashboard self-signed certificate... 2024-05-15T13:50:07.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:06 smithi033 ceph-mon[31650]: [15/May/2024:13:50:05] ENGINE Bus STARTING 2024-05-15T13:50:07.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:06 smithi033 ceph-mon[31650]: [15/May/2024:13:50:05] ENGINE Serving on http://172.21.15.33:8765 2024-05-15T13:50:07.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:06 smithi033 ceph-mon[31650]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-15T13:50:07.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:06 smithi033 ceph-mon[31650]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-15T13:50:07.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:06 smithi033 ceph-mon[31650]: [15/May/2024:13:50:05] ENGINE Serving on https://172.21.15.33:7150 2024-05-15T13:50:07.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:06 smithi033 ceph-mon[31650]: [15/May/2024:13:50:05] ENGINE Bus STARTED 2024-05-15T13:50:07.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:06 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:08.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-15T13:50:08.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.297+0000 7fbe107ad700 1 Processor -- start 2024-05-15T13:50:08.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.297+0000 7fbe107ad700 1 -- start start 2024-05-15T13:50:08.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.298+0000 7fbe107ad700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08109080 0x7fbe081094a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:08.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.298+0000 7fbe107ad700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbe081099e0 con 0x7fbe08109080 2024-05-15T13:50:08.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.298+0000 7fbe0e549700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08109080 0x7fbe081094a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:08.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.298+0000 7fbe0e549700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08109080 0x7fbe081094a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54834/0 (socket says 172.21.15.33:54834) 2024-05-15T13:50:08.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.298+0000 7fbe0e549700 1 -- 172.21.15.33:0/2708680177 learned_addr learned my addr 172.21.15.33:0/2708680177 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:50:08.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.298+0000 7fbe0e549700 1 -- 172.21.15.33:0/2708680177 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe08109b60 con 0x7fbe08109080 2024-05-15T13:50:08.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.298+0000 7fbe0e549700 1 --2- 172.21.15.33:0/2708680177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08109080 0x7fbe081094a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0098a0 tx=0x7fbdfc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3638c07cb725e1b server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:08.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.298+0000 7fbe0d547700 1 -- 172.21.15.33:0/2708680177 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbdfc004160 con 0x7fbe08109080 2024-05-15T13:50:08.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.298+0000 7fbe0d547700 1 -- 172.21.15.33:0/2708680177 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdfc004300 con 0x7fbe08109080 2024-05-15T13:50:08.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.298+0000 7fbe0d547700 1 -- 172.21.15.33:0/2708680177 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbdfc011a20 con 0x7fbe08109080 2024-05-15T13:50:08.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.299+0000 7fbe107ad700 1 -- 172.21.15.33:0/2708680177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08109080 msgr2=0x7fbe081094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:08.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.299+0000 7fbe107ad700 1 --2- 172.21.15.33:0/2708680177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08109080 0x7fbe081094a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0098a0 tx=0x7fbdfc009bb0 comp rx=0 tx=0).stop 2024-05-15T13:50:08.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.299+0000 7fbe107ad700 1 -- 172.21.15.33:0/2708680177 shutdown_connections 2024-05-15T13:50:08.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.299+0000 7fbe107ad700 1 --2- 172.21.15.33:0/2708680177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08109080 0x7fbe081094a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:08.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.299+0000 7fbe107ad700 1 -- 172.21.15.33:0/2708680177 >> 172.21.15.33:0/2708680177 conn(0x7fbe08104720 msgr2=0x7fbe08106b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:50:08.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.299+0000 7fbe107ad700 1 -- 172.21.15.33:0/2708680177 shutdown_connections 2024-05-15T13:50:08.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.299+0000 7fbe107ad700 1 -- 172.21.15.33:0/2708680177 wait complete. 2024-05-15T13:50:08.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.300+0000 7fbe107ad700 1 Processor -- start 2024-05-15T13:50:08.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.300+0000 7fbe107ad700 1 -- start start 2024-05-15T13:50:08.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.300+0000 7fbe107ad700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08109080 0x7fbe0819cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:08.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.300+0000 7fbe107ad700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0810b0c0 con 0x7fbe08109080 2024-05-15T13:50:08.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.300+0000 7fbe0e549700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08109080 0x7fbe0819cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:08.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.300+0000 7fbe0e549700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08109080 0x7fbe0819cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54842/0 (socket says 172.21.15.33:54842) 2024-05-15T13:50:08.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.300+0000 7fbe0e549700 1 -- 172.21.15.33:0/712433502 learned_addr learned my addr 172.21.15.33:0/712433502 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:50:08.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.301+0000 7fbe0e549700 1 -- 172.21.15.33:0/712433502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe0819d430 con 0x7fbe08109080 2024-05-15T13:50:08.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.301+0000 7fbe0e549700 1 --2- 172.21.15.33:0/712433502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08109080 0x7fbe0819cef0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0116e0 tx=0x7fbdfc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:08.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.301+0000 7fbdfb7fe700 1 -- 172.21.15.33:0/712433502 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbdfc003a60 con 0x7fbe08109080 2024-05-15T13:50:08.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.301+0000 7fbe107ad700 1 -- 172.21.15.33:0/712433502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe0819d6c0 con 0x7fbe08109080 2024-05-15T13:50:08.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.301+0000 7fbdfb7fe700 1 -- 172.21.15.33:0/712433502 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdfc01b8f0 con 0x7fbe08109080 2024-05-15T13:50:08.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.301+0000 7fbdfb7fe700 1 -- 172.21.15.33:0/712433502 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbdfc019860 con 0x7fbe08109080 2024-05-15T13:50:08.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.301+0000 7fbe107ad700 1 -- 172.21.15.33:0/712433502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe081a03b0 con 0x7fbe08109080 2024-05-15T13:50:08.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.302+0000 7fbe107ad700 1 -- 172.21.15.33:0/712433502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe08064ba0 con 0x7fbe08109080 2024-05-15T13:50:08.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.302+0000 7fbdfb7fe700 1 -- 172.21.15.33:0/712433502 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7fbdfc019b60 con 0x7fbe08109080 2024-05-15T13:50:08.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.302+0000 7fbdfb7fe700 1 --2- 172.21.15.33:0/712433502 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbdf403e360 0x7fbdf4040820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:08.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.302+0000 7fbdfb7fe700 1 -- 172.21.15.33:0/712433502 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbdfc053bf0 con 0x7fbe08109080 2024-05-15T13:50:08.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.303+0000 7fbe0dd48700 1 --2- 172.21.15.33:0/712433502 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbdf403e360 0x7fbdf4040820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:08.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.303+0000 7fbe0dd48700 1 --2- 172.21.15.33:0/712433502 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbdf403e360 0x7fbdf4040820 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbe04006fd0 tx=0x7fbe04006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:08.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.307+0000 7fbdfb7fe700 1 -- 172.21.15.33:0/712433502 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbdfc05a080 con 0x7fbe08109080 2024-05-15T13:50:08.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.492+0000 7fbdfb7fe700 1 -- 172.21.15.33:0/712433502 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fbdfc018530 con 0x7fbe08109080 2024-05-15T13:50:08.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.534+0000 7fbe107ad700 1 -- 172.21.15.33:0/712433502 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fbe08105880 con 0x7fbdf403e360 2024-05-15T13:50:08.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.628+0000 7fbdfb7fe700 1 -- 172.21.15.33:0/712433502 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fbe08105880 con 0x7fbdf403e360 2024-05-15T13:50:08.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.630+0000 7fbe107ad700 1 -- 172.21.15.33:0/712433502 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbdf403e360 msgr2=0x7fbdf4040820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:08.263 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.630+0000 7fbe107ad700 1 --2- 172.21.15.33:0/712433502 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbdf403e360 0x7fbdf4040820 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbe04006fd0 tx=0x7fbe04006e40 comp rx=0 tx=0).stop 2024-05-15T13:50:08.263 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.630+0000 7fbe107ad700 1 -- 172.21.15.33:0/712433502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08109080 msgr2=0x7fbe0819cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:08.263 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.630+0000 7fbe107ad700 1 --2- 172.21.15.33:0/712433502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08109080 0x7fbe0819cef0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0116e0 tx=0x7fbdfc011aa0 comp rx=0 tx=0).stop 2024-05-15T13:50:08.263 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.630+0000 7fbe107ad700 1 -- 172.21.15.33:0/712433502 shutdown_connections 2024-05-15T13:50:08.263 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.630+0000 7fbe107ad700 1 --2- 172.21.15.33:0/712433502 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbdf403e360 0x7fbdf4040820 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:08.263 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.630+0000 7fbe107ad700 1 --2- 172.21.15.33:0/712433502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08109080 0x7fbe0819cef0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:08.263 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.630+0000 7fbe107ad700 1 -- 172.21.15.33:0/712433502 >> 172.21.15.33:0/712433502 conn(0x7fbe08104720 msgr2=0x7fbe08105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:50:08.263 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.630+0000 7fbe107ad700 1 -- 172.21.15.33:0/712433502 shutdown_connections 2024-05-15T13:50:08.263 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:07.630+0000 7fbe107ad700 1 -- 172.21.15.33:0/712433502 wait complete. 2024-05-15T13:50:08.263 INFO:teuthology.orchestra.run.smithi033.stdout:Creating initial admin user... 2024-05-15T13:50:08.679 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:08 smithi033 ceph-mon[31650]: mgrmap e12: x(active, since 2s) 2024-05-15T13:50:08.679 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:08 smithi033 ceph-mon[31650]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:50:08.679 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:08 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:08.679 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:08 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:09.749 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$jurXtWBA5ED7AP9uk9Un6ORmZOVfX67c6f60WrZGWN32qFr4dglWS", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715781009, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-15T13:50:09.749 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.701+0000 7fdb9b8d8700 1 Processor -- start 2024-05-15T13:50:09.749 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.701+0000 7fdb9b8d8700 1 -- start start 2024-05-15T13:50:09.749 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.702+0000 7fdb9b8d8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb94107f10 0x7fdb94108350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:09.749 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.702+0000 7fdb9b8d8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdb94108890 con 0x7fdb94107f10 2024-05-15T13:50:09.749 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.702+0000 7fdb99674700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb94107f10 0x7fdb94108350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:09.749 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.702+0000 7fdb99674700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb94107f10 0x7fdb94108350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54858/0 (socket says 172.21.15.33:54858) 2024-05-15T13:50:09.749 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.702+0000 7fdb99674700 1 -- 172.21.15.33:0/526599148 learned_addr learned my addr 172.21.15.33:0/526599148 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:50:09.749 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.702+0000 7fdb99674700 1 -- 172.21.15.33:0/526599148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb941090f0 con 0x7fdb94107f10 2024-05-15T13:50:09.749 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.702+0000 7fdb99674700 1 --2- 172.21.15.33:0/526599148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb94107f10 0x7fdb94108350 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdb900098a0 tx=0x7fdb90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2119cd78459a71f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:09.749 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.702+0000 7fdb8bfff700 1 -- 172.21.15.33:0/526599148 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb90004160 con 0x7fdb94107f10 2024-05-15T13:50:09.750 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.702+0000 7fdb8bfff700 1 -- 172.21.15.33:0/526599148 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb90004300 con 0x7fdb94107f10 2024-05-15T13:50:09.750 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.703+0000 7fdb8bfff700 1 -- 172.21.15.33:0/526599148 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb90011a20 con 0x7fdb94107f10 2024-05-15T13:50:09.750 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.703+0000 7fdb9b8d8700 1 -- 172.21.15.33:0/526599148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb94107f10 msgr2=0x7fdb94108350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:09.750 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.703+0000 7fdb9b8d8700 1 --2- 172.21.15.33:0/526599148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb94107f10 0x7fdb94108350 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdb900098a0 tx=0x7fdb90009bb0 comp rx=0 tx=0).stop 2024-05-15T13:50:09.750 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.703+0000 7fdb9b8d8700 1 -- 172.21.15.33:0/526599148 shutdown_connections 2024-05-15T13:50:09.750 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.703+0000 7fdb9b8d8700 1 --2- 172.21.15.33:0/526599148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb94107f10 0x7fdb94108350 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:09.750 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.703+0000 7fdb9b8d8700 1 -- 172.21.15.33:0/526599148 >> 172.21.15.33:0/526599148 conn(0x7fdb941034e0 msgr2=0x7fdb94105940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:50:09.750 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.703+0000 7fdb9b8d8700 1 -- 172.21.15.33:0/526599148 shutdown_connections 2024-05-15T13:50:09.750 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.703+0000 7fdb9b8d8700 1 -- 172.21.15.33:0/526599148 wait complete. 2024-05-15T13:50:09.750 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.704+0000 7fdb9b8d8700 1 Processor -- start 2024-05-15T13:50:09.750 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.704+0000 7fdb9b8d8700 1 -- start start 2024-05-15T13:50:09.750 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.704+0000 7fdb9b8d8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb94107f10 0x7fdb94193070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:09.750 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.704+0000 7fdb9b8d8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9410c3b0 con 0x7fdb94107f10 2024-05-15T13:50:09.751 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.704+0000 7fdb99674700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb94107f10 0x7fdb94193070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:09.751 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.704+0000 7fdb99674700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb94107f10 0x7fdb94193070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54866/0 (socket says 172.21.15.33:54866) 2024-05-15T13:50:09.751 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.704+0000 7fdb99674700 1 -- 172.21.15.33:0/3799952481 learned_addr learned my addr 172.21.15.33:0/3799952481 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:50:09.751 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.704+0000 7fdb99674700 1 -- 172.21.15.33:0/3799952481 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb941935b0 con 0x7fdb94107f10 2024-05-15T13:50:09.751 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.704+0000 7fdb99674700 1 --2- 172.21.15.33:0/3799952481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb94107f10 0x7fdb94193070 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdb90000c00 tx=0x7fdb90011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:09.751 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.705+0000 7fdb8a7fc700 1 -- 172.21.15.33:0/3799952481 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb90003aa0 con 0x7fdb94107f10 2024-05-15T13:50:09.751 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.705+0000 7fdb9b8d8700 1 -- 172.21.15.33:0/3799952481 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb941917a0 con 0x7fdb94107f10 2024-05-15T13:50:09.751 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.705+0000 7fdb8a7fc700 1 -- 172.21.15.33:0/3799952481 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb9001b8f0 con 0x7fdb94107f10 2024-05-15T13:50:09.751 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.705+0000 7fdb8a7fc700 1 -- 172.21.15.33:0/3799952481 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb90023890 con 0x7fdb94107f10 2024-05-15T13:50:09.751 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.705+0000 7fdb9b8d8700 1 -- 172.21.15.33:0/3799952481 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb94191ce0 con 0x7fdb94107f10 2024-05-15T13:50:09.751 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.706+0000 7fdb8a7fc700 1 -- 172.21.15.33:0/3799952481 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fdb9002c3f0 con 0x7fdb94107f10 2024-05-15T13:50:09.751 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.706+0000 7fdb9b8d8700 1 -- 172.21.15.33:0/3799952481 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb9410c530 con 0x7fdb94107f10 2024-05-15T13:50:09.751 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.706+0000 7fdb8a7fc700 1 --2- 172.21.15.33:0/3799952481 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fdb8003e4c0 0x7fdb80040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:09.751 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.706+0000 7fdb8a7fc700 1 -- 172.21.15.33:0/3799952481 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdb900540e0 con 0x7fdb94107f10 2024-05-15T13:50:09.751 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.706+0000 7fdb98e73700 1 --2- 172.21.15.33:0/3799952481 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fdb8003e4c0 0x7fdb80040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:09.752 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.706+0000 7fdb98e73700 1 --2- 172.21.15.33:0/3799952481 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fdb8003e4c0 0x7fdb80040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdb84006fd0 tx=0x7fdb84006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:09.752 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.711+0000 7fdb8a7fc700 1 -- 172.21.15.33:0/3799952481 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdb9001f030 con 0x7fdb94107f10 2024-05-15T13:50:09.752 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:08.947+0000 7fdb9b8d8700 1 -- 172.21.15.33:0/3799952481 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fdb94064ba0 con 0x7fdb8003e4c0 2024-05-15T13:50:09.752 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:09.196+0000 7fdb8a7fc700 1 -- 172.21.15.33:0/3799952481 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fdb94064ba0 con 0x7fdb8003e4c0 2024-05-15T13:50:09.752 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:09.198+0000 7fdb9b8d8700 1 -- 172.21.15.33:0/3799952481 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fdb8003e4c0 msgr2=0x7fdb80040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:09.752 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:09.198+0000 7fdb9b8d8700 1 --2- 172.21.15.33:0/3799952481 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fdb8003e4c0 0x7fdb80040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdb84006fd0 tx=0x7fdb84006e40 comp rx=0 tx=0).stop 2024-05-15T13:50:09.752 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:09.198+0000 7fdb9b8d8700 1 -- 172.21.15.33:0/3799952481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb94107f10 msgr2=0x7fdb94193070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:09.752 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:09.198+0000 7fdb9b8d8700 1 --2- 172.21.15.33:0/3799952481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb94107f10 0x7fdb94193070 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdb90000c00 tx=0x7fdb90011aa0 comp rx=0 tx=0).stop 2024-05-15T13:50:09.752 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:09.198+0000 7fdb9b8d8700 1 -- 172.21.15.33:0/3799952481 shutdown_connections 2024-05-15T13:50:09.752 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:09.198+0000 7fdb9b8d8700 1 --2- 172.21.15.33:0/3799952481 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fdb8003e4c0 0x7fdb80040980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:09.752 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:09.198+0000 7fdb9b8d8700 1 --2- 172.21.15.33:0/3799952481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb94107f10 0x7fdb94193070 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:09.752 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:09.198+0000 7fdb9b8d8700 1 -- 172.21.15.33:0/3799952481 >> 172.21.15.33:0/3799952481 conn(0x7fdb941034e0 msgr2=0x7fdb94104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:50:09.752 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:09.198+0000 7fdb9b8d8700 1 -- 172.21.15.33:0/3799952481 shutdown_connections 2024-05-15T13:50:09.752 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:09.198+0000 7fdb9b8d8700 1 -- 172.21.15.33:0/3799952481 wait complete. 2024-05-15T13:50:09.752 INFO:teuthology.orchestra.run.smithi033.stdout:Fetching dashboard port number... 2024-05-15T13:50:10.092 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:09 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:10.092 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:09 smithi033 ceph-mon[31650]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:50:10.092 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:09 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:11.126 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:10 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/3087442900' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-15T13:50:11.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 8443 2024-05-15T13:50:11.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.299+0000 7fd8c5dbb700 1 Processor -- start 2024-05-15T13:50:11.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.299+0000 7fd8c5dbb700 1 -- start start 2024-05-15T13:50:11.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.299+0000 7fd8c5dbb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c0109080 0x7fd8c01094a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:11.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.299+0000 7fd8c5dbb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c01099e0 con 0x7fd8c0109080 2024-05-15T13:50:11.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.299+0000 7fd8bf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c0109080 0x7fd8c01094a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:11.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.299+0000 7fd8bf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c0109080 0x7fd8c01094a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54870/0 (socket says 172.21.15.33:54870) 2024-05-15T13:50:11.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.299+0000 7fd8bf7fe700 1 -- 172.21.15.33:0/3987746644 learned_addr learned my addr 172.21.15.33:0/3987746644 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:50:11.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.299+0000 7fd8bf7fe700 1 -- 172.21.15.33:0/3987746644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c0109b60 con 0x7fd8c0109080 2024-05-15T13:50:11.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.300+0000 7fd8bf7fe700 1 --2- 172.21.15.33:0/3987746644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c0109080 0x7fd8c01094a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00098a0 tx=0x7fd8b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4424505047c4889 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:11.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.300+0000 7fd8be7fc700 1 -- 172.21.15.33:0/3987746644 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8b0004160 con 0x7fd8c0109080 2024-05-15T13:50:11.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.300+0000 7fd8be7fc700 1 -- 172.21.15.33:0/3987746644 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8b0004300 con 0x7fd8c0109080 2024-05-15T13:50:11.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.300+0000 7fd8be7fc700 1 -- 172.21.15.33:0/3987746644 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8b0011a20 con 0x7fd8c0109080 2024-05-15T13:50:11.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.300+0000 7fd8c5dbb700 1 -- 172.21.15.33:0/3987746644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c0109080 msgr2=0x7fd8c01094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:11.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.300+0000 7fd8c5dbb700 1 --2- 172.21.15.33:0/3987746644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c0109080 0x7fd8c01094a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00098a0 tx=0x7fd8b0009bb0 comp rx=0 tx=0).stop 2024-05-15T13:50:11.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.300+0000 7fd8c5dbb700 1 -- 172.21.15.33:0/3987746644 shutdown_connections 2024-05-15T13:50:11.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.300+0000 7fd8c5dbb700 1 --2- 172.21.15.33:0/3987746644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c0109080 0x7fd8c01094a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:11.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.300+0000 7fd8c5dbb700 1 -- 172.21.15.33:0/3987746644 >> 172.21.15.33:0/3987746644 conn(0x7fd8c0104720 msgr2=0x7fd8c0106b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:50:11.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.301+0000 7fd8c5dbb700 1 -- 172.21.15.33:0/3987746644 shutdown_connections 2024-05-15T13:50:11.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.301+0000 7fd8c5dbb700 1 -- 172.21.15.33:0/3987746644 wait complete. 2024-05-15T13:50:11.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.301+0000 7fd8c5dbb700 1 Processor -- start 2024-05-15T13:50:11.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.301+0000 7fd8c5dbb700 1 -- start start 2024-05-15T13:50:11.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.301+0000 7fd8c5dbb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c0109080 0x7fd8c019cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:11.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.302+0000 7fd8c5dbb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c010b0c0 con 0x7fd8c0109080 2024-05-15T13:50:11.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.302+0000 7fd8bf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c0109080 0x7fd8c019cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:11.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.302+0000 7fd8bf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c0109080 0x7fd8c019cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54876/0 (socket says 172.21.15.33:54876) 2024-05-15T13:50:11.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.302+0000 7fd8bf7fe700 1 -- 172.21.15.33:0/3087442900 learned_addr learned my addr 172.21.15.33:0/3087442900 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:50:11.530 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.302+0000 7fd8bf7fe700 1 -- 172.21.15.33:0/3087442900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c019d430 con 0x7fd8c0109080 2024-05-15T13:50:11.530 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.302+0000 7fd8bf7fe700 1 --2- 172.21.15.33:0/3087442900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c0109080 0x7fd8c019cef0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00116e0 tx=0x7fd8b0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:11.530 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.302+0000 7fd8bcff9700 1 -- 172.21.15.33:0/3087442900 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8b0003a60 con 0x7fd8c0109080 2024-05-15T13:50:11.530 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.302+0000 7fd8c5dbb700 1 -- 172.21.15.33:0/3087442900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c019d6c0 con 0x7fd8c0109080 2024-05-15T13:50:11.530 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.302+0000 7fd8bcff9700 1 -- 172.21.15.33:0/3087442900 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8b001b8f0 con 0x7fd8c0109080 2024-05-15T13:50:11.530 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.302+0000 7fd8bcff9700 1 -- 172.21.15.33:0/3087442900 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8b0019860 con 0x7fd8c0109080 2024-05-15T13:50:11.530 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.302+0000 7fd8c5dbb700 1 -- 172.21.15.33:0/3087442900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8c01a03b0 con 0x7fd8c0109080 2024-05-15T13:50:11.530 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.303+0000 7fd8bcff9700 1 -- 172.21.15.33:0/3087442900 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fd8b0019b60 con 0x7fd8c0109080 2024-05-15T13:50:11.530 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.303+0000 7fd8c5dbb700 1 -- 172.21.15.33:0/3087442900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8c0064ba0 con 0x7fd8c0109080 2024-05-15T13:50:11.530 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.303+0000 7fd8bcff9700 1 --2- 172.21.15.33:0/3087442900 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fd8a803e470 0x7fd8a8040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:11.530 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.304+0000 7fd8bcff9700 1 -- 172.21.15.33:0/3087442900 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd8b0053db0 con 0x7fd8c0109080 2024-05-15T13:50:11.530 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.304+0000 7fd8beffd700 1 --2- 172.21.15.33:0/3087442900 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fd8a803e470 0x7fd8a8040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:11.530 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.304+0000 7fd8beffd700 1 --2- 172.21.15.33:0/3087442900 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fd8a803e470 0x7fd8a8040930 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4006fd0 tx=0x7fd8b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:11.530 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.309+0000 7fd8bcff9700 1 -- 172.21.15.33:0/3087442900 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd8b001f030 con 0x7fd8c0109080 2024-05-15T13:50:11.531 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.527+0000 7fd8c5dbb700 1 -- 172.21.15.33:0/3087442900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fd8c01a0960 con 0x7fd8c0109080 2024-05-15T13:50:11.531 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.527+0000 7fd8bcff9700 1 -- 172.21.15.33:0/3087442900 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fd8b0028de0 con 0x7fd8c0109080 2024-05-15T13:50:11.531 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.529+0000 7fd8c5dbb700 1 -- 172.21.15.33:0/3087442900 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fd8a803e470 msgr2=0x7fd8a8040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:11.531 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.529+0000 7fd8c5dbb700 1 --2- 172.21.15.33:0/3087442900 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fd8a803e470 0x7fd8a8040930 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4006fd0 tx=0x7fd8b4006e40 comp rx=0 tx=0).stop 2024-05-15T13:50:11.531 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.529+0000 7fd8c5dbb700 1 -- 172.21.15.33:0/3087442900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c0109080 msgr2=0x7fd8c019cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:11.531 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.529+0000 7fd8c5dbb700 1 --2- 172.21.15.33:0/3087442900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c0109080 0x7fd8c019cef0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00116e0 tx=0x7fd8b0011aa0 comp rx=0 tx=0).stop 2024-05-15T13:50:11.531 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.529+0000 7fd8c5dbb700 1 -- 172.21.15.33:0/3087442900 shutdown_connections 2024-05-15T13:50:11.531 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.529+0000 7fd8c5dbb700 1 --2- 172.21.15.33:0/3087442900 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fd8a803e470 0x7fd8a8040930 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:11.531 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.529+0000 7fd8c5dbb700 1 --2- 172.21.15.33:0/3087442900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c0109080 0x7fd8c019cef0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:11.531 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.529+0000 7fd8c5dbb700 1 -- 172.21.15.33:0/3087442900 >> 172.21.15.33:0/3087442900 conn(0x7fd8c0104720 msgr2=0x7fd8c0105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:50:11.531 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.530+0000 7fd8c5dbb700 1 -- 172.21.15.33:0/3087442900 shutdown_connections 2024-05-15T13:50:11.531 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:10.530+0000 7fd8c5dbb700 1 -- 172.21.15.33:0/3087442900 wait complete. 2024-05-15T13:50:11.538 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-15T13:50:11.538 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-05-15T13:50:11.547 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-15T13:50:11.547 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-05-15T13:50:11.547 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-05-15T13:50:11.547 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-15T13:50:11.549 INFO:teuthology.orchestra.run.smithi033.stdout:Ceph Dashboard is now available at: 2024-05-15T13:50:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:50:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: URL: https://smithi033.front.sepia.ceph.com:8443/ 2024-05-15T13:50:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: User: admin 2024-05-15T13:50:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: Password: x5wumg60zy 2024-05-15T13:50:11.550 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:50:11.550 INFO:teuthology.orchestra.run.smithi033.stdout:Saving cluster configuration to /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config directory 2024-05-15T13:50:11.550 INFO:teuthology.orchestra.run.smithi033.stdout:Enabling autotune for osd_memory_target 2024-05-15T13:50:13.179 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:12 smithi033 ceph-mon[31650]: mgrmap e13: x(active, since 6s) 2024-05-15T13:50:13.186 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.315+0000 7ff15f57c700 1 Processor -- start 2024-05-15T13:50:13.187 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.315+0000 7ff15f57c700 1 -- start start 2024-05-15T13:50:13.187 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.316+0000 7ff15f57c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff158109540 0x7ff158109960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:13.187 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.316+0000 7ff15f57c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff158109f30 con 0x7ff158109540 2024-05-15T13:50:13.187 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.316+0000 7ff15d318700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff158109540 0x7ff158109960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:13.187 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.316+0000 7ff15d318700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff158109540 0x7ff158109960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54882/0 (socket says 172.21.15.33:54882) 2024-05-15T13:50:13.187 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.316+0000 7ff15d318700 1 -- 172.21.15.33:0/1325254628 learned_addr learned my addr 172.21.15.33:0/1325254628 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:50:13.187 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.316+0000 7ff15d318700 1 -- 172.21.15.33:0/1325254628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1581081f0 con 0x7ff158109540 2024-05-15T13:50:13.187 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.316+0000 7ff15d318700 1 --2- 172.21.15.33:0/1325254628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff158109540 0x7ff158109960 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff148009e80 tx=0x7ff14800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c46dd244fa7f57c server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:13.187 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.317+0000 7ff157fff700 1 -- 172.21.15.33:0/1325254628 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff148017070 con 0x7ff158109540 2024-05-15T13:50:13.187 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.317+0000 7ff157fff700 1 -- 172.21.15.33:0/1325254628 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff14800b8c0 con 0x7ff158109540 2024-05-15T13:50:13.187 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.317+0000 7ff157fff700 1 -- 172.21.15.33:0/1325254628 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff148004490 con 0x7ff158109540 2024-05-15T13:50:13.187 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.317+0000 7ff15f57c700 1 -- 172.21.15.33:0/1325254628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff158109540 msgr2=0x7ff158109960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:13.187 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.317+0000 7ff15f57c700 1 --2- 172.21.15.33:0/1325254628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff158109540 0x7ff158109960 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff148009e80 tx=0x7ff14800b330 comp rx=0 tx=0).stop 2024-05-15T13:50:13.187 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.317+0000 7ff15f57c700 1 -- 172.21.15.33:0/1325254628 shutdown_connections 2024-05-15T13:50:13.188 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.317+0000 7ff15f57c700 1 --2- 172.21.15.33:0/1325254628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff158109540 0x7ff158109960 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:13.188 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.317+0000 7ff15f57c700 1 -- 172.21.15.33:0/1325254628 >> 172.21.15.33:0/1325254628 conn(0x7ff158102c80 msgr2=0x7ff1581050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:50:13.188 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.317+0000 7ff15f57c700 1 -- 172.21.15.33:0/1325254628 shutdown_connections 2024-05-15T13:50:13.188 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.317+0000 7ff15f57c700 1 -- 172.21.15.33:0/1325254628 wait complete. 2024-05-15T13:50:13.188 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.318+0000 7ff15f57c700 1 Processor -- start 2024-05-15T13:50:13.188 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.318+0000 7ff15f57c700 1 -- start start 2024-05-15T13:50:13.188 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.318+0000 7ff15f57c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff158109540 0x7ff15819cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:13.188 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.318+0000 7ff15f57c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff15810c5b0 con 0x7ff158109540 2024-05-15T13:50:13.188 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.318+0000 7ff15d318700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff158109540 0x7ff15819cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:13.188 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.318+0000 7ff15d318700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff158109540 0x7ff15819cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54884/0 (socket says 172.21.15.33:54884) 2024-05-15T13:50:13.188 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.318+0000 7ff15d318700 1 -- 172.21.15.33:0/2720093737 learned_addr learned my addr 172.21.15.33:0/2720093737 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:50:13.188 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.319+0000 7ff15d318700 1 -- 172.21.15.33:0/2720093737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff15819d4f0 con 0x7ff158109540 2024-05-15T13:50:13.188 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.319+0000 7ff15d318700 1 --2- 172.21.15.33:0/2720093737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff158109540 0x7ff15819cfb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff148009e50 tx=0x7ff14800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:13.188 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.319+0000 7ff1567fc700 1 -- 172.21.15.33:0/2720093737 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff148020020 con 0x7ff158109540 2024-05-15T13:50:13.189 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.319+0000 7ff1567fc700 1 -- 172.21.15.33:0/2720093737 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff148003b50 con 0x7ff158109540 2024-05-15T13:50:13.189 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.319+0000 7ff15f57c700 1 -- 172.21.15.33:0/2720093737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff15819d780 con 0x7ff158109540 2024-05-15T13:50:13.189 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.319+0000 7ff1567fc700 1 -- 172.21.15.33:0/2720093737 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff148017040 con 0x7ff158109540 2024-05-15T13:50:13.189 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.319+0000 7ff15f57c700 1 -- 172.21.15.33:0/2720093737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1581a0470 con 0x7ff158109540 2024-05-15T13:50:13.189 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.320+0000 7ff15f57c700 1 -- 172.21.15.33:0/2720093737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff158064ba0 con 0x7ff158109540 2024-05-15T13:50:13.189 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.320+0000 7ff1567fc700 1 -- 172.21.15.33:0/2720093737 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7ff148023020 con 0x7ff158109540 2024-05-15T13:50:13.189 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.320+0000 7ff1567fc700 1 --2- 172.21.15.33:0/2720093737 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff14003e570 0x7ff140040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:13.189 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.320+0000 7ff1567fc700 1 -- 172.21.15.33:0/2720093737 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff14800f050 con 0x7ff158109540 2024-05-15T13:50:13.189 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.320+0000 7ff15cb17700 1 --2- 172.21.15.33:0/2720093737 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff14003e570 0x7ff140040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:13.189 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.321+0000 7ff15cb17700 1 --2- 172.21.15.33:0/2720093737 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff14003e570 0x7ff140040a30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff14c006fd0 tx=0x7ff14c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:13.189 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.325+0000 7ff1567fc700 1 -- 172.21.15.33:0/2720093737 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff14802a020 con 0x7ff158109540 2024-05-15T13:50:13.189 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.566+0000 7ff15f57c700 1 -- 172.21.15.33:0/2720093737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7ff15802e910 con 0x7ff158109540 2024-05-15T13:50:13.189 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.567+0000 7ff1567fc700 1 -- 172.21.15.33:0/2720093737 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7ff148025dc0 con 0x7ff158109540 2024-05-15T13:50:13.189 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.569+0000 7ff15f57c700 1 -- 172.21.15.33:0/2720093737 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff14003e570 msgr2=0x7ff140040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:13.189 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.569+0000 7ff15f57c700 1 --2- 172.21.15.33:0/2720093737 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff14003e570 0x7ff140040a30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff14c006fd0 tx=0x7ff14c006e40 comp rx=0 tx=0).stop 2024-05-15T13:50:13.190 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.569+0000 7ff15f57c700 1 -- 172.21.15.33:0/2720093737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff158109540 msgr2=0x7ff15819cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:13.190 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.569+0000 7ff15f57c700 1 --2- 172.21.15.33:0/2720093737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff158109540 0x7ff15819cfb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff148009e50 tx=0x7ff14800bd50 comp rx=0 tx=0).stop 2024-05-15T13:50:13.190 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.569+0000 7ff15f57c700 1 -- 172.21.15.33:0/2720093737 shutdown_connections 2024-05-15T13:50:13.190 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.569+0000 7ff15f57c700 1 --2- 172.21.15.33:0/2720093737 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff14003e570 0x7ff140040a30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:13.190 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.569+0000 7ff15f57c700 1 --2- 172.21.15.33:0/2720093737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff158109540 0x7ff15819cfb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:13.190 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.569+0000 7ff15f57c700 1 -- 172.21.15.33:0/2720093737 >> 172.21.15.33:0/2720093737 conn(0x7ff158102c80 msgr2=0x7ff158103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:50:13.190 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.569+0000 7ff15f57c700 1 -- 172.21.15.33:0/2720093737 shutdown_connections 2024-05-15T13:50:13.190 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:12.569+0000 7ff15f57c700 1 -- 172.21.15.33:0/2720093737 wait complete. 2024-05-15T13:50:14.495 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.563+0000 7f801186a700 1 Processor -- start 2024-05-15T13:50:14.495 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.563+0000 7f801186a700 1 -- start start 2024-05-15T13:50:14.495 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.564+0000 7f801186a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f800c109060 0x7f800c109480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:14.495 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.564+0000 7f801186a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f800c109a50 con 0x7f800c109060 2024-05-15T13:50:14.495 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.564+0000 7f800affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f800c109060 0x7f800c109480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:14.495 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.564+0000 7f800affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f800c109060 0x7f800c109480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34996/0 (socket says 172.21.15.33:34996) 2024-05-15T13:50:14.495 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.564+0000 7f800affd700 1 -- 172.21.15.33:0/551321655 learned_addr learned my addr 172.21.15.33:0/551321655 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:50:14.495 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.564+0000 7f800affd700 1 -- 172.21.15.33:0/551321655 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f800c10a210 con 0x7f800c109060 2024-05-15T13:50:14.495 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.564+0000 7f800affd700 1 --2- 172.21.15.33:0/551321655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f800c109060 0x7f800c109480 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc0098a0 tx=0x7f7ffc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbbb113e1de91763 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:14.496 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.564+0000 7f8009ffb700 1 -- 172.21.15.33:0/551321655 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ffc004160 con 0x7f800c109060 2024-05-15T13:50:14.496 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.564+0000 7f8009ffb700 1 -- 172.21.15.33:0/551321655 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ffc004300 con 0x7f800c109060 2024-05-15T13:50:14.496 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.565+0000 7f8009ffb700 1 -- 172.21.15.33:0/551321655 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ffc011a20 con 0x7f800c109060 2024-05-15T13:50:14.496 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.565+0000 7f801186a700 1 -- 172.21.15.33:0/551321655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f800c109060 msgr2=0x7f800c109480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:14.496 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.565+0000 7f801186a700 1 --2- 172.21.15.33:0/551321655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f800c109060 0x7f800c109480 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc0098a0 tx=0x7f7ffc009bb0 comp rx=0 tx=0).stop 2024-05-15T13:50:14.496 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.565+0000 7f801186a700 1 -- 172.21.15.33:0/551321655 shutdown_connections 2024-05-15T13:50:14.496 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.565+0000 7f801186a700 1 --2- 172.21.15.33:0/551321655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f800c109060 0x7f800c109480 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:14.496 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.565+0000 7f801186a700 1 -- 172.21.15.33:0/551321655 >> 172.21.15.33:0/551321655 conn(0x7f800c104660 msgr2=0x7f800c106a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:50:14.496 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.565+0000 7f801186a700 1 -- 172.21.15.33:0/551321655 shutdown_connections 2024-05-15T13:50:14.496 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.565+0000 7f801186a700 1 -- 172.21.15.33:0/551321655 wait complete. 2024-05-15T13:50:14.496 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.566+0000 7f801186a700 1 Processor -- start 2024-05-15T13:50:14.496 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.566+0000 7f801186a700 1 -- start start 2024-05-15T13:50:14.496 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.566+0000 7f801186a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f800c109060 0x7f800c19cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:14.496 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.566+0000 7f801186a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f800c10d5c0 con 0x7f800c109060 2024-05-15T13:50:14.496 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.566+0000 7f800affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f800c109060 0x7f800c19cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:14.497 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.566+0000 7f800affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f800c109060 0x7f800c19cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34998/0 (socket says 172.21.15.33:34998) 2024-05-15T13:50:14.497 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.566+0000 7f800affd700 1 -- 172.21.15.33:0/2059278018 learned_addr learned my addr 172.21.15.33:0/2059278018 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:50:14.497 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.567+0000 7f800affd700 1 -- 172.21.15.33:0/2059278018 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f800c19d4e0 con 0x7f800c109060 2024-05-15T13:50:14.497 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.567+0000 7f800affd700 1 --2- 172.21.15.33:0/2059278018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f800c109060 0x7f800c19cfa0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc000c00 tx=0x7f7ffc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:14.497 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.567+0000 7f8010868700 1 -- 172.21.15.33:0/2059278018 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ffc003aa0 con 0x7f800c109060 2024-05-15T13:50:14.497 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.567+0000 7f8010868700 1 -- 172.21.15.33:0/2059278018 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ffc01b8f0 con 0x7f800c109060 2024-05-15T13:50:14.497 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.567+0000 7f801186a700 1 -- 172.21.15.33:0/2059278018 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f800c19d770 con 0x7f800c109060 2024-05-15T13:50:14.497 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.567+0000 7f8010868700 1 -- 172.21.15.33:0/2059278018 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ffc023890 con 0x7f800c109060 2024-05-15T13:50:14.497 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.567+0000 7f801186a700 1 -- 172.21.15.33:0/2059278018 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f800c1a0460 con 0x7f800c109060 2024-05-15T13:50:14.497 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.568+0000 7f8010868700 1 -- 172.21.15.33:0/2059278018 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f7ffc01b440 con 0x7f800c109060 2024-05-15T13:50:14.497 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.568+0000 7f801186a700 1 -- 172.21.15.33:0/2059278018 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f800c064ba0 con 0x7f800c109060 2024-05-15T13:50:14.497 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.568+0000 7f8010868700 1 --2- 172.21.15.33:0/2059278018 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f7ff403e560 0x7f7ff4040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:14.497 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.568+0000 7f8010868700 1 -- 172.21.15.33:0/2059278018 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7ffc054010 con 0x7f800c109060 2024-05-15T13:50:14.497 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.568+0000 7f800a7fc700 1 --2- 172.21.15.33:0/2059278018 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f7ff403e560 0x7f7ff4040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:14.497 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.569+0000 7f800a7fc700 1 --2- 172.21.15.33:0/2059278018 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f7ff403e560 0x7f7ff4040a20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8000006fd0 tx=0x7f8000006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:14.497 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.573+0000 7f8010868700 1 -- 172.21.15.33:0/2059278018 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7ffc01f030 con 0x7f800c109060 2024-05-15T13:50:14.498 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.904+0000 7f801186a700 1 -- 172.21.15.33:0/2059278018 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f800c19dc50 con 0x7f800c109060 2024-05-15T13:50:14.498 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.907+0000 7f8010868700 1 -- 172.21.15.33:0/2059278018 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f800c19dc50 con 0x7f800c109060 2024-05-15T13:50:14.498 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.909+0000 7f801186a700 1 -- 172.21.15.33:0/2059278018 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f7ff403e560 msgr2=0x7f7ff4040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:14.498 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.909+0000 7f801186a700 1 --2- 172.21.15.33:0/2059278018 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f7ff403e560 0x7f7ff4040a20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8000006fd0 tx=0x7f8000006e40 comp rx=0 tx=0).stop 2024-05-15T13:50:14.498 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.909+0000 7f801186a700 1 -- 172.21.15.33:0/2059278018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f800c109060 msgr2=0x7f800c19cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:14.498 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.909+0000 7f801186a700 1 --2- 172.21.15.33:0/2059278018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f800c109060 0x7f800c19cfa0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc000c00 tx=0x7f7ffc011aa0 comp rx=0 tx=0).stop 2024-05-15T13:50:14.498 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.909+0000 7f801186a700 1 -- 172.21.15.33:0/2059278018 shutdown_connections 2024-05-15T13:50:14.498 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.909+0000 7f801186a700 1 --2- 172.21.15.33:0/2059278018 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f7ff403e560 0x7f7ff4040a20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:14.498 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.909+0000 7f801186a700 1 --2- 172.21.15.33:0/2059278018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f800c109060 0x7f800c19cfa0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:14.498 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.909+0000 7f801186a700 1 -- 172.21.15.33:0/2059278018 >> 172.21.15.33:0/2059278018 conn(0x7f800c104660 msgr2=0x7f800c1051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:50:14.498 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.909+0000 7f801186a700 1 -- 172.21.15.33:0/2059278018 shutdown_connections 2024-05-15T13:50:14.498 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-15T13:50:13.909+0000 7f801186a700 1 -- 172.21.15.33:0/2059278018 wait complete. 2024-05-15T13:50:14.498 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-15T13:50:14.498 INFO:teuthology.orchestra.run.smithi033.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-15T13:50:14.498 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:50:14.499 INFO:teuthology.orchestra.run.smithi033.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-15T13:50:14.499 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:50:14.499 INFO:teuthology.orchestra.run.smithi033.stdout:Or, if you are only running a single cluster on this host: 2024-05-15T13:50:14.499 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:50:14.499 INFO:teuthology.orchestra.run.smithi033.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-15T13:50:14.499 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:50:14.499 INFO:teuthology.orchestra.run.smithi033.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-15T13:50:14.499 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:50:14.499 INFO:teuthology.orchestra.run.smithi033.stdout: ceph telemetry on 2024-05-15T13:50:14.499 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:50:14.499 INFO:teuthology.orchestra.run.smithi033.stdout:For more information see: 2024-05-15T13:50:14.500 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:50:14.500 INFO:teuthology.orchestra.run.smithi033.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-15T13:50:14.500 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:50:14.500 INFO:teuthology.orchestra.run.smithi033.stdout:Bootstrap complete. 2024-05-15T13:50:14.538 INFO:tasks.cephadm:Fetching config... 2024-05-15T13:50:14.538 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-15T13:50:14.538 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-15T13:50:14.556 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-15T13:50:14.556 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-15T13:50:14.556 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-15T13:50:14.612 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-15T13:50:14.612 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-15T13:50:14.613 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-15T13:50:14.679 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-15T13:50:14.679 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-15T13:50:14.679 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-15T13:50:14.735 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-15T13:50:14.735 DEBUG:teuthology.orchestra.run.smithi033:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCqjB+aTRhJp3xaxbjRkjYKgziFcZNRQIV8T0Plg01lMcLUr4hsC9nik2NM6iUyDmyvz2O4kDw0M2p96Uj9DDMoH68LKcZEVwklbmnaELTl8hazoOBabH219US+CfyP9jhdqE2JX3JOeBAGAQi5oyhHsGrTSJ9HZ2tVGH3+rbbnwz2cLHjmhuyLarwkAabZyuPOjLhGzLQT6ta+pd6PzbHcBpMYKRJN6flIZjBHoEAz5OTTMP8qokDEuwzVyFQpMyfn/AzNB6wu2RYcqQMdHS2l2KQ0fgxzQeyISiF3fRocYqsQ92Om/bFPlHoy/m8Yv2Kll3xnNCRH9ApB6DHkiEix292eAFE5FFMwt96DTUavx2BcRK7FSt5H1R8dniFG7llRuxeCw8oWpZxjNKWgjM+qwTlWKdeco4bllesW2t/xk2WnwRujORiBq5xIBaQMr0beHnB5eNETiw0vliw1WdcOpOSsip7rmY8KZ/EG9kqOYWmNDI8IOQYFWAmCZRZPxHM= ceph-bca78ca4-12c1-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-15T13:50:14.811 INFO:teuthology.orchestra.run.smithi033.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCqjB+aTRhJp3xaxbjRkjYKgziFcZNRQIV8T0Plg01lMcLUr4hsC9nik2NM6iUyDmyvz2O4kDw0M2p96Uj9DDMoH68LKcZEVwklbmnaELTl8hazoOBabH219US+CfyP9jhdqE2JX3JOeBAGAQi5oyhHsGrTSJ9HZ2tVGH3+rbbnwz2cLHjmhuyLarwkAabZyuPOjLhGzLQT6ta+pd6PzbHcBpMYKRJN6flIZjBHoEAz5OTTMP8qokDEuwzVyFQpMyfn/AzNB6wu2RYcqQMdHS2l2KQ0fgxzQeyISiF3fRocYqsQ92Om/bFPlHoy/m8Yv2Kll3xnNCRH9ApB6DHkiEix292eAFE5FFMwt96DTUavx2BcRK7FSt5H1R8dniFG7llRuxeCw8oWpZxjNKWgjM+qwTlWKdeco4bllesW2t/xk2WnwRujORiBq5xIBaQMr0beHnB5eNETiw0vliw1WdcOpOSsip7rmY8KZ/EG9kqOYWmNDI8IOQYFWAmCZRZPxHM= ceph-bca78ca4-12c1-11ef-bc98-c7b262605968 2024-05-15T13:50:14.822 DEBUG:teuthology.orchestra.run.smithi052:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCqjB+aTRhJp3xaxbjRkjYKgziFcZNRQIV8T0Plg01lMcLUr4hsC9nik2NM6iUyDmyvz2O4kDw0M2p96Uj9DDMoH68LKcZEVwklbmnaELTl8hazoOBabH219US+CfyP9jhdqE2JX3JOeBAGAQi5oyhHsGrTSJ9HZ2tVGH3+rbbnwz2cLHjmhuyLarwkAabZyuPOjLhGzLQT6ta+pd6PzbHcBpMYKRJN6flIZjBHoEAz5OTTMP8qokDEuwzVyFQpMyfn/AzNB6wu2RYcqQMdHS2l2KQ0fgxzQeyISiF3fRocYqsQ92Om/bFPlHoy/m8Yv2Kll3xnNCRH9ApB6DHkiEix292eAFE5FFMwt96DTUavx2BcRK7FSt5H1R8dniFG7llRuxeCw8oWpZxjNKWgjM+qwTlWKdeco4bllesW2t/xk2WnwRujORiBq5xIBaQMr0beHnB5eNETiw0vliw1WdcOpOSsip7rmY8KZ/EG9kqOYWmNDI8IOQYFWAmCZRZPxHM= ceph-bca78ca4-12c1-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-15T13:50:14.865 INFO:teuthology.orchestra.run.smithi052.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCqjB+aTRhJp3xaxbjRkjYKgziFcZNRQIV8T0Plg01lMcLUr4hsC9nik2NM6iUyDmyvz2O4kDw0M2p96Uj9DDMoH68LKcZEVwklbmnaELTl8hazoOBabH219US+CfyP9jhdqE2JX3JOeBAGAQi5oyhHsGrTSJ9HZ2tVGH3+rbbnwz2cLHjmhuyLarwkAabZyuPOjLhGzLQT6ta+pd6PzbHcBpMYKRJN6flIZjBHoEAz5OTTMP8qokDEuwzVyFQpMyfn/AzNB6wu2RYcqQMdHS2l2KQ0fgxzQeyISiF3fRocYqsQ92Om/bFPlHoy/m8Yv2Kll3xnNCRH9ApB6DHkiEix292eAFE5FFMwt96DTUavx2BcRK7FSt5H1R8dniFG7llRuxeCw8oWpZxjNKWgjM+qwTlWKdeco4bllesW2t/xk2WnwRujORiBq5xIBaQMr0beHnB5eNETiw0vliw1WdcOpOSsip7rmY8KZ/EG9kqOYWmNDI8IOQYFWAmCZRZPxHM= ceph-bca78ca4-12c1-11ef-bc98-c7b262605968 2024-05-15T13:50:14.879 DEBUG:teuthology.orchestra.run.smithi081:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCqjB+aTRhJp3xaxbjRkjYKgziFcZNRQIV8T0Plg01lMcLUr4hsC9nik2NM6iUyDmyvz2O4kDw0M2p96Uj9DDMoH68LKcZEVwklbmnaELTl8hazoOBabH219US+CfyP9jhdqE2JX3JOeBAGAQi5oyhHsGrTSJ9HZ2tVGH3+rbbnwz2cLHjmhuyLarwkAabZyuPOjLhGzLQT6ta+pd6PzbHcBpMYKRJN6flIZjBHoEAz5OTTMP8qokDEuwzVyFQpMyfn/AzNB6wu2RYcqQMdHS2l2KQ0fgxzQeyISiF3fRocYqsQ92Om/bFPlHoy/m8Yv2Kll3xnNCRH9ApB6DHkiEix292eAFE5FFMwt96DTUavx2BcRK7FSt5H1R8dniFG7llRuxeCw8oWpZxjNKWgjM+qwTlWKdeco4bllesW2t/xk2WnwRujORiBq5xIBaQMr0beHnB5eNETiw0vliw1WdcOpOSsip7rmY8KZ/EG9kqOYWmNDI8IOQYFWAmCZRZPxHM= ceph-bca78ca4-12c1-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-15T13:50:14.924 INFO:teuthology.orchestra.run.smithi081.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCqjB+aTRhJp3xaxbjRkjYKgziFcZNRQIV8T0Plg01lMcLUr4hsC9nik2NM6iUyDmyvz2O4kDw0M2p96Uj9DDMoH68LKcZEVwklbmnaELTl8hazoOBabH219US+CfyP9jhdqE2JX3JOeBAGAQi5oyhHsGrTSJ9HZ2tVGH3+rbbnwz2cLHjmhuyLarwkAabZyuPOjLhGzLQT6ta+pd6PzbHcBpMYKRJN6flIZjBHoEAz5OTTMP8qokDEuwzVyFQpMyfn/AzNB6wu2RYcqQMdHS2l2KQ0fgxzQeyISiF3fRocYqsQ92Om/bFPlHoy/m8Yv2Kll3xnNCRH9ApB6DHkiEix292eAFE5FFMwt96DTUavx2BcRK7FSt5H1R8dniFG7llRuxeCw8oWpZxjNKWgjM+qwTlWKdeco4bllesW2t/xk2WnwRujORiBq5xIBaQMr0beHnB5eNETiw0vliw1WdcOpOSsip7rmY8KZ/EG9kqOYWmNDI8IOQYFWAmCZRZPxHM= ceph-bca78ca4-12c1-11ef-bc98-c7b262605968 2024-05-15T13:50:14.939 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-15T13:50:15.186 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:50:15.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:14 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/2059278018' entity='client.admin' 2024-05-15T13:50:17.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.477+0000 7f411edea700 1 -- 172.21.15.33:0/3576748925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4118101e50 msgr2=0x7f4118102270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:17.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.477+0000 7f411edea700 1 --2- 172.21.15.33:0/3576748925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4118101e50 0x7f4118102270 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f410c009910 tx=0x7f410c009c20 comp rx=0 tx=0).stop 2024-05-15T13:50:17.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.477+0000 7f411edea700 1 -- 172.21.15.33:0/3576748925 shutdown_connections 2024-05-15T13:50:17.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.477+0000 7f411edea700 1 --2- 172.21.15.33:0/3576748925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4118101e50 0x7f4118102270 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:17.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.477+0000 7f411edea700 1 -- 172.21.15.33:0/3576748925 >> 172.21.15.33:0/3576748925 conn(0x7f41180fd440 msgr2=0x7f41180ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:50:17.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.477+0000 7f411edea700 1 -- 172.21.15.33:0/3576748925 shutdown_connections 2024-05-15T13:50:17.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.477+0000 7f411edea700 1 -- 172.21.15.33:0/3576748925 wait complete. 2024-05-15T13:50:17.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.478+0000 7f411edea700 1 Processor -- start 2024-05-15T13:50:17.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.478+0000 7f411edea700 1 -- start start 2024-05-15T13:50:17.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.478+0000 7f411edea700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4118101e50 0x7f411819a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:17.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.478+0000 7f411edea700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4118107d50 con 0x7f4118101e50 2024-05-15T13:50:17.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.478+0000 7f411cb86700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4118101e50 0x7f411819a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:17.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.479+0000 7f411cb86700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4118101e50 0x7f411819a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35008/0 (socket says 172.21.15.33:35008) 2024-05-15T13:50:17.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.479+0000 7f411cb86700 1 -- 172.21.15.33:0/1756856093 learned_addr learned my addr 172.21.15.33:0/1756856093 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:50:17.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.479+0000 7f411cb86700 1 -- 172.21.15.33:0/1756856093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411819ad30 con 0x7f4118101e50 2024-05-15T13:50:17.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.479+0000 7f411cb86700 1 --2- 172.21.15.33:0/1756856093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4118101e50 0x7f411819a7f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f410c005800 tx=0x7f410c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:17.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.479+0000 7f4115ffb700 1 -- 172.21.15.33:0/1756856093 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f410c027070 con 0x7f4118101e50 2024-05-15T13:50:17.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.479+0000 7f4115ffb700 1 -- 172.21.15.33:0/1756856093 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f410c00be10 con 0x7f4118101e50 2024-05-15T13:50:17.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.479+0000 7f411edea700 1 -- 172.21.15.33:0/1756856093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f411819afc0 con 0x7f4118101e50 2024-05-15T13:50:17.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.479+0000 7f4115ffb700 1 -- 172.21.15.33:0/1756856093 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f410c00f460 con 0x7f4118101e50 2024-05-15T13:50:17.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.479+0000 7f411edea700 1 -- 172.21.15.33:0/1756856093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f411819dcb0 con 0x7f4118101e50 2024-05-15T13:50:17.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.480+0000 7f411edea700 1 -- 172.21.15.33:0/1756856093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41181084d0 con 0x7f4118101e50 2024-05-15T13:50:17.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.480+0000 7f4115ffb700 1 -- 172.21.15.33:0/1756856093 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f410c02a070 con 0x7f4118101e50 2024-05-15T13:50:17.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.480+0000 7f4115ffb700 1 --2- 172.21.15.33:0/1756856093 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f410803e510 0x7f41080409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:17.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.481+0000 7f4115ffb700 1 -- 172.21.15.33:0/1756856093 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f410c01e030 con 0x7f4118101e50 2024-05-15T13:50:17.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.481+0000 7f4117fff700 1 --2- 172.21.15.33:0/1756856093 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f410803e510 0x7f41080409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:17.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.481+0000 7f4117fff700 1 --2- 172.21.15.33:0/1756856093 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f410803e510 0x7f41080409d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4104006fd0 tx=0x7f4104006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:17.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.486+0000 7f4115ffb700 1 -- 172.21.15.33:0/1756856093 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f410c013070 con 0x7f4118101e50 2024-05-15T13:50:17.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.699+0000 7f411edea700 1 -- 172.21.15.33:0/1756856093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f411802e670 con 0x7f4118101e50 2024-05-15T13:50:17.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.704+0000 7f4115ffb700 1 -- 172.21.15.33:0/1756856093 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f410c0310c0 con 0x7f4118101e50 2024-05-15T13:50:17.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.707+0000 7f411edea700 1 -- 172.21.15.33:0/1756856093 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f410803e510 msgr2=0x7f41080409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:17.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.707+0000 7f411edea700 1 --2- 172.21.15.33:0/1756856093 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f410803e510 0x7f41080409d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4104006fd0 tx=0x7f4104006e40 comp rx=0 tx=0).stop 2024-05-15T13:50:17.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.707+0000 7f411edea700 1 -- 172.21.15.33:0/1756856093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4118101e50 msgr2=0x7f411819a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:17.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.707+0000 7f411edea700 1 --2- 172.21.15.33:0/1756856093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4118101e50 0x7f411819a7f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f410c005800 tx=0x7f410c009f60 comp rx=0 tx=0).stop 2024-05-15T13:50:17.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.707+0000 7f411edea700 1 -- 172.21.15.33:0/1756856093 shutdown_connections 2024-05-15T13:50:17.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.707+0000 7f411edea700 1 --2- 172.21.15.33:0/1756856093 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f410803e510 0x7f41080409d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:17.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.707+0000 7f411edea700 1 --2- 172.21.15.33:0/1756856093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4118101e50 0x7f411819a7f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:17.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.707+0000 7f411edea700 1 -- 172.21.15.33:0/1756856093 >> 172.21.15.33:0/1756856093 conn(0x7f41180fd440 msgr2=0x7f4118191800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:50:17.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.707+0000 7f411edea700 1 -- 172.21.15.33:0/1756856093 shutdown_connections 2024-05-15T13:50:17.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:17.707+0000 7f411edea700 1 -- 172.21.15.33:0/1756856093 wait complete. 2024-05-15T13:50:17.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:17 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:17.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:17 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:17.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:17 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi033", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:50:17.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:17 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:17.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:17 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:50:17.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:17 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:18.297 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-15T13:50:18.297 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-15T13:50:18.542 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:50:18.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:18 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/1756856093' entity='client.admin' 2024-05-15T13:50:18.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:18 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:50:18.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:18 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:20.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.078+0000 7f019f664700 1 -- 172.21.15.33:0/743628438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0198101e50 msgr2=0x7f0198102270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:20.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.078+0000 7f019f664700 1 --2- 172.21.15.33:0/743628438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0198101e50 0x7f0198102270 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0194009910 tx=0x7f0194009c20 comp rx=0 tx=0).stop 2024-05-15T13:50:20.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.078+0000 7f019f664700 1 -- 172.21.15.33:0/743628438 shutdown_connections 2024-05-15T13:50:20.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.078+0000 7f019f664700 1 --2- 172.21.15.33:0/743628438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0198101e50 0x7f0198102270 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:20.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.078+0000 7f019f664700 1 -- 172.21.15.33:0/743628438 >> 172.21.15.33:0/743628438 conn(0x7f01980fd440 msgr2=0x7f01980ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:50:20.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.078+0000 7f019f664700 1 -- 172.21.15.33:0/743628438 shutdown_connections 2024-05-15T13:50:20.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.078+0000 7f019f664700 1 -- 172.21.15.33:0/743628438 wait complete. 2024-05-15T13:50:20.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.079+0000 7f019f664700 1 Processor -- start 2024-05-15T13:50:20.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.079+0000 7f019f664700 1 -- start start 2024-05-15T13:50:20.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.079+0000 7f019f664700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0198101e50 0x7f01980ff300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:20.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.079+0000 7f019f664700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0198107d50 con 0x7f0198101e50 2024-05-15T13:50:20.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.079+0000 7f019d400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0198101e50 0x7f01980ff300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:20.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.079+0000 7f019d400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0198101e50 0x7f01980ff300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35028/0 (socket says 172.21.15.33:35028) 2024-05-15T13:50:20.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.079+0000 7f019d400700 1 -- 172.21.15.33:0/1477321417 learned_addr learned my addr 172.21.15.33:0/1477321417 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:50:20.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.080+0000 7f019d400700 1 -- 172.21.15.33:0/1477321417 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01980ff840 con 0x7f0198101e50 2024-05-15T13:50:20.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.080+0000 7f019d400700 1 --2- 172.21.15.33:0/1477321417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0198101e50 0x7f01980ff300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0194005950 tx=0x7f0194004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:20.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.080+0000 7f018e7fc700 1 -- 172.21.15.33:0/1477321417 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0194026070 con 0x7f0198101e50 2024-05-15T13:50:20.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.080+0000 7f018e7fc700 1 -- 172.21.15.33:0/1477321417 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f019400bc00 con 0x7f0198101e50 2024-05-15T13:50:20.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.080+0000 7f018e7fc700 1 -- 172.21.15.33:0/1477321417 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0194003a10 con 0x7f0198101e50 2024-05-15T13:50:20.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.080+0000 7f019f664700 1 -- 172.21.15.33:0/1477321417 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01980fda30 con 0x7f0198101e50 2024-05-15T13:50:20.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.080+0000 7f019f664700 1 -- 172.21.15.33:0/1477321417 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01980fdf70 con 0x7f0198101e50 2024-05-15T13:50:20.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.081+0000 7f018e7fc700 1 -- 172.21.15.33:0/1477321417 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f0194029070 con 0x7f0198101e50 2024-05-15T13:50:20.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.081+0000 7f019f664700 1 -- 172.21.15.33:0/1477321417 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01981084d0 con 0x7f0198101e50 2024-05-15T13:50:20.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.081+0000 7f018e7fc700 1 --2- 172.21.15.33:0/1477321417 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f018403e570 0x7f0184040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:20.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.081+0000 7f018e7fc700 1 -- 172.21.15.33:0/1477321417 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f019401e070 con 0x7f0198101e50 2024-05-15T13:50:20.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.081+0000 7f019cbff700 1 --2- 172.21.15.33:0/1477321417 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f018403e570 0x7f0184040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:20.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.082+0000 7f019cbff700 1 --2- 172.21.15.33:0/1477321417 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f018403e570 0x7f0184040a30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0188006fd0 tx=0x7f0188006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:20.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.086+0000 7f018e7fc700 1 -- 172.21.15.33:0/1477321417 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0194013070 con 0x7f0198101e50 2024-05-15T13:50:20.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.305+0000 7f019f664700 1 -- 172.21.15.33:0/1477321417 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f0198191b40 con 0x7f018403e570 2024-05-15T13:50:20.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.309+0000 7f018e7fc700 1 -- 172.21.15.33:0/1477321417 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0198191b40 con 0x7f018403e570 2024-05-15T13:50:20.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.311+0000 7f019f664700 1 -- 172.21.15.33:0/1477321417 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f018403e570 msgr2=0x7f0184040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:20.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.311+0000 7f019f664700 1 --2- 172.21.15.33:0/1477321417 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f018403e570 0x7f0184040a30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0188006fd0 tx=0x7f0188006e40 comp rx=0 tx=0).stop 2024-05-15T13:50:20.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.311+0000 7f019f664700 1 -- 172.21.15.33:0/1477321417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0198101e50 msgr2=0x7f01980ff300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:20.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.311+0000 7f019f664700 1 --2- 172.21.15.33:0/1477321417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0198101e50 0x7f01980ff300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0194005950 tx=0x7f0194004fa0 comp rx=0 tx=0).stop 2024-05-15T13:50:20.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.311+0000 7f019f664700 1 -- 172.21.15.33:0/1477321417 shutdown_connections 2024-05-15T13:50:20.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.311+0000 7f019f664700 1 --2- 172.21.15.33:0/1477321417 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f018403e570 0x7f0184040a30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:20.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.311+0000 7f019f664700 1 --2- 172.21.15.33:0/1477321417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0198101e50 0x7f01980ff300 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:20.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.311+0000 7f019f664700 1 -- 172.21.15.33:0/1477321417 >> 172.21.15.33:0/1477321417 conn(0x7f01980fd440 msgr2=0x7f019806ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:50:20.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.311+0000 7f019f664700 1 -- 172.21.15.33:0/1477321417 shutdown_connections 2024-05-15T13:50:20.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:20.311+0000 7f019f664700 1 -- 172.21.15.33:0/1477321417 wait complete. 2024-05-15T13:50:21.046 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi052 2024-05-15T13:50:21.046 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-15T13:50:21.046 DEBUG:teuthology.orchestra.run.smithi052:> dd of=/etc/ceph/ceph.conf 2024-05-15T13:50:21.065 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-15T13:50:21.065 DEBUG:teuthology.orchestra.run.smithi052:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-15T13:50:21.123 INFO:tasks.cephadm:Adding host smithi052 to orchestrator... 2024-05-15T13:50:21.123 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph orch host add smithi052 2024-05-15T13:50:21.373 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:50:21.610 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:21 smithi033 ceph-mon[31650]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:50:21.610 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:21 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:21.611 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:21 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:50:21.611 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:21 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:50:21.611 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:21 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:50:21.611 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:21 smithi033 ceph-mon[31650]: Updating smithi033:/etc/ceph/ceph.conf 2024-05-15T13:50:22.603 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:22 smithi033 ceph-mon[31650]: Updating smithi033:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T13:50:22.603 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:22 smithi033 ceph-mon[31650]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-05-15T13:50:22.603 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:22 smithi033 ceph-mon[31650]: Updating smithi033:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-15T13:50:22.603 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:22 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:22.603 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:22 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:22.603 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:22 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:22.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:22.822+0000 7fdc8f22e700 1 -- 172.21.15.33:0/2552406686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc88104150 msgr2=0x7fdc88104530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:22.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:22.822+0000 7fdc8f22e700 1 --2- 172.21.15.33:0/2552406686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc88104150 0x7fdc88104530 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c009910 tx=0x7fdc7c009c20 comp rx=0 tx=0).stop 2024-05-15T13:50:22.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:22.822+0000 7fdc8f22e700 1 -- 172.21.15.33:0/2552406686 shutdown_connections 2024-05-15T13:50:22.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:22.822+0000 7fdc8f22e700 1 --2- 172.21.15.33:0/2552406686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc88104150 0x7fdc88104530 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:22.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:22.822+0000 7fdc8f22e700 1 -- 172.21.15.33:0/2552406686 >> 172.21.15.33:0/2552406686 conn(0x7fdc880fd9a0 msgr2=0x7fdc880ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:50:22.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:22.822+0000 7fdc8f22e700 1 -- 172.21.15.33:0/2552406686 shutdown_connections 2024-05-15T13:50:22.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:22.822+0000 7fdc8f22e700 1 -- 172.21.15.33:0/2552406686 wait complete. 2024-05-15T13:50:22.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:22.823+0000 7fdc8f22e700 1 Processor -- start 2024-05-15T13:50:22.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:22.823+0000 7fdc8f22e700 1 -- start start 2024-05-15T13:50:22.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:22.823+0000 7fdc8f22e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc88104150 0x7fdc8819e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:22.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:22.823+0000 7fdc8f22e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8810d4d0 con 0x7fdc88104150 2024-05-15T13:50:22.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:22.823+0000 7fdc8cfca700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc88104150 0x7fdc8819e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:22.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:22.823+0000 7fdc8cfca700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc88104150 0x7fdc8819e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56098/0 (socket says 172.21.15.33:56098) 2024-05-15T13:50:22.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:22.823+0000 7fdc8cfca700 1 -- 172.21.15.33:0/173883986 learned_addr learned my addr 172.21.15.33:0/173883986 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:50:22.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:22.823+0000 7fdc8cfca700 1 -- 172.21.15.33:0/173883986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc8819ec20 con 0x7fdc88104150 2024-05-15T13:50:22.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:22.824+0000 7fdc8cfca700 1 --2- 172.21.15.33:0/173883986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc88104150 0x7fdc8819e6e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c0056d0 tx=0x7fdc7c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:22.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:22.824+0000 7fdc85ffb700 1 -- 172.21.15.33:0/173883986 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc7c026070 con 0x7fdc88104150 2024-05-15T13:50:22.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:22.824+0000 7fdc8f22e700 1 -- 172.21.15.33:0/173883986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc8819eeb0 con 0x7fdc88104150 2024-05-15T13:50:22.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:22.824+0000 7fdc85ffb700 1 -- 172.21.15.33:0/173883986 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc7c004370 con 0x7fdc88104150 2024-05-15T13:50:22.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:22.824+0000 7fdc85ffb700 1 -- 172.21.15.33:0/173883986 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc7c0039b0 con 0x7fdc88104150 2024-05-15T13:50:22.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:22.824+0000 7fdc8f22e700 1 -- 172.21.15.33:0/173883986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc881988a0 con 0x7fdc88104150 2024-05-15T13:50:22.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:22.825+0000 7fdc8f22e700 1 -- 172.21.15.33:0/173883986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc8806bc90 con 0x7fdc88104150 2024-05-15T13:50:22.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:22.825+0000 7fdc85ffb700 1 -- 172.21.15.33:0/173883986 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fdc7c029070 con 0x7fdc88104150 2024-05-15T13:50:22.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:22.825+0000 7fdc85ffb700 1 --2- 172.21.15.33:0/173883986 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fdc7803e510 0x7fdc780409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:22.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:22.825+0000 7fdc85ffb700 1 -- 172.21.15.33:0/173883986 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdc7c01e070 con 0x7fdc88104150 2024-05-15T13:50:22.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:22.826+0000 7fdc87fff700 1 --2- 172.21.15.33:0/173883986 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fdc7803e510 0x7fdc780409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:22.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:22.826+0000 7fdc87fff700 1 --2- 172.21.15.33:0/173883986 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fdc7803e510 0x7fdc780409d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdc74006fd0 tx=0x7fdc74006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:22.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:22.831+0000 7fdc85ffb700 1 -- 172.21.15.33:0/173883986 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdc7c013070 con 0x7fdc88104150 2024-05-15T13:50:23.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:23.049+0000 7fdc8f22e700 1 -- 172.21.15.33:0/173883986 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi052", "target": ["mon-mgr", ""]}) -- 0x7fdc88199dd0 con 0x7fdc7803e510 2024-05-15T13:50:23.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:23 smithi033 ceph-mon[31650]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi052", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:50:24.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:24 smithi033 ceph-mon[31650]: Deploying cephadm binary to smithi052 2024-05-15T13:50:26.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:25 smithi033 ceph-mon[31650]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:50:27.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:27.316+0000 7fdc85ffb700 1 -- 172.21.15.33:0/173883986 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fdc88199dd0 con 0x7fdc7803e510 2024-05-15T13:50:27.319 INFO:teuthology.orchestra.run.smithi033.stdout:Added host 'smithi052' with addr '172.21.15.52' 2024-05-15T13:50:27.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:27.319+0000 7fdc8f22e700 1 -- 172.21.15.33:0/173883986 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fdc7803e510 msgr2=0x7fdc780409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:27.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:27.319+0000 7fdc8f22e700 1 --2- 172.21.15.33:0/173883986 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fdc7803e510 0x7fdc780409d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdc74006fd0 tx=0x7fdc74006e40 comp rx=0 tx=0).stop 2024-05-15T13:50:27.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:27.319+0000 7fdc8f22e700 1 -- 172.21.15.33:0/173883986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc88104150 msgr2=0x7fdc8819e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:27.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:27.319+0000 7fdc8f22e700 1 --2- 172.21.15.33:0/173883986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc88104150 0x7fdc8819e6e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c0056d0 tx=0x7fdc7c0057b0 comp rx=0 tx=0).stop 2024-05-15T13:50:27.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:27.319+0000 7fdc8f22e700 1 -- 172.21.15.33:0/173883986 shutdown_connections 2024-05-15T13:50:27.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:27.319+0000 7fdc8f22e700 1 --2- 172.21.15.33:0/173883986 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fdc7803e510 0x7fdc780409d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:27.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:27.319+0000 7fdc8f22e700 1 --2- 172.21.15.33:0/173883986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc88104150 0x7fdc8819e6e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:27.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:27.319+0000 7fdc8f22e700 1 -- 172.21.15.33:0/173883986 >> 172.21.15.33:0/173883986 conn(0x7fdc880fd9a0 msgr2=0x7fdc88104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:50:27.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:27.319+0000 7fdc8f22e700 1 -- 172.21.15.33:0/173883986 shutdown_connections 2024-05-15T13:50:27.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:27.319+0000 7fdc8f22e700 1 -- 172.21.15.33:0/173883986 wait complete. 2024-05-15T13:50:27.988 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-15T13:50:28.240 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:50:28.569 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:28 smithi033 ceph-mon[31650]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:50:28.569 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:28 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:28.569 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:28 smithi033 ceph-mon[31650]: Added host smithi052 2024-05-15T13:50:28.569 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:28 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:50:28.569 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:28 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:29.622 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:29 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:29.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:29.851+0000 7f3bfd2d6700 1 -- 172.21.15.33:0/1440251427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bf81020f0 msgr2=0x7f3bf81024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:29.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:29.851+0000 7f3bfd2d6700 1 --2- 172.21.15.33:0/1440251427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bf81020f0 0x7f3bf81024d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3be8009910 tx=0x7f3be8009c20 comp rx=0 tx=0).stop 2024-05-15T13:50:29.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:29.851+0000 7f3bfd2d6700 1 -- 172.21.15.33:0/1440251427 shutdown_connections 2024-05-15T13:50:29.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:29.851+0000 7f3bfd2d6700 1 --2- 172.21.15.33:0/1440251427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bf81020f0 0x7f3bf81024d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:29.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:29.851+0000 7f3bfd2d6700 1 -- 172.21.15.33:0/1440251427 >> 172.21.15.33:0/1440251427 conn(0x7f3bf80fd9a0 msgr2=0x7f3bf80ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:50:29.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:29.851+0000 7f3bfd2d6700 1 -- 172.21.15.33:0/1440251427 shutdown_connections 2024-05-15T13:50:29.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:29.851+0000 7f3bfd2d6700 1 -- 172.21.15.33:0/1440251427 wait complete. 2024-05-15T13:50:29.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:29.852+0000 7f3bfd2d6700 1 Processor -- start 2024-05-15T13:50:29.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:29.852+0000 7f3bfd2d6700 1 -- start start 2024-05-15T13:50:29.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:29.852+0000 7f3bfd2d6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bf81020f0 0x7f3bf81a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:29.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:29.852+0000 7f3bfd2d6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf810c370 con 0x7f3bf81020f0 2024-05-15T13:50:29.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:29.852+0000 7f3bf6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bf81020f0 0x7f3bf81a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:29.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:29.852+0000 7f3bf6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bf81020f0 0x7f3bf81a0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56110/0 (socket says 172.21.15.33:56110) 2024-05-15T13:50:29.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:29.852+0000 7f3bf6ffd700 1 -- 172.21.15.33:0/2686403536 learned_addr learned my addr 172.21.15.33:0/2686403536 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:50:29.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:29.853+0000 7f3bf6ffd700 1 -- 172.21.15.33:0/2686403536 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf81a0db0 con 0x7f3bf81020f0 2024-05-15T13:50:29.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:29.853+0000 7f3bf6ffd700 1 --2- 172.21.15.33:0/2686403536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bf81020f0 0x7f3bf81a0870 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3be80056d0 tx=0x7f3be80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:29.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:29.853+0000 7f3beffff700 1 -- 172.21.15.33:0/2686403536 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3be8026070 con 0x7f3bf81020f0 2024-05-15T13:50:29.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:29.853+0000 7f3beffff700 1 -- 172.21.15.33:0/2686403536 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3be800bba0 con 0x7f3bf81020f0 2024-05-15T13:50:29.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:29.853+0000 7f3beffff700 1 -- 172.21.15.33:0/2686403536 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3be8003920 con 0x7f3bf81020f0 2024-05-15T13:50:29.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:29.853+0000 7f3bfd2d6700 1 -- 172.21.15.33:0/2686403536 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bf81a10a0 con 0x7f3bf81020f0 2024-05-15T13:50:29.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:29.853+0000 7f3bfd2d6700 1 -- 172.21.15.33:0/2686403536 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bf819aa30 con 0x7f3bf81020f0 2024-05-15T13:50:29.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:29.854+0000 7f3beffff700 1 -- 172.21.15.33:0/2686403536 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f3be8029070 con 0x7f3bf81020f0 2024-05-15T13:50:29.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:29.854+0000 7f3bfd2d6700 1 -- 172.21.15.33:0/2686403536 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bf81035d0 con 0x7f3bf81020f0 2024-05-15T13:50:29.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:29.854+0000 7f3beffff700 1 --2- 172.21.15.33:0/2686403536 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f3be403e510 0x7f3be40409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:29.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:29.854+0000 7f3beffff700 1 -- 172.21.15.33:0/2686403536 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3be801e070 con 0x7f3bf81020f0 2024-05-15T13:50:29.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:29.854+0000 7f3bf67fc700 1 --2- 172.21.15.33:0/2686403536 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f3be403e510 0x7f3be40409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:29.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:29.855+0000 7f3bf67fc700 1 --2- 172.21.15.33:0/2686403536 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f3be403e510 0x7f3be40409d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3be0006fd0 tx=0x7f3be0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:29.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:29.860+0000 7f3beffff700 1 -- 172.21.15.33:0/2686403536 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3be8013070 con 0x7f3bf81020f0 2024-05-15T13:50:30.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:30.082+0000 7f3bfd2d6700 1 -- 172.21.15.33:0/2686403536 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3bf819bf60 con 0x7f3be403e510 2024-05-15T13:50:30.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:30.083+0000 7f3beffff700 1 -- 172.21.15.33:0/2686403536 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f3bf819bf60 con 0x7f3be403e510 2024-05-15T13:50:30.086 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:50:30.086 INFO:teuthology.orchestra.run.smithi033.stdout:[{"addr": "172.21.15.33", "hostname": "smithi033", "labels": [], "status": ""}, {"addr": "172.21.15.52", "hostname": "smithi052", "labels": [], "status": ""}] 2024-05-15T13:50:30.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:30.086+0000 7f3bfd2d6700 1 -- 172.21.15.33:0/2686403536 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f3be403e510 msgr2=0x7f3be40409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:30.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:30.086+0000 7f3bfd2d6700 1 --2- 172.21.15.33:0/2686403536 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f3be403e510 0x7f3be40409d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3be0006fd0 tx=0x7f3be0006e40 comp rx=0 tx=0).stop 2024-05-15T13:50:30.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:30.086+0000 7f3bfd2d6700 1 -- 172.21.15.33:0/2686403536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bf81020f0 msgr2=0x7f3bf81a0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:30.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:30.086+0000 7f3bfd2d6700 1 --2- 172.21.15.33:0/2686403536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bf81020f0 0x7f3bf81a0870 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3be80056d0 tx=0x7f3be80057b0 comp rx=0 tx=0).stop 2024-05-15T13:50:30.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:30.086+0000 7f3bfd2d6700 1 -- 172.21.15.33:0/2686403536 shutdown_connections 2024-05-15T13:50:30.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:30.086+0000 7f3bfd2d6700 1 --2- 172.21.15.33:0/2686403536 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f3be403e510 0x7f3be40409d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:30.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:30.086+0000 7f3bfd2d6700 1 --2- 172.21.15.33:0/2686403536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bf81020f0 0x7f3bf81a0870 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:30.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:30.086+0000 7f3bfd2d6700 1 -- 172.21.15.33:0/2686403536 >> 172.21.15.33:0/2686403536 conn(0x7f3bf80fd9a0 msgr2=0x7f3bf806b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:50:30.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:30.086+0000 7f3bfd2d6700 1 -- 172.21.15.33:0/2686403536 shutdown_connections 2024-05-15T13:50:30.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:30.086+0000 7f3bfd2d6700 1 -- 172.21.15.33:0/2686403536 wait complete. 2024-05-15T13:50:30.660 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:30 smithi033 ceph-mon[31650]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:50:30.660 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:30 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:30.687 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi081 2024-05-15T13:50:30.687 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-15T13:50:30.687 DEBUG:teuthology.orchestra.run.smithi081:> dd of=/etc/ceph/ceph.conf 2024-05-15T13:50:30.707 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-15T13:50:30.707 DEBUG:teuthology.orchestra.run.smithi081:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-15T13:50:30.764 INFO:tasks.cephadm:Adding host smithi081 to orchestrator... 2024-05-15T13:50:30.764 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph orch host add smithi081 2024-05-15T13:50:31.006 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:50:31.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:31 smithi033 ceph-mon[31650]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-15T13:50:32.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:32.599+0000 7f9c38fa7700 1 -- 172.21.15.33:0/2258536156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c34104110 msgr2=0x7f9c341044f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:32.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:32.599+0000 7f9c38fa7700 1 --2- 172.21.15.33:0/2258536156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c34104110 0x7f9c341044f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9c24009910 tx=0x7f9c24009c20 comp rx=0 tx=0).stop 2024-05-15T13:50:32.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:32.599+0000 7f9c38fa7700 1 -- 172.21.15.33:0/2258536156 shutdown_connections 2024-05-15T13:50:32.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:32.599+0000 7f9c38fa7700 1 --2- 172.21.15.33:0/2258536156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c34104110 0x7f9c341044f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:32.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:32.599+0000 7f9c38fa7700 1 -- 172.21.15.33:0/2258536156 >> 172.21.15.33:0/2258536156 conn(0x7f9c340fd940 msgr2=0x7f9c340ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:50:32.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:32.599+0000 7f9c38fa7700 1 -- 172.21.15.33:0/2258536156 shutdown_connections 2024-05-15T13:50:32.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:32.599+0000 7f9c38fa7700 1 -- 172.21.15.33:0/2258536156 wait complete. 2024-05-15T13:50:32.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:32.600+0000 7f9c38fa7700 1 Processor -- start 2024-05-15T13:50:32.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:32.600+0000 7f9c38fa7700 1 -- start start 2024-05-15T13:50:32.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:32.600+0000 7f9c38fa7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c34104110 0x7f9c3419e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:32.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:32.600+0000 7f9c38fa7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3410d4b0 con 0x7f9c34104110 2024-05-15T13:50:32.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:32.600+0000 7f9c3259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c34104110 0x7f9c3419e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:32.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:32.600+0000 7f9c3259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c34104110 0x7f9c3419e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56118/0 (socket says 172.21.15.33:56118) 2024-05-15T13:50:32.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:32.600+0000 7f9c3259c700 1 -- 172.21.15.33:0/1190295594 learned_addr learned my addr 172.21.15.33:0/1190295594 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:50:32.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:32.600+0000 7f9c3259c700 1 -- 172.21.15.33:0/1190295594 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c3419ebc0 con 0x7f9c34104110 2024-05-15T13:50:32.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:32.601+0000 7f9c3259c700 1 --2- 172.21.15.33:0/1190295594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c34104110 0x7f9c3419e680 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9c24006010 tx=0x7f9c24005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:32.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:32.601+0000 7f9c237fe700 1 -- 172.21.15.33:0/1190295594 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c24026070 con 0x7f9c34104110 2024-05-15T13:50:32.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:32.601+0000 7f9c237fe700 1 -- 172.21.15.33:0/1190295594 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c2400bc00 con 0x7f9c34104110 2024-05-15T13:50:32.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:32.601+0000 7f9c38fa7700 1 -- 172.21.15.33:0/1190295594 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c3419ee50 con 0x7f9c34104110 2024-05-15T13:50:32.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:32.601+0000 7f9c237fe700 1 -- 172.21.15.33:0/1190295594 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c24003910 con 0x7f9c34104110 2024-05-15T13:50:32.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:32.601+0000 7f9c38fa7700 1 -- 172.21.15.33:0/1190295594 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c34198850 con 0x7f9c34104110 2024-05-15T13:50:32.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:32.602+0000 7f9c237fe700 1 -- 172.21.15.33:0/1190295594 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f9c24029070 con 0x7f9c34104110 2024-05-15T13:50:32.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:32.602+0000 7f9c38fa7700 1 -- 172.21.15.33:0/1190295594 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c3406bc90 con 0x7f9c34104110 2024-05-15T13:50:32.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:32.602+0000 7f9c237fe700 1 --2- 172.21.15.33:0/1190295594 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f9c1c03e510 0x7f9c1c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:32.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:32.602+0000 7f9c237fe700 1 -- 172.21.15.33:0/1190295594 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9c2401e070 con 0x7f9c34104110 2024-05-15T13:50:32.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:32.602+0000 7f9c31d9b700 1 --2- 172.21.15.33:0/1190295594 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f9c1c03e510 0x7f9c1c0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:32.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:32.603+0000 7f9c31d9b700 1 --2- 172.21.15.33:0/1190295594 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f9c1c03e510 0x7f9c1c0409d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9c28006fd0 tx=0x7f9c28006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:32.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:32.607+0000 7f9c237fe700 1 -- 172.21.15.33:0/1190295594 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9c24013070 con 0x7f9c34104110 2024-05-15T13:50:32.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:32 smithi033 ceph-mon[31650]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:50:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:32.828+0000 7f9c38fa7700 1 -- 172.21.15.33:0/1190295594 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi081", "target": ["mon-mgr", ""]}) -- 0x7f9c34064ba0 con 0x7f9c1c03e510 2024-05-15T13:50:34.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:34 smithi033 ceph-mon[31650]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi081", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:50:34.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:34 smithi033 ceph-mon[31650]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:50:34.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:34 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:34.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:34 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:34.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:34 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:34.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:34 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:34.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:34 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi052", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:50:34.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:34 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:50:34.443 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:34 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:50:34.443 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:34 smithi033 ceph-mon[31650]: Updating smithi052:/etc/ceph/ceph.conf 2024-05-15T13:50:34.443 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:34 smithi033 ceph-mon[31650]: Updating smithi052:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T13:50:34.443 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:34 smithi033 ceph-mon[31650]: Deploying cephadm binary to smithi081 2024-05-15T13:50:35.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:35 smithi033 ceph-mon[31650]: Updating smithi052:/etc/ceph/ceph.client.admin.keyring 2024-05-15T13:50:35.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:35 smithi033 ceph-mon[31650]: Updating smithi052:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-15T13:50:36.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:36 smithi033 ceph-mon[31650]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:50:37.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:37.080+0000 7f9c237fe700 1 -- 172.21.15.33:0/1190295594 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f9c34064ba0 con 0x7f9c1c03e510 2024-05-15T13:50:37.084 INFO:teuthology.orchestra.run.smithi033.stdout:Added host 'smithi081' with addr '172.21.15.81' 2024-05-15T13:50:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:37.083+0000 7f9c38fa7700 1 -- 172.21.15.33:0/1190295594 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f9c1c03e510 msgr2=0x7f9c1c0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:37.083+0000 7f9c38fa7700 1 --2- 172.21.15.33:0/1190295594 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f9c1c03e510 0x7f9c1c0409d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9c28006fd0 tx=0x7f9c28006e40 comp rx=0 tx=0).stop 2024-05-15T13:50:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:37.083+0000 7f9c38fa7700 1 -- 172.21.15.33:0/1190295594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c34104110 msgr2=0x7f9c3419e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:37.083+0000 7f9c38fa7700 1 --2- 172.21.15.33:0/1190295594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c34104110 0x7f9c3419e680 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9c24006010 tx=0x7f9c24005780 comp rx=0 tx=0).stop 2024-05-15T13:50:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:37.083+0000 7f9c38fa7700 1 -- 172.21.15.33:0/1190295594 shutdown_connections 2024-05-15T13:50:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:37.083+0000 7f9c38fa7700 1 --2- 172.21.15.33:0/1190295594 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f9c1c03e510 0x7f9c1c0409d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:37.083+0000 7f9c38fa7700 1 --2- 172.21.15.33:0/1190295594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c34104110 0x7f9c3419e680 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:37.083+0000 7f9c38fa7700 1 -- 172.21.15.33:0/1190295594 >> 172.21.15.33:0/1190295594 conn(0x7f9c340fd940 msgr2=0x7f9c34104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:50:37.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:37.084+0000 7f9c38fa7700 1 -- 172.21.15.33:0/1190295594 shutdown_connections 2024-05-15T13:50:37.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:37.084+0000 7f9c38fa7700 1 -- 172.21.15.33:0/1190295594 wait complete. 2024-05-15T13:50:37.663 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:37 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:37.663 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:37 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:37.663 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:37 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:37.663 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:37 smithi033 ceph-mon[31650]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:50:37.663 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:37 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:37.664 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:37 smithi033 ceph-mon[31650]: Added host smithi081 2024-05-15T13:50:37.664 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:37 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:50:37.665 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-15T13:50:37.914 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:50:38.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:38 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:38.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:38 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:39.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.338+0000 7f337bed3700 1 -- 172.21.15.33:0/1269489525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33741031b0 msgr2=0x7f3374103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:39.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.338+0000 7f337bed3700 1 --2- 172.21.15.33:0/1269489525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33741031b0 0x7f3374103590 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f337000b0a0 tx=0x7f337000b3b0 comp rx=0 tx=0).stop 2024-05-15T13:50:39.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.338+0000 7f337bed3700 1 -- 172.21.15.33:0/1269489525 shutdown_connections 2024-05-15T13:50:39.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.338+0000 7f337bed3700 1 --2- 172.21.15.33:0/1269489525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33741031b0 0x7f3374103590 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:39.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.338+0000 7f337bed3700 1 -- 172.21.15.33:0/1269489525 >> 172.21.15.33:0/1269489525 conn(0x7f33740fea80 msgr2=0x7f3374100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:50:39.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.339+0000 7f337bed3700 1 -- 172.21.15.33:0/1269489525 shutdown_connections 2024-05-15T13:50:39.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.339+0000 7f337bed3700 1 -- 172.21.15.33:0/1269489525 wait complete. 2024-05-15T13:50:39.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.339+0000 7f337bed3700 1 Processor -- start 2024-05-15T13:50:39.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.339+0000 7f337bed3700 1 -- start start 2024-05-15T13:50:39.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.339+0000 7f337bed3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33741031b0 0x7f3374075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:39.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.339+0000 7f337bed3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f337410d430 con 0x7f33741031b0 2024-05-15T13:50:39.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.340+0000 7f3379c6f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33741031b0 0x7f3374075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:39.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.340+0000 7f3379c6f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33741031b0 0x7f3374075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37992/0 (socket says 172.21.15.33:37992) 2024-05-15T13:50:39.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.340+0000 7f3379c6f700 1 -- 172.21.15.33:0/486973829 learned_addr learned my addr 172.21.15.33:0/486973829 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:50:39.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.340+0000 7f3379c6f700 1 -- 172.21.15.33:0/486973829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3374077730 con 0x7f33741031b0 2024-05-15T13:50:39.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.340+0000 7f3379c6f700 1 --2- 172.21.15.33:0/486973829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33741031b0 0x7f3374075840 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3370006130 tx=0x7f33700097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:39.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.340+0000 7f336affd700 1 -- 172.21.15.33:0/486973829 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3370021020 con 0x7f33741031b0 2024-05-15T13:50:39.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.340+0000 7f336affd700 1 -- 172.21.15.33:0/486973829 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33700079b0 con 0x7f33741031b0 2024-05-15T13:50:39.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.340+0000 7f337bed3700 1 -- 172.21.15.33:0/486973829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3374075de0 con 0x7f33741031b0 2024-05-15T13:50:39.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.340+0000 7f336affd700 1 -- 172.21.15.33:0/486973829 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3370017040 con 0x7f33741031b0 2024-05-15T13:50:39.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.340+0000 7f337bed3700 1 -- 172.21.15.33:0/486973829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33740762a0 con 0x7f33741031b0 2024-05-15T13:50:39.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.341+0000 7f336affd700 1 -- 172.21.15.33:0/486973829 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f3370024020 con 0x7f33741031b0 2024-05-15T13:50:39.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.341+0000 7f337bed3700 1 -- 172.21.15.33:0/486973829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3374104690 con 0x7f33741031b0 2024-05-15T13:50:39.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.341+0000 7f336affd700 1 --2- 172.21.15.33:0/486973829 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f336003e570 0x7f3360040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:39.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.341+0000 7f336affd700 1 -- 172.21.15.33:0/486973829 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f337000f050 con 0x7f33741031b0 2024-05-15T13:50:39.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.341+0000 7f337946e700 1 --2- 172.21.15.33:0/486973829 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f336003e570 0x7f3360040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:39.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.342+0000 7f337946e700 1 --2- 172.21.15.33:0/486973829 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f336003e570 0x7f3360040a30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3364006fd0 tx=0x7f3364006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:39.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.347+0000 7f336affd700 1 -- 172.21.15.33:0/486973829 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f337002b020 con 0x7f33741031b0 2024-05-15T13:50:39.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.564+0000 7f337bed3700 1 -- 172.21.15.33:0/486973829 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3374076de0 con 0x7f336003e570 2024-05-15T13:50:39.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.566+0000 7f336affd700 1 -- 172.21.15.33:0/486973829 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+238 (secure 0 0 0) 0x7f3374076de0 con 0x7f336003e570 2024-05-15T13:50:39.568 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:50:39.568 INFO:teuthology.orchestra.run.smithi033.stdout:[{"addr": "172.21.15.33", "hostname": "smithi033", "labels": [], "status": ""}, {"addr": "172.21.15.52", "hostname": "smithi052", "labels": [], "status": ""}, {"addr": "172.21.15.81", "hostname": "smithi081", "labels": [], "status": ""}] 2024-05-15T13:50:39.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.568+0000 7f337bed3700 1 -- 172.21.15.33:0/486973829 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f336003e570 msgr2=0x7f3360040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:39.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.568+0000 7f337bed3700 1 --2- 172.21.15.33:0/486973829 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f336003e570 0x7f3360040a30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3364006fd0 tx=0x7f3364006e40 comp rx=0 tx=0).stop 2024-05-15T13:50:39.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.568+0000 7f337bed3700 1 -- 172.21.15.33:0/486973829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33741031b0 msgr2=0x7f3374075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:39.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.568+0000 7f337bed3700 1 --2- 172.21.15.33:0/486973829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33741031b0 0x7f3374075840 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3370006130 tx=0x7f33700097d0 comp rx=0 tx=0).stop 2024-05-15T13:50:39.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.568+0000 7f337bed3700 1 -- 172.21.15.33:0/486973829 shutdown_connections 2024-05-15T13:50:39.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.568+0000 7f337bed3700 1 --2- 172.21.15.33:0/486973829 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f336003e570 0x7f3360040a30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:39.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.568+0000 7f337bed3700 1 --2- 172.21.15.33:0/486973829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33741031b0 0x7f3374075840 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:39.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.568+0000 7f337bed3700 1 -- 172.21.15.33:0/486973829 >> 172.21.15.33:0/486973829 conn(0x7f33740fea80 msgr2=0x7f3374104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:50:39.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.568+0000 7f337bed3700 1 -- 172.21.15.33:0/486973829 shutdown_connections 2024-05-15T13:50:39.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:39.568+0000 7f337bed3700 1 -- 172.21.15.33:0/486973829 wait complete. 2024-05-15T13:50:39.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:39 smithi033 ceph-mon[31650]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:50:40.218 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-15T13:50:40.218 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph osd crush tunables default 2024-05-15T13:50:40.465 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:50:40.906 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:40 smithi033 ceph-mon[31650]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-15T13:50:40.906 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:40 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:41.884 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:41 smithi033 ceph-mon[31650]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:50:42.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.105+0000 7ff82be05700 1 -- 172.21.15.33:0/1357744007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff82406abc0 msgr2=0x7ff82406afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:42.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.105+0000 7ff82be05700 1 --2- 172.21.15.33:0/1357744007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff82406abc0 0x7ff82406afa0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff82000b0a0 tx=0x7ff82000b3b0 comp rx=0 tx=0).stop 2024-05-15T13:50:42.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.105+0000 7ff82be05700 1 -- 172.21.15.33:0/1357744007 shutdown_connections 2024-05-15T13:50:42.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.105+0000 7ff82be05700 1 --2- 172.21.15.33:0/1357744007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff82406abc0 0x7ff82406afa0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:42.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.105+0000 7ff82be05700 1 -- 172.21.15.33:0/1357744007 >> 172.21.15.33:0/1357744007 conn(0x7ff8240779d0 msgr2=0x7ff824077de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:50:42.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.105+0000 7ff82be05700 1 -- 172.21.15.33:0/1357744007 shutdown_connections 2024-05-15T13:50:42.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.105+0000 7ff82be05700 1 -- 172.21.15.33:0/1357744007 wait complete. 2024-05-15T13:50:42.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.106+0000 7ff82be05700 1 Processor -- start 2024-05-15T13:50:42.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.106+0000 7ff82be05700 1 -- start start 2024-05-15T13:50:42.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.106+0000 7ff82be05700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff82406abc0 0x7ff824101f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:42.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.106+0000 7ff82be05700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff82410d430 con 0x7ff82406abc0 2024-05-15T13:50:42.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.106+0000 7ff829ba1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff82406abc0 0x7ff824101f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:42.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.106+0000 7ff829ba1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff82406abc0 0x7ff824101f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37998/0 (socket says 172.21.15.33:37998) 2024-05-15T13:50:42.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.106+0000 7ff829ba1700 1 -- 172.21.15.33:0/266324991 learned_addr learned my addr 172.21.15.33:0/266324991 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:50:42.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.106+0000 7ff829ba1700 1 -- 172.21.15.33:0/266324991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff824103e50 con 0x7ff82406abc0 2024-05-15T13:50:42.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.107+0000 7ff829ba1700 1 --2- 172.21.15.33:0/266324991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff82406abc0 0x7ff824101f60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff820006130 tx=0x7ff8200097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:42.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.107+0000 7ff81affd700 1 -- 172.21.15.33:0/266324991 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff820021020 con 0x7ff82406abc0 2024-05-15T13:50:42.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.107+0000 7ff81affd700 1 -- 172.21.15.33:0/266324991 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8200079b0 con 0x7ff82406abc0 2024-05-15T13:50:42.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.107+0000 7ff82be05700 1 -- 172.21.15.33:0/266324991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff824102500 con 0x7ff82406abc0 2024-05-15T13:50:42.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.107+0000 7ff81affd700 1 -- 172.21.15.33:0/266324991 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff820017040 con 0x7ff82406abc0 2024-05-15T13:50:42.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.107+0000 7ff82be05700 1 -- 172.21.15.33:0/266324991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8241029c0 con 0x7ff82406abc0 2024-05-15T13:50:42.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.108+0000 7ff82be05700 1 -- 172.21.15.33:0/266324991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8241046c0 con 0x7ff82406abc0 2024-05-15T13:50:42.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.108+0000 7ff81affd700 1 -- 172.21.15.33:0/266324991 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7ff820024020 con 0x7ff82406abc0 2024-05-15T13:50:42.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.108+0000 7ff81affd700 1 --2- 172.21.15.33:0/266324991 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff81003e570 0x7ff810040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:42.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.108+0000 7ff81affd700 1 -- 172.21.15.33:0/266324991 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff82000f050 con 0x7ff82406abc0 2024-05-15T13:50:42.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.108+0000 7ff8293a0700 1 --2- 172.21.15.33:0/266324991 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff81003e570 0x7ff810040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:42.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.109+0000 7ff8293a0700 1 --2- 172.21.15.33:0/266324991 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff81003e570 0x7ff810040a30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff814006fd0 tx=0x7ff814006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:42.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.113+0000 7ff81affd700 1 -- 172.21.15.33:0/266324991 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff82002b020 con 0x7ff82406abc0 2024-05-15T13:50:42.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.333+0000 7ff82be05700 1 -- 172.21.15.33:0/266324991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7ff824103490 con 0x7ff82406abc0 2024-05-15T13:50:42.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.682+0000 7ff81affd700 1 -- 172.21.15.33:0/266324991 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7ff82001b3f0 con 0x7ff82406abc0 2024-05-15T13:50:42.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.684+0000 7ff82be05700 1 -- 172.21.15.33:0/266324991 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff81003e570 msgr2=0x7ff810040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:42.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.684+0000 7ff82be05700 1 --2- 172.21.15.33:0/266324991 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff81003e570 0x7ff810040a30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff814006fd0 tx=0x7ff814006e40 comp rx=0 tx=0).stop 2024-05-15T13:50:42.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.684+0000 7ff82be05700 1 -- 172.21.15.33:0/266324991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff82406abc0 msgr2=0x7ff824101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:42.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.684+0000 7ff82be05700 1 --2- 172.21.15.33:0/266324991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff82406abc0 0x7ff824101f60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff820006130 tx=0x7ff8200097d0 comp rx=0 tx=0).stop 2024-05-15T13:50:42.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.684+0000 7ff82be05700 1 -- 172.21.15.33:0/266324991 shutdown_connections 2024-05-15T13:50:42.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.684+0000 7ff82be05700 1 --2- 172.21.15.33:0/266324991 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff81003e570 0x7ff810040a30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:42.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.684+0000 7ff82be05700 1 --2- 172.21.15.33:0/266324991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff82406abc0 0x7ff824101f60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:42.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.684+0000 7ff82be05700 1 -- 172.21.15.33:0/266324991 >> 172.21.15.33:0/266324991 conn(0x7ff8240779d0 msgr2=0x7ff824104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:50:42.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.685+0000 7ff82be05700 1 -- 172.21.15.33:0/266324991 shutdown_connections 2024-05-15T13:50:42.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:50:42.685+0000 7ff82be05700 1 -- 172.21.15.33:0/266324991 wait complete. 2024-05-15T13:50:42.688 INFO:teuthology.orchestra.run.smithi033.stderr:adjusted tunables profile to default 2024-05-15T13:50:42.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:42 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/266324991' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-15T13:50:42.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:42 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:42.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:42 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:42.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:42 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:42.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:42 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:42.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:42 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi081", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:50:42.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:42 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:50:42.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:42 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:50:43.372 INFO:tasks.cephadm:Adding mon.a on smithi033 2024-05-15T13:50:43.372 INFO:tasks.cephadm:Adding mon.b on smithi052 2024-05-15T13:50:43.372 INFO:tasks.cephadm:Adding mon.c on smithi081 2024-05-15T13:50:43.372 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph orch apply mon '3;smithi033:172.21.15.33=a;smithi052:172.21.15.52=b;smithi081:172.21.15.81=c' 2024-05-15T13:50:43.603 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T13:50:43.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:43 smithi033 ceph-mon[31650]: Updating smithi081:/etc/ceph/ceph.conf 2024-05-15T13:50:43.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:43 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/266324991' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-15T13:50:43.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:43 smithi033 ceph-mon[31650]: osdmap e4: 0 total, 0 up, 0 in 2024-05-15T13:50:43.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:43 smithi033 ceph-mon[31650]: Updating smithi081:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T13:50:43.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:43 smithi033 ceph-mon[31650]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:50:43.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:43 smithi033 ceph-mon[31650]: Updating smithi081:/etc/ceph/ceph.client.admin.keyring 2024-05-15T13:50:43.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:43 smithi033 ceph-mon[31650]: Updating smithi081:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-15T13:50:43.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:43 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:43.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:43 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:43.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:43 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:44.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:44.883+0000 7f615cbaf700 1 -- 172.21.15.81:0/4039195117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61581020b0 msgr2=0x7f6158102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:44.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:44.883+0000 7f615cbaf700 1 --2- 172.21.15.81:0/4039195117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61581020b0 0x7f6158102490 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f614c009910 tx=0x7f614c009c20 comp rx=0 tx=0).stop 2024-05-15T13:50:44.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:44.883+0000 7f615cbaf700 1 -- 172.21.15.81:0/4039195117 shutdown_connections 2024-05-15T13:50:44.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:44.883+0000 7f615cbaf700 1 --2- 172.21.15.81:0/4039195117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61581020b0 0x7f6158102490 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:44.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:44.883+0000 7f615cbaf700 1 -- 172.21.15.81:0/4039195117 >> 172.21.15.81:0/4039195117 conn(0x7f61580fd960 msgr2=0x7f61580ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:50:44.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:44.883+0000 7f615cbaf700 1 -- 172.21.15.81:0/4039195117 shutdown_connections 2024-05-15T13:50:44.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:44.883+0000 7f615cbaf700 1 -- 172.21.15.81:0/4039195117 wait complete. 2024-05-15T13:50:44.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:44.884+0000 7f615cbaf700 1 Processor -- start 2024-05-15T13:50:44.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:44.884+0000 7f615cbaf700 1 -- start start 2024-05-15T13:50:44.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:44.884+0000 7f615cbaf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61581020b0 0x7f615819e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:44.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:44.884+0000 7f615cbaf700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f615810c330 con 0x7f61581020b0 2024-05-15T13:50:44.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:44.884+0000 7f615659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61581020b0 0x7f615819e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:44.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:44.884+0000 7f615659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61581020b0 0x7f615819e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.81:34194/0 (socket says 172.21.15.81:34194) 2024-05-15T13:50:44.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:44.884+0000 7f615659c700 1 -- 172.21.15.81:0/3269702704 learned_addr learned my addr 172.21.15.81:0/3269702704 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-15T13:50:44.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:44.885+0000 7f615659c700 1 -- 172.21.15.81:0/3269702704 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f615819eb80 con 0x7f61581020b0 2024-05-15T13:50:44.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:44.885+0000 7f615659c700 1 --2- 172.21.15.81:0/3269702704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61581020b0 0x7f615819e640 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f614c0056d0 tx=0x7f614c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:44.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:44.885+0000 7f61477fe700 1 -- 172.21.15.81:0/3269702704 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f614c026070 con 0x7f61581020b0 2024-05-15T13:50:44.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:44.885+0000 7f615cbaf700 1 -- 172.21.15.81:0/3269702704 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f615819ee70 con 0x7f61581020b0 2024-05-15T13:50:44.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:44.885+0000 7f61477fe700 1 -- 172.21.15.81:0/3269702704 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f614c00bba0 con 0x7f61581020b0 2024-05-15T13:50:44.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:44.885+0000 7f61477fe700 1 -- 172.21.15.81:0/3269702704 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f614c003920 con 0x7f61581020b0 2024-05-15T13:50:44.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:44.885+0000 7f615cbaf700 1 -- 172.21.15.81:0/3269702704 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61581987c0 con 0x7f61581020b0 2024-05-15T13:50:44.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:44.886+0000 7f615cbaf700 1 -- 172.21.15.81:0/3269702704 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6158103590 con 0x7f61581020b0 2024-05-15T13:50:44.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:44.886+0000 7f61477fe700 1 -- 172.21.15.81:0/3269702704 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f614c029070 con 0x7f61581020b0 2024-05-15T13:50:44.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:44.886+0000 7f61477fe700 1 --2- 172.21.15.81:0/3269702704 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f614003e510 0x7f61400409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:44.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:44.886+0000 7f61477fe700 1 -- 172.21.15.81:0/3269702704 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f614c01e070 con 0x7f61581020b0 2024-05-15T13:50:44.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:44.886+0000 7f6155d9b700 1 --2- 172.21.15.81:0/3269702704 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f614003e510 0x7f61400409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:44.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:44.887+0000 7f6155d9b700 1 --2- 172.21.15.81:0/3269702704 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f614003e510 0x7f61400409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6148006fd0 tx=0x7f6148006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:44.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:44.892+0000 7f61477fe700 1 -- 172.21.15.81:0/3269702704 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f614c013070 con 0x7f61581020b0 2024-05-15T13:50:45.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:45.104+0000 7f615cbaf700 1 -- 172.21.15.81:0/3269702704 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi033:172.21.15.33=a;smithi052:172.21.15.52=b;smithi081:172.21.15.81=c", "target": ["mon-mgr", ""]}) -- 0x7f6158199da0 con 0x7f614003e510 2024-05-15T13:50:45.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:45.111+0000 7f61477fe700 1 -- 172.21.15.81:0/3269702704 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6158199da0 con 0x7f614003e510 2024-05-15T13:50:45.113 INFO:teuthology.orchestra.run.smithi081.stdout:Scheduled mon update... 2024-05-15T13:50:45.115 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:45.113+0000 7f615cbaf700 1 -- 172.21.15.81:0/3269702704 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f614003e510 msgr2=0x7f61400409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:45.115 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:45.113+0000 7f615cbaf700 1 --2- 172.21.15.81:0/3269702704 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f614003e510 0x7f61400409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6148006fd0 tx=0x7f6148006e40 comp rx=0 tx=0).stop 2024-05-15T13:50:45.115 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:45.113+0000 7f615cbaf700 1 -- 172.21.15.81:0/3269702704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61581020b0 msgr2=0x7f615819e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:45.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:45.113+0000 7f615cbaf700 1 --2- 172.21.15.81:0/3269702704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61581020b0 0x7f615819e640 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f614c0056d0 tx=0x7f614c0057b0 comp rx=0 tx=0).stop 2024-05-15T13:50:45.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:45.113+0000 7f615cbaf700 1 -- 172.21.15.81:0/3269702704 shutdown_connections 2024-05-15T13:50:45.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:45.113+0000 7f615cbaf700 1 --2- 172.21.15.81:0/3269702704 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f614003e510 0x7f61400409d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:45.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:45.113+0000 7f615cbaf700 1 --2- 172.21.15.81:0/3269702704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61581020b0 0x7f615819e640 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:45.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:45.113+0000 7f615cbaf700 1 -- 172.21.15.81:0/3269702704 >> 172.21.15.81:0/3269702704 conn(0x7f61580fd960 msgr2=0x7f615806b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:50:45.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:45.113+0000 7f615cbaf700 1 -- 172.21.15.81:0/3269702704 shutdown_connections 2024-05-15T13:50:45.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:45.113+0000 7f615cbaf700 1 -- 172.21.15.81:0/3269702704 wait complete. 2024-05-15T13:50:45.700 DEBUG:teuthology.orchestra.run.smithi052:mon.b> sudo journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mon.b.service 2024-05-15T13:50:45.702 DEBUG:teuthology.orchestra.run.smithi081:mon.c> sudo journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mon.c.service 2024-05-15T13:50:45.705 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-15T13:50:45.705 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-15T13:50:46.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:46 smithi033 ceph-mon[31650]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:50:46.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:46 smithi033 ceph-mon[31650]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi033:172.21.15.33=a;smithi052:172.21.15.52=b;smithi081:172.21.15.81=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:50:46.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:46 smithi033 ceph-mon[31650]: Saving service mon spec with placement smithi033:172.21.15.33=a;smithi052:172.21.15.52=b;smithi081:172.21.15.81=c;count:3 2024-05-15T13:50:46.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:46 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:46.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:46 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:50:46.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:46 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:50:46.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:46 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:50:46.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:46 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:46.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:46 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-15T13:50:46.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:46 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:50:46.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:46 smithi033 ceph-mon[31650]: Deploying daemon mon.c on smithi081 2024-05-15T13:50:46.857 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T13:50:48.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:48 smithi033 ceph-mon[31650]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:50:49.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:49.694+0000 7f7e87641700 1 -- 172.21.15.81:0/2368713024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e801031b0 msgr2=0x7f7e80103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:49.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:49.694+0000 7f7e87641700 1 --2- 172.21.15.81:0/2368713024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e801031b0 0x7f7e80103590 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c009910 tx=0x7f7e7c009c20 comp rx=0 tx=0).stop 2024-05-15T13:50:49.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:49.694+0000 7f7e87641700 1 -- 172.21.15.81:0/2368713024 shutdown_connections 2024-05-15T13:50:49.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:49.694+0000 7f7e87641700 1 --2- 172.21.15.81:0/2368713024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e801031b0 0x7f7e80103590 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:49.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:49.694+0000 7f7e87641700 1 -- 172.21.15.81:0/2368713024 >> 172.21.15.81:0/2368713024 conn(0x7f7e800fea80 msgr2=0x7f7e80100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:50:49.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:49.694+0000 7f7e87641700 1 -- 172.21.15.81:0/2368713024 shutdown_connections 2024-05-15T13:50:49.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:49.694+0000 7f7e87641700 1 -- 172.21.15.81:0/2368713024 wait complete. 2024-05-15T13:50:49.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:49.695+0000 7f7e87641700 1 Processor -- start 2024-05-15T13:50:49.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:49.695+0000 7f7e87641700 1 -- start start 2024-05-15T13:50:49.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:49.695+0000 7f7e87641700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e801031b0 0x7f7e80075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:49.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:49.695+0000 7f7e87641700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8010d430 con 0x7f7e801031b0 2024-05-15T13:50:49.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:49.695+0000 7f7e853dd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e801031b0 0x7f7e80075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:49.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:49.695+0000 7f7e853dd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e801031b0 0x7f7e80075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.81:34198/0 (socket says 172.21.15.81:34198) 2024-05-15T13:50:49.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:49.695+0000 7f7e853dd700 1 -- 172.21.15.81:0/281402504 learned_addr learned my addr 172.21.15.81:0/281402504 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-15T13:50:49.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:49.696+0000 7f7e853dd700 1 -- 172.21.15.81:0/281402504 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e80077730 con 0x7f7e801031b0 2024-05-15T13:50:49.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:49.696+0000 7f7e853dd700 1 --2- 172.21.15.81:0/281402504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e801031b0 0x7f7e80075840 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c006010 tx=0x7f7e7c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:49.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:49.696+0000 7f7e767fc700 1 -- 172.21.15.81:0/281402504 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e7c026070 con 0x7f7e801031b0 2024-05-15T13:50:49.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:49.696+0000 7f7e87641700 1 -- 172.21.15.81:0/281402504 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e80075de0 con 0x7f7e801031b0 2024-05-15T13:50:49.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:49.696+0000 7f7e767fc700 1 -- 172.21.15.81:0/281402504 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e7c00bc00 con 0x7f7e801031b0 2024-05-15T13:50:49.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:49.696+0000 7f7e767fc700 1 -- 172.21.15.81:0/281402504 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e7c003910 con 0x7f7e801031b0 2024-05-15T13:50:49.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:49.696+0000 7f7e87641700 1 -- 172.21.15.81:0/281402504 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e800762a0 con 0x7f7e801031b0 2024-05-15T13:50:49.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:49.697+0000 7f7e87641700 1 -- 172.21.15.81:0/281402504 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e80104690 con 0x7f7e801031b0 2024-05-15T13:50:49.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:49.697+0000 7f7e767fc700 1 -- 172.21.15.81:0/281402504 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f7e7c029070 con 0x7f7e801031b0 2024-05-15T13:50:49.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:49.697+0000 7f7e767fc700 1 --2- 172.21.15.81:0/281402504 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f7e6c03e510 0x7f7e6c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:49.702 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:49.697+0000 7f7e767fc700 1 -- 172.21.15.81:0/281402504 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7e7c01e070 con 0x7f7e801031b0 2024-05-15T13:50:49.702 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:49.698+0000 7f7e84bdc700 1 --2- 172.21.15.81:0/281402504 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f7e6c03e510 0x7f7e6c0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:49.702 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:49.698+0000 7f7e84bdc700 1 --2- 172.21.15.81:0/281402504 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f7e6c03e510 0x7f7e6c0409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7e70006fd0 tx=0x7f7e70006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:49.705 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:49.703+0000 7f7e767fc700 1 -- 172.21.15.81:0/281402504 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7e7c013070 con 0x7f7e801031b0 2024-05-15T13:50:49.844 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:49.841+0000 7f7e767fc700 1 -- 172.21.15.81:0/281402504 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f7e7c052ec0 con 0x7f7e801031b0 2024-05-15T13:50:50.013 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:50.010+0000 7f7e87641700 1 -- 172.21.15.81:0/281402504 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f7e80076e10 con 0x7f7e801031b0 2024-05-15T13:50:54.855 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:54.852+0000 7f7e767fc700 1 -- 172.21.15.81:0/281402504 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7f7e7c030090 con 0x7f7e801031b0 2024-05-15T13:50:54.855 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-15T13:50:54.855 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":2,"fsid":"bca78ca4-12c1-11ef-bc98-c7b262605968","modified":"2024-05-15T13:50:49.841294Z","created":"2024-05-15T13:49:07.404471Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:3300","nonce":0},{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-15T13:50:54.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:54.856+0000 7f7e87641700 1 -- 172.21.15.81:0/281402504 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f7e6c03e510 msgr2=0x7f7e6c0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:54.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:54.856+0000 7f7e87641700 1 --2- 172.21.15.81:0/281402504 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f7e6c03e510 0x7f7e6c0409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7e70006fd0 tx=0x7f7e70006e40 comp rx=0 tx=0).stop 2024-05-15T13:50:54.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:54.856+0000 7f7e87641700 1 -- 172.21.15.81:0/281402504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e801031b0 msgr2=0x7f7e80075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:54.858 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:54.856+0000 7f7e87641700 1 --2- 172.21.15.81:0/281402504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e801031b0 0x7f7e80075840 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c006010 tx=0x7f7e7c005780 comp rx=0 tx=0).stop 2024-05-15T13:50:54.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:54.856+0000 7f7e87641700 1 -- 172.21.15.81:0/281402504 shutdown_connections 2024-05-15T13:50:54.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:54.856+0000 7f7e87641700 1 --2- 172.21.15.81:0/281402504 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f7e6c03e510 0x7f7e6c0409d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:54.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:54.856+0000 7f7e87641700 1 --2- 172.21.15.81:0/281402504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e801031b0 0x7f7e80075840 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:54.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:54.856+0000 7f7e87641700 1 -- 172.21.15.81:0/281402504 >> 172.21.15.81:0/281402504 conn(0x7f7e800fea80 msgr2=0x7f7e80104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:50:54.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:54.856+0000 7f7e87641700 1 -- 172.21.15.81:0/281402504 shutdown_connections 2024-05-15T13:50:54.859 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:54.856+0000 7f7e87641700 1 -- 172.21.15.81:0/281402504 wait complete. 2024-05-15T13:50:54.859 INFO:teuthology.orchestra.run.smithi081.stderr:dumped monmap epoch 2 2024-05-15T13:50:55.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:50:54 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:55.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:54 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-15T13:50:55.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:54 smithi033 ceph-mon[31650]: mon.a calling monitor election 2024-05-15T13:50:55.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:54 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-15T13:50:55.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:54 smithi033 ceph-mon[31650]: from='client.? 172.21.15.81:0/281402504' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T13:50:55.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:54 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-15T13:50:55.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:54 smithi033 ceph-mon[31650]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:50:55.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:54 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-15T13:50:55.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:54 smithi033 ceph-mon[31650]: mon.c calling monitor election 2024-05-15T13:50:55.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:54 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-15T13:50:55.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:54 smithi033 ceph-mon[31650]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:50:55.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:54 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-15T13:50:55.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:54 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-15T13:50:55.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:54 smithi033 ceph-mon[31650]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-15T13:50:55.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:54 smithi033 ceph-mon[31650]: monmap epoch 2 2024-05-15T13:50:55.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:54 smithi033 ceph-mon[31650]: fsid bca78ca4-12c1-11ef-bc98-c7b262605968 2024-05-15T13:50:55.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:54 smithi033 ceph-mon[31650]: last_changed 2024-05-15T13:50:49.841294+0000 2024-05-15T13:50:55.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:54 smithi033 ceph-mon[31650]: created 2024-05-15T13:49:07.404471+0000 2024-05-15T13:50:55.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:54 smithi033 ceph-mon[31650]: min_mon_release 19 (squid) 2024-05-15T13:50:55.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:54 smithi033 ceph-mon[31650]: election_strategy: 1 2024-05-15T13:50:55.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:54 smithi033 ceph-mon[31650]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-05-15T13:50:55.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:54 smithi033 ceph-mon[31650]: 1: [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] mon.c 2024-05-15T13:50:55.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:54 smithi033 ceph-mon[31650]: fsmap 2024-05-15T13:50:55.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:54 smithi033 ceph-mon[31650]: osdmap e4: 0 total, 0 up, 0 in 2024-05-15T13:50:55.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:54 smithi033 ceph-mon[31650]: mgrmap e13: x(active, since 49s) 2024-05-15T13:50:55.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:54 smithi033 ceph-mon[31650]: overall HEALTH_OK 2024-05-15T13:50:55.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:54 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:55.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:54 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:56.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:55 smithi033 ceph-mon[31650]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:50:56.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:55 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:56.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:55 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-15T13:50:56.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:55 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:50:56.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:55 smithi033 ceph-mon[31650]: Deploying daemon mon.b on smithi052 2024-05-15T13:50:56.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:55 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-15T13:50:56.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:50:55 smithi081 ceph-mon[36434]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:50:56.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:50:55 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:56.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:50:55 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-15T13:50:56.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:50:55 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:50:56.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:50:55 smithi081 ceph-mon[36434]: Deploying daemon mon.b on smithi052 2024-05-15T13:50:56.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:50:55 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-15T13:50:56.444 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-15T13:50:56.444 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-15T13:50:56.682 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.c/config 2024-05-15T13:50:58.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.034+0000 7fbaf36be700 1 -- 172.21.15.81:0/1383021516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbaec102000 msgr2=0x7fbaec1023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:58.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.034+0000 7fbaf36be700 1 --2- 172.21.15.81:0/1383021516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbaec102000 0x7fbaec1023e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbae8009910 tx=0x7fbae8009c20 comp rx=0 tx=0).stop 2024-05-15T13:50:58.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.034+0000 7fbaf36be700 1 -- 172.21.15.81:0/1383021516 shutdown_connections 2024-05-15T13:50:58.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.034+0000 7fbaf36be700 1 --2- 172.21.15.81:0/1383021516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbaec102000 0x7fbaec1023e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:58.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.034+0000 7fbaf36be700 1 -- 172.21.15.81:0/1383021516 >> 172.21.15.81:0/1383021516 conn(0x7fbaec0fda80 msgr2=0x7fbaec0ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:50:58.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.034+0000 7fbaf36be700 1 -- 172.21.15.81:0/1383021516 shutdown_connections 2024-05-15T13:50:58.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.034+0000 7fbaf36be700 1 -- 172.21.15.81:0/1383021516 wait complete. 2024-05-15T13:50:58.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.035+0000 7fbaf36be700 1 Processor -- start 2024-05-15T13:50:58.039 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.035+0000 7fbaf36be700 1 -- start start 2024-05-15T13:50:58.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.035+0000 7fbaf36be700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbaec102000 0x7fbaec19f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:58.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.035+0000 7fbaf36be700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbaec19f7c0 0x7fbaec1a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:58.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.035+0000 7fbaf36be700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec113bb0 con 0x7fbaec102000 2024-05-15T13:50:58.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.035+0000 7fbaf36be700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec113a30 con 0x7fbaec19f7c0 2024-05-15T13:50:58.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.035+0000 7fbaf145a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbaec102000 0x7fbaec19f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:58.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.035+0000 7fbaf0c59700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbaec19f7c0 0x7fbaec1a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:58.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.035+0000 7fbaf145a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbaec102000 0x7fbaec19f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.81:48862/0 (socket says 172.21.15.81:48862) 2024-05-15T13:50:58.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.035+0000 7fbaf145a700 1 -- 172.21.15.81:0/3005178748 learned_addr learned my addr 172.21.15.81:0/3005178748 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-15T13:50:58.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.035+0000 7fbaf0c59700 1 -- 172.21.15.81:0/3005178748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbaec19f7c0 msgr2=0x7fbaec1a3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-15T13:50:58.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.035+0000 7fbaf0c59700 1 -- 172.21.15.81:0/3005178748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbaec19f7c0 msgr2=0x7fbaec1a3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-15T13:50:58.040 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.035+0000 7fbaf0c59700 1 --2- 172.21.15.81:0/3005178748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbaec19f7c0 0x7fbaec1a3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-15T13:50:58.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.036+0000 7fbaf0c59700 1 --2- 172.21.15.81:0/3005178748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbaec19f7c0 0x7fbaec1a3b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-15T13:50:58.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.036+0000 7fbaf145a700 1 -- 172.21.15.81:0/3005178748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbaec19f7c0 msgr2=0x7fbaec1a3b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-15T13:50:58.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.036+0000 7fbaf145a700 1 --2- 172.21.15.81:0/3005178748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbaec19f7c0 0x7fbaec1a3b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:58.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.036+0000 7fbaf145a700 1 -- 172.21.15.81:0/3005178748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaec19fe00 con 0x7fbaec102000 2024-05-15T13:50:58.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.036+0000 7fbaf145a700 1 --2- 172.21.15.81:0/3005178748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbaec102000 0x7fbaec19f280 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbae8006010 tx=0x7fbae8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:58.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.036+0000 7fbae27fc700 1 -- 172.21.15.81:0/3005178748 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fbae8005dc0 con 0x7fbaec102000 2024-05-15T13:50:58.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.036+0000 7fbaf36be700 1 -- 172.21.15.81:0/3005178748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaec1a4090 con 0x7fbaec102000 2024-05-15T13:50:58.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.036+0000 7fbae27fc700 1 -- 172.21.15.81:0/3005178748 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbae800be10 con 0x7fbaec102000 2024-05-15T13:50:58.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.036+0000 7fbae27fc700 1 -- 172.21.15.81:0/3005178748 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fbae800f460 con 0x7fbaec102000 2024-05-15T13:50:58.041 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.036+0000 7fbaf36be700 1 -- 172.21.15.81:0/3005178748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaec1a4590 con 0x7fbaec102000 2024-05-15T13:50:58.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.037+0000 7fbae27fc700 1 -- 172.21.15.81:0/3005178748 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fbae802a070 con 0x7fbaec102000 2024-05-15T13:50:58.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.037+0000 7fbaf36be700 1 -- 172.21.15.81:0/3005178748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaec06c030 con 0x7fbaec102000 2024-05-15T13:50:58.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.037+0000 7fbae27fc700 1 --2- 172.21.15.81:0/3005178748 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbad803e2b0 0x7fbad8040770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:50:58.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.037+0000 7fbae27fc700 1 -- 172.21.15.81:0/3005178748 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbae801e030 con 0x7fbaec102000 2024-05-15T13:50:58.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.038+0000 7fbaf0c59700 1 --2- 172.21.15.81:0/3005178748 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbad803e2b0 0x7fbad8040770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:50:58.042 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.038+0000 7fbaf0c59700 1 --2- 172.21.15.81:0/3005178748 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbad803e2b0 0x7fbad8040770 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbadc009850 tx=0x7fbadc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:50:58.046 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.043+0000 7fbae27fc700 1 -- 172.21.15.81:0/3005178748 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbae8013070 con 0x7fbaec102000 2024-05-15T13:50:58.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:50:57 smithi081 ceph-mon[36434]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:50:58.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:50:57 smithi033 ceph-mon[31650]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:50:58.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.327+0000 7fbaf36be700 1 -- 172.21.15.81:0/3005178748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fbaec1a4840 con 0x7fbaec102000 2024-05-15T13:50:58.330 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.328+0000 7fbae27fc700 1 -- 172.21.15.81:0/3005178748 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7fbae8031090 con 0x7fbaec102000 2024-05-15T13:50:58.330 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-15T13:50:58.330 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":2,"fsid":"bca78ca4-12c1-11ef-bc98-c7b262605968","modified":"2024-05-15T13:50:49.841294Z","created":"2024-05-15T13:49:07.404471Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:3300","nonce":0},{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-15T13:50:58.332 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.330+0000 7fbaf36be700 1 -- 172.21.15.81:0/3005178748 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbad803e2b0 msgr2=0x7fbad8040770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:58.332 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.330+0000 7fbaf36be700 1 --2- 172.21.15.81:0/3005178748 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbad803e2b0 0x7fbad8040770 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbadc009850 tx=0x7fbadc008040 comp rx=0 tx=0).stop 2024-05-15T13:50:58.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.330+0000 7fbaf36be700 1 -- 172.21.15.81:0/3005178748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbaec102000 msgr2=0x7fbaec19f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:50:58.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.330+0000 7fbaf36be700 1 --2- 172.21.15.81:0/3005178748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbaec102000 0x7fbaec19f280 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbae8006010 tx=0x7fbae8005030 comp rx=0 tx=0).stop 2024-05-15T13:50:58.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.330+0000 7fbaf36be700 1 -- 172.21.15.81:0/3005178748 shutdown_connections 2024-05-15T13:50:58.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.330+0000 7fbaf36be700 1 --2- 172.21.15.81:0/3005178748 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbad803e2b0 0x7fbad8040770 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:58.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.330+0000 7fbaf36be700 1 --2- 172.21.15.81:0/3005178748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbaec102000 0x7fbaec19f280 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:58.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.330+0000 7fbaf36be700 1 --2- 172.21.15.81:0/3005178748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbaec19f7c0 0x7fbaec1a3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:50:58.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.330+0000 7fbaf36be700 1 -- 172.21.15.81:0/3005178748 >> 172.21.15.81:0/3005178748 conn(0x7fbaec0fda80 msgr2=0x7fbaec0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:50:58.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.330+0000 7fbaf36be700 1 -- 172.21.15.81:0/3005178748 shutdown_connections 2024-05-15T13:50:58.333 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:50:58.330+0000 7fbaf36be700 1 -- 172.21.15.81:0/3005178748 wait complete. 2024-05-15T13:50:58.333 INFO:teuthology.orchestra.run.smithi081.stderr:dumped monmap epoch 2 2024-05-15T13:50:58.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 podman[36170]: 2024-05-15 13:50:58.474518325 +0000 UTC m=+0.017945985 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-15T13:50:59.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 podman[36170]: 2024-05-15 13:50:58.741818136 +0000 UTC m=+0.285245777 container init 9ec142c69ed61ec3a0db7de69bc144c11a46dfc349b4e578d269f17587c9d651 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mon-b, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-15T13:50:59.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 podman[36170]: 2024-05-15 13:50:58.745221413 +0000 UTC m=+0.288649059 container start 9ec142c69ed61ec3a0db7de69bc144c11a46dfc349b4e578d269f17587c9d651 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mon-b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T13:50:59.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: set uid:gid to 167:167 (ceph:ceph) 2024-05-15T13:50:59.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: ceph version 19.0.0-3687-g9b638668 (9b63866860f0b71216b2c0a73aedd2e0a8a5c98b) squid (dev), process ceph-mon, pid 2 2024-05-15T13:50:59.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: pidfile_write: ignore empty --pid-file 2024-05-15T13:50:59.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: load: jerasure load: lrc 2024-05-15T13:50:59.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: RocksDB version: 7.9.2 2024-05-15T13:50:59.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Git sha 0 2024-05-15T13:50:59.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Compile date 2024-05-14 14:11:45 2024-05-15T13:50:59.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: DB SUMMARY 2024-05-15T13:50:59.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: DB Session ID: TATRCD8D6B8B1KBYD3LL 2024-05-15T13:50:59.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: CURRENT file: CURRENT 2024-05-15T13:50:59.093 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: IDENTITY file: IDENTITY 2024-05-15T13:50:59.093 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-15T13:50:59.093 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-05-15T13:50:59.093 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-05-15T13:50:59.093 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.error_if_exists: 0 2024-05-15T13:50:59.093 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.create_if_missing: 0 2024-05-15T13:50:59.093 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.paranoid_checks: 1 2024-05-15T13:50:59.093 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-15T13:50:59.093 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-15T13:50:59.093 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-15T13:50:59.093 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.env: 0x55c99b074cc0 2024-05-15T13:50:59.093 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.fs: PosixFileSystem 2024-05-15T13:50:59.093 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.info_log: 0x55c99ce9bda0 2024-05-15T13:50:59.093 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.max_file_opening_threads: 16 2024-05-15T13:50:59.094 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.statistics: (nil) 2024-05-15T13:50:59.094 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.use_fsync: 0 2024-05-15T13:50:59.094 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.max_log_file_size: 0 2024-05-15T13:50:59.094 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-15T13:50:59.094 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-15T13:50:59.098 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.keep_log_file_num: 1000 2024-05-15T13:50:59.098 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.recycle_log_file_num: 0 2024-05-15T13:50:59.098 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.allow_fallocate: 1 2024-05-15T13:50:59.098 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.allow_mmap_reads: 0 2024-05-15T13:50:59.098 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.allow_mmap_writes: 0 2024-05-15T13:50:59.098 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.use_direct_reads: 0 2024-05-15T13:50:59.098 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-15T13:50:59.098 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.create_missing_column_families: 0 2024-05-15T13:50:59.099 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.db_log_dir: 2024-05-15T13:50:59.099 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.wal_dir: 2024-05-15T13:50:59.099 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-15T13:50:59.099 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-15T13:50:59.099 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-15T13:50:59.099 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-15T13:50:59.099 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-15T13:50:59.099 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-15T13:50:59.099 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.advise_random_on_open: 1 2024-05-15T13:50:59.099 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.db_write_buffer_size: 0 2024-05-15T13:50:59.099 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.write_buffer_manager: 0x55c99ce6d0e0 2024-05-15T13:50:59.099 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-15T13:50:59.099 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-15T13:50:59.099 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-15T13:50:59.099 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.rate_limiter: (nil) 2024-05-15T13:50:59.100 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-15T13:50:59.100 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.wal_recovery_mode: 2 2024-05-15T13:50:59.100 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.enable_thread_tracking: 0 2024-05-15T13:50:59.100 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.enable_pipelined_write: 0 2024-05-15T13:50:59.100 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.unordered_write: 0 2024-05-15T13:50:59.100 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-15T13:50:59.100 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-15T13:50:59.100 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-15T13:50:59.100 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-15T13:50:59.100 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.row_cache: None 2024-05-15T13:50:59.100 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.wal_filter: None 2024-05-15T13:50:59.102 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-15T13:50:59.102 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.allow_ingest_behind: 0 2024-05-15T13:50:59.102 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.two_write_queues: 0 2024-05-15T13:50:59.102 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.manual_wal_flush: 0 2024-05-15T13:50:59.102 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.wal_compression: 0 2024-05-15T13:50:59.102 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.atomic_flush: 0 2024-05-15T13:50:59.102 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-15T13:50:59.102 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-15T13:50:59.103 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-15T13:50:59.103 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.log_readahead_size: 0 2024-05-15T13:50:59.103 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-15T13:50:59.103 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.best_efforts_recovery: 0 2024-05-15T13:50:59.103 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-15T13:50:59.103 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-15T13:50:59.103 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.allow_data_in_errors: 0 2024-05-15T13:50:59.103 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.db_host_id: __hostname__ 2024-05-15T13:50:59.103 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-15T13:50:59.103 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.max_background_jobs: 2 2024-05-15T13:50:59.103 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.max_background_compactions: -1 2024-05-15T13:50:59.103 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.max_subcompactions: 1 2024-05-15T13:50:59.103 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-15T13:50:59.103 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-15T13:50:59.104 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-15T13:50:59.104 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.max_total_wal_size: 0 2024-05-15T13:50:59.104 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-15T13:50:59.104 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-15T13:50:59.104 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-15T13:50:59.104 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-15T13:50:59.104 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.max_open_files: -1 2024-05-15T13:50:59.104 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.bytes_per_sync: 0 2024-05-15T13:50:59.104 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-15T13:50:59.104 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-15T13:50:59.104 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.compaction_readahead_size: 0 2024-05-15T13:50:59.104 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.max_background_flushes: -1 2024-05-15T13:50:59.104 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Compression algorithms supported: 2024-05-15T13:50:59.104 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-15T13:50:59.104 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: kZSTD supported: 0 2024-05-15T13:50:59.105 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: kXpressCompression supported: 0 2024-05-15T13:50:59.105 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: kLZ4HCCompression supported: 1 2024-05-15T13:50:59.105 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: kZlibCompression supported: 1 2024-05-15T13:50:59.105 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: kSnappyCompression supported: 1 2024-05-15T13:50:59.105 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: kLZ4Compression supported: 1 2024-05-15T13:50:59.105 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: kBZip2Compression supported: 0 2024-05-15T13:50:59.105 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-15T13:50:59.105 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-15T13:50:59.105 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-05-15T13:50:59.105 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-15T13:50:59.105 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-15T13:50:59.105 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.merge_operator: 2024-05-15T13:50:59.105 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.compaction_filter: None 2024-05-15T13:50:59.105 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.compaction_filter_factory: None 2024-05-15T13:50:59.105 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.sst_partitioner_factory: None 2024-05-15T13:50:59.106 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-15T13:50:59.106 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-15T13:50:59.106 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c99ce9bf60) 2024-05-15T13:50:59.106 INFO:journalctl@ceph.mon.b.smithi052.stdout: cache_index_and_filter_blocks: 1 2024-05-15T13:50:59.106 INFO:journalctl@ceph.mon.b.smithi052.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-15T13:50:59.106 INFO:journalctl@ceph.mon.b.smithi052.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-15T13:50:59.106 INFO:journalctl@ceph.mon.b.smithi052.stdout: pin_top_level_index_and_filter: 1 2024-05-15T13:50:59.106 INFO:journalctl@ceph.mon.b.smithi052.stdout: index_type: 0 2024-05-15T13:50:59.106 INFO:journalctl@ceph.mon.b.smithi052.stdout: data_block_index_type: 0 2024-05-15T13:50:59.106 INFO:journalctl@ceph.mon.b.smithi052.stdout: index_shortening: 1 2024-05-15T13:50:59.106 INFO:journalctl@ceph.mon.b.smithi052.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-15T13:50:59.106 INFO:journalctl@ceph.mon.b.smithi052.stdout: checksum: 4 2024-05-15T13:50:59.106 INFO:journalctl@ceph.mon.b.smithi052.stdout: no_block_cache: 0 2024-05-15T13:50:59.106 INFO:journalctl@ceph.mon.b.smithi052.stdout: block_cache: 0x55c99ce9f1f0 2024-05-15T13:50:59.106 INFO:journalctl@ceph.mon.b.smithi052.stdout: block_cache_name: BinnedLRUCache 2024-05-15T13:50:59.107 INFO:journalctl@ceph.mon.b.smithi052.stdout: block_cache_options: 2024-05-15T13:50:59.107 INFO:journalctl@ceph.mon.b.smithi052.stdout: capacity : 536870912 2024-05-15T13:50:59.107 INFO:journalctl@ceph.mon.b.smithi052.stdout: num_shard_bits : 4 2024-05-15T13:50:59.107 INFO:journalctl@ceph.mon.b.smithi052.stdout: strict_capacity_limit : 0 2024-05-15T13:50:59.107 INFO:journalctl@ceph.mon.b.smithi052.stdout: high_pri_pool_ratio: 0.000 2024-05-15T13:50:59.107 INFO:journalctl@ceph.mon.b.smithi052.stdout: block_cache_compressed: (nil) 2024-05-15T13:50:59.107 INFO:journalctl@ceph.mon.b.smithi052.stdout: persistent_cache: (nil) 2024-05-15T13:50:59.107 INFO:journalctl@ceph.mon.b.smithi052.stdout: block_size: 4096 2024-05-15T13:50:59.107 INFO:journalctl@ceph.mon.b.smithi052.stdout: block_size_deviation: 10 2024-05-15T13:50:59.107 INFO:journalctl@ceph.mon.b.smithi052.stdout: block_restart_interval: 16 2024-05-15T13:50:59.107 INFO:journalctl@ceph.mon.b.smithi052.stdout: index_block_restart_interval: 1 2024-05-15T13:50:59.107 INFO:journalctl@ceph.mon.b.smithi052.stdout: metadata_block_size: 4096 2024-05-15T13:50:59.107 INFO:journalctl@ceph.mon.b.smithi052.stdout: partition_filters: 0 2024-05-15T13:50:59.107 INFO:journalctl@ceph.mon.b.smithi052.stdout: use_delta_encoding: 1 2024-05-15T13:50:59.107 INFO:journalctl@ceph.mon.b.smithi052.stdout: filter_policy: bloomfilter 2024-05-15T13:50:59.107 INFO:journalctl@ceph.mon.b.smithi052.stdout: whole_key_filtering: 1 2024-05-15T13:50:59.108 INFO:journalctl@ceph.mon.b.smithi052.stdout: verify_compression: 0 2024-05-15T13:50:59.108 INFO:journalctl@ceph.mon.b.smithi052.stdout: read_amp_bytes_per_bit: 0 2024-05-15T13:50:59.108 INFO:journalctl@ceph.mon.b.smithi052.stdout: format_version: 5 2024-05-15T13:50:59.108 INFO:journalctl@ceph.mon.b.smithi052.stdout: enable_index_compression: 1 2024-05-15T13:50:59.108 INFO:journalctl@ceph.mon.b.smithi052.stdout: block_align: 0 2024-05-15T13:50:59.108 INFO:journalctl@ceph.mon.b.smithi052.stdout: max_auto_readahead_size: 262144 2024-05-15T13:50:59.108 INFO:journalctl@ceph.mon.b.smithi052.stdout: prepopulate_block_cache: 0 2024-05-15T13:50:59.108 INFO:journalctl@ceph.mon.b.smithi052.stdout: initial_auto_readahead_size: 8192 2024-05-15T13:50:59.108 INFO:journalctl@ceph.mon.b.smithi052.stdout: num_file_reads_for_auto_readahead: 2 2024-05-15T13:50:59.108 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.write_buffer_size: 33554432 2024-05-15T13:50:59.108 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.max_write_buffer_number: 2 2024-05-15T13:50:59.108 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.compression: NoCompression 2024-05-15T13:50:59.108 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.bottommost_compression: Disabled 2024-05-15T13:50:59.108 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.prefix_extractor: nullptr 2024-05-15T13:50:59.109 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-15T13:50:59.109 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.num_levels: 7 2024-05-15T13:50:59.109 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-15T13:50:59.109 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-15T13:50:59.109 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-15T13:50:59.109 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-15T13:50:59.109 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-15T13:50:59.109 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-15T13:50:59.109 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-15T13:50:59.109 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-15T13:50:59.109 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-15T13:50:59.109 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-15T13:50:59.109 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-15T13:50:59.109 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-15T13:50:59.109 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-15T13:50:59.110 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.compression_opts.level: 32767 2024-05-15T13:50:59.110 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.compression_opts.strategy: 0 2024-05-15T13:50:59.110 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-15T13:50:59.110 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-15T13:50:59.110 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-15T13:50:59.110 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-15T13:50:59.110 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.compression_opts.enabled: false 2024-05-15T13:50:59.110 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-15T13:50:59.110 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-15T13:50:59.110 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-15T13:50:59.110 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-15T13:50:59.110 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.target_file_size_base: 67108864 2024-05-15T13:50:59.110 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-15T13:50:59.110 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-15T13:50:59.110 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-15T13:50:59.111 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-15T13:50:59.111 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-15T13:50:59.111 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-15T13:50:59.111 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-15T13:50:59.111 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-15T13:50:59.111 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-15T13:50:59.111 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-15T13:50:59.111 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-15T13:50:59.111 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-15T13:50:59.111 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-15T13:50:59.111 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-15T13:50:59.111 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.arena_block_size: 1048576 2024-05-15T13:50:59.111 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-15T13:50:59.111 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-15T13:50:59.111 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.disable_auto_compactions: 0 2024-05-15T13:50:59.112 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-15T13:50:59.112 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-15T13:50:59.112 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-15T13:50:59.112 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-15T13:50:59.112 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-15T13:50:59.112 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-15T13:50:59.112 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-15T13:50:59.112 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-15T13:50:59.112 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-15T13:50:59.112 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-15T13:50:59.112 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-15T13:50:59.112 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.inplace_update_support: 0 2024-05-15T13:50:59.112 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-15T13:50:59.112 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-15T13:50:59.112 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-15T13:50:59.113 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-15T13:50:59.113 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.bloom_locality: 0 2024-05-15T13:50:59.113 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.max_successive_merges: 0 2024-05-15T13:50:59.113 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-15T13:50:59.113 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.paranoid_file_checks: 0 2024-05-15T13:50:59.113 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.force_consistency_checks: 1 2024-05-15T13:50:59.113 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.report_bg_io_stats: 0 2024-05-15T13:50:59.113 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.ttl: 2592000 2024-05-15T13:50:59.113 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-15T13:50:59.113 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-15T13:50:59.113 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-15T13:50:59.113 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.enable_blob_files: false 2024-05-15T13:50:59.113 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.min_blob_size: 0 2024-05-15T13:50:59.113 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.blob_file_size: 268435456 2024-05-15T13:50:59.113 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-15T13:50:59.114 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-15T13:50:59.114 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-15T13:50:59.114 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-15T13:50:59.114 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-15T13:50:59.114 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.blob_file_starting_level: 0 2024-05-15T13:50:59.114 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-15T13:50:59.114 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-15T13:50:59.114 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-15T13:50:59.114 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 804c2cac-4c6d-4306-8b6d-f58623f2889d 2024-05-15T13:50:59.114 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715781058775884, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-15T13:50:59.114 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-15T13:50:59.114 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715781058776308, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715781058, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "804c2cac-4c6d-4306-8b6d-f58623f2889d", "db_session_id": "TATRCD8D6B8B1KBYD3LL", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-15T13:50:59.114 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715781058776390, "job": 1, "event": "recovery_finished"} 2024-05-15T13:50:59.114 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-15T13:50:59.114 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-15T13:50:59.115 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c99cfbc000 2024-05-15T13:50:59.115 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: DB pointer 0x55c99cfa8000 2024-05-15T13:50:59.115 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-15T13:50:59.115 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-15T13:50:59.115 INFO:journalctl@ceph.mon.b.smithi052.stdout: ** DB Stats ** 2024-05-15T13:50:59.115 INFO:journalctl@ceph.mon.b.smithi052.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-15T13:50:59.115 INFO:journalctl@ceph.mon.b.smithi052.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-15T13:50:59.115 INFO:journalctl@ceph.mon.b.smithi052.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-15T13:50:59.115 INFO:journalctl@ceph.mon.b.smithi052.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-15T13:50:59.115 INFO:journalctl@ceph.mon.b.smithi052.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-15T13:50:59.115 INFO:journalctl@ceph.mon.b.smithi052.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-15T13:50:59.115 INFO:journalctl@ceph.mon.b.smithi052.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-15T13:50:59.115 INFO:journalctl@ceph.mon.b.smithi052.stdout: 2024-05-15T13:50:59.115 INFO:journalctl@ceph.mon.b.smithi052.stdout: ** Compaction Stats [default] ** 2024-05-15T13:50:59.115 INFO:journalctl@ceph.mon.b.smithi052.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-15T13:50:59.116 INFO:journalctl@ceph.mon.b.smithi052.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-15T13:50:59.116 INFO:journalctl@ceph.mon.b.smithi052.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-15T13:50:59.116 INFO:journalctl@ceph.mon.b.smithi052.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-15T13:50:59.116 INFO:journalctl@ceph.mon.b.smithi052.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-15T13:50:59.116 INFO:journalctl@ceph.mon.b.smithi052.stdout: 2024-05-15T13:50:59.116 INFO:journalctl@ceph.mon.b.smithi052.stdout: ** Compaction Stats [default] ** 2024-05-15T13:50:59.116 INFO:journalctl@ceph.mon.b.smithi052.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-15T13:50:59.116 INFO:journalctl@ceph.mon.b.smithi052.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-15T13:50:59.116 INFO:journalctl@ceph.mon.b.smithi052.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-15T13:50:59.116 INFO:journalctl@ceph.mon.b.smithi052.stdout: 2024-05-15T13:50:59.116 INFO:journalctl@ceph.mon.b.smithi052.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-15T13:50:59.116 INFO:journalctl@ceph.mon.b.smithi052.stdout: 2024-05-15T13:50:59.116 INFO:journalctl@ceph.mon.b.smithi052.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-15T13:50:59.116 INFO:journalctl@ceph.mon.b.smithi052.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-15T13:50:59.116 INFO:journalctl@ceph.mon.b.smithi052.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-15T13:50:59.117 INFO:journalctl@ceph.mon.b.smithi052.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-15T13:50:59.117 INFO:journalctl@ceph.mon.b.smithi052.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-15T13:50:59.117 INFO:journalctl@ceph.mon.b.smithi052.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-15T13:50:59.117 INFO:journalctl@ceph.mon.b.smithi052.stdout: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-15T13:50:59.117 INFO:journalctl@ceph.mon.b.smithi052.stdout: Interval compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-15T13:50:59.117 INFO:journalctl@ceph.mon.b.smithi052.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-15T13:50:59.117 INFO:journalctl@ceph.mon.b.smithi052.stdout: Block cache BinnedLRUCache@0x55c99ce9f1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-05-15T13:50:59.117 INFO:journalctl@ceph.mon.b.smithi052.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-15T13:50:59.117 INFO:journalctl@ceph.mon.b.smithi052.stdout: 2024-05-15T13:50:59.117 INFO:journalctl@ceph.mon.b.smithi052.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-15T13:50:59.117 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-05-15T13:50:59.117 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: using public_addr v2:172.21.15.52:0/0 -> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] 2024-05-15T13:50:59.117 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: starting mon.b rank -1 at public addrs [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] at bind addrs [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid bca78ca4-12c1-11ef-bc98-c7b262605968 2024-05-15T13:50:59.117 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: mon.b@-1(???) e0 preinit fsid bca78ca4-12c1-11ef-bc98-c7b262605968 2024-05-15T13:50:59.118 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: mon.b@-1(synchronizing).mds e1 new map 2024-05-15T13:50:59.118 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-15T13:50:59.118 INFO:journalctl@ceph.mon.b.smithi052.stdout: e1 2024-05-15T13:50:59.118 INFO:journalctl@ceph.mon.b.smithi052.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T13:50:59.118 INFO:journalctl@ceph.mon.b.smithi052.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-15T13:50:59.118 INFO:journalctl@ceph.mon.b.smithi052.stdout: legacy client fscid: -1 2024-05-15T13:50:59.118 INFO:journalctl@ceph.mon.b.smithi052.stdout: 2024-05-15T13:50:59.118 INFO:journalctl@ceph.mon.b.smithi052.stdout: No filesystems configured 2024-05-15T13:50:59.118 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-15T13:50:59.118 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-15T13:50:59.118 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-15T13:50:59.118 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-15T13:50:59.118 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-15T13:50:59.118 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-15T13:50:59.118 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-15T13:50:59.119 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-15T13:50:59.119 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-15T13:50:59.119 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-15T13:50:59.119 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:50:59.119 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:59.119 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-15T13:50:59.119 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:50:59.119 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi081", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:50:59.119 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:50:59.119 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:59.119 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:59.119 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:59.119 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:59.119 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi052", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:50:59.119 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:50:59.120 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:50:59.120 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: Updating smithi052:/etc/ceph/ceph.conf 2024-05-15T13:50:59.120 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: Updating smithi052:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T13:50:59.120 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: Deploying cephadm binary to smithi081 2024-05-15T13:50:59.120 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: Updating smithi052:/etc/ceph/ceph.client.admin.keyring 2024-05-15T13:50:59.120 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: Updating smithi052:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-15T13:50:59.120 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:50:59.120 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:59.120 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:59.120 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:59.120 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:50:59.120 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:59.120 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: Added host smithi081 2024-05-15T13:50:59.120 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:50:59.120 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:59.121 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:59.121 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:50:59.121 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-15T13:50:59.121 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:59.121 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:50:59.121 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/266324991' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-15T13:50:59.121 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:59.121 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:59.121 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:59.121 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:59.121 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi081", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:50:59.121 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:50:59.121 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:50:59.121 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: Updating smithi081:/etc/ceph/ceph.conf 2024-05-15T13:50:59.121 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/266324991' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-15T13:50:59.122 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: osdmap e4: 0 total, 0 up, 0 in 2024-05-15T13:50:59.122 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: Updating smithi081:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T13:50:59.122 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:50:59.122 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: Updating smithi081:/etc/ceph/ceph.client.admin.keyring 2024-05-15T13:50:59.122 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: Updating smithi081:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-15T13:50:59.122 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:59.122 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:59.122 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:59.122 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:50:59.122 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi033:172.21.15.33=a;smithi052:172.21.15.52=b;smithi081:172.21.15.81=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:50:59.122 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: Saving service mon spec with placement smithi033:172.21.15.33=a;smithi052:172.21.15.52=b;smithi081:172.21.15.81=c;count:3 2024-05-15T13:50:59.122 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:59.122 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:50:59.122 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:50:59.122 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:50:59.123 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:59.123 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-15T13:50:59.123 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:50:59.123 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: Deploying daemon mon.c on smithi081 2024-05-15T13:50:59.123 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:50:59.123 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-15T13:50:59.123 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: mon.a calling monitor election 2024-05-15T13:50:59.123 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-15T13:50:59.123 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='client.? 172.21.15.81:0/281402504' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T13:50:59.123 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-15T13:50:59.123 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:50:59.123 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-15T13:50:59.123 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: mon.c calling monitor election 2024-05-15T13:50:59.123 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-15T13:50:59.123 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:50:59.124 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-15T13:50:59.124 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-15T13:50:59.124 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-15T13:50:59.124 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: monmap epoch 2 2024-05-15T13:50:59.124 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: fsid bca78ca4-12c1-11ef-bc98-c7b262605968 2024-05-15T13:50:59.124 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: last_changed 2024-05-15T13:50:49.841294+0000 2024-05-15T13:50:59.124 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: created 2024-05-15T13:49:07.404471+0000 2024-05-15T13:50:59.124 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: min_mon_release 19 (squid) 2024-05-15T13:50:59.124 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: election_strategy: 1 2024-05-15T13:50:59.124 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-05-15T13:50:59.124 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: 1: [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] mon.c 2024-05-15T13:50:59.124 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: fsmap 2024-05-15T13:50:59.124 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: osdmap e4: 0 total, 0 up, 0 in 2024-05-15T13:50:59.124 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: mgrmap e13: x(active, since 49s) 2024-05-15T13:50:59.124 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: overall HEALTH_OK 2024-05-15T13:50:59.124 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:59.125 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:59.125 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:50:59.125 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:50:59.125 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-15T13:50:59.125 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:50:59.125 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: Deploying daemon mon.b on smithi052 2024-05-15T13:50:59.125 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-15T13:50:59.125 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:50:59.125 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 ceph-mon[36190]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-15T13:50:59.125 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 bash[36170]: 9ec142c69ed61ec3a0db7de69bc144c11a46dfc349b4e578d269f17587c9d651 2024-05-15T13:50:59.125 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:50:58 smithi052 systemd[1]: Started Ceph mon.b for bca78ca4-12c1-11ef-bc98-c7b262605968. 2024-05-15T13:50:59.935 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-15T13:50:59.936 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-15T13:51:00.179 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.c/config 2024-05-15T13:51:04.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:03 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-15T13:51:04.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:03 smithi033 ceph-mon[31650]: mon.a calling monitor election 2024-05-15T13:51:04.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:03 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-15T13:51:04.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:03 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-15T13:51:04.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:03 smithi033 ceph-mon[31650]: mon.c calling monitor election 2024-05-15T13:51:04.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:03 smithi033 ceph-mon[31650]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:04.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:03 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-15T13:51:04.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:03 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-15T13:51:04.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:03 smithi033 ceph-mon[31650]: mon.b calling monitor election 2024-05-15T13:51:04.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:03 smithi033 ceph-mon[31650]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:04.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:03 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-15T13:51:04.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:03 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-15T13:51:04.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:03 smithi033 ceph-mon[31650]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:04.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:03 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-15T13:51:04.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:03 smithi033 ceph-mon[31650]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-15T13:51:04.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:03 smithi033 ceph-mon[31650]: monmap epoch 3 2024-05-15T13:51:04.194 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:03 smithi033 ceph-mon[31650]: fsid bca78ca4-12c1-11ef-bc98-c7b262605968 2024-05-15T13:51:04.194 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:03 smithi033 ceph-mon[31650]: last_changed 2024-05-15T13:50:58.800290+0000 2024-05-15T13:51:04.194 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:03 smithi033 ceph-mon[31650]: created 2024-05-15T13:49:07.404471+0000 2024-05-15T13:51:04.194 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:03 smithi033 ceph-mon[31650]: min_mon_release 19 (squid) 2024-05-15T13:51:04.194 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:03 smithi033 ceph-mon[31650]: election_strategy: 1 2024-05-15T13:51:04.194 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:03 smithi033 ceph-mon[31650]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-05-15T13:51:04.194 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:03 smithi033 ceph-mon[31650]: 1: [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] mon.c 2024-05-15T13:51:04.194 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:03 smithi033 ceph-mon[31650]: 2: [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon.b 2024-05-15T13:51:04.194 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:03 smithi033 ceph-mon[31650]: fsmap 2024-05-15T13:51:04.194 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:03 smithi033 ceph-mon[31650]: osdmap e4: 0 total, 0 up, 0 in 2024-05-15T13:51:04.194 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:03 smithi033 ceph-mon[31650]: mgrmap e13: x(active, since 58s) 2024-05-15T13:51:04.195 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:03 smithi033 ceph-mon[31650]: overall HEALTH_OK 2024-05-15T13:51:04.195 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:03 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:04.195 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:03 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:04.195 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:03 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:04.195 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:03 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:04.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:03 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-15T13:51:04.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:03 smithi052 ceph-mon[36190]: mon.a calling monitor election 2024-05-15T13:51:04.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:03 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-15T13:51:04.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:03 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-15T13:51:04.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:03 smithi052 ceph-mon[36190]: mon.c calling monitor election 2024-05-15T13:51:04.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:03 smithi052 ceph-mon[36190]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:04.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:03 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-15T13:51:04.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:03 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-15T13:51:04.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:03 smithi052 ceph-mon[36190]: mon.b calling monitor election 2024-05-15T13:51:04.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:03 smithi052 ceph-mon[36190]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:04.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:03 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-15T13:51:04.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:03 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-15T13:51:04.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:03 smithi052 ceph-mon[36190]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:04.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:03 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-15T13:51:04.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:03 smithi052 ceph-mon[36190]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-15T13:51:04.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:03 smithi052 ceph-mon[36190]: monmap epoch 3 2024-05-15T13:51:04.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:03 smithi052 ceph-mon[36190]: fsid bca78ca4-12c1-11ef-bc98-c7b262605968 2024-05-15T13:51:04.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:03 smithi052 ceph-mon[36190]: last_changed 2024-05-15T13:50:58.800290+0000 2024-05-15T13:51:04.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:03 smithi052 ceph-mon[36190]: created 2024-05-15T13:49:07.404471+0000 2024-05-15T13:51:04.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:03 smithi052 ceph-mon[36190]: min_mon_release 19 (squid) 2024-05-15T13:51:04.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:03 smithi052 ceph-mon[36190]: election_strategy: 1 2024-05-15T13:51:04.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:03 smithi052 ceph-mon[36190]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-05-15T13:51:04.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:03 smithi052 ceph-mon[36190]: 1: [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] mon.c 2024-05-15T13:51:04.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:03 smithi052 ceph-mon[36190]: 2: [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon.b 2024-05-15T13:51:04.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:03 smithi052 ceph-mon[36190]: fsmap 2024-05-15T13:51:04.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:03 smithi052 ceph-mon[36190]: osdmap e4: 0 total, 0 up, 0 in 2024-05-15T13:51:04.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:03 smithi052 ceph-mon[36190]: mgrmap e13: x(active, since 58s) 2024-05-15T13:51:04.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:03 smithi052 ceph-mon[36190]: overall HEALTH_OK 2024-05-15T13:51:04.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:03 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:04.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:03 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:04.344 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:03 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:04.344 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:03 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:04.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:03 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-15T13:51:04.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:03 smithi081 ceph-mon[36434]: mon.a calling monitor election 2024-05-15T13:51:04.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:03 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-15T13:51:04.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:03 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-15T13:51:04.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:03 smithi081 ceph-mon[36434]: mon.c calling monitor election 2024-05-15T13:51:04.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:03 smithi081 ceph-mon[36434]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:04.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:03 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-15T13:51:04.359 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:03 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-15T13:51:04.359 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:03 smithi081 ceph-mon[36434]: mon.b calling monitor election 2024-05-15T13:51:04.359 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:03 smithi081 ceph-mon[36434]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:04.359 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:03 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-15T13:51:04.359 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:03 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-15T13:51:04.359 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:03 smithi081 ceph-mon[36434]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:04.359 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:03 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-15T13:51:04.359 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:03 smithi081 ceph-mon[36434]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-15T13:51:04.359 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:03 smithi081 ceph-mon[36434]: monmap epoch 3 2024-05-15T13:51:04.359 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:03 smithi081 ceph-mon[36434]: fsid bca78ca4-12c1-11ef-bc98-c7b262605968 2024-05-15T13:51:04.359 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:03 smithi081 ceph-mon[36434]: last_changed 2024-05-15T13:50:58.800290+0000 2024-05-15T13:51:04.359 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:03 smithi081 ceph-mon[36434]: created 2024-05-15T13:49:07.404471+0000 2024-05-15T13:51:04.359 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:03 smithi081 ceph-mon[36434]: min_mon_release 19 (squid) 2024-05-15T13:51:04.359 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:03 smithi081 ceph-mon[36434]: election_strategy: 1 2024-05-15T13:51:04.360 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:03 smithi081 ceph-mon[36434]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-05-15T13:51:04.360 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:03 smithi081 ceph-mon[36434]: 1: [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] mon.c 2024-05-15T13:51:04.360 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:03 smithi081 ceph-mon[36434]: 2: [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon.b 2024-05-15T13:51:04.360 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:03 smithi081 ceph-mon[36434]: fsmap 2024-05-15T13:51:04.360 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:03 smithi081 ceph-mon[36434]: osdmap e4: 0 total, 0 up, 0 in 2024-05-15T13:51:04.360 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:03 smithi081 ceph-mon[36434]: mgrmap e13: x(active, since 58s) 2024-05-15T13:51:04.360 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:03 smithi081 ceph-mon[36434]: overall HEALTH_OK 2024-05-15T13:51:04.360 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:03 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:04.360 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:03 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:04.360 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:03 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:04.360 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:03 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:04.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.572+0000 7f29600bc700 1 -- 172.21.15.81:0/3743252122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29440035c0 msgr2=0x7f2944005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:51:04.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.572+0000 7f29600bc700 1 --2- 172.21.15.81:0/3743252122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29440035c0 0x7f2944005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2958101e70 tx=0x7f295400feb0 comp rx=0 tx=0).stop 2024-05-15T13:51:04.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.572+0000 7f29600bc700 1 -- 172.21.15.81:0/3743252122 shutdown_connections 2024-05-15T13:51:04.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.572+0000 7f29600bc700 1 --2- 172.21.15.81:0/3743252122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29440035c0 0x7f2944005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:04.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.572+0000 7f29600bc700 1 -- 172.21.15.81:0/3743252122 >> 172.21.15.81:0/3743252122 conn(0x7f29580fda50 msgr2=0x7f29580ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:51:04.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.572+0000 7f29600bc700 1 -- 172.21.15.81:0/3743252122 shutdown_connections 2024-05-15T13:51:04.575 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.572+0000 7f29600bc700 1 -- 172.21.15.81:0/3743252122 wait complete. 2024-05-15T13:51:04.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.573+0000 7f29600bc700 1 Processor -- start 2024-05-15T13:51:04.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.573+0000 7f29600bc700 1 -- start start 2024-05-15T13:51:04.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.573+0000 7f29600bc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29440035c0 0x7f295819aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:51:04.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.573+0000 7f29600bc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f295819b400 0x7f295819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:51:04.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.573+0000 7f29600bc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f295819fcd0 0x7f29581a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:51:04.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.573+0000 7f29600bc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2958109e30 con 0x7f295819fcd0 2024-05-15T13:51:04.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.573+0000 7f29600bc700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2958109cb0 con 0x7f29440035c0 2024-05-15T13:51:04.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.573+0000 7f29600bc700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2958109fb0 con 0x7f295819b400 2024-05-15T13:51:04.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.573+0000 7f295d657700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f295819b400 0x7f295819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:51:04.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.573+0000 7f295d657700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f295819b400 0x7f295819f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42862/0 (socket says 172.21.15.81:42862) 2024-05-15T13:51:04.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.573+0000 7f295d657700 1 -- 172.21.15.81:0/1457347986 learned_addr learned my addr 172.21.15.81:0/1457347986 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-15T13:51:04.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.573+0000 7f295de58700 1 --2- 172.21.15.81:0/1457347986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29440035c0 0x7f295819aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:51:04.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.573+0000 7f295d657700 1 -- 172.21.15.81:0/1457347986 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f295819b400 msgr2=0x7f295819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-15T13:51:04.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.573+0000 7f295d657700 1 -- 172.21.15.81:0/1457347986 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f295819b400 msgr2=0x7f295819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-15T13:51:04.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.573+0000 7f295d657700 1 --2- 172.21.15.81:0/1457347986 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f295819b400 0x7f295819f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-15T13:51:04.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.573+0000 7f295d657700 1 --2- 172.21.15.81:0/1457347986 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f295819b400 0x7f295819f790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-15T13:51:04.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.573+0000 7f295e659700 1 --2- 172.21.15.81:0/1457347986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f295819fcd0 0x7f29581a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:51:04.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.573+0000 7f295de58700 1 -- 172.21.15.81:0/1457347986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29440035c0 msgr2=0x7f295819aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-15T13:51:04.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.573+0000 7f295de58700 1 -- 172.21.15.81:0/1457347986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29440035c0 msgr2=0x7f295819aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-15T13:51:04.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.573+0000 7f295de58700 1 --2- 172.21.15.81:0/1457347986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29440035c0 0x7f295819aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-15T13:51:04.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.573+0000 7f295de58700 1 --2- 172.21.15.81:0/1457347986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29440035c0 0x7f295819aec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-15T13:51:04.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.574+0000 7f295e659700 1 -- 172.21.15.81:0/1457347986 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f295819b400 msgr2=0x7f295819f790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-15T13:51:04.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.574+0000 7f295e659700 1 --2- 172.21.15.81:0/1457347986 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f295819b400 0x7f295819f790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:04.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.574+0000 7f295e659700 1 -- 172.21.15.81:0/1457347986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29440035c0 msgr2=0x7f295819aec0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-15T13:51:04.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.574+0000 7f295e659700 1 --2- 172.21.15.81:0/1457347986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29440035c0 0x7f295819aec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:04.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.574+0000 7f295e659700 1 -- 172.21.15.81:0/1457347986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29581a7a40 con 0x7f295819fcd0 2024-05-15T13:51:04.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.574+0000 7f295e659700 1 --2- 172.21.15.81:0/1457347986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f295819fcd0 0x7f29581a0180 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f295000bd50 tx=0x7f295000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:51:04.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.574+0000 7f294effd700 1 -- 172.21.15.81:0/1457347986 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2950019070 con 0x7f295819fcd0 2024-05-15T13:51:04.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.574+0000 7f29600bc700 1 -- 172.21.15.81:0/1457347986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29581a7c70 con 0x7f295819fcd0 2024-05-15T13:51:04.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.574+0000 7f294effd700 1 -- 172.21.15.81:0/1457347986 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f295000cd10 con 0x7f295819fcd0 2024-05-15T13:51:04.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.574+0000 7f294effd700 1 -- 172.21.15.81:0/1457347986 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2950012700 con 0x7f295819fcd0 2024-05-15T13:51:04.579 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.574+0000 7f29600bc700 1 -- 172.21.15.81:0/1457347986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29581a81b0 con 0x7f295819fcd0 2024-05-15T13:51:04.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.575+0000 7f294effd700 1 -- 172.21.15.81:0/1457347986 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f295001c070 con 0x7f295819fcd0 2024-05-15T13:51:04.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.575+0000 7f294effd700 1 --2- 172.21.15.81:0/1457347986 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f293c03e9a0 0x7f293c040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:51:04.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.575+0000 7f294effd700 1 -- 172.21.15.81:0/1457347986 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2950052080 con 0x7f295819fcd0 2024-05-15T13:51:04.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.575+0000 7f29600bc700 1 -- 172.21.15.81:0/1457347986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2958064ba0 con 0x7f295819fcd0 2024-05-15T13:51:04.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.576+0000 7f295de58700 1 --2- 172.21.15.81:0/1457347986 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f293c03e9a0 0x7f293c040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:51:04.580 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.576+0000 7f295de58700 1 --2- 172.21.15.81:0/1457347986 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f293c03e9a0 0x7f293c040e60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2954001eb0 tx=0x7f2954005e60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:51:04.583 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.581+0000 7f294effd700 1 -- 172.21.15.81:0/1457347986 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f295001c360 con 0x7f295819fcd0 2024-05-15T13:51:04.884 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.881+0000 7f29600bc700 1 -- 172.21.15.81:0/1457347986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f29581a83e0 con 0x7f295819fcd0 2024-05-15T13:51:04.884 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.882+0000 7f294effd700 1 -- 172.21.15.81:0/1457347986 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1277 (secure 0 0 0) 0x7f2950014a10 con 0x7f295819fcd0 2024-05-15T13:51:04.885 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-15T13:51:04.885 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":3,"fsid":"bca78ca4-12c1-11ef-bc98-c7b262605968","modified":"2024-05-15T13:50:58.800290Z","created":"2024-05-15T13:49:07.404471Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:3300","nonce":0},{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:3300","nonce":0},{"type":"v1","addr":"172.21.15.52:6789","nonce":0}]},"addr":"172.21.15.52:6789/0","public_addr":"172.21.15.52:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-15T13:51:04.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.884+0000 7f29600bc700 1 -- 172.21.15.81:0/1457347986 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f293c03e9a0 msgr2=0x7f293c040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:51:04.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.884+0000 7f29600bc700 1 --2- 172.21.15.81:0/1457347986 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f293c03e9a0 0x7f293c040e60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2954001eb0 tx=0x7f2954005e60 comp rx=0 tx=0).stop 2024-05-15T13:51:04.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.884+0000 7f29600bc700 1 -- 172.21.15.81:0/1457347986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f295819fcd0 msgr2=0x7f29581a0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:51:04.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.884+0000 7f29600bc700 1 --2- 172.21.15.81:0/1457347986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f295819fcd0 0x7f29581a0180 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f295000bd50 tx=0x7f295000be30 comp rx=0 tx=0).stop 2024-05-15T13:51:04.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.884+0000 7f29600bc700 1 -- 172.21.15.81:0/1457347986 shutdown_connections 2024-05-15T13:51:04.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.884+0000 7f29600bc700 1 --2- 172.21.15.81:0/1457347986 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f293c03e9a0 0x7f293c040e60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:04.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.884+0000 7f29600bc700 1 --2- 172.21.15.81:0/1457347986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f295819fcd0 0x7f29581a0180 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:04.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.884+0000 7f29600bc700 1 --2- 172.21.15.81:0/1457347986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29440035c0 0x7f295819aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:04.887 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.884+0000 7f29600bc700 1 --2- 172.21.15.81:0/1457347986 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f295819b400 0x7f295819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:04.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.884+0000 7f29600bc700 1 -- 172.21.15.81:0/1457347986 >> 172.21.15.81:0/1457347986 conn(0x7f29580fda50 msgr2=0x7f2958105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:51:04.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.884+0000 7f29600bc700 1 -- 172.21.15.81:0/1457347986 shutdown_connections 2024-05-15T13:51:04.888 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:04.884+0000 7f29600bc700 1 -- 172.21.15.81:0/1457347986 wait complete. 2024-05-15T13:51:04.888 INFO:teuthology.orchestra.run.smithi081.stderr:dumped monmap epoch 3 2024-05-15T13:51:05.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:04 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:51:05.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:04 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-15T13:51:05.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:04 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:51:05.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:04 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-15T13:51:05.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:04 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:51:05.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:04 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-15T13:51:05.817 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-15T13:51:05.817 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph config generate-minimal-conf 2024-05-15T13:51:06.090 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:51:06.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:05 smithi081 ceph-mon[36434]: from='client.? 172.21.15.81:0/1457347986' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T13:51:06.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:05 smithi081 ceph-mon[36434]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:06.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:05 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:06.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:05 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:06.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:05 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:06.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:05 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:06.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:05 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:06.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:05 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:06.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:05 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:51:06.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:05 smithi033 ceph-mon[31650]: from='client.? 172.21.15.81:0/1457347986' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T13:51:06.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:05 smithi033 ceph-mon[31650]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:06.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:05 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:06.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:05 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:06.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:05 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:06.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:05 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:06.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:05 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:06.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:05 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:06.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:05 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:51:06.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:05 smithi052 ceph-mon[36190]: from='client.? 172.21.15.81:0/1457347986' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-15T13:51:06.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:05 smithi052 ceph-mon[36190]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:06.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:05 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:06.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:05 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:06.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:05 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:06.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:05 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:06.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:05 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:06.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:05 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:06.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:05 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:51:07.158 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:07 smithi033 ceph-mon[31650]: Updating smithi033:/etc/ceph/ceph.conf 2024-05-15T13:51:07.159 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:07 smithi033 ceph-mon[31650]: Updating smithi052:/etc/ceph/ceph.conf 2024-05-15T13:51:07.159 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:07 smithi033 ceph-mon[31650]: Updating smithi081:/etc/ceph/ceph.conf 2024-05-15T13:51:07.159 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:07 smithi033 ceph-mon[31650]: Updating smithi052:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T13:51:07.159 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:07 smithi033 ceph-mon[31650]: Updating smithi033:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T13:51:07.159 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:07 smithi033 ceph-mon[31650]: Updating smithi081:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T13:51:07.159 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:07 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:07.159 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:07 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:07.159 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:07 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:07.159 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:07 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:07.159 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:07 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:07.159 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:07 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:07.160 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:07 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:07.160 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:07 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:07.160 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:07 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:07.160 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:07 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:07.160 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:07 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:07.160 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:07 smithi033 ceph-mon[31650]: Reconfiguring mon.a (unknown last config time)... 2024-05-15T13:51:07.160 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:07 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-15T13:51:07.160 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:07 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-15T13:51:07.160 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:07 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:07.160 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:07 smithi033 ceph-mon[31650]: Reconfiguring daemon mon.a on smithi033 2024-05-15T13:51:07.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:07 smithi052 ceph-mon[36190]: Updating smithi033:/etc/ceph/ceph.conf 2024-05-15T13:51:07.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:07 smithi052 ceph-mon[36190]: Updating smithi052:/etc/ceph/ceph.conf 2024-05-15T13:51:07.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:07 smithi052 ceph-mon[36190]: Updating smithi081:/etc/ceph/ceph.conf 2024-05-15T13:51:07.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:07 smithi052 ceph-mon[36190]: Updating smithi052:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T13:51:07.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:07 smithi052 ceph-mon[36190]: Updating smithi033:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T13:51:07.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:07 smithi052 ceph-mon[36190]: Updating smithi081:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T13:51:07.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:07 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:07.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:07 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:07.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:07 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:07.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:07 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:07.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:07 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:07.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:07 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:07.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:07 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:07.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:07 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:07.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:07 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:07.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:07 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:07.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:07 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:07.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:07 smithi052 ceph-mon[36190]: Reconfiguring mon.a (unknown last config time)... 2024-05-15T13:51:07.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:07 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-15T13:51:07.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:07 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-15T13:51:07.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:07 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:07.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:07 smithi052 ceph-mon[36190]: Reconfiguring daemon mon.a on smithi033 2024-05-15T13:51:07.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:07 smithi081 ceph-mon[36434]: Updating smithi033:/etc/ceph/ceph.conf 2024-05-15T13:51:07.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:07 smithi081 ceph-mon[36434]: Updating smithi052:/etc/ceph/ceph.conf 2024-05-15T13:51:07.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:07 smithi081 ceph-mon[36434]: Updating smithi081:/etc/ceph/ceph.conf 2024-05-15T13:51:07.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:07 smithi081 ceph-mon[36434]: Updating smithi052:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T13:51:07.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:07 smithi081 ceph-mon[36434]: Updating smithi033:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T13:51:07.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:07 smithi081 ceph-mon[36434]: Updating smithi081:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T13:51:07.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:07 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:07.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:07 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:07.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:07 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:07.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:07 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:07.359 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:07 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:07.359 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:07 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:07.359 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:07 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:07.359 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:07 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:07.359 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:07 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:07.359 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:07 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:07.359 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:07 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:07.359 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:07 smithi081 ceph-mon[36434]: Reconfiguring mon.a (unknown last config time)... 2024-05-15T13:51:07.359 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:07 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-15T13:51:07.359 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:07 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-15T13:51:07.359 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:07 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:07.359 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:07 smithi081 ceph-mon[36434]: Reconfiguring daemon mon.a on smithi033 2024-05-15T13:51:08.416 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:08 smithi033 ceph-mon[31650]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:08.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:08 smithi052 ceph-mon[36190]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:08.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:08 smithi081 ceph-mon[36434]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:08.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.677+0000 7fca6be92700 1 -- 172.21.15.33:0/4129869037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca641020f0 msgr2=0x7fca641024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:51:08.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.677+0000 7fca6be92700 1 --2- 172.21.15.33:0/4129869037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca641020f0 0x7fca641024d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fca5400b0a0 tx=0x7fca5400b3b0 comp rx=0 tx=0).stop 2024-05-15T13:51:08.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.677+0000 7fca6be92700 1 -- 172.21.15.33:0/4129869037 shutdown_connections 2024-05-15T13:51:08.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.677+0000 7fca6be92700 1 --2- 172.21.15.33:0/4129869037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca641020f0 0x7fca641024d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:08.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.677+0000 7fca6be92700 1 -- 172.21.15.33:0/4129869037 >> 172.21.15.33:0/4129869037 conn(0x7fca640779d0 msgr2=0x7fca64077de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:51:08.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.677+0000 7fca6be92700 1 -- 172.21.15.33:0/4129869037 shutdown_connections 2024-05-15T13:51:08.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.677+0000 7fca6be92700 1 -- 172.21.15.33:0/4129869037 wait complete. 2024-05-15T13:51:08.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.678+0000 7fca6be92700 1 Processor -- start 2024-05-15T13:51:08.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.678+0000 7fca6be92700 1 -- start start 2024-05-15T13:51:08.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.678+0000 7fca6be92700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca641020f0 0x7fca6419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:51:08.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.678+0000 7fca6be92700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca6419ede0 0x7fca64198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:51:08.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.678+0000 7fca6be92700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca64198fa0 0x7fca64199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:51:08.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.678+0000 7fca6be92700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fca6410c310 con 0x7fca64198fa0 2024-05-15T13:51:08.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.678+0000 7fca6be92700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fca6410c190 con 0x7fca6419ede0 2024-05-15T13:51:08.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.678+0000 7fca6be92700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fca6410c490 con 0x7fca641020f0 2024-05-15T13:51:08.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.678+0000 7fca6a42f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca64198fa0 0x7fca64199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:51:08.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.679+0000 7fca6a42f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca64198fa0 0x7fca64199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34130/0 (socket says 172.21.15.33:34130) 2024-05-15T13:51:08.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.679+0000 7fca6a42f700 1 -- 172.21.15.33:0/2654271419 learned_addr learned my addr 172.21.15.33:0/2654271419 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:51:08.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.679+0000 7fca6942d700 1 --2- 172.21.15.33:0/2654271419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca6419ede0 0x7fca64198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:51:08.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.679+0000 7fca69c2e700 1 --2- 172.21.15.33:0/2654271419 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca641020f0 0x7fca6419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:51:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.679+0000 7fca6a42f700 1 -- 172.21.15.33:0/2654271419 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca641020f0 msgr2=0x7fca6419e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:51:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.679+0000 7fca6a42f700 1 --2- 172.21.15.33:0/2654271419 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca641020f0 0x7fca6419e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.679+0000 7fca6a42f700 1 -- 172.21.15.33:0/2654271419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca6419ede0 msgr2=0x7fca64198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:51:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.679+0000 7fca6a42f700 1 --2- 172.21.15.33:0/2654271419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca6419ede0 0x7fca64198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.679+0000 7fca6a42f700 1 -- 172.21.15.33:0/2654271419 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca641a5770 con 0x7fca64198fa0 2024-05-15T13:51:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.679+0000 7fca6a42f700 1 --2- 172.21.15.33:0/2654271419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca64198fa0 0x7fca64199450 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fca5c00b550 tx=0x7fca5c00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:51:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.679+0000 7fca62ffd700 1 -- 172.21.15.33:0/2654271419 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c015020 con 0x7fca64198fa0 2024-05-15T13:51:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.679+0000 7fca62ffd700 1 -- 172.21.15.33:0/2654271419 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca5c011ce0 con 0x7fca64198fa0 2024-05-15T13:51:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.679+0000 7fca6be92700 1 -- 172.21.15.33:0/2654271419 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca641a59a0 con 0x7fca64198fa0 2024-05-15T13:51:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.679+0000 7fca62ffd700 1 -- 172.21.15.33:0/2654271419 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c010ca0 con 0x7fca64198fa0 2024-05-15T13:51:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.679+0000 7fca6be92700 1 -- 172.21.15.33:0/2654271419 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca641a5eb0 con 0x7fca64198fa0 2024-05-15T13:51:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.680+0000 7fca62ffd700 1 -- 172.21.15.33:0/2654271419 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fca5c018020 con 0x7fca64198fa0 2024-05-15T13:51:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.680+0000 7fca6be92700 1 -- 172.21.15.33:0/2654271419 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca64064ba0 con 0x7fca64198fa0 2024-05-15T13:51:08.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.681+0000 7fca62ffd700 1 --2- 172.21.15.33:0/2654271419 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fca4c03e9a0 0x7fca4c040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:51:08.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.681+0000 7fca62ffd700 1 -- 172.21.15.33:0/2654271419 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fca5c051fd0 con 0x7fca64198fa0 2024-05-15T13:51:08.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.681+0000 7fca69c2e700 1 --2- 172.21.15.33:0/2654271419 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fca4c03e9a0 0x7fca4c040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:51:08.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.681+0000 7fca69c2e700 1 --2- 172.21.15.33:0/2654271419 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fca4c03e9a0 0x7fca4c040e60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fca540062a0 tx=0x7fca54005040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:51:08.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.686+0000 7fca62ffd700 1 -- 172.21.15.33:0/2654271419 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fca5c01f020 con 0x7fca64198fa0 2024-05-15T13:51:08.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.901+0000 7fca6be92700 1 -- 172.21.15.33:0/2654271419 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fca641a6230 con 0x7fca64198fa0 2024-05-15T13:51:08.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.901+0000 7fca62ffd700 1 -- 172.21.15.33:0/2654271419 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+271 (secure 0 0 0) 0x7fca5c0187b0 con 0x7fca64198fa0 2024-05-15T13:51:08.904 INFO:teuthology.orchestra.run.smithi033.stdout:# minimal ceph.conf for bca78ca4-12c1-11ef-bc98-c7b262605968 2024-05-15T13:51:08.904 INFO:teuthology.orchestra.run.smithi033.stdout:[global] 2024-05-15T13:51:08.904 INFO:teuthology.orchestra.run.smithi033.stdout: fsid = bca78ca4-12c1-11ef-bc98-c7b262605968 2024-05-15T13:51:08.904 INFO:teuthology.orchestra.run.smithi033.stdout: mon_host = [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] 2024-05-15T13:51:08.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.903+0000 7fca6be92700 1 -- 172.21.15.33:0/2654271419 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fca4c03e9a0 msgr2=0x7fca4c040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:51:08.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.903+0000 7fca6be92700 1 --2- 172.21.15.33:0/2654271419 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fca4c03e9a0 0x7fca4c040e60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fca540062a0 tx=0x7fca54005040 comp rx=0 tx=0).stop 2024-05-15T13:51:08.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.903+0000 7fca6be92700 1 -- 172.21.15.33:0/2654271419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca64198fa0 msgr2=0x7fca64199450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:51:08.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.903+0000 7fca6be92700 1 --2- 172.21.15.33:0/2654271419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca64198fa0 0x7fca64199450 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fca5c00b550 tx=0x7fca5c00b860 comp rx=0 tx=0).stop 2024-05-15T13:51:08.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.903+0000 7fca6be92700 1 -- 172.21.15.33:0/2654271419 shutdown_connections 2024-05-15T13:51:08.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.903+0000 7fca6be92700 1 --2- 172.21.15.33:0/2654271419 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fca4c03e9a0 0x7fca4c040e60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:08.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.903+0000 7fca6be92700 1 --2- 172.21.15.33:0/2654271419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca64198fa0 0x7fca64199450 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:08.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.903+0000 7fca6be92700 1 --2- 172.21.15.33:0/2654271419 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca641020f0 0x7fca6419e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:08.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.903+0000 7fca6be92700 1 --2- 172.21.15.33:0/2654271419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca6419ede0 0x7fca64198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:08.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.903+0000 7fca6be92700 1 -- 172.21.15.33:0/2654271419 >> 172.21.15.33:0/2654271419 conn(0x7fca640779d0 msgr2=0x7fca6406a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:51:08.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.903+0000 7fca6be92700 1 -- 172.21.15.33:0/2654271419 shutdown_connections 2024-05-15T13:51:08.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:08.903+0000 7fca6be92700 1 -- 172.21.15.33:0/2654271419 wait complete. 2024-05-15T13:51:09.475 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-15T13:51:09.475 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-15T13:51:09.476 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.conf 2024-05-15T13:51:09.505 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-15T13:51:09.505 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-15T13:51:09.572 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-15T13:51:09.572 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/ceph/ceph.conf 2024-05-15T13:51:09.602 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-15T13:51:09.602 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-15T13:51:09.668 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-15T13:51:09.668 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/ceph/ceph.conf 2024-05-15T13:51:09.700 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-15T13:51:09.700 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-15T13:51:09.765 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:09 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:09.765 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:09 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:09.765 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:09 smithi081 ceph-mon[36434]: Reconfiguring mon.b (monmap changed)... 2024-05-15T13:51:09.765 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:09 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-15T13:51:09.765 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:09 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-15T13:51:09.765 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:09 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:09.765 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:09 smithi081 ceph-mon[36434]: Reconfiguring daemon mon.b on smithi052 2024-05-15T13:51:09.765 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:09 smithi081 ceph-mon[36434]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:09.765 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:09 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/2654271419' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:09.769 INFO:tasks.cephadm:Adding mgr.x on smithi033 2024-05-15T13:51:09.769 INFO:tasks.cephadm:Adding mgr.y on smithi052 2024-05-15T13:51:09.769 INFO:tasks.cephadm:Adding mgr.z on smithi081 2024-05-15T13:51:09.769 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph orch apply mgr '3;smithi033=x;smithi052=y;smithi081=z' 2024-05-15T13:51:09.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:09 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:09.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:09 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:09.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:09 smithi052 ceph-mon[36190]: Reconfiguring mon.b (monmap changed)... 2024-05-15T13:51:09.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:09 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-15T13:51:09.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:09 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-15T13:51:09.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:09 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:09.842 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:09 smithi052 ceph-mon[36190]: Reconfiguring daemon mon.b on smithi052 2024-05-15T13:51:09.842 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:09 smithi052 ceph-mon[36190]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:09.842 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:09 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/2654271419' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:09.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:09 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:09.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:09 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:09.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:09 smithi033 ceph-mon[31650]: Reconfiguring mon.b (monmap changed)... 2024-05-15T13:51:09.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:09 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-15T13:51:09.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:09 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-15T13:51:09.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:09 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:09.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:09 smithi033 ceph-mon[31650]: Reconfiguring daemon mon.b on smithi052 2024-05-15T13:51:09.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:09 smithi033 ceph-mon[31650]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:09.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:09 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/2654271419' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:10.045 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.c/config 2024-05-15T13:51:11.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:11 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:11.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:11 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:11.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:11 smithi081 ceph-mon[36434]: Reconfiguring mon.c (monmap changed)... 2024-05-15T13:51:11.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:11 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-15T13:51:11.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:11 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-15T13:51:11.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:11 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:11.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:11 smithi081 ceph-mon[36434]: Reconfiguring daemon mon.c on smithi081 2024-05-15T13:51:11.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:11 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:11.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:11 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:11.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:11 smithi033 ceph-mon[31650]: Reconfiguring mon.c (monmap changed)... 2024-05-15T13:51:11.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:11 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-15T13:51:11.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:11 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-15T13:51:11.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:11 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:11.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:11 smithi033 ceph-mon[31650]: Reconfiguring daemon mon.c on smithi081 2024-05-15T13:51:11.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:11 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:11.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:11 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:11.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:11 smithi052 ceph-mon[36190]: Reconfiguring mon.c (monmap changed)... 2024-05-15T13:51:11.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:11 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-15T13:51:11.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:11 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-15T13:51:11.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:11 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:11.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:11 smithi052 ceph-mon[36190]: Reconfiguring daemon mon.c on smithi081 2024-05-15T13:51:12.423 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:12 smithi081 ceph-mon[36434]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:12.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:12 smithi033 ceph-mon[31650]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:12.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.522+0000 7f903dfa3700 1 -- 172.21.15.81:0/3007491203 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f90381036b0 msgr2=0x7f9038103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:51:12.526 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.522+0000 7f903dfa3700 1 --2- 172.21.15.81:0/3007491203 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f90381036b0 0x7f9038103b10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f902c0098f0 tx=0x7f902c009c00 comp rx=0 tx=0).stop 2024-05-15T13:51:12.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.522+0000 7f903dfa3700 1 -- 172.21.15.81:0/3007491203 shutdown_connections 2024-05-15T13:51:12.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.522+0000 7f903dfa3700 1 --2- 172.21.15.81:0/3007491203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9038104050 0x7f903810a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:12.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.522+0000 7f903dfa3700 1 --2- 172.21.15.81:0/3007491203 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90381024b0 0x7f90381028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:12.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.522+0000 7f903dfa3700 1 --2- 172.21.15.81:0/3007491203 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f90381036b0 0x7f9038103b10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:12.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.522+0000 7f903dfa3700 1 -- 172.21.15.81:0/3007491203 >> 172.21.15.81:0/3007491203 conn(0x7f90380fda70 msgr2=0x7f90380ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:51:12.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.522+0000 7f903dfa3700 1 -- 172.21.15.81:0/3007491203 shutdown_connections 2024-05-15T13:51:12.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.522+0000 7f903dfa3700 1 -- 172.21.15.81:0/3007491203 wait complete. 2024-05-15T13:51:12.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.522+0000 7f903dfa3700 1 Processor -- start 2024-05-15T13:51:12.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.523+0000 7f903dfa3700 1 -- start start 2024-05-15T13:51:12.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.523+0000 7f903dfa3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90381024b0 0x7f9038106630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:51:12.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.523+0000 7f903dfa3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90381036b0 0x7f9038104cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:51:12.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.523+0000 7f903dfa3700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9038104050 0x7f9038105210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:51:12.527 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.523+0000 7f903dfa3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f903810df50 con 0x7f90381036b0 2024-05-15T13:51:12.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.523+0000 7f903dfa3700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f903810ddd0 con 0x7f90381024b0 2024-05-15T13:51:12.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.523+0000 7f903dfa3700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f903810e0d0 con 0x7f9038104050 2024-05-15T13:51:12.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.523+0000 7f9037fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9038104050 0x7f9038105210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:51:12.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.523+0000 7f9037fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9038104050 0x7f9038105210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:44002/0 (socket says 172.21.15.81:44002) 2024-05-15T13:51:12.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.523+0000 7f9037fff700 1 -- 172.21.15.81:0/4049058650 learned_addr learned my addr 172.21.15.81:0/4049058650 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-15T13:51:12.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.523+0000 7f9036ffd700 1 --2- 172.21.15.81:0/4049058650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90381036b0 0x7f9038104cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:51:12.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.523+0000 7f90377fe700 1 --2- 172.21.15.81:0/4049058650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90381024b0 0x7f9038106630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:51:12.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.523+0000 7f9037fff700 1 -- 172.21.15.81:0/4049058650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90381024b0 msgr2=0x7f9038106630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:51:12.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.523+0000 7f9037fff700 1 --2- 172.21.15.81:0/4049058650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90381024b0 0x7f9038106630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:12.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.523+0000 7f9037fff700 1 -- 172.21.15.81:0/4049058650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90381036b0 msgr2=0x7f9038104cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:51:12.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.523+0000 7f9037fff700 1 --2- 172.21.15.81:0/4049058650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90381036b0 0x7f9038104cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:12.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.523+0000 7f9037fff700 1 -- 172.21.15.81:0/4049058650 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9038105ae0 con 0x7f9038104050 2024-05-15T13:51:12.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.523+0000 7f9037fff700 1 --2- 172.21.15.81:0/4049058650 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9038104050 0x7f9038105210 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f903000c110 tx=0x7f903000c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:51:12.528 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.524+0000 7f9034ff9700 1 -- 172.21.15.81:0/4049058650 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90300092e0 con 0x7f9038104050 2024-05-15T13:51:12.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.524+0000 7f9034ff9700 1 -- 172.21.15.81:0/4049058650 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9030004bb0 con 0x7f9038104050 2024-05-15T13:51:12.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.524+0000 7f903dfa3700 1 -- 172.21.15.81:0/4049058650 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90381a3950 con 0x7f9038104050 2024-05-15T13:51:12.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.524+0000 7f9034ff9700 1 -- 172.21.15.81:0/4049058650 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9030007ce0 con 0x7f9038104050 2024-05-15T13:51:12.529 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.524+0000 7f903dfa3700 1 -- 172.21.15.81:0/4049058650 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90381a3e60 con 0x7f9038104050 2024-05-15T13:51:12.530 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.524+0000 7f9034ff9700 1 -- 172.21.15.81:0/4049058650 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f903001a020 con 0x7f9038104050 2024-05-15T13:51:12.530 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.525+0000 7f903dfa3700 1 -- 172.21.15.81:0/4049058650 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f903806a230 con 0x7f9038104050 2024-05-15T13:51:12.530 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.525+0000 7f9034ff9700 1 --2- 172.21.15.81:0/4049058650 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f902003e750 0x7f9020040c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:51:12.530 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.525+0000 7f9034ff9700 1 -- 172.21.15.81:0/4049058650 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9030052640 con 0x7f9038104050 2024-05-15T13:51:12.530 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.525+0000 7f90377fe700 1 --2- 172.21.15.81:0/4049058650 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f902003e750 0x7f9020040c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:51:12.530 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.525+0000 7f90377fe700 1 --2- 172.21.15.81:0/4049058650 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f902003e750 0x7f9020040c10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9038103510 tx=0x7f9028009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:51:12.532 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.530+0000 7f9034ff9700 1 -- 172.21.15.81:0/4049058650 <== mon.1 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9030010040 con 0x7f9038104050 2024-05-15T13:51:12.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:12 smithi052 ceph-mon[36190]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:12.747 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.744+0000 7f903dfa3700 1 -- 172.21.15.81:0/4049058650 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi033=x;smithi052=y;smithi081=z", "target": ["mon-mgr", ""]}) -- 0x7f903806c3a0 con 0x7f902003e750 2024-05-15T13:51:12.752 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.750+0000 7f9034ff9700 1 -- 172.21.15.81:0/4049058650 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f903806c3a0 con 0x7f902003e750 2024-05-15T13:51:12.753 INFO:teuthology.orchestra.run.smithi081.stdout:Scheduled mgr update... 2024-05-15T13:51:12.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.752+0000 7f903dfa3700 1 -- 172.21.15.81:0/4049058650 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f902003e750 msgr2=0x7f9020040c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:51:12.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.752+0000 7f903dfa3700 1 --2- 172.21.15.81:0/4049058650 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f902003e750 0x7f9020040c10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9038103510 tx=0x7f9028009260 comp rx=0 tx=0).stop 2024-05-15T13:51:12.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.752+0000 7f903dfa3700 1 -- 172.21.15.81:0/4049058650 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9038104050 msgr2=0x7f9038105210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:51:12.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.752+0000 7f903dfa3700 1 --2- 172.21.15.81:0/4049058650 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9038104050 0x7f9038105210 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f903000c110 tx=0x7f903000c420 comp rx=0 tx=0).stop 2024-05-15T13:51:12.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.752+0000 7f903dfa3700 1 -- 172.21.15.81:0/4049058650 shutdown_connections 2024-05-15T13:51:12.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.752+0000 7f903dfa3700 1 --2- 172.21.15.81:0/4049058650 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f902003e750 0x7f9020040c10 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:12.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.752+0000 7f903dfa3700 1 --2- 172.21.15.81:0/4049058650 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9038104050 0x7f9038105210 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:12.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.752+0000 7f903dfa3700 1 --2- 172.21.15.81:0/4049058650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90381024b0 0x7f9038106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:12.755 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.752+0000 7f903dfa3700 1 --2- 172.21.15.81:0/4049058650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90381036b0 0x7f9038104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:12.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.752+0000 7f903dfa3700 1 -- 172.21.15.81:0/4049058650 >> 172.21.15.81:0/4049058650 conn(0x7f90380fda70 msgr2=0x7f903806bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:51:12.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.753+0000 7f903dfa3700 1 -- 172.21.15.81:0/4049058650 shutdown_connections 2024-05-15T13:51:12.756 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:51:12.753+0000 7f903dfa3700 1 -- 172.21.15.81:0/4049058650 wait complete. 2024-05-15T13:51:13.370 DEBUG:teuthology.orchestra.run.smithi052:mgr.y> sudo journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mgr.y.service 2024-05-15T13:51:13.373 DEBUG:teuthology.orchestra.run.smithi081:mgr.z> sudo journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mgr.z.service 2024-05-15T13:51:13.375 INFO:tasks.cephadm:Deploying OSDs... 2024-05-15T13:51:13.375 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-15T13:51:13.375 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/scratch_devs of=/dev/stdout 2024-05-15T13:51:13.394 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-15T13:51:13.394 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_1 2024-05-15T13:51:13.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:13 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:13.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:13 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:13.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:13 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:51:13.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:13 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:13.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:13 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:13.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:13 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:51:13.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:13 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:13.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:13 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-15T13:51:13.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:13 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-15T13:51:13.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:13 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-15T13:51:13.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:13 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:13.451 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-15T13:51:13.451 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-15T13:51:13.451 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-15T13:51:13.452 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-15T13:51:13.452 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-05-15T13:51:13.452 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-05-15 13:50:15.559500760 +0000 2024-05-15T13:51:13.452 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-05-15 13:45:10.540314348 +0000 2024-05-15T13:51:13.452 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-05-15 13:45:10.540314348 +0000 2024-05-15T13:51:13.452 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-05-15 13:45:10.540314348 +0000 2024-05-15T13:51:13.452 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-15T13:51:13.519 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-05-15T13:51:13.519 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-05-15T13:51:13.519 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000212995 s, 2.4 MB/s 2024-05-15T13:51:13.520 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-15T13:51:13.556 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:13 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:13.556 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:13 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:13.556 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:13 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:51:13.556 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:13 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:13.556 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:13 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:13.556 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:13 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:51:13.556 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:13 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:13.556 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:13 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-15T13:51:13.556 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:13 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-15T13:51:13.556 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:13 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-15T13:51:13.557 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:13 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:13.579 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_2 2024-05-15T13:51:13.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:13 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:13.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:13 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:13.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:13 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:51:13.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:13 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:13.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:13 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:13.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:13 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:51:13.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:13 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:13.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:13 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-15T13:51:13.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:13 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-15T13:51:13.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:13 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-15T13:51:13.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:13 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:13.637 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-15T13:51:13.638 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-15T13:51:13.638 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-15T13:51:13.638 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-15T13:51:13.638 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-05-15T13:51:13.638 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-05-15 13:50:15.587500323 +0000 2024-05-15T13:51:13.638 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-05-15 13:45:10.895308811 +0000 2024-05-15T13:51:13.638 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-05-15 13:45:10.895308811 +0000 2024-05-15T13:51:13.638 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-05-15 13:45:10.895308811 +0000 2024-05-15T13:51:13.638 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-15T13:51:13.704 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-05-15T13:51:13.705 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-05-15T13:51:13.705 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000204549 s, 2.5 MB/s 2024-05-15T13:51:13.706 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-15T13:51:13.766 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_3 2024-05-15T13:51:13.824 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-15T13:51:13.824 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-15T13:51:13.824 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-15T13:51:13.824 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-15T13:51:13.824 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-05-15T13:51:13.824 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-05-15 13:50:15.613499918 +0000 2024-05-15T13:51:13.824 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-05-15 13:45:11.243303383 +0000 2024-05-15T13:51:13.824 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-05-15 13:45:11.243303383 +0000 2024-05-15T13:51:13.824 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-05-15 13:45:11.243303383 +0000 2024-05-15T13:51:13.825 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-15T13:51:13.891 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-05-15T13:51:13.891 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-05-15T13:51:13.891 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000202811 s, 2.5 MB/s 2024-05-15T13:51:13.892 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-15T13:51:13.951 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_4 2024-05-15T13:51:14.009 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-15T13:51:14.009 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-15T13:51:14.009 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-05-15T13:51:14.009 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-15T13:51:14.009 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-05-15T13:51:14.010 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-05-15 13:50:15.645499419 +0000 2024-05-15T13:51:14.010 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-05-15 13:45:11.586298031 +0000 2024-05-15T13:51:14.010 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-05-15 13:45:11.586298031 +0000 2024-05-15T13:51:14.010 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-05-15 13:45:11.586298031 +0000 2024-05-15T13:51:14.010 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-15T13:51:14.076 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-05-15T13:51:14.076 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-05-15T13:51:14.076 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000197703 s, 2.6 MB/s 2024-05-15T13:51:14.078 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-15T13:51:14.136 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-05-15T13:51:14.136 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/scratch_devs of=/dev/stdout 2024-05-15T13:51:14.155 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-15T13:51:14.155 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_1 2024-05-15T13:51:14.215 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-15T13:51:14.215 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-15T13:51:14.215 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-15T13:51:14.215 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-15T13:51:14.215 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2024-05-15T13:51:14.215 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-05-15 13:50:58.258379711 +0000 2024-05-15T13:51:14.215 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-05-15 13:45:24.796853720 +0000 2024-05-15T13:51:14.215 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-05-15 13:45:24.796853720 +0000 2024-05-15T13:51:14.215 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-05-15 13:45:24.796853720 +0000 2024-05-15T13:51:14.216 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-15T13:51:14.283 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-05-15T13:51:14.284 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-05-15T13:51:14.284 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.000129931 s, 3.9 MB/s 2024-05-15T13:51:14.285 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-15T13:51:14.346 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_2 2024-05-15T13:51:14.405 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-15T13:51:14.405 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-15T13:51:14.405 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-15T13:51:14.405 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-15T13:51:14.405 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2024-05-15T13:51:14.405 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-05-15 13:50:58.259379699 +0000 2024-05-15T13:51:14.405 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-05-15 13:45:25.126849492 +0000 2024-05-15T13:51:14.405 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-05-15 13:45:25.126849492 +0000 2024-05-15T13:51:14.405 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-05-15 13:45:25.126849492 +0000 2024-05-15T13:51:14.406 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-15T13:51:14.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:14 smithi033 ceph-mon[31650]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi033=x;smithi052=y;smithi081=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:51:14.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:14 smithi033 ceph-mon[31650]: Saving service mgr spec with placement smithi033=x;smithi052=y;smithi081=z;count:3 2024-05-15T13:51:14.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:14 smithi033 ceph-mon[31650]: Deploying daemon mgr.z on smithi081 2024-05-15T13:51:14.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:14 smithi033 ceph-mon[31650]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:14.469 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:14 smithi052 ceph-mon[36190]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi033=x;smithi052=y;smithi081=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:51:14.469 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:14 smithi052 ceph-mon[36190]: Saving service mgr spec with placement smithi033=x;smithi052=y;smithi081=z;count:3 2024-05-15T13:51:14.469 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:14 smithi052 ceph-mon[36190]: Deploying daemon mgr.z on smithi081 2024-05-15T13:51:14.469 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:14 smithi052 ceph-mon[36190]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:14.471 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-05-15T13:51:14.471 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-05-15T13:51:14.471 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.0001179 s, 4.3 MB/s 2024-05-15T13:51:14.472 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-15T13:51:14.531 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_3 2024-05-15T13:51:14.589 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-15T13:51:14.589 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-15T13:51:14.589 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-05-15T13:51:14.589 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-15T13:51:14.589 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2024-05-15T13:51:14.589 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-05-15 13:50:58.259379699 +0000 2024-05-15T13:51:14.589 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-05-15 13:45:25.472845059 +0000 2024-05-15T13:51:14.589 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-05-15 13:45:25.472845059 +0000 2024-05-15T13:51:14.589 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-05-15 13:45:25.472845059 +0000 2024-05-15T13:51:14.590 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-15T13:51:14.591 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:14 smithi081 ceph-mon[36434]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi033=x;smithi052=y;smithi081=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:51:14.591 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:14 smithi081 ceph-mon[36434]: Saving service mgr spec with placement smithi033=x;smithi052=y;smithi081=z;count:3 2024-05-15T13:51:14.591 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:14 smithi081 ceph-mon[36434]: Deploying daemon mgr.z on smithi081 2024-05-15T13:51:14.591 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:14 smithi081 ceph-mon[36434]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:14.654 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-05-15T13:51:14.655 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-05-15T13:51:14.655 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.000106966 s, 4.8 MB/s 2024-05-15T13:51:14.656 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-15T13:51:14.714 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_4 2024-05-15T13:51:14.772 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-15T13:51:14.772 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-15T13:51:14.772 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-05-15T13:51:14.773 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-15T13:51:14.773 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2024-05-15T13:51:14.773 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-05-15 13:50:58.260379687 +0000 2024-05-15T13:51:14.773 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-05-15 13:45:25.841840331 +0000 2024-05-15T13:51:14.773 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-05-15 13:45:25.841840331 +0000 2024-05-15T13:51:14.773 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-05-15 13:45:25.841840331 +0000 2024-05-15T13:51:14.773 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-15T13:51:14.838 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-05-15T13:51:14.838 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-05-15T13:51:14.838 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.000114369 s, 4.5 MB/s 2024-05-15T13:51:14.839 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-15T13:51:14.857 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 13:51:14 smithi081 systemd[1]: Starting Ceph mgr.z for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T13:51:14.898 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-15T13:51:14.898 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/scratch_devs of=/dev/stdout 2024-05-15T13:51:14.916 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-15T13:51:14.916 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_1 2024-05-15T13:51:14.971 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-15T13:51:14.971 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-15T13:51:14.971 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-05-15T13:51:14.971 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-15T13:51:14.971 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-05-15T13:51:14.971 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-05-15 13:51:14.694041745 +0000 2024-05-15T13:51:14.972 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-05-15 13:45:14.654088298 +0000 2024-05-15T13:51:14.972 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-05-15 13:45:14.654088298 +0000 2024-05-15T13:51:14.972 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-05-15 13:45:14.654088298 +0000 2024-05-15T13:51:14.972 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-15T13:51:15.036 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-05-15T13:51:15.036 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-05-15T13:51:15.036 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000208062 s, 2.5 MB/s 2024-05-15T13:51:15.037 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-15T13:51:15.092 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_2 2024-05-15T13:51:15.191 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-15T13:51:15.191 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-15T13:51:15.191 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-15T13:51:15.191 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-15T13:51:15.191 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-05-15T13:51:15.191 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-05-15 13:51:14.694041745 +0000 2024-05-15T13:51:15.191 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-05-15 13:45:14.999082082 +0000 2024-05-15T13:51:15.191 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-05-15 13:45:14.999082082 +0000 2024-05-15T13:51:15.191 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-05-15 13:45:14.999082082 +0000 2024-05-15T13:51:15.192 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-15T13:51:15.224 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 13:51:14 smithi081 podman[38047]: 2024-05-15 13:51:14.87226682 +0000 UTC m=+0.016435504 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-15T13:51:15.224 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 13:51:15 smithi081 podman[38047]: 2024-05-15 13:51:15.001076991 +0000 UTC m=+0.145245656 container create 9f81e1ae812cab180772f94d1b95355371948ee107288c81f81e80b34c8a8a38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-15T13:51:15.224 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 13:51:15 smithi081 podman[38047]: 2024-05-15 13:51:15.195367018 +0000 UTC m=+0.339535689 container init 9f81e1ae812cab180772f94d1b95355371948ee107288c81f81e80b34c8a8a38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T13:51:15.224 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 13:51:15 smithi081 podman[38047]: 2024-05-15 13:51:15.19789468 +0000 UTC m=+0.342063343 container start 9f81e1ae812cab180772f94d1b95355371948ee107288c81f81e80b34c8a8a38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-15T13:51:15.248 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-05-15T13:51:15.248 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-05-15T13:51:15.248 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000211723 s, 2.4 MB/s 2024-05-15T13:51:15.249 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-15T13:51:15.305 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_3 2024-05-15T13:51:15.360 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-15T13:51:15.360 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-15T13:51:15.360 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-05-15T13:51:15.360 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-15T13:51:15.360 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-05-15T13:51:15.361 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-05-15 13:51:14.695041728 +0000 2024-05-15T13:51:15.361 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-05-15 13:45:15.341075921 +0000 2024-05-15T13:51:15.361 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-05-15 13:45:15.341075921 +0000 2024-05-15T13:51:15.361 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-05-15 13:45:15.341075921 +0000 2024-05-15T13:51:15.361 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-15T13:51:15.423 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-05-15T13:51:15.423 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-05-15T13:51:15.423 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000215071 s, 2.4 MB/s 2024-05-15T13:51:15.424 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-15T13:51:15.480 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_4 2024-05-15T13:51:15.535 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-15T13:51:15.536 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-15T13:51:15.536 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-05-15T13:51:15.536 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-15T13:51:15.536 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-05-15T13:51:15.536 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-05-15 13:51:14.695041728 +0000 2024-05-15T13:51:15.536 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-05-15 13:45:15.675069903 +0000 2024-05-15T13:51:15.536 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-05-15 13:45:15.675069903 +0000 2024-05-15T13:51:15.536 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-05-15 13:45:15.675069903 +0000 2024-05-15T13:51:15.536 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-15T13:51:15.547 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 13:51:15 smithi081 ceph-mgr[38151]: -- 172.21.15.81:0/3128156701 <== mon.1 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ae030fc5a0 con 0x55ae030d1000 2024-05-15T13:51:15.547 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 13:51:15 smithi081 bash[38047]: 9f81e1ae812cab180772f94d1b95355371948ee107288c81f81e80b34c8a8a38 2024-05-15T13:51:15.547 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 13:51:15 smithi081 systemd[1]: Started Ceph mgr.z for bca78ca4-12c1-11ef-bc98-c7b262605968. 2024-05-15T13:51:15.547 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 13:51:15 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T13:51:15.353+0000 7fb793793200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-15T13:51:15.547 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 13:51:15 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T13:51:15.544+0000 7fb793793200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-15T13:51:15.569 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-05-15T13:51:15.569 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-05-15T13:51:15.569 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000216532 s, 2.4 MB/s 2024-05-15T13:51:15.570 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-15T13:51:15.625 INFO:tasks.cephadm:Deploying osd.0 on smithi033 with /dev/vg_nvme/lv_4... 2024-05-15T13:51:15.626 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-15T13:51:15.857 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 13:51:15 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T13:51:15.650+0000 7fb793793200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-15T13:51:15.879 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:51:16.347 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 13:51:15 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T13:51:15.957+0000 7fb793793200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-15T13:51:16.347 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 13:51:16 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T13:51:16.041+0000 7fb793793200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-15T13:51:16.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:16 smithi052 ceph-mon[36190]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:16.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:16 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:16.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:16 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:16.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:16 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:16.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:16 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-15T13:51:16.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:16 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-15T13:51:16.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:16 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-15T13:51:16.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:16 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:16.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:16 smithi052 ceph-mon[36190]: Deploying daemon mgr.y on smithi052 2024-05-15T13:51:16.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:16 smithi081 ceph-mon[36434]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:16.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:16 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:16.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:16 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:16.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:16 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:16.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:16 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-15T13:51:16.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:16 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-15T13:51:16.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:16 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-15T13:51:16.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:16 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:16.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:16 smithi081 ceph-mon[36434]: Deploying daemon mgr.y on smithi052 2024-05-15T13:51:16.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:16 smithi033 ceph-mon[31650]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:16.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:16 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:16.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:16 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:16.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:16 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:16.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:16 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-15T13:51:16.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:16 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-15T13:51:16.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:16 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-15T13:51:16.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:16 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:16.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:16 smithi033 ceph-mon[31650]: Deploying daemon mgr.y on smithi052 2024-05-15T13:51:17.543 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 13:51:17 smithi052 systemd[1]: Starting Ceph mgr.y for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T13:51:17.607 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 13:51:17 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T13:51:17.255+0000 7fb793793200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-15T13:51:17.607 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 13:51:17 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T13:51:17.430+0000 7fb793793200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-15T13:51:17.607 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 13:51:17 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T13:51:17.603+0000 7fb793793200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-15T13:51:17.736 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-15T13:51:17.736 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-15T13:51:17.736 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-05-15T13:51:17.736 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-05-15T13:51:17.736 INFO:teuthology.orchestra.run.smithi033.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111757 s, 938 MB/s 2024-05-15T13:51:17.736 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-05-15T13:51:17.804 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 13:51:17 smithi052 podman[37562]: 2024-05-15 13:51:17.444042376 +0000 UTC m=+0.018082426 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-15T13:51:17.804 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 13:51:17 smithi052 podman[37562]: 2024-05-15 13:51:17.541815467 +0000 UTC m=+0.115855570 container create c0b639a4f8996095e6496e0a510008e7560a8122dcc085d15a03b304c1586b6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T13:51:17.804 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 13:51:17 smithi052 podman[37562]: 2024-05-15 13:51:17.735966006 +0000 UTC m=+0.310006055 container init c0b639a4f8996095e6496e0a510008e7560a8122dcc085d15a03b304c1586b6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=) 2024-05-15T13:51:17.805 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 13:51:17 smithi052 podman[37562]: 2024-05-15 13:51:17.738642791 +0000 UTC m=+0.312682827 container start c0b639a4f8996095e6496e0a510008e7560a8122dcc085d15a03b304c1586b6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-15T13:51:17.805 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 13:51:17 smithi052 ceph-mgr[37581]: -- 172.21.15.52:0/1512579941 <== mon.2 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564de12d25a0 con 0x564de12a7000 2024-05-15T13:51:17.805 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 13:51:17 smithi052 bash[37562]: c0b639a4f8996095e6496e0a510008e7560a8122dcc085d15a03b304c1586b6b 2024-05-15T13:51:18.091 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 13:51:17 smithi052 systemd[1]: Started Ceph mgr.y for bca78ca4-12c1-11ef-bc98-c7b262605968. 2024-05-15T13:51:18.091 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 13:51:17 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T13:51:17.904+0000 7f27a80bb200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-15T13:51:18.202 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 13:51:17 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T13:51:17.902+0000 7fb793793200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-15T13:51:18.202 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 13:51:17 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T13:51:17.993+0000 7fb793793200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-15T13:51:18.203 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 13:51:18 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T13:51:18.199+0000 7fb793793200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-15T13:51:18.350 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:18 smithi033 ceph-mon[31650]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:18.350 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:18 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:18.350 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:18 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:18.350 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:18 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:18.351 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:18 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:18.351 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:18 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:51:18.351 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:18 smithi052 ceph-mon[36190]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:18.352 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:18 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:18.352 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:18 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:18.352 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 13:51:18 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T13:51:18.097+0000 7f27a80bb200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-15T13:51:18.352 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 13:51:18 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T13:51:18.208+0000 7f27a80bb200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-15T13:51:18.386 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_4 2024-05-15T13:51:18.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:18 smithi081 ceph-mon[36434]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:18.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:18 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:18.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:18 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:18.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:18 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:18.609 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:18 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:18.609 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:18 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:51:18.609 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 13:51:18 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T13:51:18.402+0000 7fb793793200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-15T13:51:18.630 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:51:18.646 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 13:51:18 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T13:51:18.542+0000 7f27a80bb200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-15T13:51:18.647 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:18 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:18.647 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:18 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:18.647 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:18 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:51:18.909 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 13:51:18 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T13:51:18.644+0000 7f27a80bb200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-15T13:51:19.257 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 13:51:19 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T13:51:19.033+0000 7fb793793200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-15T13:51:19.257 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 13:51:19 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T13:51:19.254+0000 7fb793793200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-15T13:51:20.107 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 13:51:19 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T13:51:19.680+0000 7fb793793200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-15T13:51:20.108 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 13:51:19 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T13:51:19.774+0000 7fb793793200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-15T13:51:20.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.179+0000 7f068508f700 1 -- 172.21.15.33:0/1670551992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0680109290 msgr2=0x7f068010b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:51:20.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.179+0000 7f068508f700 1 --2- 172.21.15.33:0/1670551992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0680109290 0x7f068010b700 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0674009990 tx=0x7f0674009ca0 comp rx=0 tx=0).stop 2024-05-15T13:51:20.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.179+0000 7f068508f700 1 -- 172.21.15.33:0/1670551992 shutdown_connections 2024-05-15T13:51:20.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.179+0000 7f068508f700 1 --2- 172.21.15.33:0/1670551992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0680109290 0x7f068010b700 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:20.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.179+0000 7f068508f700 1 --2- 172.21.15.33:0/1670551992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0680104030 0x7f0680106420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:20.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.179+0000 7f068508f700 1 --2- 172.21.15.33:0/1670551992 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0680106960 0x7f0680108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:20.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.179+0000 7f068508f700 1 -- 172.21.15.33:0/1670551992 >> 172.21.15.33:0/1670551992 conn(0x7f06800fda60 msgr2=0x7f06800ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:51:20.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.179+0000 7f068508f700 1 -- 172.21.15.33:0/1670551992 shutdown_connections 2024-05-15T13:51:20.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.179+0000 7f068508f700 1 -- 172.21.15.33:0/1670551992 wait complete. 2024-05-15T13:51:20.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.180+0000 7f068508f700 1 Processor -- start 2024-05-15T13:51:20.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.180+0000 7f068508f700 1 -- start start 2024-05-15T13:51:20.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.180+0000 7f068508f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0680104030 0x7f06801996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:51:20.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.180+0000 7f068508f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0680106960 0x7f0680199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:51:20.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.180+0000 7f068508f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0680109290 0x7f06801a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:51:20.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.180+0000 7f068508f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f068010f0b0 con 0x7f0680109290 2024-05-15T13:51:20.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.180+0000 7f068508f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f068010ef30 con 0x7f0680104030 2024-05-15T13:51:20.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.180+0000 7f068508f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f068010f230 con 0x7f0680106960 2024-05-15T13:51:20.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.180+0000 7f067f59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0680109290 0x7f06801a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:51:20.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.180+0000 7f067e59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0680106960 0x7f0680199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:51:20.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.180+0000 7f067f59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0680109290 0x7f06801a0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44020/0 (socket says 172.21.15.33:44020) 2024-05-15T13:51:20.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.180+0000 7f067ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0680104030 0x7f06801996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:51:20.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.180+0000 7f067f59e700 1 -- 172.21.15.33:0/4006254 learned_addr learned my addr 172.21.15.33:0/4006254 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:51:20.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.181+0000 7f067f59e700 1 -- 172.21.15.33:0/4006254 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0680106960 msgr2=0x7f0680199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:51:20.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.181+0000 7f067f59e700 1 --2- 172.21.15.33:0/4006254 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0680106960 0x7f0680199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:20.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.181+0000 7f067f59e700 1 -- 172.21.15.33:0/4006254 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0680104030 msgr2=0x7f06801996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:51:20.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.181+0000 7f067f59e700 1 --2- 172.21.15.33:0/4006254 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0680104030 0x7f06801996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:20.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.181+0000 7f067f59e700 1 -- 172.21.15.33:0/4006254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06801a1300 con 0x7f0680109290 2024-05-15T13:51:20.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.181+0000 7f067ed9d700 1 --2- 172.21.15.33:0/4006254 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0680104030 0x7f06801996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T13:51:20.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.181+0000 7f067e59c700 1 --2- 172.21.15.33:0/4006254 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0680106960 0x7f0680199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T13:51:20.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.181+0000 7f067f59e700 1 --2- 172.21.15.33:0/4006254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0680109290 0x7f06801a0c60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0674009510 tx=0x7f0674009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:51:20.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.181+0000 7f0667fff700 1 -- 172.21.15.33:0/4006254 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067400fa80 con 0x7f0680109290 2024-05-15T13:51:20.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.181+0000 7f068508f700 1 -- 172.21.15.33:0/4006254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06801a1590 con 0x7f0680109290 2024-05-15T13:51:20.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.181+0000 7f0667fff700 1 -- 172.21.15.33:0/4006254 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0674004cc0 con 0x7f0680109290 2024-05-15T13:51:20.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.181+0000 7f0667fff700 1 -- 172.21.15.33:0/4006254 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06740055c0 con 0x7f0680109290 2024-05-15T13:51:20.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.181+0000 7f068508f700 1 -- 172.21.15.33:0/4006254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06801a19e0 con 0x7f0680109290 2024-05-15T13:51:20.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.182+0000 7f0667fff700 1 -- 172.21.15.33:0/4006254 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f0674009d10 con 0x7f0680109290 2024-05-15T13:51:20.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.182+0000 7f068508f700 1 -- 172.21.15.33:0/4006254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f068006a230 con 0x7f0680109290 2024-05-15T13:51:20.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.182+0000 7f0667fff700 1 --2- 172.21.15.33:0/4006254 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f066803e750 0x7f0668040c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:51:20.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.182+0000 7f0667fff700 1 -- 172.21.15.33:0/4006254 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f06740535c0 con 0x7f0680109290 2024-05-15T13:51:20.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.183+0000 7f067ed9d700 1 --2- 172.21.15.33:0/4006254 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f066803e750 0x7f0668040c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:51:20.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.183+0000 7f067ed9d700 1 --2- 172.21.15.33:0/4006254 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f066803e750 0x7f0668040c10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0670011490 tx=0x7f0670009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:51:20.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.188+0000 7f0667fff700 1 -- 172.21.15.33:0/4006254 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0674013070 con 0x7f0680109290 2024-05-15T13:51:20.302 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 13:51:19 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T13:51:19.945+0000 7f27a80bb200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-15T13:51:20.302 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 13:51:20 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T13:51:20.122+0000 7f27a80bb200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-15T13:51:20.302 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 13:51:20 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T13:51:20.299+0000 7f27a80bb200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-15T13:51:20.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:20.414+0000 7f068508f700 1 -- 172.21.15.33:0/4006254 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f0680063980 con 0x7f066803e750 2024-05-15T13:51:20.488 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 13:51:20 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T13:51:20.136+0000 7fb793793200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-15T13:51:20.488 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 13:51:20 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T13:51:20.485+0000 7fb793793200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-15T13:51:20.488 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:20 smithi081 ceph-mon[36434]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:20.488 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:20 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:20.488 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:20 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:20.488 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:20 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:20.488 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:20 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:20.488 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:20 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:20.489 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:20 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:51:20.489 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:20 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:20.489 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:20 smithi081 ceph-mon[36434]: Reconfiguring mgr.x (unknown last config time)... 2024-05-15T13:51:20.489 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:20 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-15T13:51:20.489 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:20 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-15T13:51:20.489 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:20 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:20.489 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:20 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:20.521 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:20 smithi033 ceph-mon[31650]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:20.521 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:20 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:20.521 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:20 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:20.521 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:20 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:20.521 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:20 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:20.521 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:20 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:20.521 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:20 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:51:20.521 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:20 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:20.521 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:20 smithi033 ceph-mon[31650]: Reconfiguring mgr.x (unknown last config time)... 2024-05-15T13:51:20.521 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:20 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-15T13:51:20.521 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:20 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-15T13:51:20.522 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:20 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:20.522 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:20 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:20.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:20 smithi052 ceph-mon[36190]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:20.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:20 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:20.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:20 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:20.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:20 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:20.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:20 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:20.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:20 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:20.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:20 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:51:20.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:20 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:20.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:20 smithi052 ceph-mon[36190]: Reconfiguring mgr.x (unknown last config time)... 2024-05-15T13:51:20.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:20 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-15T13:51:20.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:20 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-15T13:51:20.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:20 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:20.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:20 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:20.815 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 13:51:20 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T13:51:20.576+0000 7fb793793200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-15T13:51:20.909 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 13:51:20 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T13:51:20.600+0000 7f27a80bb200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-15T13:51:20.909 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 13:51:20 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T13:51:20.695+0000 7f27a80bb200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-15T13:51:20.909 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 13:51:20 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T13:51:20.907+0000 7f27a80bb200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-15T13:51:21.107 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 13:51:20 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T13:51:20.812+0000 7fb793793200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-15T13:51:21.108 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 13:51:20 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T13:51:20.943+0000 7fb793793200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-15T13:51:21.108 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 13:51:21 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T13:51:21.036+0000 7fb793793200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-15T13:51:21.321 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 13:51:21 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T13:51:21.106+0000 7f27a80bb200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-15T13:51:21.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:21 smithi033 ceph-mon[31650]: Reconfiguring daemon mgr.x on smithi033 2024-05-15T13:51:21.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:21 smithi033 ceph-mon[31650]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:51:21.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:21 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:51:21.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:21 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:51:21.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:21 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:21.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:21 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:21.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:21 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:21.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:21 smithi052 ceph-mon[36190]: Reconfiguring daemon mgr.x on smithi033 2024-05-15T13:51:21.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:21 smithi052 ceph-mon[36190]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:51:21.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:21 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:51:21.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:21 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:51:21.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:21 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:21.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:21 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:21.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:21 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:21.607 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 13:51:21 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T13:51:21.357+0000 7fb793793200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-15T13:51:21.607 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 13:51:21 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T13:51:21.447+0000 7fb793793200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-15T13:51:21.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:21 smithi081 ceph-mon[36434]: Reconfiguring daemon mgr.x on smithi033 2024-05-15T13:51:21.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:21 smithi081 ceph-mon[36434]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:51:21.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:21 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:51:21.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:21 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:51:21.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:21 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:21.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:21 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:21.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:21 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:22.091 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 13:51:21 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T13:51:21.741+0000 7f27a80bb200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-15T13:51:22.091 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 13:51:21 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T13:51:21.969+0000 7f27a80bb200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-15T13:51:22.323 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 13:51:22 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T13:51:22.039+0000 7fb793793200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-15T13:51:22.323 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 13:51:22 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T13:51:22.251+0000 7fb793793200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-15T13:51:22.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:22.339+0000 7f0667fff700 1 -- 172.21.15.33:0/4006254 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 14) ==== 99937+0+0 (secure 0 0 0) 0x7f067400fcb0 con 0x7f0680109290 2024-05-15T13:51:22.402 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:22 smithi033 ceph-mon[31650]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:22.402 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:22 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:51:22.402 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:22 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:22.402 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:22 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:51:22.402 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:22 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:22.402 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:22 smithi033 ceph-mon[31650]: Standby manager daemon z started 2024-05-15T13:51:22.402 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:22 smithi033 ceph-mon[31650]: from='mgr.? 172.21.15.81:0/4131104495' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-15T13:51:22.402 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:22 smithi033 ceph-mon[31650]: from='mgr.? 172.21.15.81:0/4131104495' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-15T13:51:22.403 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:22 smithi033 ceph-mon[31650]: from='mgr.? 172.21.15.81:0/4131104495' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-15T13:51:22.403 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:22 smithi033 ceph-mon[31650]: from='mgr.? 172.21.15.81:0/4131104495' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-15T13:51:22.591 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 13:51:22 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T13:51:22.410+0000 7f27a80bb200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-15T13:51:22.591 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 13:51:22 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T13:51:22.509+0000 7f27a80bb200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-15T13:51:22.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:22 smithi052 ceph-mon[36190]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:22.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:22 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:51:22.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:22 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:22.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:22 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:51:22.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:22 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:22.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:22 smithi052 ceph-mon[36190]: Standby manager daemon z started 2024-05-15T13:51:22.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:22 smithi052 ceph-mon[36190]: from='mgr.? 172.21.15.81:0/4131104495' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-15T13:51:22.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:22 smithi052 ceph-mon[36190]: from='mgr.? 172.21.15.81:0/4131104495' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-15T13:51:22.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:22 smithi052 ceph-mon[36190]: from='mgr.? 172.21.15.81:0/4131104495' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-15T13:51:22.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:22 smithi052 ceph-mon[36190]: from='mgr.? 172.21.15.81:0/4131104495' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-15T13:51:22.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:22 smithi081 ceph-mon[36434]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:22.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:22 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:51:22.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:22 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:22.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:22 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:51:22.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:22 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:22.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:22 smithi081 ceph-mon[36434]: Standby manager daemon z started 2024-05-15T13:51:22.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:22 smithi081 ceph-mon[36434]: from='mgr.? 172.21.15.81:0/4131104495' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-15T13:51:22.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:22 smithi081 ceph-mon[36434]: from='mgr.? 172.21.15.81:0/4131104495' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-15T13:51:22.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:22 smithi081 ceph-mon[36434]: from='mgr.? 172.21.15.81:0/4131104495' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-15T13:51:22.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:22 smithi081 ceph-mon[36434]: from='mgr.? 172.21.15.81:0/4131104495' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-15T13:51:23.267 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 13:51:22 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T13:51:22.894+0000 7f27a80bb200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-15T13:51:23.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:23 smithi052 ceph-mon[36190]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-15T13:51:23.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:23 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-15T13:51:23.591 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 13:51:23 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T13:51:23.265+0000 7f27a80bb200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-15T13:51:23.591 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 13:51:23 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T13:51:23.362+0000 7f27a80bb200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-15T13:51:23.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:23 smithi081 ceph-mon[36434]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-15T13:51:23.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:23 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-15T13:51:23.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:23 smithi033 ceph-mon[31650]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-15T13:51:23.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:23 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-15T13:51:24.091 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 13:51:23 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T13:51:23.617+0000 7f27a80bb200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-15T13:51:24.091 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 13:51:23 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T13:51:23.756+0000 7f27a80bb200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-15T13:51:24.091 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 13:51:23 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T13:51:23.855+0000 7f27a80bb200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-15T13:51:24.591 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 13:51:24 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T13:51:24.195+0000 7f27a80bb200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-15T13:51:24.591 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 13:51:24 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T13:51:24.290+0000 7f27a80bb200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-15T13:51:24.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:24 smithi052 ceph-mon[36190]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:24.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:24 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/455844284' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b434111-810f-450a-b63d-c97080e9f8fc"}]: dispatch 2024-05-15T13:51:24.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:24 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/455844284' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1b434111-810f-450a-b63d-c97080e9f8fc"}]': finished 2024-05-15T13:51:24.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:24 smithi052 ceph-mon[36190]: osdmap e5: 1 total, 0 up, 1 in 2024-05-15T13:51:24.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:24 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T13:51:24.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:24 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/1339656890' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:51:24.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:24 smithi033 ceph-mon[31650]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:24.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:24 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/455844284' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b434111-810f-450a-b63d-c97080e9f8fc"}]: dispatch 2024-05-15T13:51:24.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:24 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/455844284' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1b434111-810f-450a-b63d-c97080e9f8fc"}]': finished 2024-05-15T13:51:24.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:24 smithi033 ceph-mon[31650]: osdmap e5: 1 total, 0 up, 1 in 2024-05-15T13:51:24.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:24 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T13:51:24.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:24 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/1339656890' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:51:24.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:24 smithi081 ceph-mon[36434]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:24.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:24 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/455844284' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b434111-810f-450a-b63d-c97080e9f8fc"}]: dispatch 2024-05-15T13:51:24.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:24 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/455844284' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1b434111-810f-450a-b63d-c97080e9f8fc"}]': finished 2024-05-15T13:51:24.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:24 smithi081 ceph-mon[36434]: osdmap e5: 1 total, 0 up, 1 in 2024-05-15T13:51:24.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:24 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T13:51:24.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:24 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/1339656890' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:51:25.341 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 13:51:24 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T13:51:24.922+0000 7f27a80bb200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-15T13:51:25.341 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 13:51:25 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T13:51:25.145+0000 7f27a80bb200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-15T13:51:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:25.390+0000 7f0667fff700 1 -- 172.21.15.33:0/4006254 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f067400e3e0 con 0x7f0680109290 2024-05-15T13:51:25.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:25 smithi033 ceph-mon[31650]: Standby manager daemon y started 2024-05-15T13:51:25.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:25 smithi033 ceph-mon[31650]: from='mgr.? 172.21.15.52:0/2585483434' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-15T13:51:25.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:25 smithi033 ceph-mon[31650]: from='mgr.? 172.21.15.52:0/2585483434' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-15T13:51:25.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:25 smithi033 ceph-mon[31650]: from='mgr.? 172.21.15.52:0/2585483434' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-15T13:51:25.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:25 smithi033 ceph-mon[31650]: from='mgr.? 172.21.15.52:0/2585483434' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-15T13:51:25.829 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:25 smithi052 ceph-mon[36190]: Standby manager daemon y started 2024-05-15T13:51:25.829 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:25 smithi052 ceph-mon[36190]: from='mgr.? 172.21.15.52:0/2585483434' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-15T13:51:25.829 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:25 smithi052 ceph-mon[36190]: from='mgr.? 172.21.15.52:0/2585483434' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-15T13:51:25.829 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:25 smithi052 ceph-mon[36190]: from='mgr.? 172.21.15.52:0/2585483434' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-15T13:51:25.829 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:25 smithi052 ceph-mon[36190]: from='mgr.? 172.21.15.52:0/2585483434' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-15T13:51:25.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:25 smithi081 ceph-mon[36434]: Standby manager daemon y started 2024-05-15T13:51:25.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:25 smithi081 ceph-mon[36434]: from='mgr.? 172.21.15.52:0/2585483434' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-15T13:51:25.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:25 smithi081 ceph-mon[36434]: from='mgr.? 172.21.15.52:0/2585483434' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-15T13:51:25.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:25 smithi081 ceph-mon[36434]: from='mgr.? 172.21.15.52:0/2585483434' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-15T13:51:25.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:25 smithi081 ceph-mon[36434]: from='mgr.? 172.21.15.52:0/2585483434' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-15T13:51:26.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:26 smithi033 ceph-mon[31650]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:26.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:26 smithi033 ceph-mon[31650]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-15T13:51:26.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:26 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-15T13:51:26.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:26 smithi052 ceph-mon[36190]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:26.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:26 smithi052 ceph-mon[36190]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-15T13:51:26.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:26 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-15T13:51:26.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:26 smithi081 ceph-mon[36434]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:26.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:26 smithi081 ceph-mon[36434]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-15T13:51:26.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:26 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-15T13:51:28.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:28 smithi033 ceph-mon[31650]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:28.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:28 smithi052 ceph-mon[36190]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:28.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:28 smithi081 ceph-mon[36434]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:30.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:30 smithi033 ceph-mon[31650]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:30.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:30 smithi052 ceph-mon[36190]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:30.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:30 smithi081 ceph-mon[36434]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:31.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:31 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-15T13:51:31.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:31 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:31.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:31 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-15T13:51:31.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:31 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:31.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:31 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-15T13:51:31.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:31 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:32.645 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:32 smithi033 ceph-mon[31650]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:32.645 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:32 smithi033 ceph-mon[31650]: Deploying daemon osd.0 on smithi033 2024-05-15T13:51:32.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:32 smithi052 ceph-mon[36190]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:32.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:32 smithi052 ceph-mon[36190]: Deploying daemon osd.0 on smithi033 2024-05-15T13:51:32.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:32 smithi081 ceph-mon[36434]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:32.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:32 smithi081 ceph-mon[36434]: Deploying daemon osd.0 on smithi033 2024-05-15T13:51:34.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:34 smithi033 ceph-mon[31650]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:34.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:34 smithi052 ceph-mon[36190]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:34.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:34 smithi081 ceph-mon[36434]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:36.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:36 smithi033 ceph-mon[31650]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:36.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:36 smithi052 ceph-mon[36190]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:36.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:36 smithi081 ceph-mon[36434]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:37.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:37 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:37.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:37 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:37.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:37 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:51:37.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:37 smithi052 ceph-mon[36190]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:37.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:37 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:37.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:37 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:37.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:37 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:51:37.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:37 smithi081 ceph-mon[36434]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:37.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:37 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:37.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:37 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:37.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:37 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:51:37.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:37 smithi033 ceph-mon[31650]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:40.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:40.253+0000 7f0667fff700 1 -- 172.21.15.33:0/4006254 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0680063980 con 0x7f066803e750 2024-05-15T13:51:40.256 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 0 on host 'smithi033' 2024-05-15T13:51:40.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:40.255+0000 7f068508f700 1 -- 172.21.15.33:0/4006254 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f066803e750 msgr2=0x7f0668040c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:51:40.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:40.255+0000 7f068508f700 1 --2- 172.21.15.33:0/4006254 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f066803e750 0x7f0668040c10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0670011490 tx=0x7f0670009260 comp rx=0 tx=0).stop 2024-05-15T13:51:40.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:40.255+0000 7f068508f700 1 -- 172.21.15.33:0/4006254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0680109290 msgr2=0x7f06801a0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:51:40.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:40.255+0000 7f068508f700 1 --2- 172.21.15.33:0/4006254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0680109290 0x7f06801a0c60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0674009510 tx=0x7f0674009540 comp rx=0 tx=0).stop 2024-05-15T13:51:40.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:40.255+0000 7f068508f700 1 -- 172.21.15.33:0/4006254 shutdown_connections 2024-05-15T13:51:40.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:40.255+0000 7f068508f700 1 --2- 172.21.15.33:0/4006254 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f066803e750 0x7f0668040c10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:40.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:40.255+0000 7f068508f700 1 --2- 172.21.15.33:0/4006254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0680109290 0x7f06801a0c60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:40.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:40.255+0000 7f068508f700 1 --2- 172.21.15.33:0/4006254 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0680104030 0x7f06801996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:40.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:40.255+0000 7f068508f700 1 --2- 172.21.15.33:0/4006254 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0680106960 0x7f0680199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:40.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:40.256+0000 7f068508f700 1 -- 172.21.15.33:0/4006254 >> 172.21.15.33:0/4006254 conn(0x7f06800fda60 msgr2=0x7f0680104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:51:40.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:40.256+0000 7f068508f700 1 -- 172.21.15.33:0/4006254 shutdown_connections 2024-05-15T13:51:40.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:51:40.256+0000 7f068508f700 1 -- 172.21.15.33:0/4006254 wait complete. 2024-05-15T13:51:40.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:40 smithi033 ceph-mon[31650]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:40.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:40 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:40.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:40 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:40.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:40 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:40.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:40 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:51:40.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:40 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:40.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:40 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:40.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:40 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:40.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:40 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:51:40.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:40 smithi052 ceph-mon[36190]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:40.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:40 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:40.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:40 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:40.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:40 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:40.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:40 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:51:40.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:40 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:40.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:40 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:40.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:40 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:40.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:40 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:51:40.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:40 smithi081 ceph-mon[36434]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:40.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:40 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:40.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:40 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:40.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:40 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:40.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:40 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:51:40.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:40 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:40.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:40 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:40.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:40 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:40.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:40 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:51:40.882 DEBUG:teuthology.orchestra.run.smithi033:osd.0> sudo journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.0.service 2024-05-15T13:51:40.884 INFO:tasks.cephadm:Deploying osd.1 on smithi052 with /dev/vg_nvme/lv_4... 2024-05-15T13:51:40.885 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-15T13:51:41.140 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.b/config 2024-05-15T13:51:41.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:41 smithi033 ceph-mon[31650]: from='osd.0 [v2:172.21.15.33:6802/887409130,v1:172.21.15.33:6803/887409130]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-15T13:51:41.567 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:41 smithi052 ceph-mon[36190]: from='osd.0 [v2:172.21.15.33:6802/887409130,v1:172.21.15.33:6803/887409130]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-15T13:51:41.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:41 smithi081 ceph-mon[36434]: from='osd.0 [v2:172.21.15.33:6802/887409130,v1:172.21.15.33:6803/887409130]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-15T13:51:42.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:42 smithi033 ceph-mon[31650]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:42.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:42 smithi033 ceph-mon[31650]: from='osd.0 [v2:172.21.15.33:6802/887409130,v1:172.21.15.33:6803/887409130]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-15T13:51:42.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:42 smithi033 ceph-mon[31650]: osdmap e6: 1 total, 0 up, 1 in 2024-05-15T13:51:42.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:42 smithi033 ceph-mon[31650]: from='osd.0 [v2:172.21.15.33:6802/887409130,v1:172.21.15.33:6803/887409130]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-15T13:51:42.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:42 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T13:51:42.476 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:42 smithi052 ceph-mon[36190]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:42.476 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:42 smithi052 ceph-mon[36190]: from='osd.0 [v2:172.21.15.33:6802/887409130,v1:172.21.15.33:6803/887409130]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-15T13:51:42.476 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:42 smithi052 ceph-mon[36190]: osdmap e6: 1 total, 0 up, 1 in 2024-05-15T13:51:42.476 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:42 smithi052 ceph-mon[36190]: from='osd.0 [v2:172.21.15.33:6802/887409130,v1:172.21.15.33:6803/887409130]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-15T13:51:42.476 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:42 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T13:51:42.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:42 smithi081 ceph-mon[36434]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:42.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:42 smithi081 ceph-mon[36434]: from='osd.0 [v2:172.21.15.33:6802/887409130,v1:172.21.15.33:6803/887409130]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-15T13:51:42.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:42 smithi081 ceph-mon[36434]: osdmap e6: 1 total, 0 up, 1 in 2024-05-15T13:51:42.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:42 smithi081 ceph-mon[36434]: from='osd.0 [v2:172.21.15.33:6802/887409130,v1:172.21.15.33:6803/887409130]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-15T13:51:42.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:42 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T13:51:43.051 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-15T13:51:43.051 INFO:teuthology.orchestra.run.smithi052.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-15T13:51:43.051 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10+0 records in 2024-05-15T13:51:43.051 INFO:teuthology.orchestra.run.smithi052.stderr:10+0 records out 2024-05-15T13:51:43.051 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00825497 s, 1.3 GB/s 2024-05-15T13:51:43.051 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping successful for: 2024-05-15T13:51:43.374 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:43 smithi052 ceph-mon[36190]: from='osd.0 [v2:172.21.15.33:6802/887409130,v1:172.21.15.33:6803/887409130]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-15T13:51:43.374 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:43 smithi052 ceph-mon[36190]: osdmap e7: 1 total, 0 up, 1 in 2024-05-15T13:51:43.374 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:43 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T13:51:43.375 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:43 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T13:51:43.375 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:43 smithi052 ceph-mon[36190]: from='osd.0 [v2:172.21.15.33:6802/887409130,v1:172.21.15.33:6803/887409130]' entity='osd.0' 2024-05-15T13:51:43.375 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:43 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T13:51:43.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:43 smithi033 ceph-mon[31650]: from='osd.0 [v2:172.21.15.33:6802/887409130,v1:172.21.15.33:6803/887409130]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-15T13:51:43.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:43 smithi033 ceph-mon[31650]: osdmap e7: 1 total, 0 up, 1 in 2024-05-15T13:51:43.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:43 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T13:51:43.546 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:43 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T13:51:43.546 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:43 smithi033 ceph-mon[31650]: from='osd.0 [v2:172.21.15.33:6802/887409130,v1:172.21.15.33:6803/887409130]' entity='osd.0' 2024-05-15T13:51:43.546 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:43 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T13:51:43.546 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 15 13:51:43 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-0[43694]: 2024-05-15T13:51:43.266+0000 7f4f572d5700 -1 osd.0 0 waiting for initial osdmap 2024-05-15T13:51:43.546 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 15 13:51:43 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-0[43694]: 2024-05-15T13:51:43.341+0000 7f4f4f0c0700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-15T13:51:43.665 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi052:vg_nvme/lv_4 2024-05-15T13:51:43.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:43 smithi081 ceph-mon[36434]: from='osd.0 [v2:172.21.15.33:6802/887409130,v1:172.21.15.33:6803/887409130]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-15T13:51:43.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:43 smithi081 ceph-mon[36434]: osdmap e7: 1 total, 0 up, 1 in 2024-05-15T13:51:43.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:43 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T13:51:43.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:43 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T13:51:43.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:43 smithi081 ceph-mon[36434]: from='osd.0 [v2:172.21.15.33:6802/887409130,v1:172.21.15.33:6803/887409130]' entity='osd.0' 2024-05-15T13:51:43.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:43 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T13:51:43.911 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.b/config 2024-05-15T13:51:44.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:44 smithi033 ceph-mon[31650]: purged_snaps scrub starts 2024-05-15T13:51:44.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:44 smithi033 ceph-mon[31650]: purged_snaps scrub ok 2024-05-15T13:51:44.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:44 smithi033 ceph-mon[31650]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:44.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:44 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:44.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:44 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:44.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:44 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:51:44.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:44 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:44.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:44 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:44.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:44 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:51:44.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:44 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:44.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:44 smithi033 ceph-mon[31650]: osd.0 [v2:172.21.15.33:6802/887409130,v1:172.21.15.33:6803/887409130] boot 2024-05-15T13:51:44.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:44 smithi033 ceph-mon[31650]: osdmap e8: 1 total, 1 up, 1 in 2024-05-15T13:51:44.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:44 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T13:51:44.707 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:44 smithi052 ceph-mon[36190]: purged_snaps scrub starts 2024-05-15T13:51:44.708 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:44 smithi052 ceph-mon[36190]: purged_snaps scrub ok 2024-05-15T13:51:44.708 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:44 smithi052 ceph-mon[36190]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:44.708 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:44 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:44.708 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:44 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:44.708 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:44 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:51:44.708 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:44 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:44.708 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:44 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:44.708 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:44 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:51:44.708 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:44 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:44.708 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:44 smithi052 ceph-mon[36190]: osd.0 [v2:172.21.15.33:6802/887409130,v1:172.21.15.33:6803/887409130] boot 2024-05-15T13:51:44.708 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:44 smithi052 ceph-mon[36190]: osdmap e8: 1 total, 1 up, 1 in 2024-05-15T13:51:44.708 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:44 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T13:51:44.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:44 smithi081 ceph-mon[36434]: purged_snaps scrub starts 2024-05-15T13:51:44.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:44 smithi081 ceph-mon[36434]: purged_snaps scrub ok 2024-05-15T13:51:44.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:44 smithi081 ceph-mon[36434]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-15T13:51:44.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:44 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:44.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:44 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:44.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:44 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:51:44.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:44 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:44.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:44 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:44.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:44 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:51:44.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:44 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:51:44.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:44 smithi081 ceph-mon[36434]: osd.0 [v2:172.21.15.33:6802/887409130,v1:172.21.15.33:6803/887409130] boot 2024-05-15T13:51:44.859 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:44 smithi081 ceph-mon[36434]: osdmap e8: 1 total, 1 up, 1 in 2024-05-15T13:51:44.859 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:44 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T13:51:45.519 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:45 smithi052 ceph-mon[36190]: Detected new or changed devices on smithi033 2024-05-15T13:51:45.519 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:45 smithi052 ceph-mon[36190]: Adjusting osd_memory_target on smithi033 to 17093M 2024-05-15T13:51:45.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.542+0000 7fb281aa3700 1 -- 172.21.15.52:0/144223758 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb27c1040f0 msgr2=0x7fb27c1064e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:51:45.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.542+0000 7fb281aa3700 1 --2- 172.21.15.52:0/144223758 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb27c1040f0 0x7fb27c1064e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb26c009990 tx=0x7fb26c009ca0 comp rx=0 tx=0).stop 2024-05-15T13:51:45.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.542+0000 7fb281aa3700 1 -- 172.21.15.52:0/144223758 shutdown_connections 2024-05-15T13:51:45.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.542+0000 7fb281aa3700 1 --2- 172.21.15.52:0/144223758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb27c109350 0x7fb27c10b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:45.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.542+0000 7fb281aa3700 1 --2- 172.21.15.52:0/144223758 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb27c1040f0 0x7fb27c1064e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:45.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.542+0000 7fb281aa3700 1 --2- 172.21.15.52:0/144223758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb27c106a20 0x7fb27c108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:45.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.542+0000 7fb281aa3700 1 -- 172.21.15.52:0/144223758 >> 172.21.15.52:0/144223758 conn(0x7fb27c0fda60 msgr2=0x7fb27c0ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:51:45.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.542+0000 7fb281aa3700 1 -- 172.21.15.52:0/144223758 shutdown_connections 2024-05-15T13:51:45.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.542+0000 7fb281aa3700 1 -- 172.21.15.52:0/144223758 wait complete. 2024-05-15T13:51:45.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.543+0000 7fb281aa3700 1 Processor -- start 2024-05-15T13:51:45.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.543+0000 7fb281aa3700 1 -- start start 2024-05-15T13:51:45.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.543+0000 7fb281aa3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb27c1040f0 0x7fb27c103a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:51:45.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.543+0000 7fb281aa3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb27c106a20 0x7fb27c1020a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:51:45.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.543+0000 7fb281aa3700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb27c109350 0x7fb27c1025e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:51:45.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.543+0000 7fb281aa3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb27c10f0a0 con 0x7fb27c1040f0 2024-05-15T13:51:45.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.543+0000 7fb281aa3700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb27c10ef20 con 0x7fb27c106a20 2024-05-15T13:51:45.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.543+0000 7fb281aa3700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb27c10f220 con 0x7fb27c109350 2024-05-15T13:51:45.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.544+0000 7fb27affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb27c106a20 0x7fb27c1020a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:51:45.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.544+0000 7fb27affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb27c106a20 0x7fb27c1020a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44258/0 (socket says 172.21.15.52:44258) 2024-05-15T13:51:45.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.544+0000 7fb27affd700 1 -- 172.21.15.52:0/807804289 learned_addr learned my addr 172.21.15.52:0/807804289 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-15T13:51:45.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.544+0000 7fb27b7fe700 1 --2- 172.21.15.52:0/807804289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb27c1040f0 0x7fb27c103a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:51:45.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.544+0000 7fb27affd700 1 -- 172.21.15.52:0/807804289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb27c109350 msgr2=0x7fb27c1025e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:51:45.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.544+0000 7fb27bfff700 1 --2- 172.21.15.52:0/807804289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb27c109350 0x7fb27c1025e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:51:45.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.544+0000 7fb27affd700 1 --2- 172.21.15.52:0/807804289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb27c109350 0x7fb27c1025e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:45.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.544+0000 7fb27affd700 1 -- 172.21.15.52:0/807804289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb27c1040f0 msgr2=0x7fb27c103a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:51:45.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.544+0000 7fb27affd700 1 --2- 172.21.15.52:0/807804289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb27c1040f0 0x7fb27c103a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:51:45.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.544+0000 7fb27affd700 1 -- 172.21.15.52:0/807804289 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27c102e90 con 0x7fb27c106a20 2024-05-15T13:51:45.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.544+0000 7fb27b7fe700 1 --2- 172.21.15.52:0/807804289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb27c1040f0 0x7fb27c103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T13:51:45.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.544+0000 7fb27affd700 1 --2- 172.21.15.52:0/807804289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb27c106a20 0x7fb27c1020a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb27000b530 tx=0x7fb27000b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:51:45.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.544+0000 7fb278ff9700 1 -- 172.21.15.52:0/807804289 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb270015020 con 0x7fb27c106a20 2024-05-15T13:51:45.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.544+0000 7fb278ff9700 1 -- 172.21.15.52:0/807804289 <== mon.2 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb270011ce0 con 0x7fb27c106a20 2024-05-15T13:51:45.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.544+0000 7fb281aa3700 1 -- 172.21.15.52:0/807804289 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb27c1a38a0 con 0x7fb27c106a20 2024-05-15T13:51:45.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.544+0000 7fb278ff9700 1 -- 172.21.15.52:0/807804289 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb270010d20 con 0x7fb27c106a20 2024-05-15T13:51:45.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.544+0000 7fb281aa3700 1 -- 172.21.15.52:0/807804289 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb27c1a3e60 con 0x7fb27c106a20 2024-05-15T13:51:45.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.545+0000 7fb281aa3700 1 -- 172.21.15.52:0/807804289 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb27c06a230 con 0x7fb27c106a20 2024-05-15T13:51:45.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.546+0000 7fb278ff9700 1 -- 172.21.15.52:0/807804289 <== mon.2 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7fb270018020 con 0x7fb27c106a20 2024-05-15T13:51:45.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.547+0000 7fb278ff9700 1 --2- 172.21.15.52:0/807804289 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fb2640b2870 0x7fb2640b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:51:45.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.547+0000 7fb278ff9700 1 -- 172.21.15.52:0/807804289 <== mon.2 v2:172.21.15.52:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fb2700dff30 con 0x7fb27c106a20 2024-05-15T13:51:45.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.547+0000 7fb27b7fe700 1 --2- 172.21.15.52:0/807804289 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fb2640b2870 0x7fb2640b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:51:45.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.548+0000 7fb27b7fe700 1 --2- 172.21.15.52:0/807804289 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fb2640b2870 0x7fb2640b4d30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb26c00b5c0 tx=0x7fb26c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:51:45.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.552+0000 7fb278ff9700 1 -- 172.21.15.52:0/807804289 <== mon.2 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb27001f020 con 0x7fb27c106a20 2024-05-15T13:51:45.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:45 smithi033 ceph-mon[31650]: Detected new or changed devices on smithi033 2024-05-15T13:51:45.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:45 smithi033 ceph-mon[31650]: Adjusting osd_memory_target on smithi033 to 17093M 2024-05-15T13:51:45.758 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:45 smithi081 ceph-mon[36434]: Detected new or changed devices on smithi033 2024-05-15T13:51:45.759 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:45 smithi081 ceph-mon[36434]: Adjusting osd_memory_target on smithi033 to 17093M 2024-05-15T13:51:45.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:51:45.773+0000 7fb281aa3700 1 -- 172.21.15.52:0/807804289 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fb27c063980 con 0x7fb2640b2870 2024-05-15T13:51:46.671 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:46 smithi052 ceph-mon[36190]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:51:46.671 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:46 smithi052 ceph-mon[36190]: osdmap e9: 1 total, 1 up, 1 in 2024-05-15T13:51:46.672 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:46 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:51:46.672 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:46 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:51:46.672 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:46 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:46.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:46 smithi033 ceph-mon[31650]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:51:46.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:46 smithi033 ceph-mon[31650]: osdmap e9: 1 total, 1 up, 1 in 2024-05-15T13:51:46.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:46 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:51:46.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:46 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:51:46.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:46 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:46.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:46 smithi081 ceph-mon[36434]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:51:46.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:46 smithi081 ceph-mon[36434]: osdmap e9: 1 total, 1 up, 1 in 2024-05-15T13:51:46.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:46 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:51:46.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:46 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:51:46.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:46 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:47.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:47 smithi033 ceph-mon[31650]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:51:47.720 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:47 smithi052 ceph-mon[36190]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:51:47.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:47 smithi081 ceph-mon[36434]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:51:48.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:48 smithi033 ceph-mon[31650]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:51:48.698 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:48 smithi052 ceph-mon[36190]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:51:48.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:48 smithi081 ceph-mon[36434]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:51:49.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:49 smithi033 ceph-mon[31650]: from='client.? 172.21.15.52:0/1673119819' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b578d4e-a841-4b27-bc95-d6c3bae580fd"}]: dispatch 2024-05-15T13:51:49.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:49 smithi033 ceph-mon[31650]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b578d4e-a841-4b27-bc95-d6c3bae580fd"}]: dispatch 2024-05-15T13:51:49.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:49 smithi033 ceph-mon[31650]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b578d4e-a841-4b27-bc95-d6c3bae580fd"}]': finished 2024-05-15T13:51:49.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:49 smithi033 ceph-mon[31650]: osdmap e10: 2 total, 1 up, 2 in 2024-05-15T13:51:49.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:49 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T13:51:49.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:49 smithi033 ceph-mon[31650]: from='client.? 172.21.15.52:0/288087886' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:51:49.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:49 smithi052 ceph-mon[36190]: from='client.? 172.21.15.52:0/1673119819' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b578d4e-a841-4b27-bc95-d6c3bae580fd"}]: dispatch 2024-05-15T13:51:49.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:49 smithi052 ceph-mon[36190]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b578d4e-a841-4b27-bc95-d6c3bae580fd"}]: dispatch 2024-05-15T13:51:49.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:49 smithi052 ceph-mon[36190]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b578d4e-a841-4b27-bc95-d6c3bae580fd"}]': finished 2024-05-15T13:51:49.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:49 smithi052 ceph-mon[36190]: osdmap e10: 2 total, 1 up, 2 in 2024-05-15T13:51:49.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:49 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T13:51:49.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:49 smithi052 ceph-mon[36190]: from='client.? 172.21.15.52:0/288087886' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:51:49.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:49 smithi081 ceph-mon[36434]: from='client.? 172.21.15.52:0/1673119819' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b578d4e-a841-4b27-bc95-d6c3bae580fd"}]: dispatch 2024-05-15T13:51:49.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:49 smithi081 ceph-mon[36434]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b578d4e-a841-4b27-bc95-d6c3bae580fd"}]: dispatch 2024-05-15T13:51:49.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:49 smithi081 ceph-mon[36434]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b578d4e-a841-4b27-bc95-d6c3bae580fd"}]': finished 2024-05-15T13:51:49.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:49 smithi081 ceph-mon[36434]: osdmap e10: 2 total, 1 up, 2 in 2024-05-15T13:51:49.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:49 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T13:51:49.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:49 smithi081 ceph-mon[36434]: from='client.? 172.21.15.52:0/288087886' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:51:50.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:50 smithi033 ceph-mon[31650]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:51:50.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:50 smithi052 ceph-mon[36190]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:51:50.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:50 smithi081 ceph-mon[36434]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:51:52.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:52 smithi052 ceph-mon[36190]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:51:52.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:52 smithi033 ceph-mon[31650]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:51:52.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:52 smithi081 ceph-mon[36434]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:51:54.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:54 smithi052 ceph-mon[36190]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:51:54.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:54 smithi033 ceph-mon[31650]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:51:54.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:54 smithi081 ceph-mon[36434]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:51:55.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:55 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-15T13:51:55.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:55 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:55.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:55 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-15T13:51:55.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:55 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:55.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:55 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-15T13:51:55.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:55 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:51:56.549 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:56 smithi052 ceph-mon[36190]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:51:56.549 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:56 smithi052 ceph-mon[36190]: Deploying daemon osd.1 on smithi052 2024-05-15T13:51:56.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:56 smithi033 ceph-mon[31650]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:51:56.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:56 smithi033 ceph-mon[31650]: Deploying daemon osd.1 on smithi052 2024-05-15T13:51:56.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:56 smithi081 ceph-mon[36434]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:51:56.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:56 smithi081 ceph-mon[36434]: Deploying daemon osd.1 on smithi052 2024-05-15T13:51:58.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:51:58 smithi033 ceph-mon[31650]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:51:58.792 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:51:58 smithi052 ceph-mon[36190]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:51:58.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:51:58 smithi081 ceph-mon[36434]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:52:00.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:00 smithi052 ceph-mon[36190]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:52:00.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:00 smithi033 ceph-mon[31650]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:52:00.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:00 smithi081 ceph-mon[36434]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:52:02.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:01 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:02.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:01 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:02.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:01 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:52:02.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:01 smithi052 ceph-mon[36190]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:52:02.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:01 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:02.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:01 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:02.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:01 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:52:02.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:01 smithi081 ceph-mon[36434]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:52:02.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:01 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:02.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:01 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:02.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:01 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:52:02.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:01 smithi033 ceph-mon[31650]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:52:04.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:03 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:04.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:03 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:04.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:03 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:52:04.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:03 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:52:04.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:03 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:04.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:03 smithi052 ceph-mon[36190]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:52:04.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:03 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:04.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:03 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:04.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:03 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:52:04.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:03 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:52:04.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:03 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:04.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:03 smithi081 ceph-mon[36434]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:52:04.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:03 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:04.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:03 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:04.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:03 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:52:04.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:03 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:52:04.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:03 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:04.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:03 smithi033 ceph-mon[31650]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:52:04.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:04.643+0000 7fb278ff9700 1 -- 172.21.15.52:0/807804289 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb27c063980 con 0x7fb2640b2870 2024-05-15T13:52:04.646 INFO:teuthology.orchestra.run.smithi052.stdout:Created osd(s) 1 on host 'smithi052' 2024-05-15T13:52:04.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:04.646+0000 7fb281aa3700 1 -- 172.21.15.52:0/807804289 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fb2640b2870 msgr2=0x7fb2640b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:52:04.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:04.646+0000 7fb281aa3700 1 --2- 172.21.15.52:0/807804289 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fb2640b2870 0x7fb2640b4d30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb26c00b5c0 tx=0x7fb26c005fd0 comp rx=0 tx=0).stop 2024-05-15T13:52:04.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:04.646+0000 7fb281aa3700 1 -- 172.21.15.52:0/807804289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb27c106a20 msgr2=0x7fb27c1020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:52:04.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:04.646+0000 7fb281aa3700 1 --2- 172.21.15.52:0/807804289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb27c106a20 0x7fb27c1020a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb27000b530 tx=0x7fb27000b8f0 comp rx=0 tx=0).stop 2024-05-15T13:52:04.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:04.646+0000 7fb281aa3700 1 -- 172.21.15.52:0/807804289 shutdown_connections 2024-05-15T13:52:04.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:04.646+0000 7fb281aa3700 1 --2- 172.21.15.52:0/807804289 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fb2640b2870 0x7fb2640b4d30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:52:04.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:04.646+0000 7fb281aa3700 1 --2- 172.21.15.52:0/807804289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb27c109350 0x7fb27c1025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:52:04.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:04.646+0000 7fb281aa3700 1 --2- 172.21.15.52:0/807804289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb27c1040f0 0x7fb27c103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:52:04.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:04.646+0000 7fb281aa3700 1 --2- 172.21.15.52:0/807804289 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb27c106a20 0x7fb27c1020a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:52:04.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:04.646+0000 7fb281aa3700 1 -- 172.21.15.52:0/807804289 >> 172.21.15.52:0/807804289 conn(0x7fb27c0fda60 msgr2=0x7fb27c104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:52:04.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:04.646+0000 7fb281aa3700 1 -- 172.21.15.52:0/807804289 shutdown_connections 2024-05-15T13:52:04.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:04.646+0000 7fb281aa3700 1 -- 172.21.15.52:0/807804289 wait complete. 2024-05-15T13:52:05.212 DEBUG:teuthology.orchestra.run.smithi052:osd.1> sudo journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.1.service 2024-05-15T13:52:05.214 INFO:tasks.cephadm:Deploying osd.2 on smithi081 with /dev/vg_nvme/lv_4... 2024-05-15T13:52:05.214 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-15T13:52:05.456 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.c/config 2024-05-15T13:52:05.829 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:05 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:05.829 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:05 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:05.829 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:05 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:52:05.829 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:05 smithi081 ceph-mon[36434]: from='osd.1 [v2:172.21.15.52:6800/2794180003,v1:172.21.15.52:6801/2794180003]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-15T13:52:05.829 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:05 smithi081 ceph-mon[36434]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-15T13:52:05.830 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:05 smithi081 ceph-mon[36434]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:52:05.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:05 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:05.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:05 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:05.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:05 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:52:05.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:05 smithi052 ceph-mon[36190]: from='osd.1 [v2:172.21.15.52:6800/2794180003,v1:172.21.15.52:6801/2794180003]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-15T13:52:05.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:05 smithi052 ceph-mon[36190]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-15T13:52:05.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:05 smithi052 ceph-mon[36190]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:52:05.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:05 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:05.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:05 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:05.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:05 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:52:05.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:05 smithi033 ceph-mon[31650]: from='osd.1 [v2:172.21.15.52:6800/2794180003,v1:172.21.15.52:6801/2794180003]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-15T13:52:05.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:05 smithi033 ceph-mon[31650]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-15T13:52:05.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:05 smithi033 ceph-mon[31650]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:52:06.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:06 smithi052 ceph-mon[36190]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-15T13:52:06.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:06 smithi052 ceph-mon[36190]: from='osd.1 [v2:172.21.15.52:6800/2794180003,v1:172.21.15.52:6801/2794180003]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-05-15T13:52:06.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:06 smithi052 ceph-mon[36190]: osdmap e11: 2 total, 1 up, 2 in 2024-05-15T13:52:06.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:06 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T13:52:06.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:06 smithi052 ceph-mon[36190]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-05-15T13:52:06.841 INFO:journalctl@ceph.osd.1.smithi052.stdout:May 15 13:52:06 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-1[40752]: 2024-05-15T13:52:06.605+0000 7f15c1df8700 -1 osd.1 0 waiting for initial osdmap 2024-05-15T13:52:06.842 INFO:journalctl@ceph.osd.1.smithi052.stdout:May 15 13:52:06 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-1[40752]: 2024-05-15T13:52:06.609+0000 7f15bdbeb700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-15T13:52:06.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:06 smithi081 ceph-mon[36434]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-15T13:52:06.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:06 smithi081 ceph-mon[36434]: from='osd.1 [v2:172.21.15.52:6800/2794180003,v1:172.21.15.52:6801/2794180003]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-05-15T13:52:06.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:06 smithi081 ceph-mon[36434]: osdmap e11: 2 total, 1 up, 2 in 2024-05-15T13:52:06.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:06 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T13:52:06.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:06 smithi081 ceph-mon[36434]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-05-15T13:52:06.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:06 smithi033 ceph-mon[31650]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-15T13:52:06.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:06 smithi033 ceph-mon[31650]: from='osd.1 [v2:172.21.15.52:6800/2794180003,v1:172.21.15.52:6801/2794180003]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-05-15T13:52:06.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:06 smithi033 ceph-mon[31650]: osdmap e11: 2 total, 1 up, 2 in 2024-05-15T13:52:06.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:06 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T13:52:06.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:06 smithi033 ceph-mon[31650]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-05-15T13:52:07.467 INFO:teuthology.orchestra.run.smithi081.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-15T13:52:07.467 INFO:teuthology.orchestra.run.smithi081.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-15T13:52:07.467 INFO:teuthology.orchestra.run.smithi081.stderr: stderr: 10+0 records in 2024-05-15T13:52:07.467 INFO:teuthology.orchestra.run.smithi081.stderr:10+0 records out 2024-05-15T13:52:07.467 INFO:teuthology.orchestra.run.smithi081.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0133138 s, 788 MB/s 2024-05-15T13:52:07.467 INFO:teuthology.orchestra.run.smithi081.stderr:--> Zapping successful for: 2024-05-15T13:52:07.806 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:07 smithi052 ceph-mon[36190]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-05-15T13:52:07.806 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:07 smithi052 ceph-mon[36190]: osdmap e12: 2 total, 1 up, 2 in 2024-05-15T13:52:07.806 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:07 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T13:52:07.806 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:07 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T13:52:07.806 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:07 smithi052 ceph-mon[36190]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:52:07.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:07 smithi033 ceph-mon[31650]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-05-15T13:52:07.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:07 smithi033 ceph-mon[31650]: osdmap e12: 2 total, 1 up, 2 in 2024-05-15T13:52:07.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:07 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T13:52:07.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:07 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T13:52:07.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:07 smithi033 ceph-mon[31650]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:52:08.061 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:07 smithi081 ceph-mon[36434]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-05-15T13:52:08.061 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:07 smithi081 ceph-mon[36434]: osdmap e12: 2 total, 1 up, 2 in 2024-05-15T13:52:08.061 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:07 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T13:52:08.061 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:07 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T13:52:08.061 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:07 smithi081 ceph-mon[36434]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-15T13:52:08.088 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi081:vg_nvme/lv_4 2024-05-15T13:52:08.326 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.c/config 2024-05-15T13:52:08.731 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:08 smithi081 ceph-mon[36434]: purged_snaps scrub starts 2024-05-15T13:52:08.731 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:08 smithi081 ceph-mon[36434]: purged_snaps scrub ok 2024-05-15T13:52:08.732 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:08 smithi081 ceph-mon[36434]: osd.1 [v2:172.21.15.52:6800/2794180003,v1:172.21.15.52:6801/2794180003] boot 2024-05-15T13:52:08.732 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:08 smithi081 ceph-mon[36434]: osdmap e13: 2 total, 2 up, 2 in 2024-05-15T13:52:08.732 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:08 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T13:52:08.732 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:08 smithi081 ceph-mon[36434]: Detected new or changed devices on smithi052 2024-05-15T13:52:08.732 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:08 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:08.732 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:08 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:08.732 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:08 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:52:08.732 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:08 smithi081 ceph-mon[36434]: Adjusting osd_memory_target on smithi052 to 17093M 2024-05-15T13:52:08.732 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:08 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:08.732 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:08 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:52:08.732 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:08 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:52:08.732 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:08 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:08.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:08 smithi033 ceph-mon[31650]: purged_snaps scrub starts 2024-05-15T13:52:08.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:08 smithi033 ceph-mon[31650]: purged_snaps scrub ok 2024-05-15T13:52:08.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:08 smithi033 ceph-mon[31650]: osd.1 [v2:172.21.15.52:6800/2794180003,v1:172.21.15.52:6801/2794180003] boot 2024-05-15T13:52:08.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:08 smithi033 ceph-mon[31650]: osdmap e13: 2 total, 2 up, 2 in 2024-05-15T13:52:08.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:08 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T13:52:08.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:08 smithi033 ceph-mon[31650]: Detected new or changed devices on smithi052 2024-05-15T13:52:08.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:08 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:08.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:08 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:08.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:08 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:52:08.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:08 smithi033 ceph-mon[31650]: Adjusting osd_memory_target on smithi052 to 17093M 2024-05-15T13:52:08.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:08 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:08.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:08 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:52:08.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:08 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:52:08.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:08 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:09.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:08 smithi052 ceph-mon[36190]: purged_snaps scrub starts 2024-05-15T13:52:09.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:08 smithi052 ceph-mon[36190]: purged_snaps scrub ok 2024-05-15T13:52:09.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:08 smithi052 ceph-mon[36190]: osd.1 [v2:172.21.15.52:6800/2794180003,v1:172.21.15.52:6801/2794180003] boot 2024-05-15T13:52:09.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:08 smithi052 ceph-mon[36190]: osdmap e13: 2 total, 2 up, 2 in 2024-05-15T13:52:09.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:08 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T13:52:09.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:08 smithi052 ceph-mon[36190]: Detected new or changed devices on smithi052 2024-05-15T13:52:09.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:08 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:09.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:08 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:09.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:08 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:52:09.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:08 smithi052 ceph-mon[36190]: Adjusting osd_memory_target on smithi052 to 17093M 2024-05-15T13:52:09.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:08 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:09.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:08 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:52:09.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:08 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:52:09.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:08 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:09.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.689+0000 7f3fd9234700 1 -- 172.21.15.81:0/2396882553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fd41036c0 msgr2=0x7f3fd4103b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:52:09.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.689+0000 7f3fd9234700 1 --2- 172.21.15.81:0/2396882553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fd41036c0 0x7f3fd4103b20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4009990 tx=0x7f3fc4009ca0 comp rx=0 tx=0).stop 2024-05-15T13:52:09.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.689+0000 7f3fd9234700 1 -- 172.21.15.81:0/2396882553 shutdown_connections 2024-05-15T13:52:09.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.689+0000 7f3fd9234700 1 --2- 172.21.15.81:0/2396882553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fd4104060 0x7f3fd410a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:52:09.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.689+0000 7f3fd9234700 1 --2- 172.21.15.81:0/2396882553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3fd41024c0 0x7f3fd41028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:52:09.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.689+0000 7f3fd9234700 1 --2- 172.21.15.81:0/2396882553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fd41036c0 0x7f3fd4103b20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:52:09.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.689+0000 7f3fd9234700 1 -- 172.21.15.81:0/2396882553 >> 172.21.15.81:0/2396882553 conn(0x7f3fd40fda60 msgr2=0x7f3fd40ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:52:09.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.689+0000 7f3fd9234700 1 -- 172.21.15.81:0/2396882553 shutdown_connections 2024-05-15T13:52:09.692 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.689+0000 7f3fd9234700 1 -- 172.21.15.81:0/2396882553 wait complete. 2024-05-15T13:52:09.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.690+0000 7f3fd9234700 1 Processor -- start 2024-05-15T13:52:09.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.690+0000 7f3fd9234700 1 -- start start 2024-05-15T13:52:09.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.690+0000 7f3fd9234700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3fd41024c0 0x7f3fd419b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:52:09.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.690+0000 7f3fd9234700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fd41036c0 0x7f3fd419beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:52:09.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.690+0000 7f3fd9234700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fd4104060 0x7f3fd41a2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:52:09.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.690+0000 7f3fd9234700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd410dfe0 con 0x7f3fd4104060 2024-05-15T13:52:09.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.690+0000 7f3fd9234700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd410de60 con 0x7f3fd41024c0 2024-05-15T13:52:09.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.690+0000 7f3fd9234700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd410e160 con 0x7f3fd41036c0 2024-05-15T13:52:09.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.690+0000 7f3fd259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fd41036c0 0x7f3fd419beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:52:09.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.690+0000 7f3fd259c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fd41036c0 0x7f3fd419beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39116/0 (socket says 172.21.15.81:39116) 2024-05-15T13:52:09.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.690+0000 7f3fd259c700 1 -- 172.21.15.81:0/2554618358 learned_addr learned my addr 172.21.15.81:0/2554618358 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-15T13:52:09.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.690+0000 7f3fd2d9d700 1 --2- 172.21.15.81:0/2554618358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3fd41024c0 0x7f3fd419b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:52:09.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.690+0000 7f3fd359e700 1 --2- 172.21.15.81:0/2554618358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fd4104060 0x7f3fd41a2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:52:09.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.691+0000 7f3fd259c700 1 -- 172.21.15.81:0/2554618358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3fd41024c0 msgr2=0x7f3fd419b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:52:09.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.691+0000 7f3fd259c700 1 --2- 172.21.15.81:0/2554618358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3fd41024c0 0x7f3fd419b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:52:09.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.691+0000 7f3fd259c700 1 -- 172.21.15.81:0/2554618358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fd4104060 msgr2=0x7f3fd41a2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:52:09.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.691+0000 7f3fd259c700 1 --2- 172.21.15.81:0/2554618358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fd4104060 0x7f3fd41a2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:52:09.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.691+0000 7f3fd259c700 1 -- 172.21.15.81:0/2554618358 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fd41a35d0 con 0x7f3fd41036c0 2024-05-15T13:52:09.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.691+0000 7f3fd259c700 1 --2- 172.21.15.81:0/2554618358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fd41036c0 0x7f3fd419beb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3fc40038a0 tx=0x7f3fc4003940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:52:09.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.691+0000 7f3fbbfff700 1 -- 172.21.15.81:0/2554618358 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc4027070 con 0x7f3fd41036c0 2024-05-15T13:52:09.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.691+0000 7f3fd9234700 1 -- 172.21.15.81:0/2554618358 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fd41a3860 con 0x7f3fd41036c0 2024-05-15T13:52:09.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.691+0000 7f3fbbfff700 1 -- 172.21.15.81:0/2554618358 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fc4022470 con 0x7f3fd41036c0 2024-05-15T13:52:09.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.691+0000 7f3fbbfff700 1 -- 172.21.15.81:0/2554618358 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc400f6c0 con 0x7f3fd41036c0 2024-05-15T13:52:09.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.691+0000 7f3fd9234700 1 -- 172.21.15.81:0/2554618358 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fd41a3cc0 con 0x7f3fd41036c0 2024-05-15T13:52:09.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.692+0000 7f3fd9234700 1 -- 172.21.15.81:0/2554618358 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fd406a230 con 0x7f3fd41036c0 2024-05-15T13:52:09.697 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.693+0000 7f3fbbfff700 1 -- 172.21.15.81:0/2554618358 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f3fc402a070 con 0x7f3fd41036c0 2024-05-15T13:52:09.697 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.694+0000 7f3fbbfff700 1 --2- 172.21.15.81:0/2554618358 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f3fbc0b27f0 0x7f3fbc0b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:52:09.697 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.694+0000 7f3fbbfff700 1 -- 172.21.15.81:0/2554618358 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f3fc40e1cb0 con 0x7f3fd41036c0 2024-05-15T13:52:09.697 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.694+0000 7f3fd2d9d700 1 --2- 172.21.15.81:0/2554618358 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f3fbc0b27f0 0x7f3fbc0b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:52:09.697 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.695+0000 7f3fd2d9d700 1 --2- 172.21.15.81:0/2554618358 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f3fbc0b27f0 0x7f3fbc0b4cb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4103520 tx=0x7f3fc8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:52:09.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.698+0000 7f3fbbfff700 1 -- 172.21.15.81:0/2554618358 <== mon.1 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3fc4013070 con 0x7f3fd41036c0 2024-05-15T13:52:09.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:09 smithi081 ceph-mon[36434]: osdmap e14: 2 total, 2 up, 2 in 2024-05-15T13:52:09.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:09 smithi081 ceph-mon[36434]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:09.915 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:09.912+0000 7f3fd9234700 1 -- 172.21.15.81:0/2554618358 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f3fd40ffc70 con 0x7f3fbc0b27f0 2024-05-15T13:52:09.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:09 smithi033 ceph-mon[31650]: osdmap e14: 2 total, 2 up, 2 in 2024-05-15T13:52:09.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:09 smithi033 ceph-mon[31650]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:10.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:09 smithi052 ceph-mon[36190]: osdmap e14: 2 total, 2 up, 2 in 2024-05-15T13:52:10.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:09 smithi052 ceph-mon[36190]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:10.824 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:10 smithi081 ceph-mon[36434]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:52:10.824 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:10 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:52:10.824 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:10 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:52:10.824 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:10 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:52:10.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:10 smithi033 ceph-mon[31650]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:52:10.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:10 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:52:10.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:10 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:52:10.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:10 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:52:11.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:10 smithi052 ceph-mon[36190]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:52:11.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:10 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:52:11.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:10 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:52:11.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:10 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:52:11.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:11 smithi081 ceph-mon[36434]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:11.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:11 smithi033 ceph-mon[31650]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:12.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:11 smithi052 ceph-mon[36190]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:12.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:12 smithi033 ceph-mon[31650]: from='client.? 172.21.15.81:0/1807310519' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e9c9390-ed33-464b-8f39-bca75941e05e"}]: dispatch 2024-05-15T13:52:12.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:12 smithi033 ceph-mon[31650]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e9c9390-ed33-464b-8f39-bca75941e05e"}]: dispatch 2024-05-15T13:52:12.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:12 smithi033 ceph-mon[31650]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e9c9390-ed33-464b-8f39-bca75941e05e"}]': finished 2024-05-15T13:52:12.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:12 smithi033 ceph-mon[31650]: osdmap e15: 3 total, 2 up, 3 in 2024-05-15T13:52:12.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:12 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T13:52:13.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:12 smithi052 ceph-mon[36190]: from='client.? 172.21.15.81:0/1807310519' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e9c9390-ed33-464b-8f39-bca75941e05e"}]: dispatch 2024-05-15T13:52:13.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:12 smithi052 ceph-mon[36190]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e9c9390-ed33-464b-8f39-bca75941e05e"}]: dispatch 2024-05-15T13:52:13.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:12 smithi052 ceph-mon[36190]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e9c9390-ed33-464b-8f39-bca75941e05e"}]': finished 2024-05-15T13:52:13.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:12 smithi052 ceph-mon[36190]: osdmap e15: 3 total, 2 up, 3 in 2024-05-15T13:52:13.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:12 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T13:52:13.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:12 smithi081 ceph-mon[36434]: from='client.? 172.21.15.81:0/1807310519' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e9c9390-ed33-464b-8f39-bca75941e05e"}]: dispatch 2024-05-15T13:52:13.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:12 smithi081 ceph-mon[36434]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e9c9390-ed33-464b-8f39-bca75941e05e"}]: dispatch 2024-05-15T13:52:13.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:12 smithi081 ceph-mon[36434]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e9c9390-ed33-464b-8f39-bca75941e05e"}]': finished 2024-05-15T13:52:13.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:12 smithi081 ceph-mon[36434]: osdmap e15: 3 total, 2 up, 3 in 2024-05-15T13:52:13.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:12 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T13:52:13.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:13 smithi033 ceph-mon[31650]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:13.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:13 smithi033 ceph-mon[31650]: from='client.? 172.21.15.81:0/3741811919' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:52:14.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:13 smithi052 ceph-mon[36190]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:14.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:13 smithi052 ceph-mon[36190]: from='client.? 172.21.15.81:0/3741811919' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:52:14.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:13 smithi081 ceph-mon[36434]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:14.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:13 smithi081 ceph-mon[36434]: from='client.? 172.21.15.81:0/3741811919' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:52:16.309 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:15 smithi081 ceph-mon[36434]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:16.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:15 smithi052 ceph-mon[36190]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:16.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:15 smithi033 ceph-mon[31650]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:18.092 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:17 smithi081 ceph-mon[36434]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:18.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:17 smithi052 ceph-mon[36190]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:18.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:17 smithi033 ceph-mon[31650]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:20.307 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:19 smithi081 ceph-mon[36434]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:20.307 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:19 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-15T13:52:20.307 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:19 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:52:20.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:19 smithi052 ceph-mon[36190]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:20.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:19 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-15T13:52:20.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:19 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:52:20.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:19 smithi033 ceph-mon[31650]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:20.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:19 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-15T13:52:20.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:19 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:52:21.333 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:20 smithi081 ceph-mon[36434]: Deploying daemon osd.2 on smithi081 2024-05-15T13:52:21.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:20 smithi052 ceph-mon[36190]: Deploying daemon osd.2 on smithi081 2024-05-15T13:52:21.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:20 smithi033 ceph-mon[31650]: Deploying daemon osd.2 on smithi081 2024-05-15T13:52:22.336 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:21 smithi081 ceph-mon[36434]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:22.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:21 smithi052 ceph-mon[36190]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:22.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:21 smithi033 ceph-mon[31650]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:24.078 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:23 smithi081 ceph-mon[36434]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:24.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:23 smithi052 ceph-mon[36190]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:24.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:23 smithi033 ceph-mon[31650]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:26.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:25 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:26.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:25 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:26.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:25 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:52:26.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:25 smithi052 ceph-mon[36190]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:26.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:25 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:26.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:25 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:26.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:25 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:52:26.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:25 smithi081 ceph-mon[36434]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:26.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:25 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:26.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:25 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:26.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:25 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:52:26.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:25 smithi033 ceph-mon[31650]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:28.499 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:28 smithi081 ceph-mon[36434]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:28.499 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:28 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:28.499 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:28 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:28.499 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:28 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:52:28.499 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:28 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:52:28.499 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:28 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:28.538 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:28.536+0000 7f3fbbfff700 1 -- 172.21.15.81:0/2554618358 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3fd40ffc70 con 0x7f3fbc0b27f0 2024-05-15T13:52:28.538 INFO:teuthology.orchestra.run.smithi081.stdout:Created osd(s) 2 on host 'smithi081' 2024-05-15T13:52:28.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:28.538+0000 7f3fd9234700 1 -- 172.21.15.81:0/2554618358 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f3fbc0b27f0 msgr2=0x7f3fbc0b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:52:28.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:28.538+0000 7f3fd9234700 1 --2- 172.21.15.81:0/2554618358 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f3fbc0b27f0 0x7f3fbc0b4cb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4103520 tx=0x7f3fc8009190 comp rx=0 tx=0).stop 2024-05-15T13:52:28.540 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:28.538+0000 7f3fd9234700 1 -- 172.21.15.81:0/2554618358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fd41036c0 msgr2=0x7f3fd419beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:52:28.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:28.538+0000 7f3fd9234700 1 --2- 172.21.15.81:0/2554618358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fd41036c0 0x7f3fd419beb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3fc40038a0 tx=0x7f3fc4003940 comp rx=0 tx=0).stop 2024-05-15T13:52:28.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:28.538+0000 7f3fd9234700 1 -- 172.21.15.81:0/2554618358 shutdown_connections 2024-05-15T13:52:28.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:28.538+0000 7f3fd9234700 1 --2- 172.21.15.81:0/2554618358 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f3fbc0b27f0 0x7f3fbc0b4cb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:52:28.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:28.538+0000 7f3fd9234700 1 --2- 172.21.15.81:0/2554618358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fd4104060 0x7f3fd41a2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:52:28.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:28.538+0000 7f3fd9234700 1 --2- 172.21.15.81:0/2554618358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3fd41024c0 0x7f3fd419b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:52:28.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:28.538+0000 7f3fd9234700 1 --2- 172.21.15.81:0/2554618358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fd41036c0 0x7f3fd419beb0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:52:28.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:28.538+0000 7f3fd9234700 1 -- 172.21.15.81:0/2554618358 >> 172.21.15.81:0/2554618358 conn(0x7f3fd40fda60 msgr2=0x7f3fd40ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:52:28.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:28.538+0000 7f3fd9234700 1 -- 172.21.15.81:0/2554618358 shutdown_connections 2024-05-15T13:52:28.541 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:52:28.538+0000 7f3fd9234700 1 -- 172.21.15.81:0/2554618358 wait complete. 2024-05-15T13:52:28.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:28 smithi033 ceph-mon[31650]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:28.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:28 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:28.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:28 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:28.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:28 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:52:28.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:28 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:52:28.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:28 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:28.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:28 smithi052 ceph-mon[36190]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:28.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:28 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:28.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:28 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:28.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:28 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:52:28.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:28 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:52:28.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:28 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:29.164 DEBUG:teuthology.orchestra.run.smithi081:osd.2> sudo journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.2.service 2024-05-15T13:52:29.166 INFO:tasks.cephadm:Deploying osd.3 on smithi033 with /dev/vg_nvme/lv_3... 2024-05-15T13:52:29.167 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-15T13:52:29.426 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:52:29.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:29 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:29.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:29 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:29.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:29 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:52:29.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:29 smithi052 ceph-mon[36190]: from='osd.2 [v2:172.21.15.81:6800/315610978,v1:172.21.15.81:6801/315610978]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-15T13:52:29.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:29 smithi052 ceph-mon[36190]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-15T13:52:29.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:29 smithi052 ceph-mon[36190]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:29.844 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:29 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:29.844 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:29 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:29.844 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:29 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:52:29.844 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:29 smithi033 ceph-mon[31650]: from='osd.2 [v2:172.21.15.81:6800/315610978,v1:172.21.15.81:6801/315610978]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-15T13:52:29.844 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:29 smithi033 ceph-mon[31650]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-15T13:52:29.844 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:29 smithi033 ceph-mon[31650]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:29.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:29 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:29.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:29 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:29.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:29 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:52:29.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:29 smithi081 ceph-mon[36434]: from='osd.2 [v2:172.21.15.81:6800/315610978,v1:172.21.15.81:6801/315610978]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-15T13:52:29.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:29 smithi081 ceph-mon[36434]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-15T13:52:29.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:29 smithi081 ceph-mon[36434]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:30.760 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:30 smithi033 ceph-mon[31650]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-15T13:52:30.760 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:30 smithi033 ceph-mon[31650]: osdmap e16: 3 total, 2 up, 3 in 2024-05-15T13:52:30.760 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:30 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T13:52:30.760 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:30 smithi033 ceph-mon[31650]: from='osd.2 [v2:172.21.15.81:6800/315610978,v1:172.21.15.81:6801/315610978]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-05-15T13:52:30.760 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:30 smithi033 ceph-mon[31650]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-05-15T13:52:30.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:30 smithi052 ceph-mon[36190]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-15T13:52:30.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:30 smithi052 ceph-mon[36190]: osdmap e16: 3 total, 2 up, 3 in 2024-05-15T13:52:30.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:30 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T13:52:30.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:30 smithi052 ceph-mon[36190]: from='osd.2 [v2:172.21.15.81:6800/315610978,v1:172.21.15.81:6801/315610978]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-05-15T13:52:30.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:30 smithi052 ceph-mon[36190]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-05-15T13:52:30.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:30 smithi081 ceph-mon[36434]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-15T13:52:30.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:30 smithi081 ceph-mon[36434]: osdmap e16: 3 total, 2 up, 3 in 2024-05-15T13:52:30.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:30 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T13:52:30.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:30 smithi081 ceph-mon[36434]: from='osd.2 [v2:172.21.15.81:6800/315610978,v1:172.21.15.81:6801/315610978]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-05-15T13:52:30.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:30 smithi081 ceph-mon[36434]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-05-15T13:52:30.858 INFO:journalctl@ceph.osd.2.smithi081.stdout:May 15 13:52:30 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-2[41560]: 2024-05-15T13:52:30.544+0000 7f89dc5f5700 -1 osd.2 0 waiting for initial osdmap 2024-05-15T13:52:30.858 INFO:journalctl@ceph.osd.2.smithi081.stdout:May 15 13:52:30 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-2[41560]: 2024-05-15T13:52:30.550+0000 7f89d83e8700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-15T13:52:31.386 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-15T13:52:31.386 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-15T13:52:31.386 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-05-15T13:52:31.386 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-05-15T13:52:31.386 INFO:teuthology.orchestra.run.smithi033.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111988 s, 936 MB/s 2024-05-15T13:52:31.387 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-05-15T13:52:31.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:31 smithi052 ceph-mon[36190]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-05-15T13:52:31.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:31 smithi052 ceph-mon[36190]: osdmap e17: 3 total, 2 up, 3 in 2024-05-15T13:52:31.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:31 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T13:52:31.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:31 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T13:52:31.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:31 smithi052 ceph-mon[36190]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:31.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:31 smithi081 ceph-mon[36434]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-05-15T13:52:31.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:31 smithi081 ceph-mon[36434]: osdmap e17: 3 total, 2 up, 3 in 2024-05-15T13:52:31.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:31 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T13:52:31.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:31 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T13:52:31.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:31 smithi081 ceph-mon[36434]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:31.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:31 smithi033 ceph-mon[31650]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-05-15T13:52:31.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:31 smithi033 ceph-mon[31650]: osdmap e17: 3 total, 2 up, 3 in 2024-05-15T13:52:31.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:31 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T13:52:31.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:31 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T13:52:31.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:31 smithi033 ceph-mon[31650]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-15T13:52:31.999 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_3 2024-05-15T13:52:32.248 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:52:32.624 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:32 smithi033 ceph-mon[31650]: purged_snaps scrub starts 2024-05-15T13:52:32.624 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:32 smithi033 ceph-mon[31650]: purged_snaps scrub ok 2024-05-15T13:52:32.624 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:32 smithi033 ceph-mon[31650]: osd.2 [v2:172.21.15.81:6800/315610978,v1:172.21.15.81:6801/315610978] boot 2024-05-15T13:52:32.624 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:32 smithi033 ceph-mon[31650]: osdmap e18: 3 total, 3 up, 3 in 2024-05-15T13:52:32.624 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:32 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T13:52:32.625 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:32 smithi033 ceph-mon[31650]: Detected new or changed devices on smithi081 2024-05-15T13:52:32.625 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:32 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:32.625 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:32 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:32.625 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:32 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:52:32.625 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:32 smithi033 ceph-mon[31650]: Adjusting osd_memory_target on smithi081 to 17088M 2024-05-15T13:52:32.625 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:32 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:32.625 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:32 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:52:32.625 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:32 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:52:32.625 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:32 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:32.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:32 smithi052 ceph-mon[36190]: purged_snaps scrub starts 2024-05-15T13:52:32.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:32 smithi052 ceph-mon[36190]: purged_snaps scrub ok 2024-05-15T13:52:32.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:32 smithi052 ceph-mon[36190]: osd.2 [v2:172.21.15.81:6800/315610978,v1:172.21.15.81:6801/315610978] boot 2024-05-15T13:52:32.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:32 smithi052 ceph-mon[36190]: osdmap e18: 3 total, 3 up, 3 in 2024-05-15T13:52:32.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:32 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T13:52:32.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:32 smithi052 ceph-mon[36190]: Detected new or changed devices on smithi081 2024-05-15T13:52:32.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:32 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:32.842 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:32 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:32.842 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:32 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:52:32.842 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:32 smithi052 ceph-mon[36190]: Adjusting osd_memory_target on smithi081 to 17088M 2024-05-15T13:52:32.842 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:32 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:32.842 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:32 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:52:32.842 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:32 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:52:32.842 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:32 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:32.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:32 smithi081 ceph-mon[36434]: purged_snaps scrub starts 2024-05-15T13:52:32.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:32 smithi081 ceph-mon[36434]: purged_snaps scrub ok 2024-05-15T13:52:32.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:32 smithi081 ceph-mon[36434]: osd.2 [v2:172.21.15.81:6800/315610978,v1:172.21.15.81:6801/315610978] boot 2024-05-15T13:52:32.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:32 smithi081 ceph-mon[36434]: osdmap e18: 3 total, 3 up, 3 in 2024-05-15T13:52:32.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:32 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T13:52:32.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:32 smithi081 ceph-mon[36434]: Detected new or changed devices on smithi081 2024-05-15T13:52:32.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:32 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:32.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:32 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:32.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:32 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:52:32.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:32 smithi081 ceph-mon[36434]: Adjusting osd_memory_target on smithi081 to 17088M 2024-05-15T13:52:32.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:32 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:32.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:32 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:52:32.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:32 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:52:32.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:32 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:33.671 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:33 smithi033 ceph-mon[31650]: osdmap e19: 3 total, 3 up, 3 in 2024-05-15T13:52:33.671 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:33 smithi033 ceph-mon[31650]: pgmap v83: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:33.671 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:33 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-15T13:52:33.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.779+0000 7ff0a8bf6700 1 -- 172.21.15.33:0/4085524159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0a41040e0 msgr2=0x7ff0a41064d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:52:33.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.779+0000 7ff0a8bf6700 1 --2- 172.21.15.33:0/4085524159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0a41040e0 0x7ff0a41064d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff09400b0a0 tx=0x7ff09400b3b0 comp rx=0 tx=0).stop 2024-05-15T13:52:33.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.779+0000 7ff0a8bf6700 1 -- 172.21.15.33:0/4085524159 shutdown_connections 2024-05-15T13:52:33.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.779+0000 7ff0a8bf6700 1 --2- 172.21.15.33:0/4085524159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0a4109340 0x7ff0a410b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:52:33.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.779+0000 7ff0a8bf6700 1 --2- 172.21.15.33:0/4085524159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0a41040e0 0x7ff0a41064d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:52:33.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.779+0000 7ff0a8bf6700 1 --2- 172.21.15.33:0/4085524159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0a4106a10 0x7ff0a4108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:52:33.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.779+0000 7ff0a8bf6700 1 -- 172.21.15.33:0/4085524159 >> 172.21.15.33:0/4085524159 conn(0x7ff0a40fda70 msgr2=0x7ff0a40ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:52:33.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.779+0000 7ff0a8bf6700 1 -- 172.21.15.33:0/4085524159 shutdown_connections 2024-05-15T13:52:33.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.779+0000 7ff0a8bf6700 1 -- 172.21.15.33:0/4085524159 wait complete. 2024-05-15T13:52:33.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.780+0000 7ff0a8bf6700 1 Processor -- start 2024-05-15T13:52:33.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.780+0000 7ff0a8bf6700 1 -- start start 2024-05-15T13:52:33.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.780+0000 7ff0a8bf6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0a41040e0 0x7ff0a41997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:52:33.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.780+0000 7ff0a8bf6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0a4106a10 0x7ff0a4199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:52:33.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.780+0000 7ff0a8bf6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0a4109340 0x7ff0a41a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:52:33.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.780+0000 7ff0a8bf6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a410f080 con 0x7ff0a41040e0 2024-05-15T13:52:33.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.780+0000 7ff0a8bf6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a410ef00 con 0x7ff0a4106a10 2024-05-15T13:52:33.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.780+0000 7ff0a8bf6700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a410f200 con 0x7ff0a4109340 2024-05-15T13:52:33.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.780+0000 7ff0a259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0a41040e0 0x7ff0a41997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:52:33.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.780+0000 7ff0a259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0a41040e0 0x7ff0a41997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42808/0 (socket says 172.21.15.33:42808) 2024-05-15T13:52:33.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.780+0000 7ff0a259c700 1 -- 172.21.15.33:0/970543609 learned_addr learned my addr 172.21.15.33:0/970543609 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:52:33.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.780+0000 7ff0a2d9d700 1 --2- 172.21.15.33:0/970543609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0a4109340 0x7ff0a41a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:52:33.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.780+0000 7ff0a1d9b700 1 --2- 172.21.15.33:0/970543609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0a4106a10 0x7ff0a4199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:52:33.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.780+0000 7ff0a259c700 1 -- 172.21.15.33:0/970543609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0a4109340 msgr2=0x7ff0a41a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:52:33.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.780+0000 7ff0a259c700 1 --2- 172.21.15.33:0/970543609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0a4109340 0x7ff0a41a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:52:33.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.780+0000 7ff0a259c700 1 -- 172.21.15.33:0/970543609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0a4106a10 msgr2=0x7ff0a4199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:52:33.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.780+0000 7ff0a259c700 1 --2- 172.21.15.33:0/970543609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0a4106a10 0x7ff0a4199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:52:33.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.780+0000 7ff0a259c700 1 -- 172.21.15.33:0/970543609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a41a1430 con 0x7ff0a41040e0 2024-05-15T13:52:33.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.780+0000 7ff0a1d9b700 1 --2- 172.21.15.33:0/970543609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0a4106a10 0x7ff0a4199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T13:52:33.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.780+0000 7ff0a259c700 1 --2- 172.21.15.33:0/970543609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0a41040e0 0x7ff0a41997d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff09400d010 tx=0x7ff094004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:52:33.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.781+0000 7ff0937fe700 1 -- 172.21.15.33:0/970543609 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094021070 con 0x7ff0a41040e0 2024-05-15T13:52:33.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.781+0000 7ff0a8bf6700 1 -- 172.21.15.33:0/970543609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0a41a16c0 con 0x7ff0a41040e0 2024-05-15T13:52:33.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.781+0000 7ff0937fe700 1 -- 172.21.15.33:0/970543609 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff094003c00 con 0x7ff0a41040e0 2024-05-15T13:52:33.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.781+0000 7ff0937fe700 1 -- 172.21.15.33:0/970543609 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094004880 con 0x7ff0a41040e0 2024-05-15T13:52:33.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.781+0000 7ff0a8bf6700 1 -- 172.21.15.33:0/970543609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0a41a1bc0 con 0x7ff0a41040e0 2024-05-15T13:52:33.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.782+0000 7ff0a8bf6700 1 -- 172.21.15.33:0/970543609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0a406a230 con 0x7ff0a41040e0 2024-05-15T13:52:33.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.783+0000 7ff0937fe700 1 -- 172.21.15.33:0/970543609 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7ff094007a00 con 0x7ff0a41040e0 2024-05-15T13:52:33.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.784+0000 7ff0937fe700 1 --2- 172.21.15.33:0/970543609 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff08c0b27f0 0x7ff08c0b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:52:33.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.784+0000 7ff0a1d9b700 1 --2- 172.21.15.33:0/970543609 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff08c0b27f0 0x7ff08c0b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:52:33.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.784+0000 7ff0937fe700 1 -- 172.21.15.33:0/970543609 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3226+0+0 (secure 0 0 0) 0x7ff09400f020 con 0x7ff0a41040e0 2024-05-15T13:52:33.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.784+0000 7ff0a1d9b700 1 --2- 172.21.15.33:0/970543609 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff08c0b27f0 0x7ff08c0b4cb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff098009be0 tx=0x7ff098009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:52:33.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:33.788+0000 7ff0937fe700 1 -- 172.21.15.33:0/970543609 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff09402a020 con 0x7ff0a41040e0 2024-05-15T13:52:33.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:33 smithi052 ceph-mon[36190]: osdmap e19: 3 total, 3 up, 3 in 2024-05-15T13:52:33.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:33 smithi052 ceph-mon[36190]: pgmap v83: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:33.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:33 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-15T13:52:33.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:33 smithi081 ceph-mon[36434]: osdmap e19: 3 total, 3 up, 3 in 2024-05-15T13:52:33.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:33 smithi081 ceph-mon[36434]: pgmap v83: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:33.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:33 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-15T13:52:34.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:34.012+0000 7ff0a8bf6700 1 -- 172.21.15.33:0/970543609 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ff0a4063980 con 0x7ff08c0b27f0 2024-05-15T13:52:34.830 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:34 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-15T13:52:34.830 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:34 smithi033 ceph-mon[31650]: osdmap e20: 3 total, 3 up, 3 in 2024-05-15T13:52:34.830 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:34 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-15T13:52:34.830 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:34 smithi033 ceph-mon[31650]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:52:34.830 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:34 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:52:34.830 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:34 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:52:34.830 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:34 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:52:34.830 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 15 13:52:34 smithi033 sudo[48488]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-15T13:52:34.831 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 15 13:52:34 smithi033 sudo[48488]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-15T13:52:34.831 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 15 13:52:34 smithi033 sudo[48488]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-15T13:52:34.831 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 15 13:52:34 smithi033 sudo[48488]: pam_unix(sudo:session): session closed for user root 2024-05-15T13:52:34.831 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 15 13:52:34 smithi033 sudo[48492]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-15T13:52:34.831 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 15 13:52:34 smithi033 sudo[48492]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-15T13:52:34.831 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 15 13:52:34 smithi033 sudo[48492]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-15T13:52:34.831 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 15 13:52:34 smithi033 sudo[48492]: pam_unix(sudo:session): session closed for user root 2024-05-15T13:52:34.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:34 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-15T13:52:34.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:34 smithi052 ceph-mon[36190]: osdmap e20: 3 total, 3 up, 3 in 2024-05-15T13:52:34.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:34 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-15T13:52:34.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:34 smithi052 ceph-mon[36190]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:52:34.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:34 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:52:34.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:34 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:52:34.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:34 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:52:34.842 INFO:journalctl@ceph.osd.1.smithi052.stdout:May 15 13:52:34 smithi052 sudo[44889]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-15T13:52:34.842 INFO:journalctl@ceph.osd.1.smithi052.stdout:May 15 13:52:34 smithi052 sudo[44889]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-15T13:52:34.842 INFO:journalctl@ceph.osd.1.smithi052.stdout:May 15 13:52:34 smithi052 sudo[44889]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-15T13:52:34.842 INFO:journalctl@ceph.osd.1.smithi052.stdout:May 15 13:52:34 smithi052 sudo[44889]: pam_unix(sudo:session): session closed for user root 2024-05-15T13:52:34.842 INFO:journalctl@ceph.osd.1.smithi052.stdout:May 15 13:52:34 smithi052 sudo[44892]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-15T13:52:34.842 INFO:journalctl@ceph.osd.1.smithi052.stdout:May 15 13:52:34 smithi052 sudo[44892]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-15T13:52:34.842 INFO:journalctl@ceph.osd.1.smithi052.stdout:May 15 13:52:34 smithi052 sudo[44892]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-15T13:52:34.842 INFO:journalctl@ceph.osd.1.smithi052.stdout:May 15 13:52:34 smithi052 sudo[44892]: pam_unix(sudo:session): session closed for user root 2024-05-15T13:52:34.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:34 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-15T13:52:34.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:34 smithi081 ceph-mon[36434]: osdmap e20: 3 total, 3 up, 3 in 2024-05-15T13:52:34.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:34 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-15T13:52:34.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:34 smithi081 ceph-mon[36434]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:52:34.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:34 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:52:34.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:34 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:52:34.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:34 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:52:34.858 INFO:journalctl@ceph.osd.2.smithi081.stdout:May 15 13:52:34 smithi081 sudo[45153]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-15T13:52:34.858 INFO:journalctl@ceph.osd.2.smithi081.stdout:May 15 13:52:34 smithi081 sudo[45153]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-15T13:52:34.858 INFO:journalctl@ceph.osd.2.smithi081.stdout:May 15 13:52:34 smithi081 sudo[45153]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-15T13:52:34.858 INFO:journalctl@ceph.osd.2.smithi081.stdout:May 15 13:52:34 smithi081 sudo[45153]: pam_unix(sudo:session): session closed for user root 2024-05-15T13:52:34.858 INFO:journalctl@ceph.osd.2.smithi081.stdout:May 15 13:52:34 smithi081 sudo[45156]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-15T13:52:34.858 INFO:journalctl@ceph.osd.2.smithi081.stdout:May 15 13:52:34 smithi081 sudo[45156]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-15T13:52:34.858 INFO:journalctl@ceph.osd.2.smithi081.stdout:May 15 13:52:34 smithi081 sudo[45156]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-15T13:52:34.858 INFO:journalctl@ceph.osd.2.smithi081.stdout:May 15 13:52:34 smithi081 sudo[45156]: pam_unix(sudo:session): session closed for user root 2024-05-15T13:52:35.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:34 smithi033 sudo[48499]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-15T13:52:35.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:34 smithi033 sudo[48499]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-15T13:52:35.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:34 smithi033 sudo[48499]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-15T13:52:35.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:34 smithi033 sudo[48499]: pam_unix(sudo:session): session closed for user root 2024-05-15T13:52:35.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:34 smithi033 sudo[48502]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-15T13:52:35.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:34 smithi033 sudo[48502]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-15T13:52:35.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:34 smithi033 sudo[48502]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-15T13:52:35.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:34 smithi033 sudo[48502]: pam_unix(sudo:session): session closed for user root 2024-05-15T13:52:35.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:34 smithi052 sudo[44895]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-15T13:52:35.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:34 smithi052 sudo[44895]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-15T13:52:35.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:34 smithi052 sudo[44895]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-15T13:52:35.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:34 smithi052 sudo[44895]: pam_unix(sudo:session): session closed for user root 2024-05-15T13:52:35.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:34 smithi052 sudo[44898]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-15T13:52:35.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:34 smithi052 sudo[44898]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-15T13:52:35.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:34 smithi052 sudo[44898]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-15T13:52:35.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:34 smithi052 sudo[44898]: pam_unix(sudo:session): session closed for user root 2024-05-15T13:52:35.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:34 smithi081 sudo[45159]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-15T13:52:35.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:34 smithi081 sudo[45159]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-15T13:52:35.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:34 smithi081 sudo[45159]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-15T13:52:35.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:34 smithi081 sudo[45159]: pam_unix(sudo:session): session closed for user root 2024-05-15T13:52:35.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:34 smithi081 sudo[45162]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-15T13:52:35.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:34 smithi081 sudo[45162]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-15T13:52:35.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:34 smithi081 sudo[45162]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-15T13:52:35.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:34 smithi081 sudo[45162]: pam_unix(sudo:session): session closed for user root 2024-05-15T13:52:35.831 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:35 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-15T13:52:35.831 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:35 smithi033 ceph-mon[31650]: osdmap e21: 3 total, 3 up, 3 in 2024-05-15T13:52:35.831 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:35 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:35.831 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:35 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:35.831 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:35 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:35.831 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:35 smithi033 ceph-mon[31650]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-15T13:52:35.832 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:35 smithi033 ceph-mon[31650]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-15T13:52:35.832 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:35 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-15T13:52:35.832 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:35 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-15T13:52:35.832 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:35 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-15T13:52:35.832 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:35 smithi033 ceph-mon[31650]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-15T13:52:35.832 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:35 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-15T13:52:35.832 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:35 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-15T13:52:35.832 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:35 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-15T13:52:35.832 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:35 smithi033 ceph-mon[31650]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-15T13:52:35.832 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:35 smithi033 ceph-mon[31650]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-15T13:52:35.832 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:35 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-15T13:52:35.832 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:35 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-15T13:52:35.832 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:35 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-15T13:52:35.832 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:35 smithi033 ceph-mon[31650]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:35.832 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:35 smithi033 ceph-mon[31650]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-15T13:52:35.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:35.828+0000 7ff0937fe700 1 -- 172.21.15.33:0/970543609 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff0940a7390 con 0x7ff0a41040e0 2024-05-15T13:52:35.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:35 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-15T13:52:35.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:35 smithi052 ceph-mon[36190]: osdmap e21: 3 total, 3 up, 3 in 2024-05-15T13:52:35.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:35 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:35.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:35 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:35.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:35 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:35.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:35 smithi052 ceph-mon[36190]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-15T13:52:35.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:35 smithi052 ceph-mon[36190]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-15T13:52:35.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:35 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-15T13:52:35.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:35 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-15T13:52:35.842 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:35 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-15T13:52:35.842 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:35 smithi052 ceph-mon[36190]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-15T13:52:35.842 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:35 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-15T13:52:35.842 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:35 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-15T13:52:35.842 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:35 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-15T13:52:35.842 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:35 smithi052 ceph-mon[36190]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-15T13:52:35.842 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:35 smithi052 ceph-mon[36190]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-15T13:52:35.842 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:35 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-15T13:52:35.842 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:35 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-15T13:52:35.842 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:35 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-15T13:52:35.842 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:35 smithi052 ceph-mon[36190]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:35.842 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:35 smithi052 ceph-mon[36190]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-15T13:52:35.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:35 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-15T13:52:35.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:35 smithi081 ceph-mon[36434]: osdmap e21: 3 total, 3 up, 3 in 2024-05-15T13:52:35.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:35 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:35.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:35 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:35.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:35 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:35.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:35 smithi081 ceph-mon[36434]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-15T13:52:35.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:35 smithi081 ceph-mon[36434]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-15T13:52:35.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:35 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-15T13:52:35.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:35 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-15T13:52:35.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:35 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-15T13:52:35.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:35 smithi081 ceph-mon[36434]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-15T13:52:35.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:35 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-15T13:52:35.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:35 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-15T13:52:35.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:35 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-15T13:52:35.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:35 smithi081 ceph-mon[36434]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-15T13:52:35.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:35 smithi081 ceph-mon[36434]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-15T13:52:35.859 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:35 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-15T13:52:35.859 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:35 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-15T13:52:35.859 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:35 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-15T13:52:35.859 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:35 smithi081 ceph-mon[36434]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:35.859 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:35 smithi081 ceph-mon[36434]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-15T13:52:36.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:36 smithi052 ceph-mon[36190]: osdmap e22: 3 total, 3 up, 3 in 2024-05-15T13:52:36.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:36 smithi052 ceph-mon[36190]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-15T13:52:36.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:36 smithi081 ceph-mon[36434]: osdmap e22: 3 total, 3 up, 3 in 2024-05-15T13:52:36.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:36 smithi081 ceph-mon[36434]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-15T13:52:36.887 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:36 smithi033 ceph-mon[31650]: osdmap e22: 3 total, 3 up, 3 in 2024-05-15T13:52:36.887 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:36 smithi033 ceph-mon[31650]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-15T13:52:37.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:37 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/2200650349' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ecdb24c1-fb15-42a5-9bae-436bbae9512a"}]: dispatch 2024-05-15T13:52:37.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:37 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/2200650349' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ecdb24c1-fb15-42a5-9bae-436bbae9512a"}]': finished 2024-05-15T13:52:37.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:37 smithi033 ceph-mon[31650]: osdmap e23: 4 total, 3 up, 4 in 2024-05-15T13:52:37.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:37 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T13:52:37.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:37 smithi033 ceph-mon[31650]: pgmap v89: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:37.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:37 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/2782043933' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:52:38.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:37 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/2200650349' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ecdb24c1-fb15-42a5-9bae-436bbae9512a"}]: dispatch 2024-05-15T13:52:38.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:37 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/2200650349' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ecdb24c1-fb15-42a5-9bae-436bbae9512a"}]': finished 2024-05-15T13:52:38.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:37 smithi052 ceph-mon[36190]: osdmap e23: 4 total, 3 up, 4 in 2024-05-15T13:52:38.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:37 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T13:52:38.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:37 smithi052 ceph-mon[36190]: pgmap v89: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:38.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:37 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/2782043933' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:52:38.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:37 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/2200650349' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ecdb24c1-fb15-42a5-9bae-436bbae9512a"}]: dispatch 2024-05-15T13:52:38.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:37 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/2200650349' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ecdb24c1-fb15-42a5-9bae-436bbae9512a"}]': finished 2024-05-15T13:52:38.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:37 smithi081 ceph-mon[36434]: osdmap e23: 4 total, 3 up, 4 in 2024-05-15T13:52:38.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:37 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T13:52:38.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:37 smithi081 ceph-mon[36434]: pgmap v89: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:38.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:37 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/2782043933' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:52:40.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:39 smithi052 ceph-mon[36190]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:40.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:39 smithi081 ceph-mon[36434]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:40.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:39 smithi033 ceph-mon[31650]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:42.247 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:41 smithi033 ceph-mon[31650]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:42.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:41 smithi052 ceph-mon[36190]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:42.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:41 smithi081 ceph-mon[36434]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:44.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:43 smithi033 ceph-mon[31650]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:44.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:43 smithi052 ceph-mon[36190]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:44.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:43 smithi081 ceph-mon[36434]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:45.310 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:44 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-15T13:52:45.311 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:44 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:52:45.311 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:44 smithi033 ceph-mon[31650]: Deploying daemon osd.3 on smithi033 2024-05-15T13:52:45.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:44 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-15T13:52:45.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:44 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:52:45.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:44 smithi052 ceph-mon[36190]: Deploying daemon osd.3 on smithi033 2024-05-15T13:52:45.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:44 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-15T13:52:45.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:44 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:52:45.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:44 smithi081 ceph-mon[36434]: Deploying daemon osd.3 on smithi033 2024-05-15T13:52:46.032 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:45 smithi033 ceph-mon[31650]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:46.242 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:45 smithi052 ceph-mon[36190]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:46.258 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:45 smithi081 ceph-mon[36434]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:48.321 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:47 smithi033 ceph-mon[31650]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:48.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:47 smithi052 ceph-mon[36190]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:48.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:47 smithi081 ceph-mon[36434]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:50.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:49 smithi033 ceph-mon[31650]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:50.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:49 smithi052 ceph-mon[36190]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:50.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:49 smithi081 ceph-mon[36434]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:51.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:51 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:51.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:51 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:51.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:51 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:52:51.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:51 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:51.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:51 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:51.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:51 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:52:51.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:51 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:51.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:51 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:51.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:51 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:52:52.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:52 smithi052 ceph-mon[36190]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:52.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:52 smithi081 ceph-mon[36434]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:52.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:52 smithi033 ceph-mon[31650]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:53.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:53.829+0000 7ff0937fe700 1 -- 172.21.15.33:0/970543609 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff0a4063980 con 0x7ff08c0b27f0 2024-05-15T13:52:53.831 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 3 on host 'smithi033' 2024-05-15T13:52:53.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:53.831+0000 7ff0a8bf6700 1 -- 172.21.15.33:0/970543609 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff08c0b27f0 msgr2=0x7ff08c0b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:52:53.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:53.831+0000 7ff0a8bf6700 1 --2- 172.21.15.33:0/970543609 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff08c0b27f0 0x7ff08c0b4cb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff098009be0 tx=0x7ff098009260 comp rx=0 tx=0).stop 2024-05-15T13:52:53.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:53.831+0000 7ff0a8bf6700 1 -- 172.21.15.33:0/970543609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0a41040e0 msgr2=0x7ff0a41997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:52:53.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:53.831+0000 7ff0a8bf6700 1 --2- 172.21.15.33:0/970543609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0a41040e0 0x7ff0a41997d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff09400d010 tx=0x7ff094004d70 comp rx=0 tx=0).stop 2024-05-15T13:52:53.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:53.831+0000 7ff0a8bf6700 1 -- 172.21.15.33:0/970543609 shutdown_connections 2024-05-15T13:52:53.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:53.831+0000 7ff0a8bf6700 1 --2- 172.21.15.33:0/970543609 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff08c0b27f0 0x7ff08c0b4cb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:52:53.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:53.831+0000 7ff0a8bf6700 1 --2- 172.21.15.33:0/970543609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0a4109340 0x7ff0a41a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:52:53.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:53.831+0000 7ff0a8bf6700 1 --2- 172.21.15.33:0/970543609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0a41040e0 0x7ff0a41997d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:52:53.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:53.831+0000 7ff0a8bf6700 1 --2- 172.21.15.33:0/970543609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0a4106a10 0x7ff0a4199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:52:53.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:53.831+0000 7ff0a8bf6700 1 -- 172.21.15.33:0/970543609 >> 172.21.15.33:0/970543609 conn(0x7ff0a40fda70 msgr2=0x7ff0a4105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:52:53.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:53.831+0000 7ff0a8bf6700 1 -- 172.21.15.33:0/970543609 shutdown_connections 2024-05-15T13:52:53.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:52:53.831+0000 7ff0a8bf6700 1 -- 172.21.15.33:0/970543609 wait complete. 2024-05-15T13:52:54.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:53 smithi033 ceph-mon[31650]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:54.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:53 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:54.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:53 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:54.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:53 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:52:54.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:53 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:52:54.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:53 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:54.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:53 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:54.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:53 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:54.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:53 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:52:54.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:53 smithi033 ceph-mon[31650]: from='osd.3 [v2:172.21.15.33:6810/818533501,v1:172.21.15.33:6811/818533501]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-15T13:52:54.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:53 smithi052 ceph-mon[36190]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:54.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:53 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:54.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:53 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:54.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:53 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:52:54.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:53 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:52:54.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:53 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:54.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:53 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:54.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:53 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:54.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:53 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:52:54.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:53 smithi052 ceph-mon[36190]: from='osd.3 [v2:172.21.15.33:6810/818533501,v1:172.21.15.33:6811/818533501]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-15T13:52:54.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:53 smithi081 ceph-mon[36434]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:54.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:53 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:54.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:53 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:54.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:53 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:52:54.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:53 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:52:54.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:53 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:54.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:53 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:54.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:53 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:54.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:53 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:52:54.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:53 smithi081 ceph-mon[36434]: from='osd.3 [v2:172.21.15.33:6810/818533501,v1:172.21.15.33:6811/818533501]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-15T13:52:54.497 DEBUG:teuthology.orchestra.run.smithi033:osd.3> sudo journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.3.service 2024-05-15T13:52:54.499 INFO:tasks.cephadm:Deploying osd.4 on smithi052 with /dev/vg_nvme/lv_3... 2024-05-15T13:52:54.499 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-15T13:52:54.754 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.b/config 2024-05-15T13:52:55.903 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:55 smithi033 ceph-mon[31650]: from='osd.3 [v2:172.21.15.33:6810/818533501,v1:172.21.15.33:6811/818533501]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-15T13:52:55.904 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:55 smithi033 ceph-mon[31650]: osdmap e24: 4 total, 3 up, 4 in 2024-05-15T13:52:55.904 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:55 smithi033 ceph-mon[31650]: from='osd.3 [v2:172.21.15.33:6810/818533501,v1:172.21.15.33:6811/818533501]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-15T13:52:55.904 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:55 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T13:52:55.904 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:55 smithi033 ceph-mon[31650]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:55.904 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 15 13:52:55 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-3[50628]: 2024-05-15T13:52:55.833+0000 7f9c7068f700 -1 osd.3 0 waiting for initial osdmap 2024-05-15T13:52:55.904 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 15 13:52:55 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-3[50628]: 2024-05-15T13:52:55.839+0000 7f9c6c482700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-15T13:52:56.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:55 smithi052 ceph-mon[36190]: from='osd.3 [v2:172.21.15.33:6810/818533501,v1:172.21.15.33:6811/818533501]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-15T13:52:56.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:55 smithi052 ceph-mon[36190]: osdmap e24: 4 total, 3 up, 4 in 2024-05-15T13:52:56.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:55 smithi052 ceph-mon[36190]: from='osd.3 [v2:172.21.15.33:6810/818533501,v1:172.21.15.33:6811/818533501]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-15T13:52:56.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:55 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T13:52:56.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:55 smithi052 ceph-mon[36190]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:56.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:55 smithi081 ceph-mon[36434]: from='osd.3 [v2:172.21.15.33:6810/818533501,v1:172.21.15.33:6811/818533501]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-15T13:52:56.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:55 smithi081 ceph-mon[36434]: osdmap e24: 4 total, 3 up, 4 in 2024-05-15T13:52:56.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:55 smithi081 ceph-mon[36434]: from='osd.3 [v2:172.21.15.33:6810/818533501,v1:172.21.15.33:6811/818533501]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-15T13:52:56.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:55 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T13:52:56.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:55 smithi081 ceph-mon[36434]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-15T13:52:56.691 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-15T13:52:56.691 INFO:teuthology.orchestra.run.smithi052.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-15T13:52:56.691 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10+0 records in 2024-05-15T13:52:56.692 INFO:teuthology.orchestra.run.smithi052.stderr:10+0 records out 2024-05-15T13:52:56.692 INFO:teuthology.orchestra.run.smithi052.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00815516 s, 1.3 GB/s 2024-05-15T13:52:56.692 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping successful for: 2024-05-15T13:52:56.865 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:56 smithi052 ceph-mon[36190]: from='osd.3 [v2:172.21.15.33:6810/818533501,v1:172.21.15.33:6811/818533501]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-15T13:52:56.865 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:56 smithi052 ceph-mon[36190]: osdmap e25: 4 total, 3 up, 4 in 2024-05-15T13:52:56.865 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:56 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T13:52:56.865 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:56 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T13:52:57.083 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:56 smithi033 ceph-mon[31650]: from='osd.3 [v2:172.21.15.33:6810/818533501,v1:172.21.15.33:6811/818533501]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-15T13:52:57.083 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:56 smithi033 ceph-mon[31650]: osdmap e25: 4 total, 3 up, 4 in 2024-05-15T13:52:57.083 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:56 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T13:52:57.083 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:56 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T13:52:57.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:56 smithi081 ceph-mon[36434]: from='osd.3 [v2:172.21.15.33:6810/818533501,v1:172.21.15.33:6811/818533501]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-15T13:52:57.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:56 smithi081 ceph-mon[36434]: osdmap e25: 4 total, 3 up, 4 in 2024-05-15T13:52:57.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:56 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T13:52:57.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:56 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T13:52:57.336 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi052:vg_nvme/lv_3 2024-05-15T13:52:57.598 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.b/config 2024-05-15T13:52:57.953 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:57 smithi052 ceph-mon[36190]: purged_snaps scrub starts 2024-05-15T13:52:57.953 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:57 smithi052 ceph-mon[36190]: purged_snaps scrub ok 2024-05-15T13:52:57.954 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:57 smithi052 ceph-mon[36190]: osd.3 [v2:172.21.15.33:6810/818533501,v1:172.21.15.33:6811/818533501] boot 2024-05-15T13:52:57.954 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:57 smithi052 ceph-mon[36190]: osdmap e26: 4 total, 4 up, 4 in 2024-05-15T13:52:57.954 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:57 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T13:52:57.954 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:57 smithi052 ceph-mon[36190]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-05-15T13:52:57.954 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:57 smithi052 ceph-mon[36190]: Detected new or changed devices on smithi033 2024-05-15T13:52:57.954 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:57 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:57.954 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:57 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:57.954 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:57 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:52:57.954 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:57 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:52:57.954 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:57 smithi052 ceph-mon[36190]: Adjusting osd_memory_target on smithi033 to 8546M 2024-05-15T13:52:57.954 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:57 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:57.954 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:57 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:52:57.955 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:57 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:52:57.955 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:57 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:57.955 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:57 smithi052 ceph-mon[36190]: osdmap e27: 4 total, 4 up, 4 in 2024-05-15T13:52:58.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:57 smithi081 ceph-mon[36434]: purged_snaps scrub starts 2024-05-15T13:52:58.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:57 smithi081 ceph-mon[36434]: purged_snaps scrub ok 2024-05-15T13:52:58.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:57 smithi081 ceph-mon[36434]: osd.3 [v2:172.21.15.33:6810/818533501,v1:172.21.15.33:6811/818533501] boot 2024-05-15T13:52:58.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:57 smithi081 ceph-mon[36434]: osdmap e26: 4 total, 4 up, 4 in 2024-05-15T13:52:58.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:57 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T13:52:58.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:57 smithi081 ceph-mon[36434]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-05-15T13:52:58.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:57 smithi081 ceph-mon[36434]: Detected new or changed devices on smithi033 2024-05-15T13:52:58.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:57 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:58.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:57 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:58.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:57 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:52:58.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:57 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:52:58.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:57 smithi081 ceph-mon[36434]: Adjusting osd_memory_target on smithi033 to 8546M 2024-05-15T13:52:58.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:57 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:58.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:57 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:52:58.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:57 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:52:58.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:57 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:58.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:57 smithi081 ceph-mon[36434]: osdmap e27: 4 total, 4 up, 4 in 2024-05-15T13:52:58.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:57 smithi033 ceph-mon[31650]: purged_snaps scrub starts 2024-05-15T13:52:58.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:57 smithi033 ceph-mon[31650]: purged_snaps scrub ok 2024-05-15T13:52:58.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:57 smithi033 ceph-mon[31650]: osd.3 [v2:172.21.15.33:6810/818533501,v1:172.21.15.33:6811/818533501] boot 2024-05-15T13:52:58.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:57 smithi033 ceph-mon[31650]: osdmap e26: 4 total, 4 up, 4 in 2024-05-15T13:52:58.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:57 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T13:52:58.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:57 smithi033 ceph-mon[31650]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail 2024-05-15T13:52:58.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:57 smithi033 ceph-mon[31650]: Detected new or changed devices on smithi033 2024-05-15T13:52:58.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:57 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:58.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:57 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:58.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:57 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:52:58.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:57 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:52:58.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:57 smithi033 ceph-mon[31650]: Adjusting osd_memory_target on smithi033 to 8546M 2024-05-15T13:52:58.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:57 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:58.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:57 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:52:58.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:57 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:52:58.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:57 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:52:58.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:57 smithi033 ceph-mon[31650]: osdmap e27: 4 total, 4 up, 4 in 2024-05-15T13:52:59.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.049+0000 7f42f8f73700 1 -- 172.21.15.52:0/1815579075 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42f41069a0 msgr2=0x7f42f4108d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:52:59.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.049+0000 7f42f8f73700 1 --2- 172.21.15.52:0/1815579075 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42f41069a0 0x7f42f4108d90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f42e8009990 tx=0x7f42e8009ca0 comp rx=0 tx=0).stop 2024-05-15T13:52:59.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.049+0000 7f42f8f73700 1 -- 172.21.15.52:0/1815579075 shutdown_connections 2024-05-15T13:52:59.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.049+0000 7f42f8f73700 1 --2- 172.21.15.52:0/1815579075 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42f41092d0 0x7f42f410b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:52:59.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.049+0000 7f42f8f73700 1 --2- 172.21.15.52:0/1815579075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f4104070 0x7f42f4106460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:52:59.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.049+0000 7f42f8f73700 1 --2- 172.21.15.52:0/1815579075 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42f41069a0 0x7f42f4108d90 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:52:59.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.049+0000 7f42f8f73700 1 -- 172.21.15.52:0/1815579075 >> 172.21.15.52:0/1815579075 conn(0x7f42f40fdaa0 msgr2=0x7f42f40ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:52:59.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.049+0000 7f42f8f73700 1 -- 172.21.15.52:0/1815579075 shutdown_connections 2024-05-15T13:52:59.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.049+0000 7f42f8f73700 1 -- 172.21.15.52:0/1815579075 wait complete. 2024-05-15T13:52:59.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.050+0000 7f42f8f73700 1 Processor -- start 2024-05-15T13:52:59.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.050+0000 7f42f8f73700 1 -- start start 2024-05-15T13:52:59.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.050+0000 7f42f8f73700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42f4104070 0x7f42f41996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:52:59.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.050+0000 7f42f8f73700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42f41069a0 0x7f42f4199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:52:59.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.050+0000 7f42f8f73700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f41092d0 0x7f42f41a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:52:59.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.050+0000 7f42f8f73700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f42f410f0a0 con 0x7f42f41092d0 2024-05-15T13:52:59.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.050+0000 7f42f8f73700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f42f410ef20 con 0x7f42f4104070 2024-05-15T13:52:59.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.050+0000 7f42f8f73700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f42f410f220 con 0x7f42f41069a0 2024-05-15T13:52:59.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.050+0000 7f42f259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42f4104070 0x7f42f41996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:52:59.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.050+0000 7f42f259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42f4104070 0x7f42f41996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60392/0 (socket says 172.21.15.52:60392) 2024-05-15T13:52:59.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.050+0000 7f42f259c700 1 -- 172.21.15.52:0/290957185 learned_addr learned my addr 172.21.15.52:0/290957185 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-15T13:52:59.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.050+0000 7f42f2d9d700 1 --2- 172.21.15.52:0/290957185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f41092d0 0x7f42f41a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:52:59.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.050+0000 7f42f1d9b700 1 --2- 172.21.15.52:0/290957185 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42f41069a0 0x7f42f4199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:52:59.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.050+0000 7f42f259c700 1 -- 172.21.15.52:0/290957185 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42f41069a0 msgr2=0x7f42f4199c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:52:59.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.050+0000 7f42f259c700 1 --2- 172.21.15.52:0/290957185 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42f41069a0 0x7f42f4199c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:52:59.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.050+0000 7f42f259c700 1 -- 172.21.15.52:0/290957185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f41092d0 msgr2=0x7f42f41a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:52:59.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.050+0000 7f42f259c700 1 --2- 172.21.15.52:0/290957185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f41092d0 0x7f42f41a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:52:59.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.050+0000 7f42f259c700 1 -- 172.21.15.52:0/290957185 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f41a1320 con 0x7f42f4104070 2024-05-15T13:52:59.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.051+0000 7f42f259c700 1 --2- 172.21.15.52:0/290957185 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42f4104070 0x7f42f41996c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f42e400d950 tx=0x7f42e400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:52:59.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.051+0000 7f42e37fe700 1 -- 172.21.15.52:0/290957185 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e4016070 con 0x7f42f4104070 2024-05-15T13:52:59.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.051+0000 7f42f8f73700 1 -- 172.21.15.52:0/290957185 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42f41a1610 con 0x7f42f4104070 2024-05-15T13:52:59.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.051+0000 7f42e37fe700 1 -- 172.21.15.52:0/290957185 <== mon.2 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42e4009df0 con 0x7f42f4104070 2024-05-15T13:52:59.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.051+0000 7f42e37fe700 1 -- 172.21.15.52:0/290957185 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e400f620 con 0x7f42f4104070 2024-05-15T13:52:59.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.051+0000 7f42f8f73700 1 -- 172.21.15.52:0/290957185 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42f41a1ba0 con 0x7f42f4104070 2024-05-15T13:52:59.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.052+0000 7f42f8f73700 1 -- 172.21.15.52:0/290957185 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42f406a230 con 0x7f42f4104070 2024-05-15T13:52:59.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.053+0000 7f42e37fe700 1 -- 172.21.15.52:0/290957185 <== mon.2 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f42e4019070 con 0x7f42f4104070 2024-05-15T13:52:59.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.054+0000 7f42e37fe700 1 --2- 172.21.15.52:0/290957185 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f42dc0b2880 0x7f42dc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:52:59.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.054+0000 7f42e37fe700 1 -- 172.21.15.52:0/290957185 <== mon.2 v2:172.21.15.52:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3669+0+0 (secure 0 0 0) 0x7f42e40e0f90 con 0x7f42f4104070 2024-05-15T13:52:59.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.054+0000 7f42f1d9b700 1 --2- 172.21.15.52:0/290957185 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f42dc0b2880 0x7f42dc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:52:59.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.054+0000 7f42f1d9b700 1 --2- 172.21.15.52:0/290957185 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f42dc0b2880 0x7f42dc0b4d40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f42e800b5c0 tx=0x7f42e8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:52:59.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.058+0000 7f42e37fe700 1 -- 172.21.15.52:0/290957185 <== mon.2 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f42e40a9e30 con 0x7f42f4104070 2024-05-15T13:52:59.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:52:59.284+0000 7f42f8f73700 1 -- 172.21.15.52:0/290957185 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f42f4063980 con 0x7f42dc0b2880 2024-05-15T13:53:00.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:59 smithi052 ceph-mon[36190]: osdmap e28: 4 total, 4 up, 4 in 2024-05-15T13:53:00.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:59 smithi052 ceph-mon[36190]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-15T13:53:00.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:59 smithi052 ceph-mon[36190]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:53:00.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:59 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:53:00.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:59 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:53:00.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:52:59 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:53:00.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:59 smithi081 ceph-mon[36434]: osdmap e28: 4 total, 4 up, 4 in 2024-05-15T13:53:00.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:59 smithi081 ceph-mon[36434]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-15T13:53:00.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:59 smithi081 ceph-mon[36434]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:53:00.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:59 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:53:00.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:59 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:53:00.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:52:59 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:53:00.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:59 smithi033 ceph-mon[31650]: osdmap e28: 4 total, 4 up, 4 in 2024-05-15T13:53:00.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:59 smithi033 ceph-mon[31650]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-15T13:53:00.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:59 smithi033 ceph-mon[31650]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:53:00.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:59 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:53:00.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:59 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:53:00.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:52:59 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:53:02.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:01 smithi033 ceph-mon[31650]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-05-15T13:53:02.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:01 smithi033 ceph-mon[31650]: from='client.? 172.21.15.52:0/685624501' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "706752a0-b17f-4563-a7a9-d5078bf0f3b8"}]: dispatch 2024-05-15T13:53:02.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:01 smithi033 ceph-mon[31650]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "706752a0-b17f-4563-a7a9-d5078bf0f3b8"}]: dispatch 2024-05-15T13:53:02.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:01 smithi033 ceph-mon[31650]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "706752a0-b17f-4563-a7a9-d5078bf0f3b8"}]': finished 2024-05-15T13:53:02.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:01 smithi033 ceph-mon[31650]: osdmap e29: 5 total, 4 up, 5 in 2024-05-15T13:53:02.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:01 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T13:53:02.290 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:01 smithi052 ceph-mon[36190]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-05-15T13:53:02.290 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:01 smithi052 ceph-mon[36190]: from='client.? 172.21.15.52:0/685624501' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "706752a0-b17f-4563-a7a9-d5078bf0f3b8"}]: dispatch 2024-05-15T13:53:02.290 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:01 smithi052 ceph-mon[36190]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "706752a0-b17f-4563-a7a9-d5078bf0f3b8"}]: dispatch 2024-05-15T13:53:02.290 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:01 smithi052 ceph-mon[36190]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "706752a0-b17f-4563-a7a9-d5078bf0f3b8"}]': finished 2024-05-15T13:53:02.291 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:01 smithi052 ceph-mon[36190]: osdmap e29: 5 total, 4 up, 5 in 2024-05-15T13:53:02.291 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:01 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T13:53:02.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:01 smithi081 ceph-mon[36434]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 358 GiB / 358 GiB avail 2024-05-15T13:53:02.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:01 smithi081 ceph-mon[36434]: from='client.? 172.21.15.52:0/685624501' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "706752a0-b17f-4563-a7a9-d5078bf0f3b8"}]: dispatch 2024-05-15T13:53:02.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:01 smithi081 ceph-mon[36434]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "706752a0-b17f-4563-a7a9-d5078bf0f3b8"}]: dispatch 2024-05-15T13:53:02.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:01 smithi081 ceph-mon[36434]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "706752a0-b17f-4563-a7a9-d5078bf0f3b8"}]': finished 2024-05-15T13:53:02.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:01 smithi081 ceph-mon[36434]: osdmap e29: 5 total, 4 up, 5 in 2024-05-15T13:53:02.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:01 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T13:53:03.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:02 smithi052 ceph-mon[36190]: from='client.? 172.21.15.52:0/2002547276' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:53:03.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:02 smithi081 ceph-mon[36434]: from='client.? 172.21.15.52:0/2002547276' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:53:03.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:02 smithi033 ceph-mon[31650]: from='client.? 172.21.15.52:0/2002547276' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:53:04.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:03 smithi052 ceph-mon[36190]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-15T13:53:04.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:03 smithi081 ceph-mon[36434]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-15T13:53:04.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:03 smithi033 ceph-mon[31650]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-15T13:53:06.234 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:05 smithi052 ceph-mon[36190]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-05-15T13:53:06.243 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:05 smithi033 ceph-mon[31650]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-05-15T13:53:06.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:05 smithi081 ceph-mon[36434]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-05-15T13:53:08.241 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:07 smithi052 ceph-mon[36190]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-05-15T13:53:08.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:07 smithi081 ceph-mon[36434]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-05-15T13:53:08.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:07 smithi033 ceph-mon[31650]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-05-15T13:53:10.041 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:09 smithi052 ceph-mon[36190]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-15T13:53:10.041 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:09 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-15T13:53:10.042 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:09 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:53:10.042 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:09 smithi052 ceph-mon[36190]: Deploying daemon osd.4 on smithi052 2024-05-15T13:53:10.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:09 smithi081 ceph-mon[36434]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-15T13:53:10.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:09 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-15T13:53:10.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:09 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:53:10.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:09 smithi081 ceph-mon[36434]: Deploying daemon osd.4 on smithi052 2024-05-15T13:53:10.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:09 smithi033 ceph-mon[31650]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-15T13:53:10.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:09 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-15T13:53:10.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:09 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:53:10.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:09 smithi033 ceph-mon[31650]: Deploying daemon osd.4 on smithi052 2024-05-15T13:53:12.262 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:11 smithi052 ceph-mon[36190]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-15T13:53:12.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:11 smithi081 ceph-mon[36434]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-15T13:53:12.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:11 smithi033 ceph-mon[31650]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-15T13:53:14.290 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:13 smithi052 ceph-mon[36190]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-15T13:53:14.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:13 smithi081 ceph-mon[36434]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-15T13:53:14.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:13 smithi033 ceph-mon[31650]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-15T13:53:15.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:15 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:15.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:15 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:15.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:15 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:53:15.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:15 smithi052 ceph-mon[36190]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-15T13:53:15.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:15 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:15.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:15 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:15.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:15 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:53:15.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:15 smithi081 ceph-mon[36434]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-15T13:53:15.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:15 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:15.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:15 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:15.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:15 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:53:15.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:15 smithi033 ceph-mon[31650]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-15T13:53:18.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:53:18.200+0000 7f42e37fe700 1 -- 172.21.15.52:0/290957185 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f42f4063980 con 0x7f42dc0b2880 2024-05-15T13:53:18.203 INFO:teuthology.orchestra.run.smithi052.stdout:Created osd(s) 4 on host 'smithi052' 2024-05-15T13:53:18.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:53:18.202+0000 7f42f8f73700 1 -- 172.21.15.52:0/290957185 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f42dc0b2880 msgr2=0x7f42dc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:53:18.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:53:18.202+0000 7f42f8f73700 1 --2- 172.21.15.52:0/290957185 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f42dc0b2880 0x7f42dc0b4d40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f42e800b5c0 tx=0x7f42e8005fd0 comp rx=0 tx=0).stop 2024-05-15T13:53:18.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:53:18.202+0000 7f42f8f73700 1 -- 172.21.15.52:0/290957185 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42f4104070 msgr2=0x7f42f41996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:53:18.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:53:18.202+0000 7f42f8f73700 1 --2- 172.21.15.52:0/290957185 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42f4104070 0x7f42f41996c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f42e400d950 tx=0x7f42e400dc60 comp rx=0 tx=0).stop 2024-05-15T13:53:18.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:53:18.202+0000 7f42f8f73700 1 -- 172.21.15.52:0/290957185 shutdown_connections 2024-05-15T13:53:18.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:53:18.202+0000 7f42f8f73700 1 --2- 172.21.15.52:0/290957185 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f42dc0b2880 0x7f42dc0b4d40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:53:18.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:53:18.202+0000 7f42f8f73700 1 --2- 172.21.15.52:0/290957185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f41092d0 0x7f42f41a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:53:18.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:53:18.202+0000 7f42f8f73700 1 --2- 172.21.15.52:0/290957185 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42f4104070 0x7f42f41996c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:53:18.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:53:18.202+0000 7f42f8f73700 1 --2- 172.21.15.52:0/290957185 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42f41069a0 0x7f42f4199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:53:18.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:53:18.202+0000 7f42f8f73700 1 -- 172.21.15.52:0/290957185 >> 172.21.15.52:0/290957185 conn(0x7f42f40fdaa0 msgr2=0x7f42f4104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:53:18.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:53:18.202+0000 7f42f8f73700 1 -- 172.21.15.52:0/290957185 shutdown_connections 2024-05-15T13:53:18.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:53:18.202+0000 7f42f8f73700 1 -- 172.21.15.52:0/290957185 wait complete. 2024-05-15T13:53:18.445 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:18 smithi052 ceph-mon[36190]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-15T13:53:18.445 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:18 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:18.445 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:18 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:18.446 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:18 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:53:18.446 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:18 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:53:18.446 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:18 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:18.446 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:18 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:18.446 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:18 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:18.446 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:18 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:53:18.561 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:18 smithi081 ceph-mon[36434]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-15T13:53:18.561 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:18 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:18.561 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:18 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:18.561 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:18 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:53:18.561 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:18 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:53:18.561 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:18 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:18.561 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:18 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:18.561 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:18 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:18.562 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:18 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:53:18.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:18 smithi033 ceph-mon[31650]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-15T13:53:18.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:18 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:18.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:18 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:18.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:18 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:53:18.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:18 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:53:18.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:18 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:18.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:18 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:18.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:18 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:18.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:18 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:53:18.769 DEBUG:teuthology.orchestra.run.smithi052:osd.4> sudo journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.4.service 2024-05-15T13:53:18.772 INFO:tasks.cephadm:Deploying osd.5 on smithi081 with /dev/vg_nvme/lv_3... 2024-05-15T13:53:18.772 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-15T13:53:19.016 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.c/config 2024-05-15T13:53:19.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:19 smithi052 ceph-mon[36190]: from='osd.4 [v2:172.21.15.52:6808/3010133776,v1:172.21.15.52:6809/3010133776]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-15T13:53:19.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:19 smithi052 ceph-mon[36190]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-15T13:53:19.418 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:19 smithi081 ceph-mon[36434]: from='osd.4 [v2:172.21.15.52:6808/3010133776,v1:172.21.15.52:6809/3010133776]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-15T13:53:19.419 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:19 smithi081 ceph-mon[36434]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-15T13:53:19.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:19 smithi033 ceph-mon[31650]: from='osd.4 [v2:172.21.15.52:6808/3010133776,v1:172.21.15.52:6809/3010133776]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-15T13:53:19.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:19 smithi033 ceph-mon[31650]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-15T13:53:20.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:20 smithi052 ceph-mon[36190]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-15T13:53:20.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:20 smithi052 ceph-mon[36190]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-15T13:53:20.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:20 smithi052 ceph-mon[36190]: osdmap e30: 5 total, 4 up, 5 in 2024-05-15T13:53:20.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:20 smithi052 ceph-mon[36190]: from='osd.4 [v2:172.21.15.52:6808/3010133776,v1:172.21.15.52:6809/3010133776]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-05-15T13:53:20.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:20 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T13:53:20.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:20 smithi052 ceph-mon[36190]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-05-15T13:53:20.592 INFO:journalctl@ceph.osd.4.smithi052.stdout:May 15 13:53:20 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-4[47696]: 2024-05-15T13:53:20.312+0000 7f5440107700 -1 osd.4 0 waiting for initial osdmap 2024-05-15T13:53:20.592 INFO:journalctl@ceph.osd.4.smithi052.stdout:May 15 13:53:20 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-4[47696]: 2024-05-15T13:53:20.320+0000 7f543befa700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-15T13:53:20.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:20 smithi081 ceph-mon[36434]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-15T13:53:20.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:20 smithi081 ceph-mon[36434]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-15T13:53:20.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:20 smithi081 ceph-mon[36434]: osdmap e30: 5 total, 4 up, 5 in 2024-05-15T13:53:20.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:20 smithi081 ceph-mon[36434]: from='osd.4 [v2:172.21.15.52:6808/3010133776,v1:172.21.15.52:6809/3010133776]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-05-15T13:53:20.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:20 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T13:53:20.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:20 smithi081 ceph-mon[36434]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-05-15T13:53:20.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:20 smithi033 ceph-mon[31650]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-15T13:53:20.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:20 smithi033 ceph-mon[31650]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-15T13:53:20.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:20 smithi033 ceph-mon[31650]: osdmap e30: 5 total, 4 up, 5 in 2024-05-15T13:53:20.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:20 smithi033 ceph-mon[31650]: from='osd.4 [v2:172.21.15.52:6808/3010133776,v1:172.21.15.52:6809/3010133776]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-05-15T13:53:20.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:20 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T13:53:20.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:20 smithi033 ceph-mon[31650]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-05-15T13:53:20.883 INFO:teuthology.orchestra.run.smithi081.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-15T13:53:20.883 INFO:teuthology.orchestra.run.smithi081.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-15T13:53:20.883 INFO:teuthology.orchestra.run.smithi081.stderr: stderr: 10+0 records in 2024-05-15T13:53:20.883 INFO:teuthology.orchestra.run.smithi081.stderr:10+0 records out 2024-05-15T13:53:20.883 INFO:teuthology.orchestra.run.smithi081.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0118069 s, 888 MB/s 2024-05-15T13:53:20.883 INFO:teuthology.orchestra.run.smithi081.stderr:--> Zapping successful for: 2024-05-15T13:53:21.432 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:21 smithi052 ceph-mon[36190]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-05-15T13:53:21.432 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:21 smithi052 ceph-mon[36190]: osdmap e31: 5 total, 4 up, 5 in 2024-05-15T13:53:21.432 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:21 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T13:53:21.432 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:21 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T13:53:21.437 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:21 smithi081 ceph-mon[36434]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-05-15T13:53:21.437 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:21 smithi081 ceph-mon[36434]: osdmap e31: 5 total, 4 up, 5 in 2024-05-15T13:53:21.437 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:21 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T13:53:21.437 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:21 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T13:53:21.464 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi081:vg_nvme/lv_3 2024-05-15T13:53:21.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:21 smithi033 ceph-mon[31650]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-05-15T13:53:21.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:21 smithi033 ceph-mon[31650]: osdmap e31: 5 total, 4 up, 5 in 2024-05-15T13:53:21.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:21 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T13:53:21.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:21 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T13:53:21.708 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.c/config 2024-05-15T13:53:22.594 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:22 smithi052 ceph-mon[36190]: purged_snaps scrub starts 2024-05-15T13:53:22.594 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:22 smithi052 ceph-mon[36190]: purged_snaps scrub ok 2024-05-15T13:53:22.594 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:22 smithi052 ceph-mon[36190]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-15T13:53:22.594 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:22 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T13:53:22.594 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:22 smithi052 ceph-mon[36190]: osd.4 [v2:172.21.15.52:6808/3010133776,v1:172.21.15.52:6809/3010133776] boot 2024-05-15T13:53:22.594 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:22 smithi052 ceph-mon[36190]: osdmap e32: 5 total, 5 up, 5 in 2024-05-15T13:53:22.594 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:22 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T13:53:22.594 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:22 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:22.594 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:22 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:22.594 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:22 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:53:22.594 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:22 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:53:22.595 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:22 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:22.595 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:22 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:53:22.595 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:22 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:53:22.595 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:22 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:22.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:22 smithi081 ceph-mon[36434]: purged_snaps scrub starts 2024-05-15T13:53:22.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:22 smithi081 ceph-mon[36434]: purged_snaps scrub ok 2024-05-15T13:53:22.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:22 smithi081 ceph-mon[36434]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-15T13:53:22.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:22 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T13:53:22.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:22 smithi081 ceph-mon[36434]: osd.4 [v2:172.21.15.52:6808/3010133776,v1:172.21.15.52:6809/3010133776] boot 2024-05-15T13:53:22.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:22 smithi081 ceph-mon[36434]: osdmap e32: 5 total, 5 up, 5 in 2024-05-15T13:53:22.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:22 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T13:53:22.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:22 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:22.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:22 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:22.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:22 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:53:22.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:22 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:53:22.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:22 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:22.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:22 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:53:22.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:22 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:53:22.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:22 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:22.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:22 smithi033 ceph-mon[31650]: purged_snaps scrub starts 2024-05-15T13:53:22.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:22 smithi033 ceph-mon[31650]: purged_snaps scrub ok 2024-05-15T13:53:22.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:22 smithi033 ceph-mon[31650]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-05-15T13:53:22.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:22 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T13:53:22.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:22 smithi033 ceph-mon[31650]: osd.4 [v2:172.21.15.52:6808/3010133776,v1:172.21.15.52:6809/3010133776] boot 2024-05-15T13:53:22.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:22 smithi033 ceph-mon[31650]: osdmap e32: 5 total, 5 up, 5 in 2024-05-15T13:53:22.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:22 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T13:53:22.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:22 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:22.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:22 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:22.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:22 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:53:22.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:22 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:53:22.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:22 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:22.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:22 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:53:22.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:22 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:53:22.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:22 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:23.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.181+0000 7f3349f67700 1 -- 172.21.15.81:0/3434946449 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33441036c0 msgr2=0x7f3344103b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:53:23.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.181+0000 7f3349f67700 1 --2- 172.21.15.81:0/3434946449 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33441036c0 0x7f3344103b20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3338009990 tx=0x7f3338009ca0 comp rx=0 tx=0).stop 2024-05-15T13:53:23.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.181+0000 7f3349f67700 1 -- 172.21.15.81:0/3434946449 shutdown_connections 2024-05-15T13:53:23.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.181+0000 7f3349f67700 1 --2- 172.21.15.81:0/3434946449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3344104060 0x7f334410a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:53:23.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.181+0000 7f3349f67700 1 --2- 172.21.15.81:0/3434946449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33441024c0 0x7f33441028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:53:23.185 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.181+0000 7f3349f67700 1 --2- 172.21.15.81:0/3434946449 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33441036c0 0x7f3344103b20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:53:23.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.181+0000 7f3349f67700 1 -- 172.21.15.81:0/3434946449 >> 172.21.15.81:0/3434946449 conn(0x7f33440fda60 msgr2=0x7f33440ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:53:23.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.181+0000 7f3349f67700 1 -- 172.21.15.81:0/3434946449 shutdown_connections 2024-05-15T13:53:23.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.181+0000 7f3349f67700 1 -- 172.21.15.81:0/3434946449 wait complete. 2024-05-15T13:53:23.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.181+0000 7f3349f67700 1 Processor -- start 2024-05-15T13:53:23.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.182+0000 7f3349f67700 1 -- start start 2024-05-15T13:53:23.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.182+0000 7f3349f67700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33441024c0 0x7f33441065a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:53:23.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.182+0000 7f3349f67700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33441036c0 0x7f3344104bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:53:23.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.182+0000 7f3349f67700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3344104060 0x7f3344105130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:53:23.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.182+0000 7f3349f67700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f334410df90 con 0x7f33441024c0 2024-05-15T13:53:23.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.182+0000 7f3349f67700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f334410de10 con 0x7f3344104060 2024-05-15T13:53:23.186 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.182+0000 7f3349f67700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f334410e110 con 0x7f33441036c0 2024-05-15T13:53:23.187 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.182+0000 7f3342ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33441036c0 0x7f3344104bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:53:23.187 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.182+0000 7f3342ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33441036c0 0x7f3344104bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34162/0 (socket says 172.21.15.81:34162) 2024-05-15T13:53:23.187 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.182+0000 7f3342ffd700 1 -- 172.21.15.81:0/4219372092 learned_addr learned my addr 172.21.15.81:0/4219372092 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-15T13:53:23.187 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.182+0000 7f33437fe700 1 --2- 172.21.15.81:0/4219372092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33441024c0 0x7f33441065a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:53:23.187 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.182+0000 7f3342ffd700 1 -- 172.21.15.81:0/4219372092 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3344104060 msgr2=0x7f3344105130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:53:23.187 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.182+0000 7f3342ffd700 1 --2- 172.21.15.81:0/4219372092 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3344104060 0x7f3344105130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:53:23.187 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.182+0000 7f3342ffd700 1 -- 172.21.15.81:0/4219372092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33441024c0 msgr2=0x7f33441065a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:53:23.187 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.182+0000 7f3342ffd700 1 --2- 172.21.15.81:0/4219372092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33441024c0 0x7f33441065a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:53:23.187 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.182+0000 7f3342ffd700 1 -- 172.21.15.81:0/4219372092 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33441059e0 con 0x7f33441036c0 2024-05-15T13:53:23.187 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.182+0000 7f3342ffd700 1 --2- 172.21.15.81:0/4219372092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33441036c0 0x7f3344104bf0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3338003790 tx=0x7f3338009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:53:23.187 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.182+0000 7f3340ff9700 1 -- 172.21.15.81:0/4219372092 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3338027070 con 0x7f33441036c0 2024-05-15T13:53:23.188 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.183+0000 7f3349f67700 1 -- 172.21.15.81:0/4219372092 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33441a3860 con 0x7f33441036c0 2024-05-15T13:53:23.188 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.183+0000 7f3340ff9700 1 -- 172.21.15.81:0/4219372092 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f333800bd90 con 0x7f33441036c0 2024-05-15T13:53:23.188 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.183+0000 7f3340ff9700 1 -- 172.21.15.81:0/4219372092 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3338022e50 con 0x7f33441036c0 2024-05-15T13:53:23.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.183+0000 7f3349f67700 1 -- 172.21.15.81:0/4219372092 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33441a3cb0 con 0x7f33441036c0 2024-05-15T13:53:23.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.184+0000 7f3349f67700 1 -- 172.21.15.81:0/4219372092 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33441a3f10 con 0x7f33441036c0 2024-05-15T13:53:23.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.185+0000 7f3340ff9700 1 -- 172.21.15.81:0/4219372092 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f333802a070 con 0x7f33441036c0 2024-05-15T13:53:23.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.185+0000 7f3340ff9700 1 --2- 172.21.15.81:0/4219372092 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f332c0b2a20 0x7f332c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:53:23.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.186+0000 7f3340ff9700 1 -- 172.21.15.81:0/4219372092 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4101+0+0 (secure 0 0 0) 0x7f33380e2230 con 0x7f33441036c0 2024-05-15T13:53:23.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.186+0000 7f33437fe700 1 --2- 172.21.15.81:0/4219372092 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f332c0b2a20 0x7f332c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:53:23.189 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.186+0000 7f33437fe700 1 --2- 172.21.15.81:0/4219372092 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f332c0b2a20 0x7f332c0b4ee0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3344103520 tx=0x7f333400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:53:23.191 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.189+0000 7f3340ff9700 1 -- 172.21.15.81:0/4219372092 <== mon.1 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3338013070 con 0x7f33441036c0 2024-05-15T13:53:23.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:23.402+0000 7f3349f67700 1 -- 172.21.15.81:0/4219372092 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f334406c380 con 0x7f332c0b2a20 2024-05-15T13:53:23.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:23 smithi052 ceph-mon[36190]: Detected new or changed devices on smithi052 2024-05-15T13:53:23.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:23 smithi052 ceph-mon[36190]: Adjusting osd_memory_target on smithi052 to 8546M 2024-05-15T13:53:23.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:23 smithi052 ceph-mon[36190]: osdmap e33: 5 total, 5 up, 5 in 2024-05-15T13:53:23.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:23 smithi081 ceph-mon[36434]: Detected new or changed devices on smithi052 2024-05-15T13:53:23.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:23 smithi081 ceph-mon[36434]: Adjusting osd_memory_target on smithi052 to 8546M 2024-05-15T13:53:23.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:23 smithi081 ceph-mon[36434]: osdmap e33: 5 total, 5 up, 5 in 2024-05-15T13:53:23.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:23 smithi033 ceph-mon[31650]: Detected new or changed devices on smithi052 2024-05-15T13:53:23.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:23 smithi033 ceph-mon[31650]: Adjusting osd_memory_target on smithi052 to 8546M 2024-05-15T13:53:23.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:23 smithi033 ceph-mon[31650]: osdmap e33: 5 total, 5 up, 5 in 2024-05-15T13:53:24.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:24 smithi052 ceph-mon[36190]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:24.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:24 smithi052 ceph-mon[36190]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:53:24.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:24 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:53:24.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:24 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:53:24.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:24 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:53:24.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:24 smithi081 ceph-mon[36434]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:24.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:24 smithi081 ceph-mon[36434]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:53:24.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:24 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:53:24.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:24 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:53:24.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:24 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:53:24.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:24 smithi033 ceph-mon[31650]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:24.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:24 smithi033 ceph-mon[31650]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:53:24.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:24 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:53:24.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:24 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:53:24.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:24 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:53:26.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:26 smithi052 ceph-mon[36190]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:26.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:26 smithi052 ceph-mon[36190]: from='client.? 172.21.15.81:0/628851196' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d45ab97-f364-4903-974d-8acc961363d4"}]: dispatch 2024-05-15T13:53:26.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:26 smithi052 ceph-mon[36190]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d45ab97-f364-4903-974d-8acc961363d4"}]: dispatch 2024-05-15T13:53:26.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:26 smithi052 ceph-mon[36190]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d45ab97-f364-4903-974d-8acc961363d4"}]': finished 2024-05-15T13:53:26.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:26 smithi052 ceph-mon[36190]: osdmap e34: 6 total, 5 up, 6 in 2024-05-15T13:53:26.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:26 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T13:53:26.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:26 smithi081 ceph-mon[36434]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:26.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:26 smithi081 ceph-mon[36434]: from='client.? 172.21.15.81:0/628851196' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d45ab97-f364-4903-974d-8acc961363d4"}]: dispatch 2024-05-15T13:53:26.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:26 smithi081 ceph-mon[36434]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d45ab97-f364-4903-974d-8acc961363d4"}]: dispatch 2024-05-15T13:53:26.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:26 smithi081 ceph-mon[36434]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d45ab97-f364-4903-974d-8acc961363d4"}]': finished 2024-05-15T13:53:26.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:26 smithi081 ceph-mon[36434]: osdmap e34: 6 total, 5 up, 6 in 2024-05-15T13:53:26.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:26 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T13:53:26.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:26 smithi033 ceph-mon[31650]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:26.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:26 smithi033 ceph-mon[31650]: from='client.? 172.21.15.81:0/628851196' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d45ab97-f364-4903-974d-8acc961363d4"}]: dispatch 2024-05-15T13:53:26.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:26 smithi033 ceph-mon[31650]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3d45ab97-f364-4903-974d-8acc961363d4"}]: dispatch 2024-05-15T13:53:26.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:26 smithi033 ceph-mon[31650]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3d45ab97-f364-4903-974d-8acc961363d4"}]': finished 2024-05-15T13:53:26.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:26 smithi033 ceph-mon[31650]: osdmap e34: 6 total, 5 up, 6 in 2024-05-15T13:53:26.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:26 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T13:53:27.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:27 smithi052 ceph-mon[36190]: from='client.? 172.21.15.81:0/1935953381' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:53:27.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:27 smithi081 ceph-mon[36434]: from='client.? 172.21.15.81:0/1935953381' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:53:27.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:27 smithi033 ceph-mon[31650]: from='client.? 172.21.15.81:0/1935953381' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:53:28.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:28 smithi052 ceph-mon[36190]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:28.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:28 smithi081 ceph-mon[36434]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:28.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:28 smithi033 ceph-mon[31650]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:30.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:30 smithi052 ceph-mon[36190]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:30.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:30 smithi081 ceph-mon[36434]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:30.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:30 smithi033 ceph-mon[31650]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:32.521 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:32 smithi081 ceph-mon[36434]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:32.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:32 smithi052 ceph-mon[36190]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:32.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:32 smithi033 ceph-mon[31650]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:34.555 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:34 smithi081 ceph-mon[36434]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:34.555 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:34 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-15T13:53:34.555 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:34 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:53:34.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:34 smithi033 ceph-mon[31650]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:34.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:34 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-15T13:53:34.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:34 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:53:34.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:34 smithi052 ceph-mon[36190]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:34.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:34 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-15T13:53:34.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:34 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:53:35.468 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:35 smithi081 ceph-mon[36434]: Deploying daemon osd.5 on smithi081 2024-05-15T13:53:35.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:35 smithi033 ceph-mon[31650]: Deploying daemon osd.5 on smithi081 2024-05-15T13:53:35.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:35 smithi052 ceph-mon[36190]: Deploying daemon osd.5 on smithi081 2024-05-15T13:53:36.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:36 smithi081 ceph-mon[36434]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:36.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:36 smithi033 ceph-mon[31650]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:36.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:36 smithi052 ceph-mon[36190]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:38.457 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:38 smithi081 ceph-mon[36434]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:38.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:38 smithi033 ceph-mon[31650]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:38.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:38 smithi052 ceph-mon[36190]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:40.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:40 smithi052 ceph-mon[36190]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:40.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:40 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:40.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:40 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:40.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:40 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:53:40.342 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:40 smithi081 ceph-mon[36434]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:40.342 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:40 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:40.342 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:40 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:40.342 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:40 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:53:40.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:40 smithi033 ceph-mon[31650]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:40.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:40 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:40.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:40 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:40.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:40 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:53:42.084 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:42 smithi081 ceph-mon[36434]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:42.085 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:42 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:42.085 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:42 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:42.085 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:42 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:53:42.085 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:42 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:53:42.085 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:42 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:42.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:42 smithi052 ceph-mon[36190]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:42.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:42 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:42.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:42 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:42.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:42 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:53:42.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:42 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:53:42.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:42 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:42.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:42 smithi033 ceph-mon[31650]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:42.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:42 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:42.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:42 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:42.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:42 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:53:42.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:42 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:53:42.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:42 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:42.726 INFO:teuthology.orchestra.run.smithi081.stdout:Created osd(s) 5 on host 'smithi081' 2024-05-15T13:53:42.726 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:42.724+0000 7f3340ff9700 1 -- 172.21.15.81:0/4219372092 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f334406c380 con 0x7f332c0b2a20 2024-05-15T13:53:42.728 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:42.726+0000 7f3349f67700 1 -- 172.21.15.81:0/4219372092 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f332c0b2a20 msgr2=0x7f332c0b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:53:42.728 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:42.726+0000 7f3349f67700 1 --2- 172.21.15.81:0/4219372092 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f332c0b2a20 0x7f332c0b4ee0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3344103520 tx=0x7f333400b410 comp rx=0 tx=0).stop 2024-05-15T13:53:42.728 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:42.726+0000 7f3349f67700 1 -- 172.21.15.81:0/4219372092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33441036c0 msgr2=0x7f3344104bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:53:42.728 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:42.726+0000 7f3349f67700 1 --2- 172.21.15.81:0/4219372092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33441036c0 0x7f3344104bf0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3338003790 tx=0x7f3338009580 comp rx=0 tx=0).stop 2024-05-15T13:53:42.728 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:42.726+0000 7f3349f67700 1 -- 172.21.15.81:0/4219372092 shutdown_connections 2024-05-15T13:53:42.728 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:42.726+0000 7f3349f67700 1 --2- 172.21.15.81:0/4219372092 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f332c0b2a20 0x7f332c0b4ee0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:53:42.729 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:42.726+0000 7f3349f67700 1 --2- 172.21.15.81:0/4219372092 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3344104060 0x7f3344105130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:53:42.729 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:42.726+0000 7f3349f67700 1 --2- 172.21.15.81:0/4219372092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33441024c0 0x7f33441065a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:53:42.729 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:42.726+0000 7f3349f67700 1 --2- 172.21.15.81:0/4219372092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33441036c0 0x7f3344104bf0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:53:42.729 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:42.726+0000 7f3349f67700 1 -- 172.21.15.81:0/4219372092 >> 172.21.15.81:0/4219372092 conn(0x7f33440fda60 msgr2=0x7f334406bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:53:42.729 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:42.726+0000 7f3349f67700 1 -- 172.21.15.81:0/4219372092 shutdown_connections 2024-05-15T13:53:42.729 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:53:42.726+0000 7f3349f67700 1 -- 172.21.15.81:0/4219372092 wait complete. 2024-05-15T13:53:43.310 DEBUG:teuthology.orchestra.run.smithi081:osd.5> sudo journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.5.service 2024-05-15T13:53:43.313 INFO:tasks.cephadm:Deploying osd.6 on smithi033 with /dev/vg_nvme/lv_2... 2024-05-15T13:53:43.313 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-15T13:53:43.573 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:53:43.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:43 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:43.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:43 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:43.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:43 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:53:43.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:43 smithi081 ceph-mon[36434]: from='osd.5 [v2:172.21.15.81:6808/2132616297,v1:172.21.15.81:6809/2132616297]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-15T13:53:43.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:43 smithi081 ceph-mon[36434]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-15T13:53:43.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:43 smithi081 ceph-mon[36434]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:43.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:43 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:43.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:43 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:43.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:43 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:53:43.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:43 smithi033 ceph-mon[31650]: from='osd.5 [v2:172.21.15.81:6808/2132616297,v1:172.21.15.81:6809/2132616297]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-15T13:53:43.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:43 smithi033 ceph-mon[31650]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-15T13:53:43.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:43 smithi033 ceph-mon[31650]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:44.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:43 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:44.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:43 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:44.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:43 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:53:44.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:43 smithi052 ceph-mon[36190]: from='osd.5 [v2:172.21.15.81:6808/2132616297,v1:172.21.15.81:6809/2132616297]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-15T13:53:44.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:43 smithi052 ceph-mon[36190]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-15T13:53:44.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:43 smithi052 ceph-mon[36190]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:44.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:44 smithi081 ceph-mon[36434]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-15T13:53:44.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:44 smithi081 ceph-mon[36434]: osdmap e35: 6 total, 5 up, 6 in 2024-05-15T13:53:44.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:44 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T13:53:44.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:44 smithi081 ceph-mon[36434]: from='osd.5 [v2:172.21.15.81:6808/2132616297,v1:172.21.15.81:6809/2132616297]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-05-15T13:53:44.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:44 smithi081 ceph-mon[36434]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-05-15T13:53:44.969 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:44 smithi033 ceph-mon[31650]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-15T13:53:44.969 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:44 smithi033 ceph-mon[31650]: osdmap e35: 6 total, 5 up, 6 in 2024-05-15T13:53:44.969 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:44 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T13:53:44.969 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:44 smithi033 ceph-mon[31650]: from='osd.5 [v2:172.21.15.81:6808/2132616297,v1:172.21.15.81:6809/2132616297]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-05-15T13:53:44.969 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:44 smithi033 ceph-mon[31650]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-05-15T13:53:45.090 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:44 smithi052 ceph-mon[36190]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-15T13:53:45.090 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:44 smithi052 ceph-mon[36190]: osdmap e35: 6 total, 5 up, 6 in 2024-05-15T13:53:45.090 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:44 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T13:53:45.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:44 smithi052 ceph-mon[36190]: from='osd.5 [v2:172.21.15.81:6808/2132616297,v1:172.21.15.81:6809/2132616297]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-05-15T13:53:45.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:44 smithi052 ceph-mon[36190]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-05-15T13:53:45.732 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-15T13:53:45.733 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-15T13:53:45.733 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-05-15T13:53:45.733 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-05-15T13:53:45.733 INFO:teuthology.orchestra.run.smithi033.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0119506 s, 877 MB/s 2024-05-15T13:53:45.733 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-05-15T13:53:45.797 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:45 smithi081 ceph-mon[36434]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-05-15T13:53:45.797 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:45 smithi081 ceph-mon[36434]: osdmap e36: 6 total, 5 up, 6 in 2024-05-15T13:53:45.797 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:45 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T13:53:45.797 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:45 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T13:53:45.797 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:45 smithi081 ceph-mon[36434]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:45.797 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:45 smithi081 ceph-mon[36434]: osdmap e37: 6 total, 5 up, 6 in 2024-05-15T13:53:45.797 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:45 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T13:53:45.797 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:45 smithi081 ceph-mon[36434]: from='osd.5 ' entity='osd.5' 2024-05-15T13:53:45.798 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 13:53:45 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5[48018]: 2024-05-15T13:53:45.646+0000 7f5ee883a700 -1 osd.5 0 waiting for initial osdmap 2024-05-15T13:53:45.798 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 13:53:45 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5[48018]: 2024-05-15T13:53:45.794+0000 7f5ee0625700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-15T13:53:45.798 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:45 smithi033 ceph-mon[31650]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-05-15T13:53:45.798 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:45 smithi033 ceph-mon[31650]: osdmap e36: 6 total, 5 up, 6 in 2024-05-15T13:53:45.798 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:45 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T13:53:45.798 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:45 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T13:53:45.798 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:45 smithi033 ceph-mon[31650]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:45.798 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:45 smithi033 ceph-mon[31650]: osdmap e37: 6 total, 5 up, 6 in 2024-05-15T13:53:45.799 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:45 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T13:53:45.799 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:45 smithi033 ceph-mon[31650]: from='osd.5 ' entity='osd.5' 2024-05-15T13:53:46.090 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:45 smithi052 ceph-mon[36190]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-05-15T13:53:46.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:45 smithi052 ceph-mon[36190]: osdmap e36: 6 total, 5 up, 6 in 2024-05-15T13:53:46.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:45 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T13:53:46.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:45 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T13:53:46.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:45 smithi052 ceph-mon[36190]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-05-15T13:53:46.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:45 smithi052 ceph-mon[36190]: osdmap e37: 6 total, 5 up, 6 in 2024-05-15T13:53:46.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:45 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T13:53:46.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:45 smithi052 ceph-mon[36190]: from='osd.5 ' entity='osd.5' 2024-05-15T13:53:46.367 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_2 2024-05-15T13:53:46.617 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:53:46.992 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:46 smithi052 ceph-mon[36190]: purged_snaps scrub starts 2024-05-15T13:53:46.992 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:46 smithi052 ceph-mon[36190]: purged_snaps scrub ok 2024-05-15T13:53:46.992 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:46 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T13:53:46.992 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:46 smithi052 ceph-mon[36190]: Detected new or changed devices on smithi081 2024-05-15T13:53:46.993 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:46 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:46.993 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:46 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:46.993 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:46 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:53:46.993 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:46 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:53:46.993 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:46 smithi052 ceph-mon[36190]: Adjusting osd_memory_target on smithi081 to 8544M 2024-05-15T13:53:46.993 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:46 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:46.993 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:46 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:53:46.993 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:46 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:53:46.993 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:46 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:46.993 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:46 smithi052 ceph-mon[36190]: osd.5 [v2:172.21.15.81:6808/2132616297,v1:172.21.15.81:6809/2132616297] boot 2024-05-15T13:53:46.993 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:46 smithi052 ceph-mon[36190]: osdmap e38: 6 total, 6 up, 6 in 2024-05-15T13:53:46.993 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:46 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T13:53:47.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:46 smithi033 ceph-mon[31650]: purged_snaps scrub starts 2024-05-15T13:53:47.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:46 smithi033 ceph-mon[31650]: purged_snaps scrub ok 2024-05-15T13:53:47.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:46 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T13:53:47.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:46 smithi033 ceph-mon[31650]: Detected new or changed devices on smithi081 2024-05-15T13:53:47.067 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:46 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:47.067 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:46 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:47.067 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:46 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:53:47.067 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:46 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:53:47.067 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:46 smithi033 ceph-mon[31650]: Adjusting osd_memory_target on smithi081 to 8544M 2024-05-15T13:53:47.067 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:46 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:47.067 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:46 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:53:47.067 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:46 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:53:47.067 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:46 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:47.067 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:46 smithi033 ceph-mon[31650]: osd.5 [v2:172.21.15.81:6808/2132616297,v1:172.21.15.81:6809/2132616297] boot 2024-05-15T13:53:47.067 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:46 smithi033 ceph-mon[31650]: osdmap e38: 6 total, 6 up, 6 in 2024-05-15T13:53:47.067 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:46 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T13:53:47.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:46 smithi081 ceph-mon[36434]: purged_snaps scrub starts 2024-05-15T13:53:47.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:46 smithi081 ceph-mon[36434]: purged_snaps scrub ok 2024-05-15T13:53:47.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:46 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T13:53:47.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:46 smithi081 ceph-mon[36434]: Detected new or changed devices on smithi081 2024-05-15T13:53:47.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:46 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:47.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:46 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:47.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:46 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:53:47.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:46 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:53:47.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:46 smithi081 ceph-mon[36434]: Adjusting osd_memory_target on smithi081 to 8544M 2024-05-15T13:53:47.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:46 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:47.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:46 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:53:47.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:46 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:53:47.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:46 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:53:47.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:46 smithi081 ceph-mon[36434]: osd.5 [v2:172.21.15.81:6808/2132616297,v1:172.21.15.81:6809/2132616297] boot 2024-05-15T13:53:47.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:46 smithi081 ceph-mon[36434]: osdmap e38: 6 total, 6 up, 6 in 2024-05-15T13:53:47.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:46 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T13:53:48.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.237+0000 7f1260d8d700 1 -- 172.21.15.33:0/3390710989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f125c1047a0 msgr2=0x7f125c104c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:53:48.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.237+0000 7f1260d8d700 1 --2- 172.21.15.33:0/3390710989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f125c1047a0 0x7f125c104c00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f12500099c0 tx=0x7f1250009cd0 comp rx=0 tx=0).stop 2024-05-15T13:53:48.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.237+0000 7f1260d8d700 1 -- 172.21.15.33:0/3390710989 shutdown_connections 2024-05-15T13:53:48.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.237+0000 7f1260d8d700 1 --2- 172.21.15.33:0/3390710989 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f125c105140 0x7f125c10b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:53:48.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.237+0000 7f1260d8d700 1 --2- 172.21.15.33:0/3390710989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f125c1035a0 0x7f125c1039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:53:48.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.237+0000 7f1260d8d700 1 --2- 172.21.15.33:0/3390710989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f125c1047a0 0x7f125c104c00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:53:48.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.237+0000 7f1260d8d700 1 -- 172.21.15.33:0/3390710989 >> 172.21.15.33:0/3390710989 conn(0x7f125c0feb40 msgr2=0x7f125c100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:53:48.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.237+0000 7f1260d8d700 1 -- 172.21.15.33:0/3390710989 shutdown_connections 2024-05-15T13:53:48.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.237+0000 7f1260d8d700 1 -- 172.21.15.33:0/3390710989 wait complete. 2024-05-15T13:53:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.238+0000 7f1260d8d700 1 Processor -- start 2024-05-15T13:53:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.238+0000 7f1260d8d700 1 -- start start 2024-05-15T13:53:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.238+0000 7f1260d8d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f125c1035a0 0x7f125c1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:53:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.238+0000 7f1260d8d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f125c1047a0 0x7f125c199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:53:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.238+0000 7f1260d8d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f125c105140 0x7f125c1a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:53:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.238+0000 7f1260d8d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f125c10f040 con 0x7f125c1047a0 2024-05-15T13:53:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.238+0000 7f1260d8d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f125c10eec0 con 0x7f125c1035a0 2024-05-15T13:53:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.238+0000 7f1260d8d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f125c10f1c0 con 0x7f125c105140 2024-05-15T13:53:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.239+0000 7f1259d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f125c1047a0 0x7f125c199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:53:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.239+0000 7f125a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f125c1035a0 0x7f125c1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:53:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.239+0000 7f1259d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f125c1047a0 0x7f125c199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38326/0 (socket says 172.21.15.33:38326) 2024-05-15T13:53:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.239+0000 7f1259d9b700 1 -- 172.21.15.33:0/52285879 learned_addr learned my addr 172.21.15.33:0/52285879 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:53:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.239+0000 7f125ad9d700 1 --2- 172.21.15.33:0/52285879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f125c105140 0x7f125c1a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:53:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.239+0000 7f1259d9b700 1 -- 172.21.15.33:0/52285879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f125c105140 msgr2=0x7f125c1a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:53:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.239+0000 7f1259d9b700 1 --2- 172.21.15.33:0/52285879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f125c105140 0x7f125c1a0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:53:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.239+0000 7f1259d9b700 1 -- 172.21.15.33:0/52285879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f125c1035a0 msgr2=0x7f125c1996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:53:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.239+0000 7f1259d9b700 1 --2- 172.21.15.33:0/52285879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f125c1035a0 0x7f125c1996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:53:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.239+0000 7f1259d9b700 1 -- 172.21.15.33:0/52285879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f125c1a1300 con 0x7f125c1047a0 2024-05-15T13:53:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.239+0000 7f125a59c700 1 --2- 172.21.15.33:0/52285879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f125c1035a0 0x7f125c1996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T13:53:48.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.239+0000 7f1259d9b700 1 --2- 172.21.15.33:0/52285879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f125c1047a0 0x7f125c199be0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f125000b580 tx=0x7f125000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:53:48.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.239+0000 7f124b7fe700 1 -- 172.21.15.33:0/52285879 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1250027070 con 0x7f125c1047a0 2024-05-15T13:53:48.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.239+0000 7f1260d8d700 1 -- 172.21.15.33:0/52285879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f125c1a1590 con 0x7f125c1047a0 2024-05-15T13:53:48.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.239+0000 7f124b7fe700 1 -- 172.21.15.33:0/52285879 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1250022920 con 0x7f125c1047a0 2024-05-15T13:53:48.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.239+0000 7f124b7fe700 1 -- 172.21.15.33:0/52285879 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1250009e60 con 0x7f125c1047a0 2024-05-15T13:53:48.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.239+0000 7f1260d8d700 1 -- 172.21.15.33:0/52285879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f125c1a1a40 con 0x7f125c1047a0 2024-05-15T13:53:48.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.240+0000 7f1260d8d700 1 -- 172.21.15.33:0/52285879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f125c06a230 con 0x7f125c1047a0 2024-05-15T13:53:48.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.243+0000 7f124b7fe700 1 -- 172.21.15.33:0/52285879 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f125002a070 con 0x7f125c1047a0 2024-05-15T13:53:48.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.243+0000 7f124b7fe700 1 --2- 172.21.15.33:0/52285879 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f12440b2880 0x7f12440b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:53:48.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.243+0000 7f125a59c700 1 --2- 172.21.15.33:0/52285879 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f12440b2880 0x7f12440b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:53:48.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.243+0000 7f124b7fe700 1 -- 172.21.15.33:0/52285879 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4533+0+0 (secure 0 0 0) 0x7f12500e2330 con 0x7f125c1047a0 2024-05-15T13:53:48.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.244+0000 7f125a59c700 1 --2- 172.21.15.33:0/52285879 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f12440b2880 0x7f12440b4d40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f125c104600 tx=0x7f124c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:53:48.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.247+0000 7f124b7fe700 1 -- 172.21.15.33:0/52285879 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1250013070 con 0x7f125c1047a0 2024-05-15T13:53:48.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:53:48.472+0000 7f1260d8d700 1 -- 172.21.15.33:0/52285879 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f125c063980 con 0x7f12440b2880 2024-05-15T13:53:48.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:48 smithi052 ceph-mon[36190]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-15T13:53:48.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:48 smithi052 ceph-mon[36190]: osdmap e39: 6 total, 6 up, 6 in 2024-05-15T13:53:48.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:48 smithi081 ceph-mon[36434]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-15T13:53:48.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:48 smithi081 ceph-mon[36434]: osdmap e39: 6 total, 6 up, 6 in 2024-05-15T13:53:48.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:48 smithi033 ceph-mon[31650]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-15T13:53:48.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:48 smithi033 ceph-mon[31650]: osdmap e39: 6 total, 6 up, 6 in 2024-05-15T13:53:49.418 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:49 smithi033 ceph-mon[31650]: osdmap e40: 6 total, 6 up, 6 in 2024-05-15T13:53:49.419 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:49 smithi033 ceph-mon[31650]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:53:49.419 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:49 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:53:49.419 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:49 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:53:49.419 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:49 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:53:49.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:49 smithi052 ceph-mon[36190]: osdmap e40: 6 total, 6 up, 6 in 2024-05-15T13:53:49.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:49 smithi052 ceph-mon[36190]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:53:49.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:49 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:53:49.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:49 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:53:49.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:49 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:53:49.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:49 smithi081 ceph-mon[36434]: osdmap e40: 6 total, 6 up, 6 in 2024-05-15T13:53:49.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:49 smithi081 ceph-mon[36434]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:53:49.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:49 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:53:49.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:49 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:53:49.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:49 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:53:50.584 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:50 smithi033 ceph-mon[31650]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-15T13:53:50.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:50 smithi052 ceph-mon[36190]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-15T13:53:50.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:50 smithi081 ceph-mon[36434]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-15T13:53:51.593 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:51 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/2027045576' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3be524c5-dcf9-4742-b896-ea9a44a6b391"}]: dispatch 2024-05-15T13:53:51.593 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:51 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/2027045576' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3be524c5-dcf9-4742-b896-ea9a44a6b391"}]': finished 2024-05-15T13:53:51.593 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:51 smithi052 ceph-mon[36190]: osdmap e41: 7 total, 6 up, 7 in 2024-05-15T13:53:51.593 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:51 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-15T13:53:51.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:51 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/2027045576' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3be524c5-dcf9-4742-b896-ea9a44a6b391"}]: dispatch 2024-05-15T13:53:51.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:51 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/2027045576' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3be524c5-dcf9-4742-b896-ea9a44a6b391"}]': finished 2024-05-15T13:53:51.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:51 smithi081 ceph-mon[36434]: osdmap e41: 7 total, 6 up, 7 in 2024-05-15T13:53:51.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:51 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-15T13:53:51.654 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:51 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/2027045576' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3be524c5-dcf9-4742-b896-ea9a44a6b391"}]: dispatch 2024-05-15T13:53:51.655 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:51 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/2027045576' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3be524c5-dcf9-4742-b896-ea9a44a6b391"}]': finished 2024-05-15T13:53:51.655 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:51 smithi033 ceph-mon[31650]: osdmap e41: 7 total, 6 up, 7 in 2024-05-15T13:53:51.655 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:51 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-15T13:53:52.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:52 smithi033 ceph-mon[31650]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-15T13:53:52.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:52 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/4060353040' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:53:52.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:52 smithi052 ceph-mon[36190]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-15T13:53:52.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:52 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/4060353040' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:53:52.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:52 smithi081 ceph-mon[36434]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-15T13:53:52.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:52 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/4060353040' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:53:54.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:54 smithi033 ceph-mon[31650]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-15T13:53:54.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:54 smithi052 ceph-mon[36190]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-15T13:53:54.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:54 smithi081 ceph-mon[36434]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-15T13:53:56.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:56 smithi033 ceph-mon[31650]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-05-15T13:53:56.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:56 smithi052 ceph-mon[36190]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-05-15T13:53:56.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:56 smithi081 ceph-mon[36434]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-05-15T13:53:58.611 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:58 smithi033 ceph-mon[31650]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-15T13:53:58.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:58 smithi052 ceph-mon[36190]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-15T13:53:58.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:58 smithi081 ceph-mon[36434]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-15T13:53:59.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:59 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-15T13:53:59.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:53:59 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:53:59.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:59 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-15T13:53:59.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:53:59 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:53:59.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:59 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-15T13:53:59.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:53:59 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:54:00.511 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:00 smithi033 ceph-mon[31650]: Deploying daemon osd.6 on smithi033 2024-05-15T13:54:00.511 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:00 smithi033 ceph-mon[31650]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-15T13:54:00.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:00 smithi052 ceph-mon[36190]: Deploying daemon osd.6 on smithi033 2024-05-15T13:54:00.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:00 smithi052 ceph-mon[36190]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-15T13:54:00.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:00 smithi081 ceph-mon[36434]: Deploying daemon osd.6 on smithi033 2024-05-15T13:54:00.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:00 smithi081 ceph-mon[36434]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-15T13:54:02.659 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:02 smithi033 ceph-mon[31650]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-15T13:54:02.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:02 smithi052 ceph-mon[36190]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-15T13:54:02.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:02 smithi081 ceph-mon[36434]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-15T13:54:04.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:04 smithi033 ceph-mon[31650]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-15T13:54:04.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:04 smithi052 ceph-mon[36190]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-15T13:54:04.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:04 smithi081 ceph-mon[36434]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-15T13:54:05.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:05 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:05.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:05 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:54:05.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:05 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:05.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:05 smithi052 ceph-mon[36190]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-15T13:54:05.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:05 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:05.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:05 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:54:05.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:05 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:05.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:05 smithi081 ceph-mon[36434]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-15T13:54:05.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:05 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:05.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:05 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:54:05.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:05 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:05.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:05 smithi033 ceph-mon[31650]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-15T13:54:08.188 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 6 on host 'smithi033' 2024-05-15T13:54:08.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:54:08.185+0000 7f124b7fe700 1 -- 172.21.15.33:0/52285879 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f125c063980 con 0x7f12440b2880 2024-05-15T13:54:08.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:54:08.188+0000 7f1260d8d700 1 -- 172.21.15.33:0/52285879 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f12440b2880 msgr2=0x7f12440b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:54:08.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:54:08.188+0000 7f1260d8d700 1 --2- 172.21.15.33:0/52285879 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f12440b2880 0x7f12440b4d40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f125c104600 tx=0x7f124c009260 comp rx=0 tx=0).stop 2024-05-15T13:54:08.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:54:08.188+0000 7f1260d8d700 1 -- 172.21.15.33:0/52285879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f125c1047a0 msgr2=0x7f125c199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:54:08.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:54:08.188+0000 7f1260d8d700 1 --2- 172.21.15.33:0/52285879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f125c1047a0 0x7f125c199be0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f125000b580 tx=0x7f125000fa00 comp rx=0 tx=0).stop 2024-05-15T13:54:08.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:54:08.188+0000 7f1260d8d700 1 -- 172.21.15.33:0/52285879 shutdown_connections 2024-05-15T13:54:08.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:54:08.188+0000 7f1260d8d700 1 --2- 172.21.15.33:0/52285879 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f12440b2880 0x7f12440b4d40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:54:08.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:54:08.188+0000 7f1260d8d700 1 --2- 172.21.15.33:0/52285879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f125c105140 0x7f125c1a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:54:08.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:54:08.188+0000 7f1260d8d700 1 --2- 172.21.15.33:0/52285879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f125c1035a0 0x7f125c1996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:54:08.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:54:08.188+0000 7f1260d8d700 1 --2- 172.21.15.33:0/52285879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f125c1047a0 0x7f125c199be0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:54:08.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:54:08.188+0000 7f1260d8d700 1 -- 172.21.15.33:0/52285879 >> 172.21.15.33:0/52285879 conn(0x7f125c0feb40 msgr2=0x7f125c107b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:54:08.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:54:08.188+0000 7f1260d8d700 1 -- 172.21.15.33:0/52285879 shutdown_connections 2024-05-15T13:54:08.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:54:08.188+0000 7f1260d8d700 1 -- 172.21.15.33:0/52285879 wait complete. 2024-05-15T13:54:08.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:07 smithi052 ceph-mon[36190]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-15T13:54:08.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:07 smithi081 ceph-mon[36434]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-15T13:54:08.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:07 smithi033 ceph-mon[31650]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-15T13:54:08.885 DEBUG:teuthology.orchestra.run.smithi033:osd.6> sudo journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.6.service 2024-05-15T13:54:08.888 INFO:tasks.cephadm:Deploying osd.7 on smithi052 with /dev/vg_nvme/lv_2... 2024-05-15T13:54:08.888 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-15T13:54:09.146 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.b/config 2024-05-15T13:54:09.268 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:09 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:09.269 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:09 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:09.269 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:09 smithi052 ceph-mon[36190]: from='osd.6 [v2:172.21.15.33:6818/1260691801,v1:172.21.15.33:6819/1260691801]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-15T13:54:09.269 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:09 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:09.269 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:09 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:09.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:09 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:09.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:09 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:09.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:09 smithi033 ceph-mon[31650]: from='osd.6 [v2:172.21.15.33:6818/1260691801,v1:172.21.15.33:6819/1260691801]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-15T13:54:09.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:09 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:09.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:09 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:09.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:09 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:09.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:09 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:09.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:09 smithi081 ceph-mon[36434]: from='osd.6 [v2:172.21.15.33:6818/1260691801,v1:172.21.15.33:6819/1260691801]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-15T13:54:09.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:09 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:09.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:09 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:10.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:10 smithi033 ceph-mon[31650]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-15T13:54:10.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:10 smithi033 ceph-mon[31650]: from='osd.6 [v2:172.21.15.33:6818/1260691801,v1:172.21.15.33:6819/1260691801]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-15T13:54:10.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:10 smithi033 ceph-mon[31650]: osdmap e42: 7 total, 6 up, 7 in 2024-05-15T13:54:10.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:10 smithi033 ceph-mon[31650]: from='osd.6 [v2:172.21.15.33:6818/1260691801,v1:172.21.15.33:6819/1260691801]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-15T13:54:10.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:10 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-15T13:54:10.485 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:10 smithi052 ceph-mon[36190]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-15T13:54:10.485 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:10 smithi052 ceph-mon[36190]: from='osd.6 [v2:172.21.15.33:6818/1260691801,v1:172.21.15.33:6819/1260691801]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-15T13:54:10.485 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:10 smithi052 ceph-mon[36190]: osdmap e42: 7 total, 6 up, 7 in 2024-05-15T13:54:10.485 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:10 smithi052 ceph-mon[36190]: from='osd.6 [v2:172.21.15.33:6818/1260691801,v1:172.21.15.33:6819/1260691801]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-15T13:54:10.485 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:10 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-15T13:54:10.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:10 smithi081 ceph-mon[36434]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-15T13:54:10.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:10 smithi081 ceph-mon[36434]: from='osd.6 [v2:172.21.15.33:6818/1260691801,v1:172.21.15.33:6819/1260691801]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-15T13:54:10.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:10 smithi081 ceph-mon[36434]: osdmap e42: 7 total, 6 up, 7 in 2024-05-15T13:54:10.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:10 smithi081 ceph-mon[36434]: from='osd.6 [v2:172.21.15.33:6818/1260691801,v1:172.21.15.33:6819/1260691801]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-15T13:54:10.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:10 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-15T13:54:11.160 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-15T13:54:11.160 INFO:teuthology.orchestra.run.smithi052.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-15T13:54:11.160 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10+0 records in 2024-05-15T13:54:11.160 INFO:teuthology.orchestra.run.smithi052.stderr:10+0 records out 2024-05-15T13:54:11.160 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00843297 s, 1.2 GB/s 2024-05-15T13:54:11.160 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping successful for: 2024-05-15T13:54:11.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:11 smithi033 ceph-mon[31650]: from='osd.6 [v2:172.21.15.33:6818/1260691801,v1:172.21.15.33:6819/1260691801]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-15T13:54:11.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:11 smithi033 ceph-mon[31650]: osdmap e43: 7 total, 6 up, 7 in 2024-05-15T13:54:11.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:11 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-15T13:54:11.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:11 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-15T13:54:11.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:11 smithi033 ceph-mon[31650]: from='osd.6 [v2:172.21.15.33:6818/1260691801,v1:172.21.15.33:6819/1260691801]' entity='osd.6' 2024-05-15T13:54:11.442 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 13:54:11 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T13:54:11.099+0000 7f523ffc3700 -1 osd.6 0 waiting for initial osdmap 2024-05-15T13:54:11.442 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 13:54:11 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T13:54:11.206+0000 7f52375ad700 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-15T13:54:11.461 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:11 smithi052 ceph-mon[36190]: from='osd.6 [v2:172.21.15.33:6818/1260691801,v1:172.21.15.33:6819/1260691801]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-15T13:54:11.461 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:11 smithi052 ceph-mon[36190]: osdmap e43: 7 total, 6 up, 7 in 2024-05-15T13:54:11.461 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:11 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-15T13:54:11.461 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:11 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-15T13:54:11.461 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:11 smithi052 ceph-mon[36190]: from='osd.6 [v2:172.21.15.33:6818/1260691801,v1:172.21.15.33:6819/1260691801]' entity='osd.6' 2024-05-15T13:54:11.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:11 smithi081 ceph-mon[36434]: from='osd.6 [v2:172.21.15.33:6818/1260691801,v1:172.21.15.33:6819/1260691801]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-15T13:54:11.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:11 smithi081 ceph-mon[36434]: osdmap e43: 7 total, 6 up, 7 in 2024-05-15T13:54:11.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:11 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-15T13:54:11.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:11 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-15T13:54:11.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:11 smithi081 ceph-mon[36434]: from='osd.6 [v2:172.21.15.33:6818/1260691801,v1:172.21.15.33:6819/1260691801]' entity='osd.6' 2024-05-15T13:54:11.756 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi052:vg_nvme/lv_2 2024-05-15T13:54:12.010 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.b/config 2024-05-15T13:54:12.405 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:12 smithi052 ceph-mon[36190]: purged_snaps scrub starts 2024-05-15T13:54:12.405 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:12 smithi052 ceph-mon[36190]: purged_snaps scrub ok 2024-05-15T13:54:12.406 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:12 smithi052 ceph-mon[36190]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-15T13:54:12.406 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:12 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-15T13:54:12.406 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:12 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-15T13:54:12.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:12 smithi033 ceph-mon[31650]: purged_snaps scrub starts 2024-05-15T13:54:12.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:12 smithi033 ceph-mon[31650]: purged_snaps scrub ok 2024-05-15T13:54:12.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:12 smithi033 ceph-mon[31650]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-15T13:54:12.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:12 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-15T13:54:12.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:12 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-15T13:54:12.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:12 smithi081 ceph-mon[36434]: purged_snaps scrub starts 2024-05-15T13:54:12.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:12 smithi081 ceph-mon[36434]: purged_snaps scrub ok 2024-05-15T13:54:12.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:12 smithi081 ceph-mon[36434]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-15T13:54:12.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:12 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-15T13:54:12.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:12 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-15T13:54:13.396 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:13 smithi052 ceph-mon[36190]: osd.6 [v2:172.21.15.33:6818/1260691801,v1:172.21.15.33:6819/1260691801] boot 2024-05-15T13:54:13.397 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:13 smithi052 ceph-mon[36190]: osdmap e44: 7 total, 7 up, 7 in 2024-05-15T13:54:13.397 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:13 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-15T13:54:13.397 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:13 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:13.397 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:13 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:13.397 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:13 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:54:13.397 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:13 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:54:13.397 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:13 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:54:13.397 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:13 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:13.397 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:13 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:54:13.397 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:13 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:54:13.397 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:13 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:13.397 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:13 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:54:13.397 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:13 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:54:13.397 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:13 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:54:13.398 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:13 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:13.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:13 smithi033 ceph-mon[31650]: osd.6 [v2:172.21.15.33:6818/1260691801,v1:172.21.15.33:6819/1260691801] boot 2024-05-15T13:54:13.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:13 smithi033 ceph-mon[31650]: osdmap e44: 7 total, 7 up, 7 in 2024-05-15T13:54:13.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:13 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-15T13:54:13.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:13 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:13.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:13 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:13.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:13 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:54:13.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:13 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:54:13.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:13 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:54:13.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:13 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:13.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:13 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:54:13.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:13 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:54:13.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:13 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:13.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:13 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:54:13.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:13 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:54:13.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:13 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:54:13.461 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:13 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:13.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.602+0000 7f875ffff700 1 -- 172.21.15.52:0/2535701954 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8760103f60 msgr2=0x7f876010a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:54:13.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.602+0000 7f875ffff700 1 --2- 172.21.15.52:0/2535701954 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8760103f60 0x7f876010a810 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f875800b0d0 tx=0x7f875800b3e0 comp rx=0 tx=0).stop 2024-05-15T13:54:13.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.602+0000 7f875ffff700 1 -- 172.21.15.52:0/2535701954 shutdown_connections 2024-05-15T13:54:13.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.602+0000 7f875ffff700 1 --2- 172.21.15.52:0/2535701954 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8760103f60 0x7f876010a810 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:54:13.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.602+0000 7f875ffff700 1 --2- 172.21.15.52:0/2535701954 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87601023c0 0x7f87601027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:54:13.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.602+0000 7f875ffff700 1 --2- 172.21.15.52:0/2535701954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87601035c0 0x7f8760103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:54:13.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.602+0000 7f875ffff700 1 -- 172.21.15.52:0/2535701954 >> 172.21.15.52:0/2535701954 conn(0x7f87600fd960 msgr2=0x7f87600ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:54:13.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.602+0000 7f875ffff700 1 -- 172.21.15.52:0/2535701954 shutdown_connections 2024-05-15T13:54:13.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.602+0000 7f875ffff700 1 -- 172.21.15.52:0/2535701954 wait complete. 2024-05-15T13:54:13.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.602+0000 7f875ffff700 1 Processor -- start 2024-05-15T13:54:13.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.603+0000 7f875ffff700 1 -- start start 2024-05-15T13:54:13.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.603+0000 7f875ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87601023c0 0x7f876019b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:54:13.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.603+0000 7f875ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87601035c0 0x7f876019bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:54:13.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.603+0000 7f875ffff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8760103f60 0x7f87601a2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:54:13.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.603+0000 7f875ffff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f876010de40 con 0x7f87601023c0 2024-05-15T13:54:13.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.603+0000 7f875ffff700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f876010dcc0 con 0x7f87601035c0 2024-05-15T13:54:13.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.603+0000 7f875ffff700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f876010dfc0 con 0x7f8760103f60 2024-05-15T13:54:13.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.603+0000 7f875e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87601035c0 0x7f876019bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:54:13.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.603+0000 7f875e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87601035c0 0x7f876019bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45804/0 (socket says 172.21.15.52:45804) 2024-05-15T13:54:13.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.603+0000 7f875e7fc700 1 -- 172.21.15.52:0/3454582850 learned_addr learned my addr 172.21.15.52:0/3454582850 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-15T13:54:13.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.603+0000 7f875effd700 1 --2- 172.21.15.52:0/3454582850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87601023c0 0x7f876019b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:54:13.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.603+0000 7f875f7fe700 1 --2- 172.21.15.52:0/3454582850 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8760103f60 0x7f87601a2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:54:13.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.603+0000 7f875e7fc700 1 -- 172.21.15.52:0/3454582850 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8760103f60 msgr2=0x7f87601a2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:54:13.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.603+0000 7f875e7fc700 1 --2- 172.21.15.52:0/3454582850 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8760103f60 0x7f87601a2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:54:13.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.603+0000 7f875e7fc700 1 -- 172.21.15.52:0/3454582850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87601023c0 msgr2=0x7f876019b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:54:13.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.603+0000 7f875e7fc700 1 --2- 172.21.15.52:0/3454582850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87601023c0 0x7f876019b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:54:13.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.603+0000 7f875e7fc700 1 -- 172.21.15.52:0/3454582850 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87601a33b0 con 0x7f87601035c0 2024-05-15T13:54:13.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.603+0000 7f875e7fc700 1 --2- 172.21.15.52:0/3454582850 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87601035c0 0x7f876019bc90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f875400d950 tx=0x7f875400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:54:13.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.604+0000 7f8747fff700 1 -- 172.21.15.52:0/3454582850 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8754016070 con 0x7f87601035c0 2024-05-15T13:54:13.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.604+0000 7f8747fff700 1 -- 172.21.15.52:0/3454582850 <== mon.2 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8754009df0 con 0x7f87601035c0 2024-05-15T13:54:13.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.604+0000 7f8747fff700 1 -- 172.21.15.52:0/3454582850 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875400f620 con 0x7f87601035c0 2024-05-15T13:54:13.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.604+0000 7f875ffff700 1 -- 172.21.15.52:0/3454582850 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87601a36a0 con 0x7f87601035c0 2024-05-15T13:54:13.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.604+0000 7f875ffff700 1 -- 172.21.15.52:0/3454582850 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87601a3c30 con 0x7f87601035c0 2024-05-15T13:54:13.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.605+0000 7f875ffff700 1 -- 172.21.15.52:0/3454582850 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f876006a230 con 0x7f87601035c0 2024-05-15T13:54:13.610 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:13 smithi081 ceph-mon[36434]: osd.6 [v2:172.21.15.33:6818/1260691801,v1:172.21.15.33:6819/1260691801] boot 2024-05-15T13:54:13.611 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:13 smithi081 ceph-mon[36434]: osdmap e44: 7 total, 7 up, 7 in 2024-05-15T13:54:13.611 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:13 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-15T13:54:13.611 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:13 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:13.611 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:13 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:13.611 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:13 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:54:13.611 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:13 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:54:13.611 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:13 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:54:13.611 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:13 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:13.611 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:13 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:54:13.611 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:13 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:54:13.611 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:13 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:13.611 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:13 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:54:13.611 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:13 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:54:13.612 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:13 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:54:13.612 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:13 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:13.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.606+0000 7f8747fff700 1 -- 172.21.15.52:0/3454582850 <== mon.2 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8754019070 con 0x7f87601035c0 2024-05-15T13:54:13.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.607+0000 7f8747fff700 1 --2- 172.21.15.52:0/3454582850 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f87480b2900 0x7f87480b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:54:13.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.607+0000 7f8747fff700 1 -- 172.21.15.52:0/3454582850 <== mon.2 v2:172.21.15.52:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 4965+0+0 (secure 0 0 0) 0x7f87540e0f60 con 0x7f87601035c0 2024-05-15T13:54:13.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.607+0000 7f875effd700 1 --2- 172.21.15.52:0/3454582850 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f87480b2900 0x7f87480b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:54:13.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.607+0000 7f875effd700 1 --2- 172.21.15.52:0/3454582850 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f87480b2900 0x7f87480b4dc0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8760103420 tx=0x7f8750007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:54:13.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.610+0000 7f8747fff700 1 -- 172.21.15.52:0/3454582850 <== mon.2 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f87540a9990 con 0x7f87601035c0 2024-05-15T13:54:13.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:13.831+0000 7f875ffff700 1 -- 172.21.15.52:0/3454582850 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f87600ffb10 con 0x7f87480b2900 2024-05-15T13:54:14.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:14 smithi033 ceph-mon[31650]: Detected new or changed devices on smithi033 2024-05-15T13:54:14.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:14 smithi033 ceph-mon[31650]: Adjusting osd_memory_target on smithi033 to 5697M 2024-05-15T13:54:14.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:14 smithi033 ceph-mon[31650]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-15T13:54:14.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:14 smithi033 ceph-mon[31650]: osdmap e45: 7 total, 7 up, 7 in 2024-05-15T13:54:14.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:14 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:54:14.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:14 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:54:14.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:14 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:54:14.471 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:14 smithi052 ceph-mon[36190]: Detected new or changed devices on smithi033 2024-05-15T13:54:14.471 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:14 smithi052 ceph-mon[36190]: Adjusting osd_memory_target on smithi033 to 5697M 2024-05-15T13:54:14.472 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:14 smithi052 ceph-mon[36190]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-15T13:54:14.472 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:14 smithi052 ceph-mon[36190]: osdmap e45: 7 total, 7 up, 7 in 2024-05-15T13:54:14.472 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:14 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:54:14.472 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:14 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:54:14.472 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:14 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:54:14.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:14 smithi081 ceph-mon[36434]: Detected new or changed devices on smithi033 2024-05-15T13:54:14.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:14 smithi081 ceph-mon[36434]: Adjusting osd_memory_target on smithi033 to 5697M 2024-05-15T13:54:14.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:14 smithi081 ceph-mon[36434]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-05-15T13:54:14.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:14 smithi081 ceph-mon[36434]: osdmap e45: 7 total, 7 up, 7 in 2024-05-15T13:54:14.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:14 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:54:14.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:14 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:54:14.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:14 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:54:15.474 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:15 smithi052 ceph-mon[36190]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:54:15.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:15 smithi081 ceph-mon[36434]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:54:15.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:15 smithi033 ceph-mon[31650]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:54:16.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:16 smithi052 ceph-mon[36190]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-15T13:54:16.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:16 smithi081 ceph-mon[36434]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-15T13:54:16.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:16 smithi033 ceph-mon[31650]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-15T13:54:17.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:17 smithi052 ceph-mon[36190]: from='client.? 172.21.15.52:0/3314782349' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bda2f920-4f2c-4eed-8127-83c1973676e8"}]: dispatch 2024-05-15T13:54:17.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:17 smithi052 ceph-mon[36190]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bda2f920-4f2c-4eed-8127-83c1973676e8"}]: dispatch 2024-05-15T13:54:17.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:17 smithi052 ceph-mon[36190]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bda2f920-4f2c-4eed-8127-83c1973676e8"}]': finished 2024-05-15T13:54:17.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:17 smithi052 ceph-mon[36190]: osdmap e46: 8 total, 7 up, 8 in 2024-05-15T13:54:17.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:17 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-15T13:54:17.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:17 smithi052 ceph-mon[36190]: from='client.? 172.21.15.52:0/3024543021' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:54:17.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:17 smithi081 ceph-mon[36434]: from='client.? 172.21.15.52:0/3314782349' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bda2f920-4f2c-4eed-8127-83c1973676e8"}]: dispatch 2024-05-15T13:54:17.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:17 smithi081 ceph-mon[36434]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bda2f920-4f2c-4eed-8127-83c1973676e8"}]: dispatch 2024-05-15T13:54:17.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:17 smithi081 ceph-mon[36434]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bda2f920-4f2c-4eed-8127-83c1973676e8"}]': finished 2024-05-15T13:54:17.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:17 smithi081 ceph-mon[36434]: osdmap e46: 8 total, 7 up, 8 in 2024-05-15T13:54:17.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:17 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-15T13:54:17.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:17 smithi081 ceph-mon[36434]: from='client.? 172.21.15.52:0/3024543021' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:54:17.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:17 smithi033 ceph-mon[31650]: from='client.? 172.21.15.52:0/3314782349' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bda2f920-4f2c-4eed-8127-83c1973676e8"}]: dispatch 2024-05-15T13:54:17.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:17 smithi033 ceph-mon[31650]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bda2f920-4f2c-4eed-8127-83c1973676e8"}]: dispatch 2024-05-15T13:54:17.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:17 smithi033 ceph-mon[31650]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bda2f920-4f2c-4eed-8127-83c1973676e8"}]': finished 2024-05-15T13:54:17.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:17 smithi033 ceph-mon[31650]: osdmap e46: 8 total, 7 up, 8 in 2024-05-15T13:54:17.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:17 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-15T13:54:17.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:17 smithi033 ceph-mon[31650]: from='client.? 172.21.15.52:0/3024543021' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:54:18.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:18 smithi033 ceph-mon[31650]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-15T13:54:18.778 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:18 smithi081 ceph-mon[36434]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-15T13:54:18.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:18 smithi052 ceph-mon[36190]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-15T13:54:20.729 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:20 smithi052 ceph-mon[36190]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-15T13:54:20.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:20 smithi081 ceph-mon[36434]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-15T13:54:20.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:20 smithi033 ceph-mon[31650]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-15T13:54:22.658 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:22 smithi052 ceph-mon[36190]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-15T13:54:22.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:22 smithi081 ceph-mon[36434]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-15T13:54:22.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:22 smithi033 ceph-mon[31650]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-05-15T13:54:24.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:24 smithi052 ceph-mon[36190]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-15T13:54:24.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:24 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-15T13:54:24.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:24 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:54:24.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:24 smithi081 ceph-mon[36434]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-15T13:54:24.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:24 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-15T13:54:24.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:24 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:54:24.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:24 smithi033 ceph-mon[31650]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-15T13:54:24.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:24 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-15T13:54:24.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:24 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:54:25.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:25 smithi052 ceph-mon[36190]: Deploying daemon osd.7 on smithi052 2024-05-15T13:54:25.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:25 smithi081 ceph-mon[36434]: Deploying daemon osd.7 on smithi052 2024-05-15T13:54:25.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:25 smithi033 ceph-mon[31650]: Deploying daemon osd.7 on smithi052 2024-05-15T13:54:26.782 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:26 smithi052 ceph-mon[36190]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-15T13:54:26.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:26 smithi081 ceph-mon[36434]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-15T13:54:26.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:26 smithi033 ceph-mon[31650]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-15T13:54:28.538 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:28 smithi052 ceph-mon[36190]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-15T13:54:28.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:28 smithi081 ceph-mon[36434]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-15T13:54:28.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:28 smithi033 ceph-mon[31650]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-15T13:54:30.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:30 smithi033 ceph-mon[31650]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-15T13:54:30.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:30 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:30.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:30 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:54:30.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:30 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:30.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:30 smithi052 ceph-mon[36190]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-15T13:54:30.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:30 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:30.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:30 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:54:30.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:30 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:30.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:30 smithi081 ceph-mon[36434]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-15T13:54:30.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:30 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:30.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:30 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:54:30.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:30 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:32.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:32 smithi033 ceph-mon[31650]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-15T13:54:32.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:32 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:32.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:32 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:32.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:32 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:54:32.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:32 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:54:32.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:32 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:32.466 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:32 smithi052 ceph-mon[36190]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-15T13:54:32.466 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:32 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:32.466 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:32 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:32.466 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:32 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:54:32.466 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:32 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:54:32.466 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:32 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:32.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:32 smithi081 ceph-mon[36434]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-15T13:54:32.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:32 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:32.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:32 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:32.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:32 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:54:32.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:32 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:54:32.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:32 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:32.803 INFO:teuthology.orchestra.run.smithi052.stdout:Created osd(s) 7 on host 'smithi052' 2024-05-15T13:54:32.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:32.800+0000 7f8747fff700 1 -- 172.21.15.52:0/3454582850 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f87600ffb10 con 0x7f87480b2900 2024-05-15T13:54:32.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:32.802+0000 7f875ffff700 1 -- 172.21.15.52:0/3454582850 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f87480b2900 msgr2=0x7f87480b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:54:32.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:32.802+0000 7f875ffff700 1 --2- 172.21.15.52:0/3454582850 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f87480b2900 0x7f87480b4dc0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8760103420 tx=0x7f8750007400 comp rx=0 tx=0).stop 2024-05-15T13:54:32.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:32.802+0000 7f875ffff700 1 -- 172.21.15.52:0/3454582850 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87601035c0 msgr2=0x7f876019bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:54:32.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:32.802+0000 7f875ffff700 1 --2- 172.21.15.52:0/3454582850 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87601035c0 0x7f876019bc90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f875400d950 tx=0x7f875400dc60 comp rx=0 tx=0).stop 2024-05-15T13:54:32.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:32.802+0000 7f875ffff700 1 -- 172.21.15.52:0/3454582850 shutdown_connections 2024-05-15T13:54:32.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:32.802+0000 7f875ffff700 1 --2- 172.21.15.52:0/3454582850 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f87480b2900 0x7f87480b4dc0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:54:32.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:32.802+0000 7f875ffff700 1 --2- 172.21.15.52:0/3454582850 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8760103f60 0x7f87601a2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:54:32.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:32.802+0000 7f875ffff700 1 --2- 172.21.15.52:0/3454582850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87601023c0 0x7f876019b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:54:32.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:32.802+0000 7f875ffff700 1 --2- 172.21.15.52:0/3454582850 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f87601035c0 0x7f876019bc90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:54:32.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:32.802+0000 7f875ffff700 1 -- 172.21.15.52:0/3454582850 >> 172.21.15.52:0/3454582850 conn(0x7f87600fd960 msgr2=0x7f87600ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:54:32.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:32.802+0000 7f875ffff700 1 -- 172.21.15.52:0/3454582850 shutdown_connections 2024-05-15T13:54:32.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:54:32.802+0000 7f875ffff700 1 -- 172.21.15.52:0/3454582850 wait complete. 2024-05-15T13:54:33.371 DEBUG:teuthology.orchestra.run.smithi052:osd.7> sudo journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.7.service 2024-05-15T13:54:33.373 INFO:tasks.cephadm:Deploying osd.8 on smithi081 with /dev/vg_nvme/lv_2... 2024-05-15T13:54:33.374 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-15T13:54:33.618 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.c/config 2024-05-15T13:54:34.041 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:33 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:34.041 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:33 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:34.041 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:33 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:54:34.041 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:33 smithi081 ceph-mon[36434]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-15T13:54:34.041 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:33 smithi081 ceph-mon[36434]: from='osd.7 [v2:172.21.15.52:6816/1654595813,v1:172.21.15.52:6817/1654595813]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-15T13:54:34.041 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:33 smithi081 ceph-mon[36434]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-15T13:54:34.070 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:33 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:34.070 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:33 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:34.070 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:33 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:54:34.070 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:33 smithi052 ceph-mon[36190]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-15T13:54:34.070 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:33 smithi052 ceph-mon[36190]: from='osd.7 [v2:172.21.15.52:6816/1654595813,v1:172.21.15.52:6817/1654595813]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-15T13:54:34.071 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:33 smithi052 ceph-mon[36190]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-15T13:54:34.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:33 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:34.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:33 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:34.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:33 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:54:34.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:33 smithi033 ceph-mon[31650]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-15T13:54:34.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:33 smithi033 ceph-mon[31650]: from='osd.7 [v2:172.21.15.52:6816/1654595813,v1:172.21.15.52:6817/1654595813]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-15T13:54:34.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:33 smithi033 ceph-mon[31650]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-15T13:54:34.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:34 smithi081 ceph-mon[36434]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-15T13:54:34.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:34 smithi081 ceph-mon[36434]: from='osd.7 [v2:172.21.15.52:6816/1654595813,v1:172.21.15.52:6817/1654595813]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-05-15T13:54:34.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:34 smithi081 ceph-mon[36434]: osdmap e47: 8 total, 7 up, 8 in 2024-05-15T13:54:34.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:34 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-15T13:54:34.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:34 smithi081 ceph-mon[36434]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-05-15T13:54:35.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:34 smithi052 ceph-mon[36190]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-15T13:54:35.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:34 smithi052 ceph-mon[36190]: from='osd.7 [v2:172.21.15.52:6816/1654595813,v1:172.21.15.52:6817/1654595813]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-05-15T13:54:35.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:34 smithi052 ceph-mon[36190]: osdmap e47: 8 total, 7 up, 8 in 2024-05-15T13:54:35.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:34 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-15T13:54:35.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:34 smithi052 ceph-mon[36190]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-05-15T13:54:35.091 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 13:54:34 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T13:54:34.794+0000 7f0766524700 -1 osd.7 0 waiting for initial osdmap 2024-05-15T13:54:35.091 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 13:54:34 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T13:54:34.804+0000 7f0762317700 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-15T13:54:35.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:34 smithi033 ceph-mon[31650]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-15T13:54:35.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:34 smithi033 ceph-mon[31650]: from='osd.7 [v2:172.21.15.52:6816/1654595813,v1:172.21.15.52:6817/1654595813]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-05-15T13:54:35.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:34 smithi033 ceph-mon[31650]: osdmap e47: 8 total, 7 up, 8 in 2024-05-15T13:54:35.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:34 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-15T13:54:35.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:34 smithi033 ceph-mon[31650]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-05-15T13:54:35.472 INFO:teuthology.orchestra.run.smithi081.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-15T13:54:35.472 INFO:teuthology.orchestra.run.smithi081.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-15T13:54:35.472 INFO:teuthology.orchestra.run.smithi081.stderr: stderr: 10+0 records in 2024-05-15T13:54:35.473 INFO:teuthology.orchestra.run.smithi081.stderr:10+0 records out 2024-05-15T13:54:35.473 INFO:teuthology.orchestra.run.smithi081.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115221 s, 910 MB/s 2024-05-15T13:54:35.473 INFO:teuthology.orchestra.run.smithi081.stderr:--> Zapping successful for: 2024-05-15T13:54:36.002 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:35 smithi081 ceph-mon[36434]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-05-15T13:54:36.002 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:35 smithi081 ceph-mon[36434]: osdmap e48: 8 total, 7 up, 8 in 2024-05-15T13:54:36.002 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:35 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-15T13:54:36.002 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:35 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-15T13:54:36.002 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:35 smithi081 ceph-mon[36434]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-15T13:54:36.002 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:35 smithi081 ceph-mon[36434]: osd.7 [v2:172.21.15.52:6816/1654595813,v1:172.21.15.52:6817/1654595813] boot 2024-05-15T13:54:36.003 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:35 smithi081 ceph-mon[36434]: osdmap e49: 8 total, 8 up, 8 in 2024-05-15T13:54:36.003 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:35 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-15T13:54:36.031 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi081:vg_nvme/lv_2 2024-05-15T13:54:36.045 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:35 smithi052 ceph-mon[36190]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-05-15T13:54:36.045 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:35 smithi052 ceph-mon[36190]: osdmap e48: 8 total, 7 up, 8 in 2024-05-15T13:54:36.045 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:35 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-15T13:54:36.045 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:35 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-15T13:54:36.045 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:35 smithi052 ceph-mon[36190]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-15T13:54:36.046 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:35 smithi052 ceph-mon[36190]: osd.7 [v2:172.21.15.52:6816/1654595813,v1:172.21.15.52:6817/1654595813] boot 2024-05-15T13:54:36.046 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:35 smithi052 ceph-mon[36190]: osdmap e49: 8 total, 8 up, 8 in 2024-05-15T13:54:36.046 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:35 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-15T13:54:36.092 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:35 smithi033 ceph-mon[31650]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-05-15T13:54:36.092 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:35 smithi033 ceph-mon[31650]: osdmap e48: 8 total, 7 up, 8 in 2024-05-15T13:54:36.092 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:35 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-15T13:54:36.092 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:35 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-15T13:54:36.092 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:35 smithi033 ceph-mon[31650]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-05-15T13:54:36.092 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:35 smithi033 ceph-mon[31650]: osd.7 [v2:172.21.15.52:6816/1654595813,v1:172.21.15.52:6817/1654595813] boot 2024-05-15T13:54:36.092 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:35 smithi033 ceph-mon[31650]: osdmap e49: 8 total, 8 up, 8 in 2024-05-15T13:54:36.092 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:35 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-15T13:54:36.276 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.c/config 2024-05-15T13:54:36.950 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:36 smithi081 ceph-mon[36434]: purged_snaps scrub starts 2024-05-15T13:54:36.951 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:36 smithi081 ceph-mon[36434]: purged_snaps scrub ok 2024-05-15T13:54:36.951 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:36 smithi081 ceph-mon[36434]: Detected new or changed devices on smithi052 2024-05-15T13:54:36.951 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:36 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:36.951 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:36 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:36.951 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:36 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:54:36.951 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:36 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:54:36.951 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:36 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:54:36.951 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:36 smithi081 ceph-mon[36434]: Adjusting osd_memory_target on smithi052 to 5697M 2024-05-15T13:54:36.951 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:36 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:36.951 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:36 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:54:36.951 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:36 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:54:36.951 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:36 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:36.952 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:36 smithi081 ceph-mon[36434]: osdmap e50: 8 total, 8 up, 8 in 2024-05-15T13:54:37.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:36 smithi052 ceph-mon[36190]: purged_snaps scrub starts 2024-05-15T13:54:37.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:36 smithi052 ceph-mon[36190]: purged_snaps scrub ok 2024-05-15T13:54:37.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:36 smithi052 ceph-mon[36190]: Detected new or changed devices on smithi052 2024-05-15T13:54:37.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:36 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:37.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:36 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:37.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:36 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:54:37.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:36 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:54:37.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:36 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:54:37.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:36 smithi052 ceph-mon[36190]: Adjusting osd_memory_target on smithi052 to 5697M 2024-05-15T13:54:37.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:36 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:37.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:36 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:54:37.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:36 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:54:37.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:36 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:37.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:36 smithi052 ceph-mon[36190]: osdmap e50: 8 total, 8 up, 8 in 2024-05-15T13:54:37.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:36 smithi033 ceph-mon[31650]: purged_snaps scrub starts 2024-05-15T13:54:37.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:36 smithi033 ceph-mon[31650]: purged_snaps scrub ok 2024-05-15T13:54:37.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:36 smithi033 ceph-mon[31650]: Detected new or changed devices on smithi052 2024-05-15T13:54:37.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:36 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:37.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:36 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:37.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:36 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:54:37.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:36 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:54:37.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:36 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:54:37.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:36 smithi033 ceph-mon[31650]: Adjusting osd_memory_target on smithi052 to 5697M 2024-05-15T13:54:37.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:36 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:37.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:36 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:54:37.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:36 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:54:37.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:36 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:37.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:36 smithi033 ceph-mon[31650]: osdmap e50: 8 total, 8 up, 8 in 2024-05-15T13:54:37.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.705+0000 7f1d451d4700 1 -- 172.21.15.81:0/1707734549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d40104bd0 msgr2=0x7f1d40104ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:54:37.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.705+0000 7f1d451d4700 1 --2- 172.21.15.81:0/1707734549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d40104bd0 0x7f1d40104ff0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1d30009990 tx=0x7f1d30009ca0 comp rx=0 tx=0).stop 2024-05-15T13:54:37.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.705+0000 7f1d451d4700 1 -- 172.21.15.81:0/1707734549 shutdown_connections 2024-05-15T13:54:37.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.705+0000 7f1d451d4700 1 --2- 172.21.15.81:0/1707734549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d40101ff0 0x7f1d4010bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:54:37.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.706+0000 7f1d451d4700 1 --2- 172.21.15.81:0/1707734549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d40104bd0 0x7f1d40104ff0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:54:37.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.706+0000 7f1d451d4700 1 --2- 172.21.15.81:0/1707734549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d40101650 0x7f1d40101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:54:37.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.706+0000 7f1d451d4700 1 -- 172.21.15.81:0/1707734549 >> 172.21.15.81:0/1707734549 conn(0x7f1d400fd1b0 msgr2=0x7f1d400ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:54:37.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.706+0000 7f1d451d4700 1 -- 172.21.15.81:0/1707734549 shutdown_connections 2024-05-15T13:54:37.708 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.706+0000 7f1d451d4700 1 -- 172.21.15.81:0/1707734549 wait complete. 2024-05-15T13:54:37.710 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.706+0000 7f1d451d4700 1 Processor -- start 2024-05-15T13:54:37.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.707+0000 7f1d451d4700 1 -- start start 2024-05-15T13:54:37.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.707+0000 7f1d451d4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d40101650 0x7f1d4019b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:54:37.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.707+0000 7f1d451d4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d40101ff0 0x7f1d4019be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:54:37.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.707+0000 7f1d451d4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d40104bd0 0x7f1d401a2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:54:37.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.707+0000 7f1d451d4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4010f090 con 0x7f1d40101650 2024-05-15T13:54:37.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.707+0000 7f1d451d4700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4010ef10 con 0x7f1d40101ff0 2024-05-15T13:54:37.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.707+0000 7f1d451d4700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4010f210 con 0x7f1d40104bd0 2024-05-15T13:54:37.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.707+0000 7f1d3f59e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d40104bd0 0x7f1d401a2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:54:37.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.707+0000 7f1d3f59e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d40104bd0 0x7f1d401a2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39812/0 (socket says 172.21.15.81:39812) 2024-05-15T13:54:37.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.707+0000 7f1d3f59e700 1 -- 172.21.15.81:0/1308943076 learned_addr learned my addr 172.21.15.81:0/1308943076 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-15T13:54:37.711 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.707+0000 7f1d3ed9d700 1 --2- 172.21.15.81:0/1308943076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d40101650 0x7f1d4019b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:54:37.712 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.707+0000 7f1d3e59c700 1 --2- 172.21.15.81:0/1308943076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d40101ff0 0x7f1d4019be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:54:37.712 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.707+0000 7f1d3f59e700 1 -- 172.21.15.81:0/1308943076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d40101ff0 msgr2=0x7f1d4019be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:54:37.712 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.707+0000 7f1d3f59e700 1 --2- 172.21.15.81:0/1308943076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d40101ff0 0x7f1d4019be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:54:37.712 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.707+0000 7f1d3f59e700 1 -- 172.21.15.81:0/1308943076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d40101650 msgr2=0x7f1d4019b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:54:37.712 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.707+0000 7f1d3f59e700 1 --2- 172.21.15.81:0/1308943076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d40101650 0x7f1d4019b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:54:37.712 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.707+0000 7f1d3f59e700 1 -- 172.21.15.81:0/1308943076 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d401a3550 con 0x7f1d40104bd0 2024-05-15T13:54:37.712 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.707+0000 7f1d3e59c700 1 --2- 172.21.15.81:0/1308943076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d40101ff0 0x7f1d4019be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T13:54:37.712 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.707+0000 7f1d3f59e700 1 --2- 172.21.15.81:0/1308943076 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d40104bd0 0x7f1d401a2eb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1d38007d40 tx=0x7f1d3800a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:54:37.712 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.708+0000 7f1d27fff700 1 -- 172.21.15.81:0/1308943076 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d38016020 con 0x7f1d40104bd0 2024-05-15T13:54:37.712 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.708+0000 7f1d451d4700 1 -- 172.21.15.81:0/1308943076 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d401a3840 con 0x7f1d40104bd0 2024-05-15T13:54:37.712 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.708+0000 7f1d27fff700 1 -- 172.21.15.81:0/1308943076 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d3800ad60 con 0x7f1d40104bd0 2024-05-15T13:54:37.712 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.708+0000 7f1d27fff700 1 -- 172.21.15.81:0/1308943076 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d380115e0 con 0x7f1d40104bd0 2024-05-15T13:54:37.712 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.708+0000 7f1d451d4700 1 -- 172.21.15.81:0/1308943076 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d401a3e50 con 0x7f1d40104bd0 2024-05-15T13:54:37.712 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.709+0000 7f1d451d4700 1 -- 172.21.15.81:0/1308943076 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d4006a230 con 0x7f1d40104bd0 2024-05-15T13:54:37.713 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.710+0000 7f1d27fff700 1 -- 172.21.15.81:0/1308943076 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f1d38019020 con 0x7f1d40104bd0 2024-05-15T13:54:37.713 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.710+0000 7f1d27fff700 1 --2- 172.21.15.81:0/1308943076 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f1d280b2980 0x7f1d280b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:54:37.713 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.711+0000 7f1d27fff700 1 -- 172.21.15.81:0/1308943076 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5397+0+0 (secure 0 0 0) 0x7f1d380e1a90 con 0x7f1d40104bd0 2024-05-15T13:54:37.713 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.711+0000 7f1d3ed9d700 1 --2- 172.21.15.81:0/1308943076 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f1d280b2980 0x7f1d280b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:54:37.713 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.711+0000 7f1d3ed9d700 1 --2- 172.21.15.81:0/1308943076 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f1d280b2980 0x7f1d280b4e40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1d300038a0 tx=0x7f1d30005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:54:37.716 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.714+0000 7f1d27fff700 1 -- 172.21.15.81:0/1308943076 <== mon.1 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1d38020020 con 0x7f1d40104bd0 2024-05-15T13:54:37.932 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:37.930+0000 7f1d451d4700 1 -- 172.21.15.81:0/1308943076 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f1d40103e80 con 0x7f1d280b2980 2024-05-15T13:54:38.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:38 smithi052 ceph-mon[36190]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-05-15T13:54:38.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:38 smithi052 ceph-mon[36190]: osdmap e51: 8 total, 8 up, 8 in 2024-05-15T13:54:38.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:38 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:54:38.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:38 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:54:38.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:38 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:54:38.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:38 smithi081 ceph-mon[36434]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-05-15T13:54:38.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:38 smithi081 ceph-mon[36434]: osdmap e51: 8 total, 8 up, 8 in 2024-05-15T13:54:38.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:38 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:54:38.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:38 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:54:38.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:38 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:54:38.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:38 smithi033 ceph-mon[31650]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-05-15T13:54:38.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:38 smithi033 ceph-mon[31650]: osdmap e51: 8 total, 8 up, 8 in 2024-05-15T13:54:38.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:38 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:54:38.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:38 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:54:38.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:38 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:54:39.469 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:39 smithi081 ceph-mon[36434]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:54:39.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:39 smithi052 ceph-mon[36190]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:54:39.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:39 smithi033 ceph-mon[31650]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:54:40.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:40 smithi052 ceph-mon[36190]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-15T13:54:40.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:40 smithi081 ceph-mon[36434]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-15T13:54:40.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:40 smithi033 ceph-mon[31650]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-15T13:54:41.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:41 smithi052 ceph-mon[36190]: from='client.? 172.21.15.81:0/116942749' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "550585b8-0df4-49ee-95c8-3d084744dbd4"}]: dispatch 2024-05-15T13:54:41.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:41 smithi052 ceph-mon[36190]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "550585b8-0df4-49ee-95c8-3d084744dbd4"}]: dispatch 2024-05-15T13:54:41.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:41 smithi052 ceph-mon[36190]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "550585b8-0df4-49ee-95c8-3d084744dbd4"}]': finished 2024-05-15T13:54:41.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:41 smithi052 ceph-mon[36190]: osdmap e52: 9 total, 8 up, 9 in 2024-05-15T13:54:41.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:41 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-15T13:54:41.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:41 smithi052 ceph-mon[36190]: from='client.? 172.21.15.81:0/2494132166' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:54:41.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:41 smithi081 ceph-mon[36434]: from='client.? 172.21.15.81:0/116942749' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "550585b8-0df4-49ee-95c8-3d084744dbd4"}]: dispatch 2024-05-15T13:54:41.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:41 smithi081 ceph-mon[36434]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "550585b8-0df4-49ee-95c8-3d084744dbd4"}]: dispatch 2024-05-15T13:54:41.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:41 smithi081 ceph-mon[36434]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "550585b8-0df4-49ee-95c8-3d084744dbd4"}]': finished 2024-05-15T13:54:41.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:41 smithi081 ceph-mon[36434]: osdmap e52: 9 total, 8 up, 9 in 2024-05-15T13:54:41.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:41 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-15T13:54:41.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:41 smithi081 ceph-mon[36434]: from='client.? 172.21.15.81:0/2494132166' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:54:41.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:41 smithi033 ceph-mon[31650]: from='client.? 172.21.15.81:0/116942749' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "550585b8-0df4-49ee-95c8-3d084744dbd4"}]: dispatch 2024-05-15T13:54:41.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:41 smithi033 ceph-mon[31650]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "550585b8-0df4-49ee-95c8-3d084744dbd4"}]: dispatch 2024-05-15T13:54:41.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:41 smithi033 ceph-mon[31650]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "550585b8-0df4-49ee-95c8-3d084744dbd4"}]': finished 2024-05-15T13:54:41.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:41 smithi033 ceph-mon[31650]: osdmap e52: 9 total, 8 up, 9 in 2024-05-15T13:54:41.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:41 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-15T13:54:41.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:41 smithi033 ceph-mon[31650]: from='client.? 172.21.15.81:0/2494132166' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:54:42.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:42 smithi052 ceph-mon[36190]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-05-15T13:54:42.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:42 smithi081 ceph-mon[36434]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-05-15T13:54:42.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:42 smithi033 ceph-mon[31650]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-05-15T13:54:44.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:44 smithi052 ceph-mon[36190]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-05-15T13:54:44.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:44 smithi081 ceph-mon[36434]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-05-15T13:54:44.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:44 smithi033 ceph-mon[31650]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-05-15T13:54:46.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:46 smithi081 ceph-mon[36434]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-15T13:54:46.685 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:46 smithi052 ceph-mon[36190]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-15T13:54:46.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:46 smithi033 ceph-mon[31650]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-15T13:54:48.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:48 smithi081 ceph-mon[36434]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-15T13:54:48.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:48 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-15T13:54:48.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:48 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:54:48.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:48 smithi033 ceph-mon[31650]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-15T13:54:48.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:48 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-15T13:54:48.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:48 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:54:48.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:48 smithi052 ceph-mon[36190]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-15T13:54:48.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:48 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-15T13:54:48.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:48 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:54:49.511 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:49 smithi081 ceph-mon[36434]: Deploying daemon osd.8 on smithi081 2024-05-15T13:54:49.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:49 smithi033 ceph-mon[31650]: Deploying daemon osd.8 on smithi081 2024-05-15T13:54:49.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:49 smithi052 ceph-mon[36190]: Deploying daemon osd.8 on smithi081 2024-05-15T13:54:50.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:50 smithi081 ceph-mon[36434]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-15T13:54:50.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:50 smithi033 ceph-mon[31650]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-15T13:54:50.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:50 smithi052 ceph-mon[36190]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-15T13:54:52.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:52 smithi081 ceph-mon[36434]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-05-15T13:54:52.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:52 smithi033 ceph-mon[31650]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-05-15T13:54:52.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:52 smithi052 ceph-mon[36190]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-05-15T13:54:54.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:54 smithi081 ceph-mon[36434]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-15T13:54:54.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:54 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:54.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:54 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:54.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:54 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:54:54.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:54 smithi033 ceph-mon[31650]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-15T13:54:54.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:54 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:54.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:54 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:54.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:54 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:54:54.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:54 smithi052 ceph-mon[36190]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-15T13:54:54.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:54 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:54.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:54 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:54.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:54 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:54:56.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:56 smithi081 ceph-mon[36434]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-15T13:54:56.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:56 smithi033 ceph-mon[31650]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-15T13:54:56.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:56 smithi052 ceph-mon[36190]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-15T13:54:57.356 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:57.354+0000 7f1d27fff700 1 -- 172.21.15.81:0/1308943076 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1d40103e80 con 0x7f1d280b2980 2024-05-15T13:54:57.356 INFO:teuthology.orchestra.run.smithi081.stdout:Created osd(s) 8 on host 'smithi081' 2024-05-15T13:54:57.358 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:57.356+0000 7f1d451d4700 1 -- 172.21.15.81:0/1308943076 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f1d280b2980 msgr2=0x7f1d280b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:54:57.358 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:57.356+0000 7f1d451d4700 1 --2- 172.21.15.81:0/1308943076 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f1d280b2980 0x7f1d280b4e40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1d300038a0 tx=0x7f1d30005fb0 comp rx=0 tx=0).stop 2024-05-15T13:54:57.358 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:57.356+0000 7f1d451d4700 1 -- 172.21.15.81:0/1308943076 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d40104bd0 msgr2=0x7f1d401a2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:54:57.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:57.356+0000 7f1d451d4700 1 --2- 172.21.15.81:0/1308943076 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d40104bd0 0x7f1d401a2eb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1d38007d40 tx=0x7f1d3800a340 comp rx=0 tx=0).stop 2024-05-15T13:54:57.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:57.356+0000 7f1d451d4700 1 -- 172.21.15.81:0/1308943076 shutdown_connections 2024-05-15T13:54:57.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:57.356+0000 7f1d451d4700 1 --2- 172.21.15.81:0/1308943076 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f1d280b2980 0x7f1d280b4e40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:54:57.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:57.356+0000 7f1d451d4700 1 --2- 172.21.15.81:0/1308943076 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d40104bd0 0x7f1d401a2eb0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:54:57.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:57.356+0000 7f1d451d4700 1 --2- 172.21.15.81:0/1308943076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d40101650 0x7f1d4019b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:54:57.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:57.356+0000 7f1d451d4700 1 --2- 172.21.15.81:0/1308943076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d40101ff0 0x7f1d4019be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:54:57.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:57.356+0000 7f1d451d4700 1 -- 172.21.15.81:0/1308943076 >> 172.21.15.81:0/1308943076 conn(0x7f1d400fd1b0 msgr2=0x7f1d40102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:54:57.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:57.356+0000 7f1d451d4700 1 -- 172.21.15.81:0/1308943076 shutdown_connections 2024-05-15T13:54:57.359 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:54:57.356+0000 7f1d451d4700 1 -- 172.21.15.81:0/1308943076 wait complete. 2024-05-15T13:54:57.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:57 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:57.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:57 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:57.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:57 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:54:57.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:57 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:54:57.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:57 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:57.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:57 smithi081 ceph-mon[36434]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-15T13:54:57.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:57 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:57.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:57 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:57.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:57 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:54:57.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:57 smithi081 ceph-mon[36434]: from='osd.8 [v2:172.21.15.81:6816/3009661748,v1:172.21.15.81:6817/3009661748]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-15T13:54:57.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:57 smithi081 ceph-mon[36434]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-15T13:54:57.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:57 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:57.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:57 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:57.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:57 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:54:57.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:57 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:54:57.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:57 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:57.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:57 smithi033 ceph-mon[31650]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-15T13:54:57.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:57 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:57.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:57 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:57.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:57 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:54:57.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:57 smithi033 ceph-mon[31650]: from='osd.8 [v2:172.21.15.81:6816/3009661748,v1:172.21.15.81:6817/3009661748]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-15T13:54:57.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:57 smithi033 ceph-mon[31650]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-15T13:54:57.974 DEBUG:teuthology.orchestra.run.smithi081:osd.8> sudo journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.8.service 2024-05-15T13:54:57.976 INFO:tasks.cephadm:Deploying osd.9 on smithi033 with /dev/vg_nvme/lv_1... 2024-05-15T13:54:57.976 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-15T13:54:58.090 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:57 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:58.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:57 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:58.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:57 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:54:58.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:57 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:54:58.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:57 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:58.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:57 smithi052 ceph-mon[36190]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-15T13:54:58.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:57 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:58.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:57 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:54:58.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:57 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:54:58.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:57 smithi052 ceph-mon[36190]: from='osd.8 [v2:172.21.15.81:6816/3009661748,v1:172.21.15.81:6817/3009661748]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-15T13:54:58.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:57 smithi052 ceph-mon[36190]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-15T13:54:58.234 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:54:59.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:59 smithi052 ceph-mon[36190]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-15T13:54:59.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:59 smithi052 ceph-mon[36190]: osdmap e53: 9 total, 8 up, 9 in 2024-05-15T13:54:59.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:59 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-15T13:54:59.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:59 smithi052 ceph-mon[36190]: from='osd.8 [v2:172.21.15.81:6816/3009661748,v1:172.21.15.81:6817/3009661748]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-05-15T13:54:59.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:54:59 smithi052 ceph-mon[36190]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-05-15T13:54:59.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:59 smithi081 ceph-mon[36434]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-15T13:54:59.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:59 smithi081 ceph-mon[36434]: osdmap e53: 9 total, 8 up, 9 in 2024-05-15T13:54:59.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:59 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-15T13:54:59.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:59 smithi081 ceph-mon[36434]: from='osd.8 [v2:172.21.15.81:6816/3009661748,v1:172.21.15.81:6817/3009661748]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-05-15T13:54:59.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:54:59 smithi081 ceph-mon[36434]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-05-15T13:54:59.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:59 smithi033 ceph-mon[31650]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-15T13:54:59.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:59 smithi033 ceph-mon[31650]: osdmap e53: 9 total, 8 up, 9 in 2024-05-15T13:54:59.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:59 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-15T13:54:59.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:59 smithi033 ceph-mon[31650]: from='osd.8 [v2:172.21.15.81:6816/3009661748,v1:172.21.15.81:6817/3009661748]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-05-15T13:54:59.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:54:59 smithi033 ceph-mon[31650]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-05-15T13:55:00.435 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-15T13:55:00.435 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-15T13:55:00.435 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-05-15T13:55:00.435 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-05-15T13:55:00.435 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110778 s, 947 MB/s 2024-05-15T13:55:00.436 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-05-15T13:55:00.503 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:00 smithi081 ceph-mon[36434]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-15T13:55:00.503 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:00 smithi081 ceph-mon[36434]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-05-15T13:55:00.503 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:00 smithi081 ceph-mon[36434]: osdmap e54: 9 total, 8 up, 9 in 2024-05-15T13:55:00.504 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:00 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-15T13:55:00.504 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:00 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-15T13:55:00.504 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:00 smithi081 ceph-mon[36434]: from='osd.8 ' entity='osd.8' 2024-05-15T13:55:00.504 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:00 smithi081 ceph-mon[36434]: osd.8 [v2:172.21.15.81:6816/3009661748,v1:172.21.15.81:6817/3009661748] boot 2024-05-15T13:55:00.504 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:00 smithi081 ceph-mon[36434]: osdmap e55: 9 total, 9 up, 9 in 2024-05-15T13:55:00.504 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:00 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-15T13:55:00.504 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 13:55:00 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T13:55:00.219+0000 7f0a6329f700 -1 osd.8 0 waiting for initial osdmap 2024-05-15T13:55:00.504 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 13:55:00 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T13:55:00.306+0000 7f0a5b08a700 -1 osd.8 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-15T13:55:00.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:00 smithi052 ceph-mon[36190]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-15T13:55:00.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:00 smithi052 ceph-mon[36190]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-05-15T13:55:00.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:00 smithi052 ceph-mon[36190]: osdmap e54: 9 total, 8 up, 9 in 2024-05-15T13:55:00.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:00 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-15T13:55:00.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:00 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-15T13:55:00.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:00 smithi052 ceph-mon[36190]: from='osd.8 ' entity='osd.8' 2024-05-15T13:55:00.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:00 smithi052 ceph-mon[36190]: osd.8 [v2:172.21.15.81:6816/3009661748,v1:172.21.15.81:6817/3009661748] boot 2024-05-15T13:55:00.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:00 smithi052 ceph-mon[36190]: osdmap e55: 9 total, 9 up, 9 in 2024-05-15T13:55:00.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:00 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-15T13:55:00.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:00 smithi033 ceph-mon[31650]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-05-15T13:55:00.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:00 smithi033 ceph-mon[31650]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-05-15T13:55:00.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:00 smithi033 ceph-mon[31650]: osdmap e54: 9 total, 8 up, 9 in 2024-05-15T13:55:00.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:00 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-15T13:55:00.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:00 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-15T13:55:00.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:00 smithi033 ceph-mon[31650]: from='osd.8 ' entity='osd.8' 2024-05-15T13:55:00.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:00 smithi033 ceph-mon[31650]: osd.8 [v2:172.21.15.81:6816/3009661748,v1:172.21.15.81:6817/3009661748] boot 2024-05-15T13:55:00.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:00 smithi033 ceph-mon[31650]: osdmap e55: 9 total, 9 up, 9 in 2024-05-15T13:55:00.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:00 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-15T13:55:01.096 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_1 2024-05-15T13:55:01.351 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:55:01.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:01 smithi052 ceph-mon[36190]: purged_snaps scrub starts 2024-05-15T13:55:01.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:01 smithi052 ceph-mon[36190]: purged_snaps scrub ok 2024-05-15T13:55:01.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:01 smithi052 ceph-mon[36190]: osdmap e56: 9 total, 9 up, 9 in 2024-05-15T13:55:01.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:01 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:01.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:01 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:01.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:01 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:55:01.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:01 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:55:01.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:01 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:55:01.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:01 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:01.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:01 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:55:01.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:01 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:55:01.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:01 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:01.593 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:01 smithi033 ceph-mon[31650]: purged_snaps scrub starts 2024-05-15T13:55:01.594 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:01 smithi033 ceph-mon[31650]: purged_snaps scrub ok 2024-05-15T13:55:01.594 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:01 smithi033 ceph-mon[31650]: osdmap e56: 9 total, 9 up, 9 in 2024-05-15T13:55:01.594 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:01 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:01.594 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:01 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:01.594 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:01 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:55:01.594 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:01 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:55:01.594 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:01 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:55:01.594 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:01 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:01.594 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:01 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:55:01.594 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:01 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:55:01.594 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:01 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:01.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:01 smithi081 ceph-mon[36434]: purged_snaps scrub starts 2024-05-15T13:55:01.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:01 smithi081 ceph-mon[36434]: purged_snaps scrub ok 2024-05-15T13:55:01.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:01 smithi081 ceph-mon[36434]: osdmap e56: 9 total, 9 up, 9 in 2024-05-15T13:55:01.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:01 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:01.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:01 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:01.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:01 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:55:01.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:01 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:55:01.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:01 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:55:01.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:01 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:01.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:01 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:55:01.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:01 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:55:01.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:01 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:02.435 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:02 smithi033 ceph-mon[31650]: Detected new or changed devices on smithi081 2024-05-15T13:55:02.435 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:02 smithi033 ceph-mon[31650]: Adjusting osd_memory_target on smithi081 to 5696M 2024-05-15T13:55:02.435 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:02 smithi033 ceph-mon[31650]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 567 MiB used, 804 GiB / 805 GiB avail 2024-05-15T13:55:02.435 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:02 smithi033 ceph-mon[31650]: osdmap e57: 9 total, 9 up, 9 in 2024-05-15T13:55:02.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:02 smithi081 ceph-mon[36434]: Detected new or changed devices on smithi081 2024-05-15T13:55:02.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:02 smithi081 ceph-mon[36434]: Adjusting osd_memory_target on smithi081 to 5696M 2024-05-15T13:55:02.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:02 smithi081 ceph-mon[36434]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 567 MiB used, 804 GiB / 805 GiB avail 2024-05-15T13:55:02.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:02 smithi081 ceph-mon[36434]: osdmap e57: 9 total, 9 up, 9 in 2024-05-15T13:55:02.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:02 smithi052 ceph-mon[36190]: Detected new or changed devices on smithi081 2024-05-15T13:55:02.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:02 smithi052 ceph-mon[36190]: Adjusting osd_memory_target on smithi081 to 5696M 2024-05-15T13:55:02.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:02 smithi052 ceph-mon[36190]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 567 MiB used, 804 GiB / 805 GiB avail 2024-05-15T13:55:02.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:02 smithi052 ceph-mon[36190]: osdmap e57: 9 total, 9 up, 9 in 2024-05-15T13:55:02.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.945+0000 7f314821f700 1 -- 172.21.15.33:0/1983960483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3140102490 msgr2=0x7f31401028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:55:02.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.945+0000 7f314821f700 1 --2- 172.21.15.33:0/1983960483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3140102490 0x7f31401028b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f313c0099c0 tx=0x7f313c009cd0 comp rx=0 tx=0).stop 2024-05-15T13:55:02.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.945+0000 7f314821f700 1 -- 172.21.15.33:0/1983960483 shutdown_connections 2024-05-15T13:55:02.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.945+0000 7f314821f700 1 --2- 172.21.15.33:0/1983960483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3140104030 0x7f314010a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:55:02.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.945+0000 7f314821f700 1 --2- 172.21.15.33:0/1983960483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3140102490 0x7f31401028b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:55:02.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.945+0000 7f314821f700 1 --2- 172.21.15.33:0/1983960483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3140103690 0x7f3140103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:55:02.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.945+0000 7f314821f700 1 -- 172.21.15.33:0/1983960483 >> 172.21.15.33:0/1983960483 conn(0x7f31400fda30 msgr2=0x7f31400ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:55:02.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.945+0000 7f314821f700 1 -- 172.21.15.33:0/1983960483 shutdown_connections 2024-05-15T13:55:02.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.945+0000 7f314821f700 1 -- 172.21.15.33:0/1983960483 wait complete. 2024-05-15T13:55:02.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.946+0000 7f314821f700 1 Processor -- start 2024-05-15T13:55:02.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.946+0000 7f314821f700 1 -- start start 2024-05-15T13:55:02.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.946+0000 7f314821f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3140102490 0x7f314019b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:55:02.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.946+0000 7f314821f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3140103690 0x7f314019be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:55:02.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.946+0000 7f314821f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3140104030 0x7f31401a2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:55:02.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.946+0000 7f314821f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f314010df80 con 0x7f3140103690 2024-05-15T13:55:02.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.946+0000 7f314821f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f314010de00 con 0x7f3140104030 2024-05-15T13:55:02.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.946+0000 7f314821f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f314010e100 con 0x7f3140102490 2024-05-15T13:55:02.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.946+0000 7f31457ba700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3140103690 0x7f314019be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:55:02.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.946+0000 7f31457ba700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3140103690 0x7f314019be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41294/0 (socket says 172.21.15.33:41294) 2024-05-15T13:55:02.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.946+0000 7f31457ba700 1 -- 172.21.15.33:0/3581509237 learned_addr learned my addr 172.21.15.33:0/3581509237 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:55:02.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.946+0000 7f3145fbb700 1 --2- 172.21.15.33:0/3581509237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3140102490 0x7f314019b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:55:02.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.946+0000 7f31467bc700 1 --2- 172.21.15.33:0/3581509237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3140104030 0x7f31401a2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:55:02.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.946+0000 7f31457ba700 1 -- 172.21.15.33:0/3581509237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3140102490 msgr2=0x7f314019b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:55:02.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.946+0000 7f31457ba700 1 --2- 172.21.15.33:0/3581509237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3140102490 0x7f314019b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:55:02.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.946+0000 7f31457ba700 1 -- 172.21.15.33:0/3581509237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3140104030 msgr2=0x7f31401a2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:55:02.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.946+0000 7f31457ba700 1 --2- 172.21.15.33:0/3581509237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3140104030 0x7f31401a2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:55:02.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.946+0000 7f31457ba700 1 -- 172.21.15.33:0/3581509237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31401a35b0 con 0x7f3140103690 2024-05-15T13:55:02.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.946+0000 7f31467bc700 1 --2- 172.21.15.33:0/3581509237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3140104030 0x7f31401a2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T13:55:02.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.947+0000 7f31457ba700 1 --2- 172.21.15.33:0/3581509237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3140103690 0x7f314019be90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f313000d980 tx=0x7f313000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:55:02.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.947+0000 7f3136ffd700 1 -- 172.21.15.33:0/3581509237 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3130016070 con 0x7f3140103690 2024-05-15T13:55:02.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.947+0000 7f3136ffd700 1 -- 172.21.15.33:0/3581509237 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3130010460 con 0x7f3140103690 2024-05-15T13:55:02.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.947+0000 7f3136ffd700 1 -- 172.21.15.33:0/3581509237 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313000f620 con 0x7f3140103690 2024-05-15T13:55:02.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.947+0000 7f314821f700 1 -- 172.21.15.33:0/3581509237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31401a38a0 con 0x7f3140103690 2024-05-15T13:55:02.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.947+0000 7f314821f700 1 -- 172.21.15.33:0/3581509237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31401a3d90 con 0x7f3140103690 2024-05-15T13:55:02.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.948+0000 7f314821f700 1 -- 172.21.15.33:0/3581509237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f314006a230 con 0x7f3140103690 2024-05-15T13:55:02.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.949+0000 7f3136ffd700 1 -- 172.21.15.33:0/3581509237 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3130019070 con 0x7f3140103690 2024-05-15T13:55:02.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.950+0000 7f3136ffd700 1 --2- 172.21.15.33:0/3581509237 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f312c0b2900 0x7f312c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:55:02.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.950+0000 7f3145fbb700 1 --2- 172.21.15.33:0/3581509237 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f312c0b2900 0x7f312c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:55:02.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.950+0000 7f3136ffd700 1 -- 172.21.15.33:0/3581509237 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5829+0+0 (secure 0 0 0) 0x7f31300e1430 con 0x7f3140103690 2024-05-15T13:55:02.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.951+0000 7f3145fbb700 1 --2- 172.21.15.33:0/3581509237 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f312c0b2900 0x7f312c0b4dc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f313c00b580 tx=0x7f313c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:55:02.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:02.954+0000 7f3136ffd700 1 -- 172.21.15.33:0/3581509237 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f31300a9a60 con 0x7f3140103690 2024-05-15T13:55:03.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:03.180+0000 7f314821f700 1 -- 172.21.15.33:0/3581509237 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f314006c1f0 con 0x7f312c0b2900 2024-05-15T13:55:03.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:03 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:55:03.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:03 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:55:03.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:03 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:55:03.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:03 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:55:03.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:03 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:55:03.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:03 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:55:03.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:03 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:55:03.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:03 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:55:03.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:03 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:55:04.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:04 smithi081 ceph-mon[36434]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 567 MiB used, 804 GiB / 805 GiB avail 2024-05-15T13:55:04.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:04 smithi081 ceph-mon[36434]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:55:04.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:04 smithi033 ceph-mon[31650]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 567 MiB used, 804 GiB / 805 GiB avail 2024-05-15T13:55:04.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:04 smithi033 ceph-mon[31650]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:55:04.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:04 smithi052 ceph-mon[36190]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 567 MiB used, 804 GiB / 805 GiB avail 2024-05-15T13:55:04.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:04 smithi052 ceph-mon[36190]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:55:06.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:06 smithi081 ceph-mon[36434]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 567 MiB used, 804 GiB / 805 GiB avail 2024-05-15T13:55:06.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:06 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/1318872048' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d781846-26f3-4b23-b22c-c994a8509a71"}]: dispatch 2024-05-15T13:55:06.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:06 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/1318872048' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d781846-26f3-4b23-b22c-c994a8509a71"}]': finished 2024-05-15T13:55:06.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:06 smithi081 ceph-mon[36434]: osdmap e58: 10 total, 9 up, 10 in 2024-05-15T13:55:06.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:06 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-15T13:55:06.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:06 smithi033 ceph-mon[31650]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 567 MiB used, 804 GiB / 805 GiB avail 2024-05-15T13:55:06.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:06 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/1318872048' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d781846-26f3-4b23-b22c-c994a8509a71"}]: dispatch 2024-05-15T13:55:06.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:06 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/1318872048' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d781846-26f3-4b23-b22c-c994a8509a71"}]': finished 2024-05-15T13:55:06.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:06 smithi033 ceph-mon[31650]: osdmap e58: 10 total, 9 up, 10 in 2024-05-15T13:55:06.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:06 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-15T13:55:06.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:06 smithi052 ceph-mon[36190]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 567 MiB used, 804 GiB / 805 GiB avail 2024-05-15T13:55:06.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:06 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/1318872048' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d781846-26f3-4b23-b22c-c994a8509a71"}]: dispatch 2024-05-15T13:55:06.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:06 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/1318872048' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d781846-26f3-4b23-b22c-c994a8509a71"}]': finished 2024-05-15T13:55:06.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:06 smithi052 ceph-mon[36190]: osdmap e58: 10 total, 9 up, 10 in 2024-05-15T13:55:06.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:06 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-15T13:55:07.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:07 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/1405979206' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:55:07.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:07 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/1405979206' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:55:07.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:07 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/1405979206' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:55:08.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:08 smithi033 ceph-mon[31650]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-05-15T13:55:08.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:08 smithi052 ceph-mon[36190]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-05-15T13:55:08.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:08 smithi081 ceph-mon[36434]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-05-15T13:55:10.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:10 smithi033 ceph-mon[31650]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-15T13:55:10.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:10 smithi052 ceph-mon[36190]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-15T13:55:10.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:10 smithi081 ceph-mon[36434]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-15T13:55:12.671 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:12 smithi033 ceph-mon[31650]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-15T13:55:12.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:12 smithi052 ceph-mon[36190]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-15T13:55:12.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:12 smithi081 ceph-mon[36434]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-15T13:55:14.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:14 smithi033 ceph-mon[31650]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-15T13:55:14.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:14 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-15T13:55:14.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:14 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:55:14.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:14 smithi052 ceph-mon[36190]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-15T13:55:14.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:14 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-15T13:55:14.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:14 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:55:14.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:14 smithi081 ceph-mon[36434]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-15T13:55:14.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:14 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-15T13:55:14.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:14 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:55:15.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:15 smithi033 ceph-mon[31650]: Deploying daemon osd.9 on smithi033 2024-05-15T13:55:15.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:15 smithi052 ceph-mon[36190]: Deploying daemon osd.9 on smithi033 2024-05-15T13:55:15.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:15 smithi081 ceph-mon[36434]: Deploying daemon osd.9 on smithi033 2024-05-15T13:55:16.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:16 smithi033 ceph-mon[31650]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-15T13:55:16.742 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:16 smithi052 ceph-mon[36190]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-15T13:55:16.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:16 smithi081 ceph-mon[36434]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-15T13:55:18.659 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:18 smithi081 ceph-mon[36434]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-15T13:55:18.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:18 smithi033 ceph-mon[31650]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-15T13:55:18.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:18 smithi052 ceph-mon[36190]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-15T13:55:20.620 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:20 smithi033 ceph-mon[31650]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-15T13:55:20.620 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:20 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:20.620 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:20 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:20.620 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:20 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:55:20.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:20 smithi052 ceph-mon[36190]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-15T13:55:20.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:20 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:20.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:20 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:20.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:20 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:55:20.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:20 smithi081 ceph-mon[36434]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-15T13:55:20.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:20 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:20.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:20 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:20.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:20 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:55:22.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:22 smithi033 ceph-mon[31650]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-15T13:55:22.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:22 smithi052 ceph-mon[36190]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-15T13:55:22.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:22 smithi081 ceph-mon[36434]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-15T13:55:23.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:23.593+0000 7f3136ffd700 1 -- 172.21.15.33:0/3581509237 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f314006c1f0 con 0x7f312c0b2900 2024-05-15T13:55:23.596 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 9 on host 'smithi033' 2024-05-15T13:55:23.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:23.595+0000 7f314821f700 1 -- 172.21.15.33:0/3581509237 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f312c0b2900 msgr2=0x7f312c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:55:23.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:23.595+0000 7f314821f700 1 --2- 172.21.15.33:0/3581509237 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f312c0b2900 0x7f312c0b4dc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f313c00b580 tx=0x7f313c005fb0 comp rx=0 tx=0).stop 2024-05-15T13:55:23.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:23.595+0000 7f314821f700 1 -- 172.21.15.33:0/3581509237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3140103690 msgr2=0x7f314019be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:55:23.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:23.595+0000 7f314821f700 1 --2- 172.21.15.33:0/3581509237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3140103690 0x7f314019be90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f313000d980 tx=0x7f313000dd40 comp rx=0 tx=0).stop 2024-05-15T13:55:23.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:23.595+0000 7f314821f700 1 -- 172.21.15.33:0/3581509237 shutdown_connections 2024-05-15T13:55:23.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:23.595+0000 7f314821f700 1 --2- 172.21.15.33:0/3581509237 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f312c0b2900 0x7f312c0b4dc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:55:23.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:23.595+0000 7f314821f700 1 --2- 172.21.15.33:0/3581509237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3140104030 0x7f31401a2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:55:23.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:23.595+0000 7f314821f700 1 --2- 172.21.15.33:0/3581509237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3140102490 0x7f314019b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:55:23.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:23.595+0000 7f314821f700 1 --2- 172.21.15.33:0/3581509237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3140103690 0x7f314019be90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:55:23.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:23.595+0000 7f314821f700 1 -- 172.21.15.33:0/3581509237 >> 172.21.15.33:0/3581509237 conn(0x7f31400fda30 msgr2=0x7f314006bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:55:23.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:23.596+0000 7f314821f700 1 -- 172.21.15.33:0/3581509237 shutdown_connections 2024-05-15T13:55:23.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:55:23.596+0000 7f314821f700 1 -- 172.21.15.33:0/3581509237 wait complete. 2024-05-15T13:55:23.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:23 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:23.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:23 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:23.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:23 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:55:23.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:23 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:55:23.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:23 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:23.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:23 smithi033 ceph-mon[31650]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-15T13:55:23.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:23 smithi033 ceph-mon[31650]: from='osd.9 [v2:172.21.15.33:6826/2843220858,v1:172.21.15.33:6827/2843220858]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-15T13:55:23.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:23 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:23.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:23 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:23.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:23 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:55:24.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:23 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:24.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:23 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:24.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:23 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:55:24.094 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:23 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:55:24.094 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:23 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:24.094 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:23 smithi052 ceph-mon[36190]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-15T13:55:24.094 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:23 smithi052 ceph-mon[36190]: from='osd.9 [v2:172.21.15.33:6826/2843220858,v1:172.21.15.33:6827/2843220858]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-15T13:55:24.094 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:23 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:24.094 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:23 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:24.094 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:23 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:55:24.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:23 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:24.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:23 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:24.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:23 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:55:24.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:23 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:55:24.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:23 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:24.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:23 smithi081 ceph-mon[36434]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-15T13:55:24.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:23 smithi081 ceph-mon[36434]: from='osd.9 [v2:172.21.15.33:6826/2843220858,v1:172.21.15.33:6827/2843220858]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-15T13:55:24.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:23 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:24.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:23 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:24.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:23 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:55:24.224 DEBUG:teuthology.orchestra.run.smithi033:osd.9> sudo journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.9.service 2024-05-15T13:55:24.227 INFO:tasks.cephadm:Deploying osd.10 on smithi052 with /dev/vg_nvme/lv_1... 2024-05-15T13:55:24.227 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-15T13:55:24.484 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.b/config 2024-05-15T13:55:25.030 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:24 smithi052 ceph-mon[36190]: from='osd.9 [v2:172.21.15.33:6826/2843220858,v1:172.21.15.33:6827/2843220858]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-15T13:55:25.030 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:24 smithi052 ceph-mon[36190]: osdmap e59: 10 total, 9 up, 10 in 2024-05-15T13:55:25.030 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:24 smithi052 ceph-mon[36190]: from='osd.9 [v2:172.21.15.33:6826/2843220858,v1:172.21.15.33:6827/2843220858]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-15T13:55:25.030 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:24 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-15T13:55:25.103 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:24 smithi033 ceph-mon[31650]: from='osd.9 [v2:172.21.15.33:6826/2843220858,v1:172.21.15.33:6827/2843220858]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-15T13:55:25.104 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:24 smithi033 ceph-mon[31650]: osdmap e59: 10 total, 9 up, 10 in 2024-05-15T13:55:25.104 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:24 smithi033 ceph-mon[31650]: from='osd.9 [v2:172.21.15.33:6826/2843220858,v1:172.21.15.33:6827/2843220858]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-15T13:55:25.104 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:24 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-15T13:55:25.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:24 smithi081 ceph-mon[36434]: from='osd.9 [v2:172.21.15.33:6826/2843220858,v1:172.21.15.33:6827/2843220858]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-15T13:55:25.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:24 smithi081 ceph-mon[36434]: osdmap e59: 10 total, 9 up, 10 in 2024-05-15T13:55:25.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:24 smithi081 ceph-mon[36434]: from='osd.9 [v2:172.21.15.33:6826/2843220858,v1:172.21.15.33:6827/2843220858]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-15T13:55:25.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:24 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-15T13:55:25.862 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:25 smithi052 ceph-mon[36190]: from='osd.9 [v2:172.21.15.33:6826/2843220858,v1:172.21.15.33:6827/2843220858]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-15T13:55:25.862 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:25 smithi052 ceph-mon[36190]: osdmap e60: 10 total, 9 up, 10 in 2024-05-15T13:55:25.862 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:25 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-15T13:55:25.862 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:25 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-15T13:55:25.862 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:25 smithi052 ceph-mon[36190]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-15T13:55:25.862 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:25 smithi052 ceph-mon[36190]: from='osd.9 [v2:172.21.15.33:6826/2843220858,v1:172.21.15.33:6827/2843220858]' entity='osd.9' 2024-05-15T13:55:25.862 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:25 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-15T13:55:25.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:25 smithi033 ceph-mon[31650]: from='osd.9 [v2:172.21.15.33:6826/2843220858,v1:172.21.15.33:6827/2843220858]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-15T13:55:25.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:25 smithi033 ceph-mon[31650]: osdmap e60: 10 total, 9 up, 10 in 2024-05-15T13:55:25.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:25 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-15T13:55:25.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:25 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-15T13:55:25.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:25 smithi033 ceph-mon[31650]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-15T13:55:25.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:25 smithi033 ceph-mon[31650]: from='osd.9 [v2:172.21.15.33:6826/2843220858,v1:172.21.15.33:6827/2843220858]' entity='osd.9' 2024-05-15T13:55:25.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:25 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-15T13:55:25.942 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 13:55:25 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T13:55:25.689+0000 7f82471ef700 -1 osd.9 0 waiting for initial osdmap 2024-05-15T13:55:25.942 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 13:55:25 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T13:55:25.796+0000 7f823efda700 -1 osd.9 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-15T13:55:26.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:25 smithi081 ceph-mon[36434]: from='osd.9 [v2:172.21.15.33:6826/2843220858,v1:172.21.15.33:6827/2843220858]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-15T13:55:26.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:25 smithi081 ceph-mon[36434]: osdmap e60: 10 total, 9 up, 10 in 2024-05-15T13:55:26.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:25 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-15T13:55:26.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:25 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-15T13:55:26.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:25 smithi081 ceph-mon[36434]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-15T13:55:26.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:25 smithi081 ceph-mon[36434]: from='osd.9 [v2:172.21.15.33:6826/2843220858,v1:172.21.15.33:6827/2843220858]' entity='osd.9' 2024-05-15T13:55:26.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:25 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-15T13:55:26.445 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-15T13:55:26.445 INFO:teuthology.orchestra.run.smithi052.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-15T13:55:26.445 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10+0 records in 2024-05-15T13:55:26.445 INFO:teuthology.orchestra.run.smithi052.stderr:10+0 records out 2024-05-15T13:55:26.445 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00847203 s, 1.2 GB/s 2024-05-15T13:55:26.445 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping successful for: 2024-05-15T13:55:27.090 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:26 smithi052 ceph-mon[36190]: purged_snaps scrub starts 2024-05-15T13:55:27.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:26 smithi052 ceph-mon[36190]: purged_snaps scrub ok 2024-05-15T13:55:27.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:26 smithi052 ceph-mon[36190]: osd.9 [v2:172.21.15.33:6826/2843220858,v1:172.21.15.33:6827/2843220858] boot 2024-05-15T13:55:27.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:26 smithi052 ceph-mon[36190]: osdmap e61: 10 total, 10 up, 10 in 2024-05-15T13:55:27.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:26 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-15T13:55:27.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:26 smithi081 ceph-mon[36434]: purged_snaps scrub starts 2024-05-15T13:55:27.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:26 smithi081 ceph-mon[36434]: purged_snaps scrub ok 2024-05-15T13:55:27.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:26 smithi081 ceph-mon[36434]: osd.9 [v2:172.21.15.33:6826/2843220858,v1:172.21.15.33:6827/2843220858] boot 2024-05-15T13:55:27.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:26 smithi081 ceph-mon[36434]: osdmap e61: 10 total, 10 up, 10 in 2024-05-15T13:55:27.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:26 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-15T13:55:27.137 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:26 smithi033 ceph-mon[31650]: purged_snaps scrub starts 2024-05-15T13:55:27.137 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:26 smithi033 ceph-mon[31650]: purged_snaps scrub ok 2024-05-15T13:55:27.137 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:26 smithi033 ceph-mon[31650]: osd.9 [v2:172.21.15.33:6826/2843220858,v1:172.21.15.33:6827/2843220858] boot 2024-05-15T13:55:27.137 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:26 smithi033 ceph-mon[31650]: osdmap e61: 10 total, 10 up, 10 in 2024-05-15T13:55:27.137 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:26 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-15T13:55:27.238 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi052:vg_nvme/lv_1 2024-05-15T13:55:27.486 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.b/config 2024-05-15T13:55:28.507 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:28 smithi052 ceph-mon[36190]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:28.507 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:28 smithi052 ceph-mon[36190]: Detected new or changed devices on smithi033 2024-05-15T13:55:28.507 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:28 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:28.507 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:28 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:28.507 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:28 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:55:28.507 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:28 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:55:28.507 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:28 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:55:28.507 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:28 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:55:28.508 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:28 smithi052 ceph-mon[36190]: Adjusting osd_memory_target on smithi033 to 4273M 2024-05-15T13:55:28.508 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:28 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:28.508 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:28 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:55:28.508 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:28 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:55:28.508 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:28 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:28.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:28 smithi081 ceph-mon[36434]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:28.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:28 smithi081 ceph-mon[36434]: Detected new or changed devices on smithi033 2024-05-15T13:55:28.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:28 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:28.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:28 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:28.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:28 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:55:28.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:28 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:55:28.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:28 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:55:28.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:28 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:55:28.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:28 smithi081 ceph-mon[36434]: Adjusting osd_memory_target on smithi033 to 4273M 2024-05-15T13:55:28.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:28 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:28.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:28 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:55:28.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:28 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:55:28.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:28 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:28.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:28 smithi033 ceph-mon[31650]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:28.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:28 smithi033 ceph-mon[31650]: Detected new or changed devices on smithi033 2024-05-15T13:55:28.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:28 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:28.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:28 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:28.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:28 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:55:28.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:28 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:55:28.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:28 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:55:28.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:28 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:55:28.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:28 smithi033 ceph-mon[31650]: Adjusting osd_memory_target on smithi033 to 4273M 2024-05-15T13:55:28.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:28 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:28.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:28 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:55:28.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:28 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:55:28.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:28 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:29.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.127+0000 7ff2bdb98700 1 -- 172.21.15.52:0/220659376 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2b81047d0 msgr2=0x7ff2b8104c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:55:29.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.127+0000 7ff2bdb98700 1 --2- 172.21.15.52:0/220659376 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2b81047d0 0x7ff2b8104c30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac0099c0 tx=0x7ff2ac009cd0 comp rx=0 tx=0).stop 2024-05-15T13:55:29.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.128+0000 7ff2bdb98700 1 -- 172.21.15.52:0/220659376 shutdown_connections 2024-05-15T13:55:29.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.128+0000 7ff2bdb98700 1 --2- 172.21.15.52:0/220659376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b8105170 0x7ff2b810ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:55:29.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.128+0000 7ff2bdb98700 1 --2- 172.21.15.52:0/220659376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2b81035d0 0x7ff2b81039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:55:29.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.128+0000 7ff2bdb98700 1 --2- 172.21.15.52:0/220659376 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2b81047d0 0x7ff2b8104c30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:55:29.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.128+0000 7ff2bdb98700 1 -- 172.21.15.52:0/220659376 >> 172.21.15.52:0/220659376 conn(0x7ff2b80feb90 msgr2=0x7ff2b8100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:55:29.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.128+0000 7ff2bdb98700 1 -- 172.21.15.52:0/220659376 shutdown_connections 2024-05-15T13:55:29.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.128+0000 7ff2bdb98700 1 -- 172.21.15.52:0/220659376 wait complete. 2024-05-15T13:55:29.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.128+0000 7ff2bdb98700 1 Processor -- start 2024-05-15T13:55:29.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.128+0000 7ff2bdb98700 1 -- start start 2024-05-15T13:55:29.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.128+0000 7ff2bdb98700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b81035d0 0x7ff2b80772e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:55:29.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.129+0000 7ff2bdb98700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2b81047d0 0x7ff2b8075930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:55:29.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.129+0000 7ff2bdb98700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2b8105170 0x7ff2b8075e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:55:29.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.129+0000 7ff2bdb98700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b810f070 con 0x7ff2b81035d0 2024-05-15T13:55:29.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.129+0000 7ff2bdb98700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b810eef0 con 0x7ff2b81047d0 2024-05-15T13:55:29.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.129+0000 7ff2bdb98700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b810f1f0 con 0x7ff2b8105170 2024-05-15T13:55:29.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.129+0000 7ff2b6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2b81047d0 0x7ff2b8075930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:55:29.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.129+0000 7ff2b6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2b81047d0 0x7ff2b8075930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38164/0 (socket says 172.21.15.52:38164) 2024-05-15T13:55:29.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.129+0000 7ff2b6ffd700 1 -- 172.21.15.52:0/2977753102 learned_addr learned my addr 172.21.15.52:0/2977753102 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-05-15T13:55:29.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.129+0000 7ff2b77fe700 1 --2- 172.21.15.52:0/2977753102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b81035d0 0x7ff2b80772e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:55:29.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.129+0000 7ff2b7fff700 1 --2- 172.21.15.52:0/2977753102 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2b8105170 0x7ff2b8075e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:55:29.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.129+0000 7ff2b6ffd700 1 -- 172.21.15.52:0/2977753102 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2b8105170 msgr2=0x7ff2b8075e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:55:29.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.129+0000 7ff2b6ffd700 1 --2- 172.21.15.52:0/2977753102 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2b8105170 0x7ff2b8075e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:55:29.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.129+0000 7ff2b6ffd700 1 -- 172.21.15.52:0/2977753102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b81035d0 msgr2=0x7ff2b80772e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:55:29.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.129+0000 7ff2b6ffd700 1 --2- 172.21.15.52:0/2977753102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b81035d0 0x7ff2b80772e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:55:29.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.129+0000 7ff2b6ffd700 1 -- 172.21.15.52:0/2977753102 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b8076720 con 0x7ff2b81047d0 2024-05-15T13:55:29.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.129+0000 7ff2b6ffd700 1 --2- 172.21.15.52:0/2977753102 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2b81047d0 0x7ff2b8075930 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac003790 tx=0x7ff2ac005b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:55:29.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.129+0000 7ff2b4ff9700 1 -- 172.21.15.52:0/2977753102 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac027070 con 0x7ff2b81047d0 2024-05-15T13:55:29.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.129+0000 7ff2b4ff9700 1 -- 172.21.15.52:0/2977753102 <== mon.2 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2ac0040e0 con 0x7ff2b81047d0 2024-05-15T13:55:29.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.129+0000 7ff2bdb98700 1 -- 172.21.15.52:0/2977753102 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b81a37e0 con 0x7ff2b81047d0 2024-05-15T13:55:29.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.129+0000 7ff2b4ff9700 1 -- 172.21.15.52:0/2977753102 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac022d60 con 0x7ff2b81047d0 2024-05-15T13:55:29.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.130+0000 7ff2bdb98700 1 -- 172.21.15.52:0/2977753102 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b81a3c90 con 0x7ff2b81047d0 2024-05-15T13:55:29.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.131+0000 7ff2bdb98700 1 -- 172.21.15.52:0/2977753102 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2b806a230 con 0x7ff2b81047d0 2024-05-15T13:55:29.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.132+0000 7ff2b4ff9700 1 -- 172.21.15.52:0/2977753102 <== mon.2 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff2ac02a070 con 0x7ff2b81047d0 2024-05-15T13:55:29.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.133+0000 7ff2b4ff9700 1 --2- 172.21.15.52:0/2977753102 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff2a00b2880 0x7ff2a00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:55:29.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.133+0000 7ff2b4ff9700 1 -- 172.21.15.52:0/2977753102 <== mon.2 v2:172.21.15.52:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6261+0+0 (secure 0 0 0) 0x7ff2ac0e2b90 con 0x7ff2b81047d0 2024-05-15T13:55:29.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.133+0000 7ff2b77fe700 1 --2- 172.21.15.52:0/2977753102 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff2a00b2880 0x7ff2a00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:55:29.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.133+0000 7ff2b77fe700 1 --2- 172.21.15.52:0/2977753102 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff2a00b2880 0x7ff2a00b4d40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8104630 tx=0x7ff2a800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:55:29.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.136+0000 7ff2b4ff9700 1 -- 172.21.15.52:0/2977753102 <== mon.2 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff2ac013070 con 0x7ff2b81047d0 2024-05-15T13:55:29.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:29.366+0000 7ff2bdb98700 1 -- 172.21.15.52:0/2977753102 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ff2b8063980 con 0x7ff2a00b2880 2024-05-15T13:55:29.369 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:29 smithi052 ceph-mon[36190]: osdmap e62: 10 total, 10 up, 10 in 2024-05-15T13:55:29.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:29 smithi081 ceph-mon[36434]: osdmap e62: 10 total, 10 up, 10 in 2024-05-15T13:55:29.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:29 smithi033 ceph-mon[31650]: osdmap e62: 10 total, 10 up, 10 in 2024-05-15T13:55:30.297 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:30 smithi052 ceph-mon[36190]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:30.297 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:30 smithi052 ceph-mon[36190]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:55:30.297 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:30 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:55:30.297 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:30 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:55:30.297 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:30 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:55:30.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:30 smithi081 ceph-mon[36434]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:30.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:30 smithi081 ceph-mon[36434]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:55:30.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:30 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:55:30.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:30 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:55:30.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:30 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:55:30.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:30 smithi033 ceph-mon[31650]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:30.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:30 smithi033 ceph-mon[31650]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:55:30.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:30 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:55:30.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:30 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:55:30.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:30 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:55:32.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:32 smithi052 ceph-mon[36190]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:32.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:32 smithi081 ceph-mon[36434]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:32.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:32 smithi033 ceph-mon[31650]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:33.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:33 smithi052 ceph-mon[36190]: from='client.? 172.21.15.52:0/2812754784' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af58c940-fdc4-418b-a638-7dc44438b34c"}]: dispatch 2024-05-15T13:55:33.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:33 smithi052 ceph-mon[36190]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af58c940-fdc4-418b-a638-7dc44438b34c"}]: dispatch 2024-05-15T13:55:33.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:33 smithi052 ceph-mon[36190]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "af58c940-fdc4-418b-a638-7dc44438b34c"}]': finished 2024-05-15T13:55:33.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:33 smithi052 ceph-mon[36190]: osdmap e63: 11 total, 10 up, 11 in 2024-05-15T13:55:33.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:33 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-15T13:55:33.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:33 smithi052 ceph-mon[36190]: from='client.? 172.21.15.52:0/530949609' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:55:33.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:33 smithi081 ceph-mon[36434]: from='client.? 172.21.15.52:0/2812754784' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af58c940-fdc4-418b-a638-7dc44438b34c"}]: dispatch 2024-05-15T13:55:33.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:33 smithi081 ceph-mon[36434]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af58c940-fdc4-418b-a638-7dc44438b34c"}]: dispatch 2024-05-15T13:55:33.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:33 smithi081 ceph-mon[36434]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "af58c940-fdc4-418b-a638-7dc44438b34c"}]': finished 2024-05-15T13:55:33.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:33 smithi081 ceph-mon[36434]: osdmap e63: 11 total, 10 up, 11 in 2024-05-15T13:55:33.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:33 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-15T13:55:33.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:33 smithi081 ceph-mon[36434]: from='client.? 172.21.15.52:0/530949609' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:55:33.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:33 smithi033 ceph-mon[31650]: from='client.? 172.21.15.52:0/2812754784' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af58c940-fdc4-418b-a638-7dc44438b34c"}]: dispatch 2024-05-15T13:55:33.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:33 smithi033 ceph-mon[31650]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "af58c940-fdc4-418b-a638-7dc44438b34c"}]: dispatch 2024-05-15T13:55:33.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:33 smithi033 ceph-mon[31650]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "af58c940-fdc4-418b-a638-7dc44438b34c"}]': finished 2024-05-15T13:55:33.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:33 smithi033 ceph-mon[31650]: osdmap e63: 11 total, 10 up, 11 in 2024-05-15T13:55:33.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:33 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-15T13:55:33.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:33 smithi033 ceph-mon[31650]: from='client.? 172.21.15.52:0/530949609' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:55:34.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:34 smithi052 ceph-mon[36190]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:34.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:34 smithi081 ceph-mon[36434]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:34.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:34 smithi033 ceph-mon[31650]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:36.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:36 smithi052 ceph-mon[36190]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:36.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:36 smithi081 ceph-mon[36434]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:36.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:36 smithi033 ceph-mon[31650]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:38.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:38 smithi052 ceph-mon[36190]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:38.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:38 smithi081 ceph-mon[36434]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:38.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:38 smithi033 ceph-mon[31650]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:40.502 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:40 smithi052 ceph-mon[36190]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:40.502 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:40 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-15T13:55:40.502 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:40 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:55:40.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:40 smithi081 ceph-mon[36434]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:40.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:40 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-15T13:55:40.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:40 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:55:40.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:40 smithi033 ceph-mon[31650]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:40.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:40 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-15T13:55:40.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:40 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:55:41.506 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:41 smithi052 ceph-mon[36190]: Deploying daemon osd.10 on smithi052 2024-05-15T13:55:41.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:41 smithi081 ceph-mon[36434]: Deploying daemon osd.10 on smithi052 2024-05-15T13:55:41.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:41 smithi033 ceph-mon[31650]: Deploying daemon osd.10 on smithi052 2024-05-15T13:55:42.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:42 smithi052 ceph-mon[36190]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:42.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:42 smithi081 ceph-mon[36434]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:42.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:42 smithi033 ceph-mon[31650]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:44.546 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:44 smithi052 ceph-mon[36190]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:44.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:44 smithi081 ceph-mon[36434]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:44.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:44 smithi033 ceph-mon[31650]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:46.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:46 smithi052 ceph-mon[36190]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:46.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:46 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:46.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:46 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:55:46.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:46 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:46.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:46 smithi081 ceph-mon[36434]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:46.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:46 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:46.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:46 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:55:46.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:46 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:46.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:46 smithi033 ceph-mon[31650]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:46.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:46 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:46.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:46 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:55:46.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:46 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:48.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:48 smithi033 ceph-mon[31650]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:48.332 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:48 smithi052 ceph-mon[36190]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:48.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:48.331+0000 7ff2b4ff9700 1 -- 172.21.15.52:0/2977753102 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7ff2b8063980 con 0x7ff2a00b2880 2024-05-15T13:55:48.334 INFO:teuthology.orchestra.run.smithi052.stdout:Created osd(s) 10 on host 'smithi052' 2024-05-15T13:55:48.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:48.334+0000 7ff2bdb98700 1 -- 172.21.15.52:0/2977753102 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff2a00b2880 msgr2=0x7ff2a00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:55:48.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:48.334+0000 7ff2bdb98700 1 --2- 172.21.15.52:0/2977753102 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff2a00b2880 0x7ff2a00b4d40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8104630 tx=0x7ff2a800b410 comp rx=0 tx=0).stop 2024-05-15T13:55:48.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:48.334+0000 7ff2bdb98700 1 -- 172.21.15.52:0/2977753102 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2b81047d0 msgr2=0x7ff2b8075930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:55:48.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:48.334+0000 7ff2bdb98700 1 --2- 172.21.15.52:0/2977753102 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2b81047d0 0x7ff2b8075930 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac003790 tx=0x7ff2ac005b00 comp rx=0 tx=0).stop 2024-05-15T13:55:48.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:48.334+0000 7ff2bdb98700 1 -- 172.21.15.52:0/2977753102 shutdown_connections 2024-05-15T13:55:48.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:48.334+0000 7ff2bdb98700 1 --2- 172.21.15.52:0/2977753102 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff2a00b2880 0x7ff2a00b4d40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:55:48.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:48.334+0000 7ff2bdb98700 1 --2- 172.21.15.52:0/2977753102 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2b8105170 0x7ff2b8075e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:55:48.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:48.334+0000 7ff2bdb98700 1 --2- 172.21.15.52:0/2977753102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b81035d0 0x7ff2b80772e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:55:48.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:48.334+0000 7ff2bdb98700 1 --2- 172.21.15.52:0/2977753102 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2b81047d0 0x7ff2b8075930 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:55:48.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:48.334+0000 7ff2bdb98700 1 -- 172.21.15.52:0/2977753102 >> 172.21.15.52:0/2977753102 conn(0x7ff2b80feb90 msgr2=0x7ff2b8107b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:55:48.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:48.335+0000 7ff2bdb98700 1 -- 172.21.15.52:0/2977753102 shutdown_connections 2024-05-15T13:55:48.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-05-15T13:55:48.335+0000 7ff2bdb98700 1 -- 172.21.15.52:0/2977753102 wait complete. 2024-05-15T13:55:48.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:48 smithi081 ceph-mon[36434]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:49.019 DEBUG:teuthology.orchestra.run.smithi052:osd.10> sudo journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.10.service 2024-05-15T13:55:49.021 INFO:tasks.cephadm:Deploying osd.11 on smithi081 with /dev/vg_nvme/lv_1... 2024-05-15T13:55:49.022 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-15T13:55:49.269 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.c/config 2024-05-15T13:55:49.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:49 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:49.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:49 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:49.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:49 smithi052 ceph-mon[36190]: from='osd.10 [v2:172.21.15.52:6824/2422349831,v1:172.21.15.52:6825/2422349831]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-15T13:55:49.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:49 smithi052 ceph-mon[36190]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-15T13:55:49.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:49 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:49.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:49 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:49.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:49 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:49.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:49 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:49.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:49 smithi081 ceph-mon[36434]: from='osd.10 [v2:172.21.15.52:6824/2422349831,v1:172.21.15.52:6825/2422349831]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-15T13:55:49.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:49 smithi081 ceph-mon[36434]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-15T13:55:49.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:49 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:49.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:49 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:49.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:49 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:49.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:49 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:49.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:49 smithi033 ceph-mon[31650]: from='osd.10 [v2:172.21.15.52:6824/2422349831,v1:172.21.15.52:6825/2422349831]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-15T13:55:49.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:49 smithi033 ceph-mon[31650]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-15T13:55:49.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:49 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:49.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:49 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:50.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:50 smithi052 ceph-mon[36190]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:50.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:50 smithi052 ceph-mon[36190]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-15T13:55:50.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:50 smithi052 ceph-mon[36190]: from='osd.10 [v2:172.21.15.52:6824/2422349831,v1:172.21.15.52:6825/2422349831]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-05-15T13:55:50.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:50 smithi052 ceph-mon[36190]: osdmap e64: 11 total, 10 up, 11 in 2024-05-15T13:55:50.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:50 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-15T13:55:50.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:50 smithi052 ceph-mon[36190]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-05-15T13:55:50.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:50 smithi081 ceph-mon[36434]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:50.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:50 smithi081 ceph-mon[36434]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-15T13:55:50.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:50 smithi081 ceph-mon[36434]: from='osd.10 [v2:172.21.15.52:6824/2422349831,v1:172.21.15.52:6825/2422349831]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-05-15T13:55:50.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:50 smithi081 ceph-mon[36434]: osdmap e64: 11 total, 10 up, 11 in 2024-05-15T13:55:50.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:50 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-15T13:55:50.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:50 smithi081 ceph-mon[36434]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-05-15T13:55:50.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:50 smithi033 ceph-mon[31650]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:50.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:50 smithi033 ceph-mon[31650]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-15T13:55:50.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:50 smithi033 ceph-mon[31650]: from='osd.10 [v2:172.21.15.52:6824/2422349831,v1:172.21.15.52:6825/2422349831]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-05-15T13:55:50.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:50 smithi033 ceph-mon[31650]: osdmap e64: 11 total, 10 up, 11 in 2024-05-15T13:55:50.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:50 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-15T13:55:50.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:50 smithi033 ceph-mon[31650]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-05-15T13:55:51.317 INFO:teuthology.orchestra.run.smithi081.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-15T13:55:51.318 INFO:teuthology.orchestra.run.smithi081.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-15T13:55:51.318 INFO:teuthology.orchestra.run.smithi081.stderr: stderr: 10+0 records in 2024-05-15T13:55:51.318 INFO:teuthology.orchestra.run.smithi081.stderr:10+0 records out 2024-05-15T13:55:51.318 INFO:teuthology.orchestra.run.smithi081.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0117336 s, 894 MB/s 2024-05-15T13:55:51.318 INFO:teuthology.orchestra.run.smithi081.stderr: stderr: 2024-05-15T13:55:51.318 INFO:teuthology.orchestra.run.smithi081.stderr:--> Zapping successful for: 2024-05-15T13:55:51.338 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 13:55:51 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T13:55:51.073+0000 7fea2e164700 -1 osd.10 0 waiting for initial osdmap 2024-05-15T13:55:51.338 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 13:55:51 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T13:55:51.087+0000 7fea25f4f700 -1 osd.10 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-15T13:55:51.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:51 smithi052 ceph-mon[36190]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-05-15T13:55:51.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:51 smithi052 ceph-mon[36190]: osdmap e65: 11 total, 10 up, 11 in 2024-05-15T13:55:51.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:51 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-15T13:55:51.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:51 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-15T13:55:51.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:51 smithi052 ceph-mon[36190]: from='osd.10 ' entity='osd.10' 2024-05-15T13:55:51.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:51 smithi081 ceph-mon[36434]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-05-15T13:55:51.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:51 smithi081 ceph-mon[36434]: osdmap e65: 11 total, 10 up, 11 in 2024-05-15T13:55:51.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:51 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-15T13:55:51.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:51 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-15T13:55:51.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:51 smithi081 ceph-mon[36434]: from='osd.10 ' entity='osd.10' 2024-05-15T13:55:51.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:51 smithi033 ceph-mon[31650]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-05-15T13:55:51.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:51 smithi033 ceph-mon[31650]: osdmap e65: 11 total, 10 up, 11 in 2024-05-15T13:55:51.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:51 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-15T13:55:51.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:51 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-15T13:55:51.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:51 smithi033 ceph-mon[31650]: from='osd.10 ' entity='osd.10' 2024-05-15T13:55:51.938 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi081:vg_nvme/lv_1 2024-05-15T13:55:52.183 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.c/config 2024-05-15T13:55:52.549 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:52 smithi052 ceph-mon[36190]: purged_snaps scrub starts 2024-05-15T13:55:52.549 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:52 smithi052 ceph-mon[36190]: purged_snaps scrub ok 2024-05-15T13:55:52.549 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:52 smithi052 ceph-mon[36190]: pgmap v228: 1 pgs: 1 peering; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:52.550 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:52 smithi052 ceph-mon[36190]: osd.10 [v2:172.21.15.52:6824/2422349831,v1:172.21.15.52:6825/2422349831] boot 2024-05-15T13:55:52.550 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:52 smithi052 ceph-mon[36190]: osdmap e66: 11 total, 11 up, 11 in 2024-05-15T13:55:52.550 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:52 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-15T13:55:52.550 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:52 smithi052 ceph-mon[36190]: osdmap e67: 11 total, 11 up, 11 in 2024-05-15T13:55:52.586 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:52 smithi081 ceph-mon[36434]: purged_snaps scrub starts 2024-05-15T13:55:52.587 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:52 smithi081 ceph-mon[36434]: purged_snaps scrub ok 2024-05-15T13:55:52.587 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:52 smithi081 ceph-mon[36434]: pgmap v228: 1 pgs: 1 peering; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:52.587 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:52 smithi081 ceph-mon[36434]: osd.10 [v2:172.21.15.52:6824/2422349831,v1:172.21.15.52:6825/2422349831] boot 2024-05-15T13:55:52.587 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:52 smithi081 ceph-mon[36434]: osdmap e66: 11 total, 11 up, 11 in 2024-05-15T13:55:52.587 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:52 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-15T13:55:52.587 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:52 smithi081 ceph-mon[36434]: osdmap e67: 11 total, 11 up, 11 in 2024-05-15T13:55:52.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:52 smithi033 ceph-mon[31650]: purged_snaps scrub starts 2024-05-15T13:55:52.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:52 smithi033 ceph-mon[31650]: purged_snaps scrub ok 2024-05-15T13:55:52.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:52 smithi033 ceph-mon[31650]: pgmap v228: 1 pgs: 1 peering; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-15T13:55:52.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:52 smithi033 ceph-mon[31650]: osd.10 [v2:172.21.15.52:6824/2422349831,v1:172.21.15.52:6825/2422349831] boot 2024-05-15T13:55:52.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:52 smithi033 ceph-mon[31650]: osdmap e66: 11 total, 11 up, 11 in 2024-05-15T13:55:52.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:52 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-15T13:55:52.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:52 smithi033 ceph-mon[31650]: osdmap e67: 11 total, 11 up, 11 in 2024-05-15T13:55:53.731 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.727+0000 7fbf232af700 1 -- 172.21.15.81:0/723656935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf1c1036a0 msgr2=0x7fbf1c103b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:55:53.731 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.727+0000 7fbf232af700 1 --2- 172.21.15.81:0/723656935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf1c1036a0 0x7fbf1c103b00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c009990 tx=0x7fbf0c009ca0 comp rx=0 tx=0).stop 2024-05-15T13:55:53.731 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.727+0000 7fbf232af700 1 -- 172.21.15.81:0/723656935 shutdown_connections 2024-05-15T13:55:53.731 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.727+0000 7fbf232af700 1 --2- 172.21.15.81:0/723656935 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf1c104040 0x7fbf1c10a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:55:53.731 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.727+0000 7fbf232af700 1 --2- 172.21.15.81:0/723656935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf1c1024a0 0x7fbf1c1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:55:53.732 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.727+0000 7fbf232af700 1 --2- 172.21.15.81:0/723656935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf1c1036a0 0x7fbf1c103b00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:55:53.732 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.727+0000 7fbf232af700 1 -- 172.21.15.81:0/723656935 >> 172.21.15.81:0/723656935 conn(0x7fbf1c0fda40 msgr2=0x7fbf1c0ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:55:53.732 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.727+0000 7fbf232af700 1 -- 172.21.15.81:0/723656935 shutdown_connections 2024-05-15T13:55:53.732 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.727+0000 7fbf232af700 1 -- 172.21.15.81:0/723656935 wait complete. 2024-05-15T13:55:53.732 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.728+0000 7fbf232af700 1 Processor -- start 2024-05-15T13:55:53.732 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.728+0000 7fbf232af700 1 -- start start 2024-05-15T13:55:53.732 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.728+0000 7fbf232af700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf1c1024a0 0x7fbf1c199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:55:53.732 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.728+0000 7fbf232af700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf1c1036a0 0x7fbf1c199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:55:53.732 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.728+0000 7fbf232af700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf1c104040 0x7fbf1c1a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:55:53.732 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.728+0000 7fbf232af700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1c10df40 con 0x7fbf1c1024a0 2024-05-15T13:55:53.732 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.728+0000 7fbf232af700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1c10ddc0 con 0x7fbf1c104040 2024-05-15T13:55:53.732 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.728+0000 7fbf232af700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1c10e0c0 con 0x7fbf1c1036a0 2024-05-15T13:55:53.732 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.728+0000 7fbf2084a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf1c1036a0 0x7fbf1c199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:55:53.732 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.728+0000 7fbf2084a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf1c1036a0 0x7fbf1c199cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51274/0 (socket says 172.21.15.81:51274) 2024-05-15T13:55:53.733 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.728+0000 7fbf2084a700 1 -- 172.21.15.81:0/1927375782 learned_addr learned my addr 172.21.15.81:0/1927375782 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-15T13:55:53.733 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.728+0000 7fbf2104b700 1 --2- 172.21.15.81:0/1927375782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf1c1024a0 0x7fbf1c199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:55:53.733 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.728+0000 7fbf2184c700 1 --2- 172.21.15.81:0/1927375782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf1c104040 0x7fbf1c1a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:55:53.733 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.728+0000 7fbf2084a700 1 -- 172.21.15.81:0/1927375782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf1c104040 msgr2=0x7fbf1c1a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:55:53.733 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.728+0000 7fbf2084a700 1 --2- 172.21.15.81:0/1927375782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf1c104040 0x7fbf1c1a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:55:53.733 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.728+0000 7fbf2084a700 1 -- 172.21.15.81:0/1927375782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf1c1024a0 msgr2=0x7fbf1c199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:55:53.733 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.728+0000 7fbf2084a700 1 --2- 172.21.15.81:0/1927375782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf1c1024a0 0x7fbf1c199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:55:53.733 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.729+0000 7fbf2084a700 1 -- 172.21.15.81:0/1927375782 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf1c1a1380 con 0x7fbf1c1036a0 2024-05-15T13:55:53.733 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.729+0000 7fbf2084a700 1 --2- 172.21.15.81:0/1927375782 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf1c1036a0 0x7fbf1c199cb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c0098e0 tx=0x7fbf0c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:55:53.733 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.729+0000 7fbf127fc700 1 -- 172.21.15.81:0/1927375782 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c027070 con 0x7fbf1c1036a0 2024-05-15T13:55:53.733 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.729+0000 7fbf232af700 1 -- 172.21.15.81:0/1927375782 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf1c1a1610 con 0x7fbf1c1036a0 2024-05-15T13:55:53.733 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.729+0000 7fbf127fc700 1 -- 172.21.15.81:0/1927375782 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf0c00be70 con 0x7fbf1c1036a0 2024-05-15T13:55:53.733 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.729+0000 7fbf127fc700 1 -- 172.21.15.81:0/1927375782 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c017650 con 0x7fbf1c1036a0 2024-05-15T13:55:53.733 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.729+0000 7fbf232af700 1 -- 172.21.15.81:0/1927375782 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf1c1a1a60 con 0x7fbf1c1036a0 2024-05-15T13:55:53.734 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.730+0000 7fbf232af700 1 -- 172.21.15.81:0/1927375782 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf1c06a230 con 0x7fbf1c1036a0 2024-05-15T13:55:53.734 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.731+0000 7fbf127fc700 1 -- 172.21.15.81:0/1927375782 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fbf0c02a070 con 0x7fbf1c1036a0 2024-05-15T13:55:53.734 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.732+0000 7fbf127fc700 1 --2- 172.21.15.81:0/1927375782 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbf080b2880 0x7fbf080b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:55:53.734 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.732+0000 7fbf127fc700 1 -- 172.21.15.81:0/1927375782 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6694+0+0 (secure 0 0 0) 0x7fbf0c0e30b0 con 0x7fbf1c1036a0 2024-05-15T13:55:53.735 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.732+0000 7fbf2104b700 1 --2- 172.21.15.81:0/1927375782 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbf080b2880 0x7fbf080b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:55:53.735 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.732+0000 7fbf2104b700 1 --2- 172.21.15.81:0/1927375782 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbf080b2880 0x7fbf080b4d40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c103500 tx=0x7fbf1800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:55:53.737 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.735+0000 7fbf127fc700 1 -- 172.21.15.81:0/1927375782 <== mon.1 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbf0c031070 con 0x7fbf1c1036a0 2024-05-15T13:55:53.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:53 smithi081 ceph-mon[36434]: Detected new or changed devices on smithi052 2024-05-15T13:55:53.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:53 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:53.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:53 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:53.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:53 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:55:53.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:53 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:55:53.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:53 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:55:53.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:53 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:55:53.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:53 smithi081 ceph-mon[36434]: Adjusting osd_memory_target on smithi052 to 4273M 2024-05-15T13:55:53.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:53 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:53.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:53 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:55:53.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:53 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:55:53.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:53 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:53.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:53 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:55:53.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:53 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:55:53.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:53 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:55:53.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:53 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:53.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:53 smithi081 ceph-mon[36434]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 727 MiB used, 983 GiB / 983 GiB avail 2024-05-15T13:55:53.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:53 smithi081 ceph-mon[36434]: osdmap e68: 11 total, 11 up, 11 in 2024-05-15T13:55:53.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:53 smithi033 ceph-mon[31650]: Detected new or changed devices on smithi052 2024-05-15T13:55:53.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:53 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:53.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:53 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:53.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:53 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:55:53.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:53 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:55:53.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:53 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:55:53.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:53 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:55:53.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:53 smithi033 ceph-mon[31650]: Adjusting osd_memory_target on smithi052 to 4273M 2024-05-15T13:55:53.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:53 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:53.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:53 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:55:53.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:53 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:55:53.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:53 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:53.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:53 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:55:53.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:53 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:55:53.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:53 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:55:53.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:53 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:53.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:53 smithi033 ceph-mon[31650]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 727 MiB used, 983 GiB / 983 GiB avail 2024-05-15T13:55:53.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:53 smithi033 ceph-mon[31650]: osdmap e68: 11 total, 11 up, 11 in 2024-05-15T13:55:53.949 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:55:53.947+0000 7fbf232af700 1 -- 172.21.15.81:0/1927375782 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fbf1c06b930 con 0x7fbf080b2880 2024-05-15T13:55:54.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:53 smithi052 ceph-mon[36190]: Detected new or changed devices on smithi052 2024-05-15T13:55:54.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:53 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:54.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:53 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:54.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:53 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:55:54.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:53 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:55:54.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:53 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:55:54.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:53 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:55:54.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:53 smithi052 ceph-mon[36190]: Adjusting osd_memory_target on smithi052 to 4273M 2024-05-15T13:55:54.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:53 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:54.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:53 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:55:54.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:53 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:55:54.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:53 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:54.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:53 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:55:54.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:53 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:55:54.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:53 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:55:54.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:53 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:55:54.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:53 smithi052 ceph-mon[36190]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 727 MiB used, 983 GiB / 983 GiB avail 2024-05-15T13:55:54.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:53 smithi052 ceph-mon[36190]: osdmap e68: 11 total, 11 up, 11 in 2024-05-15T13:55:55.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:54 smithi081 ceph-mon[36434]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:55:55.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:54 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:55:55.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:54 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:55:55.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:54 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:55:55.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:54 smithi033 ceph-mon[31650]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:55:55.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:54 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:55:55.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:54 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:55:55.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:54 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:55:55.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:54 smithi052 ceph-mon[36190]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:55:55.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:54 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-15T13:55:55.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:54 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-15T13:55:55.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:54 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:55:56.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:55 smithi081 ceph-mon[36434]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 727 MiB used, 983 GiB / 983 GiB avail 2024-05-15T13:55:56.190 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:55 smithi033 ceph-mon[31650]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 727 MiB used, 983 GiB / 983 GiB avail 2024-05-15T13:55:56.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:55 smithi052 ceph-mon[36190]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 727 MiB used, 983 GiB / 983 GiB avail 2024-05-15T13:55:57.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:56 smithi081 ceph-mon[36434]: from='client.? 172.21.15.81:0/3171683422' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab84aaea-2086-4842-8bf4-628842fce46a"}]: dispatch 2024-05-15T13:55:57.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:56 smithi081 ceph-mon[36434]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab84aaea-2086-4842-8bf4-628842fce46a"}]: dispatch 2024-05-15T13:55:57.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:56 smithi081 ceph-mon[36434]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab84aaea-2086-4842-8bf4-628842fce46a"}]': finished 2024-05-15T13:55:57.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:56 smithi081 ceph-mon[36434]: osdmap e69: 12 total, 11 up, 12 in 2024-05-15T13:55:57.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:56 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-15T13:55:57.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:56 smithi033 ceph-mon[31650]: from='client.? 172.21.15.81:0/3171683422' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab84aaea-2086-4842-8bf4-628842fce46a"}]: dispatch 2024-05-15T13:55:57.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:56 smithi033 ceph-mon[31650]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab84aaea-2086-4842-8bf4-628842fce46a"}]: dispatch 2024-05-15T13:55:57.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:56 smithi033 ceph-mon[31650]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab84aaea-2086-4842-8bf4-628842fce46a"}]': finished 2024-05-15T13:55:57.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:56 smithi033 ceph-mon[31650]: osdmap e69: 12 total, 11 up, 12 in 2024-05-15T13:55:57.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:56 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-15T13:55:57.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:56 smithi052 ceph-mon[36190]: from='client.? 172.21.15.81:0/3171683422' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab84aaea-2086-4842-8bf4-628842fce46a"}]: dispatch 2024-05-15T13:55:57.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:56 smithi052 ceph-mon[36190]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab84aaea-2086-4842-8bf4-628842fce46a"}]: dispatch 2024-05-15T13:55:57.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:56 smithi052 ceph-mon[36190]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab84aaea-2086-4842-8bf4-628842fce46a"}]': finished 2024-05-15T13:55:57.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:56 smithi052 ceph-mon[36190]: osdmap e69: 12 total, 11 up, 12 in 2024-05-15T13:55:57.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:56 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-15T13:55:58.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:57 smithi033 ceph-mon[31650]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-05-15T13:55:58.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:55:57 smithi033 ceph-mon[31650]: from='client.? 172.21.15.81:0/2884332137' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:55:58.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:57 smithi052 ceph-mon[36190]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-05-15T13:55:58.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:55:57 smithi052 ceph-mon[36190]: from='client.? 172.21.15.81:0/2884332137' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:55:58.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:57 smithi081 ceph-mon[36434]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-05-15T13:55:58.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:55:57 smithi081 ceph-mon[36434]: from='client.? 172.21.15.81:0/2884332137' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-15T13:56:00.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:00 smithi052 ceph-mon[36190]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-05-15T13:56:00.356 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:00 smithi081 ceph-mon[36434]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-05-15T13:56:00.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:00 smithi033 ceph-mon[31650]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-05-15T13:56:02.285 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:02 smithi081 ceph-mon[36434]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-15T13:56:02.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:02 smithi052 ceph-mon[36190]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-15T13:56:02.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:02 smithi033 ceph-mon[31650]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-15T13:56:04.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:04 smithi052 ceph-mon[36190]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-15T13:56:04.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:04 smithi081 ceph-mon[36434]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-15T13:56:04.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:04 smithi033 ceph-mon[31650]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-15T13:56:05.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:05 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-15T13:56:05.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:05 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:05.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:05 smithi052 ceph-mon[36190]: Deploying daemon osd.11 on smithi081 2024-05-15T13:56:05.341 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:05 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-15T13:56:05.341 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:05 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:05.341 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:05 smithi081 ceph-mon[36434]: Deploying daemon osd.11 on smithi081 2024-05-15T13:56:05.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:05 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-15T13:56:05.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:05 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:05.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:05 smithi033 ceph-mon[31650]: Deploying daemon osd.11 on smithi081 2024-05-15T13:56:06.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:06 smithi052 ceph-mon[36190]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-15T13:56:06.342 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:06 smithi033 ceph-mon[31650]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-15T13:56:06.346 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:06 smithi081 ceph-mon[36434]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-15T13:56:08.281 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:08 smithi081 ceph-mon[36434]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-15T13:56:08.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:08 smithi052 ceph-mon[36190]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-15T13:56:08.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:08 smithi033 ceph-mon[31650]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-15T13:56:10.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:10 smithi052 ceph-mon[36190]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-15T13:56:10.356 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:10 smithi081 ceph-mon[36434]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-15T13:56:10.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:10 smithi033 ceph-mon[31650]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-15T13:56:11.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:11 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:11.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:11 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:56:11.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:11 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:11.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:11 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:11.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:11 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:56:11.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:11 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:11.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:11 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:11.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:11 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:56:11.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:11 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:12.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:12 smithi052 ceph-mon[36190]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-15T13:56:12.356 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:12 smithi081 ceph-mon[36434]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-15T13:56:12.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:12 smithi033 ceph-mon[31650]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-15T13:56:14.071 INFO:teuthology.orchestra.run.smithi081.stdout:Created osd(s) 11 on host 'smithi081' 2024-05-15T13:56:14.071 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:14.069+0000 7fbf127fc700 1 -- 172.21.15.81:0/1927375782 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fbf1c06b930 con 0x7fbf080b2880 2024-05-15T13:56:14.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:14.071+0000 7fbf232af700 1 -- 172.21.15.81:0/1927375782 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbf080b2880 msgr2=0x7fbf080b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:14.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:14.071+0000 7fbf232af700 1 --2- 172.21.15.81:0/1927375782 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbf080b2880 0x7fbf080b4d40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c103500 tx=0x7fbf1800b410 comp rx=0 tx=0).stop 2024-05-15T13:56:14.073 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:14.071+0000 7fbf232af700 1 -- 172.21.15.81:0/1927375782 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf1c1036a0 msgr2=0x7fbf1c199cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:14.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:14.071+0000 7fbf232af700 1 --2- 172.21.15.81:0/1927375782 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf1c1036a0 0x7fbf1c199cb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c0098e0 tx=0x7fbf0c00f690 comp rx=0 tx=0).stop 2024-05-15T13:56:14.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:14.072+0000 7fbf232af700 1 -- 172.21.15.81:0/1927375782 shutdown_connections 2024-05-15T13:56:14.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:14.072+0000 7fbf232af700 1 --2- 172.21.15.81:0/1927375782 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbf080b2880 0x7fbf080b4d40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:14.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:14.072+0000 7fbf232af700 1 --2- 172.21.15.81:0/1927375782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf1c104040 0x7fbf1c1a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:14.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:14.072+0000 7fbf232af700 1 --2- 172.21.15.81:0/1927375782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf1c1024a0 0x7fbf1c199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:14.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:14.072+0000 7fbf232af700 1 --2- 172.21.15.81:0/1927375782 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf1c1036a0 0x7fbf1c199cb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:14.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:14.072+0000 7fbf232af700 1 -- 172.21.15.81:0/1927375782 >> 172.21.15.81:0/1927375782 conn(0x7fbf1c0fda40 msgr2=0x7fbf1c06b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:56:14.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:14.072+0000 7fbf232af700 1 -- 172.21.15.81:0/1927375782 shutdown_connections 2024-05-15T13:56:14.074 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:14.072+0000 7fbf232af700 1 -- 172.21.15.81:0/1927375782 wait complete. 2024-05-15T13:56:14.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:14 smithi052 ceph-mon[36190]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-15T13:56:14.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:14 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:14.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:14 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:14.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:14 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:14.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:14 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:56:14.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:14 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:14.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:14 smithi052 ceph-mon[36190]: from='osd.11 [v2:172.21.15.81:6824/326313090,v1:172.21.15.81:6825/326313090]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-15T13:56:14.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:14 smithi052 ceph-mon[36190]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-15T13:56:14.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:14 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:14.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:14 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:14.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:14 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:56:14.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:14 smithi081 ceph-mon[36434]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-15T13:56:14.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:14 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:14.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:14 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:14.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:14 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:14.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:14 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:56:14.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:14 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:14.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:14 smithi081 ceph-mon[36434]: from='osd.11 [v2:172.21.15.81:6824/326313090,v1:172.21.15.81:6825/326313090]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-15T13:56:14.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:14 smithi081 ceph-mon[36434]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-15T13:56:14.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:14 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:14.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:14 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:14.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:14 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:56:14.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:14 smithi033 ceph-mon[31650]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-15T13:56:14.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:14 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:14.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:14 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:14.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:14 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:14.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:14 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:56:14.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:14 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:14.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:14 smithi033 ceph-mon[31650]: from='osd.11 [v2:172.21.15.81:6824/326313090,v1:172.21.15.81:6825/326313090]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-15T13:56:14.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:14 smithi033 ceph-mon[31650]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-15T13:56:14.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:14 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:14.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:14 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:14.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:14 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:56:14.688 DEBUG:teuthology.orchestra.run.smithi081:osd.11> sudo journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.11.service 2024-05-15T13:56:14.690 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-15T13:56:14.690 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph osd stat -f json 2024-05-15T13:56:14.951 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:56:15.338 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:15 smithi033 ceph-mon[31650]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-15T13:56:15.338 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:15 smithi033 ceph-mon[31650]: osdmap e70: 12 total, 11 up, 12 in 2024-05-15T13:56:15.338 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:15 smithi033 ceph-mon[31650]: from='osd.11 [v2:172.21.15.81:6824/326313090,v1:172.21.15.81:6825/326313090]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-05-15T13:56:15.338 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:15 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-15T13:56:15.338 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:15 smithi033 ceph-mon[31650]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-05-15T13:56:15.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:15 smithi081 ceph-mon[36434]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-15T13:56:15.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:15 smithi081 ceph-mon[36434]: osdmap e70: 12 total, 11 up, 12 in 2024-05-15T13:56:15.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:15 smithi081 ceph-mon[36434]: from='osd.11 [v2:172.21.15.81:6824/326313090,v1:172.21.15.81:6825/326313090]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-05-15T13:56:15.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:15 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-15T13:56:15.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:15 smithi081 ceph-mon[36434]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-05-15T13:56:15.358 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 13:56:15 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T13:56:15.100+0000 7ff46a024700 -1 osd.11 0 waiting for initial osdmap 2024-05-15T13:56:15.358 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 13:56:15 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T13:56:15.113+0000 7ff465e17700 -1 osd.11 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-15T13:56:15.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:15 smithi052 ceph-mon[36190]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-15T13:56:15.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:15 smithi052 ceph-mon[36190]: osdmap e70: 12 total, 11 up, 12 in 2024-05-15T13:56:15.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:15 smithi052 ceph-mon[36190]: from='osd.11 [v2:172.21.15.81:6824/326313090,v1:172.21.15.81:6825/326313090]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-05-15T13:56:15.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:15 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-15T13:56:15.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:15 smithi052 ceph-mon[36190]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-05-15T13:56:16.194 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:16 smithi081 ceph-mon[36434]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-15T13:56:16.195 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:16 smithi081 ceph-mon[36434]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-05-15T13:56:16.195 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:16 smithi081 ceph-mon[36434]: osdmap e71: 12 total, 11 up, 12 in 2024-05-15T13:56:16.195 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:16 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-15T13:56:16.195 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:16 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-15T13:56:16.362 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:16 smithi033 ceph-mon[31650]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-15T13:56:16.362 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:16 smithi033 ceph-mon[31650]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-05-15T13:56:16.362 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:16 smithi033 ceph-mon[31650]: osdmap e71: 12 total, 11 up, 12 in 2024-05-15T13:56:16.362 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:16 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-15T13:56:16.362 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:16 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-15T13:56:16.492 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:16 smithi052 ceph-mon[36190]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-15T13:56:16.492 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:16 smithi052 ceph-mon[36190]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-05-15T13:56:16.492 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:16 smithi052 ceph-mon[36190]: osdmap e71: 12 total, 11 up, 12 in 2024-05-15T13:56:16.492 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:16 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-15T13:56:16.492 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:16 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-15T13:56:16.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.718+0000 7fdd1707d700 1 -- 172.21.15.33:0/2265864652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd10103590 msgr2=0x7fdd101039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:16.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.718+0000 7fdd1707d700 1 --2- 172.21.15.33:0/2265864652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd10103590 0x7fdd101039b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdd040099c0 tx=0x7fdd04009cd0 comp rx=0 tx=0).stop 2024-05-15T13:56:16.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.718+0000 7fdd1707d700 1 -- 172.21.15.33:0/2265864652 shutdown_connections 2024-05-15T13:56:16.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.718+0000 7fdd1707d700 1 --2- 172.21.15.33:0/2265864652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd10105130 0x7fdd1010b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:16.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.718+0000 7fdd1707d700 1 --2- 172.21.15.33:0/2265864652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd10103590 0x7fdd101039b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:16.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.718+0000 7fdd1707d700 1 --2- 172.21.15.33:0/2265864652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd10104790 0x7fdd10104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:16.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.718+0000 7fdd1707d700 1 -- 172.21.15.33:0/2265864652 >> 172.21.15.33:0/2265864652 conn(0x7fdd100feb30 msgr2=0x7fdd10100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:56:16.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.718+0000 7fdd1707d700 1 -- 172.21.15.33:0/2265864652 shutdown_connections 2024-05-15T13:56:16.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.718+0000 7fdd1707d700 1 -- 172.21.15.33:0/2265864652 wait complete. 2024-05-15T13:56:16.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.719+0000 7fdd1707d700 1 Processor -- start 2024-05-15T13:56:16.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.719+0000 7fdd1707d700 1 -- start start 2024-05-15T13:56:16.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.719+0000 7fdd1707d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd10103590 0x7fdd101996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:16.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.719+0000 7fdd1707d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd10104790 0x7fdd10199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:16.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.719+0000 7fdd1707d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd10105130 0x7fdd101a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:16.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.719+0000 7fdd1707d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1010f030 con 0x7fdd10104790 2024-05-15T13:56:16.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.719+0000 7fdd1707d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1010eeb0 con 0x7fdd10103590 2024-05-15T13:56:16.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.719+0000 7fdd1707d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1010f1b0 con 0x7fdd10105130 2024-05-15T13:56:16.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.719+0000 7fdd0ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd10104790 0x7fdd10199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:16.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.719+0000 7fdd14e19700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd10103590 0x7fdd101996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:16.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.719+0000 7fdd0ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd10104790 0x7fdd10199c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56720/0 (socket says 172.21.15.33:56720) 2024-05-15T13:56:16.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.719+0000 7fdd0ffff700 1 -- 172.21.15.33:0/194813301 learned_addr learned my addr 172.21.15.33:0/194813301 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:56:16.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.719+0000 7fdd1561a700 1 --2- 172.21.15.33:0/194813301 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd10105130 0x7fdd101a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:16.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.720+0000 7fdd14e19700 1 -- 172.21.15.33:0/194813301 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd10105130 msgr2=0x7fdd101a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:16.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.720+0000 7fdd14e19700 1 --2- 172.21.15.33:0/194813301 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd10105130 0x7fdd101a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:16.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.720+0000 7fdd14e19700 1 -- 172.21.15.33:0/194813301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd10104790 msgr2=0x7fdd10199c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:16.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.720+0000 7fdd14e19700 1 --2- 172.21.15.33:0/194813301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd10104790 0x7fdd10199c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:16.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.720+0000 7fdd14e19700 1 -- 172.21.15.33:0/194813301 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd101a1330 con 0x7fdd10103590 2024-05-15T13:56:16.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.720+0000 7fdd0ffff700 1 --2- 172.21.15.33:0/194813301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd10104790 0x7fdd10199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T13:56:16.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.720+0000 7fdd1561a700 1 --2- 172.21.15.33:0/194813301 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd10105130 0x7fdd101a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T13:56:16.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.720+0000 7fdd14e19700 1 --2- 172.21.15.33:0/194813301 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd10103590 0x7fdd101996d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdd040036f0 tx=0x7fdd0400f710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:56:16.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.720+0000 7fdd0dffb700 1 -- 172.21.15.33:0/194813301 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd04009e60 con 0x7fdd10103590 2024-05-15T13:56:16.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.720+0000 7fdd0dffb700 1 -- 172.21.15.33:0/194813301 <== mon.2 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd04005940 con 0x7fdd10103590 2024-05-15T13:56:16.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.720+0000 7fdd1707d700 1 -- 172.21.15.33:0/194813301 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd101a15c0 con 0x7fdd10103590 2024-05-15T13:56:16.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.720+0000 7fdd0dffb700 1 -- 172.21.15.33:0/194813301 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd04005450 con 0x7fdd10103590 2024-05-15T13:56:16.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.720+0000 7fdd1707d700 1 -- 172.21.15.33:0/194813301 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd101a1aa0 con 0x7fdd10103590 2024-05-15T13:56:16.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.721+0000 7fdd1707d700 1 -- 172.21.15.33:0/194813301 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd1006a230 con 0x7fdd10103590 2024-05-15T13:56:16.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.722+0000 7fdd0dffb700 1 -- 172.21.15.33:0/194813301 <== mon.2 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fdd04003ac0 con 0x7fdd10103590 2024-05-15T13:56:16.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.723+0000 7fdd0dffb700 1 --2- 172.21.15.33:0/194813301 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fdd000b2880 0x7fdd000b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:16.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.723+0000 7fdd0ffff700 1 --2- 172.21.15.33:0/194813301 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fdd000b2880 0x7fdd000b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:16.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.723+0000 7fdd0dffb700 1 -- 172.21.15.33:0/194813301 <== mon.2 v2:172.21.15.52:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fdd040e2c60 con 0x7fdd10103590 2024-05-15T13:56:16.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.724+0000 7fdd0ffff700 1 --2- 172.21.15.33:0/194813301 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fdd000b2880 0x7fdd000b4d40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc011550 tx=0x7fdcfc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:56:16.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.727+0000 7fdd0dffb700 1 -- 172.21.15.33:0/194813301 <== mon.2 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdd04013070 con 0x7fdd10103590 2024-05-15T13:56:16.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.946+0000 7fdd1707d700 1 -- 172.21.15.33:0/194813301 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fdd10064150 con 0x7fdd10103590 2024-05-15T13:56:16.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.947+0000 7fdd0dffb700 1 -- 172.21.15.33:0/194813301 <== mon.2 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7fdd040038c0 con 0x7fdd10103590 2024-05-15T13:56:16.949 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:56:16.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.949+0000 7fdd1707d700 1 -- 172.21.15.33:0/194813301 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fdd000b2880 msgr2=0x7fdd000b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:16.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.949+0000 7fdd1707d700 1 --2- 172.21.15.33:0/194813301 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fdd000b2880 0x7fdd000b4d40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc011550 tx=0x7fdcfc009260 comp rx=0 tx=0).stop 2024-05-15T13:56:16.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.949+0000 7fdd1707d700 1 -- 172.21.15.33:0/194813301 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd10103590 msgr2=0x7fdd101996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:16.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.949+0000 7fdd1707d700 1 --2- 172.21.15.33:0/194813301 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd10103590 0x7fdd101996d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdd040036f0 tx=0x7fdd0400f710 comp rx=0 tx=0).stop 2024-05-15T13:56:16.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.949+0000 7fdd1707d700 1 -- 172.21.15.33:0/194813301 shutdown_connections 2024-05-15T13:56:16.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.949+0000 7fdd1707d700 1 --2- 172.21.15.33:0/194813301 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fdd000b2880 0x7fdd000b4d40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:16.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.949+0000 7fdd1707d700 1 --2- 172.21.15.33:0/194813301 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd10105130 0x7fdd101a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:16.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.949+0000 7fdd1707d700 1 --2- 172.21.15.33:0/194813301 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdd10103590 0x7fdd101996d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:16.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.949+0000 7fdd1707d700 1 --2- 172.21.15.33:0/194813301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd10104790 0x7fdd10199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:16.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.949+0000 7fdd1707d700 1 -- 172.21.15.33:0/194813301 >> 172.21.15.33:0/194813301 conn(0x7fdd100feb30 msgr2=0x7fdd10107ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:56:16.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.949+0000 7fdd1707d700 1 -- 172.21.15.33:0/194813301 shutdown_connections 2024-05-15T13:56:16.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:16.949+0000 7fdd1707d700 1 -- 172.21.15.33:0/194813301 wait complete. 2024-05-15T13:56:17.112 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:17 smithi081 ceph-mon[36434]: purged_snaps scrub starts 2024-05-15T13:56:17.112 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:17 smithi081 ceph-mon[36434]: purged_snaps scrub ok 2024-05-15T13:56:17.113 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:17 smithi081 ceph-mon[36434]: osd.11 [v2:172.21.15.81:6824/326313090,v1:172.21.15.81:6825/326313090] boot 2024-05-15T13:56:17.113 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:17 smithi081 ceph-mon[36434]: osdmap e72: 12 total, 12 up, 12 in 2024-05-15T13:56:17.113 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:17 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-15T13:56:17.113 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:17 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/194813301' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-15T13:56:17.257 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:17 smithi033 ceph-mon[31650]: purged_snaps scrub starts 2024-05-15T13:56:17.257 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:17 smithi033 ceph-mon[31650]: purged_snaps scrub ok 2024-05-15T13:56:17.257 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:17 smithi033 ceph-mon[31650]: osd.11 [v2:172.21.15.81:6824/326313090,v1:172.21.15.81:6825/326313090] boot 2024-05-15T13:56:17.257 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:17 smithi033 ceph-mon[31650]: osdmap e72: 12 total, 12 up, 12 in 2024-05-15T13:56:17.257 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:17 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-15T13:56:17.257 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:17 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/194813301' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-15T13:56:17.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:17 smithi052 ceph-mon[36190]: purged_snaps scrub starts 2024-05-15T13:56:17.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:17 smithi052 ceph-mon[36190]: purged_snaps scrub ok 2024-05-15T13:56:17.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:17 smithi052 ceph-mon[36190]: osd.11 [v2:172.21.15.81:6824/326313090,v1:172.21.15.81:6825/326313090] boot 2024-05-15T13:56:17.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:17 smithi052 ceph-mon[36190]: osdmap e72: 12 total, 12 up, 12 in 2024-05-15T13:56:17.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:17 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-15T13:56:17.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:17 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/194813301' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-15T13:56:17.711 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":72,"num_osds":12,"num_up_osds":12,"osd_up_since":1715781376,"num_in_osds":12,"osd_in_since":1715781356,"num_remapped_pgs":0} 2024-05-15T13:56:17.712 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-15T13:56:17.973 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:56:18.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:18 smithi033 ceph-mon[31650]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:18.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:18 smithi033 ceph-mon[31650]: osdmap e73: 12 total, 12 up, 12 in 2024-05-15T13:56:18.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:18 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:18.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:18 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:18.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:18 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:56:18.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:18 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:56:18.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:18 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:56:18.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:18 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:56:18.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:18 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:18.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:18 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:18.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:18 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:56:18.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:18 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:18.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:18 smithi052 ceph-mon[36190]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:18.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:18 smithi052 ceph-mon[36190]: osdmap e73: 12 total, 12 up, 12 in 2024-05-15T13:56:18.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:18 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:18.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:18 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:18.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:18 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:56:18.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:18 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:56:18.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:18 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:56:18.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:18 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:56:18.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:18 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:18.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:18 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:18.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:18 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:56:18.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:18 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:18.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:18 smithi081 ceph-mon[36434]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:18.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:18 smithi081 ceph-mon[36434]: osdmap e73: 12 total, 12 up, 12 in 2024-05-15T13:56:18.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:18 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:18.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:18 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:18.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:18 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:56:18.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:18 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:56:18.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:18 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:56:18.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:18 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-15T13:56:18.609 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:18 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:18.609 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:18 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:18.609 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:18 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:56:18.609 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:18 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:19.340 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:19 smithi033 ceph-mon[31650]: Detected new or changed devices on smithi081 2024-05-15T13:56:19.340 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:19 smithi033 ceph-mon[31650]: Adjusting osd_memory_target on smithi081 to 4272M 2024-05-15T13:56:19.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:19 smithi081 ceph-mon[36434]: Detected new or changed devices on smithi081 2024-05-15T13:56:19.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:19 smithi081 ceph-mon[36434]: Adjusting osd_memory_target on smithi081 to 4272M 2024-05-15T13:56:19.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.554+0000 7fde3cf7f700 1 -- 172.21.15.33:0/2655553206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde3806bd10 msgr2=0x7fde3806c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:19.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.554+0000 7fde3cf7f700 1 --2- 172.21.15.33:0/2655553206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde3806bd10 0x7fde3806c190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fde2c0099c0 tx=0x7fde2c009cd0 comp rx=0 tx=0).stop 2024-05-15T13:56:19.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.554+0000 7fde3cf7f700 1 -- 172.21.15.33:0/2655553206 shutdown_connections 2024-05-15T13:56:19.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.554+0000 7fde3cf7f700 1 --2- 172.21.15.33:0/2655553206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde3806c6d0 0x7fde38110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:19.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.554+0000 7fde3cf7f700 1 --2- 172.21.15.33:0/2655553206 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde38109630 0x7fde38109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:19.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.554+0000 7fde3cf7f700 1 --2- 172.21.15.33:0/2655553206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde3806bd10 0x7fde3806c190 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:19.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.554+0000 7fde3cf7f700 1 -- 172.21.15.33:0/2655553206 >> 172.21.15.33:0/2655553206 conn(0x7fde380794e0 msgr2=0x7fde380798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:56:19.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.554+0000 7fde3cf7f700 1 -- 172.21.15.33:0/2655553206 shutdown_connections 2024-05-15T13:56:19.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.554+0000 7fde3cf7f700 1 -- 172.21.15.33:0/2655553206 wait complete. 2024-05-15T13:56:19.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.555+0000 7fde3cf7f700 1 Processor -- start 2024-05-15T13:56:19.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.555+0000 7fde3cf7f700 1 -- start start 2024-05-15T13:56:19.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.555+0000 7fde3cf7f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde3806bd10 0x7fde381024b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:19.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.555+0000 7fde3cf7f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde3806c6d0 0x7fde381029f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:19.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.555+0000 7fde3cf7f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde38109630 0x7fde38102f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:19.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.555+0000 7fde3cf7f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fde38113790 con 0x7fde3806c6d0 2024-05-15T13:56:19.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.555+0000 7fde3cf7f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fde38113610 con 0x7fde3806bd10 2024-05-15T13:56:19.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.555+0000 7fde3cf7f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fde38113910 con 0x7fde38109630 2024-05-15T13:56:19.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.555+0000 7fde35d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde3806c6d0 0x7fde381029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:19.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.555+0000 7fde35d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde3806c6d0 0x7fde381029f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56736/0 (socket says 172.21.15.33:56736) 2024-05-15T13:56:19.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.555+0000 7fde35d9b700 1 -- 172.21.15.33:0/627175849 learned_addr learned my addr 172.21.15.33:0/627175849 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:56:19.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.555+0000 7fde36d9d700 1 --2- 172.21.15.33:0/627175849 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde38109630 0x7fde38102f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:19.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.555+0000 7fde3659c700 1 --2- 172.21.15.33:0/627175849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde3806bd10 0x7fde381024b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:19.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.556+0000 7fde36d9d700 1 -- 172.21.15.33:0/627175849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde3806bd10 msgr2=0x7fde381024b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:19.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.556+0000 7fde36d9d700 1 --2- 172.21.15.33:0/627175849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde3806bd10 0x7fde381024b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:19.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.556+0000 7fde36d9d700 1 -- 172.21.15.33:0/627175849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde3806c6d0 msgr2=0x7fde381029f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:19.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.556+0000 7fde36d9d700 1 --2- 172.21.15.33:0/627175849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde3806c6d0 0x7fde381029f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:19.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.556+0000 7fde36d9d700 1 -- 172.21.15.33:0/627175849 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde381a80e0 con 0x7fde38109630 2024-05-15T13:56:19.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.556+0000 7fde35d9b700 1 --2- 172.21.15.33:0/627175849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde3806c6d0 0x7fde381029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T13:56:19.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.556+0000 7fde36d9d700 1 --2- 172.21.15.33:0/627175849 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde38109630 0x7fde38102f30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fde3000efc0 tx=0x7fde3000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:56:19.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.556+0000 7fde277fe700 1 -- 172.21.15.33:0/627175849 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde300092e0 con 0x7fde38109630 2024-05-15T13:56:19.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.556+0000 7fde3cf7f700 1 -- 172.21.15.33:0/627175849 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde381a8370 con 0x7fde38109630 2024-05-15T13:56:19.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.556+0000 7fde277fe700 1 -- 172.21.15.33:0/627175849 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde30004bb0 con 0x7fde38109630 2024-05-15T13:56:19.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.556+0000 7fde277fe700 1 -- 172.21.15.33:0/627175849 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde30004020 con 0x7fde38109630 2024-05-15T13:56:19.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.556+0000 7fde3cf7f700 1 -- 172.21.15.33:0/627175849 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde381a8880 con 0x7fde38109630 2024-05-15T13:56:19.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.557+0000 7fde3cf7f700 1 -- 172.21.15.33:0/627175849 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde3810bac0 con 0x7fde38109630 2024-05-15T13:56:19.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.559+0000 7fde277fe700 1 -- 172.21.15.33:0/627175849 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fde3001b030 con 0x7fde38109630 2024-05-15T13:56:19.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.559+0000 7fde277fe700 1 --2- 172.21.15.33:0/627175849 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fde200b2880 0x7fde200b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:19.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.560+0000 7fde3659c700 1 --2- 172.21.15.33:0/627175849 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fde200b2880 0x7fde200b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:19.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.560+0000 7fde277fe700 1 -- 172.21.15.33:0/627175849 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fde300e22d0 con 0x7fde38109630 2024-05-15T13:56:19.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.560+0000 7fde3659c700 1 --2- 172.21.15.33:0/627175849 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fde200b2880 0x7fde200b4d40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fde28009be0 tx=0x7fde28009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:56:19.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.567+0000 7fde277fe700 1 -- 172.21.15.33:0/627175849 <== mon.1 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fde30022020 con 0x7fde38109630 2024-05-15T13:56:19.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:19 smithi052 ceph-mon[36190]: Detected new or changed devices on smithi081 2024-05-15T13:56:19.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:19 smithi052 ceph-mon[36190]: Adjusting osd_memory_target on smithi081 to 4272M 2024-05-15T13:56:19.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.786+0000 7fde3cf7f700 1 -- 172.21.15.33:0/627175849 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fde3810b4a0 con 0x7fde38109630 2024-05-15T13:56:19.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.787+0000 7fde277fe700 1 -- 172.21.15.33:0/627175849 <== mon.1 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+18349 (secure 0 0 0) 0x7fde30010040 con 0x7fde38109630 2024-05-15T13:56:19.790 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:56:19.790 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":73,"fsid":"bca78ca4-12c1-11ef-bc98-c7b262605968","created":"2024-05-15T13:49:10.327613+0000","modified":"2024-05-15T13:56:17.105935+0000","last_up_change":"2024-05-15T13:56:16.097711+0000","last_in_change":"2024-05-15T13:55:56.722143+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-15T13:52:32.970844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b434111-810f-450a-b63d-c97080e9f8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6803","nonce":887409130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6805","nonce":887409130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6809","nonce":887409130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6807","nonce":887409130}]},"public_addr":"172.21.15.33:6803/887409130","cluster_addr":"172.21.15.33:6805/887409130","heartbeat_back_addr":"172.21.15.33:6809/887409130","heartbeat_front_addr":"172.21.15.33:6807/887409130","state":["exists","up"]},{"osd":1,"uuid":"0b578d4e-a841-4b27-bc95-d6c3bae580fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6801","nonce":2794180003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6803","nonce":2794180003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6807","nonce":2794180003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6805","nonce":2794180003}]},"public_addr":"172.21.15.52:6801/2794180003","cluster_addr":"172.21.15.52:6803/2794180003","heartbeat_back_addr":"172.21.15.52:6807/2794180003","heartbeat_front_addr":"172.21.15.52:6805/2794180003","state":["exists","up"]},{"osd":2,"uuid":"0e9c9390-ed33-464b-8f39-bca75941e05e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6801","nonce":315610978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6803","nonce":315610978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6807","nonce":315610978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6805","nonce":315610978}]},"public_addr":"172.21.15.81:6801/315610978","cluster_addr":"172.21.15.81:6803/315610978","heartbeat_back_addr":"172.21.15.81:6807/315610978","heartbeat_front_addr":"172.21.15.81:6805/315610978","state":["exists","up"]},{"osd":3,"uuid":"ecdb24c1-fb15-42a5-9bae-436bbae9512a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6811","nonce":818533501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6813","nonce":818533501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6817","nonce":818533501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6815","nonce":818533501}]},"public_addr":"172.21.15.33:6811/818533501","cluster_addr":"172.21.15.33:6813/818533501","heartbeat_back_addr":"172.21.15.33:6817/818533501","heartbeat_front_addr":"172.21.15.33:6815/818533501","state":["exists","up"]},{"osd":4,"uuid":"706752a0-b17f-4563-a7a9-d5078bf0f3b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6809","nonce":3010133776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6811","nonce":3010133776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6815","nonce":3010133776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6813","nonce":3010133776}]},"public_addr":"172.21.15.52:6809/3010133776","cluster_addr":"172.21.15.52:6811/3010133776","heartbeat_back_addr":"172.21.15.52:6815/3010133776","heartbeat_front_addr":"172.21.15.52:6813/3010133776","state":["exists","up"]},{"osd":5,"uuid":"3d45ab97-f364-4903-974d-8acc961363d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6809","nonce":2132616297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6811","nonce":2132616297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6815","nonce":2132616297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6813","nonce":2132616297}]},"public_addr":"172.21.15.81:6809/2132616297","cluster_addr":"172.21.15.81:6811/2132616297","heartbeat_back_addr":"172.21.15.81:6815/2132616297","heartbeat_front_addr":"172.21.15.81:6813/2132616297","state":["exists","up"]},{"osd":6,"uuid":"3be524c5-dcf9-4742-b896-ea9a44a6b391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6819","nonce":1260691801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6821","nonce":1260691801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6825","nonce":1260691801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6823","nonce":1260691801}]},"public_addr":"172.21.15.33:6819/1260691801","cluster_addr":"172.21.15.33:6821/1260691801","heartbeat_back_addr":"172.21.15.33:6825/1260691801","heartbeat_front_addr":"172.21.15.33:6823/1260691801","state":["exists","up"]},{"osd":7,"uuid":"bda2f920-4f2c-4eed-8127-83c1973676e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6817","nonce":1654595813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6819","nonce":1654595813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6823","nonce":1654595813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6821","nonce":1654595813}]},"public_addr":"172.21.15.52:6817/1654595813","cluster_addr":"172.21.15.52:6819/1654595813","heartbeat_back_addr":"172.21.15.52:6823/1654595813","heartbeat_front_addr":"172.21.15.52:6821/1654595813","state":["exists","up"]},{"osd":8,"uuid":"550585b8-0df4-49ee-95c8-3d084744dbd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6817","nonce":3009661748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6819","nonce":3009661748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6823","nonce":3009661748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6821","nonce":3009661748}]},"public_addr":"172.21.15.81:6817/3009661748","cluster_addr":"172.21.15.81:6819/3009661748","heartbeat_back_addr":"172.21.15.81:6823/3009661748","heartbeat_front_addr":"172.21.15.81:6821/3009661748","state":["exists","up"]},{"osd":9,"uuid":"1d781846-26f3-4b23-b22c-c994a8509a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6827","nonce":2843220858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6829","nonce":2843220858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6833","nonce":2843220858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6831","nonce":2843220858}]},"public_addr":"172.21.15.33:6827/2843220858","cluster_addr":"172.21.15.33:6829/2843220858","heartbeat_back_addr":"172.21.15.33:6833/2843220858","heartbeat_front_addr":"172.21.15.33:6831/2843220858","state":["exists","up"]},{"osd":10,"uuid":"af58c940-fdc4-418b-a638-7dc44438b34c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6825","nonce":2422349831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6827","nonce":2422349831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6831","nonce":2422349831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6829","nonce":2422349831}]},"public_addr":"172.21.15.52:6825/2422349831","cluster_addr":"172.21.15.52:6827/2422349831","heartbeat_back_addr":"172.21.15.52:6831/2422349831","heartbeat_front_addr":"172.21.15.52:6829/2422349831","state":["exists","up"]},{"osd":11,"uuid":"ab84aaea-2086-4842-8bf4-628842fce46a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6825","nonce":326313090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6827","nonce":326313090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6831","nonce":326313090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6829","nonce":326313090}]},"public_addr":"172.21.15.81:6825/326313090","cluster_addr":"172.21.15.81:6827/326313090","heartbeat_back_addr":"172.21.15.81:6831/326313090","heartbeat_front_addr":"172.21.15.81:6829/326313090","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:51:41.493136+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:52:05.784773+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:52:29.769429+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:52:54.891620+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:53:19.412922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:53:43.921841+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:54:09.486732+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:54:34.064370+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:54:58.655304+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:55:24.548693+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:55:49.941191+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:56:14.919558+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/436121571":"2024-05-16T13:50:04.881656+0000","172.21.15.33:6801/2680375465":"2024-05-16T13:50:04.881656+0000","172.21.15.33:6800/2680375465":"2024-05-16T13:50:04.881656+0000","172.21.15.33:0/1022787179":"2024-05-16T13:49:37.564643+0000","172.21.15.33:0/3720711369":"2024-05-16T13:50:04.881656+0000","172.21.15.33:0/1282466838":"2024-05-16T13:49:37.564643+0000","172.21.15.33:0/3022248969":"2024-05-16T13:49:37.564643+0000","172.21.15.33:6801/3303495364":"2024-05-16T13:49:37.564643+0000","172.21.15.33:6800/3303495364":"2024-05-16T13:49:37.564643+0000","172.21.15.33:0/4282812692":"2024-05-16T13:50:04.881656+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-15T13:56:19.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.789+0000 7fde3cf7f700 1 -- 172.21.15.33:0/627175849 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fde200b2880 msgr2=0x7fde200b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:19.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.789+0000 7fde3cf7f700 1 --2- 172.21.15.33:0/627175849 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fde200b2880 0x7fde200b4d40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fde28009be0 tx=0x7fde28009260 comp rx=0 tx=0).stop 2024-05-15T13:56:19.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.789+0000 7fde3cf7f700 1 -- 172.21.15.33:0/627175849 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde38109630 msgr2=0x7fde38102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:19.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.789+0000 7fde3cf7f700 1 --2- 172.21.15.33:0/627175849 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde38109630 0x7fde38102f30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fde3000efc0 tx=0x7fde3000c3a0 comp rx=0 tx=0).stop 2024-05-15T13:56:19.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.789+0000 7fde3cf7f700 1 -- 172.21.15.33:0/627175849 shutdown_connections 2024-05-15T13:56:19.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.789+0000 7fde3cf7f700 1 --2- 172.21.15.33:0/627175849 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fde200b2880 0x7fde200b4d40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:19.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.789+0000 7fde3cf7f700 1 --2- 172.21.15.33:0/627175849 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde38109630 0x7fde38102f30 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:19.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.789+0000 7fde3cf7f700 1 --2- 172.21.15.33:0/627175849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde3806bd10 0x7fde381024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:19.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.789+0000 7fde3cf7f700 1 --2- 172.21.15.33:0/627175849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde3806c6d0 0x7fde381029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:19.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.789+0000 7fde3cf7f700 1 -- 172.21.15.33:0/627175849 >> 172.21.15.33:0/627175849 conn(0x7fde380794e0 msgr2=0x7fde381010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:56:19.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.790+0000 7fde3cf7f700 1 -- 172.21.15.33:0/627175849 shutdown_connections 2024-05-15T13:56:19.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:19.790+0000 7fde3cf7f700 1 -- 172.21.15.33:0/627175849 wait complete. 2024-05-15T13:56:20.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:20 smithi033 ceph-mon[31650]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:20.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:20 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/627175849' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T13:56:20.480 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-15T13:52:32.970844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-15T13:56:20.481 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-15T13:56:20.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:20 smithi052 ceph-mon[36190]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:20.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:20 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/627175849' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T13:56:20.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:20 smithi081 ceph-mon[36434]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:20.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:20 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/627175849' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T13:56:20.734 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:56:22.285 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:22 smithi033 ceph-mon[31650]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:22.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.516+0000 7f937c10d700 1 -- 172.21.15.33:0/3264948783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9374102c40 msgr2=0x7f93741030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:22.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.516+0000 7f937c10d700 1 --2- 172.21.15.33:0/3264948783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9374102c40 0x7f93741030c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9364009990 tx=0x7f9364009ca0 comp rx=0 tx=0).stop 2024-05-15T13:56:22.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.516+0000 7f937c10d700 1 -- 172.21.15.33:0/3264948783 shutdown_connections 2024-05-15T13:56:22.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.516+0000 7f937c10d700 1 --2- 172.21.15.33:0/3264948783 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9374103600 0x7f937410f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:22.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.516+0000 7f937c10d700 1 --2- 172.21.15.33:0/3264948783 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93741084c0 0x7f93741088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:22.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.516+0000 7f937c10d700 1 --2- 172.21.15.33:0/3264948783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9374102c40 0x7f93741030c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:22.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.516+0000 7f937c10d700 1 -- 172.21.15.33:0/3264948783 >> 172.21.15.33:0/3264948783 conn(0x7f9374078ab0 msgr2=0x7f93740ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:56:22.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.516+0000 7f937c10d700 1 -- 172.21.15.33:0/3264948783 shutdown_connections 2024-05-15T13:56:22.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.516+0000 7f937c10d700 1 -- 172.21.15.33:0/3264948783 wait complete. 2024-05-15T13:56:22.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.517+0000 7f937c10d700 1 Processor -- start 2024-05-15T13:56:22.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.517+0000 7f937c10d700 1 -- start start 2024-05-15T13:56:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.517+0000 7f937c10d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9374102c40 0x7f937419f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.517+0000 7f937c10d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9374103600 0x7f937419fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.517+0000 7f937c10d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93741084c0 0x7f9374199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.517+0000 7f937c10d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9374112670 con 0x7f93741084c0 2024-05-15T13:56:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.517+0000 7f937c10d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f93741124f0 con 0x7f9374102c40 2024-05-15T13:56:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.517+0000 7f937c10d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f93741127f0 con 0x7f9374103600 2024-05-15T13:56:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.518+0000 7f937a6aa700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93741084c0 0x7f9374199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.518+0000 7f937a6aa700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93741084c0 0x7f9374199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56744/0 (socket says 172.21.15.33:56744) 2024-05-15T13:56:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.518+0000 7f937a6aa700 1 -- 172.21.15.33:0/228868802 learned_addr learned my addr 172.21.15.33:0/228868802 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:56:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.518+0000 7f9379ea9700 1 --2- 172.21.15.33:0/228868802 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9374102c40 0x7f937419f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.518+0000 7f93796a8700 1 --2- 172.21.15.33:0/228868802 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9374103600 0x7f937419fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.518+0000 7f937a6aa700 1 -- 172.21.15.33:0/228868802 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9374103600 msgr2=0x7f937419fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.518+0000 7f937a6aa700 1 --2- 172.21.15.33:0/228868802 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9374103600 0x7f937419fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.518+0000 7f937a6aa700 1 -- 172.21.15.33:0/228868802 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9374102c40 msgr2=0x7f937419f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.518+0000 7f937a6aa700 1 --2- 172.21.15.33:0/228868802 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9374102c40 0x7f937419f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.518+0000 7f937a6aa700 1 -- 172.21.15.33:0/228868802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f937419a1e0 con 0x7f93741084c0 2024-05-15T13:56:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.518+0000 7f937a6aa700 1 --2- 172.21.15.33:0/228868802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93741084c0 0x7f9374199a80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f936c00b550 tx=0x7f936c00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:56:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.518+0000 7f936affd700 1 -- 172.21.15.33:0/228868802 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c015020 con 0x7f93741084c0 2024-05-15T13:56:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.518+0000 7f936affd700 1 -- 172.21.15.33:0/228868802 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f936c011ce0 con 0x7f93741084c0 2024-05-15T13:56:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.518+0000 7f937c10d700 1 -- 172.21.15.33:0/228868802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f937419a4d0 con 0x7f93741084c0 2024-05-15T13:56:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.518+0000 7f936affd700 1 -- 172.21.15.33:0/228868802 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c010dd0 con 0x7f93741084c0 2024-05-15T13:56:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.518+0000 7f937c10d700 1 -- 172.21.15.33:0/228868802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93741a6740 con 0x7f93741084c0 2024-05-15T13:56:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.519+0000 7f937c10d700 1 -- 172.21.15.33:0/228868802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f937406a230 con 0x7f93741084c0 2024-05-15T13:56:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.521+0000 7f936affd700 1 -- 172.21.15.33:0/228868802 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f936c018020 con 0x7f93741084c0 2024-05-15T13:56:22.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.522+0000 7f936affd700 1 --2- 172.21.15.33:0/228868802 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f93600b2900 0x7f93600b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:22.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.522+0000 7f9379ea9700 1 --2- 172.21.15.33:0/228868802 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f93600b2900 0x7f93600b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:22.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.522+0000 7f936affd700 1 -- 172.21.15.33:0/228868802 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f936c0e19e0 con 0x7f93741084c0 2024-05-15T13:56:22.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.522+0000 7f9379ea9700 1 --2- 172.21.15.33:0/228868802 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f93600b2900 0x7f93600b4dc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f93741075f0 tx=0x7f9370009180 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:56:22.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.525+0000 7f936affd700 1 -- 172.21.15.33:0/228868802 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f936c01f020 con 0x7f93741084c0 2024-05-15T13:56:22.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:22 smithi052 ceph-mon[36190]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:22.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:22 smithi081 ceph-mon[36434]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:22.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.739+0000 7f937c10d700 1 -- 172.21.15.33:0/228868802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f93741a6a00 con 0x7f93741084c0 2024-05-15T13:56:22.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.739+0000 7f936affd700 1 -- 172.21.15.33:0/228868802 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) ==== 93+0+10 (secure 0 0 0) 0x7f936c0a9ba0 con 0x7f93741084c0 2024-05-15T13:56:22.742 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-05-15T13:56:22.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.741+0000 7f937c10d700 1 -- 172.21.15.33:0/228868802 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f93600b2900 msgr2=0x7f93600b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:22.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.741+0000 7f937c10d700 1 --2- 172.21.15.33:0/228868802 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f93600b2900 0x7f93600b4dc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f93741075f0 tx=0x7f9370009180 comp rx=0 tx=0).stop 2024-05-15T13:56:22.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.741+0000 7f937c10d700 1 -- 172.21.15.33:0/228868802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93741084c0 msgr2=0x7f9374199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:22.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.741+0000 7f937c10d700 1 --2- 172.21.15.33:0/228868802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93741084c0 0x7f9374199a80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f936c00b550 tx=0x7f936c00b860 comp rx=0 tx=0).stop 2024-05-15T13:56:22.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.741+0000 7f937c10d700 1 -- 172.21.15.33:0/228868802 shutdown_connections 2024-05-15T13:56:22.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.741+0000 7f937c10d700 1 --2- 172.21.15.33:0/228868802 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f93600b2900 0x7f93600b4dc0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:22.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.741+0000 7f937c10d700 1 --2- 172.21.15.33:0/228868802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93741084c0 0x7f9374199a80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:22.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.741+0000 7f937c10d700 1 --2- 172.21.15.33:0/228868802 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9374102c40 0x7f937419f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:22.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.741+0000 7f937c10d700 1 --2- 172.21.15.33:0/228868802 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9374103600 0x7f937419fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:22.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.741+0000 7f937c10d700 1 -- 172.21.15.33:0/228868802 >> 172.21.15.33:0/228868802 conn(0x7f9374078ab0 msgr2=0x7f9374100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:56:22.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.741+0000 7f937c10d700 1 -- 172.21.15.33:0/228868802 shutdown_connections 2024-05-15T13:56:22.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:22.741+0000 7f937c10d700 1 -- 172.21.15.33:0/228868802 wait complete. 2024-05-15T13:56:23.312 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:23 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/228868802' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-15T13:56:23.345 INFO:tasks.cephadm:Adding mds.a on smithi033 2024-05-15T13:56:23.345 INFO:tasks.cephadm:Adding mds.d on smithi033 2024-05-15T13:56:23.345 INFO:tasks.cephadm:Adding mds.g on smithi033 2024-05-15T13:56:23.345 INFO:tasks.cephadm:Adding mds.j on smithi033 2024-05-15T13:56:23.345 INFO:tasks.cephadm:Adding mds.b on smithi052 2024-05-15T13:56:23.345 INFO:tasks.cephadm:Adding mds.e on smithi052 2024-05-15T13:56:23.345 INFO:tasks.cephadm:Adding mds.h on smithi052 2024-05-15T13:56:23.345 INFO:tasks.cephadm:Adding mds.k on smithi052 2024-05-15T13:56:23.345 INFO:tasks.cephadm:Adding mds.c on smithi081 2024-05-15T13:56:23.345 INFO:tasks.cephadm:Adding mds.f on smithi081 2024-05-15T13:56:23.345 INFO:tasks.cephadm:Adding mds.i on smithi081 2024-05-15T13:56:23.346 INFO:tasks.cephadm:Adding mds.l on smithi081 2024-05-15T13:56:23.346 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph orch apply mds all '12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi052=b;smithi052=e;smithi052=h;smithi052=k;smithi081=c;smithi081=f;smithi081=i;smithi081=l' 2024-05-15T13:56:23.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:23 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/228868802' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-15T13:56:23.594 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.c/config 2024-05-15T13:56:23.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:23 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/228868802' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-15T13:56:24.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:24 smithi081 ceph-mon[36434]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:24.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:24 smithi033 ceph-mon[31650]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:24.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:24 smithi052 ceph-mon[36190]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:25.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.163+0000 7f4627784700 1 -- 172.21.15.81:0/1504900190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46201040d0 msgr2=0x7f46201064c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:25.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.163+0000 7f4627784700 1 --2- 172.21.15.81:0/1504900190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46201040d0 0x7f46201064c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f461c009990 tx=0x7f461c009ca0 comp rx=0 tx=0).stop 2024-05-15T13:56:25.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.163+0000 7f4627784700 1 -- 172.21.15.81:0/1504900190 shutdown_connections 2024-05-15T13:56:25.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.163+0000 7f4627784700 1 --2- 172.21.15.81:0/1504900190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4620109330 0x7f462010b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:25.166 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.163+0000 7f4627784700 1 --2- 172.21.15.81:0/1504900190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46201040d0 0x7f46201064c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:25.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.163+0000 7f4627784700 1 --2- 172.21.15.81:0/1504900190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4620106a00 0x7f4620108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:25.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.163+0000 7f4627784700 1 -- 172.21.15.81:0/1504900190 >> 172.21.15.81:0/1504900190 conn(0x7f46200fdaa0 msgr2=0x7f46200ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:56:25.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.163+0000 7f4627784700 1 -- 172.21.15.81:0/1504900190 shutdown_connections 2024-05-15T13:56:25.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.163+0000 7f4627784700 1 -- 172.21.15.81:0/1504900190 wait complete. 2024-05-15T13:56:25.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.163+0000 7f4627784700 1 Processor -- start 2024-05-15T13:56:25.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.163+0000 7f4627784700 1 -- start start 2024-05-15T13:56:25.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.163+0000 7f4627784700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46201040d0 0x7f4620103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:25.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.163+0000 7f4627784700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4620106a00 0x7f4620102080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:25.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.164+0000 7f4627784700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4620109330 0x7f46201025c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:25.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.164+0000 7f4627784700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f462010f140 con 0x7f46201040d0 2024-05-15T13:56:25.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.164+0000 7f4627784700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f462010efc0 con 0x7f4620106a00 2024-05-15T13:56:25.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.164+0000 7f4627784700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f462010f2c0 con 0x7f4620109330 2024-05-15T13:56:25.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.164+0000 7f4625d21700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4620109330 0x7f46201025c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:25.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.164+0000 7f4625d21700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4620109330 0x7f46201025c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39842/0 (socket says 172.21.15.81:39842) 2024-05-15T13:56:25.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.164+0000 7f4625d21700 1 -- 172.21.15.81:0/1293293875 learned_addr learned my addr 172.21.15.81:0/1293293875 (peer_addr_for_me v2:172.21.15.81:0/0) 2024-05-15T13:56:25.167 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.164+0000 7f4625520700 1 --2- 172.21.15.81:0/1293293875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46201040d0 0x7f4620103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:25.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.164+0000 7f4624d1f700 1 --2- 172.21.15.81:0/1293293875 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4620106a00 0x7f4620102080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:25.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.164+0000 7f4625d21700 1 -- 172.21.15.81:0/1293293875 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4620106a00 msgr2=0x7f4620102080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:25.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.164+0000 7f4625d21700 1 --2- 172.21.15.81:0/1293293875 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4620106a00 0x7f4620102080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:25.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.164+0000 7f4625d21700 1 -- 172.21.15.81:0/1293293875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46201040d0 msgr2=0x7f4620103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:25.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.164+0000 7f4625d21700 1 --2- 172.21.15.81:0/1293293875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46201040d0 0x7f4620103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:25.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.164+0000 7f4625d21700 1 -- 172.21.15.81:0/1293293875 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4620102e70 con 0x7f4620109330 2024-05-15T13:56:25.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.164+0000 7f4625d21700 1 --2- 172.21.15.81:0/1293293875 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4620109330 0x7f46201025c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f461800b9e0 tx=0x7f461800bcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:56:25.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.164+0000 7f46167fc700 1 -- 172.21.15.81:0/1293293875 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461800d8e0 con 0x7f4620109330 2024-05-15T13:56:25.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.164+0000 7f46167fc700 1 -- 172.21.15.81:0/1293293875 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4618015420 con 0x7f4620109330 2024-05-15T13:56:25.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.164+0000 7f4627784700 1 -- 172.21.15.81:0/1293293875 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46201a39a0 con 0x7f4620109330 2024-05-15T13:56:25.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.164+0000 7f46167fc700 1 -- 172.21.15.81:0/1293293875 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4618014630 con 0x7f4620109330 2024-05-15T13:56:25.168 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.164+0000 7f4627784700 1 -- 172.21.15.81:0/1293293875 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46201a3f30 con 0x7f4620109330 2024-05-15T13:56:25.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.165+0000 7f4627784700 1 -- 172.21.15.81:0/1293293875 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f462006a230 con 0x7f4620109330 2024-05-15T13:56:25.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.166+0000 7f46167fc700 1 -- 172.21.15.81:0/1293293875 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f46180155d0 con 0x7f4620109330 2024-05-15T13:56:25.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.167+0000 7f46167fc700 1 --2- 172.21.15.81:0/1293293875 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f460c0b2880 0x7f460c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:25.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.167+0000 7f46167fc700 1 -- 172.21.15.81:0/1293293875 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f46180e1680 con 0x7f4620109330 2024-05-15T13:56:25.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.167+0000 7f4625520700 1 --2- 172.21.15.81:0/1293293875 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f460c0b2880 0x7f460c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:25.169 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.168+0000 7f4625520700 1 --2- 172.21.15.81:0/1293293875 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f460c0b2880 0x7f460c0b4d40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f461c00b5c0 tx=0x7f461c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:56:25.173 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.172+0000 7f46167fc700 1 -- 172.21.15.81:0/1293293875 <== mon.1 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f46180a9840 con 0x7f4620109330 2024-05-15T13:56:25.391 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.389+0000 7f4627784700 1 -- 172.21.15.81:0/1293293875 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi052=b;smithi052=e;smithi052=h;smithi052=k;smithi081=c;smithi081=f;smithi081=i;smithi081=l", "target": ["mon-mgr", ""]}) -- 0x7f4620063c40 con 0x7f460c0b2880 2024-05-15T13:56:25.401 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.399+0000 7f46167fc700 1 -- 172.21.15.81:0/1293293875 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f4620063c40 con 0x7f460c0b2880 2024-05-15T13:56:25.401 INFO:teuthology.orchestra.run.smithi081.stdout:Scheduled mds.all update... 2024-05-15T13:56:25.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.401+0000 7f4627784700 1 -- 172.21.15.81:0/1293293875 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f460c0b2880 msgr2=0x7f460c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:25.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.401+0000 7f4627784700 1 --2- 172.21.15.81:0/1293293875 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f460c0b2880 0x7f460c0b4d40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f461c00b5c0 tx=0x7f461c005fb0 comp rx=0 tx=0).stop 2024-05-15T13:56:25.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.401+0000 7f4627784700 1 -- 172.21.15.81:0/1293293875 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4620109330 msgr2=0x7f46201025c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:25.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.401+0000 7f4627784700 1 --2- 172.21.15.81:0/1293293875 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4620109330 0x7f46201025c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f461800b9e0 tx=0x7f461800bcf0 comp rx=0 tx=0).stop 2024-05-15T13:56:25.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.401+0000 7f4627784700 1 -- 172.21.15.81:0/1293293875 shutdown_connections 2024-05-15T13:56:25.403 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.401+0000 7f4627784700 1 --2- 172.21.15.81:0/1293293875 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f460c0b2880 0x7f460c0b4d40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:25.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.401+0000 7f4627784700 1 --2- 172.21.15.81:0/1293293875 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4620109330 0x7f46201025c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:25.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.401+0000 7f4627784700 1 --2- 172.21.15.81:0/1293293875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46201040d0 0x7f4620103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:25.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.401+0000 7f4627784700 1 --2- 172.21.15.81:0/1293293875 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4620106a00 0x7f4620102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:25.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.401+0000 7f4627784700 1 -- 172.21.15.81:0/1293293875 >> 172.21.15.81:0/1293293875 conn(0x7f46200fdaa0 msgr2=0x7f4620104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:56:25.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.401+0000 7f4627784700 1 -- 172.21.15.81:0/1293293875 shutdown_connections 2024-05-15T13:56:25.404 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-15T13:56:25.401+0000 7f4627784700 1 -- 172.21.15.81:0/1293293875 wait complete. 2024-05-15T13:56:26.058 DEBUG:teuthology.orchestra.run.smithi033:mds.a> sudo journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.a.service 2024-05-15T13:56:26.061 DEBUG:teuthology.orchestra.run.smithi033:mds.d> sudo journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.d.service 2024-05-15T13:56:26.103 DEBUG:teuthology.orchestra.run.smithi033:mds.g> sudo journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.g.service 2024-05-15T13:56:26.146 DEBUG:teuthology.orchestra.run.smithi033:mds.j> sudo journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.j.service 2024-05-15T13:56:26.189 DEBUG:teuthology.orchestra.run.smithi052:mds.b> sudo journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.b.service 2024-05-15T13:56:26.192 DEBUG:teuthology.orchestra.run.smithi052:mds.e> sudo journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.e.service 2024-05-15T13:56:26.235 DEBUG:teuthology.orchestra.run.smithi052:mds.h> sudo journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.h.service 2024-05-15T13:56:26.278 DEBUG:teuthology.orchestra.run.smithi052:mds.k> sudo journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.k.service 2024-05-15T13:56:26.321 DEBUG:teuthology.orchestra.run.smithi081:mds.c> sudo journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.c.service 2024-05-15T13:56:26.323 DEBUG:teuthology.orchestra.run.smithi081:mds.f> sudo journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.f.service 2024-05-15T13:56:26.366 DEBUG:teuthology.orchestra.run.smithi081:mds.i> sudo journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.i.service 2024-05-15T13:56:26.409 DEBUG:teuthology.orchestra.run.smithi081:mds.l> sudo journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.l.service 2024-05-15T13:56:26.452 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-15T13:56:26.452 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T13:56:26.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:26 smithi052 ceph-mon[36190]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:26.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:26 smithi052 ceph-mon[36190]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi052=b;smithi052=e;smithi052=h;smithi052=k;smithi081=c;smithi081=f;smithi081=i;smithi081=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:56:26.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:26 smithi052 ceph-mon[36190]: Saving service mds.all spec with placement smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi052=b;smithi052=e;smithi052=h;smithi052=k;smithi081=c;smithi081=f;smithi081=i;smithi081=l;count:12 2024-05-15T13:56:26.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:26 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:26.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:26 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:56:26.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:26 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:26.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:26 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:56:26.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:26 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:26.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:26 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:56:26.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:26 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:56:26.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:26 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:26.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:26 smithi081 ceph-mon[36434]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:26.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:26 smithi081 ceph-mon[36434]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi052=b;smithi052=e;smithi052=h;smithi052=k;smithi081=c;smithi081=f;smithi081=i;smithi081=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:56:26.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:26 smithi081 ceph-mon[36434]: Saving service mds.all spec with placement smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi052=b;smithi052=e;smithi052=h;smithi052=k;smithi081=c;smithi081=f;smithi081=i;smithi081=l;count:12 2024-05-15T13:56:26.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:26 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:26.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:26 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:56:26.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:26 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:26.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:26 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:56:26.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:26 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:26.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:26 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:56:26.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:26 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:56:26.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:26 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:26.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:26 smithi033 ceph-mon[31650]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:26.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:26 smithi033 ceph-mon[31650]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi052=b;smithi052=e;smithi052=h;smithi052=k;smithi081=c;smithi081=f;smithi081=i;smithi081=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:56:26.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:26 smithi033 ceph-mon[31650]: Saving service mds.all spec with placement smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi052=b;smithi052=e;smithi052=h;smithi052=k;smithi081=c;smithi081=f;smithi081=i;smithi081=l;count:12 2024-05-15T13:56:26.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:26 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:26.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:26 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:56:26.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:26 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:26.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:26 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:56:26.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:26 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:26.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:26 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:56:26.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:26 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:56:26.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:26 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:26.700 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:56:27.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:27 smithi052 ceph-mon[36190]: Deploying daemon mds.l on smithi081 2024-05-15T13:56:27.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:27 smithi081 ceph-mon[36434]: Deploying daemon mds.l on smithi081 2024-05-15T13:56:27.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:27 smithi033 ceph-mon[31650]: Deploying daemon mds.l on smithi081 2024-05-15T13:56:28.357 INFO:journalctl@ceph.mds.l.smithi081.stdout:May 15 13:56:28 smithi081 systemd[1]: Starting Ceph mds.l for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T13:56:28.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.390+0000 7f75c42a1700 1 -- 172.21.15.33:0/988191109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75bc1088f0 msgr2=0x7f75bc110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:28.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.390+0000 7f75c42a1700 1 --2- 172.21.15.33:0/988191109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75bc1088f0 0x7f75bc110470 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f75b400b0d0 tx=0x7f75b400b3e0 comp rx=0 tx=0).stop 2024-05-15T13:56:28.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.390+0000 7f75c42a1700 1 -- 172.21.15.33:0/988191109 shutdown_connections 2024-05-15T13:56:28.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.390+0000 7f75c42a1700 1 --2- 172.21.15.33:0/988191109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75bc1088f0 0x7f75bc110470 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:28.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.390+0000 7f75c42a1700 1 --2- 172.21.15.33:0/988191109 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75bc1037f0 0x7f75bc103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:28.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.390+0000 7f75c42a1700 1 --2- 172.21.15.33:0/988191109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75bc1041a0 0x7f75bc1081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:28.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.390+0000 7f75c42a1700 1 -- 172.21.15.33:0/988191109 >> 172.21.15.33:0/988191109 conn(0x7f75bc07a6a0 msgr2=0x7f75bc07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:56:28.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.390+0000 7f75c42a1700 1 -- 172.21.15.33:0/988191109 shutdown_connections 2024-05-15T13:56:28.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.390+0000 7f75c42a1700 1 -- 172.21.15.33:0/988191109 wait complete. 2024-05-15T13:56:28.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.391+0000 7f75c42a1700 1 Processor -- start 2024-05-15T13:56:28.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.391+0000 7f75c42a1700 1 -- start start 2024-05-15T13:56:28.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.391+0000 7f75c42a1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75bc1037f0 0x7f75bc1a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:28.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.391+0000 7f75c42a1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75bc1041a0 0x7f75bc1a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:28.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.391+0000 7f75c42a1700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75bc1088f0 0x7f75bc19bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:28.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.391+0000 7f75c42a1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc1136e0 con 0x7f75bc1041a0 2024-05-15T13:56:28.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.391+0000 7f75c42a1700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc113560 con 0x7f75bc1037f0 2024-05-15T13:56:28.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.391+0000 7f75c42a1700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f75bc113860 con 0x7f75bc1088f0 2024-05-15T13:56:28.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.391+0000 7f75c183c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75bc1041a0 0x7f75bc1a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:28.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.391+0000 7f75c283e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75bc1088f0 0x7f75bc19bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:28.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.391+0000 7f75c183c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75bc1041a0 0x7f75bc1a1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56306/0 (socket says 172.21.15.33:56306) 2024-05-15T13:56:28.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.391+0000 7f75c183c700 1 -- 172.21.15.33:0/3233604212 learned_addr learned my addr 172.21.15.33:0/3233604212 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:56:28.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.391+0000 7f75c203d700 1 --2- 172.21.15.33:0/3233604212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75bc1037f0 0x7f75bc1a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:28.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.391+0000 7f75c283e700 1 -- 172.21.15.33:0/3233604212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75bc1037f0 msgr2=0x7f75bc1a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:28.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.391+0000 7f75c283e700 1 --2- 172.21.15.33:0/3233604212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75bc1037f0 0x7f75bc1a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:28.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.391+0000 7f75c283e700 1 -- 172.21.15.33:0/3233604212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75bc1041a0 msgr2=0x7f75bc1a1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:28.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.391+0000 7f75c283e700 1 --2- 172.21.15.33:0/3233604212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75bc1041a0 0x7f75bc1a1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:28.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.391+0000 7f75c283e700 1 -- 172.21.15.33:0/3233604212 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75bc19c3f0 con 0x7f75bc1088f0 2024-05-15T13:56:28.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.391+0000 7f75c203d700 1 --2- 172.21.15.33:0/3233604212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75bc1037f0 0x7f75bc1a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T13:56:28.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.392+0000 7f75c283e700 1 --2- 172.21.15.33:0/3233604212 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75bc1088f0 0x7f75bc19bb10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f75b400b0d0 tx=0x7f75b4007a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:56:28.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.392+0000 7f75b37fe700 1 -- 172.21.15.33:0/3233604212 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b401f070 con 0x7f75bc1088f0 2024-05-15T13:56:28.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.392+0000 7f75b37fe700 1 -- 172.21.15.33:0/3233604212 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75b4004c50 con 0x7f75bc1088f0 2024-05-15T13:56:28.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.392+0000 7f75c42a1700 1 -- 172.21.15.33:0/3233604212 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75bc19c680 con 0x7f75bc1088f0 2024-05-15T13:56:28.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.392+0000 7f75b37fe700 1 -- 172.21.15.33:0/3233604212 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b401b690 con 0x7f75bc1088f0 2024-05-15T13:56:28.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.392+0000 7f75c42a1700 1 -- 172.21.15.33:0/3233604212 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75bc1a87f0 con 0x7f75bc1088f0 2024-05-15T13:56:28.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.393+0000 7f75c42a1700 1 -- 172.21.15.33:0/3233604212 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75bc104c80 con 0x7f75bc1088f0 2024-05-15T13:56:28.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.394+0000 7f75b37fe700 1 -- 172.21.15.33:0/3233604212 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f75b4022070 con 0x7f75bc1088f0 2024-05-15T13:56:28.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.395+0000 7f75b37fe700 1 --2- 172.21.15.33:0/3233604212 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f75a80b2880 0x7f75a80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:28.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.395+0000 7f75c203d700 1 --2- 172.21.15.33:0/3233604212 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f75a80b2880 0x7f75a80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:28.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.395+0000 7f75b37fe700 1 -- 172.21.15.33:0/3233604212 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f75b400f020 con 0x7f75bc1088f0 2024-05-15T13:56:28.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.395+0000 7f75c203d700 1 --2- 172.21.15.33:0/3233604212 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f75a80b2880 0x7f75a80b4d40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f75b8009cb0 tx=0x7f75b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:56:28.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.398+0000 7f75b37fe700 1 -- 172.21.15.33:0/3233604212 <== mon.1 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f75b4029020 con 0x7f75bc1088f0 2024-05-15T13:56:28.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.667+0000 7f75c42a1700 1 -- 172.21.15.33:0/3233604212 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75bc06a230 con 0x7f75bc1088f0 2024-05-15T13:56:28.670 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:28 smithi033 ceph-mon[31650]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:28.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.668+0000 7f75b37fe700 1 -- 172.21.15.33:0/3233604212 <== mon.1 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f75bc06a230 con 0x7f75bc1088f0 2024-05-15T13:56:28.671 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:56:28.671 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-05-15T13:56:28.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.670+0000 7f75c42a1700 1 -- 172.21.15.33:0/3233604212 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f75a80b2880 msgr2=0x7f75a80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:28.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.670+0000 7f75c42a1700 1 --2- 172.21.15.33:0/3233604212 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f75a80b2880 0x7f75a80b4d40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f75b8009cb0 tx=0x7f75b8009260 comp rx=0 tx=0).stop 2024-05-15T13:56:28.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.670+0000 7f75c42a1700 1 -- 172.21.15.33:0/3233604212 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75bc1088f0 msgr2=0x7f75bc19bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:28.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.670+0000 7f75c42a1700 1 --2- 172.21.15.33:0/3233604212 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75bc1088f0 0x7f75bc19bb10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f75b400b0d0 tx=0x7f75b4007a00 comp rx=0 tx=0).stop 2024-05-15T13:56:28.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.670+0000 7f75c42a1700 1 -- 172.21.15.33:0/3233604212 shutdown_connections 2024-05-15T13:56:28.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.670+0000 7f75c42a1700 1 --2- 172.21.15.33:0/3233604212 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f75a80b2880 0x7f75a80b4d40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:28.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.670+0000 7f75c42a1700 1 --2- 172.21.15.33:0/3233604212 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75bc1088f0 0x7f75bc19bb10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:28.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.670+0000 7f75c42a1700 1 --2- 172.21.15.33:0/3233604212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f75bc1037f0 0x7f75bc1a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:28.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.670+0000 7f75c42a1700 1 --2- 172.21.15.33:0/3233604212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75bc1041a0 0x7f75bc1a1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:28.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.670+0000 7f75c42a1700 1 -- 172.21.15.33:0/3233604212 >> 172.21.15.33:0/3233604212 conn(0x7f75bc07a6a0 msgr2=0x7f75bc101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:56:28.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.670+0000 7f75c42a1700 1 -- 172.21.15.33:0/3233604212 shutdown_connections 2024-05-15T13:56:28.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:28.670+0000 7f75c42a1700 1 -- 172.21.15.33:0/3233604212 wait complete. 2024-05-15T13:56:28.674 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 1 2024-05-15T13:56:28.709 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:28 smithi081 ceph-mon[36434]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:28.709 INFO:journalctl@ceph.mds.l.smithi081.stdout:May 15 13:56:28 smithi081 podman[65946]: 2024-05-15 13:56:28.356905872 +0000 UTC m=+0.016905905 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-15T13:56:28.709 INFO:journalctl@ceph.mds.l.smithi081.stdout:May 15 13:56:28 smithi081 podman[65946]: 2024-05-15 13:56:28.489220154 +0000 UTC m=+0.149220174 container create e49dda6b9f585795831a079acbc35831e8ea82d5f76ecced73941de7a46317d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-l, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514) 2024-05-15T13:56:28.709 INFO:journalctl@ceph.mds.l.smithi081.stdout:May 15 13:56:28 smithi081 podman[65946]: 2024-05-15 13:56:28.708624029 +0000 UTC m=+0.368624039 container init e49dda6b9f585795831a079acbc35831e8ea82d5f76ecced73941de7a46317d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5) 2024-05-15T13:56:28.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:28 smithi052 ceph-mon[36190]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:29.107 INFO:journalctl@ceph.mds.l.smithi081.stdout:May 15 13:56:28 smithi081 podman[65946]: 2024-05-15 13:56:28.711386749 +0000 UTC m=+0.371386771 container start e49dda6b9f585795831a079acbc35831e8ea82d5f76ecced73941de7a46317d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-l, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-15T13:56:29.107 INFO:journalctl@ceph.mds.l.smithi081.stdout:May 15 13:56:28 smithi081 ceph-mds[65965]: -- 172.21.15.81:0/2129919122 <== mon.1 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fd3dacfa40 con 0x55fd3daa2c00 2024-05-15T13:56:29.107 INFO:journalctl@ceph.mds.l.smithi081.stdout:May 15 13:56:28 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-l[65961]: starting mds.l at 2024-05-15T13:56:29.107 INFO:journalctl@ceph.mds.l.smithi081.stdout:May 15 13:56:28 smithi081 bash[65946]: e49dda6b9f585795831a079acbc35831e8ea82d5f76ecced73941de7a46317d1 2024-05-15T13:56:29.107 INFO:journalctl@ceph.mds.l.smithi081.stdout:May 15 13:56:28 smithi081 systemd[1]: Started Ceph mds.l for bca78ca4-12c1-11ef-bc98-c7b262605968. 2024-05-15T13:56:29.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:29 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/3233604212' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:56:29.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:29 smithi033 ceph-mon[31650]: mds.? [v2:172.21.15.81:6832/1872861431,v1:172.21.15.81:6833/1872861431] up:boot 2024-05-15T13:56:29.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:29 smithi033 ceph-mon[31650]: fsmap 1 up:standby 2024-05-15T13:56:29.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:29 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-15T13:56:29.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:29 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:29.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:29 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:29.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:29 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:29.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:29 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:56:29.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:29 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:56:29.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:29 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:29.821 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:29 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/3233604212' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:56:29.821 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:29 smithi052 ceph-mon[36190]: mds.? [v2:172.21.15.81:6832/1872861431,v1:172.21.15.81:6833/1872861431] up:boot 2024-05-15T13:56:29.821 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:29 smithi052 ceph-mon[36190]: fsmap 1 up:standby 2024-05-15T13:56:29.821 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:29 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-15T13:56:29.821 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:29 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:29.821 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:29 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:29.821 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:29 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:29.821 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:29 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:56:29.821 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:29 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:56:29.821 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:29 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:29.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:29 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/3233604212' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:56:29.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:29 smithi081 ceph-mon[36434]: mds.? [v2:172.21.15.81:6832/1872861431,v1:172.21.15.81:6833/1872861431] up:boot 2024-05-15T13:56:29.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:29 smithi081 ceph-mon[36434]: fsmap 1 up:standby 2024-05-15T13:56:29.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:29 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-15T13:56:29.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:29 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:29.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:29 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:29.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:29 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:29.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:29 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:56:29.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:29 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:56:29.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:29 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:30.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:30 smithi033 ceph-mon[31650]: Deploying daemon mds.k on smithi052 2024-05-15T13:56:30.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:30 smithi033 ceph-mon[31650]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:30.816 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:30 smithi052 ceph-mon[36190]: Deploying daemon mds.k on smithi052 2024-05-15T13:56:30.817 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:30 smithi052 ceph-mon[36190]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:30.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:30 smithi081 ceph-mon[36434]: Deploying daemon mds.k on smithi052 2024-05-15T13:56:30.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:30 smithi081 ceph-mon[36434]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:31.355 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T13:56:31.528 INFO:journalctl@ceph.mds.k.smithi052.stdout:May 15 13:56:31 smithi052 systemd[1]: Starting Ceph mds.k for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T13:56:31.617 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:56:31.840 INFO:journalctl@ceph.mds.k.smithi052.stdout:May 15 13:56:31 smithi052 podman[65963]: 2024-05-15 13:56:31.429728554 +0000 UTC m=+0.017388310 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-15T13:56:31.841 INFO:journalctl@ceph.mds.k.smithi052.stdout:May 15 13:56:31 smithi052 podman[65963]: 2024-05-15 13:56:31.625543933 +0000 UTC m=+0.213203681 container create 3504755753aa99e33ad7df8432e4f481baf9298461041907b3b8443530c2b7a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-k, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-15T13:56:32.340 INFO:journalctl@ceph.mds.k.smithi052.stdout:May 15 13:56:31 smithi052 podman[65963]: 2024-05-15 13:56:31.844960862 +0000 UTC m=+0.432620603 container init 3504755753aa99e33ad7df8432e4f481baf9298461041907b3b8443530c2b7a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-k, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-15T13:56:32.341 INFO:journalctl@ceph.mds.k.smithi052.stdout:May 15 13:56:31 smithi052 podman[65963]: 2024-05-15 13:56:31.847661538 +0000 UTC m=+0.435321287 container start 3504755753aa99e33ad7df8432e4f481baf9298461041907b3b8443530c2b7a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-k, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True) 2024-05-15T13:56:32.341 INFO:journalctl@ceph.mds.k.smithi052.stdout:May 15 13:56:31 smithi052 ceph-mds[65984]: -- 172.21.15.52:0/3672960992 <== mon.2 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5576cd1cba40 con 0x5576cd19ec00 2024-05-15T13:56:32.341 INFO:journalctl@ceph.mds.k.smithi052.stdout:May 15 13:56:31 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-k[65980]: starting mds.k at 2024-05-15T13:56:32.341 INFO:journalctl@ceph.mds.k.smithi052.stdout:May 15 13:56:31 smithi052 bash[65963]: 3504755753aa99e33ad7df8432e4f481baf9298461041907b3b8443530c2b7a7 2024-05-15T13:56:32.341 INFO:journalctl@ceph.mds.k.smithi052.stdout:May 15 13:56:31 smithi052 systemd[1]: Started Ceph mds.k for bca78ca4-12c1-11ef-bc98-c7b262605968. 2024-05-15T13:56:32.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:32 smithi033 ceph-mon[31650]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:32.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:32 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:32.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:32 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:32.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:32 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:32.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:32 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:56:32.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:32 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:56:32.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:32 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:32.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:32 smithi052 ceph-mon[36190]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:32.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:32 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:32.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:32 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:32.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:32 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:32.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:32 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:56:32.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:32 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:56:32.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:32 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:32.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:32 smithi081 ceph-mon[36434]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:32.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:32 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:32.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:32 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:32.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:32 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:32.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:32 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:56:32.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:32 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:56:32.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:32 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:33.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.301+0000 7f62f3dbc700 1 -- 172.21.15.33:0/308311105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62ec1041a0 msgr2=0x7f62ec1081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:33.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.301+0000 7f62f3dbc700 1 --2- 172.21.15.33:0/308311105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62ec1041a0 0x7f62ec1081c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f62dc0099c0 tx=0x7f62dc009cd0 comp rx=0 tx=0).stop 2024-05-15T13:56:33.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.302+0000 7f62f3dbc700 1 -- 172.21.15.33:0/308311105 shutdown_connections 2024-05-15T13:56:33.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.302+0000 7f62f3dbc700 1 --2- 172.21.15.33:0/308311105 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62ec1088f0 0x7f62ec110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:33.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.302+0000 7f62f3dbc700 1 --2- 172.21.15.33:0/308311105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62ec1037f0 0x7f62ec103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:33.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.302+0000 7f62f3dbc700 1 --2- 172.21.15.33:0/308311105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62ec1041a0 0x7f62ec1081c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:33.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.302+0000 7f62f3dbc700 1 -- 172.21.15.33:0/308311105 >> 172.21.15.33:0/308311105 conn(0x7f62ec07a6a0 msgr2=0x7f62ec07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:56:33.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.302+0000 7f62f3dbc700 1 -- 172.21.15.33:0/308311105 shutdown_connections 2024-05-15T13:56:33.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.302+0000 7f62f3dbc700 1 -- 172.21.15.33:0/308311105 wait complete. 2024-05-15T13:56:33.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.302+0000 7f62f3dbc700 1 Processor -- start 2024-05-15T13:56:33.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.303+0000 7f62f3dbc700 1 -- start start 2024-05-15T13:56:33.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.303+0000 7f62f3dbc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62ec1037f0 0x7f62ec19f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:33.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.303+0000 7f62f3dbc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62ec1041a0 0x7f62ec19fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:33.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.303+0000 7f62f3dbc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62ec1088f0 0x7f62ec199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:33.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.303+0000 7f62f3dbc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec1136d0 con 0x7f62ec1041a0 2024-05-15T13:56:33.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.303+0000 7f62f3dbc700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec113550 con 0x7f62ec1088f0 2024-05-15T13:56:33.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.303+0000 7f62f3dbc700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec113850 con 0x7f62ec1037f0 2024-05-15T13:56:33.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.303+0000 7f62f1357700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62ec1041a0 0x7f62ec19fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:33.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.303+0000 7f62f1b58700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62ec1037f0 0x7f62ec19f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:33.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.303+0000 7f62f1b58700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62ec1037f0 0x7f62ec19f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:54754/0 (socket says 172.21.15.33:54754) 2024-05-15T13:56:33.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.303+0000 7f62f1b58700 1 -- 172.21.15.33:0/587934849 learned_addr learned my addr 172.21.15.33:0/587934849 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:56:33.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.303+0000 7f62f2359700 1 --2- 172.21.15.33:0/587934849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62ec1088f0 0x7f62ec199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:33.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.303+0000 7f62f1b58700 1 -- 172.21.15.33:0/587934849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62ec1088f0 msgr2=0x7f62ec199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:33.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.303+0000 7f62f1b58700 1 --2- 172.21.15.33:0/587934849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62ec1088f0 0x7f62ec199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:33.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.303+0000 7f62f1b58700 1 -- 172.21.15.33:0/587934849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62ec1041a0 msgr2=0x7f62ec19fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:33.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.303+0000 7f62f1b58700 1 --2- 172.21.15.33:0/587934849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62ec1041a0 0x7f62ec19fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:33.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.303+0000 7f62f1b58700 1 -- 172.21.15.33:0/587934849 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62ec19a0d0 con 0x7f62ec1037f0 2024-05-15T13:56:33.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.304+0000 7f62f1b58700 1 --2- 172.21.15.33:0/587934849 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62ec1037f0 0x7f62ec19f7b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f62e800d980 tx=0x7f62e800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:56:33.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.304+0000 7f62e2ffd700 1 -- 172.21.15.33:0/587934849 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e8017070 con 0x7f62ec1037f0 2024-05-15T13:56:33.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.304+0000 7f62e2ffd700 1 -- 172.21.15.33:0/587934849 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62e8005020 con 0x7f62ec1037f0 2024-05-15T13:56:33.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.304+0000 7f62f3dbc700 1 -- 172.21.15.33:0/587934849 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62ec19a3c0 con 0x7f62ec1037f0 2024-05-15T13:56:33.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.304+0000 7f62e2ffd700 1 -- 172.21.15.33:0/587934849 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e8009b50 con 0x7f62ec1037f0 2024-05-15T13:56:33.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.304+0000 7f62f3dbc700 1 -- 172.21.15.33:0/587934849 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62ec1a6680 con 0x7f62ec1037f0 2024-05-15T13:56:33.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.305+0000 7f62f3dbc700 1 -- 172.21.15.33:0/587934849 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62ec06a230 con 0x7f62ec1037f0 2024-05-15T13:56:33.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.307+0000 7f62e2ffd700 1 -- 172.21.15.33:0/587934849 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f62e80051d0 con 0x7f62ec1037f0 2024-05-15T13:56:33.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.307+0000 7f62e2ffd700 1 --2- 172.21.15.33:0/587934849 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f62d80b2880 0x7f62d80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:33.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.307+0000 7f62f1357700 1 --2- 172.21.15.33:0/587934849 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f62d80b2880 0x7f62d80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:33.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.307+0000 7f62e2ffd700 1 -- 172.21.15.33:0/587934849 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f62e80e21a0 con 0x7f62ec1037f0 2024-05-15T13:56:33.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.308+0000 7f62f1357700 1 --2- 172.21.15.33:0/587934849 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f62d80b2880 0x7f62d80b4d40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f62dc009990 tx=0x7f62dc009650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:56:33.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.311+0000 7f62e2ffd700 1 -- 172.21.15.33:0/587934849 <== mon.1 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f62e8020020 con 0x7f62ec1037f0 2024-05-15T13:56:33.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.574+0000 7f62f3dbc700 1 -- 172.21.15.33:0/587934849 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62ec02e560 con 0x7f62ec1037f0 2024-05-15T13:56:33.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.575+0000 7f62e2ffd700 1 -- 172.21.15.33:0/587934849 <== mon.1 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7f62e80aa2c0 con 0x7f62ec1037f0 2024-05-15T13:56:33.578 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:56:33.578 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-15T13:56:33.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.578+0000 7f62f3dbc700 1 -- 172.21.15.33:0/587934849 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f62d80b2880 msgr2=0x7f62d80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:33.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.578+0000 7f62f3dbc700 1 --2- 172.21.15.33:0/587934849 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f62d80b2880 0x7f62d80b4d40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f62dc009990 tx=0x7f62dc009650 comp rx=0 tx=0).stop 2024-05-15T13:56:33.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.578+0000 7f62f3dbc700 1 -- 172.21.15.33:0/587934849 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62ec1037f0 msgr2=0x7f62ec19f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:33.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.578+0000 7f62f3dbc700 1 --2- 172.21.15.33:0/587934849 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62ec1037f0 0x7f62ec19f7b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f62e800d980 tx=0x7f62e800dc90 comp rx=0 tx=0).stop 2024-05-15T13:56:33.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.578+0000 7f62f3dbc700 1 -- 172.21.15.33:0/587934849 shutdown_connections 2024-05-15T13:56:33.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.578+0000 7f62f3dbc700 1 --2- 172.21.15.33:0/587934849 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f62d80b2880 0x7f62d80b4d40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:33.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.578+0000 7f62f3dbc700 1 --2- 172.21.15.33:0/587934849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62ec1088f0 0x7f62ec199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:33.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.578+0000 7f62f3dbc700 1 --2- 172.21.15.33:0/587934849 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62ec1037f0 0x7f62ec19f7b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:33.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.578+0000 7f62f3dbc700 1 --2- 172.21.15.33:0/587934849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62ec1041a0 0x7f62ec19fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:33.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.578+0000 7f62f3dbc700 1 -- 172.21.15.33:0/587934849 >> 172.21.15.33:0/587934849 conn(0x7f62ec07a6a0 msgr2=0x7f62ec109a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:56:33.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.578+0000 7f62f3dbc700 1 -- 172.21.15.33:0/587934849 shutdown_connections 2024-05-15T13:56:33.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:33.578+0000 7f62f3dbc700 1 -- 172.21.15.33:0/587934849 wait complete. 2024-05-15T13:56:33.581 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 3 2024-05-15T13:56:33.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:33 smithi033 ceph-mon[31650]: Deploying daemon mds.j on smithi033 2024-05-15T13:56:33.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:33 smithi033 ceph-mon[31650]: mds.? [v2:172.21.15.52:6832/2083190067,v1:172.21.15.52:6833/2083190067] up:boot 2024-05-15T13:56:33.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:33 smithi033 ceph-mon[31650]: fsmap 2 up:standby 2024-05-15T13:56:33.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:33 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-15T13:56:33.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:33 smithi052 ceph-mon[36190]: Deploying daemon mds.j on smithi033 2024-05-15T13:56:33.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:33 smithi052 ceph-mon[36190]: mds.? [v2:172.21.15.52:6832/2083190067,v1:172.21.15.52:6833/2083190067] up:boot 2024-05-15T13:56:33.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:33 smithi052 ceph-mon[36190]: fsmap 2 up:standby 2024-05-15T13:56:33.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:33 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-15T13:56:33.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:33 smithi081 ceph-mon[36434]: Deploying daemon mds.j on smithi033 2024-05-15T13:56:33.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:33 smithi081 ceph-mon[36434]: mds.? [v2:172.21.15.52:6832/2083190067,v1:172.21.15.52:6833/2083190067] up:boot 2024-05-15T13:56:33.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:33 smithi081 ceph-mon[36434]: fsmap 2 up:standby 2024-05-15T13:56:33.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:33 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-15T13:56:34.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:34 smithi052 ceph-mon[36190]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:34.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:34 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/587934849' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:56:34.680 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:34 smithi033 ceph-mon[31650]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:34.680 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:34 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/587934849' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:56:34.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:34 smithi081 ceph-mon[36434]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:34.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:34 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/587934849' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:56:34.940 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 15 13:56:34 smithi033 systemd[1]: Starting Ceph mds.j for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T13:56:35.439 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 15 13:56:35 smithi033 podman[69342]: 2024-05-15 13:56:35.175402541 +0000 UTC m=+0.018095434 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-15T13:56:35.440 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 15 13:56:35 smithi033 podman[69342]: 2024-05-15 13:56:35.275163069 +0000 UTC m=+0.117855990 container create 8e2339d6cdaa9d69d4d0071b9e6c11d2b7472b13d8e32c510a2b10c790d19230 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-15T13:56:35.691 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 15 13:56:35 smithi033 podman[69342]: 2024-05-15 13:56:35.561055473 +0000 UTC m=+0.403748346 container init 8e2339d6cdaa9d69d4d0071b9e6c11d2b7472b13d8e32c510a2b10c790d19230 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-15T13:56:35.691 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 15 13:56:35 smithi033 podman[69342]: 2024-05-15 13:56:35.564073397 +0000 UTC m=+0.406766278 container start 8e2339d6cdaa9d69d4d0071b9e6c11d2b7472b13d8e32c510a2b10c790d19230 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-j, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T13:56:35.691 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 15 13:56:35 smithi033 ceph-mds[69361]: -- 172.21.15.33:0/1462467562 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5580ff78fa40 con 0x5580ff762c00 2024-05-15T13:56:35.691 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 15 13:56:35 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-j[69357]: starting mds.j at 2024-05-15T13:56:35.691 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 15 13:56:35 smithi033 bash[69342]: 8e2339d6cdaa9d69d4d0071b9e6c11d2b7472b13d8e32c510a2b10c790d19230 2024-05-15T13:56:35.691 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 15 13:56:35 smithi033 systemd[1]: Started Ceph mds.j for bca78ca4-12c1-11ef-bc98-c7b262605968. 2024-05-15T13:56:36.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:36 smithi033 ceph-mon[31650]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:36.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:36 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:36.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:36 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:36.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:36 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:36.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:36 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:56:36.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:36 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:56:36.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:36 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:36.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:36 smithi052 ceph-mon[36190]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:36.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:36 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:36.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:36 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:36.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:36 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:36.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:36 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:56:36.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:36 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:56:36.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:36 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:36.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:36 smithi081 ceph-mon[36434]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:36.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:36 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:36.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:36 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:36.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:36 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:36.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:36 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:56:36.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:36 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:56:36.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:36 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:37.102 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T13:56:37.485 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:56:37.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:37 smithi033 ceph-mon[31650]: Deploying daemon mds.d on smithi033 2024-05-15T13:56:37.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:37 smithi033 ceph-mon[31650]: mds.? [v2:172.21.15.33:6834/3975341851,v1:172.21.15.33:6835/3975341851] up:boot 2024-05-15T13:56:37.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:37 smithi033 ceph-mon[31650]: fsmap 3 up:standby 2024-05-15T13:56:37.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:37 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-15T13:56:37.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:37 smithi052 ceph-mon[36190]: Deploying daemon mds.d on smithi033 2024-05-15T13:56:37.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:37 smithi052 ceph-mon[36190]: mds.? [v2:172.21.15.33:6834/3975341851,v1:172.21.15.33:6835/3975341851] up:boot 2024-05-15T13:56:37.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:37 smithi052 ceph-mon[36190]: fsmap 3 up:standby 2024-05-15T13:56:37.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:37 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-15T13:56:37.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:37 smithi081 ceph-mon[36434]: Deploying daemon mds.d on smithi033 2024-05-15T13:56:37.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:37 smithi081 ceph-mon[36434]: mds.? [v2:172.21.15.33:6834/3975341851,v1:172.21.15.33:6835/3975341851] up:boot 2024-05-15T13:56:37.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:37 smithi081 ceph-mon[36434]: fsmap 3 up:standby 2024-05-15T13:56:37.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:37 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-15T13:56:38.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:38 smithi033 ceph-mon[31650]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:38.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:38 smithi052 ceph-mon[36190]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:38.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:38 smithi081 ceph-mon[36434]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:39.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.262+0000 7f13512f1700 1 -- 172.21.15.33:0/883326252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f134c108920 msgr2=0x7f134c1104a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:39.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.262+0000 7f13512f1700 1 --2- 172.21.15.33:0/883326252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f134c108920 0x7f134c1104a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f134400b0d0 tx=0x7f134400b3e0 comp rx=0 tx=0).stop 2024-05-15T13:56:39.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.262+0000 7f13512f1700 1 -- 172.21.15.33:0/883326252 shutdown_connections 2024-05-15T13:56:39.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.262+0000 7f13512f1700 1 --2- 172.21.15.33:0/883326252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f134c108920 0x7f134c1104a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:39.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.262+0000 7f13512f1700 1 --2- 172.21.15.33:0/883326252 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f134c103850 0x7f134c103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:39.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.262+0000 7f13512f1700 1 --2- 172.21.15.33:0/883326252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f134c104200 0x7f134c1081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:39.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.262+0000 7f13512f1700 1 -- 172.21.15.33:0/883326252 >> 172.21.15.33:0/883326252 conn(0x7f134c07a5d0 msgr2=0x7f134c07a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:56:39.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.262+0000 7f13512f1700 1 -- 172.21.15.33:0/883326252 shutdown_connections 2024-05-15T13:56:39.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.262+0000 7f13512f1700 1 -- 172.21.15.33:0/883326252 wait complete. 2024-05-15T13:56:39.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.263+0000 7f13512f1700 1 Processor -- start 2024-05-15T13:56:39.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.264+0000 7f13512f1700 1 -- start start 2024-05-15T13:56:39.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.264+0000 7f13512f1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f134c103850 0x7f134c19f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:39.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.264+0000 7f13512f1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f134c104200 0x7f134c19fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:39.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.264+0000 7f13512f1700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f134c108920 0x7f134c199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.264+0000 7f13512f1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f134c1136f0 con 0x7f134c103850 2024-05-15T13:56:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.264+0000 7f13512f1700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f134c113570 con 0x7f134c104200 2024-05-15T13:56:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.264+0000 7f13512f1700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f134c113870 con 0x7f134c108920 2024-05-15T13:56:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.264+0000 7f134affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f134c103850 0x7f134c19f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.264+0000 7f134affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f134c103850 0x7f134c19f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50160/0 (socket says 172.21.15.33:50160) 2024-05-15T13:56:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.264+0000 7f134affd700 1 -- 172.21.15.33:0/3334922486 learned_addr learned my addr 172.21.15.33:0/3334922486 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:56:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.264+0000 7f134a7fc700 1 --2- 172.21.15.33:0/3334922486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f134c104200 0x7f134c19fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.265+0000 7f134b7fe700 1 --2- 172.21.15.33:0/3334922486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f134c108920 0x7f134c199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.265+0000 7f134affd700 1 -- 172.21.15.33:0/3334922486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f134c108920 msgr2=0x7f134c199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.265+0000 7f134affd700 1 --2- 172.21.15.33:0/3334922486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f134c108920 0x7f134c199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.265+0000 7f134affd700 1 -- 172.21.15.33:0/3334922486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f134c104200 msgr2=0x7f134c19fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.265+0000 7f134affd700 1 --2- 172.21.15.33:0/3334922486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f134c104200 0x7f134c19fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.265+0000 7f134affd700 1 -- 172.21.15.33:0/3334922486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f134c199ff0 con 0x7f134c103850 2024-05-15T13:56:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.265+0000 7f134b7fe700 1 --2- 172.21.15.33:0/3334922486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f134c108920 0x7f134c199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T13:56:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.265+0000 7f134affd700 1 --2- 172.21.15.33:0/3334922486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f134c103850 0x7f134c19f690 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f133c00c9b0 tx=0x7f133c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:56:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.265+0000 7f1333fff700 1 -- 172.21.15.33:0/3334922486 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133c007b00 con 0x7f134c103850 2024-05-15T13:56:39.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.265+0000 7f13512f1700 1 -- 172.21.15.33:0/3334922486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f134c19a2e0 con 0x7f134c103850 2024-05-15T13:56:39.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.265+0000 7f13512f1700 1 -- 172.21.15.33:0/3334922486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f134c1a64f0 con 0x7f134c103850 2024-05-15T13:56:39.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.265+0000 7f1333fff700 1 -- 172.21.15.33:0/3334922486 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f133c004ce0 con 0x7f134c103850 2024-05-15T13:56:39.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.265+0000 7f1333fff700 1 -- 172.21.15.33:0/3334922486 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133c005530 con 0x7f134c103850 2024-05-15T13:56:39.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.267+0000 7f13512f1700 1 -- 172.21.15.33:0/3334922486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1338005330 con 0x7f134c103850 2024-05-15T13:56:39.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.268+0000 7f1333fff700 1 -- 172.21.15.33:0/3334922486 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f133c004750 con 0x7f134c103850 2024-05-15T13:56:39.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.269+0000 7f1333fff700 1 --2- 172.21.15.33:0/3334922486 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f13340b2900 0x7f13340b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:39.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.270+0000 7f134a7fc700 1 --2- 172.21.15.33:0/3334922486 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f13340b2900 0x7f13340b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:39.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.270+0000 7f1333fff700 1 -- 172.21.15.33:0/3334922486 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f133c0e1d50 con 0x7f134c103850 2024-05-15T13:56:39.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.270+0000 7f134a7fc700 1 --2- 172.21.15.33:0/3334922486 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f13340b2900 0x7f13340b4dc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f134c109c20 tx=0x7f134000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:56:39.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.275+0000 7f1333fff700 1 -- 172.21.15.33:0/3334922486 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f133c014030 con 0x7f134c103850 2024-05-15T13:56:39.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.561+0000 7f13512f1700 1 -- 172.21.15.33:0/3334922486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1338005620 con 0x7f134c103850 2024-05-15T13:56:39.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.561+0000 7f1333fff700 1 -- 172.21.15.33:0/3334922486 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7f133c0a9e70 con 0x7f134c103850 2024-05-15T13:56:39.563 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:56:39.563 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-15T13:56:39.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.564+0000 7f13512f1700 1 -- 172.21.15.33:0/3334922486 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f13340b2900 msgr2=0x7f13340b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:39.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.564+0000 7f13512f1700 1 --2- 172.21.15.33:0/3334922486 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f13340b2900 0x7f13340b4dc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f134c109c20 tx=0x7f134000a380 comp rx=0 tx=0).stop 2024-05-15T13:56:39.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.564+0000 7f13512f1700 1 -- 172.21.15.33:0/3334922486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f134c103850 msgr2=0x7f134c19f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:39.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.564+0000 7f13512f1700 1 --2- 172.21.15.33:0/3334922486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f134c103850 0x7f134c19f690 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f133c00c9b0 tx=0x7f133c00ccc0 comp rx=0 tx=0).stop 2024-05-15T13:56:39.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.564+0000 7f13512f1700 1 -- 172.21.15.33:0/3334922486 shutdown_connections 2024-05-15T13:56:39.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.564+0000 7f13512f1700 1 --2- 172.21.15.33:0/3334922486 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f13340b2900 0x7f13340b4dc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:39.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.564+0000 7f13512f1700 1 --2- 172.21.15.33:0/3334922486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f134c108920 0x7f134c199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:39.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.564+0000 7f13512f1700 1 --2- 172.21.15.33:0/3334922486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f134c103850 0x7f134c19f690 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:39.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.564+0000 7f13512f1700 1 --2- 172.21.15.33:0/3334922486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f134c104200 0x7f134c19fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:39.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.564+0000 7f13512f1700 1 -- 172.21.15.33:0/3334922486 >> 172.21.15.33:0/3334922486 conn(0x7f134c07a5d0 msgr2=0x7f134c101560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:56:39.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.564+0000 7f13512f1700 1 -- 172.21.15.33:0/3334922486 shutdown_connections 2024-05-15T13:56:39.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:39.564+0000 7f13512f1700 1 -- 172.21.15.33:0/3334922486 wait complete. 2024-05-15T13:56:39.566 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 4 2024-05-15T13:56:39.941 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 15 13:56:39 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-d[69810]: starting mds.d at 2024-05-15T13:56:40.441 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 15 13:56:40 smithi033 bash[69775]: e29496fc34f2bec113367ed8eda431d81cd14da1ec7cd871dd3f492e592dab85 2024-05-15T13:56:40.441 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 15 13:56:40 smithi033 systemd[1]: Started Ceph mds.d for bca78ca4-12c1-11ef-bc98-c7b262605968. 2024-05-15T13:56:40.774 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:40 smithi052 ceph-mon[36190]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:40.774 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:40 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/3334922486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:56:40.774 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:40 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:40.774 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:40 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:40.774 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:40 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:40.774 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:40 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:56:40.775 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:40 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:56:40.775 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:40 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:40.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:40 smithi081 ceph-mon[36434]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:40.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:40 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/3334922486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:56:40.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:40 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:40.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:40 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:40.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:40 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:40.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:40 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:56:40.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:40 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:56:40.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:40 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:40.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:40 smithi033 ceph-mon[31650]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:40.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:40 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/3334922486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:56:40.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:40 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:40.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:40 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:40.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:40 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:40.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:40 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:56:40.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:40 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:56:40.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:40 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:41.738 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:41 smithi052 ceph-mon[36190]: Deploying daemon mds.e on smithi052 2024-05-15T13:56:41.738 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:41 smithi052 ceph-mon[36190]: mds.? [v2:172.21.15.33:6836/3558077615,v1:172.21.15.33:6837/3558077615] up:boot 2024-05-15T13:56:41.738 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:41 smithi052 ceph-mon[36190]: fsmap 4 up:standby 2024-05-15T13:56:41.738 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:41 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-15T13:56:41.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:41 smithi081 ceph-mon[36434]: Deploying daemon mds.e on smithi052 2024-05-15T13:56:41.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:41 smithi081 ceph-mon[36434]: mds.? [v2:172.21.15.33:6836/3558077615,v1:172.21.15.33:6837/3558077615] up:boot 2024-05-15T13:56:41.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:41 smithi081 ceph-mon[36434]: fsmap 4 up:standby 2024-05-15T13:56:41.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:41 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-15T13:56:41.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:41 smithi033 ceph-mon[31650]: Deploying daemon mds.e on smithi052 2024-05-15T13:56:41.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:41 smithi033 ceph-mon[31650]: mds.? [v2:172.21.15.33:6836/3558077615,v1:172.21.15.33:6837/3558077615] up:boot 2024-05-15T13:56:41.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:41 smithi033 ceph-mon[31650]: fsmap 4 up:standby 2024-05-15T13:56:41.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:41 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-15T13:56:42.522 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T13:56:42.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:42 smithi052 ceph-mon[36190]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:42.591 INFO:journalctl@ceph.mds.e.smithi052.stdout:May 15 13:56:42 smithi052 podman[66280]: 2024-05-15 13:56:42.530970849 +0000 UTC m=+0.145704951 container create ce9ab971fa680cd130262bbcd5a70fc9311e3b941dd557cffddd05ae9ea2a3c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-e, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True) 2024-05-15T13:56:42.787 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:56:42.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:42 smithi081 ceph-mon[36434]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:42.909 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:42 smithi033 ceph-mon[31650]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:43.091 INFO:journalctl@ceph.mds.e.smithi052.stdout:May 15 13:56:42 smithi052 podman[66280]: 2024-05-15 13:56:42.742531983 +0000 UTC m=+0.357266088 container init ce9ab971fa680cd130262bbcd5a70fc9311e3b941dd557cffddd05ae9ea2a3c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-05-15T13:56:43.091 INFO:journalctl@ceph.mds.e.smithi052.stdout:May 15 13:56:42 smithi052 podman[66280]: 2024-05-15 13:56:42.745282129 +0000 UTC m=+0.360016249 container start ce9ab971fa680cd130262bbcd5a70fc9311e3b941dd557cffddd05ae9ea2a3c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-15T13:56:43.091 INFO:journalctl@ceph.mds.e.smithi052.stdout:May 15 13:56:42 smithi052 ceph-mds[66299]: -- 172.21.15.52:0/2789362953 <== mon.2 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555d65029a40 con 0x555d64ffcc00 2024-05-15T13:56:43.091 INFO:journalctl@ceph.mds.e.smithi052.stdout:May 15 13:56:42 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-e[66295]: starting mds.e at 2024-05-15T13:56:43.091 INFO:journalctl@ceph.mds.e.smithi052.stdout:May 15 13:56:42 smithi052 bash[66280]: ce9ab971fa680cd130262bbcd5a70fc9311e3b941dd557cffddd05ae9ea2a3c5 2024-05-15T13:56:43.091 INFO:journalctl@ceph.mds.e.smithi052.stdout:May 15 13:56:42 smithi052 systemd[1]: Started Ceph mds.e for bca78ca4-12c1-11ef-bc98-c7b262605968. 2024-05-15T13:56:44.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:43 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:44.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:43 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:44.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:43 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:44.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:43 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:56:44.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:43 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:56:44.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:43 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:44.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:43 smithi081 ceph-mon[36434]: Deploying daemon mds.f on smithi081 2024-05-15T13:56:44.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:43 smithi081 ceph-mon[36434]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:44.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:43 smithi081 ceph-mon[36434]: mds.? [v2:172.21.15.52:6834/3191964924,v1:172.21.15.52:6835/3191964924] up:boot 2024-05-15T13:56:44.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:43 smithi081 ceph-mon[36434]: fsmap 5 up:standby 2024-05-15T13:56:44.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:43 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-15T13:56:44.141 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:43 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:44.141 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:43 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:44.141 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:43 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:44.141 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:43 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:56:44.142 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:43 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:56:44.142 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:43 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:44.142 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:43 smithi033 ceph-mon[31650]: Deploying daemon mds.f on smithi081 2024-05-15T13:56:44.142 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:43 smithi033 ceph-mon[31650]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:44.142 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:43 smithi033 ceph-mon[31650]: mds.? [v2:172.21.15.52:6834/3191964924,v1:172.21.15.52:6835/3191964924] up:boot 2024-05-15T13:56:44.142 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:43 smithi033 ceph-mon[31650]: fsmap 5 up:standby 2024-05-15T13:56:44.142 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:43 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-15T13:56:44.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:43 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:44.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:43 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:44.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:43 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:44.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:43 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:56:44.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:43 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:56:44.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:43 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:44.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:43 smithi052 ceph-mon[36190]: Deploying daemon mds.f on smithi081 2024-05-15T13:56:44.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:43 smithi052 ceph-mon[36190]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:44.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:43 smithi052 ceph-mon[36190]: mds.? [v2:172.21.15.52:6834/3191964924,v1:172.21.15.52:6835/3191964924] up:boot 2024-05-15T13:56:44.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:43 smithi052 ceph-mon[36190]: fsmap 5 up:standby 2024-05-15T13:56:44.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:43 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-15T13:56:44.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.484+0000 7f9941b73700 1 -- 172.21.15.33:0/1557902410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f993c103fc0 msgr2=0x7f993c1103b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:44.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.484+0000 7f9941b73700 1 --2- 172.21.15.33:0/1557902410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f993c103fc0 0x7f993c1103b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f993400b0d0 tx=0x7f993400b3e0 comp rx=0 tx=0).stop 2024-05-15T13:56:44.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.484+0000 7f9941b73700 1 -- 172.21.15.33:0/1557902410 shutdown_connections 2024-05-15T13:56:44.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.484+0000 7f9941b73700 1 --2- 172.21.15.33:0/1557902410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f993c103fc0 0x7f993c1103b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:44.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.484+0000 7f9941b73700 1 --2- 172.21.15.33:0/1557902410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f993c1095e0 0x7f993c1099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:44.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.484+0000 7f9941b73700 1 --2- 172.21.15.33:0/1557902410 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f993c103600 0x7f993c103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:44.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.484+0000 7f9941b73700 1 -- 172.21.15.33:0/1557902410 >> 172.21.15.33:0/1557902410 conn(0x7f993c07a700 msgr2=0x7f993c07ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:56:44.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.484+0000 7f9941b73700 1 -- 172.21.15.33:0/1557902410 shutdown_connections 2024-05-15T13:56:44.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.484+0000 7f9941b73700 1 -- 172.21.15.33:0/1557902410 wait complete. 2024-05-15T13:56:44.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.484+0000 7f9941b73700 1 Processor -- start 2024-05-15T13:56:44.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.484+0000 7f9941b73700 1 -- start start 2024-05-15T13:56:44.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.485+0000 7f9941b73700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f993c103600 0x7f993c19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:44.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.485+0000 7f9941b73700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f993c103fc0 0x7f993c19fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:44.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.485+0000 7f9941b73700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f993c1095e0 0x7f993c199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:44.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.485+0000 7f9941b73700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f993c1137b0 con 0x7f993c103600 2024-05-15T13:56:44.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.485+0000 7f9941b73700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f993c113630 con 0x7f993c103fc0 2024-05-15T13:56:44.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.485+0000 7f9941b73700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f993c113930 con 0x7f993c1095e0 2024-05-15T13:56:44.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.485+0000 7f993b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f993c103600 0x7f993c19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:44.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.485+0000 7f993bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f993c1095e0 0x7f993c199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:44.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.485+0000 7f993b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f993c103600 0x7f993c19f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34614/0 (socket says 172.21.15.33:34614) 2024-05-15T13:56:44.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.485+0000 7f993b7fe700 1 -- 172.21.15.33:0/407276264 learned_addr learned my addr 172.21.15.33:0/407276264 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:56:44.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.485+0000 7f993affd700 1 --2- 172.21.15.33:0/407276264 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f993c103fc0 0x7f993c19fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:44.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.485+0000 7f993b7fe700 1 -- 172.21.15.33:0/407276264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f993c1095e0 msgr2=0x7f993c199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:44.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.485+0000 7f993b7fe700 1 --2- 172.21.15.33:0/407276264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f993c1095e0 0x7f993c199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:44.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.485+0000 7f993b7fe700 1 -- 172.21.15.33:0/407276264 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f993c103fc0 msgr2=0x7f993c19fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:44.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.485+0000 7f993b7fe700 1 --2- 172.21.15.33:0/407276264 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f993c103fc0 0x7f993c19fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:44.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.485+0000 7f993b7fe700 1 -- 172.21.15.33:0/407276264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f993c19a190 con 0x7f993c103600 2024-05-15T13:56:44.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.485+0000 7f993b7fe700 1 --2- 172.21.15.33:0/407276264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f993c103600 0x7f993c19f820 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f992c00b530 tx=0x7f992c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:56:44.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.486+0000 7f9938ff9700 1 -- 172.21.15.33:0/407276264 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c015020 con 0x7f993c103600 2024-05-15T13:56:44.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.486+0000 7f9941b73700 1 -- 172.21.15.33:0/407276264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f993c19a480 con 0x7f993c103600 2024-05-15T13:56:44.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.486+0000 7f9938ff9700 1 -- 172.21.15.33:0/407276264 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f992c011ce0 con 0x7f993c103600 2024-05-15T13:56:44.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.486+0000 7f9938ff9700 1 -- 172.21.15.33:0/407276264 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c010dd0 con 0x7f993c103600 2024-05-15T13:56:44.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.486+0000 7f9941b73700 1 -- 172.21.15.33:0/407276264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f993c1a6770 con 0x7f993c103600 2024-05-15T13:56:44.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.487+0000 7f9941b73700 1 -- 172.21.15.33:0/407276264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f993c10ba70 con 0x7f993c103600 2024-05-15T13:56:44.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.488+0000 7f9938ff9700 1 -- 172.21.15.33:0/407276264 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f992c018020 con 0x7f993c103600 2024-05-15T13:56:44.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.489+0000 7f9938ff9700 1 --2- 172.21.15.33:0/407276264 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f99240b2880 0x7f99240b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:44.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.489+0000 7f9938ff9700 1 -- 172.21.15.33:0/407276264 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f992c0e19d0 con 0x7f993c103600 2024-05-15T13:56:44.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.489+0000 7f993affd700 1 --2- 172.21.15.33:0/407276264 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f99240b2880 0x7f99240b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:44.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.489+0000 7f993affd700 1 --2- 172.21.15.33:0/407276264 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f99240b2880 0x7f99240b4d40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f993c07ad00 tx=0x7f993000b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:56:44.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.492+0000 7f9938ff9700 1 -- 172.21.15.33:0/407276264 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f992c01f020 con 0x7f993c103600 2024-05-15T13:56:44.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.751+0000 7f9941b73700 1 -- 172.21.15.33:0/407276264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f993c06a230 con 0x7f993c103600 2024-05-15T13:56:44.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.752+0000 7f9938ff9700 1 -- 172.21.15.33:0/407276264 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7f992c0a9b90 con 0x7f993c103600 2024-05-15T13:56:44.753 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:56:44.753 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-15T13:56:44.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.754+0000 7f9941b73700 1 -- 172.21.15.33:0/407276264 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f99240b2880 msgr2=0x7f99240b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:44.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.754+0000 7f9941b73700 1 --2- 172.21.15.33:0/407276264 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f99240b2880 0x7f99240b4d40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f993c07ad00 tx=0x7f993000b3c0 comp rx=0 tx=0).stop 2024-05-15T13:56:44.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.754+0000 7f9941b73700 1 -- 172.21.15.33:0/407276264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f993c103600 msgr2=0x7f993c19f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:44.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.754+0000 7f9941b73700 1 --2- 172.21.15.33:0/407276264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f993c103600 0x7f993c19f820 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f992c00b530 tx=0x7f992c00b840 comp rx=0 tx=0).stop 2024-05-15T13:56:44.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.754+0000 7f9941b73700 1 -- 172.21.15.33:0/407276264 shutdown_connections 2024-05-15T13:56:44.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.754+0000 7f9941b73700 1 --2- 172.21.15.33:0/407276264 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f99240b2880 0x7f99240b4d40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:44.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.754+0000 7f9941b73700 1 --2- 172.21.15.33:0/407276264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f993c1095e0 0x7f993c199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:44.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.754+0000 7f9941b73700 1 --2- 172.21.15.33:0/407276264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f993c103600 0x7f993c19f820 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:44.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.754+0000 7f9941b73700 1 --2- 172.21.15.33:0/407276264 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f993c103fc0 0x7f993c19fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:44.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.754+0000 7f9941b73700 1 -- 172.21.15.33:0/407276264 >> 172.21.15.33:0/407276264 conn(0x7f993c07a700 msgr2=0x7f993c102200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:56:44.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.754+0000 7f9941b73700 1 -- 172.21.15.33:0/407276264 shutdown_connections 2024-05-15T13:56:44.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:44.754+0000 7f9941b73700 1 -- 172.21.15.33:0/407276264 wait complete. 2024-05-15T13:56:44.757 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 6 2024-05-15T13:56:45.090 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:44 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/407276264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:56:45.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:44 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/407276264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:56:45.107 INFO:journalctl@ceph.mds.f.smithi081.stdout:May 15 13:56:44 smithi081 systemd[1]: Starting Ceph mds.f for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T13:56:45.155 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:44 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/407276264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:56:45.438 INFO:journalctl@ceph.mds.f.smithi081.stdout:May 15 13:56:45 smithi081 podman[66249]: 2024-05-15 13:56:45.068582844 +0000 UTC m=+0.017318754 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-15T13:56:45.438 INFO:journalctl@ceph.mds.f.smithi081.stdout:May 15 13:56:45 smithi081 podman[66249]: 2024-05-15 13:56:45.167235705 +0000 UTC m=+0.115971612 container create 031e85e5354540e45372fc76d2b9bf24c1416b65b3736c92c271e50a76077d09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-15T13:56:45.439 INFO:journalctl@ceph.mds.f.smithi081.stdout:May 15 13:56:45 smithi081 podman[66249]: 2024-05-15 13:56:45.41144506 +0000 UTC m=+0.360180969 container init 031e85e5354540e45372fc76d2b9bf24c1416b65b3736c92c271e50a76077d09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-f, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-15T13:56:45.439 INFO:journalctl@ceph.mds.f.smithi081.stdout:May 15 13:56:45 smithi081 podman[66249]: 2024-05-15 13:56:45.414120101 +0000 UTC m=+0.362856001 container start 031e85e5354540e45372fc76d2b9bf24c1416b65b3736c92c271e50a76077d09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T13:56:45.857 INFO:journalctl@ceph.mds.f.smithi081.stdout:May 15 13:56:45 smithi081 ceph-mds[66268]: -- 172.21.15.81:0/876552879 <== mon.1 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557b9fc95a40 con 0x557b9fc68800 2024-05-15T13:56:45.857 INFO:journalctl@ceph.mds.f.smithi081.stdout:May 15 13:56:45 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-f[66264]: starting mds.f at 2024-05-15T13:56:45.857 INFO:journalctl@ceph.mds.f.smithi081.stdout:May 15 13:56:45 smithi081 bash[66249]: 031e85e5354540e45372fc76d2b9bf24c1416b65b3736c92c271e50a76077d09 2024-05-15T13:56:45.857 INFO:journalctl@ceph.mds.f.smithi081.stdout:May 15 13:56:45 smithi081 systemd[1]: Started Ceph mds.f for bca78ca4-12c1-11ef-bc98-c7b262605968. 2024-05-15T13:56:46.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:45 smithi033 ceph-mon[31650]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:46.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:45 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:46.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:45 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:46.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:45 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:46.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:45 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:56:46.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:45 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:56:46.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:45 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:46.230 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:45 smithi081 ceph-mon[36434]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:46.230 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:45 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:46.230 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:45 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:46.231 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:45 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:46.231 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:45 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:56:46.231 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:45 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:56:46.231 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:45 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:46.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:45 smithi052 ceph-mon[36190]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:46.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:45 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:46.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:45 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:46.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:45 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:46.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:45 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:56:46.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:45 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:56:46.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:45 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:47.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:46 smithi033 ceph-mon[31650]: Deploying daemon mds.c on smithi081 2024-05-15T13:56:47.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:46 smithi033 ceph-mon[31650]: mds.? [v2:172.21.15.81:6834/3773839758,v1:172.21.15.81:6835/3773839758] up:boot 2024-05-15T13:56:47.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:46 smithi033 ceph-mon[31650]: fsmap 6 up:standby 2024-05-15T13:56:47.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:46 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-15T13:56:47.227 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:46 smithi081 ceph-mon[36434]: Deploying daemon mds.c on smithi081 2024-05-15T13:56:47.228 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:46 smithi081 ceph-mon[36434]: mds.? [v2:172.21.15.81:6834/3773839758,v1:172.21.15.81:6835/3773839758] up:boot 2024-05-15T13:56:47.228 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:46 smithi081 ceph-mon[36434]: fsmap 6 up:standby 2024-05-15T13:56:47.228 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:46 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-15T13:56:47.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:46 smithi052 ceph-mon[36190]: Deploying daemon mds.c on smithi081 2024-05-15T13:56:47.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:46 smithi052 ceph-mon[36190]: mds.? [v2:172.21.15.81:6834/3773839758,v1:172.21.15.81:6835/3773839758] up:boot 2024-05-15T13:56:47.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:46 smithi052 ceph-mon[36190]: fsmap 6 up:standby 2024-05-15T13:56:47.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:46 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-15T13:56:47.466 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T13:56:47.729 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:56:47.857 INFO:journalctl@ceph.mds.c.smithi081.stdout:May 15 13:56:47 smithi081 systemd[1]: Starting Ceph mds.c for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T13:56:48.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:47 smithi033 ceph-mon[31650]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:48.225 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:47 smithi081 ceph-mon[36434]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:48.226 INFO:journalctl@ceph.mds.c.smithi081.stdout:May 15 13:56:47 smithi081 podman[66550]: 2024-05-15 13:56:47.814653076 +0000 UTC m=+0.017539490 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-15T13:56:48.226 INFO:journalctl@ceph.mds.c.smithi081.stdout:May 15 13:56:47 smithi081 podman[66550]: 2024-05-15 13:56:47.945718032 +0000 UTC m=+0.148604432 container create b722e0a0be650978392c956dc5573d0028a832f4ef87cdd769fe3f9d0445691f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-15T13:56:48.226 INFO:journalctl@ceph.mds.c.smithi081.stdout:May 15 13:56:48 smithi081 podman[66550]: 2024-05-15 13:56:48.223797822 +0000 UTC m=+0.426684223 container init b722e0a0be650978392c956dc5573d0028a832f4ef87cdd769fe3f9d0445691f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True) 2024-05-15T13:56:48.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:47 smithi052 ceph-mon[36190]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:48.607 INFO:journalctl@ceph.mds.c.smithi081.stdout:May 15 13:56:48 smithi081 podman[66550]: 2024-05-15 13:56:48.226966492 +0000 UTC m=+0.429852905 container start b722e0a0be650978392c956dc5573d0028a832f4ef87cdd769fe3f9d0445691f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-15T13:56:48.607 INFO:journalctl@ceph.mds.c.smithi081.stdout:May 15 13:56:48 smithi081 ceph-mds[66570]: -- 172.21.15.81:0/1622844649 <== mon.1 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bd62425a40 con 0x55bd623f9000 2024-05-15T13:56:48.607 INFO:journalctl@ceph.mds.c.smithi081.stdout:May 15 13:56:48 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-c[66566]: starting mds.c at 2024-05-15T13:56:48.607 INFO:journalctl@ceph.mds.c.smithi081.stdout:May 15 13:56:48 smithi081 bash[66550]: b722e0a0be650978392c956dc5573d0028a832f4ef87cdd769fe3f9d0445691f 2024-05-15T13:56:48.607 INFO:journalctl@ceph.mds.c.smithi081.stdout:May 15 13:56:48 smithi081 systemd[1]: Started Ceph mds.c for bca78ca4-12c1-11ef-bc98-c7b262605968. 2024-05-15T13:56:49.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.336+0000 7fd0baaac700 1 -- 172.21.15.33:0/2892718597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0b4102930 msgr2=0x7fd0b4110430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:49.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.336+0000 7fd0baaac700 1 --2- 172.21.15.33:0/2892718597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0b4102930 0x7fd0b4110430 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac00b0a0 tx=0x7fd0ac00b3b0 comp rx=0 tx=0).stop 2024-05-15T13:56:49.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.336+0000 7fd0baaac700 1 -- 172.21.15.33:0/2892718597 shutdown_connections 2024-05-15T13:56:49.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.336+0000 7fd0baaac700 1 --2- 172.21.15.33:0/2892718597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0b4102930 0x7fd0b4110430 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:49.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.336+0000 7fd0baaac700 1 --2- 172.21.15.33:0/2892718597 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0b4109720 0x7fd0b4109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:49.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.336+0000 7fd0baaac700 1 --2- 172.21.15.33:0/2892718597 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0b4101f70 0x7fd0b41023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:49.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.336+0000 7fd0baaac700 1 -- 172.21.15.33:0/2892718597 >> 172.21.15.33:0/2892718597 conn(0x7fd0b40776b0 msgr2=0x7fd0b4077ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:56:49.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.336+0000 7fd0baaac700 1 -- 172.21.15.33:0/2892718597 shutdown_connections 2024-05-15T13:56:49.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.336+0000 7fd0baaac700 1 -- 172.21.15.33:0/2892718597 wait complete. 2024-05-15T13:56:49.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.336+0000 7fd0baaac700 1 Processor -- start 2024-05-15T13:56:49.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.337+0000 7fd0baaac700 1 -- start start 2024-05-15T13:56:49.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.337+0000 7fd0baaac700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0b4101f70 0x7fd0b4103150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:49.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.337+0000 7fd0baaac700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0b4102930 0x7fd0b4103690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:49.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.337+0000 7fd0baaac700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0b4109720 0x7fd0b4103bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:49.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.337+0000 7fd0baaac700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b4113750 con 0x7fd0b4109720 2024-05-15T13:56:49.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.337+0000 7fd0baaac700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b41135d0 con 0x7fd0b4101f70 2024-05-15T13:56:49.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.337+0000 7fd0baaac700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b41138d0 con 0x7fd0b4102930 2024-05-15T13:56:49.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.337+0000 7fd0b9049700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0b4109720 0x7fd0b4103bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:49.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.337+0000 7fd0b9049700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0b4109720 0x7fd0b4103bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34628/0 (socket says 172.21.15.33:34628) 2024-05-15T13:56:49.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.337+0000 7fd0b9049700 1 -- 172.21.15.33:0/941640693 learned_addr learned my addr 172.21.15.33:0/941640693 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:56:49.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.337+0000 7fd0b3fff700 1 --2- 172.21.15.33:0/941640693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0b4102930 0x7fd0b4103690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:49.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.337+0000 7fd0b8848700 1 --2- 172.21.15.33:0/941640693 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0b4101f70 0x7fd0b4103150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:49.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.337+0000 7fd0b9049700 1 -- 172.21.15.33:0/941640693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0b4102930 msgr2=0x7fd0b4103690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:49.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.337+0000 7fd0b9049700 1 --2- 172.21.15.33:0/941640693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0b4102930 0x7fd0b4103690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:49.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.337+0000 7fd0b9049700 1 -- 172.21.15.33:0/941640693 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0b4101f70 msgr2=0x7fd0b4103150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:49.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.337+0000 7fd0b9049700 1 --2- 172.21.15.33:0/941640693 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0b4101f70 0x7fd0b4103150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:49.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.337+0000 7fd0b9049700 1 -- 172.21.15.33:0/941640693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b41a8170 con 0x7fd0b4109720 2024-05-15T13:56:49.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.337+0000 7fd0b3fff700 1 --2- 172.21.15.33:0/941640693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0b4102930 0x7fd0b4103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T13:56:49.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.338+0000 7fd0b9049700 1 --2- 172.21.15.33:0/941640693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0b4109720 0x7fd0b4103bd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac00b830 tx=0x7fd0ac004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:56:49.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.338+0000 7fd0b1ffb700 1 -- 172.21.15.33:0/941640693 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac021070 con 0x7fd0b4109720 2024-05-15T13:56:49.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.338+0000 7fd0baaac700 1 -- 172.21.15.33:0/941640693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0b41a8400 con 0x7fd0b4109720 2024-05-15T13:56:49.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.338+0000 7fd0b1ffb700 1 -- 172.21.15.33:0/941640693 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0ac007e30 con 0x7fd0b4109720 2024-05-15T13:56:49.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.338+0000 7fd0b1ffb700 1 -- 172.21.15.33:0/941640693 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac004700 con 0x7fd0b4109720 2024-05-15T13:56:49.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.338+0000 7fd0baaac700 1 -- 172.21.15.33:0/941640693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0b41a8900 con 0x7fd0b4109720 2024-05-15T13:56:49.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.339+0000 7fd0baaac700 1 -- 172.21.15.33:0/941640693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0b406a230 con 0x7fd0b4109720 2024-05-15T13:56:49.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.341+0000 7fd0b1ffb700 1 -- 172.21.15.33:0/941640693 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd0ac003d70 con 0x7fd0b4109720 2024-05-15T13:56:49.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.341+0000 7fd0b1ffb700 1 --2- 172.21.15.33:0/941640693 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fd0a40b2880 0x7fd0a40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:49.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.342+0000 7fd0b8848700 1 --2- 172.21.15.33:0/941640693 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fd0a40b2880 0x7fd0a40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:49.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.342+0000 7fd0b1ffb700 1 -- 172.21.15.33:0/941640693 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fd0ac00f020 con 0x7fd0b4109720 2024-05-15T13:56:49.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.342+0000 7fd0b8848700 1 --2- 172.21.15.33:0/941640693 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fd0a40b2880 0x7fd0a40b4d40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8005950 tx=0x7fd0a800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:56:49.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.349+0000 7fd0b1ffb700 1 -- 172.21.15.33:0/941640693 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd0ac02a020 con 0x7fd0b4109720 2024-05-15T13:56:49.387 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:49 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:49.387 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:49 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:49.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.612+0000 7fd0baaac700 1 -- 172.21.15.33:0/941640693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0b402e560 con 0x7fd0b4109720 2024-05-15T13:56:49.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.612+0000 7fd0b1ffb700 1 -- 172.21.15.33:0/941640693 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6340 (secure 0 0 0) 0x7fd0ac0ab890 con 0x7fd0b4109720 2024-05-15T13:56:49.614 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:56:49.614 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/3773839758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3773839758},{"type":"v1","addr":"172.21.15.81:6835","nonce":3773839758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-15T13:56:49.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.615+0000 7fd0baaac700 1 -- 172.21.15.33:0/941640693 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fd0a40b2880 msgr2=0x7fd0a40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:49.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.615+0000 7fd0baaac700 1 --2- 172.21.15.33:0/941640693 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fd0a40b2880 0x7fd0a40b4d40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8005950 tx=0x7fd0a800b410 comp rx=0 tx=0).stop 2024-05-15T13:56:49.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.615+0000 7fd0baaac700 1 -- 172.21.15.33:0/941640693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0b4109720 msgr2=0x7fd0b4103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:49.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.615+0000 7fd0baaac700 1 --2- 172.21.15.33:0/941640693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0b4109720 0x7fd0b4103bd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac00b830 tx=0x7fd0ac004ce0 comp rx=0 tx=0).stop 2024-05-15T13:56:49.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.615+0000 7fd0baaac700 1 -- 172.21.15.33:0/941640693 shutdown_connections 2024-05-15T13:56:49.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.615+0000 7fd0baaac700 1 --2- 172.21.15.33:0/941640693 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fd0a40b2880 0x7fd0a40b4d40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:49.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.615+0000 7fd0baaac700 1 --2- 172.21.15.33:0/941640693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0b4109720 0x7fd0b4103bd0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:49.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.615+0000 7fd0baaac700 1 --2- 172.21.15.33:0/941640693 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0b4101f70 0x7fd0b4103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:49.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.615+0000 7fd0baaac700 1 --2- 172.21.15.33:0/941640693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0b4102930 0x7fd0b4103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:49.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.615+0000 7fd0baaac700 1 -- 172.21.15.33:0/941640693 >> 172.21.15.33:0/941640693 conn(0x7fd0b40776b0 msgr2=0x7fd0b4100a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:56:49.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.615+0000 7fd0baaac700 1 -- 172.21.15.33:0/941640693 shutdown_connections 2024-05-15T13:56:49.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:49.615+0000 7fd0baaac700 1 -- 172.21.15.33:0/941640693 wait complete. 2024-05-15T13:56:49.618 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 8 2024-05-15T13:56:49.657 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:49 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:49.657 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:49 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:49.657 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:49 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:49.657 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:49 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:56:49.657 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:49 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:56:49.657 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:49 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:49.657 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:49 smithi081 ceph-mon[36434]: Deploying daemon mds.h on smithi052 2024-05-15T13:56:49.657 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:49 smithi081 ceph-mon[36434]: mds.? [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] up:boot 2024-05-15T13:56:49.658 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:49 smithi081 ceph-mon[36434]: fsmap 7 up:standby 2024-05-15T13:56:49.658 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:49 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-15T13:56:49.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:49 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:49.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:49 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:56:49.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:49 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:56:49.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:49 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:49.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:49 smithi033 ceph-mon[31650]: Deploying daemon mds.h on smithi052 2024-05-15T13:56:49.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:49 smithi033 ceph-mon[31650]: mds.? [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] up:boot 2024-05-15T13:56:49.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:49 smithi033 ceph-mon[31650]: fsmap 7 up:standby 2024-05-15T13:56:49.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:49 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-15T13:56:49.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:49 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:49.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:49 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:49.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:49 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:49.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:49 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:56:49.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:49 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:56:49.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:49 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:49.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:49 smithi052 ceph-mon[36190]: Deploying daemon mds.h on smithi052 2024-05-15T13:56:49.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:49 smithi052 ceph-mon[36190]: mds.? [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] up:boot 2024-05-15T13:56:49.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:49 smithi052 ceph-mon[36190]: fsmap 7 up:standby 2024-05-15T13:56:49.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:49 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-15T13:56:50.552 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:50 smithi033 ceph-mon[31650]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:50.552 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:50 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/941640693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:56:50.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:50 smithi052 ceph-mon[36190]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:50.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:50 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/941640693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:56:50.591 INFO:journalctl@ceph.mds.h.smithi052.stdout:May 15 13:56:50 smithi052 systemd[1]: Starting Ceph mds.h for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T13:56:50.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:50 smithi081 ceph-mon[36434]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:50.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:50 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/941640693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:56:51.018 INFO:journalctl@ceph.mds.h.smithi052.stdout:May 15 13:56:50 smithi052 podman[66598]: 2024-05-15 13:56:50.618763829 +0000 UTC m=+0.017928933 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-15T13:56:51.018 INFO:journalctl@ceph.mds.h.smithi052.stdout:May 15 13:56:50 smithi052 podman[66598]: 2024-05-15 13:56:50.722543119 +0000 UTC m=+0.121708225 container create ac645799e591769b04b962c73adcf5b72e1ae2b57f67f859888cd19a2106eec9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-h, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-15T13:56:51.018 INFO:journalctl@ceph.mds.h.smithi052.stdout:May 15 13:56:50 smithi052 podman[66598]: 2024-05-15 13:56:50.925171499 +0000 UTC m=+0.324336603 container init ac645799e591769b04b962c73adcf5b72e1ae2b57f67f859888cd19a2106eec9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-h, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240514) 2024-05-15T13:56:51.018 INFO:journalctl@ceph.mds.h.smithi052.stdout:May 15 13:56:50 smithi052 podman[66598]: 2024-05-15 13:56:50.928052104 +0000 UTC m=+0.327217208 container start ac645799e591769b04b962c73adcf5b72e1ae2b57f67f859888cd19a2106eec9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-h, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-15T13:56:51.018 INFO:journalctl@ceph.mds.h.smithi052.stdout:May 15 13:56:50 smithi052 ceph-mds[66617]: -- 172.21.15.52:0/2616383288 <== mon.2 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f11c6b3a40 con 0x55f11c686c00 2024-05-15T13:56:51.018 INFO:journalctl@ceph.mds.h.smithi052.stdout:May 15 13:56:50 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-h[66613]: starting mds.h at 2024-05-15T13:56:51.018 INFO:journalctl@ceph.mds.h.smithi052.stdout:May 15 13:56:51 smithi052 bash[66598]: ac645799e591769b04b962c73adcf5b72e1ae2b57f67f859888cd19a2106eec9 2024-05-15T13:56:51.340 INFO:journalctl@ceph.mds.h.smithi052.stdout:May 15 13:56:51 smithi052 systemd[1]: Started Ceph mds.h for bca78ca4-12c1-11ef-bc98-c7b262605968. 2024-05-15T13:56:52.270 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T13:56:52.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:52 smithi052 ceph-mon[36190]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:52.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:52 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:52.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:52 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:52.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:52 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:52.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:52 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:56:52.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:52 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:56:52.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:52 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:52.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:52 smithi052 ceph-mon[36190]: Deploying daemon mds.i on smithi081 2024-05-15T13:56:52.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:52 smithi052 ceph-mon[36190]: mds.? [v2:172.21.15.52:6836/1629697841,v1:172.21.15.52:6837/1629697841] up:boot 2024-05-15T13:56:52.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:52 smithi052 ceph-mon[36190]: fsmap 8 up:standby 2024-05-15T13:56:52.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:52 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-15T13:56:52.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:52 smithi081 ceph-mon[36434]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:52.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:52 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:52.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:52 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:52.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:52 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:52.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:52 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:56:52.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:52 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:56:52.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:52 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:52.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:52 smithi081 ceph-mon[36434]: Deploying daemon mds.i on smithi081 2024-05-15T13:56:52.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:52 smithi081 ceph-mon[36434]: mds.? [v2:172.21.15.52:6836/1629697841,v1:172.21.15.52:6837/1629697841] up:boot 2024-05-15T13:56:52.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:52 smithi081 ceph-mon[36434]: fsmap 8 up:standby 2024-05-15T13:56:52.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:52 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-15T13:56:52.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:52 smithi033 ceph-mon[31650]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:52.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:52 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:52.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:52 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:52.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:52 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:52.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:52 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:56:52.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:52 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:56:52.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:52 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:52.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:52 smithi033 ceph-mon[31650]: Deploying daemon mds.i on smithi081 2024-05-15T13:56:52.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:52 smithi033 ceph-mon[31650]: mds.? [v2:172.21.15.52:6836/1629697841,v1:172.21.15.52:6837/1629697841] up:boot 2024-05-15T13:56:52.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:52 smithi033 ceph-mon[31650]: fsmap 8 up:standby 2024-05-15T13:56:52.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:52 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-15T13:56:52.535 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:56:53.357 INFO:journalctl@ceph.mds.i.smithi081.stdout:May 15 13:56:53 smithi081 systemd[1]: Starting Ceph mds.i for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T13:56:53.663 INFO:journalctl@ceph.mds.i.smithi081.stdout:May 15 13:56:53 smithi081 podman[66869]: 2024-05-15 13:56:53.303970172 +0000 UTC m=+0.017330728 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-15T13:56:53.663 INFO:journalctl@ceph.mds.i.smithi081.stdout:May 15 13:56:53 smithi081 podman[66869]: 2024-05-15 13:56:53.408755594 +0000 UTC m=+0.122116142 container create 63576537a4ee5c900faad68ff9a5eeb06d039f0bb760e4f00e92b9f3938407fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-i, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-15T13:56:53.664 INFO:journalctl@ceph.mds.i.smithi081.stdout:May 15 13:56:53 smithi081 podman[66869]: 2024-05-15 13:56:53.636353935 +0000 UTC m=+0.349714485 container init 63576537a4ee5c900faad68ff9a5eeb06d039f0bb760e4f00e92b9f3938407fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-i, ceph=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T13:56:53.664 INFO:journalctl@ceph.mds.i.smithi081.stdout:May 15 13:56:53 smithi081 podman[66869]: 2024-05-15 13:56:53.638942727 +0000 UTC m=+0.352303279 container start 63576537a4ee5c900faad68ff9a5eeb06d039f0bb760e4f00e92b9f3938407fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-i, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-15T13:56:53.664 INFO:journalctl@ceph.mds.i.smithi081.stdout:May 15 13:56:53 smithi081 ceph-mds[66888]: -- 172.21.15.81:0/2519795347 <== mon.1 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564b995e1a40 con 0x564b995b5000 2024-05-15T13:56:54.069 INFO:journalctl@ceph.mds.i.smithi081.stdout:May 15 13:56:53 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-i[66884]: starting mds.i at 2024-05-15T13:56:54.070 INFO:journalctl@ceph.mds.i.smithi081.stdout:May 15 13:56:53 smithi081 bash[66869]: 63576537a4ee5c900faad68ff9a5eeb06d039f0bb760e4f00e92b9f3938407fa 2024-05-15T13:56:54.070 INFO:journalctl@ceph.mds.i.smithi081.stdout:May 15 13:56:53 smithi081 systemd[1]: Started Ceph mds.i for bca78ca4-12c1-11ef-bc98-c7b262605968. 2024-05-15T13:56:54.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.163+0000 7fd373135700 1 -- 172.21.15.33:0/206907571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd36c1095b0 msgr2=0x7fd36c109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:54.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.163+0000 7fd373135700 1 --2- 172.21.15.33:0/206907571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd36c1095b0 0x7fd36c109990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd368009990 tx=0x7fd368009ca0 comp rx=0 tx=0).stop 2024-05-15T13:56:54.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.163+0000 7fd373135700 1 -- 172.21.15.33:0/206907571 shutdown_connections 2024-05-15T13:56:54.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.163+0000 7fd373135700 1 --2- 172.21.15.33:0/206907571 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd36c06bbe0 0x7fd36c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:54.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.163+0000 7fd373135700 1 --2- 172.21.15.33:0/206907571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd36c1095b0 0x7fd36c109990 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:54.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.163+0000 7fd373135700 1 --2- 172.21.15.33:0/206907571 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd36c06b220 0x7fd36c06b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:54.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.163+0000 7fd373135700 1 -- 172.21.15.33:0/206907571 >> 172.21.15.33:0/206907571 conn(0x7fd36c0793f0 msgr2=0x7fd36c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:56:54.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.163+0000 7fd373135700 1 -- 172.21.15.33:0/206907571 shutdown_connections 2024-05-15T13:56:54.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.163+0000 7fd373135700 1 -- 172.21.15.33:0/206907571 wait complete. 2024-05-15T13:56:54.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.164+0000 7fd373135700 1 Processor -- start 2024-05-15T13:56:54.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.164+0000 7fd373135700 1 -- start start 2024-05-15T13:56:54.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.164+0000 7fd373135700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd36c06b220 0x7fd36c102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:54.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.164+0000 7fd373135700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd36c06bbe0 0x7fd36c102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:54.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.164+0000 7fd373135700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd36c1095b0 0x7fd36c102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:54.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.164+0000 7fd373135700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c113740 con 0x7fd36c1095b0 2024-05-15T13:56:54.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.164+0000 7fd373135700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c1135c0 con 0x7fd36c06bbe0 2024-05-15T13:56:54.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.164+0000 7fd373135700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c1138c0 con 0x7fd36c06b220 2024-05-15T13:56:54.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.165+0000 7fd3716d2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd36c1095b0 0x7fd36c102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:54.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.165+0000 7fd3716d2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd36c1095b0 0x7fd36c102f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39106/0 (socket says 172.21.15.33:39106) 2024-05-15T13:56:54.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.165+0000 7fd3716d2700 1 -- 172.21.15.33:0/2730680527 learned_addr learned my addr 172.21.15.33:0/2730680527 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:56:54.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.165+0000 7fd363fff700 1 --2- 172.21.15.33:0/2730680527 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd36c06bbe0 0x7fd36c102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:54.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.165+0000 7fd370ed1700 1 --2- 172.21.15.33:0/2730680527 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd36c06b220 0x7fd36c102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:54.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.165+0000 7fd3716d2700 1 -- 172.21.15.33:0/2730680527 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd36c06b220 msgr2=0x7fd36c102510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:54.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.165+0000 7fd3716d2700 1 --2- 172.21.15.33:0/2730680527 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd36c06b220 0x7fd36c102510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:54.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.165+0000 7fd3716d2700 1 -- 172.21.15.33:0/2730680527 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd36c06bbe0 msgr2=0x7fd36c102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:54.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.165+0000 7fd3716d2700 1 --2- 172.21.15.33:0/2730680527 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd36c06bbe0 0x7fd36c102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:54.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.165+0000 7fd3716d2700 1 -- 172.21.15.33:0/2730680527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd36c1a8180 con 0x7fd36c1095b0 2024-05-15T13:56:54.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.165+0000 7fd370ed1700 1 --2- 172.21.15.33:0/2730680527 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd36c06b220 0x7fd36c102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T13:56:54.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.165+0000 7fd363fff700 1 --2- 172.21.15.33:0/2730680527 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd36c06bbe0 0x7fd36c102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T13:56:54.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.165+0000 7fd3716d2700 1 --2- 172.21.15.33:0/2730680527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd36c1095b0 0x7fd36c102f90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd36400b970 tx=0x7fd36400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:56:54.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.165+0000 7fd361ffb700 1 -- 172.21.15.33:0/2730680527 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36400d860 con 0x7fd36c1095b0 2024-05-15T13:56:54.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.165+0000 7fd373135700 1 -- 172.21.15.33:0/2730680527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd36c1a8470 con 0x7fd36c1095b0 2024-05-15T13:56:54.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.165+0000 7fd361ffb700 1 -- 172.21.15.33:0/2730680527 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd36400e430 con 0x7fd36c1095b0 2024-05-15T13:56:54.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.165+0000 7fd361ffb700 1 -- 172.21.15.33:0/2730680527 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd364017660 con 0x7fd36c1095b0 2024-05-15T13:56:54.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.166+0000 7fd373135700 1 -- 172.21.15.33:0/2730680527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd36c1a89b0 con 0x7fd36c1095b0 2024-05-15T13:56:54.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.167+0000 7fd373135700 1 -- 172.21.15.33:0/2730680527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd36c10ba90 con 0x7fd36c1095b0 2024-05-15T13:56:54.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.168+0000 7fd361ffb700 1 -- 172.21.15.33:0/2730680527 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd36400da00 con 0x7fd36c1095b0 2024-05-15T13:56:54.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.169+0000 7fd361ffb700 1 --2- 172.21.15.33:0/2730680527 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fd3540b2880 0x7fd3540b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:56:54.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.169+0000 7fd370ed1700 1 --2- 172.21.15.33:0/2730680527 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fd3540b2880 0x7fd3540b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:56:54.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.169+0000 7fd361ffb700 1 -- 172.21.15.33:0/2730680527 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fd3640e1bd0 con 0x7fd36c1095b0 2024-05-15T13:56:54.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.170+0000 7fd370ed1700 1 --2- 172.21.15.33:0/2730680527 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fd3540b2880 0x7fd3540b4d40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd36800b5c0 tx=0x7fd368005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:56:54.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.172+0000 7fd361ffb700 1 -- 172.21.15.33:0/2730680527 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd364053df0 con 0x7fd36c1095b0 2024-05-15T13:56:54.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:54 smithi052 ceph-mon[36190]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:54.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:54 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:54.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:54 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:54.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:54 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:54.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:54 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:56:54.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:54 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:56:54.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:54 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:54.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:54 smithi081 ceph-mon[36434]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:54.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:54 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:54.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:54 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:54.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:54 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:54.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:54 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:56:54.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:54 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:56:54.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:54 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:54.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:54 smithi033 ceph-mon[31650]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:54.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:54 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:54.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:54 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:54.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:54 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:54.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:54 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:56:54.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:54 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:56:54.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:54 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:54.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.464+0000 7fd373135700 1 -- 172.21.15.33:0/2730680527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd36c10b470 con 0x7fd36c1095b0 2024-05-15T13:56:54.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.464+0000 7fd361ffb700 1 -- 172.21.15.33:0/2730680527 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7992 (secure 0 0 0) 0x7fd364013030 con 0x7fd36c1095b0 2024-05-15T13:56:54.466 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:56:54.466 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/1629697841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1629697841},{"type":"v1","addr":"172.21.15.52:6837","nonce":1629697841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/3773839758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3773839758},{"type":"v1","addr":"172.21.15.81:6835","nonce":3773839758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6839/1554395940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":1554395940},{"type":"v1","addr":"172.21.15.81:6839","nonce":1554395940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-15T13:56:54.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.467+0000 7fd373135700 1 -- 172.21.15.33:0/2730680527 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fd3540b2880 msgr2=0x7fd3540b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:54.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.467+0000 7fd373135700 1 --2- 172.21.15.33:0/2730680527 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fd3540b2880 0x7fd3540b4d40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd36800b5c0 tx=0x7fd368005960 comp rx=0 tx=0).stop 2024-05-15T13:56:54.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.467+0000 7fd373135700 1 -- 172.21.15.33:0/2730680527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd36c1095b0 msgr2=0x7fd36c102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:56:54.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.467+0000 7fd373135700 1 --2- 172.21.15.33:0/2730680527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd36c1095b0 0x7fd36c102f90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd36400b970 tx=0x7fd36400bc80 comp rx=0 tx=0).stop 2024-05-15T13:56:54.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.467+0000 7fd373135700 1 -- 172.21.15.33:0/2730680527 shutdown_connections 2024-05-15T13:56:54.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.467+0000 7fd373135700 1 --2- 172.21.15.33:0/2730680527 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fd3540b2880 0x7fd3540b4d40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:54.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.467+0000 7fd373135700 1 --2- 172.21.15.33:0/2730680527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd36c1095b0 0x7fd36c102f90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:54.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.467+0000 7fd373135700 1 --2- 172.21.15.33:0/2730680527 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd36c06b220 0x7fd36c102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:54.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.467+0000 7fd373135700 1 --2- 172.21.15.33:0/2730680527 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd36c06bbe0 0x7fd36c102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:56:54.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.467+0000 7fd373135700 1 -- 172.21.15.33:0/2730680527 >> 172.21.15.33:0/2730680527 conn(0x7fd36c0793f0 msgr2=0x7fd36c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:56:54.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.467+0000 7fd373135700 1 -- 172.21.15.33:0/2730680527 shutdown_connections 2024-05-15T13:56:54.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:56:54.467+0000 7fd373135700 1 -- 172.21.15.33:0/2730680527 wait complete. 2024-05-15T13:56:54.470 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 10 2024-05-15T13:56:55.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:55 smithi052 ceph-mon[36190]: Deploying daemon mds.g on smithi033 2024-05-15T13:56:55.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:55 smithi052 ceph-mon[36190]: mds.? [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] up:boot 2024-05-15T13:56:55.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:55 smithi052 ceph-mon[36190]: fsmap 9 up:standby 2024-05-15T13:56:55.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:55 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-15T13:56:55.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:55 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/2730680527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:56:55.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:55 smithi081 ceph-mon[36434]: Deploying daemon mds.g on smithi033 2024-05-15T13:56:55.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:55 smithi081 ceph-mon[36434]: mds.? [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] up:boot 2024-05-15T13:56:55.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:55 smithi081 ceph-mon[36434]: fsmap 9 up:standby 2024-05-15T13:56:55.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:55 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-15T13:56:55.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:55 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/2730680527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:56:55.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:55 smithi033 ceph-mon[31650]: Deploying daemon mds.g on smithi033 2024-05-15T13:56:55.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:55 smithi033 ceph-mon[31650]: mds.? [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] up:boot 2024-05-15T13:56:55.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:55 smithi033 ceph-mon[31650]: fsmap 9 up:standby 2024-05-15T13:56:55.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:55 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-15T13:56:55.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:55 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/2730680527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:56:56.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:56 smithi052 ceph-mon[36190]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:56.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:56 smithi081 ceph-mon[36434]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:56.399 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:56 smithi033 ceph-mon[31650]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:56.690 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 15 13:56:56 smithi033 systemd[1]: Starting Ceph mds.g for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T13:56:57.113 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 15 13:56:56 smithi033 podman[70600]: 2024-05-15 13:56:56.71277346 +0000 UTC m=+0.017953674 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-15T13:56:57.113 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 15 13:56:56 smithi033 podman[70600]: 2024-05-15 13:56:56.85915678 +0000 UTC m=+0.164337054 container create a1133de4c9562db3746410cd464b2c93547d0afdc0d8419e2d7e3a0bb08dcac4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS) 2024-05-15T13:56:57.113 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 15 13:56:57 smithi033 podman[70600]: 2024-05-15 13:56:57.111702721 +0000 UTC m=+0.416882930 container init a1133de4c9562db3746410cd464b2c93547d0afdc0d8419e2d7e3a0bb08dcac4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-g, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-15T13:56:57.441 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 15 13:56:57 smithi033 podman[70600]: 2024-05-15 13:56:57.114897174 +0000 UTC m=+0.420077382 container start a1133de4c9562db3746410cd464b2c93547d0afdc0d8419e2d7e3a0bb08dcac4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-g, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-15T13:56:57.441 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 15 13:56:57 smithi033 ceph-mds[70619]: -- 172.21.15.33:0/3595578907 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559e76611a40 con 0x559e765e4800 2024-05-15T13:56:57.441 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 15 13:56:57 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-g[70615]: starting mds.g at 2024-05-15T13:56:57.441 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 15 13:56:57 smithi033 bash[70600]: a1133de4c9562db3746410cd464b2c93547d0afdc0d8419e2d7e3a0bb08dcac4 2024-05-15T13:56:57.441 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 15 13:56:57 smithi033 systemd[1]: Started Ceph mds.g for bca78ca4-12c1-11ef-bc98-c7b262605968. 2024-05-15T13:56:57.665 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T13:56:58.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:58 smithi052 ceph-mon[36190]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:58.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:58.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:58.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:58.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:56:58.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:56:58.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:58.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:58 smithi052 ceph-mon[36190]: Deploying daemon mds.a on smithi033 2024-05-15T13:56:58.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:58 smithi052 ceph-mon[36190]: mds.? [v2:172.21.15.33:6838/2030117633,v1:172.21.15.33:6839/2030117633] up:boot 2024-05-15T13:56:58.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:58 smithi052 ceph-mon[36190]: fsmap 10 up:standby 2024-05-15T13:56:58.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:56:58 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-15T13:56:58.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:58 smithi081 ceph-mon[36434]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:58.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:58 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:58.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:58 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:58.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:58 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:58.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:58 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:56:58.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:58 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:56:58.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:58 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:58.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:58 smithi081 ceph-mon[36434]: Deploying daemon mds.a on smithi033 2024-05-15T13:56:58.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:58 smithi081 ceph-mon[36434]: mds.? [v2:172.21.15.33:6838/2030117633,v1:172.21.15.33:6839/2030117633] up:boot 2024-05-15T13:56:58.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:58 smithi081 ceph-mon[36434]: fsmap 10 up:standby 2024-05-15T13:56:58.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:56:58 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-15T13:56:58.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:58 smithi033 ceph-mon[31650]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:56:58.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:58 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:58.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:58 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:58.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:58 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:56:58.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:58 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:56:58.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:58 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:56:58.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:58 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:56:58.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:58 smithi033 ceph-mon[31650]: Deploying daemon mds.a on smithi033 2024-05-15T13:56:58.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:58 smithi033 ceph-mon[31650]: mds.? [v2:172.21.15.33:6838/2030117633,v1:172.21.15.33:6839/2030117633] up:boot 2024-05-15T13:56:58.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:58 smithi033 ceph-mon[31650]: fsmap 10 up:standby 2024-05-15T13:56:58.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:56:58 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-15T13:56:58.996 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:56:59.941 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 15 13:56:59 smithi033 systemd[1]: Starting Ceph mds.a for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T13:57:00.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:00 smithi081 ceph-mon[36434]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:57:00.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:00 smithi033 ceph-mon[31650]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:57:00.481 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 15 13:57:00 smithi033 podman[71004]: 2024-05-15 13:57:00.126812357 +0000 UTC m=+0.019055324 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-15T13:57:00.482 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 15 13:57:00 smithi033 podman[71004]: 2024-05-15 13:57:00.227835236 +0000 UTC m=+0.120078224 container create 4e9df59fc51de448ad6e788147a7db999173849dcbacda3601e54e20a8352399 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, ceph=True) 2024-05-15T13:57:00.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:00 smithi052 ceph-mon[36190]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:57:00.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.783+0000 7ff6107c2700 1 -- 172.21.15.33:0/928733428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6081088f0 msgr2=0x7ff608110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:00.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.783+0000 7ff6107c2700 1 --2- 172.21.15.33:0/928733428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6081088f0 0x7ff608110470 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff6000099c0 tx=0x7ff600009cd0 comp rx=0 tx=0).stop 2024-05-15T13:57:00.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.783+0000 7ff6107c2700 1 -- 172.21.15.33:0/928733428 shutdown_connections 2024-05-15T13:57:00.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.783+0000 7ff6107c2700 1 --2- 172.21.15.33:0/928733428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6081088f0 0x7ff608110470 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:00.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.783+0000 7ff6107c2700 1 --2- 172.21.15.33:0/928733428 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6081037f0 0x7ff608103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:00.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.783+0000 7ff6107c2700 1 --2- 172.21.15.33:0/928733428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6081041a0 0x7ff6081081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:00.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.783+0000 7ff6107c2700 1 -- 172.21.15.33:0/928733428 >> 172.21.15.33:0/928733428 conn(0x7ff60807a6a0 msgr2=0x7ff60807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:00.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.783+0000 7ff6107c2700 1 -- 172.21.15.33:0/928733428 shutdown_connections 2024-05-15T13:57:00.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.783+0000 7ff6107c2700 1 -- 172.21.15.33:0/928733428 wait complete. 2024-05-15T13:57:00.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.784+0000 7ff6107c2700 1 Processor -- start 2024-05-15T13:57:00.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.784+0000 7ff6107c2700 1 -- start start 2024-05-15T13:57:00.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.784+0000 7ff6107c2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6081037f0 0x7ff60819bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:00.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.784+0000 7ff6107c2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6081041a0 0x7ff60819c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:00.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.784+0000 7ff6107c2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6081088f0 0x7ff6081a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:00.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.784+0000 7ff6107c2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff608113750 con 0x7ff6081088f0 2024-05-15T13:57:00.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.784+0000 7ff6107c2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff6081135d0 con 0x7ff6081041a0 2024-05-15T13:57:00.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.784+0000 7ff6107c2700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ff6081138d0 con 0x7ff6081037f0 2024-05-15T13:57:00.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.784+0000 7ff60ed5f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6081088f0 0x7ff6081a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:00.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.784+0000 7ff60ed5f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6081088f0 0x7ff6081a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39136/0 (socket says 172.21.15.33:39136) 2024-05-15T13:57:00.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.784+0000 7ff60ed5f700 1 -- 172.21.15.33:0/3405967159 learned_addr learned my addr 172.21.15.33:0/3405967159 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:57:00.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.784+0000 7ff60e55e700 1 --2- 172.21.15.33:0/3405967159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6081037f0 0x7ff60819bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:00.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.785+0000 7ff60dd5d700 1 --2- 172.21.15.33:0/3405967159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6081041a0 0x7ff60819c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:00.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.785+0000 7ff60ed5f700 1 -- 172.21.15.33:0/3405967159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6081037f0 msgr2=0x7ff60819bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:00.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.785+0000 7ff60ed5f700 1 --2- 172.21.15.33:0/3405967159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6081037f0 0x7ff60819bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:00.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.785+0000 7ff60ed5f700 1 -- 172.21.15.33:0/3405967159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6081041a0 msgr2=0x7ff60819c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:00.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.785+0000 7ff60ed5f700 1 --2- 172.21.15.33:0/3405967159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6081041a0 0x7ff60819c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:00.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.785+0000 7ff60ed5f700 1 -- 172.21.15.33:0/3405967159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6081a0d60 con 0x7ff6081088f0 2024-05-15T13:57:00.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.785+0000 7ff60ed5f700 1 --2- 172.21.15.33:0/3405967159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6081088f0 0x7ff6081a0620 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff600005910 tx=0x7ff600009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:00.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.785+0000 7ff6077fe700 1 -- 172.21.15.33:0/3405967159 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff600027070 con 0x7ff6081088f0 2024-05-15T13:57:00.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.785+0000 7ff6107c2700 1 -- 172.21.15.33:0/3405967159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6081a0ff0 con 0x7ff6081088f0 2024-05-15T13:57:00.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.785+0000 7ff6077fe700 1 -- 172.21.15.33:0/3405967159 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff600003f20 con 0x7ff6081088f0 2024-05-15T13:57:00.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.785+0000 7ff6077fe700 1 -- 172.21.15.33:0/3405967159 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff600022c50 con 0x7ff6081088f0 2024-05-15T13:57:00.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.785+0000 7ff6107c2700 1 -- 172.21.15.33:0/3405967159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6081a88d0 con 0x7ff6081088f0 2024-05-15T13:57:00.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.786+0000 7ff6107c2700 1 -- 172.21.15.33:0/3405967159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff60806a230 con 0x7ff6081088f0 2024-05-15T13:57:00.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.788+0000 7ff6077fe700 1 -- 172.21.15.33:0/3405967159 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff60002a070 con 0x7ff6081088f0 2024-05-15T13:57:00.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.788+0000 7ff6077fe700 1 --2- 172.21.15.33:0/3405967159 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff5f00b2880 0x7ff5f00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:00.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.788+0000 7ff60e55e700 1 --2- 172.21.15.33:0/3405967159 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff5f00b2880 0x7ff5f00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:00.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.788+0000 7ff6077fe700 1 -- 172.21.15.33:0/3405967159 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7ff6000e2bd0 con 0x7ff6081088f0 2024-05-15T13:57:00.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.789+0000 7ff60e55e700 1 --2- 172.21.15.33:0/3405967159 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff5f00b2880 0x7ff5f00b4d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8003f90 tx=0x7ff5f8009180 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:00.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:00.792+0000 7ff6077fe700 1 -- 172.21.15.33:0/3405967159 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff600013070 con 0x7ff6081088f0 2024-05-15T13:57:01.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:01.067+0000 7ff6107c2700 1 -- 172.21.15.33:0/3405967159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6081a8aa0 con 0x7ff6081088f0 2024-05-15T13:57:01.069 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 15 13:57:00 smithi033 podman[71004]: 2024-05-15 13:57:00.911750543 +0000 UTC m=+0.803993514 container init 4e9df59fc51de448ad6e788147a7db999173849dcbacda3601e54e20a8352399 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux ) 2024-05-15T13:57:01.069 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 15 13:57:00 smithi033 podman[71004]: 2024-05-15 13:57:00.91549565 +0000 UTC m=+0.807738613 container start 4e9df59fc51de448ad6e788147a7db999173849dcbacda3601e54e20a8352399 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-a, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-15T13:57:01.069 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 15 13:57:00 smithi033 ceph-mds[71068]: -- 172.21.15.33:0/598435345 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55723db4da40 con 0x55723db21000 2024-05-15T13:57:01.069 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 15 13:57:00 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-a[71037]: starting mds.a at 2024-05-15T13:57:01.069 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 15 13:57:01 smithi033 bash[71004]: 4e9df59fc51de448ad6e788147a7db999173849dcbacda3601e54e20a8352399 2024-05-15T13:57:01.069 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 15 13:57:01 smithi033 systemd[1]: Started Ceph mds.a for bca78ca4-12c1-11ef-bc98-c7b262605968. 2024-05-15T13:57:01.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:01.068+0000 7ff6077fe700 1 -- 172.21.15.33:0/3405967159 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8818 (secure 0 0 0) 0x7ff6000aae20 con 0x7ff6081088f0 2024-05-15T13:57:01.070 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:57:01.070 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2030117633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2030117633},{"type":"v1","addr":"172.21.15.33:6839","nonce":2030117633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/1629697841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1629697841},{"type":"v1","addr":"172.21.15.52:6837","nonce":1629697841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/3773839758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3773839758},{"type":"v1","addr":"172.21.15.81:6835","nonce":3773839758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6839/1554395940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":1554395940},{"type":"v1","addr":"172.21.15.81:6839","nonce":1554395940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-15T13:57:01.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:01.071+0000 7ff6107c2700 1 -- 172.21.15.33:0/3405967159 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff5f00b2880 msgr2=0x7ff5f00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:01.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:01.071+0000 7ff6107c2700 1 --2- 172.21.15.33:0/3405967159 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff5f00b2880 0x7ff5f00b4d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8003f90 tx=0x7ff5f8009180 comp rx=0 tx=0).stop 2024-05-15T13:57:01.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:01.071+0000 7ff6107c2700 1 -- 172.21.15.33:0/3405967159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6081088f0 msgr2=0x7ff6081a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:01.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:01.071+0000 7ff6107c2700 1 --2- 172.21.15.33:0/3405967159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6081088f0 0x7ff6081a0620 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff600005910 tx=0x7ff600009580 comp rx=0 tx=0).stop 2024-05-15T13:57:01.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:01.071+0000 7ff6107c2700 1 -- 172.21.15.33:0/3405967159 shutdown_connections 2024-05-15T13:57:01.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:01.071+0000 7ff6107c2700 1 --2- 172.21.15.33:0/3405967159 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff5f00b2880 0x7ff5f00b4d40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:01.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:01.071+0000 7ff6107c2700 1 --2- 172.21.15.33:0/3405967159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6081088f0 0x7ff6081a0620 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:01.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:01.071+0000 7ff6107c2700 1 --2- 172.21.15.33:0/3405967159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6081037f0 0x7ff60819bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:01.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:01.071+0000 7ff6107c2700 1 --2- 172.21.15.33:0/3405967159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff6081041a0 0x7ff60819c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:01.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:01.071+0000 7ff6107c2700 1 -- 172.21.15.33:0/3405967159 >> 172.21.15.33:0/3405967159 conn(0x7ff60807a6a0 msgr2=0x7ff6081018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:01.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:01.071+0000 7ff6107c2700 1 -- 172.21.15.33:0/3405967159 shutdown_connections 2024-05-15T13:57:01.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:01.071+0000 7ff6107c2700 1 -- 172.21.15.33:0/3405967159 wait complete. 2024-05-15T13:57:01.074 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 11 2024-05-15T13:57:01.391 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:01 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/3405967159' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:57:01.391 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:01 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:01.392 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:01 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:01.392 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:01 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:01.392 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:01 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:57:01.392 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:01 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:57:01.392 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:01 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:57:01.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:01 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/3405967159' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:57:01.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:01 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:01.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:01 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:01.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:01 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:01.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:01 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:57:01.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:01 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:57:01.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:01 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:57:01.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:01 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/3405967159' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:57:01.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:01 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:01.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:01 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:01.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:01 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:01.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:01 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-15T13:57:01.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:01 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-15T13:57:01.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:01 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:57:02.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:02 smithi052 ceph-mon[36190]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:57:02.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:02 smithi052 ceph-mon[36190]: Deploying daemon mds.b on smithi052 2024-05-15T13:57:02.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:02 smithi052 ceph-mon[36190]: mds.? [v2:172.21.15.33:6840/1918979061,v1:172.21.15.33:6841/1918979061] up:boot 2024-05-15T13:57:02.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:02 smithi052 ceph-mon[36190]: fsmap 11 up:standby 2024-05-15T13:57:02.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:02 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-15T13:57:02.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:02 smithi081 ceph-mon[36434]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:57:02.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:02 smithi081 ceph-mon[36434]: Deploying daemon mds.b on smithi052 2024-05-15T13:57:02.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:02 smithi081 ceph-mon[36434]: mds.? [v2:172.21.15.33:6840/1918979061,v1:172.21.15.33:6841/1918979061] up:boot 2024-05-15T13:57:02.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:02 smithi081 ceph-mon[36434]: fsmap 11 up:standby 2024-05-15T13:57:02.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:02 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-15T13:57:02.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:02 smithi033 ceph-mon[31650]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:57:02.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:02 smithi033 ceph-mon[31650]: Deploying daemon mds.b on smithi052 2024-05-15T13:57:02.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:02 smithi033 ceph-mon[31650]: mds.? [v2:172.21.15.33:6840/1918979061,v1:172.21.15.33:6841/1918979061] up:boot 2024-05-15T13:57:02.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:02 smithi033 ceph-mon[31650]: fsmap 11 up:standby 2024-05-15T13:57:02.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:02 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-15T13:57:03.551 INFO:journalctl@ceph.mds.b.smithi052.stdout:May 15 13:57:03 smithi052 systemd[1]: Starting Ceph mds.b for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T13:57:03.728 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T13:57:03.841 INFO:journalctl@ceph.mds.b.smithi052.stdout:May 15 13:57:03 smithi052 podman[66905]: 2024-05-15 13:57:03.45269784 +0000 UTC m=+0.018145433 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-15T13:57:03.841 INFO:journalctl@ceph.mds.b.smithi052.stdout:May 15 13:57:03 smithi052 podman[66905]: 2024-05-15 13:57:03.552204823 +0000 UTC m=+0.117652435 container create da7651bf28afbd8f163b29ece52c05c4138c839ed06eff5155936c3aafd95232 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-b, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-15T13:57:03.841 INFO:journalctl@ceph.mds.b.smithi052.stdout:May 15 13:57:03 smithi052 podman[66905]: 2024-05-15 13:57:03.762943846 +0000 UTC m=+0.328391427 container init da7651bf28afbd8f163b29ece52c05c4138c839ed06eff5155936c3aafd95232 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T13:57:03.841 INFO:journalctl@ceph.mds.b.smithi052.stdout:May 15 13:57:03 smithi052 podman[66905]: 2024-05-15 13:57:03.765661881 +0000 UTC m=+0.331109479 container start da7651bf28afbd8f163b29ece52c05c4138c839ed06eff5155936c3aafd95232 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-b, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-15T13:57:03.841 INFO:journalctl@ceph.mds.b.smithi052.stdout:May 15 13:57:03 smithi052 ceph-mds[66924]: -- 172.21.15.52:0/1718947807 <== mon.2 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c45483ba40 con 0x55c45480ec00 2024-05-15T13:57:03.841 INFO:journalctl@ceph.mds.b.smithi052.stdout:May 15 13:57:03 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-b[66920]: starting mds.b at 2024-05-15T13:57:03.998 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:57:04.249 INFO:journalctl@ceph.mds.b.smithi052.stdout:May 15 13:57:03 smithi052 bash[66905]: da7651bf28afbd8f163b29ece52c05c4138c839ed06eff5155936c3aafd95232 2024-05-15T13:57:04.249 INFO:journalctl@ceph.mds.b.smithi052.stdout:May 15 13:57:03 smithi052 systemd[1]: Started Ceph mds.b for bca78ca4-12c1-11ef-bc98-c7b262605968. 2024-05-15T13:57:04.377 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:04 smithi033 ceph-mon[31650]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:57:04.377 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:04 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:04.377 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:04 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:04.377 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:04 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:04.377 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:04 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:04.377 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:04 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:04.377 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:04 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:57:04.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:04 smithi052 ceph-mon[36190]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:57:04.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:04 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:04.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:04 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:04.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:04 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:04.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:04 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:04.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:04 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:04.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:04 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:57:04.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:04 smithi081 ceph-mon[36434]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:57:04.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:04 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:04.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:04 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:04.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:04 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:04.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:04 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:04.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:04 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:04.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:04 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:57:05.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:05 smithi081 ceph-mon[36434]: mds.? [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] up:boot 2024-05-15T13:57:05.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:05 smithi081 ceph-mon[36434]: fsmap 12 up:standby 2024-05-15T13:57:05.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:05 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-15T13:57:05.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:05 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:05.397 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:05 smithi033 ceph-mon[31650]: mds.? [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] up:boot 2024-05-15T13:57:05.398 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:05 smithi033 ceph-mon[31650]: fsmap 12 up:standby 2024-05-15T13:57:05.398 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:05 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-15T13:57:05.398 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:05 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:05.505 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:05 smithi052 ceph-mon[36190]: mds.? [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] up:boot 2024-05-15T13:57:05.505 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:05 smithi052 ceph-mon[36190]: fsmap 12 up:standby 2024-05-15T13:57:05.505 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:05 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-15T13:57:05.505 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:05 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:05.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.655+0000 7fce529cb700 1 -- 172.21.15.33:0/2193479800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce4c1037f0 msgr2=0x7fce4c103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:05.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.655+0000 7fce529cb700 1 --2- 172.21.15.33:0/2193479800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce4c1037f0 0x7fce4c103bd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0099c0 tx=0x7fce3c009cd0 comp rx=0 tx=0).stop 2024-05-15T13:57:05.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.655+0000 7fce529cb700 1 -- 172.21.15.33:0/2193479800 shutdown_connections 2024-05-15T13:57:05.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.655+0000 7fce529cb700 1 --2- 172.21.15.33:0/2193479800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce4c1088f0 0x7fce4c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:05.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.655+0000 7fce529cb700 1 --2- 172.21.15.33:0/2193479800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce4c1037f0 0x7fce4c103bd0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:05.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.656+0000 7fce529cb700 1 --2- 172.21.15.33:0/2193479800 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fce4c1041a0 0x7fce4c1081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:05.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.656+0000 7fce529cb700 1 -- 172.21.15.33:0/2193479800 >> 172.21.15.33:0/2193479800 conn(0x7fce4c07a6a0 msgr2=0x7fce4c07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:05.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.656+0000 7fce529cb700 1 -- 172.21.15.33:0/2193479800 shutdown_connections 2024-05-15T13:57:05.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.656+0000 7fce529cb700 1 -- 172.21.15.33:0/2193479800 wait complete. 2024-05-15T13:57:05.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.656+0000 7fce529cb700 1 Processor -- start 2024-05-15T13:57:05.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.656+0000 7fce529cb700 1 -- start start 2024-05-15T13:57:05.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.657+0000 7fce529cb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fce4c1037f0 0x7fce4c075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:05.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.657+0000 7fce529cb700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce4c1041a0 0x7fce4c075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:05.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.657+0000 7fce529cb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce4c1088f0 0x7fce4c0762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:05.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.657+0000 7fce529cb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fce4c1136c0 con 0x7fce4c1088f0 2024-05-15T13:57:05.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.657+0000 7fce529cb700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fce4c113540 con 0x7fce4c1037f0 2024-05-15T13:57:05.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.657+0000 7fce529cb700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fce4c113840 con 0x7fce4c1041a0 2024-05-15T13:57:05.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.657+0000 7fce4bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fce4c1037f0 0x7fce4c075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:05.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.657+0000 7fce43fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce4c1041a0 0x7fce4c075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:05.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.657+0000 7fce50f68700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce4c1088f0 0x7fce4c0762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:05.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.657+0000 7fce4bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fce4c1037f0 0x7fce4c075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:51618/0 (socket says 172.21.15.33:51618) 2024-05-15T13:57:05.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.657+0000 7fce4bfff700 1 -- 172.21.15.33:0/1445722954 learned_addr learned my addr 172.21.15.33:0/1445722954 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:57:05.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.657+0000 7fce43fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce4c1041a0 0x7fce4c075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:57238/0 (socket says 172.21.15.33:57238) 2024-05-15T13:57:05.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.657+0000 7fce43fff700 1 -- 172.21.15.33:0/1445722954 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fce4c1037f0 msgr2=0x7fce4c075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:05.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.657+0000 7fce43fff700 1 --2- 172.21.15.33:0/1445722954 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fce4c1037f0 0x7fce4c075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:05.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.657+0000 7fce43fff700 1 -- 172.21.15.33:0/1445722954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce4c1088f0 msgr2=0x7fce4c0762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:05.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.657+0000 7fce43fff700 1 --2- 172.21.15.33:0/1445722954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce4c1088f0 0x7fce4c0762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:05.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.657+0000 7fce43fff700 1 -- 172.21.15.33:0/1445722954 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce4c1a8100 con 0x7fce4c1041a0 2024-05-15T13:57:05.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.657+0000 7fce50f68700 1 --2- 172.21.15.33:0/1445722954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce4c1088f0 0x7fce4c0762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T13:57:05.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.657+0000 7fce43fff700 1 --2- 172.21.15.33:0/1445722954 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce4c1041a0 0x7fce4c075d80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fce3400d750 tx=0x7fce3400da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:05.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.657+0000 7fce49ffb700 1 -- 172.21.15.33:0/1445722954 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce34014030 con 0x7fce4c1041a0 2024-05-15T13:57:05.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.658+0000 7fce529cb700 1 -- 172.21.15.33:0/1445722954 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce4c1a83f0 con 0x7fce4c1041a0 2024-05-15T13:57:05.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.658+0000 7fce529cb700 1 -- 172.21.15.33:0/1445722954 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce4c1a8980 con 0x7fce4c1041a0 2024-05-15T13:57:05.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.658+0000 7fce49ffb700 1 -- 172.21.15.33:0/1445722954 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce340044d0 con 0x7fce4c1041a0 2024-05-15T13:57:05.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.658+0000 7fce49ffb700 1 -- 172.21.15.33:0/1445722954 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce34005420 con 0x7fce4c1041a0 2024-05-15T13:57:05.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.658+0000 7fce529cb700 1 -- 172.21.15.33:0/1445722954 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce2c005330 con 0x7fce4c1041a0 2024-05-15T13:57:05.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.660+0000 7fce49ffb700 1 -- 172.21.15.33:0/1445722954 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fce34004020 con 0x7fce4c1041a0 2024-05-15T13:57:05.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.661+0000 7fce49ffb700 1 --2- 172.21.15.33:0/1445722954 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fce380b2880 0x7fce380b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:05.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.661+0000 7fce4bfff700 1 --2- 172.21.15.33:0/1445722954 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fce380b2880 0x7fce380b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:05.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.661+0000 7fce49ffb700 1 -- 172.21.15.33:0/1445722954 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fce340e1a50 con 0x7fce4c1041a0 2024-05-15T13:57:05.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.661+0000 7fce4bfff700 1 --2- 172.21.15.33:0/1445722954 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fce380b2880 0x7fce380b4d40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fce3c009990 tx=0x7fce3c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:05.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.666+0000 7fce49ffb700 1 -- 172.21.15.33:0/1445722954 <== mon.1 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fce340a9b70 con 0x7fce4c1041a0 2024-05-15T13:57:05.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.931+0000 7fce529cb700 1 -- 172.21.15.33:0/1445722954 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce2c005620 con 0x7fce4c1041a0 2024-05-15T13:57:05.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.932+0000 7fce49ffb700 1 -- 172.21.15.33:0/1445722954 <== mon.1 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10470 (secure 0 0 0) 0x7fce340a9470 con 0x7fce4c1041a0 2024-05-15T13:57:05.934 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:57:05.934 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2030117633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2030117633},{"type":"v1","addr":"172.21.15.33:6839","nonce":2030117633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1918979061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1918979061},{"type":"v1","addr":"172.21.15.33:6841","nonce":1918979061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/1629697841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1629697841},{"type":"v1","addr":"172.21.15.52:6837","nonce":1629697841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/3773839758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3773839758},{"type":"v1","addr":"172.21.15.81:6835","nonce":3773839758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2696044471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2696044471},{"type":"v1","addr":"172.21.15.52:6839","nonce":2696044471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6839/1554395940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":1554395940},{"type":"v1","addr":"172.21.15.81:6839","nonce":1554395940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-15T13:57:05.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.934+0000 7fce529cb700 1 -- 172.21.15.33:0/1445722954 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fce380b2880 msgr2=0x7fce380b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:05.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.934+0000 7fce529cb700 1 --2- 172.21.15.33:0/1445722954 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fce380b2880 0x7fce380b4d40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fce3c009990 tx=0x7fce3c005960 comp rx=0 tx=0).stop 2024-05-15T13:57:05.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.934+0000 7fce529cb700 1 -- 172.21.15.33:0/1445722954 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce4c1041a0 msgr2=0x7fce4c075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:05.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.934+0000 7fce529cb700 1 --2- 172.21.15.33:0/1445722954 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce4c1041a0 0x7fce4c075d80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fce3400d750 tx=0x7fce3400da60 comp rx=0 tx=0).stop 2024-05-15T13:57:05.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.934+0000 7fce529cb700 1 -- 172.21.15.33:0/1445722954 shutdown_connections 2024-05-15T13:57:05.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.934+0000 7fce529cb700 1 --2- 172.21.15.33:0/1445722954 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fce380b2880 0x7fce380b4d40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:05.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.935+0000 7fce529cb700 1 --2- 172.21.15.33:0/1445722954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce4c1088f0 0x7fce4c0762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:05.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.935+0000 7fce529cb700 1 --2- 172.21.15.33:0/1445722954 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fce4c1037f0 0x7fce4c075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:05.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.935+0000 7fce529cb700 1 --2- 172.21.15.33:0/1445722954 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce4c1041a0 0x7fce4c075d80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:05.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.935+0000 7fce529cb700 1 -- 172.21.15.33:0/1445722954 >> 172.21.15.33:0/1445722954 conn(0x7fce4c07a6a0 msgr2=0x7fce4c109a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:05.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.935+0000 7fce529cb700 1 -- 172.21.15.33:0/1445722954 shutdown_connections 2024-05-15T13:57:05.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:05.935+0000 7fce529cb700 1 -- 172.21.15.33:0/1445722954 wait complete. 2024-05-15T13:57:05.937 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 13 2024-05-15T13:57:06.522 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:06 smithi052 ceph-mon[36190]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:57:06.522 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:06 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:06.522 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:06 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:06.522 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:06 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:06.522 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:06 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:06.522 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:06 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/1445722954' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:57:06.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:06 smithi081 ceph-mon[36434]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:57:06.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:06 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:06.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:06 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:06.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:06 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:06.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:06 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:06.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:06 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/1445722954' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:57:06.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:06 smithi033 ceph-mon[31650]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:57:06.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:06 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:06.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:06 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:06.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:06 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:06.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:06 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:06.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:06 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/1445722954' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:57:06.655 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-15T13:57:07.097 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:57:08.461 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:08 smithi033 ceph-mon[31650]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:57:08.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:08 smithi052 ceph-mon[36190]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:57:08.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:08 smithi081 ceph-mon[36434]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:57:08.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.641+0000 7f293bae3700 1 -- 172.21.15.33:0/3651144975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29341035c0 msgr2=0x7f2934103a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:08.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.641+0000 7f293bae3700 1 --2- 172.21.15.33:0/3651144975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29341035c0 0x7f2934103a40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2928009990 tx=0x7f2928009ca0 comp rx=0 tx=0).stop 2024-05-15T13:57:08.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.641+0000 7f293bae3700 1 -- 172.21.15.33:0/3651144975 shutdown_connections 2024-05-15T13:57:08.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.641+0000 7f293bae3700 1 --2- 172.21.15.33:0/3651144975 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2934103f80 0x7f2934110410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:08.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.641+0000 7f293bae3700 1 --2- 172.21.15.33:0/3651144975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29341095d0 0x7f29341099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:08.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.641+0000 7f293bae3700 1 --2- 172.21.15.33:0/3651144975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29341035c0 0x7f2934103a40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:08.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.641+0000 7f293bae3700 1 -- 172.21.15.33:0/3651144975 >> 172.21.15.33:0/3651144975 conn(0x7f293407a6f0 msgr2=0x7f293407ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:08.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.641+0000 7f293bae3700 1 -- 172.21.15.33:0/3651144975 shutdown_connections 2024-05-15T13:57:08.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.641+0000 7f293bae3700 1 -- 172.21.15.33:0/3651144975 wait complete. 2024-05-15T13:57:08.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.642+0000 7f293bae3700 1 Processor -- start 2024-05-15T13:57:08.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.642+0000 7f293bae3700 1 -- start start 2024-05-15T13:57:08.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.642+0000 7f293bae3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29341035c0 0x7f2934075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:08.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.642+0000 7f293bae3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2934103f80 0x7f2934075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:08.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.642+0000 7f293bae3700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f29341095d0 0x7f29340762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:08.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.642+0000 7f293bae3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2934113780 con 0x7f29341035c0 2024-05-15T13:57:08.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.642+0000 7f293bae3700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2934113600 con 0x7f2934103f80 2024-05-15T13:57:08.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.642+0000 7f293bae3700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2934113900 con 0x7f29341095d0 2024-05-15T13:57:08.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.642+0000 7f293987f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29341035c0 0x7f2934075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:08.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.642+0000 7f293987f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29341035c0 0x7f2934075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59838/0 (socket says 172.21.15.33:59838) 2024-05-15T13:57:08.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.642+0000 7f293987f700 1 -- 172.21.15.33:0/2466143750 learned_addr learned my addr 172.21.15.33:0/2466143750 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:57:08.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.642+0000 7f293907e700 1 --2- 172.21.15.33:0/2466143750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2934103f80 0x7f2934075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:08.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.642+0000 7f293a080700 1 --2- 172.21.15.33:0/2466143750 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f29341095d0 0x7f29340762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:08.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.643+0000 7f293987f700 1 -- 172.21.15.33:0/2466143750 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f29341095d0 msgr2=0x7f29340762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:08.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.643+0000 7f293987f700 1 --2- 172.21.15.33:0/2466143750 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f29341095d0 0x7f29340762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:08.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.643+0000 7f293987f700 1 -- 172.21.15.33:0/2466143750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2934103f80 msgr2=0x7f2934075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:08.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.643+0000 7f293987f700 1 --2- 172.21.15.33:0/2466143750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2934103f80 0x7f2934075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:08.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.643+0000 7f293987f700 1 -- 172.21.15.33:0/2466143750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29341a80e0 con 0x7f29341035c0 2024-05-15T13:57:08.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.643+0000 7f293987f700 1 --2- 172.21.15.33:0/2466143750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29341035c0 0x7f2934075840 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f292400e9a0 tx=0x7f292400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:08.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.643+0000 7f2932ffd700 1 -- 172.21.15.33:0/2466143750 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292400ca10 con 0x7f29341035c0 2024-05-15T13:57:08.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.643+0000 7f293bae3700 1 -- 172.21.15.33:0/2466143750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29341a83d0 con 0x7f29341035c0 2024-05-15T13:57:08.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.643+0000 7f2932ffd700 1 -- 172.21.15.33:0/2466143750 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f292400cbb0 con 0x7f29341035c0 2024-05-15T13:57:08.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.643+0000 7f2932ffd700 1 -- 172.21.15.33:0/2466143750 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2924018680 con 0x7f29341035c0 2024-05-15T13:57:08.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.643+0000 7f293bae3700 1 -- 172.21.15.33:0/2466143750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29341a8930 con 0x7f29341035c0 2024-05-15T13:57:08.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.644+0000 7f293bae3700 1 -- 172.21.15.33:0/2466143750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f293406a230 con 0x7f29341035c0 2024-05-15T13:57:08.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.647+0000 7f2932ffd700 1 -- 172.21.15.33:0/2466143750 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f2924010430 con 0x7f29341035c0 2024-05-15T13:57:08.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.648+0000 7f2932ffd700 1 --2- 172.21.15.33:0/2466143750 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f291c0b2880 0x7f291c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:08.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.648+0000 7f2932ffd700 1 -- 172.21.15.33:0/2466143750 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f29240e21e0 con 0x7f29341035c0 2024-05-15T13:57:08.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.648+0000 7f293907e700 1 --2- 172.21.15.33:0/2466143750 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f291c0b2880 0x7f291c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:08.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.649+0000 7f293907e700 1 --2- 172.21.15.33:0/2466143750 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f291c0b2880 0x7f291c0b4d40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f292800b2e0 tx=0x7f2928005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:08.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.651+0000 7f2932ffd700 1 -- 172.21.15.33:0/2466143750 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f29240aa3a0 con 0x7f29341035c0 2024-05-15T13:57:08.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.891+0000 7f293bae3700 1 -- 172.21.15.33:0/2466143750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2934076da0 con 0x7f29341035c0 2024-05-15T13:57:08.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.891+0000 7f2932ffd700 1 -- 172.21.15.33:0/2466143750 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v73) ==== 84+0+65 (secure 0 0 0) 0x7f2934076da0 con 0x7f29341035c0 2024-05-15T13:57:08.893 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:57:08.893 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2024-05-15T13:57:08.893 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T13:57:08.893 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2024-05-15T13:57:08.893 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": ".mgr" 2024-05-15T13:57:08.894 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T13:57:08.894 INFO:teuthology.orchestra.run.smithi033.stdout:] 2024-05-15T13:57:08.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.894+0000 7f293bae3700 1 -- 172.21.15.33:0/2466143750 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f291c0b2880 msgr2=0x7f291c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:08.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.894+0000 7f293bae3700 1 --2- 172.21.15.33:0/2466143750 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f291c0b2880 0x7f291c0b4d40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f292800b2e0 tx=0x7f2928005960 comp rx=0 tx=0).stop 2024-05-15T13:57:08.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.894+0000 7f293bae3700 1 -- 172.21.15.33:0/2466143750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29341035c0 msgr2=0x7f2934075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:08.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.894+0000 7f293bae3700 1 --2- 172.21.15.33:0/2466143750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29341035c0 0x7f2934075840 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f292400e9a0 tx=0x7f292400ecb0 comp rx=0 tx=0).stop 2024-05-15T13:57:08.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.894+0000 7f293bae3700 1 -- 172.21.15.33:0/2466143750 shutdown_connections 2024-05-15T13:57:08.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.894+0000 7f293bae3700 1 --2- 172.21.15.33:0/2466143750 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f291c0b2880 0x7f291c0b4d40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:08.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.894+0000 7f293bae3700 1 --2- 172.21.15.33:0/2466143750 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f29341095d0 0x7f29340762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:08.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.894+0000 7f293bae3700 1 --2- 172.21.15.33:0/2466143750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29341035c0 0x7f2934075840 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:08.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.894+0000 7f293bae3700 1 --2- 172.21.15.33:0/2466143750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2934103f80 0x7f2934075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:08.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.894+0000 7f293bae3700 1 -- 172.21.15.33:0/2466143750 >> 172.21.15.33:0/2466143750 conn(0x7f293407a6f0 msgr2=0x7f2934102350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:08.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.894+0000 7f293bae3700 1 -- 172.21.15.33:0/2466143750 shutdown_connections 2024-05-15T13:57:08.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:08.894+0000 7f293bae3700 1 -- 172.21.15.33:0/2466143750 wait complete. 2024-05-15T13:57:09.572 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-15T13:57:09.573 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-15T13:57:10.062 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:09 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:10.062 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:09 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:10.062 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:09 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:57:10.062 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:09 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:57:10.062 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:09 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:10.062 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:09 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/2466143750' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-15T13:57:10.063 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:09 smithi052 ceph-mon[36190]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:57:10.063 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:09 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:57:10.078 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:09 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:10.078 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:09 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:10.078 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:09 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:57:10.078 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:09 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:57:10.078 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:09 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:10.078 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:09 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/2466143750' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-15T13:57:10.078 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:09 smithi081 ceph-mon[36434]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:57:10.078 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:09 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:57:10.086 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:09 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:10.086 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:09 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:10.087 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:09 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:57:10.087 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:09 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:57:10.087 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:09 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:10.087 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:09 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/2466143750' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-15T13:57:10.087 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:09 smithi033 ceph-mon[31650]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:57:10.087 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:09 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:57:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:57:11.857 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:11 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:11.857 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:11 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:11.857 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:11 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:11.857 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:11 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:11.858 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:11 smithi033 ceph-mon[31650]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:57:12.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.086+0000 7f1e917c6700 1 -- 172.21.15.33:0/3427645855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e8c101f00 msgr2=0x7f1e8c102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:12.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.086+0000 7f1e917c6700 1 --2- 172.21.15.33:0/3427645855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e8c101f00 0x7f1e8c102380 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1e80009990 tx=0x7f1e80009ca0 comp rx=0 tx=0).stop 2024-05-15T13:57:12.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.086+0000 7f1e917c6700 1 -- 172.21.15.33:0/3427645855 shutdown_connections 2024-05-15T13:57:12.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.086+0000 7f1e917c6700 1 --2- 172.21.15.33:0/3427645855 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1e8c1028c0 0x7f1e8c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:12.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.086+0000 7f1e917c6700 1 --2- 172.21.15.33:0/3427645855 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1e8c108620 0x7f1e8c108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:12.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.086+0000 7f1e917c6700 1 --2- 172.21.15.33:0/3427645855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e8c101f00 0x7f1e8c102380 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:12.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.086+0000 7f1e917c6700 1 -- 172.21.15.33:0/3427645855 >> 172.21.15.33:0/3427645855 conn(0x7f1e8c077780 msgr2=0x7f1e8c077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:12.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.086+0000 7f1e917c6700 1 -- 172.21.15.33:0/3427645855 shutdown_connections 2024-05-15T13:57:12.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.086+0000 7f1e917c6700 1 -- 172.21.15.33:0/3427645855 wait complete. 2024-05-15T13:57:12.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.086+0000 7f1e917c6700 1 Processor -- start 2024-05-15T13:57:12.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.087+0000 7f1e917c6700 1 -- start start 2024-05-15T13:57:12.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.087+0000 7f1e917c6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1e8c101f00 0x7f1e8c1030e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:12.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.087+0000 7f1e917c6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1e8c1028c0 0x7f1e8c103620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:12.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.087+0000 7f1e917c6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e8c108620 0x7f1e8c103b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:12.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.087+0000 7f1e917c6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c113760 con 0x7f1e8c108620 2024-05-15T13:57:12.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.087+0000 7f1e917c6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c1135e0 con 0x7f1e8c1028c0 2024-05-15T13:57:12.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.087+0000 7f1e917c6700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8c1138e0 con 0x7f1e8c101f00 2024-05-15T13:57:12.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.087+0000 7f1e8b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e8c108620 0x7f1e8c103b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:12.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.087+0000 7f1e8b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e8c108620 0x7f1e8c103b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59868/0 (socket says 172.21.15.33:59868) 2024-05-15T13:57:12.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.087+0000 7f1e8b7fe700 1 -- 172.21.15.33:0/571772022 learned_addr learned my addr 172.21.15.33:0/571772022 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:57:12.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.087+0000 7f1e8affd700 1 --2- 172.21.15.33:0/571772022 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1e8c101f00 0x7f1e8c1030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:12.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.087+0000 7f1e8a7fc700 1 --2- 172.21.15.33:0/571772022 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1e8c1028c0 0x7f1e8c103620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:12.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.088+0000 7f1e8b7fe700 1 -- 172.21.15.33:0/571772022 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1e8c101f00 msgr2=0x7f1e8c1030e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:12.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.088+0000 7f1e8b7fe700 1 --2- 172.21.15.33:0/571772022 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1e8c101f00 0x7f1e8c1030e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:12.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.088+0000 7f1e8b7fe700 1 -- 172.21.15.33:0/571772022 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1e8c1028c0 msgr2=0x7f1e8c103620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:12.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.088+0000 7f1e8b7fe700 1 --2- 172.21.15.33:0/571772022 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1e8c1028c0 0x7f1e8c103620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:12.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.088+0000 7f1e8b7fe700 1 -- 172.21.15.33:0/571772022 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e8c1a81d0 con 0x7f1e8c108620 2024-05-15T13:57:12.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.088+0000 7f1e8b7fe700 1 --2- 172.21.15.33:0/571772022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e8c108620 0x7f1e8c103b60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1e84007ee0 tx=0x7f1e8400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:12.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.088+0000 7f1e73fff700 1 -- 172.21.15.33:0/571772022 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e84004bb0 con 0x7f1e8c108620 2024-05-15T13:57:12.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.088+0000 7f1e917c6700 1 -- 172.21.15.33:0/571772022 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e8c1a84c0 con 0x7f1e8c108620 2024-05-15T13:57:12.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.088+0000 7f1e73fff700 1 -- 172.21.15.33:0/571772022 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e84004d50 con 0x7f1e8c108620 2024-05-15T13:57:12.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.088+0000 7f1e73fff700 1 -- 172.21.15.33:0/571772022 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e84005920 con 0x7f1e8c108620 2024-05-15T13:57:12.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.088+0000 7f1e917c6700 1 -- 172.21.15.33:0/571772022 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e8c1a8980 con 0x7f1e8c108620 2024-05-15T13:57:12.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:11 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:12.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:11 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:12.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:11 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:12.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:11 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:12.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:11 smithi052 ceph-mon[36190]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:57:12.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.089+0000 7f1e917c6700 1 -- 172.21.15.33:0/571772022 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e8c06a230 con 0x7f1e8c108620 2024-05-15T13:57:12.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.090+0000 7f1e73fff700 1 -- 172.21.15.33:0/571772022 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f1e8401a020 con 0x7f1e8c108620 2024-05-15T13:57:12.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.091+0000 7f1e73fff700 1 --2- 172.21.15.33:0/571772022 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f1e740b2880 0x7f1e740b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:12.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.091+0000 7f1e8affd700 1 --2- 172.21.15.33:0/571772022 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f1e740b2880 0x7f1e740b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:12.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.092+0000 7f1e73fff700 1 -- 172.21.15.33:0/571772022 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f1e840e1420 con 0x7f1e8c108620 2024-05-15T13:57:12.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.092+0000 7f1e8affd700 1 --2- 172.21.15.33:0/571772022 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f1e740b2880 0x7f1e740b4d40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c078ce0 tx=0x7f1e7c007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:12.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.097+0000 7f1e73fff700 1 -- 172.21.15.33:0/571772022 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1e84010070 con 0x7f1e8c108620 2024-05-15T13:57:12.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:11 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:12.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:11 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:12.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:11 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:12.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:11 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:12.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:11 smithi081 ceph-mon[36434]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:57:12.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:12.320+0000 7f1e917c6700 1 -- 172.21.15.33:0/571772022 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f1e8c02e560 con 0x7f1e8c108620 2024-05-15T13:57:13.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:13.238+0000 7f1e73fff700 1 -- 172.21.15.33:0/571772022 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v74) ==== 138+0+0 (secure 0 0 0) 0x7f1e840a95e0 con 0x7f1e8c108620 2024-05-15T13:57:13.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:13.240+0000 7f1e917c6700 1 -- 172.21.15.33:0/571772022 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f1e740b2880 msgr2=0x7f1e740b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:13.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:13.240+0000 7f1e917c6700 1 --2- 172.21.15.33:0/571772022 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f1e740b2880 0x7f1e740b4d40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c078ce0 tx=0x7f1e7c007400 comp rx=0 tx=0).stop 2024-05-15T13:57:13.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:13.240+0000 7f1e917c6700 1 -- 172.21.15.33:0/571772022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e8c108620 msgr2=0x7f1e8c103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:13.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:13.240+0000 7f1e917c6700 1 --2- 172.21.15.33:0/571772022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e8c108620 0x7f1e8c103b60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1e84007ee0 tx=0x7f1e8400d3b0 comp rx=0 tx=0).stop 2024-05-15T13:57:13.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:13.240+0000 7f1e917c6700 1 -- 172.21.15.33:0/571772022 shutdown_connections 2024-05-15T13:57:13.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:13.240+0000 7f1e917c6700 1 --2- 172.21.15.33:0/571772022 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f1e740b2880 0x7f1e740b4d40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:13.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:13.240+0000 7f1e917c6700 1 --2- 172.21.15.33:0/571772022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e8c108620 0x7f1e8c103b60 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:13.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:13.240+0000 7f1e917c6700 1 --2- 172.21.15.33:0/571772022 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1e8c101f00 0x7f1e8c1030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:13.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:13.240+0000 7f1e917c6700 1 --2- 172.21.15.33:0/571772022 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1e8c1028c0 0x7f1e8c103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:13.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:13.240+0000 7f1e917c6700 1 -- 172.21.15.33:0/571772022 >> 172.21.15.33:0/571772022 conn(0x7f1e8c077780 msgr2=0x7f1e8c100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:13.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:13.241+0000 7f1e917c6700 1 -- 172.21.15.33:0/571772022 shutdown_connections 2024-05-15T13:57:13.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:13.241+0000 7f1e917c6700 1 -- 172.21.15.33:0/571772022 wait complete. 2024-05-15T13:57:13.244 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2024-05-15T13:57:13.547 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:13 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:13.547 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:13 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:13.548 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:13 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/571772022' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-15T13:57:13.548 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:13 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:57:13.548 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:13 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:57:13.548 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:13 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:13.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:13 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:13.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:13 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:13.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:13 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/571772022' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-15T13:57:13.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:13 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:57:13.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:13 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:57:13.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:13 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:13.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:13 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:13.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:13 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:13.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:13 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/571772022' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-15T13:57:13.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:13 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:57:13.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:13 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:57:13.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:13 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:57:13.860 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-15T13:57:14.109 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:57:14.543 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:14 smithi033 ceph-mon[31650]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:57:14.543 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:14 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/571772022' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-15T13:57:14.543 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:14 smithi033 ceph-mon[31650]: osdmap e74: 12 total, 12 up, 12 in 2024-05-15T13:57:14.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:14 smithi052 ceph-mon[36190]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:57:14.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:14 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/571772022' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-15T13:57:14.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:14 smithi052 ceph-mon[36190]: osdmap e74: 12 total, 12 up, 12 in 2024-05-15T13:57:14.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:14 smithi081 ceph-mon[36434]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:57:14.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:14 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/571772022' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-15T13:57:14.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:14 smithi081 ceph-mon[36434]: osdmap e74: 12 total, 12 up, 12 in 2024-05-15T13:57:15.468 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:15 smithi033 ceph-mon[31650]: osdmap e75: 12 total, 12 up, 12 in 2024-05-15T13:57:15.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:15 smithi052 ceph-mon[36190]: osdmap e75: 12 total, 12 up, 12 in 2024-05-15T13:57:15.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:15 smithi081 ceph-mon[36434]: osdmap e75: 12 total, 12 up, 12 in 2024-05-15T13:57:15.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.669+0000 7f92031e2700 1 -- 172.21.15.33:0/2898999079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91fc1041a0 msgr2=0x7f91fc1081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:15.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.669+0000 7f92031e2700 1 --2- 172.21.15.33:0/2898999079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91fc1041a0 0x7f91fc1081c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f91e80099c0 tx=0x7f91e8009cd0 comp rx=0 tx=0).stop 2024-05-15T13:57:15.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.669+0000 7f92031e2700 1 -- 172.21.15.33:0/2898999079 shutdown_connections 2024-05-15T13:57:15.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.669+0000 7f92031e2700 1 --2- 172.21.15.33:0/2898999079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91fc1088f0 0x7f91fc110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:15.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.669+0000 7f92031e2700 1 --2- 172.21.15.33:0/2898999079 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f91fc1037f0 0x7f91fc103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:15.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.669+0000 7f92031e2700 1 --2- 172.21.15.33:0/2898999079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91fc1041a0 0x7f91fc1081c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:15.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.669+0000 7f92031e2700 1 -- 172.21.15.33:0/2898999079 >> 172.21.15.33:0/2898999079 conn(0x7f91fc07a6a0 msgr2=0x7f91fc07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:15.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.669+0000 7f92031e2700 1 -- 172.21.15.33:0/2898999079 shutdown_connections 2024-05-15T13:57:15.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.669+0000 7f92031e2700 1 -- 172.21.15.33:0/2898999079 wait complete. 2024-05-15T13:57:15.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.670+0000 7f92031e2700 1 Processor -- start 2024-05-15T13:57:15.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.670+0000 7f92031e2700 1 -- start start 2024-05-15T13:57:15.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.670+0000 7f92031e2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91fc1037f0 0x7f91fc19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:15.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.671+0000 7f92031e2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f91fc1041a0 0x7f91fc19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:15.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.671+0000 7f92031e2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91fc1088f0 0x7f91fc199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:15.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.671+0000 7f92031e2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f91fc113700 con 0x7f91fc1037f0 2024-05-15T13:57:15.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.671+0000 7f92031e2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f91fc113580 con 0x7f91fc1088f0 2024-05-15T13:57:15.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.671+0000 7f9200f7e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91fc1037f0 0x7f91fc19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:15.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.671+0000 7f92031e2700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f91fc113880 con 0x7f91fc1041a0 2024-05-15T13:57:15.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.671+0000 7f9200f7e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91fc1037f0 0x7f91fc19f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45424/0 (socket says 172.21.15.33:45424) 2024-05-15T13:57:15.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.671+0000 7f9200f7e700 1 -- 172.21.15.33:0/3333807470 learned_addr learned my addr 172.21.15.33:0/3333807470 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:57:15.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.671+0000 7f91fbfff700 1 --2- 172.21.15.33:0/3333807470 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f91fc1041a0 0x7f91fc19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:15.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.671+0000 7f920177f700 1 --2- 172.21.15.33:0/3333807470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91fc1088f0 0x7f91fc199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:15.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.671+0000 7f9200f7e700 1 -- 172.21.15.33:0/3333807470 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f91fc1041a0 msgr2=0x7f91fc19fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:15.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.671+0000 7f9200f7e700 1 --2- 172.21.15.33:0/3333807470 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f91fc1041a0 0x7f91fc19fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:15.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.671+0000 7f9200f7e700 1 -- 172.21.15.33:0/3333807470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91fc1088f0 msgr2=0x7f91fc199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:15.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.671+0000 7f9200f7e700 1 --2- 172.21.15.33:0/3333807470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91fc1088f0 0x7f91fc199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:15.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.671+0000 7f9200f7e700 1 -- 172.21.15.33:0/3333807470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91fc19a0e0 con 0x7f91fc1037f0 2024-05-15T13:57:15.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.671+0000 7f9200f7e700 1 --2- 172.21.15.33:0/3333807470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91fc1037f0 0x7f91fc19f7c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f91f000c9b0 tx=0x7f91f000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:15.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.671+0000 7f91f9ffb700 1 -- 172.21.15.33:0/3333807470 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f0007b00 con 0x7f91fc1037f0 2024-05-15T13:57:15.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.671+0000 7f91f9ffb700 1 -- 172.21.15.33:0/3333807470 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91f0007ca0 con 0x7f91fc1037f0 2024-05-15T13:57:15.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.671+0000 7f92031e2700 1 -- 172.21.15.33:0/3333807470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91fc19a3d0 con 0x7f91fc1037f0 2024-05-15T13:57:15.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.671+0000 7f91f9ffb700 1 -- 172.21.15.33:0/3333807470 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f0018730 con 0x7f91fc1037f0 2024-05-15T13:57:15.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.672+0000 7f92031e2700 1 -- 172.21.15.33:0/3333807470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91fc1a6690 con 0x7f91fc1037f0 2024-05-15T13:57:15.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.672+0000 7f92031e2700 1 -- 172.21.15.33:0/3333807470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91fc06a230 con 0x7f91fc1037f0 2024-05-15T13:57:15.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.674+0000 7f91f9ffb700 1 -- 172.21.15.33:0/3333807470 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f91f000f450 con 0x7f91fc1037f0 2024-05-15T13:57:15.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.674+0000 7f91f9ffb700 1 --2- 172.21.15.33:0/3333807470 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f91ec0b2900 0x7f91ec0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:15.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.675+0000 7f91fbfff700 1 --2- 172.21.15.33:0/3333807470 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f91ec0b2900 0x7f91ec0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:15.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.675+0000 7f91f9ffb700 1 -- 172.21.15.33:0/3333807470 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7497+0+0 (secure 0 0 0) 0x7f91f00e21f0 con 0x7f91fc1037f0 2024-05-15T13:57:15.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.675+0000 7f91fbfff700 1 --2- 172.21.15.33:0/3333807470 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f91ec0b2900 0x7f91ec0b4dc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f91fc108190 tx=0x7f91e801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:15.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.678+0000 7f91f9ffb700 1 -- 172.21.15.33:0/3333807470 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f91f0014030 con 0x7f91fc1037f0 2024-05-15T13:57:15.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:15.892+0000 7f92031e2700 1 -- 172.21.15.33:0/3333807470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f91fc02e560 con 0x7f91fc1037f0 2024-05-15T13:57:16.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:16.359+0000 7f91f9ffb700 1 -- 172.21.15.33:0/3333807470 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v77) ==== 170+0+0 (secure 0 0 0) 0x7f91f00aa350 con 0x7f91fc1037f0 2024-05-15T13:57:16.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:16.362+0000 7f92031e2700 1 -- 172.21.15.33:0/3333807470 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f91ec0b2900 msgr2=0x7f91ec0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:16.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:16.362+0000 7f92031e2700 1 --2- 172.21.15.33:0/3333807470 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f91ec0b2900 0x7f91ec0b4dc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f91fc108190 tx=0x7f91e801a040 comp rx=0 tx=0).stop 2024-05-15T13:57:16.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:16.362+0000 7f92031e2700 1 -- 172.21.15.33:0/3333807470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91fc1037f0 msgr2=0x7f91fc19f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:16.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:16.362+0000 7f92031e2700 1 --2- 172.21.15.33:0/3333807470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91fc1037f0 0x7f91fc19f7c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f91f000c9b0 tx=0x7f91f000cd70 comp rx=0 tx=0).stop 2024-05-15T13:57:16.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:16.362+0000 7f92031e2700 1 -- 172.21.15.33:0/3333807470 shutdown_connections 2024-05-15T13:57:16.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:16.362+0000 7f92031e2700 1 --2- 172.21.15.33:0/3333807470 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f91ec0b2900 0x7f91ec0b4dc0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:16.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:16.362+0000 7f92031e2700 1 --2- 172.21.15.33:0/3333807470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91fc1088f0 0x7f91fc199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:16.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:16.362+0000 7f92031e2700 1 --2- 172.21.15.33:0/3333807470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91fc1037f0 0x7f91fc19f7c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:16.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:16.362+0000 7f92031e2700 1 --2- 172.21.15.33:0/3333807470 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f91fc1041a0 0x7f91fc19fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:16.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:16.362+0000 7f92031e2700 1 -- 172.21.15.33:0/3333807470 >> 172.21.15.33:0/3333807470 conn(0x7f91fc07a6a0 msgr2=0x7f91fc101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:16.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:16.362+0000 7f92031e2700 1 -- 172.21.15.33:0/3333807470 shutdown_connections 2024-05-15T13:57:16.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:16.362+0000 7f92031e2700 1 -- 172.21.15.33:0/3333807470 wait complete. 2024-05-15T13:57:16.366 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2024-05-15T13:57:16.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:16 smithi052 ceph-mon[36190]: pgmap v280: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:57:16.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:16 smithi052 ceph-mon[36190]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-15T13:57:16.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:16 smithi052 ceph-mon[36190]: osdmap e76: 12 total, 12 up, 12 in 2024-05-15T13:57:16.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:16 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/3333807470' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-15T13:57:16.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:16 smithi081 ceph-mon[36434]: pgmap v280: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:57:16.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:16 smithi081 ceph-mon[36434]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-15T13:57:16.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:16 smithi081 ceph-mon[36434]: osdmap e76: 12 total, 12 up, 12 in 2024-05-15T13:57:16.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:16 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/3333807470' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-15T13:57:16.671 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:16 smithi033 ceph-mon[31650]: pgmap v280: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:57:16.671 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:16 smithi033 ceph-mon[31650]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-15T13:57:16.671 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:16 smithi033 ceph-mon[31650]: osdmap e76: 12 total, 12 up, 12 in 2024-05-15T13:57:16.671 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:16 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/3333807470' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-15T13:57:17.029 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-15T13:57:17.284 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:57:17.649 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:17 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/3333807470' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-15T13:57:17.649 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:17 smithi033 ceph-mon[31650]: osdmap e77: 12 total, 12 up, 12 in 2024-05-15T13:57:17.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:17 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/3333807470' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-15T13:57:17.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:17 smithi052 ceph-mon[36190]: osdmap e77: 12 total, 12 up, 12 in 2024-05-15T13:57:17.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:17 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/3333807470' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-15T13:57:17.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:17 smithi081 ceph-mon[36434]: osdmap e77: 12 total, 12 up, 12 in 2024-05-15T13:57:18.542 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:18 smithi033 ceph-mon[31650]: pgmap v283: 97 pgs: 15 creating+peering, 59 unknown, 23 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:57:18.543 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:18 smithi033 ceph-mon[31650]: osdmap e78: 12 total, 12 up, 12 in 2024-05-15T13:57:18.758 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:18 smithi081 ceph-mon[36434]: pgmap v283: 97 pgs: 15 creating+peering, 59 unknown, 23 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:57:18.758 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:18 smithi081 ceph-mon[36434]: osdmap e78: 12 total, 12 up, 12 in 2024-05-15T13:57:18.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.782+0000 7fa75e908700 1 -- 172.21.15.33:0/234427958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7581095b0 msgr2=0x7fa758109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:18.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.782+0000 7fa75e908700 1 --2- 172.21.15.33:0/234427958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7581095b0 0x7fa758109990 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa748009990 tx=0x7fa748009ca0 comp rx=0 tx=0).stop 2024-05-15T13:57:18.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.782+0000 7fa75e908700 1 -- 172.21.15.33:0/234427958 shutdown_connections 2024-05-15T13:57:18.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.782+0000 7fa75e908700 1 --2- 172.21.15.33:0/234427958 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa75806bbe0 0x7fa7581103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:18.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.782+0000 7fa75e908700 1 --2- 172.21.15.33:0/234427958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7581095b0 0x7fa758109990 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:18.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.782+0000 7fa75e908700 1 --2- 172.21.15.33:0/234427958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa75806b220 0x7fa75806b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:18.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.782+0000 7fa75e908700 1 -- 172.21.15.33:0/234427958 >> 172.21.15.33:0/234427958 conn(0x7fa7580793f0 msgr2=0x7fa758079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:18.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.782+0000 7fa75e908700 1 -- 172.21.15.33:0/234427958 shutdown_connections 2024-05-15T13:57:18.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.782+0000 7fa75e908700 1 -- 172.21.15.33:0/234427958 wait complete. 2024-05-15T13:57:18.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.783+0000 7fa75e908700 1 Processor -- start 2024-05-15T13:57:18.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.783+0000 7fa75e908700 1 -- start start 2024-05-15T13:57:18.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.783+0000 7fa75e908700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa75806b220 0x7fa75819f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:18.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.783+0000 7fa75e908700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa75806bbe0 0x7fa75819fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:18.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.783+0000 7fa75e908700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7581095b0 0x7fa758199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:18.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.783+0000 7fa75e908700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa7581137c0 con 0x7fa7581095b0 2024-05-15T13:57:18.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.783+0000 7fa75e908700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa758113640 con 0x7fa75806b220 2024-05-15T13:57:18.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.783+0000 7fa75e908700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa758113940 con 0x7fa75806bbe0 2024-05-15T13:57:18.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.783+0000 7fa75cea5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7581095b0 0x7fa758199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:18.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.783+0000 7fa757fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa75806b220 0x7fa75819f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:18.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.783+0000 7fa75cea5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7581095b0 0x7fa758199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45450/0 (socket says 172.21.15.33:45450) 2024-05-15T13:57:18.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.783+0000 7fa75cea5700 1 -- 172.21.15.33:0/2486011275 learned_addr learned my addr 172.21.15.33:0/2486011275 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:57:18.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.783+0000 7fa74ffff700 1 --2- 172.21.15.33:0/2486011275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa75806bbe0 0x7fa75819fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:18.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.784+0000 7fa75cea5700 1 -- 172.21.15.33:0/2486011275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa75806bbe0 msgr2=0x7fa75819fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:18.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.784+0000 7fa75cea5700 1 --2- 172.21.15.33:0/2486011275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa75806bbe0 0x7fa75819fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:18.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.784+0000 7fa75cea5700 1 -- 172.21.15.33:0/2486011275 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa75806b220 msgr2=0x7fa75819f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:18.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.784+0000 7fa75cea5700 1 --2- 172.21.15.33:0/2486011275 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa75806b220 0x7fa75819f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:18.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.784+0000 7fa75cea5700 1 -- 172.21.15.33:0/2486011275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa75819a240 con 0x7fa7581095b0 2024-05-15T13:57:18.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.784+0000 7fa757fff700 1 --2- 172.21.15.33:0/2486011275 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa75806b220 0x7fa75819f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T13:57:18.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.784+0000 7fa75cea5700 1 --2- 172.21.15.33:0/2486011275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7581095b0 0x7fa758199ae0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa750007ee0 tx=0x7fa75000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:18.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.784+0000 7fa755ffb700 1 -- 172.21.15.33:0/2486011275 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa750004bb0 con 0x7fa7581095b0 2024-05-15T13:57:18.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.784+0000 7fa755ffb700 1 -- 172.21.15.33:0/2486011275 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa750004d50 con 0x7fa7581095b0 2024-05-15T13:57:18.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.784+0000 7fa75e908700 1 -- 172.21.15.33:0/2486011275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa75819a530 con 0x7fa7581095b0 2024-05-15T13:57:18.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.784+0000 7fa755ffb700 1 -- 172.21.15.33:0/2486011275 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa750005920 con 0x7fa7581095b0 2024-05-15T13:57:18.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.784+0000 7fa75e908700 1 -- 172.21.15.33:0/2486011275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7581a67f0 con 0x7fa7581095b0 2024-05-15T13:57:18.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.785+0000 7fa75e908700 1 -- 172.21.15.33:0/2486011275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa75810ba90 con 0x7fa7581095b0 2024-05-15T13:57:18.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.787+0000 7fa755ffb700 1 -- 172.21.15.33:0/2486011275 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fa75001a020 con 0x7fa7581095b0 2024-05-15T13:57:18.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.788+0000 7fa755ffb700 1 --2- 172.21.15.33:0/2486011275 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fa7440b2900 0x7fa7440b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:18.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.788+0000 7fa757fff700 1 --2- 172.21.15.33:0/2486011275 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fa7440b2900 0x7fa7440b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:18.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.788+0000 7fa755ffb700 1 -- 172.21.15.33:0/2486011275 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7879+0+0 (secure 0 0 0) 0x7fa7500e1d70 con 0x7fa7581095b0 2024-05-15T13:57:18.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.788+0000 7fa757fff700 1 --2- 172.21.15.33:0/2486011275 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fa7440b2900 0x7fa7440b4dc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa748006010 tx=0x7fa74800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:18.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:18.792+0000 7fa755ffb700 1 -- 172.21.15.33:0/2486011275 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa750010070 con 0x7fa7581095b0 2024-05-15T13:57:18.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:18 smithi052 ceph-mon[36190]: pgmap v283: 97 pgs: 15 creating+peering, 59 unknown, 23 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:57:18.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:18 smithi052 ceph-mon[36190]: osdmap e78: 12 total, 12 up, 12 in 2024-05-15T13:57:19.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:19.054+0000 7fa75e908700 1 -- 172.21.15.33:0/2486011275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa75810b470 con 0x7fa7581095b0 2024-05-15T13:57:19.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:19.376+0000 7fa755ffb700 1 -- 172.21.15.33:0/2486011275 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fa7500a9ba0 con 0x7fa7581095b0 2024-05-15T13:57:19.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:19.379+0000 7fa75e908700 1 -- 172.21.15.33:0/2486011275 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fa7440b2900 msgr2=0x7fa7440b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:19.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:19.379+0000 7fa75e908700 1 --2- 172.21.15.33:0/2486011275 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fa7440b2900 0x7fa7440b4dc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa748006010 tx=0x7fa74800b540 comp rx=0 tx=0).stop 2024-05-15T13:57:19.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:19.379+0000 7fa75e908700 1 -- 172.21.15.33:0/2486011275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7581095b0 msgr2=0x7fa758199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:19.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:19.379+0000 7fa75e908700 1 --2- 172.21.15.33:0/2486011275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7581095b0 0x7fa758199ae0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa750007ee0 tx=0x7fa75000d3b0 comp rx=0 tx=0).stop 2024-05-15T13:57:19.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:19.379+0000 7fa75e908700 1 -- 172.21.15.33:0/2486011275 shutdown_connections 2024-05-15T13:57:19.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:19.379+0000 7fa75e908700 1 --2- 172.21.15.33:0/2486011275 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fa7440b2900 0x7fa7440b4dc0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:19.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:19.379+0000 7fa75e908700 1 --2- 172.21.15.33:0/2486011275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7581095b0 0x7fa758199ae0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:19.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:19.379+0000 7fa75e908700 1 --2- 172.21.15.33:0/2486011275 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa75806b220 0x7fa75819f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:19.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:19.379+0000 7fa75e908700 1 --2- 172.21.15.33:0/2486011275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa75806bbe0 0x7fa75819fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:19.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:19.379+0000 7fa75e908700 1 -- 172.21.15.33:0/2486011275 >> 172.21.15.33:0/2486011275 conn(0x7fa7580793f0 msgr2=0x7fa758101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:19.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:19.379+0000 7fa75e908700 1 -- 172.21.15.33:0/2486011275 shutdown_connections 2024-05-15T13:57:19.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:19.379+0000 7fa75e908700 1 -- 172.21.15.33:0/2486011275 wait complete. 2024-05-15T13:57:19.383 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 2 and data pool 3 2024-05-15T13:57:19.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:19 smithi033 ceph-mon[31650]: osdmap e79: 12 total, 12 up, 12 in 2024-05-15T13:57:19.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:19 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/2486011275' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-15T13:57:19.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:19 smithi033 ceph-mon[31650]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-15T13:57:19.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:19 smithi052 ceph-mon[36190]: osdmap e79: 12 total, 12 up, 12 in 2024-05-15T13:57:19.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:19 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/2486011275' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-15T13:57:19.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:19 smithi052 ceph-mon[36190]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-15T13:57:19.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:19 smithi081 ceph-mon[36434]: osdmap e79: 12 total, 12 up, 12 in 2024-05-15T13:57:19.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:19 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/2486011275' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-15T13:57:19.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:19 smithi081 ceph-mon[36434]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-15T13:57:20.012 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-15T13:57:20.269 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:57:20.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:20 smithi033 ceph-mon[31650]: pgmap v286: 97 pgs: 17 creating+peering, 31 unknown, 49 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:57:20.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:20 smithi033 ceph-mon[31650]: osdmap e80: 12 total, 12 up, 12 in 2024-05-15T13:57:20.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:20 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/2486011275' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-15T13:57:20.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:20 smithi033 ceph-mon[31650]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-15T13:57:20.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:20 smithi033 ceph-mon[31650]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-15T13:57:20.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:20 smithi033 ceph-mon[31650]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-15T13:57:20.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:20 smithi033 ceph-mon[31650]: mds.? [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] up:active 2024-05-15T13:57:20.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:20 smithi033 ceph-mon[31650]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-15T13:57:20.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:20 smithi052 ceph-mon[36190]: pgmap v286: 97 pgs: 17 creating+peering, 31 unknown, 49 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:57:20.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:20 smithi052 ceph-mon[36190]: osdmap e80: 12 total, 12 up, 12 in 2024-05-15T13:57:20.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:20 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/2486011275' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-15T13:57:20.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:20 smithi052 ceph-mon[36190]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-15T13:57:20.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:20 smithi052 ceph-mon[36190]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-15T13:57:20.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:20 smithi052 ceph-mon[36190]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-15T13:57:20.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:20 smithi052 ceph-mon[36190]: mds.? [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] up:active 2024-05-15T13:57:20.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:20 smithi052 ceph-mon[36190]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-15T13:57:20.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:20 smithi081 ceph-mon[36434]: pgmap v286: 97 pgs: 17 creating+peering, 31 unknown, 49 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T13:57:20.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:20 smithi081 ceph-mon[36434]: osdmap e80: 12 total, 12 up, 12 in 2024-05-15T13:57:20.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:20 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/2486011275' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-15T13:57:20.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:20 smithi081 ceph-mon[36434]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-15T13:57:20.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:20 smithi081 ceph-mon[36434]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-15T13:57:20.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:20 smithi081 ceph-mon[36434]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-15T13:57:20.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:20 smithi081 ceph-mon[36434]: mds.? [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] up:active 2024-05-15T13:57:20.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:20 smithi081 ceph-mon[36434]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-15T13:57:21.651 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:21 smithi033 ceph-mon[31650]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-15T13:57:21.651 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:21 smithi033 ceph-mon[31650]: Cluster is now healthy 2024-05-15T13:57:21.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:21 smithi052 ceph-mon[36190]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-15T13:57:21.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:21 smithi052 ceph-mon[36190]: Cluster is now healthy 2024-05-15T13:57:21.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:21 smithi081 ceph-mon[36434]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-15T13:57:21.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:21 smithi081 ceph-mon[36434]: Cluster is now healthy 2024-05-15T13:57:21.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.961+0000 7f8833f5f700 1 -- 172.21.15.33:0/439040332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f882c06bbe0 msgr2=0x7f882c1103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:21.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.961+0000 7f8833f5f700 1 --2- 172.21.15.33:0/439040332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f882c06bbe0 0x7f882c1103d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f882400b0a0 tx=0x7f882400b3b0 comp rx=0 tx=0).stop 2024-05-15T13:57:21.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.961+0000 7f8833f5f700 1 -- 172.21.15.33:0/439040332 shutdown_connections 2024-05-15T13:57:21.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.961+0000 7f8833f5f700 1 --2- 172.21.15.33:0/439040332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f882c06bbe0 0x7f882c1103d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:21.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.961+0000 7f8833f5f700 1 --2- 172.21.15.33:0/439040332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f882c1095b0 0x7f882c109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:21.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.961+0000 7f8833f5f700 1 --2- 172.21.15.33:0/439040332 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f882c06b220 0x7f882c06b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:21.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.961+0000 7f8833f5f700 1 -- 172.21.15.33:0/439040332 >> 172.21.15.33:0/439040332 conn(0x7f882c0793f0 msgr2=0x7f882c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:21.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.961+0000 7f8833f5f700 1 -- 172.21.15.33:0/439040332 shutdown_connections 2024-05-15T13:57:21.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.961+0000 7f8833f5f700 1 -- 172.21.15.33:0/439040332 wait complete. 2024-05-15T13:57:21.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.962+0000 7f8833f5f700 1 Processor -- start 2024-05-15T13:57:21.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.962+0000 7f8833f5f700 1 -- start start 2024-05-15T13:57:21.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.962+0000 7f8833f5f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f882c06b220 0x7f882c102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:21.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.962+0000 7f8833f5f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f882c06bbe0 0x7f882c102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:21.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.962+0000 7f8833f5f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f882c1095b0 0x7f882c102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:21.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.962+0000 7f8833f5f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f882c113720 con 0x7f882c06b220 2024-05-15T13:57:21.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.962+0000 7f8833f5f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f882c1135a0 con 0x7f882c1095b0 2024-05-15T13:57:21.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.962+0000 7f8833f5f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f882c1138a0 con 0x7f882c06bbe0 2024-05-15T13:57:21.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.963+0000 7f8831cfb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f882c06b220 0x7f882c102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:21.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.963+0000 7f8831cfb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f882c06b220 0x7f882c102510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45478/0 (socket says 172.21.15.33:45478) 2024-05-15T13:57:21.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.963+0000 7f8831cfb700 1 -- 172.21.15.33:0/3687286250 learned_addr learned my addr 172.21.15.33:0/3687286250 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:57:21.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.963+0000 7f88324fc700 1 --2- 172.21.15.33:0/3687286250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f882c1095b0 0x7f882c102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:21.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.963+0000 7f88314fa700 1 --2- 172.21.15.33:0/3687286250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f882c06bbe0 0x7f882c102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:21.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.963+0000 7f8831cfb700 1 -- 172.21.15.33:0/3687286250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f882c06bbe0 msgr2=0x7f882c102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:21.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.963+0000 7f8831cfb700 1 --2- 172.21.15.33:0/3687286250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f882c06bbe0 0x7f882c102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:21.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.963+0000 7f8831cfb700 1 -- 172.21.15.33:0/3687286250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f882c1095b0 msgr2=0x7f882c102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:21.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.963+0000 7f8831cfb700 1 --2- 172.21.15.33:0/3687286250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f882c1095b0 0x7f882c102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:21.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.963+0000 7f8831cfb700 1 -- 172.21.15.33:0/3687286250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f882c1a8140 con 0x7f882c06b220 2024-05-15T13:57:21.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.963+0000 7f88324fc700 1 --2- 172.21.15.33:0/3687286250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f882c1095b0 0x7f882c102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T13:57:21.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.963+0000 7f88314fa700 1 --2- 172.21.15.33:0/3687286250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f882c06bbe0 0x7f882c102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T13:57:21.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.963+0000 7f8831cfb700 1 --2- 172.21.15.33:0/3687286250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f882c06b220 0x7f882c102510 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f882800c920 tx=0x7f882800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:21.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.963+0000 7f8822ffd700 1 -- 172.21.15.33:0/3687286250 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8828007b00 con 0x7f882c06b220 2024-05-15T13:57:21.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.963+0000 7f8822ffd700 1 -- 172.21.15.33:0/3687286250 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8828004ce0 con 0x7f882c06b220 2024-05-15T13:57:21.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.963+0000 7f8833f5f700 1 -- 172.21.15.33:0/3687286250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f882c1a8430 con 0x7f882c06b220 2024-05-15T13:57:21.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.963+0000 7f8822ffd700 1 -- 172.21.15.33:0/3687286250 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8828005650 con 0x7f882c06b220 2024-05-15T13:57:21.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.963+0000 7f8833f5f700 1 -- 172.21.15.33:0/3687286250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f882c1a89c0 con 0x7f882c06b220 2024-05-15T13:57:21.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.964+0000 7f8833f5f700 1 -- 172.21.15.33:0/3687286250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f882c06a230 con 0x7f882c06b220 2024-05-15T13:57:21.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.966+0000 7f8822ffd700 1 -- 172.21.15.33:0/3687286250 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8828004750 con 0x7f882c06b220 2024-05-15T13:57:21.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.966+0000 7f8822ffd700 1 --2- 172.21.15.33:0/3687286250 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f88180b2900 0x7f88180b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:21.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.967+0000 7f88314fa700 1 --2- 172.21.15.33:0/3687286250 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f88180b2900 0x7f88180b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:21.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.967+0000 7f8822ffd700 1 -- 172.21.15.33:0/3687286250 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f88280e1ff0 con 0x7f882c06b220 2024-05-15T13:57:21.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.967+0000 7f88314fa700 1 --2- 172.21.15.33:0/3687286250 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f88180b2900 0x7f88180b4dc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f881c005d90 tx=0x7f881c005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:21.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:21.971+0000 7f8822ffd700 1 -- 172.21.15.33:0/3687286250 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8828014030 con 0x7f882c06b220 2024-05-15T13:57:22.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:22.185+0000 7f8833f5f700 1 -- 172.21.15.33:0/3687286250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f882c02e560 con 0x7f882c06b220 2024-05-15T13:57:22.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:22.187+0000 7f8822ffd700 1 -- 172.21.15.33:0/3687286250 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22112 (secure 0 0 0) 0x7f88280a9ee0 con 0x7f882c06b220 2024-05-15T13:57:22.190 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:57:22.190 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":80,"fsid":"bca78ca4-12c1-11ef-bc98-c7b262605968","created":"2024-05-15T13:49:10.327613+0000","modified":"2024-05-15T13:57:19.056451+0000","last_up_change":"2024-05-15T13:56:16.097711+0000","last_in_change":"2024-05-15T13:55:56.722143+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-15T13:52:32.970844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-15T13:57:12.324802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-15T13:57:15.898406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b434111-810f-450a-b63d-c97080e9f8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6803","nonce":887409130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6805","nonce":887409130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6809","nonce":887409130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6807","nonce":887409130}]},"public_addr":"172.21.15.33:6803/887409130","cluster_addr":"172.21.15.33:6805/887409130","heartbeat_back_addr":"172.21.15.33:6809/887409130","heartbeat_front_addr":"172.21.15.33:6807/887409130","state":["exists","up"]},{"osd":1,"uuid":"0b578d4e-a841-4b27-bc95-d6c3bae580fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6801","nonce":2794180003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6803","nonce":2794180003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6807","nonce":2794180003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6805","nonce":2794180003}]},"public_addr":"172.21.15.52:6801/2794180003","cluster_addr":"172.21.15.52:6803/2794180003","heartbeat_back_addr":"172.21.15.52:6807/2794180003","heartbeat_front_addr":"172.21.15.52:6805/2794180003","state":["exists","up"]},{"osd":2,"uuid":"0e9c9390-ed33-464b-8f39-bca75941e05e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6801","nonce":315610978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6803","nonce":315610978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6807","nonce":315610978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6805","nonce":315610978}]},"public_addr":"172.21.15.81:6801/315610978","cluster_addr":"172.21.15.81:6803/315610978","heartbeat_back_addr":"172.21.15.81:6807/315610978","heartbeat_front_addr":"172.21.15.81:6805/315610978","state":["exists","up"]},{"osd":3,"uuid":"ecdb24c1-fb15-42a5-9bae-436bbae9512a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6811","nonce":818533501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6813","nonce":818533501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6817","nonce":818533501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6815","nonce":818533501}]},"public_addr":"172.21.15.33:6811/818533501","cluster_addr":"172.21.15.33:6813/818533501","heartbeat_back_addr":"172.21.15.33:6817/818533501","heartbeat_front_addr":"172.21.15.33:6815/818533501","state":["exists","up"]},{"osd":4,"uuid":"706752a0-b17f-4563-a7a9-d5078bf0f3b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6809","nonce":3010133776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6811","nonce":3010133776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6815","nonce":3010133776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6813","nonce":3010133776}]},"public_addr":"172.21.15.52:6809/3010133776","cluster_addr":"172.21.15.52:6811/3010133776","heartbeat_back_addr":"172.21.15.52:6815/3010133776","heartbeat_front_addr":"172.21.15.52:6813/3010133776","state":["exists","up"]},{"osd":5,"uuid":"3d45ab97-f364-4903-974d-8acc961363d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6809","nonce":2132616297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6811","nonce":2132616297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6815","nonce":2132616297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6813","nonce":2132616297}]},"public_addr":"172.21.15.81:6809/2132616297","cluster_addr":"172.21.15.81:6811/2132616297","heartbeat_back_addr":"172.21.15.81:6815/2132616297","heartbeat_front_addr":"172.21.15.81:6813/2132616297","state":["exists","up"]},{"osd":6,"uuid":"3be524c5-dcf9-4742-b896-ea9a44a6b391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6819","nonce":1260691801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6821","nonce":1260691801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6825","nonce":1260691801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6823","nonce":1260691801}]},"public_addr":"172.21.15.33:6819/1260691801","cluster_addr":"172.21.15.33:6821/1260691801","heartbeat_back_addr":"172.21.15.33:6825/1260691801","heartbeat_front_addr":"172.21.15.33:6823/1260691801","state":["exists","up"]},{"osd":7,"uuid":"bda2f920-4f2c-4eed-8127-83c1973676e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6817","nonce":1654595813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6819","nonce":1654595813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6823","nonce":1654595813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6821","nonce":1654595813}]},"public_addr":"172.21.15.52:6817/1654595813","cluster_addr":"172.21.15.52:6819/1654595813","heartbeat_back_addr":"172.21.15.52:6823/1654595813","heartbeat_front_addr":"172.21.15.52:6821/1654595813","state":["exists","up"]},{"osd":8,"uuid":"550585b8-0df4-49ee-95c8-3d084744dbd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6817","nonce":3009661748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6819","nonce":3009661748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6823","nonce":3009661748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6821","nonce":3009661748}]},"public_addr":"172.21.15.81:6817/3009661748","cluster_addr":"172.21.15.81:6819/3009661748","heartbeat_back_addr":"172.21.15.81:6823/3009661748","heartbeat_front_addr":"172.21.15.81:6821/3009661748","state":["exists","up"]},{"osd":9,"uuid":"1d781846-26f3-4b23-b22c-c994a8509a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6827","nonce":2843220858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6829","nonce":2843220858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6833","nonce":2843220858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6831","nonce":2843220858}]},"public_addr":"172.21.15.33:6827/2843220858","cluster_addr":"172.21.15.33:6829/2843220858","heartbeat_back_addr":"172.21.15.33:6833/2843220858","heartbeat_front_addr":"172.21.15.33:6831/2843220858","state":["exists","up"]},{"osd":10,"uuid":"af58c940-fdc4-418b-a638-7dc44438b34c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6825","nonce":2422349831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6827","nonce":2422349831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6831","nonce":2422349831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6829","nonce":2422349831}]},"public_addr":"172.21.15.52:6825/2422349831","cluster_addr":"172.21.15.52:6827/2422349831","heartbeat_back_addr":"172.21.15.52:6831/2422349831","heartbeat_front_addr":"172.21.15.52:6829/2422349831","state":["exists","up"]},{"osd":11,"uuid":"ab84aaea-2086-4842-8bf4-628842fce46a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6825","nonce":326313090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6827","nonce":326313090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6831","nonce":326313090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6829","nonce":326313090}]},"public_addr":"172.21.15.81:6825/326313090","cluster_addr":"172.21.15.81:6827/326313090","heartbeat_back_addr":"172.21.15.81:6831/326313090","heartbeat_front_addr":"172.21.15.81:6829/326313090","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:51:41.493136+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:52:05.784773+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:52:29.769429+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:52:54.891620+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:53:19.412922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:53:43.921841+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:54:09.486732+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:54:34.064370+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:54:58.655304+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:55:24.548693+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:55:49.941191+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:56:14.919558+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/436121571":"2024-05-16T13:50:04.881656+0000","172.21.15.33:6801/2680375465":"2024-05-16T13:50:04.881656+0000","172.21.15.33:6800/2680375465":"2024-05-16T13:50:04.881656+0000","172.21.15.33:0/1022787179":"2024-05-16T13:49:37.564643+0000","172.21.15.33:0/3720711369":"2024-05-16T13:50:04.881656+0000","172.21.15.33:0/1282466838":"2024-05-16T13:49:37.564643+0000","172.21.15.33:0/3022248969":"2024-05-16T13:49:37.564643+0000","172.21.15.33:6801/3303495364":"2024-05-16T13:49:37.564643+0000","172.21.15.33:6800/3303495364":"2024-05-16T13:49:37.564643+0000","172.21.15.33:0/4282812692":"2024-05-16T13:50:04.881656+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-15T13:57:22.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:22.189+0000 7f8833f5f700 1 -- 172.21.15.33:0/3687286250 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f88180b2900 msgr2=0x7f88180b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:22.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:22.189+0000 7f8833f5f700 1 --2- 172.21.15.33:0/3687286250 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f88180b2900 0x7f88180b4dc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f881c005d90 tx=0x7f881c005d00 comp rx=0 tx=0).stop 2024-05-15T13:57:22.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:22.189+0000 7f8833f5f700 1 -- 172.21.15.33:0/3687286250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f882c06b220 msgr2=0x7f882c102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:22.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:22.189+0000 7f8833f5f700 1 --2- 172.21.15.33:0/3687286250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f882c06b220 0x7f882c102510 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f882800c920 tx=0x7f882800cce0 comp rx=0 tx=0).stop 2024-05-15T13:57:22.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:22.189+0000 7f8833f5f700 1 -- 172.21.15.33:0/3687286250 shutdown_connections 2024-05-15T13:57:22.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:22.189+0000 7f8833f5f700 1 --2- 172.21.15.33:0/3687286250 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f88180b2900 0x7f88180b4dc0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:22.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:22.189+0000 7f8833f5f700 1 --2- 172.21.15.33:0/3687286250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f882c1095b0 0x7f882c102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:22.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:22.189+0000 7f8833f5f700 1 --2- 172.21.15.33:0/3687286250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f882c06b220 0x7f882c102510 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:22.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:22.189+0000 7f8833f5f700 1 --2- 172.21.15.33:0/3687286250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f882c06bbe0 0x7f882c102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:22.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:22.189+0000 7f8833f5f700 1 -- 172.21.15.33:0/3687286250 >> 172.21.15.33:0/3687286250 conn(0x7f882c0793f0 msgr2=0x7f882c1010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:22.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:22.189+0000 7f8833f5f700 1 -- 172.21.15.33:0/3687286250 shutdown_connections 2024-05-15T13:57:22.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:22.189+0000 7f8833f5f700 1 -- 172.21.15.33:0/3687286250 wait complete. 2024-05-15T13:57:22.490 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:22 smithi033 ceph-mon[31650]: pgmap v288: 97 pgs: 17 creating+peering, 11 unknown, 69 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 222 B/s wr, 4 op/s 2024-05-15T13:57:22.490 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:22 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/3687286250' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T13:57:22.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:22 smithi052 ceph-mon[36190]: pgmap v288: 97 pgs: 17 creating+peering, 11 unknown, 69 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 222 B/s wr, 4 op/s 2024-05-15T13:57:22.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:22 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/3687286250' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T13:57:22.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:22 smithi081 ceph-mon[36434]: pgmap v288: 97 pgs: 17 creating+peering, 11 unknown, 69 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 222 B/s wr, 4 op/s 2024-05-15T13:57:22.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:22 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/3687286250' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T13:57:22.896 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-15T13:57:23.153 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:57:24.357 INFO:journalctl@ceph.mds.i.smithi081.stdout:May 15 13:57:24 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-i[66884]: 2024-05-15T13:57:24.078+0000 7fd51b287700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-15T13:57:24.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:24 smithi033 ceph-mon[31650]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 6 op/s 2024-05-15T13:57:24.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.788+0000 7f4de7a1a700 1 -- 172.21.15.33:0/1616174603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4de006c6d0 msgr2=0x7f4de0110400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:24.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.788+0000 7f4de7a1a700 1 --2- 172.21.15.33:0/1616174603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4de006c6d0 0x7f4de0110400 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4dd80099c0 tx=0x7f4dd8009cd0 comp rx=0 tx=0).stop 2024-05-15T13:57:24.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.788+0000 7f4de7a1a700 1 -- 172.21.15.33:0/1616174603 shutdown_connections 2024-05-15T13:57:24.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.788+0000 7f4de7a1a700 1 --2- 172.21.15.33:0/1616174603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4de006c6d0 0x7f4de0110400 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:24.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.788+0000 7f4de7a1a700 1 --2- 172.21.15.33:0/1616174603 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4de0109630 0x7f4de0109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:24.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.788+0000 7f4de7a1a700 1 --2- 172.21.15.33:0/1616174603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4de006bd10 0x7f4de006c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:24.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.788+0000 7f4de7a1a700 1 -- 172.21.15.33:0/1616174603 >> 172.21.15.33:0/1616174603 conn(0x7f4de00794e0 msgr2=0x7f4de00798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:24.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.788+0000 7f4de7a1a700 1 -- 172.21.15.33:0/1616174603 shutdown_connections 2024-05-15T13:57:24.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.788+0000 7f4de7a1a700 1 -- 172.21.15.33:0/1616174603 wait complete. 2024-05-15T13:57:24.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.788+0000 7f4de7a1a700 1 Processor -- start 2024-05-15T13:57:24.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.788+0000 7f4de7a1a700 1 -- start start 2024-05-15T13:57:24.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.789+0000 7f4de7a1a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4de006bd10 0x7f4de019f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:24.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.789+0000 7f4de7a1a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4de006c6d0 0x7f4de019fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:24.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.789+0000 7f4de7a1a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4de0109630 0x7f4de0199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:24.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.789+0000 7f4de7a1a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4de0113750 con 0x7f4de006c6d0 2024-05-15T13:57:24.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.789+0000 7f4de7a1a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4de01135d0 con 0x7f4de0109630 2024-05-15T13:57:24.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.789+0000 7f4de7a1a700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4de01138d0 con 0x7f4de006bd10 2024-05-15T13:57:24.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.789+0000 7f4de4fb5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4de006c6d0 0x7f4de019fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:24.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.789+0000 7f4de4fb5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4de006c6d0 0x7f4de019fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40662/0 (socket says 172.21.15.33:40662) 2024-05-15T13:57:24.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.789+0000 7f4de4fb5700 1 -- 172.21.15.33:0/3059113599 learned_addr learned my addr 172.21.15.33:0/3059113599 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:57:24.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.789+0000 7f4de5fb7700 1 --2- 172.21.15.33:0/3059113599 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4de0109630 0x7f4de0199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:24.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.789+0000 7f4de57b6700 1 --2- 172.21.15.33:0/3059113599 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4de006bd10 0x7f4de019f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:24.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.789+0000 7f4de4fb5700 1 -- 172.21.15.33:0/3059113599 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4de006bd10 msgr2=0x7f4de019f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:24.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.789+0000 7f4de4fb5700 1 --2- 172.21.15.33:0/3059113599 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4de006bd10 0x7f4de019f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:24.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.789+0000 7f4de4fb5700 1 -- 172.21.15.33:0/3059113599 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4de0109630 msgr2=0x7f4de0199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:24.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.789+0000 7f4de4fb5700 1 --2- 172.21.15.33:0/3059113599 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4de0109630 0x7f4de0199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:24.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.789+0000 7f4de4fb5700 1 -- 172.21.15.33:0/3059113599 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de019a170 con 0x7f4de006c6d0 2024-05-15T13:57:24.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.789+0000 7f4de4fb5700 1 --2- 172.21.15.33:0/3059113599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4de006c6d0 0x7f4de019fd90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4dd000c970 tx=0x7f4dd000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:24.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.789+0000 7f4dd67fc700 1 -- 172.21.15.33:0/3059113599 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd00078d0 con 0x7f4de006c6d0 2024-05-15T13:57:24.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.789+0000 7f4de7a1a700 1 -- 172.21.15.33:0/3059113599 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4de019a460 con 0x7f4de006c6d0 2024-05-15T13:57:24.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.790+0000 7f4dd67fc700 1 -- 172.21.15.33:0/3059113599 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dd0018480 con 0x7f4de006c6d0 2024-05-15T13:57:24.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.790+0000 7f4dd67fc700 1 -- 172.21.15.33:0/3059113599 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd000f6b0 con 0x7f4de006c6d0 2024-05-15T13:57:24.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.790+0000 7f4de7a1a700 1 -- 172.21.15.33:0/3059113599 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4de01a6720 con 0x7f4de006c6d0 2024-05-15T13:57:24.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.791+0000 7f4de7a1a700 1 -- 172.21.15.33:0/3059113599 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4de006a230 con 0x7f4de006c6d0 2024-05-15T13:57:24.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.792+0000 7f4dd67fc700 1 -- 172.21.15.33:0/3059113599 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f4dd0018930 con 0x7f4de006c6d0 2024-05-15T13:57:24.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.793+0000 7f4dd67fc700 1 --2- 172.21.15.33:0/3059113599 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f4dcc0b2880 0x7f4dcc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:24.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.793+0000 7f4de57b6700 1 --2- 172.21.15.33:0/3059113599 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f4dcc0b2880 0x7f4dcc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:24.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.793+0000 7f4dd67fc700 1 -- 172.21.15.33:0/3059113599 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f4dd00e2570 con 0x7f4de006c6d0 2024-05-15T13:57:24.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.793+0000 7f4de57b6700 1 --2- 172.21.15.33:0/3059113599 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f4dcc0b2880 0x7f4dcc0b4d40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc007fd0 tx=0x7f4ddc00c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:24.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:24.796+0000 7f4dd67fc700 1 -- 172.21.15.33:0/3059113599 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4dd00aa470 con 0x7f4de006c6d0 2024-05-15T13:57:24.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:24 smithi052 ceph-mon[36190]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 6 op/s 2024-05-15T13:57:24.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:24 smithi081 ceph-mon[36434]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 6 op/s 2024-05-15T13:57:25.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:25.022+0000 7f4de7a1a700 1 -- 172.21.15.33:0/3059113599 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4de019af40 con 0x7f4de006c6d0 2024-05-15T13:57:25.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:25.024+0000 7f4dd67fc700 1 -- 172.21.15.33:0/3059113599 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22112 (secure 0 0 0) 0x7f4dd00a9d70 con 0x7f4de006c6d0 2024-05-15T13:57:25.027 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:57:25.027 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":80,"fsid":"bca78ca4-12c1-11ef-bc98-c7b262605968","created":"2024-05-15T13:49:10.327613+0000","modified":"2024-05-15T13:57:19.056451+0000","last_up_change":"2024-05-15T13:56:16.097711+0000","last_in_change":"2024-05-15T13:55:56.722143+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-15T13:52:32.970844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-15T13:57:12.324802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-15T13:57:15.898406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b434111-810f-450a-b63d-c97080e9f8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6803","nonce":887409130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6805","nonce":887409130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6809","nonce":887409130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6807","nonce":887409130}]},"public_addr":"172.21.15.33:6803/887409130","cluster_addr":"172.21.15.33:6805/887409130","heartbeat_back_addr":"172.21.15.33:6809/887409130","heartbeat_front_addr":"172.21.15.33:6807/887409130","state":["exists","up"]},{"osd":1,"uuid":"0b578d4e-a841-4b27-bc95-d6c3bae580fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6801","nonce":2794180003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6803","nonce":2794180003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6807","nonce":2794180003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6805","nonce":2794180003}]},"public_addr":"172.21.15.52:6801/2794180003","cluster_addr":"172.21.15.52:6803/2794180003","heartbeat_back_addr":"172.21.15.52:6807/2794180003","heartbeat_front_addr":"172.21.15.52:6805/2794180003","state":["exists","up"]},{"osd":2,"uuid":"0e9c9390-ed33-464b-8f39-bca75941e05e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6801","nonce":315610978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6803","nonce":315610978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6807","nonce":315610978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6805","nonce":315610978}]},"public_addr":"172.21.15.81:6801/315610978","cluster_addr":"172.21.15.81:6803/315610978","heartbeat_back_addr":"172.21.15.81:6807/315610978","heartbeat_front_addr":"172.21.15.81:6805/315610978","state":["exists","up"]},{"osd":3,"uuid":"ecdb24c1-fb15-42a5-9bae-436bbae9512a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6811","nonce":818533501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6813","nonce":818533501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6817","nonce":818533501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6815","nonce":818533501}]},"public_addr":"172.21.15.33:6811/818533501","cluster_addr":"172.21.15.33:6813/818533501","heartbeat_back_addr":"172.21.15.33:6817/818533501","heartbeat_front_addr":"172.21.15.33:6815/818533501","state":["exists","up"]},{"osd":4,"uuid":"706752a0-b17f-4563-a7a9-d5078bf0f3b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6809","nonce":3010133776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6811","nonce":3010133776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6815","nonce":3010133776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6813","nonce":3010133776}]},"public_addr":"172.21.15.52:6809/3010133776","cluster_addr":"172.21.15.52:6811/3010133776","heartbeat_back_addr":"172.21.15.52:6815/3010133776","heartbeat_front_addr":"172.21.15.52:6813/3010133776","state":["exists","up"]},{"osd":5,"uuid":"3d45ab97-f364-4903-974d-8acc961363d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6809","nonce":2132616297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6811","nonce":2132616297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6815","nonce":2132616297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6813","nonce":2132616297}]},"public_addr":"172.21.15.81:6809/2132616297","cluster_addr":"172.21.15.81:6811/2132616297","heartbeat_back_addr":"172.21.15.81:6815/2132616297","heartbeat_front_addr":"172.21.15.81:6813/2132616297","state":["exists","up"]},{"osd":6,"uuid":"3be524c5-dcf9-4742-b896-ea9a44a6b391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6819","nonce":1260691801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6821","nonce":1260691801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6825","nonce":1260691801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6823","nonce":1260691801}]},"public_addr":"172.21.15.33:6819/1260691801","cluster_addr":"172.21.15.33:6821/1260691801","heartbeat_back_addr":"172.21.15.33:6825/1260691801","heartbeat_front_addr":"172.21.15.33:6823/1260691801","state":["exists","up"]},{"osd":7,"uuid":"bda2f920-4f2c-4eed-8127-83c1973676e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6817","nonce":1654595813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6819","nonce":1654595813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6823","nonce":1654595813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6821","nonce":1654595813}]},"public_addr":"172.21.15.52:6817/1654595813","cluster_addr":"172.21.15.52:6819/1654595813","heartbeat_back_addr":"172.21.15.52:6823/1654595813","heartbeat_front_addr":"172.21.15.52:6821/1654595813","state":["exists","up"]},{"osd":8,"uuid":"550585b8-0df4-49ee-95c8-3d084744dbd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6817","nonce":3009661748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6819","nonce":3009661748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6823","nonce":3009661748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6821","nonce":3009661748}]},"public_addr":"172.21.15.81:6817/3009661748","cluster_addr":"172.21.15.81:6819/3009661748","heartbeat_back_addr":"172.21.15.81:6823/3009661748","heartbeat_front_addr":"172.21.15.81:6821/3009661748","state":["exists","up"]},{"osd":9,"uuid":"1d781846-26f3-4b23-b22c-c994a8509a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6827","nonce":2843220858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6829","nonce":2843220858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6833","nonce":2843220858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6831","nonce":2843220858}]},"public_addr":"172.21.15.33:6827/2843220858","cluster_addr":"172.21.15.33:6829/2843220858","heartbeat_back_addr":"172.21.15.33:6833/2843220858","heartbeat_front_addr":"172.21.15.33:6831/2843220858","state":["exists","up"]},{"osd":10,"uuid":"af58c940-fdc4-418b-a638-7dc44438b34c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6825","nonce":2422349831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6827","nonce":2422349831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6831","nonce":2422349831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6829","nonce":2422349831}]},"public_addr":"172.21.15.52:6825/2422349831","cluster_addr":"172.21.15.52:6827/2422349831","heartbeat_back_addr":"172.21.15.52:6831/2422349831","heartbeat_front_addr":"172.21.15.52:6829/2422349831","state":["exists","up"]},{"osd":11,"uuid":"ab84aaea-2086-4842-8bf4-628842fce46a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6825","nonce":326313090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6827","nonce":326313090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6831","nonce":326313090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6829","nonce":326313090}]},"public_addr":"172.21.15.81:6825/326313090","cluster_addr":"172.21.15.81:6827/326313090","heartbeat_back_addr":"172.21.15.81:6831/326313090","heartbeat_front_addr":"172.21.15.81:6829/326313090","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:51:41.493136+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:52:05.784773+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:52:29.769429+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:52:54.891620+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:53:19.412922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:53:43.921841+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:54:09.486732+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:54:34.064370+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:54:58.655304+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:55:24.548693+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:55:49.941191+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:56:14.919558+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/436121571":"2024-05-16T13:50:04.881656+0000","172.21.15.33:6801/2680375465":"2024-05-16T13:50:04.881656+0000","172.21.15.33:6800/2680375465":"2024-05-16T13:50:04.881656+0000","172.21.15.33:0/1022787179":"2024-05-16T13:49:37.564643+0000","172.21.15.33:0/3720711369":"2024-05-16T13:50:04.881656+0000","172.21.15.33:0/1282466838":"2024-05-16T13:49:37.564643+0000","172.21.15.33:0/3022248969":"2024-05-16T13:49:37.564643+0000","172.21.15.33:6801/3303495364":"2024-05-16T13:49:37.564643+0000","172.21.15.33:6800/3303495364":"2024-05-16T13:49:37.564643+0000","172.21.15.33:0/4282812692":"2024-05-16T13:50:04.881656+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-15T13:57:25.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:25.026+0000 7f4de7a1a700 1 -- 172.21.15.33:0/3059113599 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f4dcc0b2880 msgr2=0x7f4dcc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:25.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:25.026+0000 7f4de7a1a700 1 --2- 172.21.15.33:0/3059113599 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f4dcc0b2880 0x7f4dcc0b4d40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc007fd0 tx=0x7f4ddc00c040 comp rx=0 tx=0).stop 2024-05-15T13:57:25.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:25.026+0000 7f4de7a1a700 1 -- 172.21.15.33:0/3059113599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4de006c6d0 msgr2=0x7f4de019fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:25.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:25.026+0000 7f4de7a1a700 1 --2- 172.21.15.33:0/3059113599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4de006c6d0 0x7f4de019fd90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4dd000c970 tx=0x7f4dd000cc80 comp rx=0 tx=0).stop 2024-05-15T13:57:25.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:25.026+0000 7f4de7a1a700 1 -- 172.21.15.33:0/3059113599 shutdown_connections 2024-05-15T13:57:25.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:25.026+0000 7f4de7a1a700 1 --2- 172.21.15.33:0/3059113599 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f4dcc0b2880 0x7f4dcc0b4d40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:25.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:25.026+0000 7f4de7a1a700 1 --2- 172.21.15.33:0/3059113599 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4de0109630 0x7f4de0199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:25.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:25.026+0000 7f4de7a1a700 1 --2- 172.21.15.33:0/3059113599 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4de006bd10 0x7f4de019f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:25.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:25.026+0000 7f4de7a1a700 1 --2- 172.21.15.33:0/3059113599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4de006c6d0 0x7f4de019fd90 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:25.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:25.026+0000 7f4de7a1a700 1 -- 172.21.15.33:0/3059113599 >> 172.21.15.33:0/3059113599 conn(0x7f4de00794e0 msgr2=0x7f4de0101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:25.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:25.027+0000 7f4de7a1a700 1 -- 172.21.15.33:0/3059113599 shutdown_connections 2024-05-15T13:57:25.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:25.027+0000 7f4de7a1a700 1 -- 172.21.15.33:0/3059113599 wait complete. 2024-05-15T13:57:25.681 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:25 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/3059113599' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T13:57:25.714 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-15T13:57:25.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:25 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/3059113599' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T13:57:25.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:25 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/3059113599' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T13:57:25.970 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:57:26.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:26 smithi033 ceph-mon[31650]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-15T13:57:26.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:26 smithi052 ceph-mon[36190]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-15T13:57:26.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:26 smithi081 ceph-mon[36434]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-15T13:57:27.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.643+0000 7fc11afe5700 1 -- 172.21.15.33:0/1746795748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1141028a0 msgr2=0x7fc114110310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:27.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.643+0000 7fc11afe5700 1 --2- 172.21.15.33:0/1746795748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1141028a0 0x7fc114110310 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc10c00b0a0 tx=0x7fc10c00b3b0 comp rx=0 tx=0).stop 2024-05-15T13:57:27.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.643+0000 7fc11afe5700 1 -- 172.21.15.33:0/1746795748 shutdown_connections 2024-05-15T13:57:27.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.643+0000 7fc11afe5700 1 --2- 172.21.15.33:0/1746795748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1141028a0 0x7fc114110310 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:27.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.643+0000 7fc11afe5700 1 --2- 172.21.15.33:0/1746795748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc114108600 0x7fc1141089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:27.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.643+0000 7fc11afe5700 1 --2- 172.21.15.33:0/1746795748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc114101ee0 0x7fc114102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:27.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.643+0000 7fc11afe5700 1 -- 172.21.15.33:0/1746795748 >> 172.21.15.33:0/1746795748 conn(0x7fc114077760 msgr2=0x7fc114077b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:27.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.643+0000 7fc11afe5700 1 -- 172.21.15.33:0/1746795748 shutdown_connections 2024-05-15T13:57:27.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.643+0000 7fc11afe5700 1 -- 172.21.15.33:0/1746795748 wait complete. 2024-05-15T13:57:27.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.644+0000 7fc11afe5700 1 Processor -- start 2024-05-15T13:57:27.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.644+0000 7fc11afe5700 1 -- start start 2024-05-15T13:57:27.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.644+0000 7fc11afe5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc114101ee0 0x7fc11419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:27.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.644+0000 7fc11afe5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1141028a0 0x7fc11419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:27.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.644+0000 7fc11afe5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc114108600 0x7fc1141999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:27.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.644+0000 7fc11afe5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc1141137e0 con 0x7fc114108600 2024-05-15T13:57:27.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.644+0000 7fc11afe5700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc114113660 con 0x7fc1141028a0 2024-05-15T13:57:27.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.644+0000 7fc11afe5700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc114113960 con 0x7fc114101ee0 2024-05-15T13:57:27.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.644+0000 7fc119582700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc114108600 0x7fc1141999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:27.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.644+0000 7fc119582700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc114108600 0x7fc1141999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40670/0 (socket says 172.21.15.33:40670) 2024-05-15T13:57:27.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.644+0000 7fc119582700 1 -- 172.21.15.33:0/1427637531 learned_addr learned my addr 172.21.15.33:0/1427637531 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:57:27.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.644+0000 7fc118d81700 1 --2- 172.21.15.33:0/1427637531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc114101ee0 0x7fc11419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:27.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.644+0000 7fc113fff700 1 --2- 172.21.15.33:0/1427637531 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1141028a0 0x7fc11419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:27.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.645+0000 7fc119582700 1 -- 172.21.15.33:0/1427637531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc114101ee0 msgr2=0x7fc11419f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:27.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.645+0000 7fc119582700 1 --2- 172.21.15.33:0/1427637531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc114101ee0 0x7fc11419f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:27.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.645+0000 7fc119582700 1 -- 172.21.15.33:0/1427637531 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1141028a0 msgr2=0x7fc11419fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:27.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.645+0000 7fc119582700 1 --2- 172.21.15.33:0/1427637531 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1141028a0 0x7fc11419fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:27.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.645+0000 7fc119582700 1 -- 172.21.15.33:0/1427637531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11419a120 con 0x7fc114108600 2024-05-15T13:57:27.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.645+0000 7fc119582700 1 --2- 172.21.15.33:0/1427637531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc114108600 0x7fc1141999c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc10c009fd0 tx=0x7fc10c004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:27.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.645+0000 7fc111ffb700 1 -- 172.21.15.33:0/1427637531 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c021070 con 0x7fc114108600 2024-05-15T13:57:27.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.645+0000 7fc111ffb700 1 -- 172.21.15.33:0/1427637531 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc10c007c00 con 0x7fc114108600 2024-05-15T13:57:27.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.645+0000 7fc11afe5700 1 -- 172.21.15.33:0/1427637531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc11419a3b0 con 0x7fc114108600 2024-05-15T13:57:27.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.645+0000 7fc111ffb700 1 -- 172.21.15.33:0/1427637531 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10c0052a0 con 0x7fc114108600 2024-05-15T13:57:27.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.645+0000 7fc11afe5700 1 -- 172.21.15.33:0/1427637531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1141a66d0 con 0x7fc114108600 2024-05-15T13:57:27.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.647+0000 7fc11afe5700 1 -- 172.21.15.33:0/1427637531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc11406a230 con 0x7fc114108600 2024-05-15T13:57:27.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.648+0000 7fc111ffb700 1 -- 172.21.15.33:0/1427637531 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc10c024070 con 0x7fc114108600 2024-05-15T13:57:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.649+0000 7fc111ffb700 1 --2- 172.21.15.33:0/1427637531 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fc0fc0b2900 0x7fc0fc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.649+0000 7fc118d81700 1 --2- 172.21.15.33:0/1427637531 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fc0fc0b2900 0x7fc0fc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.649+0000 7fc111ffb700 1 -- 172.21.15.33:0/1427637531 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fc10c0e3180 con 0x7fc114108600 2024-05-15T13:57:27.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.649+0000 7fc118d81700 1 --2- 172.21.15.33:0/1427637531 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fc0fc0b2900 0x7fc0fc0b4dc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc11419ab00 tx=0x7fc10400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:27.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.653+0000 7fc111ffb700 1 -- 172.21.15.33:0/1427637531 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc10c0ab080 con 0x7fc114108600 2024-05-15T13:57:27.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:27.915+0000 7fc11afe5700 1 -- 172.21.15.33:0/1427637531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc11402e560 con 0x7fc114108600 2024-05-15T13:57:28.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:28.451+0000 7fc111ffb700 1 -- 172.21.15.33:0/1427637531 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7fc10c009cb0 con 0x7fc114108600 2024-05-15T13:57:28.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:28.454+0000 7fc11afe5700 1 -- 172.21.15.33:0/1427637531 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fc0fc0b2900 msgr2=0x7fc0fc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:28.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:28.454+0000 7fc11afe5700 1 --2- 172.21.15.33:0/1427637531 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fc0fc0b2900 0x7fc0fc0b4dc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc11419ab00 tx=0x7fc10400b410 comp rx=0 tx=0).stop 2024-05-15T13:57:28.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:28.454+0000 7fc11afe5700 1 -- 172.21.15.33:0/1427637531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc114108600 msgr2=0x7fc1141999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:28.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:28.454+0000 7fc11afe5700 1 --2- 172.21.15.33:0/1427637531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc114108600 0x7fc1141999c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc10c009fd0 tx=0x7fc10c004dc0 comp rx=0 tx=0).stop 2024-05-15T13:57:28.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:28.454+0000 7fc11afe5700 1 -- 172.21.15.33:0/1427637531 shutdown_connections 2024-05-15T13:57:28.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:28.454+0000 7fc11afe5700 1 --2- 172.21.15.33:0/1427637531 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fc0fc0b2900 0x7fc0fc0b4dc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:28.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:28.454+0000 7fc11afe5700 1 --2- 172.21.15.33:0/1427637531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc114108600 0x7fc1141999c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:28.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:28.454+0000 7fc11afe5700 1 --2- 172.21.15.33:0/1427637531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc114101ee0 0x7fc11419f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:28.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:28.454+0000 7fc11afe5700 1 --2- 172.21.15.33:0/1427637531 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1141028a0 0x7fc11419fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:28.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:28.454+0000 7fc11afe5700 1 -- 172.21.15.33:0/1427637531 >> 172.21.15.33:0/1427637531 conn(0x7fc114077760 msgr2=0x7fc114100b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:28.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:28.454+0000 7fc11afe5700 1 -- 172.21.15.33:0/1427637531 shutdown_connections 2024-05-15T13:57:28.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:28.454+0000 7fc11afe5700 1 -- 172.21.15.33:0/1427637531 wait complete. 2024-05-15T13:57:28.762 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:28 smithi033 ceph-mon[31650]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-15T13:57:28.762 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:28 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/1427637531' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-15T13:57:28.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:28 smithi052 ceph-mon[36190]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-15T13:57:28.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:28 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/1427637531' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-15T13:57:28.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:28 smithi081 ceph-mon[36434]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-15T13:57:28.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:28 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/1427637531' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-15T13:57:29.107 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-05-15T13:57:29.107 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-05-15T13:57:29.363 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:57:29.757 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:29 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/1427637531' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-15T13:57:29.757 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:29 smithi033 ceph-mon[31650]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-15T13:57:29.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:29 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/1427637531' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-15T13:57:29.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:29 smithi052 ceph-mon[36190]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-15T13:57:29.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:29 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/1427637531' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-15T13:57:29.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:29 smithi081 ceph-mon[36434]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-15T13:57:30.720 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:30 smithi033 ceph-mon[31650]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-15T13:57:30.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:30 smithi052 ceph-mon[36190]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-15T13:57:30.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:30 smithi081 ceph-mon[36434]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-15T13:57:31.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.080+0000 7f0779930700 1 -- 172.21.15.33:0/481602183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07741095b0 msgr2=0x7f0774109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:31.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.080+0000 7f0779930700 1 --2- 172.21.15.33:0/481602183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07741095b0 0x7f0774109990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f076400b0a0 tx=0x7f076400b3b0 comp rx=0 tx=0).stop 2024-05-15T13:57:31.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.080+0000 7f0779930700 1 -- 172.21.15.33:0/481602183 shutdown_connections 2024-05-15T13:57:31.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.080+0000 7f0779930700 1 --2- 172.21.15.33:0/481602183 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f077406bbe0 0x7f07741103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:31.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.080+0000 7f0779930700 1 --2- 172.21.15.33:0/481602183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07741095b0 0x7f0774109990 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:31.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.080+0000 7f0779930700 1 --2- 172.21.15.33:0/481602183 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f077406b220 0x7f077406b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:31.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.080+0000 7f0779930700 1 -- 172.21.15.33:0/481602183 >> 172.21.15.33:0/481602183 conn(0x7f07740793f0 msgr2=0x7f0774079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:31.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.080+0000 7f0779930700 1 -- 172.21.15.33:0/481602183 shutdown_connections 2024-05-15T13:57:31.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.080+0000 7f0779930700 1 -- 172.21.15.33:0/481602183 wait complete. 2024-05-15T13:57:31.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.081+0000 7f0779930700 1 Processor -- start 2024-05-15T13:57:31.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.081+0000 7f0779930700 1 -- start start 2024-05-15T13:57:31.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.081+0000 7f0779930700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f077406b220 0x7f077419f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:31.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.081+0000 7f0779930700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f077406bbe0 0x7f077419fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:31.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.081+0000 7f0779930700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07741095b0 0x7f0774199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:31.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.081+0000 7f0779930700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0774113740 con 0x7f07741095b0 2024-05-15T13:57:31.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.081+0000 7f0779930700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f07741135c0 con 0x7f077406b220 2024-05-15T13:57:31.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.081+0000 7f0779930700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f07741138c0 con 0x7f077406bbe0 2024-05-15T13:57:31.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.082+0000 7f07737fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07741095b0 0x7f0774199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:31.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.082+0000 7f07737fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07741095b0 0x7f0774199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40692/0 (socket says 172.21.15.33:40692) 2024-05-15T13:57:31.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.082+0000 7f07737fe700 1 -- 172.21.15.33:0/690344110 learned_addr learned my addr 172.21.15.33:0/690344110 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:57:31.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.082+0000 7f0772ffd700 1 --2- 172.21.15.33:0/690344110 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f077406b220 0x7f077419f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:31.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.082+0000 7f07727fc700 1 --2- 172.21.15.33:0/690344110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f077406bbe0 0x7f077419fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:31.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.082+0000 7f07737fe700 1 -- 172.21.15.33:0/690344110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f077406bbe0 msgr2=0x7f077419fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:31.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.082+0000 7f07737fe700 1 --2- 172.21.15.33:0/690344110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f077406bbe0 0x7f077419fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:31.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.082+0000 7f07737fe700 1 -- 172.21.15.33:0/690344110 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f077406b220 msgr2=0x7f077419f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:31.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.082+0000 7f07737fe700 1 --2- 172.21.15.33:0/690344110 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f077406b220 0x7f077419f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:31.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.082+0000 7f07737fe700 1 -- 172.21.15.33:0/690344110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f077419a200 con 0x7f07741095b0 2024-05-15T13:57:31.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.082+0000 7f07737fe700 1 --2- 172.21.15.33:0/690344110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07741095b0 0x7f0774199aa0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f076c00c900 tx=0x7f076c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:31.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.082+0000 7f077892e700 1 -- 172.21.15.33:0/690344110 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c007b00 con 0x7f07741095b0 2024-05-15T13:57:31.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.082+0000 7f077892e700 1 -- 172.21.15.33:0/690344110 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f076c00ce50 con 0x7f07741095b0 2024-05-15T13:57:31.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.082+0000 7f0779930700 1 -- 172.21.15.33:0/690344110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f077419a4f0 con 0x7f07741095b0 2024-05-15T13:57:31.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.082+0000 7f077892e700 1 -- 172.21.15.33:0/690344110 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c018660 con 0x7f07741095b0 2024-05-15T13:57:31.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.082+0000 7f0779930700 1 -- 172.21.15.33:0/690344110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07741a6760 con 0x7f07741095b0 2024-05-15T13:57:31.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.083+0000 7f0779930700 1 -- 172.21.15.33:0/690344110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f077410ba90 con 0x7f07741095b0 2024-05-15T13:57:31.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.085+0000 7f077892e700 1 -- 172.21.15.33:0/690344110 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f076c00f450 con 0x7f07741095b0 2024-05-15T13:57:31.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.085+0000 7f077892e700 1 --2- 172.21.15.33:0/690344110 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f07600b2900 0x7f07600b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:31.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.086+0000 7f0772ffd700 1 --2- 172.21.15.33:0/690344110 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f07600b2900 0x7f07600b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:31.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.086+0000 7f077892e700 1 -- 172.21.15.33:0/690344110 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f076c0e1fa0 con 0x7f07741095b0 2024-05-15T13:57:31.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.086+0000 7f0772ffd700 1 --2- 172.21.15.33:0/690344110 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f07600b2900 0x7f07600b4dc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0764009fd0 tx=0x7f076400b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:31.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.089+0000 7f077892e700 1 -- 172.21.15.33:0/690344110 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f076c014030 con 0x7f07741095b0 2024-05-15T13:57:31.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.352+0000 7f0779930700 1 -- 172.21.15.33:0/690344110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f077419af80 con 0x7f07741095b0 2024-05-15T13:57:31.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.466+0000 7f077892e700 1 -- 172.21.15.33:0/690344110 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v17) ==== 105+0+0 (secure 0 0 0) 0x7f076c0a9f20 con 0x7f07741095b0 2024-05-15T13:57:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.468+0000 7f0779930700 1 -- 172.21.15.33:0/690344110 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f07600b2900 msgr2=0x7f07600b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.468+0000 7f0779930700 1 --2- 172.21.15.33:0/690344110 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f07600b2900 0x7f07600b4dc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0764009fd0 tx=0x7f076400b7b0 comp rx=0 tx=0).stop 2024-05-15T13:57:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.468+0000 7f0779930700 1 -- 172.21.15.33:0/690344110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07741095b0 msgr2=0x7f0774199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.468+0000 7f0779930700 1 --2- 172.21.15.33:0/690344110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07741095b0 0x7f0774199aa0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f076c00c900 tx=0x7f076c00cc10 comp rx=0 tx=0).stop 2024-05-15T13:57:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.468+0000 7f0779930700 1 -- 172.21.15.33:0/690344110 shutdown_connections 2024-05-15T13:57:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.468+0000 7f0779930700 1 --2- 172.21.15.33:0/690344110 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f07600b2900 0x7f07600b4dc0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.468+0000 7f0779930700 1 --2- 172.21.15.33:0/690344110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07741095b0 0x7f0774199aa0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.468+0000 7f0779930700 1 --2- 172.21.15.33:0/690344110 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f077406b220 0x7f077419f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.468+0000 7f0779930700 1 --2- 172.21.15.33:0/690344110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f077406bbe0 0x7f077419fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.468+0000 7f0779930700 1 -- 172.21.15.33:0/690344110 >> 172.21.15.33:0/690344110 conn(0x7f07740793f0 msgr2=0x7f07741010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.468+0000 7f0779930700 1 -- 172.21.15.33:0/690344110 shutdown_connections 2024-05-15T13:57:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:31.468+0000 7f0779930700 1 -- 172.21.15.33:0/690344110 wait complete. 2024-05-15T13:57:31.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:31 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/690344110' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-15T13:57:31.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:31 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/690344110' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-15T13:57:31.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:31 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/690344110' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-15T13:57:32.363 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-15T13:57:32.624 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:57:32.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:32 smithi033 ceph-mon[31650]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-15T13:57:32.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:32 smithi033 ceph-mon[31650]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-15T13:57:32.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:32 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/690344110' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-15T13:57:32.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:32 smithi033 ceph-mon[31650]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-15T13:57:32.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:32 smithi033 ceph-mon[31650]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-15T13:57:32.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:32 smithi033 ceph-mon[31650]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-05-15T13:57:32.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:32 smithi033 ceph-mon[31650]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-05-15T13:57:32.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:32 smithi052 ceph-mon[36190]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-15T13:57:32.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:32 smithi052 ceph-mon[36190]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-15T13:57:32.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:32 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/690344110' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-15T13:57:32.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:32 smithi052 ceph-mon[36190]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-15T13:57:32.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:32 smithi052 ceph-mon[36190]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-15T13:57:32.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:32 smithi052 ceph-mon[36190]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-05-15T13:57:32.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:32 smithi052 ceph-mon[36190]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-05-15T13:57:32.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:32 smithi081 ceph-mon[36434]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-15T13:57:32.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:32 smithi081 ceph-mon[36434]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-15T13:57:32.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:32 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/690344110' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-15T13:57:32.859 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:32 smithi081 ceph-mon[36434]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-15T13:57:32.859 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:32 smithi081 ceph-mon[36434]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-15T13:57:32.859 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:32 smithi081 ceph-mon[36434]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-05-15T13:57:32.859 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:32 smithi081 ceph-mon[36434]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-05-15T13:57:33.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:33 smithi033 ceph-mon[31650]: mds.? [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] up:active 2024-05-15T13:57:33.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:33 smithi033 ceph-mon[31650]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-15T13:57:33.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:33 smithi033 ceph-mon[31650]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-15T13:57:33.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:33 smithi033 ceph-mon[31650]: Cluster is now healthy 2024-05-15T13:57:33.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:33 smithi033 ceph-mon[31650]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-05-15T13:57:33.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:33 smithi033 ceph-mon[31650]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:creating} 9 up:standby 2024-05-15T13:57:33.755 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:33 smithi033 ceph-mon[31650]: daemon mds.b is now active in filesystem cephfs as rank 2 2024-05-15T13:57:33.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:33 smithi052 ceph-mon[36190]: mds.? [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] up:active 2024-05-15T13:57:33.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:33 smithi052 ceph-mon[36190]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-15T13:57:33.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:33 smithi052 ceph-mon[36190]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-15T13:57:33.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:33 smithi052 ceph-mon[36190]: Cluster is now healthy 2024-05-15T13:57:33.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:33 smithi052 ceph-mon[36190]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-05-15T13:57:33.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:33 smithi052 ceph-mon[36190]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:creating} 9 up:standby 2024-05-15T13:57:33.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:33 smithi052 ceph-mon[36190]: daemon mds.b is now active in filesystem cephfs as rank 2 2024-05-15T13:57:33.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:33 smithi081 ceph-mon[36434]: mds.? [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] up:active 2024-05-15T13:57:33.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:33 smithi081 ceph-mon[36434]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-15T13:57:33.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:33 smithi081 ceph-mon[36434]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-15T13:57:33.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:33 smithi081 ceph-mon[36434]: Cluster is now healthy 2024-05-15T13:57:33.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:33 smithi081 ceph-mon[36434]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-05-15T13:57:33.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:33 smithi081 ceph-mon[36434]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:creating} 9 up:standby 2024-05-15T13:57:33.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:33 smithi081 ceph-mon[36434]: daemon mds.b is now active in filesystem cephfs as rank 2 2024-05-15T13:57:34.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.289+0000 7f6ed0c53700 1 -- 172.21.15.33:0/2345987012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ecc1088f0 msgr2=0x7f6ecc110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:34.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.289+0000 7f6ed0c53700 1 --2- 172.21.15.33:0/2345987012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ecc1088f0 0x7f6ecc110470 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6ec400b0d0 tx=0x7f6ec400b3e0 comp rx=0 tx=0).stop 2024-05-15T13:57:34.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.290+0000 7f6ed0c53700 1 -- 172.21.15.33:0/2345987012 shutdown_connections 2024-05-15T13:57:34.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.290+0000 7f6ed0c53700 1 --2- 172.21.15.33:0/2345987012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ecc1088f0 0x7f6ecc110470 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:34.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.290+0000 7f6ed0c53700 1 --2- 172.21.15.33:0/2345987012 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ecc1037f0 0x7f6ecc103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:34.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.290+0000 7f6ed0c53700 1 --2- 172.21.15.33:0/2345987012 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ecc1041a0 0x7f6ecc1081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:34.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.290+0000 7f6ed0c53700 1 -- 172.21.15.33:0/2345987012 >> 172.21.15.33:0/2345987012 conn(0x7f6ecc07a6a0 msgr2=0x7f6ecc07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:34.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.290+0000 7f6ed0c53700 1 -- 172.21.15.33:0/2345987012 shutdown_connections 2024-05-15T13:57:34.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.290+0000 7f6ed0c53700 1 -- 172.21.15.33:0/2345987012 wait complete. 2024-05-15T13:57:34.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.290+0000 7f6ed0c53700 1 Processor -- start 2024-05-15T13:57:34.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.291+0000 7f6ed0c53700 1 -- start start 2024-05-15T13:57:34.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.291+0000 7f6ed0c53700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ecc1037f0 0x7f6ecc19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:34.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.291+0000 7f6ed0c53700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ecc1041a0 0x7f6ecc19fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:34.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.291+0000 7f6ed0c53700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ecc1088f0 0x7f6ecc199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:34.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.291+0000 7f6ed0c53700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc113750 con 0x7f6ecc1037f0 2024-05-15T13:57:34.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.291+0000 7f6ed0c53700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc1135d0 con 0x7f6ecc1041a0 2024-05-15T13:57:34.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.291+0000 7f6ed0c53700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc1138d0 con 0x7f6ecc1088f0 2024-05-15T13:57:34.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.291+0000 7f6eca59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ecc1037f0 0x7f6ecc19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:34.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.291+0000 7f6eca59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ecc1037f0 0x7f6ecc19f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48510/0 (socket says 172.21.15.33:48510) 2024-05-15T13:57:34.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.291+0000 7f6eca59c700 1 -- 172.21.15.33:0/692638667 learned_addr learned my addr 172.21.15.33:0/692638667 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:57:34.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.291+0000 7f6ec9d9b700 1 --2- 172.21.15.33:0/692638667 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ecc1041a0 0x7f6ecc19fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:34.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.291+0000 7f6ecad9d700 1 --2- 172.21.15.33:0/692638667 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ecc1088f0 0x7f6ecc199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:34.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.291+0000 7f6eca59c700 1 -- 172.21.15.33:0/692638667 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ecc1088f0 msgr2=0x7f6ecc199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:34.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.291+0000 7f6eca59c700 1 --2- 172.21.15.33:0/692638667 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ecc1088f0 0x7f6ecc199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:34.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.291+0000 7f6eca59c700 1 -- 172.21.15.33:0/692638667 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ecc1041a0 msgr2=0x7f6ecc19fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:34.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.291+0000 7f6eca59c700 1 --2- 172.21.15.33:0/692638667 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ecc1041a0 0x7f6ecc19fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:34.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.291+0000 7f6eca59c700 1 -- 172.21.15.33:0/692638667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ecc19a0f0 con 0x7f6ecc1037f0 2024-05-15T13:57:34.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.291+0000 7f6ecad9d700 1 --2- 172.21.15.33:0/692638667 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ecc1088f0 0x7f6ecc199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T13:57:34.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.291+0000 7f6eca59c700 1 --2- 172.21.15.33:0/692638667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ecc1037f0 0x7f6ecc19f7d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc00d980 tx=0x7f6ebc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:34.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.292+0000 7f6ebb7fe700 1 -- 172.21.15.33:0/692638667 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc017070 con 0x7f6ecc1037f0 2024-05-15T13:57:34.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.292+0000 7f6ed0c53700 1 -- 172.21.15.33:0/692638667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ecc19a3e0 con 0x7f6ecc1037f0 2024-05-15T13:57:34.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.292+0000 7f6ebb7fe700 1 -- 172.21.15.33:0/692638667 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ebc005020 con 0x7f6ecc1037f0 2024-05-15T13:57:34.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.292+0000 7f6ebb7fe700 1 -- 172.21.15.33:0/692638667 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc009b50 con 0x7f6ecc1037f0 2024-05-15T13:57:34.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.292+0000 7f6ed0c53700 1 -- 172.21.15.33:0/692638667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ecc1a66a0 con 0x7f6ecc1037f0 2024-05-15T13:57:34.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.293+0000 7f6ed0c53700 1 -- 172.21.15.33:0/692638667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ecc06a230 con 0x7f6ecc1037f0 2024-05-15T13:57:34.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.295+0000 7f6ebb7fe700 1 -- 172.21.15.33:0/692638667 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f6ebc0051d0 con 0x7f6ecc1037f0 2024-05-15T13:57:34.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.296+0000 7f6ebb7fe700 1 --2- 172.21.15.33:0/692638667 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f6eb40b2880 0x7f6eb40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:34.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.296+0000 7f6ec9d9b700 1 --2- 172.21.15.33:0/692638667 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f6eb40b2880 0x7f6eb40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:34.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.296+0000 7f6ebb7fe700 1 -- 172.21.15.33:0/692638667 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f6ebc0e2450 con 0x7f6ecc1037f0 2024-05-15T13:57:34.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.296+0000 7f6ec9d9b700 1 --2- 172.21.15.33:0/692638667 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f6eb40b2880 0x7f6eb40b4d40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc103130 tx=0x7f6ec0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:34.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.299+0000 7f6ebb7fe700 1 -- 172.21.15.33:0/692638667 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6ebc020020 con 0x7f6ecc1037f0 2024-05-15T13:57:34.357 INFO:journalctl@ceph.mds.i.smithi081.stdout:May 15 13:57:34 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-i[66884]: 2024-05-15T13:57:34.079+0000 7fd51b287700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-15T13:57:34.357 INFO:journalctl@ceph.mds.i.smithi081.stdout:May 15 13:57:34 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-i[66884]: 2024-05-15T13:57:34.079+0000 7fd51b287700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-15T13:57:34.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:34.561+0000 7f6ed0c53700 1 -- 172.21.15.33:0/692638667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f6ecc02e560 con 0x7f6ecc1037f0 2024-05-15T13:57:34.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:34 smithi033 ceph-mon[31650]: pgmap v294: 97 pgs: 97 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 4 op/s 2024-05-15T13:57:34.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:34 smithi033 ceph-mon[31650]: mds.? [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] up:active 2024-05-15T13:57:34.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:34 smithi033 ceph-mon[31650]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 9 up:standby 2024-05-15T13:57:34.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:34 smithi052 ceph-mon[36190]: pgmap v294: 97 pgs: 97 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 4 op/s 2024-05-15T13:57:34.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:34 smithi052 ceph-mon[36190]: mds.? [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] up:active 2024-05-15T13:57:34.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:34 smithi052 ceph-mon[36190]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 9 up:standby 2024-05-15T13:57:34.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:34 smithi081 ceph-mon[36434]: pgmap v294: 97 pgs: 97 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 4 op/s 2024-05-15T13:57:34.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:34 smithi081 ceph-mon[36434]: mds.? [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] up:active 2024-05-15T13:57:34.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:34 smithi081 ceph-mon[36434]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 9 up:standby 2024-05-15T13:57:35.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:35.361+0000 7f6ebb7fe700 1 -- 172.21.15.33:0/692638667 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f6ebc0aa2c0 con 0x7f6ecc1037f0 2024-05-15T13:57:35.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:35.364+0000 7f6ed0c53700 1 -- 172.21.15.33:0/692638667 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f6eb40b2880 msgr2=0x7f6eb40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:35.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:35.364+0000 7f6ed0c53700 1 --2- 172.21.15.33:0/692638667 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f6eb40b2880 0x7f6eb40b4d40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc103130 tx=0x7f6ec0009190 comp rx=0 tx=0).stop 2024-05-15T13:57:35.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:35.364+0000 7f6ed0c53700 1 -- 172.21.15.33:0/692638667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ecc1037f0 msgr2=0x7f6ecc19f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:35.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:35.364+0000 7f6ed0c53700 1 --2- 172.21.15.33:0/692638667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ecc1037f0 0x7f6ecc19f7d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc00d980 tx=0x7f6ebc00dc90 comp rx=0 tx=0).stop 2024-05-15T13:57:35.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:35.364+0000 7f6ed0c53700 1 -- 172.21.15.33:0/692638667 shutdown_connections 2024-05-15T13:57:35.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:35.364+0000 7f6ed0c53700 1 --2- 172.21.15.33:0/692638667 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f6eb40b2880 0x7f6eb40b4d40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:35.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:35.364+0000 7f6ed0c53700 1 --2- 172.21.15.33:0/692638667 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ecc1088f0 0x7f6ecc199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:35.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:35.364+0000 7f6ed0c53700 1 --2- 172.21.15.33:0/692638667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ecc1037f0 0x7f6ecc19f7d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:35.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:35.364+0000 7f6ed0c53700 1 --2- 172.21.15.33:0/692638667 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ecc1041a0 0x7f6ecc19fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:35.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:35.365+0000 7f6ed0c53700 1 -- 172.21.15.33:0/692638667 >> 172.21.15.33:0/692638667 conn(0x7f6ecc07a6a0 msgr2=0x7f6ecc101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:35.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:35.365+0000 7f6ed0c53700 1 -- 172.21.15.33:0/692638667 shutdown_connections 2024-05-15T13:57:35.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:35.365+0000 7f6ed0c53700 1 -- 172.21.15.33:0/692638667 wait complete. 2024-05-15T13:57:35.688 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:35 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/692638667' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-15T13:57:35.688 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:35 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/692638667' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-15T13:57:35.688 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:35 smithi033 ceph-mon[31650]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T13:57:35.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:35 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/692638667' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-15T13:57:35.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:35 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/692638667' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-15T13:57:35.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:35 smithi052 ceph-mon[36190]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T13:57:35.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:35 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/692638667' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-15T13:57:35.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:35 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/692638667' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-15T13:57:35.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:35 smithi081 ceph-mon[36434]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T13:57:36.116 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-15T13:57:36.373 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:57:36.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:36 smithi033 ceph-mon[31650]: pgmap v295: 97 pgs: 97 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-05-15T13:57:36.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:36 smithi052 ceph-mon[36190]: pgmap v295: 97 pgs: 97 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-05-15T13:57:36.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:36 smithi081 ceph-mon[36434]: pgmap v295: 97 pgs: 97 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-05-15T13:57:37.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.924+0000 7fa633af9700 1 -- 172.21.15.33:0/2796822090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa62c108520 msgr2=0x7fa62c108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:37.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.924+0000 7fa633af9700 1 --2- 172.21.15.33:0/2796822090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa62c108520 0x7fa62c108900 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa62800b0a0 tx=0x7fa62800b3b0 comp rx=0 tx=0).stop 2024-05-15T13:57:37.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.924+0000 7fa633af9700 1 -- 172.21.15.33:0/2796822090 shutdown_connections 2024-05-15T13:57:37.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.924+0000 7fa633af9700 1 --2- 172.21.15.33:0/2796822090 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa62c102e00 0x7fa62c10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:37.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.924+0000 7fa633af9700 1 --2- 172.21.15.33:0/2796822090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa62c108520 0x7fa62c108900 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:37.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.924+0000 7fa633af9700 1 --2- 172.21.15.33:0/2796822090 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa62c102440 0x7fa62c1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:37.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.924+0000 7fa633af9700 1 -- 172.21.15.33:0/2796822090 >> 172.21.15.33:0/2796822090 conn(0x7fa62c0793f0 msgr2=0x7fa62c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:37.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.924+0000 7fa633af9700 1 -- 172.21.15.33:0/2796822090 shutdown_connections 2024-05-15T13:57:37.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.924+0000 7fa633af9700 1 -- 172.21.15.33:0/2796822090 wait complete. 2024-05-15T13:57:37.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.925+0000 7fa633af9700 1 Processor -- start 2024-05-15T13:57:37.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.925+0000 7fa633af9700 1 -- start start 2024-05-15T13:57:37.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.925+0000 7fa633af9700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa62c102440 0x7fa62c19f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:37.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.925+0000 7fa633af9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa62c102e00 0x7fa62c19fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:37.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.925+0000 7fa633af9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa62c108520 0x7fa62c199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:37.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.925+0000 7fa633af9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c1125d0 con 0x7fa62c108520 2024-05-15T13:57:37.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.925+0000 7fa633af9700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c112450 con 0x7fa62c102e00 2024-05-15T13:57:37.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.925+0000 7fa633af9700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c112750 con 0x7fa62c102440 2024-05-15T13:57:37.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.925+0000 7fa632096700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa62c108520 0x7fa62c199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:37.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.925+0000 7fa631094700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa62c102e00 0x7fa62c19fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:37.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.925+0000 7fa631094700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa62c102e00 0x7fa62c19fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:36842/0 (socket says 172.21.15.33:36842) 2024-05-15T13:57:37.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.925+0000 7fa631094700 1 -- 172.21.15.33:0/268387512 learned_addr learned my addr 172.21.15.33:0/268387512 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:57:37.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.926+0000 7fa631895700 1 --2- 172.21.15.33:0/268387512 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa62c102440 0x7fa62c19f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:37.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.926+0000 7fa632096700 1 -- 172.21.15.33:0/268387512 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa62c102440 msgr2=0x7fa62c19f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:37.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.926+0000 7fa632096700 1 --2- 172.21.15.33:0/268387512 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa62c102440 0x7fa62c19f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:37.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.926+0000 7fa632096700 1 -- 172.21.15.33:0/268387512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa62c102e00 msgr2=0x7fa62c19fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:37.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.926+0000 7fa632096700 1 --2- 172.21.15.33:0/268387512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa62c102e00 0x7fa62c19fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:37.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.926+0000 7fa632096700 1 -- 172.21.15.33:0/268387512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa62c19a1a0 con 0x7fa62c108520 2024-05-15T13:57:37.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.926+0000 7fa632096700 1 --2- 172.21.15.33:0/268387512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa62c108520 0x7fa62c199a40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa62400d720 tx=0x7fa62400da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:37.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.926+0000 7fa622ffd700 1 -- 172.21.15.33:0/268387512 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa624016070 con 0x7fa62c108520 2024-05-15T13:57:37.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.926+0000 7fa622ffd700 1 -- 172.21.15.33:0/268387512 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa62400f690 con 0x7fa62c108520 2024-05-15T13:57:37.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.926+0000 7fa633af9700 1 -- 172.21.15.33:0/268387512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa62c19a490 con 0x7fa62c108520 2024-05-15T13:57:37.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.927+0000 7fa622ffd700 1 -- 172.21.15.33:0/268387512 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6240039e0 con 0x7fa62c108520 2024-05-15T13:57:37.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.927+0000 7fa633af9700 1 -- 172.21.15.33:0/268387512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa62c1a6750 con 0x7fa62c108520 2024-05-15T13:57:37.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.928+0000 7fa633af9700 1 -- 172.21.15.33:0/268387512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa62c06a230 con 0x7fa62c108520 2024-05-15T13:57:37.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.929+0000 7fa622ffd700 1 -- 172.21.15.33:0/268387512 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fa624003c60 con 0x7fa62c108520 2024-05-15T13:57:37.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.930+0000 7fa622ffd700 1 --2- 172.21.15.33:0/268387512 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fa6180b2880 0x7fa6180b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:37.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.930+0000 7fa631895700 1 --2- 172.21.15.33:0/268387512 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fa6180b2880 0x7fa6180b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:37.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.930+0000 7fa622ffd700 1 -- 172.21.15.33:0/268387512 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fa6240e1720 con 0x7fa62c108520 2024-05-15T13:57:37.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.930+0000 7fa631895700 1 --2- 172.21.15.33:0/268387512 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fa6180b2880 0x7fa6180b4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa628009e60 tx=0x7fa62800b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:37.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:37.934+0000 7fa622ffd700 1 -- 172.21.15.33:0/268387512 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa62401f070 con 0x7fa62c108520 2024-05-15T13:57:38.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:38.199+0000 7fa633af9700 1 -- 172.21.15.33:0/268387512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fa62c1a6980 con 0x7fa62c108520 2024-05-15T13:57:38.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:38.501+0000 7fa622ffd700 1 -- 172.21.15.33:0/268387512 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) ==== 115+0+0 (secure 0 0 0) 0x7fa6240a9590 con 0x7fa62c108520 2024-05-15T13:57:38.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:38.503+0000 7fa633af9700 1 -- 172.21.15.33:0/268387512 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fa6180b2880 msgr2=0x7fa6180b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:38.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:38.503+0000 7fa633af9700 1 --2- 172.21.15.33:0/268387512 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fa6180b2880 0x7fa6180b4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa628009e60 tx=0x7fa62800b7b0 comp rx=0 tx=0).stop 2024-05-15T13:57:38.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:38.503+0000 7fa633af9700 1 -- 172.21.15.33:0/268387512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa62c108520 msgr2=0x7fa62c199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:38.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:38.503+0000 7fa633af9700 1 --2- 172.21.15.33:0/268387512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa62c108520 0x7fa62c199a40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa62400d720 tx=0x7fa62400da30 comp rx=0 tx=0).stop 2024-05-15T13:57:38.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:38.503+0000 7fa633af9700 1 -- 172.21.15.33:0/268387512 shutdown_connections 2024-05-15T13:57:38.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:38.503+0000 7fa633af9700 1 --2- 172.21.15.33:0/268387512 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fa6180b2880 0x7fa6180b4d40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:38.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:38.503+0000 7fa633af9700 1 --2- 172.21.15.33:0/268387512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa62c108520 0x7fa62c199a40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:38.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:38.503+0000 7fa633af9700 1 --2- 172.21.15.33:0/268387512 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa62c102440 0x7fa62c19f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:38.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:38.503+0000 7fa633af9700 1 --2- 172.21.15.33:0/268387512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa62c102e00 0x7fa62c19fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:38.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:38.503+0000 7fa633af9700 1 -- 172.21.15.33:0/268387512 >> 172.21.15.33:0/268387512 conn(0x7fa62c0793f0 msgr2=0x7fa62c06b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:38.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:38.503+0000 7fa633af9700 1 -- 172.21.15.33:0/268387512 shutdown_connections 2024-05-15T13:57:38.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:38.503+0000 7fa633af9700 1 -- 172.21.15.33:0/268387512 wait complete. 2024-05-15T13:57:38.836 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:38 smithi033 ceph-mon[31650]: pgmap v296: 97 pgs: 97 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1.8 KiB/s wr, 7 op/s 2024-05-15T13:57:38.836 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:38 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/268387512' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-15T13:57:38.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:38 smithi052 ceph-mon[36190]: pgmap v296: 97 pgs: 97 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1.8 KiB/s wr, 7 op/s 2024-05-15T13:57:38.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:38 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/268387512' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-15T13:57:38.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:38 smithi081 ceph-mon[36434]: pgmap v296: 97 pgs: 97 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1.8 KiB/s wr, 7 op/s 2024-05-15T13:57:38.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:38 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/268387512' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-15T13:57:39.181 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-05-15T13:57:39.182 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-15T13:57:39.434 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:57:39.782 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:39 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/268387512' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-15T13:57:39.782 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:39 smithi033 ceph-mon[31650]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T13:57:39.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:39 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/268387512' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-15T13:57:39.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:39 smithi052 ceph-mon[36190]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T13:57:39.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:39 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/268387512' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-15T13:57:39.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:39 smithi081 ceph-mon[36434]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T13:57:40.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:40 smithi052 ceph-mon[36190]: pgmap v297: 97 pgs: 97 active+clean; 582 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-15T13:57:40.851 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:40 smithi033 ceph-mon[31650]: pgmap v297: 97 pgs: 97 active+clean; 582 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-15T13:57:40.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:40 smithi081 ceph-mon[36434]: pgmap v297: 97 pgs: 97 active+clean; 582 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-05-15T13:57:41.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.090+0000 7f236da88700 1 -- 172.21.15.33:0/1810361270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f236806c6d0 msgr2=0x7f2368110400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:41.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.090+0000 7f236da88700 1 --2- 172.21.15.33:0/1810361270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f236806c6d0 0x7f2368110400 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f236000b0d0 tx=0x7f236000b3e0 comp rx=0 tx=0).stop 2024-05-15T13:57:41.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.090+0000 7f236da88700 1 -- 172.21.15.33:0/1810361270 shutdown_connections 2024-05-15T13:57:41.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.090+0000 7f236da88700 1 --2- 172.21.15.33:0/1810361270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f236806c6d0 0x7f2368110400 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:41.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.090+0000 7f236da88700 1 --2- 172.21.15.33:0/1810361270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2368109630 0x7f2368109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:41.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.090+0000 7f236da88700 1 --2- 172.21.15.33:0/1810361270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f236806bd10 0x7f236806c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:41.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.090+0000 7f236da88700 1 -- 172.21.15.33:0/1810361270 >> 172.21.15.33:0/1810361270 conn(0x7f23680794e0 msgr2=0x7f23680798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:41.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.090+0000 7f236da88700 1 -- 172.21.15.33:0/1810361270 shutdown_connections 2024-05-15T13:57:41.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.090+0000 7f236da88700 1 -- 172.21.15.33:0/1810361270 wait complete. 2024-05-15T13:57:41.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.090+0000 7f236da88700 1 Processor -- start 2024-05-15T13:57:41.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.091+0000 7f236da88700 1 -- start start 2024-05-15T13:57:41.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.091+0000 7f236da88700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f236806bd10 0x7f236819f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:41.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.091+0000 7f236da88700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f236806c6d0 0x7f236819fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:41.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.091+0000 7f236da88700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2368109630 0x7f23681999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:41.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.091+0000 7f236da88700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2368113790 con 0x7f2368109630 2024-05-15T13:57:41.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.091+0000 7f236da88700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2368113610 con 0x7f236806bd10 2024-05-15T13:57:41.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.091+0000 7f236da88700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2368113910 con 0x7f236806c6d0 2024-05-15T13:57:41.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.091+0000 7f2367fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2368109630 0x7f23681999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:41.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.091+0000 7f2367fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2368109630 0x7f23681999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48540/0 (socket says 172.21.15.33:48540) 2024-05-15T13:57:41.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.091+0000 7f23677fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f236806bd10 0x7f236819f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:41.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.091+0000 7f2367fff700 1 -- 172.21.15.33:0/3094402777 learned_addr learned my addr 172.21.15.33:0/3094402777 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:57:41.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.091+0000 7f2366ffd700 1 --2- 172.21.15.33:0/3094402777 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f236806c6d0 0x7f236819fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:41.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.091+0000 7f2367fff700 1 -- 172.21.15.33:0/3094402777 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f236806c6d0 msgr2=0x7f236819fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:41.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.091+0000 7f2367fff700 1 --2- 172.21.15.33:0/3094402777 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f236806c6d0 0x7f236819fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:41.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.091+0000 7f2367fff700 1 -- 172.21.15.33:0/3094402777 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f236806bd10 msgr2=0x7f236819f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:41.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.092+0000 7f2367fff700 1 --2- 172.21.15.33:0/3094402777 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f236806bd10 0x7f236819f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:41.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.092+0000 7f2367fff700 1 -- 172.21.15.33:0/3094402777 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f236819a130 con 0x7f2368109630 2024-05-15T13:57:41.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.092+0000 7f23677fe700 1 --2- 172.21.15.33:0/3094402777 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f236806bd10 0x7f236819f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T13:57:41.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.092+0000 7f2367fff700 1 --2- 172.21.15.33:0/3094402777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2368109630 0x7f23681999d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f23600095c0 tx=0x7f2360007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:41.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.092+0000 7f2364ff9700 1 -- 172.21.15.33:0/3094402777 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236001f070 con 0x7f2368109630 2024-05-15T13:57:41.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.092+0000 7f236da88700 1 -- 172.21.15.33:0/3094402777 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f236819a3c0 con 0x7f2368109630 2024-05-15T13:57:41.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.092+0000 7f2364ff9700 1 -- 172.21.15.33:0/3094402777 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2360007e20 con 0x7f2368109630 2024-05-15T13:57:41.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.092+0000 7f2364ff9700 1 -- 172.21.15.33:0/3094402777 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236001b4a0 con 0x7f2368109630 2024-05-15T13:57:41.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.092+0000 7f236da88700 1 -- 172.21.15.33:0/3094402777 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23681a66e0 con 0x7f2368109630 2024-05-15T13:57:41.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.093+0000 7f236da88700 1 -- 172.21.15.33:0/3094402777 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f236806a230 con 0x7f2368109630 2024-05-15T13:57:41.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.095+0000 7f2364ff9700 1 -- 172.21.15.33:0/3094402777 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f2360022070 con 0x7f2368109630 2024-05-15T13:57:41.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.096+0000 7f2364ff9700 1 --2- 172.21.15.33:0/3094402777 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f23500b2880 0x7f23500b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:41.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.096+0000 7f23677fe700 1 --2- 172.21.15.33:0/3094402777 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f23500b2880 0x7f23500b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:41.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.096+0000 7f2364ff9700 1 -- 172.21.15.33:0/3094402777 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f236000f020 con 0x7f2368109630 2024-05-15T13:57:41.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.097+0000 7f23677fe700 1 --2- 172.21.15.33:0/3094402777 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f23500b2880 0x7f23500b4d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f236819ab10 tx=0x7f235800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:41.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.100+0000 7f2364ff9700 1 -- 172.21.15.33:0/3094402777 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2360029020 con 0x7f2368109630 2024-05-15T13:57:41.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.346+0000 7f236da88700 1 -- 172.21.15.33:0/3094402777 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f2368063980 con 0x7f23500b2880 2024-05-15T13:57:41.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.374+0000 7f2364ff9700 1 -- 172.21.15.33:0/3094402777 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2368063980 con 0x7f23500b2880 2024-05-15T13:57:41.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.376+0000 7f236da88700 1 -- 172.21.15.33:0/3094402777 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f23500b2880 msgr2=0x7f23500b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:41.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.376+0000 7f236da88700 1 --2- 172.21.15.33:0/3094402777 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f23500b2880 0x7f23500b4d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f236819ab10 tx=0x7f235800b410 comp rx=0 tx=0).stop 2024-05-15T13:57:41.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.376+0000 7f236da88700 1 -- 172.21.15.33:0/3094402777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2368109630 msgr2=0x7f23681999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:41.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.376+0000 7f236da88700 1 --2- 172.21.15.33:0/3094402777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2368109630 0x7f23681999d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f23600095c0 tx=0x7f2360007a20 comp rx=0 tx=0).stop 2024-05-15T13:57:41.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.376+0000 7f236da88700 1 -- 172.21.15.33:0/3094402777 shutdown_connections 2024-05-15T13:57:41.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.376+0000 7f236da88700 1 --2- 172.21.15.33:0/3094402777 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f23500b2880 0x7f23500b4d40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:41.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.376+0000 7f236da88700 1 --2- 172.21.15.33:0/3094402777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2368109630 0x7f23681999d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:41.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.376+0000 7f236da88700 1 --2- 172.21.15.33:0/3094402777 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f236806bd10 0x7f236819f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:41.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.376+0000 7f236da88700 1 --2- 172.21.15.33:0/3094402777 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f236806c6d0 0x7f236819fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:41.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.376+0000 7f236da88700 1 -- 172.21.15.33:0/3094402777 >> 172.21.15.33:0/3094402777 conn(0x7f23680794e0 msgr2=0x7f2368101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:41.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.376+0000 7f236da88700 1 -- 172.21.15.33:0/3094402777 shutdown_connections 2024-05-15T13:57:41.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:41.376+0000 7f236da88700 1 -- 172.21.15.33:0/3094402777 wait complete. 2024-05-15T13:57:41.691 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:57:41 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:57:41.351+0000 7fbdf2fa5700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-15T13:57:41.691 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:57:41 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:57:41.351+0000 7fbdf2fa5700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-15T13:57:41.691 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:57:41 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:57:41.351+0000 7fbdf2fa5700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-15T13:57:41.691 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:57:41 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:57:41.351+0000 7fbdf2fa5700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-15T13:57:41.691 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 13:57:41 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T13:57:41.351+0000 7fbdf2fa5700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-15T13:57:42.006 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-15T13:57:42.256 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:57:42.681 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:42 smithi033 ceph-mon[31650]: pgmap v298: 97 pgs: 97 active+clean; 582 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-15T13:57:42.681 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:42 smithi033 ceph-mon[31650]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:57:42.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:42 smithi052 ceph-mon[36190]: pgmap v298: 97 pgs: 97 active+clean; 582 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-15T13:57:42.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:42 smithi052 ceph-mon[36190]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:57:42.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:42 smithi081 ceph-mon[36434]: pgmap v298: 97 pgs: 97 active+clean; 582 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-15T13:57:42.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:42 smithi081 ceph-mon[36434]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:57:43.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.974+0000 7f8a6c144700 1 -- 172.21.15.33:0/1449745959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a641041a0 msgr2=0x7f8a641081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:43.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.974+0000 7f8a6c144700 1 --2- 172.21.15.33:0/1449745959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a641041a0 0x7f8a641081c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8a540099c0 tx=0x7f8a54009cd0 comp rx=0 tx=0).stop 2024-05-15T13:57:43.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.975+0000 7f8a6c144700 1 -- 172.21.15.33:0/1449745959 shutdown_connections 2024-05-15T13:57:43.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.975+0000 7f8a6c144700 1 --2- 172.21.15.33:0/1449745959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a641088f0 0x7f8a64110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:43.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.975+0000 7f8a6c144700 1 --2- 172.21.15.33:0/1449745959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a641037f0 0x7f8a64103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:43.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.975+0000 7f8a6c144700 1 --2- 172.21.15.33:0/1449745959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a641041a0 0x7f8a641081c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:43.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.975+0000 7f8a6c144700 1 -- 172.21.15.33:0/1449745959 >> 172.21.15.33:0/1449745959 conn(0x7f8a6407a6a0 msgr2=0x7f8a6407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:43.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.975+0000 7f8a6c144700 1 -- 172.21.15.33:0/1449745959 shutdown_connections 2024-05-15T13:57:43.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.975+0000 7f8a6c144700 1 -- 172.21.15.33:0/1449745959 wait complete. 2024-05-15T13:57:43.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.975+0000 7f8a6c144700 1 Processor -- start 2024-05-15T13:57:43.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.975+0000 7f8a6c144700 1 -- start start 2024-05-15T13:57:43.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.976+0000 7f8a6c144700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a641037f0 0x7f8a6410f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:43.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.976+0000 7f8a6c144700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a641041a0 0x7f8a6410fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:43.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.976+0000 7f8a6c144700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a641088f0 0x7f8a641a4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:43.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.976+0000 7f8a6c144700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8a641136d0 con 0x7f8a641088f0 2024-05-15T13:57:43.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.976+0000 7f8a6c144700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8a64113550 con 0x7f8a641037f0 2024-05-15T13:57:43.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.976+0000 7f8a6c144700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f8a64113850 con 0x7f8a641041a0 2024-05-15T13:57:43.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.976+0000 7f8a6a6e1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a641088f0 0x7f8a641a4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:43.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.976+0000 7f8a6a6e1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a641088f0 0x7f8a641a4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33002/0 (socket says 172.21.15.33:33002) 2024-05-15T13:57:43.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.976+0000 7f8a6a6e1700 1 -- 172.21.15.33:0/3496690164 learned_addr learned my addr 172.21.15.33:0/3496690164 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:57:43.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.976+0000 7f8a696df700 1 --2- 172.21.15.33:0/3496690164 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a641041a0 0x7f8a6410fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:43.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.976+0000 7f8a69ee0700 1 --2- 172.21.15.33:0/3496690164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a641037f0 0x7f8a6410f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:43.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.976+0000 7f8a6a6e1700 1 -- 172.21.15.33:0/3496690164 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a641041a0 msgr2=0x7f8a6410fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:43.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.976+0000 7f8a6a6e1700 1 --2- 172.21.15.33:0/3496690164 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a641041a0 0x7f8a6410fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:43.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.976+0000 7f8a6a6e1700 1 -- 172.21.15.33:0/3496690164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a641037f0 msgr2=0x7f8a6410f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:43.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.976+0000 7f8a6a6e1700 1 --2- 172.21.15.33:0/3496690164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a641037f0 0x7f8a6410f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:43.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.976+0000 7f8a6a6e1700 1 -- 172.21.15.33:0/3496690164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a641a5140 con 0x7f8a641088f0 2024-05-15T13:57:43.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.976+0000 7f8a6a6e1700 1 --2- 172.21.15.33:0/3496690164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a641088f0 0x7f8a641a4a00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c007d70 tx=0x7f8a5c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:43.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.977+0000 7f8a5affd700 1 -- 172.21.15.33:0/3496690164 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c016020 con 0x7f8a641088f0 2024-05-15T13:57:43.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.977+0000 7f8a6c144700 1 -- 172.21.15.33:0/3496690164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a641a5430 con 0x7f8a641088f0 2024-05-15T13:57:43.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.977+0000 7f8a5affd700 1 -- 172.21.15.33:0/3496690164 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a5c00ac90 con 0x7f8a641088f0 2024-05-15T13:57:43.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.977+0000 7f8a5affd700 1 -- 172.21.15.33:0/3496690164 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c0115e0 con 0x7f8a641088f0 2024-05-15T13:57:43.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.977+0000 7f8a6c144700 1 -- 172.21.15.33:0/3496690164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a641accb0 con 0x7f8a641088f0 2024-05-15T13:57:43.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.978+0000 7f8a6c144700 1 -- 172.21.15.33:0/3496690164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a64104c80 con 0x7f8a641088f0 2024-05-15T13:57:43.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.979+0000 7f8a5affd700 1 -- 172.21.15.33:0/3496690164 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f8a5c019020 con 0x7f8a641088f0 2024-05-15T13:57:43.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.980+0000 7f8a5affd700 1 --2- 172.21.15.33:0/3496690164 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f8a500b2910 0x7f8a500b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:43.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.980+0000 7f8a69ee0700 1 --2- 172.21.15.33:0/3496690164 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f8a500b2910 0x7f8a500b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:43.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.980+0000 7f8a5affd700 1 -- 172.21.15.33:0/3496690164 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f8a5c0e2640 con 0x7f8a641088f0 2024-05-15T13:57:43.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.980+0000 7f8a69ee0700 1 --2- 172.21.15.33:0/3496690164 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f8a500b2910 0x7f8a500b4dd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8a60004630 tx=0x7f8a6000b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:43.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:43.984+0000 7f8a5affd700 1 -- 172.21.15.33:0/3496690164 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8a5c020020 con 0x7f8a641088f0 2024-05-15T13:57:44.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:44.230+0000 7f8a6c144700 1 -- 172.21.15.33:0/3496690164 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f8a64109a40 con 0x7f8a500b2910 2024-05-15T13:57:44.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:44.257+0000 7f8a5affd700 1 -- 172.21.15.33:0/3496690164 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8a64109a40 con 0x7f8a500b2910 2024-05-15T13:57:44.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:44.259+0000 7f8a6c144700 1 -- 172.21.15.33:0/3496690164 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f8a500b2910 msgr2=0x7f8a500b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:44.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:44.259+0000 7f8a6c144700 1 --2- 172.21.15.33:0/3496690164 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f8a500b2910 0x7f8a500b4dd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8a60004630 tx=0x7f8a6000b3c0 comp rx=0 tx=0).stop 2024-05-15T13:57:44.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:44.259+0000 7f8a6c144700 1 -- 172.21.15.33:0/3496690164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a641088f0 msgr2=0x7f8a641a4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:44.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:44.259+0000 7f8a6c144700 1 --2- 172.21.15.33:0/3496690164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a641088f0 0x7f8a641a4a00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c007d70 tx=0x7f8a5c00a340 comp rx=0 tx=0).stop 2024-05-15T13:57:44.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:44.259+0000 7f8a6c144700 1 -- 172.21.15.33:0/3496690164 shutdown_connections 2024-05-15T13:57:44.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:44.259+0000 7f8a6c144700 1 --2- 172.21.15.33:0/3496690164 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f8a500b2910 0x7f8a500b4dd0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:44.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:44.259+0000 7f8a6c144700 1 --2- 172.21.15.33:0/3496690164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a641088f0 0x7f8a641a4a00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:44.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:44.259+0000 7f8a6c144700 1 --2- 172.21.15.33:0/3496690164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a641037f0 0x7f8a6410f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:44.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:44.259+0000 7f8a6c144700 1 --2- 172.21.15.33:0/3496690164 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8a641041a0 0x7f8a6410fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:44.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:44.259+0000 7f8a6c144700 1 -- 172.21.15.33:0/3496690164 >> 172.21.15.33:0/3496690164 conn(0x7f8a6407a6a0 msgr2=0x7f8a64102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:44.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:44.260+0000 7f8a6c144700 1 -- 172.21.15.33:0/3496690164 shutdown_connections 2024-05-15T13:57:44.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:44.260+0000 7f8a6c144700 1 -- 172.21.15.33:0/3496690164 wait complete. 2024-05-15T13:57:44.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:44 smithi052 ceph-mon[36190]: pgmap v299: 97 pgs: 97 active+clean; 582 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-05-15T13:57:44.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:44 smithi052 ceph-mon[36190]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-15T13:57:44.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:44 smithi081 ceph-mon[36434]: pgmap v299: 97 pgs: 97 active+clean; 582 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-05-15T13:57:44.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:44 smithi081 ceph-mon[36434]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-15T13:57:44.869 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:44 smithi033 ceph-mon[31650]: pgmap v299: 97 pgs: 97 active+clean; 582 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-05-15T13:57:44.869 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:44 smithi033 ceph-mon[31650]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-15T13:57:44.897 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-15T13:57:45.149 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:57:45.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:45 smithi052 ceph-mon[36190]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:57:45.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:45 smithi081 ceph-mon[36434]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:57:45.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:45 smithi033 ceph-mon[31650]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:57:46.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:46 smithi033 ceph-mon[31650]: pgmap v300: 97 pgs: 97 active+clean; 582 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-05-15T13:57:46.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:46 smithi052 ceph-mon[36190]: pgmap v300: 97 pgs: 97 active+clean; 582 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-05-15T13:57:46.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:46 smithi081 ceph-mon[36434]: pgmap v300: 97 pgs: 97 active+clean; 582 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-05-15T13:57:46.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.865+0000 7f9fb2df2700 1 -- 172.21.15.33:0/4293214240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fac06bc70 msgr2=0x7f9fac06c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:46.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.865+0000 7f9fb2df2700 1 --2- 172.21.15.33:0/4293214240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fac06bc70 0x7f9fac06c0f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c009990 tx=0x7f9f9c009ca0 comp rx=0 tx=0).stop 2024-05-15T13:57:46.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.865+0000 7f9fb2df2700 1 -- 172.21.15.33:0/4293214240 shutdown_connections 2024-05-15T13:57:46.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.865+0000 7f9fb2df2700 1 --2- 172.21.15.33:0/4293214240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fac06c630 0x7f9fac1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:46.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.865+0000 7f9fb2df2700 1 --2- 172.21.15.33:0/4293214240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fac1095b0 0x7f9fac109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:46.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.865+0000 7f9fb2df2700 1 --2- 172.21.15.33:0/4293214240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fac06bc70 0x7f9fac06c0f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:46.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.865+0000 7f9fb2df2700 1 -- 172.21.15.33:0/4293214240 >> 172.21.15.33:0/4293214240 conn(0x7f9fac0793f0 msgr2=0x7f9fac079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:46.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.865+0000 7f9fb2df2700 1 -- 172.21.15.33:0/4293214240 shutdown_connections 2024-05-15T13:57:46.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.865+0000 7f9fb2df2700 1 -- 172.21.15.33:0/4293214240 wait complete. 2024-05-15T13:57:46.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.865+0000 7f9fb2df2700 1 Processor -- start 2024-05-15T13:57:46.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.865+0000 7f9fb2df2700 1 -- start start 2024-05-15T13:57:46.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.866+0000 7f9fb2df2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fac06bc70 0x7f9fac19be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:46.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.866+0000 7f9fb2df2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fac06c630 0x7f9fac19c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:46.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.866+0000 7f9fb2df2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fac1095b0 0x7f9fac1a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:46.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.866+0000 7f9fb2df2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac1136e0 con 0x7f9fac06c630 2024-05-15T13:57:46.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.866+0000 7f9fb2df2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac113560 con 0x7f9fac1095b0 2024-05-15T13:57:46.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.866+0000 7f9fb2df2700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac113860 con 0x7f9fac06bc70 2024-05-15T13:57:46.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.866+0000 7f9fabfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fac06c630 0x7f9fac19c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:46.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.866+0000 7f9fabfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fac06c630 0x7f9fac19c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33014/0 (socket says 172.21.15.33:33014) 2024-05-15T13:57:46.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.866+0000 7f9fabfff700 1 -- 172.21.15.33:0/2731335257 learned_addr learned my addr 172.21.15.33:0/2731335257 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:57:46.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.866+0000 7f9fb138f700 1 --2- 172.21.15.33:0/2731335257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fac1095b0 0x7f9fac1a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:46.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.866+0000 7f9fb0b8e700 1 --2- 172.21.15.33:0/2731335257 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fac06bc70 0x7f9fac19be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:46.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.866+0000 7f9fabfff700 1 -- 172.21.15.33:0/2731335257 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fac06bc70 msgr2=0x7f9fac19be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:46.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.866+0000 7f9fabfff700 1 --2- 172.21.15.33:0/2731335257 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fac06bc70 0x7f9fac19be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:46.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.866+0000 7f9fabfff700 1 -- 172.21.15.33:0/2731335257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fac1095b0 msgr2=0x7f9fac1a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:46.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.866+0000 7f9fabfff700 1 --2- 172.21.15.33:0/2731335257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fac1095b0 0x7f9fac1a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:46.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.866+0000 7f9fabfff700 1 -- 172.21.15.33:0/2731335257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fac1a0e40 con 0x7f9fac06c630 2024-05-15T13:57:46.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.866+0000 7f9fabfff700 1 --2- 172.21.15.33:0/2731335257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fac06c630 0x7f9fac19c370 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c000c00 tx=0x7f9f9c00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:46.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.866+0000 7f9fa9ffb700 1 -- 172.21.15.33:0/2731335257 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c027070 con 0x7f9fac06c630 2024-05-15T13:57:46.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.867+0000 7f9fb2df2700 1 -- 172.21.15.33:0/2731335257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fac1a10d0 con 0x7f9fac06c630 2024-05-15T13:57:46.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.867+0000 7f9fa9ffb700 1 -- 172.21.15.33:0/2731335257 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f9c022470 con 0x7f9fac06c630 2024-05-15T13:57:46.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.867+0000 7f9fa9ffb700 1 -- 172.21.15.33:0/2731335257 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c017650 con 0x7f9fac06c630 2024-05-15T13:57:46.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.867+0000 7f9fb2df2700 1 -- 172.21.15.33:0/2731335257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fac1a89b0 con 0x7f9fac06c630 2024-05-15T13:57:46.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.868+0000 7f9fb2df2700 1 -- 172.21.15.33:0/2731335257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fac06a230 con 0x7f9fac06c630 2024-05-15T13:57:46.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.869+0000 7f9fa9ffb700 1 -- 172.21.15.33:0/2731335257 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9f9c02a070 con 0x7f9fac06c630 2024-05-15T13:57:46.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.870+0000 7f9fa9ffb700 1 --2- 172.21.15.33:0/2731335257 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f9f980b2910 0x7f9f980b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:46.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.870+0000 7f9fb0b8e700 1 --2- 172.21.15.33:0/2731335257 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f9f980b2910 0x7f9f980b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:46.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.870+0000 7f9fa9ffb700 1 -- 172.21.15.33:0/2731335257 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f9f9c0e3440 con 0x7f9fac06c630 2024-05-15T13:57:46.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.870+0000 7f9fb0b8e700 1 --2- 172.21.15.33:0/2731335257 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f9f980b2910 0x7f9f980b4dd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9f94009be0 tx=0x7f9f94009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:46.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:46.873+0000 7f9fa9ffb700 1 -- 172.21.15.33:0/2731335257 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9f9c013070 con 0x7f9fac06c630 2024-05-15T13:57:47.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:47.123+0000 7f9fb2df2700 1 -- 172.21.15.33:0/2731335257 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f9fac10ee30 con 0x7f9f980b2910 2024-05-15T13:57:47.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:47.130+0000 7f9fa9ffb700 1 -- 172.21.15.33:0/2731335257 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f9fac10ee30 con 0x7f9f980b2910 2024-05-15T13:57:47.131 INFO:teuthology.orchestra.run.smithi033.stdout:/volumes/qa/sv_0/9d8fcc07-077b-48cc-9cd5-8369bfc8bdfe 2024-05-15T13:57:47.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:47.132+0000 7f9fb2df2700 1 -- 172.21.15.33:0/2731335257 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f9f980b2910 msgr2=0x7f9f980b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:47.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:47.132+0000 7f9fb2df2700 1 --2- 172.21.15.33:0/2731335257 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f9f980b2910 0x7f9f980b4dd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9f94009be0 tx=0x7f9f94009260 comp rx=0 tx=0).stop 2024-05-15T13:57:47.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:47.132+0000 7f9fb2df2700 1 -- 172.21.15.33:0/2731335257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fac06c630 msgr2=0x7f9fac19c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:47.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:47.132+0000 7f9fb2df2700 1 --2- 172.21.15.33:0/2731335257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fac06c630 0x7f9fac19c370 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c000c00 tx=0x7f9f9c00bfd0 comp rx=0 tx=0).stop 2024-05-15T13:57:47.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:47.132+0000 7f9fb2df2700 1 -- 172.21.15.33:0/2731335257 shutdown_connections 2024-05-15T13:57:47.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:47.132+0000 7f9fb2df2700 1 --2- 172.21.15.33:0/2731335257 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f9f980b2910 0x7f9f980b4dd0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:47.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:47.132+0000 7f9fb2df2700 1 --2- 172.21.15.33:0/2731335257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fac1095b0 0x7f9fac1a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:47.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:47.132+0000 7f9fb2df2700 1 --2- 172.21.15.33:0/2731335257 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9fac06bc70 0x7f9fac19be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:47.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:47.132+0000 7f9fb2df2700 1 --2- 172.21.15.33:0/2731335257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fac06c630 0x7f9fac19c370 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:47.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:47.132+0000 7f9fb2df2700 1 -- 172.21.15.33:0/2731335257 >> 172.21.15.33:0/2731335257 conn(0x7f9fac0793f0 msgr2=0x7f9fac10e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:47.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:47.132+0000 7f9fb2df2700 1 -- 172.21.15.33:0/2731335257 shutdown_connections 2024-05-15T13:57:47.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:47.132+0000 7f9fb2df2700 1 -- 172.21.15.33:0/2731335257 wait complete. 2024-05-15T13:57:47.793 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:47 smithi033 ceph-mon[31650]: pgmap v301: 97 pgs: 97 active+clean; 603 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-05-15T13:57:47.794 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:47 smithi033 ceph-mon[31650]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:57:47.825 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-15T13:57:47.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:47 smithi052 ceph-mon[36190]: pgmap v301: 97 pgs: 97 active+clean; 603 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-05-15T13:57:47.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:47 smithi052 ceph-mon[36190]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:57:47.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:47 smithi081 ceph-mon[36434]: pgmap v301: 97 pgs: 97 active+clean; 603 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-05-15T13:57:47.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:47 smithi081 ceph-mon[36434]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:57:48.078 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:57:49.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.782+0000 7f7102bdb700 1 -- 172.21.15.33:0/4050231573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70fc1041d0 msgr2=0x7f70fc1081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:49.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.782+0000 7f7102bdb700 1 --2- 172.21.15.33:0/4050231573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70fc1041d0 0x7f70fc1081e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f70f00099c0 tx=0x7f70f0009cd0 comp rx=0 tx=0).stop 2024-05-15T13:57:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.782+0000 7f7102bdb700 1 -- 172.21.15.33:0/4050231573 shutdown_connections 2024-05-15T13:57:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.782+0000 7f7102bdb700 1 --2- 172.21.15.33:0/4050231573 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70fc108910 0x7f70fc110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.782+0000 7f7102bdb700 1 --2- 172.21.15.33:0/4050231573 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70fc103820 0x7f70fc103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.782+0000 7f7102bdb700 1 --2- 172.21.15.33:0/4050231573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70fc1041d0 0x7f70fc1081e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.782+0000 7f7102bdb700 1 -- 172.21.15.33:0/4050231573 >> 172.21.15.33:0/4050231573 conn(0x7f70fc07a6b0 msgr2=0x7f70fc07aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.782+0000 7f7102bdb700 1 -- 172.21.15.33:0/4050231573 shutdown_connections 2024-05-15T13:57:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.782+0000 7f7102bdb700 1 -- 172.21.15.33:0/4050231573 wait complete. 2024-05-15T13:57:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.783+0000 7f7102bdb700 1 Processor -- start 2024-05-15T13:57:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.783+0000 7f7102bdb700 1 -- start start 2024-05-15T13:57:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.783+0000 7f7102bdb700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70fc103820 0x7f70fc19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.783+0000 7f7102bdb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70fc1041d0 0x7f70fc19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.783+0000 7f7102bdb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70fc108910 0x7f70fc199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.783+0000 7f7102bdb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f70fc1136d0 con 0x7f70fc1041d0 2024-05-15T13:57:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.783+0000 7f7102bdb700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f70fc113550 con 0x7f70fc108910 2024-05-15T13:57:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.783+0000 7f7102bdb700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f70fc113850 con 0x7f70fc103820 2024-05-15T13:57:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.783+0000 7f70fbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70fc1041d0 0x7f70fc19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.783+0000 7f70fbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70fc1041d0 0x7f70fc19fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33024/0 (socket says 172.21.15.33:33024) 2024-05-15T13:57:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.783+0000 7f7100977700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70fc103820 0x7f70fc19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.783+0000 7f70fbfff700 1 -- 172.21.15.33:0/357158702 learned_addr learned my addr 172.21.15.33:0/357158702 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:57:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.783+0000 7f7101178700 1 --2- 172.21.15.33:0/357158702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70fc108910 0x7f70fc199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.784+0000 7f70fbfff700 1 -- 172.21.15.33:0/357158702 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70fc103820 msgr2=0x7f70fc19f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.784+0000 7f70fbfff700 1 --2- 172.21.15.33:0/357158702 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70fc103820 0x7f70fc19f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.784+0000 7f70fbfff700 1 -- 172.21.15.33:0/357158702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70fc108910 msgr2=0x7f70fc199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.784+0000 7f70fbfff700 1 --2- 172.21.15.33:0/357158702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70fc108910 0x7f70fc199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.784+0000 7f70fbfff700 1 -- 172.21.15.33:0/357158702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70fc19a0b0 con 0x7f70fc1041d0 2024-05-15T13:57:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.784+0000 7f7100977700 1 --2- 172.21.15.33:0/357158702 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70fc103820 0x7f70fc19f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T13:57:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.784+0000 7f70fbfff700 1 --2- 172.21.15.33:0/357158702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70fc1041d0 0x7f70fc19fcd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f70f00098e0 tx=0x7f70f00057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.784+0000 7f70f9ffb700 1 -- 172.21.15.33:0/357158702 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f0027070 con 0x7f70fc1041d0 2024-05-15T13:57:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.784+0000 7f7102bdb700 1 -- 172.21.15.33:0/357158702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70fc19a340 con 0x7f70fc1041d0 2024-05-15T13:57:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.784+0000 7f70f9ffb700 1 -- 172.21.15.33:0/357158702 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70f0003f20 con 0x7f70fc1041d0 2024-05-15T13:57:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.784+0000 7f70f9ffb700 1 -- 172.21.15.33:0/357158702 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f0022d70 con 0x7f70fc1041d0 2024-05-15T13:57:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.784+0000 7f7102bdb700 1 -- 172.21.15.33:0/357158702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70fc1a6660 con 0x7f70fc1041d0 2024-05-15T13:57:49.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.785+0000 7f7102bdb700 1 -- 172.21.15.33:0/357158702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70fc104ca0 con 0x7f70fc1041d0 2024-05-15T13:57:49.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.787+0000 7f70f9ffb700 1 -- 172.21.15.33:0/357158702 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f70f002a070 con 0x7f70fc1041d0 2024-05-15T13:57:49.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.787+0000 7f70f9ffb700 1 --2- 172.21.15.33:0/357158702 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f70e40b2910 0x7f70e40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:49.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.787+0000 7f7100977700 1 --2- 172.21.15.33:0/357158702 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f70e40b2910 0x7f70e40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:49.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.787+0000 7f70f9ffb700 1 -- 172.21.15.33:0/357158702 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f70f00e37e0 con 0x7f70fc1041d0 2024-05-15T13:57:49.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.788+0000 7f7100977700 1 --2- 172.21.15.33:0/357158702 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f70e40b2910 0x7f70e40b4dd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f70fc19aa90 tx=0x7f70ec00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:49.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:49.791+0000 7f70f9ffb700 1 -- 172.21.15.33:0/357158702 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f70f0031020 con 0x7f70fc1041d0 2024-05-15T13:57:50.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:50.039+0000 7f7102bdb700 1 -- 172.21.15.33:0/357158702 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f70fc063980 con 0x7f70e40b2910 2024-05-15T13:57:50.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:50.068+0000 7f70f9ffb700 1 -- 172.21.15.33:0/357158702 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f70fc063980 con 0x7f70e40b2910 2024-05-15T13:57:50.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:50.070+0000 7f7102bdb700 1 -- 172.21.15.33:0/357158702 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f70e40b2910 msgr2=0x7f70e40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:50.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:50.070+0000 7f7102bdb700 1 --2- 172.21.15.33:0/357158702 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f70e40b2910 0x7f70e40b4dd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f70fc19aa90 tx=0x7f70ec00a380 comp rx=0 tx=0).stop 2024-05-15T13:57:50.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:50.070+0000 7f7102bdb700 1 -- 172.21.15.33:0/357158702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70fc1041d0 msgr2=0x7f70fc19fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:50.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:50.070+0000 7f7102bdb700 1 --2- 172.21.15.33:0/357158702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70fc1041d0 0x7f70fc19fcd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f70f00098e0 tx=0x7f70f00057c0 comp rx=0 tx=0).stop 2024-05-15T13:57:50.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:50.070+0000 7f7102bdb700 1 -- 172.21.15.33:0/357158702 shutdown_connections 2024-05-15T13:57:50.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:50.070+0000 7f7102bdb700 1 --2- 172.21.15.33:0/357158702 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f70e40b2910 0x7f70e40b4dd0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:50.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:50.070+0000 7f7102bdb700 1 --2- 172.21.15.33:0/357158702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70fc108910 0x7f70fc199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:50.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:50.070+0000 7f7102bdb700 1 --2- 172.21.15.33:0/357158702 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70fc103820 0x7f70fc19f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:50.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:50.070+0000 7f7102bdb700 1 --2- 172.21.15.33:0/357158702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70fc1041d0 0x7f70fc19fcd0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:50.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:50.070+0000 7f7102bdb700 1 -- 172.21.15.33:0/357158702 >> 172.21.15.33:0/357158702 conn(0x7f70fc07a6b0 msgr2=0x7f70fc102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:50.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:50.070+0000 7f7102bdb700 1 -- 172.21.15.33:0/357158702 shutdown_connections 2024-05-15T13:57:50.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:50.070+0000 7f7102bdb700 1 -- 172.21.15.33:0/357158702 wait complete. 2024-05-15T13:57:50.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:50 smithi052 ceph-mon[36190]: pgmap v302: 97 pgs: 97 active+clean; 603 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-05-15T13:57:50.356 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:50 smithi081 ceph-mon[36434]: pgmap v302: 97 pgs: 97 active+clean; 603 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-05-15T13:57:50.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:50 smithi033 ceph-mon[31650]: pgmap v302: 97 pgs: 97 active+clean; 603 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-05-15T13:57:50.680 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-15T13:57:50.932 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:57:51.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:51 smithi052 ceph-mon[36190]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:57:51.355 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:51 smithi033 ceph-mon[31650]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:57:51.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:51 smithi081 ceph-mon[36434]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:57:52.327 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:52 smithi033 ceph-mon[31650]: pgmap v303: 97 pgs: 97 active+clean; 603 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2024-05-15T13:57:52.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:52 smithi052 ceph-mon[36190]: pgmap v303: 97 pgs: 97 active+clean; 603 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2024-05-15T13:57:52.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:52 smithi081 ceph-mon[36434]: pgmap v303: 97 pgs: 97 active+clean; 603 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2024-05-15T13:57:52.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.709+0000 7f4bba0b4700 1 -- 172.21.15.33:0/1875592965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bb4102e30 msgr2=0x7f4bb410f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:52.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.709+0000 7f4bba0b4700 1 --2- 172.21.15.33:0/1875592965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bb4102e30 0x7f4bb410f2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4bac009990 tx=0x7f4bac009ca0 comp rx=0 tx=0).stop 2024-05-15T13:57:52.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.709+0000 7f4bba0b4700 1 -- 172.21.15.33:0/1875592965 shutdown_connections 2024-05-15T13:57:52.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.709+0000 7f4bba0b4700 1 --2- 172.21.15.33:0/1875592965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bb4102e30 0x7f4bb410f2f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:52.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.709+0000 7f4bba0b4700 1 --2- 172.21.15.33:0/1875592965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bb41084b0 0x7f4bb4108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:52.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.709+0000 7f4bba0b4700 1 --2- 172.21.15.33:0/1875592965 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bb4102470 0x7f4bb41028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:52.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.709+0000 7f4bba0b4700 1 -- 172.21.15.33:0/1875592965 >> 172.21.15.33:0/1875592965 conn(0x7f4bb40793f0 msgr2=0x7f4bb4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:52.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.709+0000 7f4bba0b4700 1 -- 172.21.15.33:0/1875592965 shutdown_connections 2024-05-15T13:57:52.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.709+0000 7f4bba0b4700 1 -- 172.21.15.33:0/1875592965 wait complete. 2024-05-15T13:57:52.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.710+0000 7f4bba0b4700 1 Processor -- start 2024-05-15T13:57:52.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.710+0000 7f4bba0b4700 1 -- start start 2024-05-15T13:57:52.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.710+0000 7f4bba0b4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bb4102470 0x7f4bb4104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:52.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.710+0000 7f4bba0b4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bb4102e30 0x7f4bb41045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:52.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.710+0000 7f4bba0b4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bb41084b0 0x7f4bb4104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:52.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.710+0000 7f4bba0b4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb4112690 con 0x7f4bb4102e30 2024-05-15T13:57:52.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.710+0000 7f4bba0b4700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb4112510 con 0x7f4bb4102470 2024-05-15T13:57:52.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.710+0000 7f4bba0b4700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb4112810 con 0x7f4bb41084b0 2024-05-15T13:57:52.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.710+0000 7f4bb2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bb4102e30 0x7f4bb41045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:52.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.710+0000 7f4bb2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bb4102e30 0x7f4bb41045c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52260/0 (socket says 172.21.15.33:52260) 2024-05-15T13:57:52.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.710+0000 7f4bb2ffd700 1 -- 172.21.15.33:0/2869737001 learned_addr learned my addr 172.21.15.33:0/2869737001 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:57:52.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.711+0000 7f4bb37fe700 1 --2- 172.21.15.33:0/2869737001 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bb4102470 0x7f4bb4104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:52.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.711+0000 7f4bb3fff700 1 --2- 172.21.15.33:0/2869737001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bb41084b0 0x7f4bb4104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:52.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.711+0000 7f4bb2ffd700 1 -- 172.21.15.33:0/2869737001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bb41084b0 msgr2=0x7f4bb4104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:52.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.711+0000 7f4bb2ffd700 1 --2- 172.21.15.33:0/2869737001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bb41084b0 0x7f4bb4104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:52.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.711+0000 7f4bb2ffd700 1 -- 172.21.15.33:0/2869737001 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bb4102470 msgr2=0x7f4bb4104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:52.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.711+0000 7f4bb2ffd700 1 --2- 172.21.15.33:0/2869737001 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bb4102470 0x7f4bb4104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:52.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.711+0000 7f4bb2ffd700 1 -- 172.21.15.33:0/2869737001 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb41a81c0 con 0x7f4bb4102e30 2024-05-15T13:57:52.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.711+0000 7f4bb2ffd700 1 --2- 172.21.15.33:0/2869737001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bb4102e30 0x7f4bb41045c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4ba800d950 tx=0x7f4ba800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:52.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.711+0000 7f4bb0ff9700 1 -- 172.21.15.33:0/2869737001 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba8016070 con 0x7f4bb4102e30 2024-05-15T13:57:52.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.711+0000 7f4bb0ff9700 1 -- 172.21.15.33:0/2869737001 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ba8009df0 con 0x7f4bb4102e30 2024-05-15T13:57:52.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.711+0000 7f4bb0ff9700 1 -- 172.21.15.33:0/2869737001 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba800f3c0 con 0x7f4bb4102e30 2024-05-15T13:57:52.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.711+0000 7f4bba0b4700 1 -- 172.21.15.33:0/2869737001 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bb41a84b0 con 0x7f4bb4102e30 2024-05-15T13:57:52.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.711+0000 7f4bba0b4700 1 -- 172.21.15.33:0/2869737001 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bb41a89a0 con 0x7f4bb4102e30 2024-05-15T13:57:52.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.712+0000 7f4bba0b4700 1 -- 172.21.15.33:0/2869737001 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bb4103b60 con 0x7f4bb4102e30 2024-05-15T13:57:52.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.714+0000 7f4bb0ff9700 1 -- 172.21.15.33:0/2869737001 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f4ba8019070 con 0x7f4bb4102e30 2024-05-15T13:57:52.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.715+0000 7f4bb0ff9700 1 --2- 172.21.15.33:0/2869737001 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f4b9c0b2990 0x7f4b9c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:52.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.715+0000 7f4bb37fe700 1 --2- 172.21.15.33:0/2869737001 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f4b9c0b2990 0x7f4b9c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:52.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.715+0000 7f4bb0ff9700 1 -- 172.21.15.33:0/2869737001 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f4ba80e1e60 con 0x7f4bb4102e30 2024-05-15T13:57:52.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.715+0000 7f4bb37fe700 1 --2- 172.21.15.33:0/2869737001 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f4b9c0b2990 0x7f4b9c0b4e50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4ba400a400 tx=0x7f4ba400d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:52.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.719+0000 7f4bb0ff9700 1 -- 172.21.15.33:0/2869737001 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4ba80a9ad0 con 0x7f4bb4102e30 2024-05-15T13:57:52.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.970+0000 7f4bba0b4700 1 -- 172.21.15.33:0/2869737001 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f4bb41002e0 con 0x7f4b9c0b2990 2024-05-15T13:57:52.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.975+0000 7f4bb0ff9700 1 -- 172.21.15.33:0/2869737001 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f4bb41002e0 con 0x7f4b9c0b2990 2024-05-15T13:57:52.977 INFO:teuthology.orchestra.run.smithi033.stdout:/volumes/qa/sv_1/ee5a4962-7db7-4578-bad4-70ba3aff515c 2024-05-15T13:57:52.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.977+0000 7f4bba0b4700 1 -- 172.21.15.33:0/2869737001 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f4b9c0b2990 msgr2=0x7f4b9c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:52.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.977+0000 7f4bba0b4700 1 --2- 172.21.15.33:0/2869737001 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f4b9c0b2990 0x7f4b9c0b4e50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4ba400a400 tx=0x7f4ba400d040 comp rx=0 tx=0).stop 2024-05-15T13:57:52.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.977+0000 7f4bba0b4700 1 -- 172.21.15.33:0/2869737001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bb4102e30 msgr2=0x7f4bb41045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:52.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.977+0000 7f4bba0b4700 1 --2- 172.21.15.33:0/2869737001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bb4102e30 0x7f4bb41045c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4ba800d950 tx=0x7f4ba800dc60 comp rx=0 tx=0).stop 2024-05-15T13:57:52.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.977+0000 7f4bba0b4700 1 -- 172.21.15.33:0/2869737001 shutdown_connections 2024-05-15T13:57:52.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.977+0000 7f4bba0b4700 1 --2- 172.21.15.33:0/2869737001 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f4b9c0b2990 0x7f4b9c0b4e50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:52.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.977+0000 7f4bba0b4700 1 --2- 172.21.15.33:0/2869737001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bb41084b0 0x7f4bb4104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:52.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.977+0000 7f4bba0b4700 1 --2- 172.21.15.33:0/2869737001 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4bb4102470 0x7f4bb4104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:52.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.977+0000 7f4bba0b4700 1 --2- 172.21.15.33:0/2869737001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bb4102e30 0x7f4bb41045c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:52.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.977+0000 7f4bba0b4700 1 -- 172.21.15.33:0/2869737001 >> 172.21.15.33:0/2869737001 conn(0x7f4bb40793f0 msgr2=0x7f4bb4106260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:52.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.978+0000 7f4bba0b4700 1 -- 172.21.15.33:0/2869737001 shutdown_connections 2024-05-15T13:57:52.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:52.978+0000 7f4bba0b4700 1 -- 172.21.15.33:0/2869737001 wait complete. 2024-05-15T13:57:53.666 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T13:57:53.928 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:57:54.238 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:54 smithi033 ceph-mon[31650]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:57:54.238 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:54 smithi033 ceph-mon[31650]: pgmap v304: 97 pgs: 97 active+clean; 622 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-05-15T13:57:54.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:54 smithi052 ceph-mon[36190]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:57:54.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:54 smithi052 ceph-mon[36190]: pgmap v304: 97 pgs: 97 active+clean; 622 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-05-15T13:57:54.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:54 smithi081 ceph-mon[36434]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:57:54.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:54 smithi081 ceph-mon[36434]: pgmap v304: 97 pgs: 97 active+clean; 622 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-05-15T13:57:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.483+0000 7fb8a0641700 1 -- 172.21.15.33:0/4287279592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb89806c6d0 msgr2=0x7fb898110400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.483+0000 7fb8a0641700 1 --2- 172.21.15.33:0/4287279592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb89806c6d0 0x7fb898110400 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb89000b0d0 tx=0x7fb89000b3e0 comp rx=0 tx=0).stop 2024-05-15T13:57:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.483+0000 7fb8a0641700 1 -- 172.21.15.33:0/4287279592 shutdown_connections 2024-05-15T13:57:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.483+0000 7fb8a0641700 1 --2- 172.21.15.33:0/4287279592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb89806c6d0 0x7fb898110400 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.483+0000 7fb8a0641700 1 --2- 172.21.15.33:0/4287279592 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb898109630 0x7fb898109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.483+0000 7fb8a0641700 1 --2- 172.21.15.33:0/4287279592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb89806bd10 0x7fb89806c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.483+0000 7fb8a0641700 1 -- 172.21.15.33:0/4287279592 >> 172.21.15.33:0/4287279592 conn(0x7fb8980794e0 msgr2=0x7fb8980798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.483+0000 7fb8a0641700 1 -- 172.21.15.33:0/4287279592 shutdown_connections 2024-05-15T13:57:55.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.483+0000 7fb8a0641700 1 -- 172.21.15.33:0/4287279592 wait complete. 2024-05-15T13:57:55.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.484+0000 7fb8a0641700 1 Processor -- start 2024-05-15T13:57:55.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.484+0000 7fb8a0641700 1 -- start start 2024-05-15T13:57:55.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.484+0000 7fb8a0641700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb89806bd10 0x7fb8981024b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:55.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.484+0000 7fb8a0641700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb89806c6d0 0x7fb8981029f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:55.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.484+0000 7fb8a0641700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb898109630 0x7fb898102f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:55.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.484+0000 7fb8a0641700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb898113740 con 0x7fb898109630 2024-05-15T13:57:55.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.484+0000 7fb8a0641700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb8981135c0 con 0x7fb89806c6d0 2024-05-15T13:57:55.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.484+0000 7fb8a0641700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb8981138c0 con 0x7fb89806bd10 2024-05-15T13:57:55.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.484+0000 7fb89ebde700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb898109630 0x7fb898102f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:55.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.485+0000 7fb89ebde700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb898109630 0x7fb898102f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52272/0 (socket says 172.21.15.33:52272) 2024-05-15T13:57:55.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.485+0000 7fb89ebde700 1 -- 172.21.15.33:0/780965608 learned_addr learned my addr 172.21.15.33:0/780965608 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:57:55.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.485+0000 7fb89dbdc700 1 --2- 172.21.15.33:0/780965608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb89806c6d0 0x7fb8981029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:55.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.485+0000 7fb89e3dd700 1 --2- 172.21.15.33:0/780965608 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb89806bd10 0x7fb8981024b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:55.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.485+0000 7fb89ebde700 1 -- 172.21.15.33:0/780965608 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb89806bd10 msgr2=0x7fb8981024b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:55.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.485+0000 7fb89ebde700 1 --2- 172.21.15.33:0/780965608 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb89806bd10 0x7fb8981024b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:55.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.485+0000 7fb89ebde700 1 -- 172.21.15.33:0/780965608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb89806c6d0 msgr2=0x7fb8981029f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:55.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.485+0000 7fb89ebde700 1 --2- 172.21.15.33:0/780965608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb89806c6d0 0x7fb8981029f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:55.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.485+0000 7fb89ebde700 1 -- 172.21.15.33:0/780965608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8981a7fd0 con 0x7fb898109630 2024-05-15T13:57:55.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.485+0000 7fb89ebde700 1 --2- 172.21.15.33:0/780965608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb898109630 0x7fb898102f30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb890009850 tx=0x7fb890007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:55.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.485+0000 7fb88f7fe700 1 -- 172.21.15.33:0/780965608 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89001f070 con 0x7fb898109630 2024-05-15T13:57:55.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.485+0000 7fb88f7fe700 1 -- 172.21.15.33:0/780965608 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb890005520 con 0x7fb898109630 2024-05-15T13:57:55.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.485+0000 7fb8a0641700 1 -- 172.21.15.33:0/780965608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8981a8260 con 0x7fb898109630 2024-05-15T13:57:55.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.485+0000 7fb88f7fe700 1 -- 172.21.15.33:0/780965608 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89001b870 con 0x7fb898109630 2024-05-15T13:57:55.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.485+0000 7fb8a0641700 1 -- 172.21.15.33:0/780965608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8981a8660 con 0x7fb898109630 2024-05-15T13:57:55.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.486+0000 7fb8a0641700 1 -- 172.21.15.33:0/780965608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb89806a230 con 0x7fb898109630 2024-05-15T13:57:55.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.488+0000 7fb88f7fe700 1 -- 172.21.15.33:0/780965608 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb890022070 con 0x7fb898109630 2024-05-15T13:57:55.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.488+0000 7fb88f7fe700 1 --2- 172.21.15.33:0/780965608 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fb8840b2910 0x7fb8840b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:55.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.489+0000 7fb89e3dd700 1 --2- 172.21.15.33:0/780965608 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fb8840b2910 0x7fb8840b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:55.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.489+0000 7fb88f7fe700 1 -- 172.21.15.33:0/780965608 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fb8900e3280 con 0x7fb898109630 2024-05-15T13:57:55.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.489+0000 7fb89e3dd700 1 --2- 172.21.15.33:0/780965608 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fb8840b2910 0x7fb8840b4dd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb894005fd0 tx=0x7fb89400af20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:55.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.492+0000 7fb88f7fe700 1 -- 172.21.15.33:0/780965608 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb8900ab080 con 0x7fb898109630 2024-05-15T13:57:55.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.761+0000 7fb8a0641700 1 -- 172.21.15.33:0/780965608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb89802e600 con 0x7fb898109630 2024-05-15T13:57:55.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.761+0000 7fb88f7fe700 1 -- 172.21.15.33:0/780965608 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11858 (secure 0 0 0) 0x7fb890009cb0 con 0x7fb898109630 2024-05-15T13:57:55.763 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:57:55.763 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2030117633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2030117633},{"type":"v1","addr":"172.21.15.33:6839","nonce":2030117633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1918979061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1918979061},{"type":"v1","addr":"172.21.15.33:6841","nonce":1918979061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T13:57:19.056417+0000","modified":"2024-05-15T13:57:38.201184+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24463,"mds_2":24461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/1629697841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1629697841},{"type":"v1","addr":"172.21.15.52:6837","nonce":1629697841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/3773839758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3773839758},{"type":"v1","addr":"172.21.15.81:6835","nonce":3773839758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.52:6839/2696044471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2696044471},{"type":"v1","addr":"172.21.15.52:6839","nonce":2696044471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.81:6839/1554395940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":1554395940},{"type":"v1","addr":"172.21.15.81:6839","nonce":1554395940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24478,24461,24463]},"id":1}]} 2024-05-15T13:57:55.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.764+0000 7fb8a0641700 1 -- 172.21.15.33:0/780965608 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fb8840b2910 msgr2=0x7fb8840b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:55.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.764+0000 7fb8a0641700 1 --2- 172.21.15.33:0/780965608 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fb8840b2910 0x7fb8840b4dd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb894005fd0 tx=0x7fb89400af20 comp rx=0 tx=0).stop 2024-05-15T13:57:55.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.764+0000 7fb8a0641700 1 -- 172.21.15.33:0/780965608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb898109630 msgr2=0x7fb898102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:55.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.764+0000 7fb8a0641700 1 --2- 172.21.15.33:0/780965608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb898109630 0x7fb898102f30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb890009850 tx=0x7fb890007ad0 comp rx=0 tx=0).stop 2024-05-15T13:57:55.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.764+0000 7fb8a0641700 1 -- 172.21.15.33:0/780965608 shutdown_connections 2024-05-15T13:57:55.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.764+0000 7fb8a0641700 1 --2- 172.21.15.33:0/780965608 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fb8840b2910 0x7fb8840b4dd0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:55.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.764+0000 7fb8a0641700 1 --2- 172.21.15.33:0/780965608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb898109630 0x7fb898102f30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:55.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.764+0000 7fb8a0641700 1 --2- 172.21.15.33:0/780965608 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb89806bd10 0x7fb8981024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:55.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.764+0000 7fb8a0641700 1 --2- 172.21.15.33:0/780965608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb89806c6d0 0x7fb8981029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:55.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.764+0000 7fb8a0641700 1 -- 172.21.15.33:0/780965608 >> 172.21.15.33:0/780965608 conn(0x7fb8980794e0 msgr2=0x7fb898100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:55.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.764+0000 7fb8a0641700 1 -- 172.21.15.33:0/780965608 shutdown_connections 2024-05-15T13:57:55.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:55.764+0000 7fb8a0641700 1 -- 172.21.15.33:0/780965608 wait complete. 2024-05-15T13:57:55.766 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 23 2024-05-15T13:57:56.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:56 smithi052 ceph-mon[36190]: pgmap v305: 97 pgs: 97 active+clean; 622 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.1 KiB/s wr, 7 op/s 2024-05-15T13:57:56.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:56 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/780965608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:57:56.356 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:56 smithi081 ceph-mon[36434]: pgmap v305: 97 pgs: 97 active+clean; 622 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.1 KiB/s wr, 7 op/s 2024-05-15T13:57:56.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:56 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/780965608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:57:56.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:56 smithi033 ceph-mon[31650]: pgmap v305: 97 pgs: 97 active+clean; 622 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.1 KiB/s wr, 7 op/s 2024-05-15T13:57:56.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:56 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/780965608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:57:56.478 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-15T13:57:56.737 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:57:58.129 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:58 smithi033 ceph-mon[31650]: pgmap v306: 97 pgs: 97 active+clean; 635 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-05-15T13:57:58.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:58 smithi052 ceph-mon[36190]: pgmap v306: 97 pgs: 97 active+clean; 635 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-05-15T13:57:58.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.342+0000 7ffbddd61700 1 -- 172.21.15.33:0/3820416807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbd81041d0 msgr2=0x7ffbd81081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:58.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.342+0000 7ffbddd61700 1 --2- 172.21.15.33:0/3820416807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbd81041d0 0x7ffbd81081e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ffbc00099c0 tx=0x7ffbc0009cd0 comp rx=0 tx=0).stop 2024-05-15T13:57:58.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.343+0000 7ffbddd61700 1 -- 172.21.15.33:0/3820416807 shutdown_connections 2024-05-15T13:57:58.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.343+0000 7ffbddd61700 1 --2- 172.21.15.33:0/3820416807 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbd8108910 0x7ffbd8110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:58.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.343+0000 7ffbddd61700 1 --2- 172.21.15.33:0/3820416807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffbd8103820 0x7ffbd8103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:58.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.343+0000 7ffbddd61700 1 --2- 172.21.15.33:0/3820416807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbd81041d0 0x7ffbd81081e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:58.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.343+0000 7ffbddd61700 1 -- 172.21.15.33:0/3820416807 >> 172.21.15.33:0/3820416807 conn(0x7ffbd807a6b0 msgr2=0x7ffbd807aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:58.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.343+0000 7ffbddd61700 1 -- 172.21.15.33:0/3820416807 shutdown_connections 2024-05-15T13:57:58.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.343+0000 7ffbddd61700 1 -- 172.21.15.33:0/3820416807 wait complete. 2024-05-15T13:57:58.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.343+0000 7ffbddd61700 1 Processor -- start 2024-05-15T13:57:58.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.343+0000 7ffbddd61700 1 -- start start 2024-05-15T13:57:58.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.344+0000 7ffbddd61700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbd8103820 0x7ffbd819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:58.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.344+0000 7ffbddd61700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbd81041d0 0x7ffbd819fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:58.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.344+0000 7ffbddd61700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffbd8108910 0x7ffbd8199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:58.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.344+0000 7ffbddd61700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd81136f0 con 0x7ffbd8103820 2024-05-15T13:57:58.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.344+0000 7ffbddd61700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd8113570 con 0x7ffbd8108910 2024-05-15T13:57:58.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.344+0000 7ffbddd61700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd8113870 con 0x7ffbd81041d0 2024-05-15T13:57:58.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.344+0000 7ffbd77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbd8103820 0x7ffbd819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:58.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.344+0000 7ffbd77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbd8103820 0x7ffbd819f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52288/0 (socket says 172.21.15.33:52288) 2024-05-15T13:57:58.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.344+0000 7ffbd77fe700 1 -- 172.21.15.33:0/178923661 learned_addr learned my addr 172.21.15.33:0/178923661 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:57:58.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.344+0000 7ffbd7fff700 1 --2- 172.21.15.33:0/178923661 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffbd8108910 0x7ffbd8199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:58.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.344+0000 7ffbceffd700 1 --2- 172.21.15.33:0/178923661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbd81041d0 0x7ffbd819fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:58.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.344+0000 7ffbd77fe700 1 -- 172.21.15.33:0/178923661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbd81041d0 msgr2=0x7ffbd819fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:58.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.344+0000 7ffbd77fe700 1 --2- 172.21.15.33:0/178923661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbd81041d0 0x7ffbd819fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:58.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.344+0000 7ffbd77fe700 1 -- 172.21.15.33:0/178923661 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffbd8108910 msgr2=0x7ffbd8199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:58.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.344+0000 7ffbd77fe700 1 --2- 172.21.15.33:0/178923661 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffbd8108910 0x7ffbd8199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:58.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.344+0000 7ffbd77fe700 1 -- 172.21.15.33:0/178923661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd819a070 con 0x7ffbd8103820 2024-05-15T13:57:58.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.344+0000 7ffbd7fff700 1 --2- 172.21.15.33:0/178923661 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffbd8108910 0x7ffbd8199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T13:57:58.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.344+0000 7ffbd77fe700 1 --2- 172.21.15.33:0/178923661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbd8103820 0x7ffbd819f750 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ffbc800c9b0 tx=0x7ffbc800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:58.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.344+0000 7ffbd57fa700 1 -- 172.21.15.33:0/178923661 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8007b00 con 0x7ffbd8103820 2024-05-15T13:57:58.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.344+0000 7ffbddd61700 1 -- 172.21.15.33:0/178923661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbd819a360 con 0x7ffbd8103820 2024-05-15T13:57:58.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.344+0000 7ffbd57fa700 1 -- 172.21.15.33:0/178923661 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbc8007ca0 con 0x7ffbd8103820 2024-05-15T13:57:58.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.344+0000 7ffbd57fa700 1 -- 172.21.15.33:0/178923661 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8018730 con 0x7ffbd8103820 2024-05-15T13:57:58.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.345+0000 7ffbddd61700 1 -- 172.21.15.33:0/178923661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbd81a6620 con 0x7ffbd8103820 2024-05-15T13:57:58.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.345+0000 7ffbddd61700 1 -- 172.21.15.33:0/178923661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbd806a230 con 0x7ffbd8103820 2024-05-15T13:57:58.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.347+0000 7ffbd57fa700 1 -- 172.21.15.33:0/178923661 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ffbc800f450 con 0x7ffbd8103820 2024-05-15T13:57:58.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.347+0000 7ffbd57fa700 1 --2- 172.21.15.33:0/178923661 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ffbc40b2910 0x7ffbc40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:57:58.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.348+0000 7ffbceffd700 1 --2- 172.21.15.33:0/178923661 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ffbc40b2910 0x7ffbc40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:57:58.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.348+0000 7ffbd57fa700 1 -- 172.21.15.33:0/178923661 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7ffbc80e2580 con 0x7ffbd8103820 2024-05-15T13:57:58.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.348+0000 7ffbceffd700 1 --2- 172.21.15.33:0/178923661 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ffbc40b2910 0x7ffbc40b4dd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0003800 tx=0x7ffbc0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:57:58.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.351+0000 7ffbd57fa700 1 -- 172.21.15.33:0/178923661 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffbc8014030 con 0x7ffbd8103820 2024-05-15T13:57:58.356 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:58 smithi081 ceph-mon[36434]: pgmap v306: 97 pgs: 97 active+clean; 635 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-05-15T13:57:58.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.571+0000 7ffbddd61700 1 -- 172.21.15.33:0/178923661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffbd819ae40 con 0x7ffbd8103820 2024-05-15T13:57:58.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.573+0000 7ffbd57fa700 1 -- 172.21.15.33:0/178923661 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22112 (secure 0 0 0) 0x7ffbc80aa380 con 0x7ffbd8103820 2024-05-15T13:57:58.575 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:57:58.575 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":80,"fsid":"bca78ca4-12c1-11ef-bc98-c7b262605968","created":"2024-05-15T13:49:10.327613+0000","modified":"2024-05-15T13:57:19.056451+0000","last_up_change":"2024-05-15T13:56:16.097711+0000","last_in_change":"2024-05-15T13:55:56.722143+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-15T13:52:32.970844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-15T13:57:12.324802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-15T13:57:15.898406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b434111-810f-450a-b63d-c97080e9f8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6803","nonce":887409130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6805","nonce":887409130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6809","nonce":887409130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6807","nonce":887409130}]},"public_addr":"172.21.15.33:6803/887409130","cluster_addr":"172.21.15.33:6805/887409130","heartbeat_back_addr":"172.21.15.33:6809/887409130","heartbeat_front_addr":"172.21.15.33:6807/887409130","state":["exists","up"]},{"osd":1,"uuid":"0b578d4e-a841-4b27-bc95-d6c3bae580fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6801","nonce":2794180003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6803","nonce":2794180003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6807","nonce":2794180003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6805","nonce":2794180003}]},"public_addr":"172.21.15.52:6801/2794180003","cluster_addr":"172.21.15.52:6803/2794180003","heartbeat_back_addr":"172.21.15.52:6807/2794180003","heartbeat_front_addr":"172.21.15.52:6805/2794180003","state":["exists","up"]},{"osd":2,"uuid":"0e9c9390-ed33-464b-8f39-bca75941e05e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6801","nonce":315610978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6803","nonce":315610978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6807","nonce":315610978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6805","nonce":315610978}]},"public_addr":"172.21.15.81:6801/315610978","cluster_addr":"172.21.15.81:6803/315610978","heartbeat_back_addr":"172.21.15.81:6807/315610978","heartbeat_front_addr":"172.21.15.81:6805/315610978","state":["exists","up"]},{"osd":3,"uuid":"ecdb24c1-fb15-42a5-9bae-436bbae9512a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6811","nonce":818533501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6813","nonce":818533501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6817","nonce":818533501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6815","nonce":818533501}]},"public_addr":"172.21.15.33:6811/818533501","cluster_addr":"172.21.15.33:6813/818533501","heartbeat_back_addr":"172.21.15.33:6817/818533501","heartbeat_front_addr":"172.21.15.33:6815/818533501","state":["exists","up"]},{"osd":4,"uuid":"706752a0-b17f-4563-a7a9-d5078bf0f3b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6809","nonce":3010133776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6811","nonce":3010133776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6815","nonce":3010133776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6813","nonce":3010133776}]},"public_addr":"172.21.15.52:6809/3010133776","cluster_addr":"172.21.15.52:6811/3010133776","heartbeat_back_addr":"172.21.15.52:6815/3010133776","heartbeat_front_addr":"172.21.15.52:6813/3010133776","state":["exists","up"]},{"osd":5,"uuid":"3d45ab97-f364-4903-974d-8acc961363d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6809","nonce":2132616297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6811","nonce":2132616297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6815","nonce":2132616297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6813","nonce":2132616297}]},"public_addr":"172.21.15.81:6809/2132616297","cluster_addr":"172.21.15.81:6811/2132616297","heartbeat_back_addr":"172.21.15.81:6815/2132616297","heartbeat_front_addr":"172.21.15.81:6813/2132616297","state":["exists","up"]},{"osd":6,"uuid":"3be524c5-dcf9-4742-b896-ea9a44a6b391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6819","nonce":1260691801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6821","nonce":1260691801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6825","nonce":1260691801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6823","nonce":1260691801}]},"public_addr":"172.21.15.33:6819/1260691801","cluster_addr":"172.21.15.33:6821/1260691801","heartbeat_back_addr":"172.21.15.33:6825/1260691801","heartbeat_front_addr":"172.21.15.33:6823/1260691801","state":["exists","up"]},{"osd":7,"uuid":"bda2f920-4f2c-4eed-8127-83c1973676e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6817","nonce":1654595813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6819","nonce":1654595813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6823","nonce":1654595813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6821","nonce":1654595813}]},"public_addr":"172.21.15.52:6817/1654595813","cluster_addr":"172.21.15.52:6819/1654595813","heartbeat_back_addr":"172.21.15.52:6823/1654595813","heartbeat_front_addr":"172.21.15.52:6821/1654595813","state":["exists","up"]},{"osd":8,"uuid":"550585b8-0df4-49ee-95c8-3d084744dbd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6817","nonce":3009661748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6819","nonce":3009661748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6823","nonce":3009661748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6821","nonce":3009661748}]},"public_addr":"172.21.15.81:6817/3009661748","cluster_addr":"172.21.15.81:6819/3009661748","heartbeat_back_addr":"172.21.15.81:6823/3009661748","heartbeat_front_addr":"172.21.15.81:6821/3009661748","state":["exists","up"]},{"osd":9,"uuid":"1d781846-26f3-4b23-b22c-c994a8509a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6827","nonce":2843220858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6829","nonce":2843220858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6833","nonce":2843220858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6831","nonce":2843220858}]},"public_addr":"172.21.15.33:6827/2843220858","cluster_addr":"172.21.15.33:6829/2843220858","heartbeat_back_addr":"172.21.15.33:6833/2843220858","heartbeat_front_addr":"172.21.15.33:6831/2843220858","state":["exists","up"]},{"osd":10,"uuid":"af58c940-fdc4-418b-a638-7dc44438b34c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6825","nonce":2422349831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6827","nonce":2422349831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6831","nonce":2422349831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6829","nonce":2422349831}]},"public_addr":"172.21.15.52:6825/2422349831","cluster_addr":"172.21.15.52:6827/2422349831","heartbeat_back_addr":"172.21.15.52:6831/2422349831","heartbeat_front_addr":"172.21.15.52:6829/2422349831","state":["exists","up"]},{"osd":11,"uuid":"ab84aaea-2086-4842-8bf4-628842fce46a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6825","nonce":326313090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6827","nonce":326313090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6831","nonce":326313090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6829","nonce":326313090}]},"public_addr":"172.21.15.81:6825/326313090","cluster_addr":"172.21.15.81:6827/326313090","heartbeat_back_addr":"172.21.15.81:6831/326313090","heartbeat_front_addr":"172.21.15.81:6829/326313090","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:51:41.493136+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:52:05.784773+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:52:29.769429+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:52:54.891620+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:53:19.412922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:53:43.921841+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:54:09.486732+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:54:34.064370+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:54:58.655304+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:55:24.548693+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:55:49.941191+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:56:14.919558+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/436121571":"2024-05-16T13:50:04.881656+0000","172.21.15.33:6801/2680375465":"2024-05-16T13:50:04.881656+0000","172.21.15.33:6800/2680375465":"2024-05-16T13:50:04.881656+0000","172.21.15.33:0/1022787179":"2024-05-16T13:49:37.564643+0000","172.21.15.33:0/3720711369":"2024-05-16T13:50:04.881656+0000","172.21.15.33:0/1282466838":"2024-05-16T13:49:37.564643+0000","172.21.15.33:0/3022248969":"2024-05-16T13:49:37.564643+0000","172.21.15.33:6801/3303495364":"2024-05-16T13:49:37.564643+0000","172.21.15.33:6800/3303495364":"2024-05-16T13:49:37.564643+0000","172.21.15.33:0/4282812692":"2024-05-16T13:50:04.881656+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-15T13:57:58.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.575+0000 7ffbddd61700 1 -- 172.21.15.33:0/178923661 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ffbc40b2910 msgr2=0x7ffbc40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:58.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.575+0000 7ffbddd61700 1 --2- 172.21.15.33:0/178923661 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ffbc40b2910 0x7ffbc40b4dd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0003800 tx=0x7ffbc0000bc0 comp rx=0 tx=0).stop 2024-05-15T13:57:58.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.575+0000 7ffbddd61700 1 -- 172.21.15.33:0/178923661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbd8103820 msgr2=0x7ffbd819f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:57:58.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.575+0000 7ffbddd61700 1 --2- 172.21.15.33:0/178923661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbd8103820 0x7ffbd819f750 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ffbc800c9b0 tx=0x7ffbc800cd70 comp rx=0 tx=0).stop 2024-05-15T13:57:58.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.575+0000 7ffbddd61700 1 -- 172.21.15.33:0/178923661 shutdown_connections 2024-05-15T13:57:58.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.575+0000 7ffbddd61700 1 --2- 172.21.15.33:0/178923661 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ffbc40b2910 0x7ffbc40b4dd0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:58.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.575+0000 7ffbddd61700 1 --2- 172.21.15.33:0/178923661 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffbd8108910 0x7ffbd8199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:58.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.575+0000 7ffbddd61700 1 --2- 172.21.15.33:0/178923661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbd8103820 0x7ffbd819f750 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:58.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.575+0000 7ffbddd61700 1 --2- 172.21.15.33:0/178923661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbd81041d0 0x7ffbd819fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:57:58.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.575+0000 7ffbddd61700 1 -- 172.21.15.33:0/178923661 >> 172.21.15.33:0/178923661 conn(0x7ffbd807a6b0 msgr2=0x7ffbd8101590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:57:58.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.575+0000 7ffbddd61700 1 -- 172.21.15.33:0/178923661 shutdown_connections 2024-05-15T13:57:58.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:57:58.576+0000 7ffbddd61700 1 -- 172.21.15.33:0/178923661 wait complete. 2024-05-15T13:57:59.183 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:57:59 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/178923661' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T13:57:59.212 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-15T13:57:59.213 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-15T13:57:59.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:57:59 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/178923661' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T13:57:59.356 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:57:59 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/178923661' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T13:57:59.462 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:58:00.190 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:00 smithi033 ceph-mon[31650]: pgmap v307: 97 pgs: 97 active+clean; 635 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-05-15T13:58:00.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:00 smithi052 ceph-mon[36190]: pgmap v307: 97 pgs: 97 active+clean; 635 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-05-15T13:58:00.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:00 smithi081 ceph-mon[36434]: pgmap v307: 97 pgs: 97 active+clean; 635 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-05-15T13:58:01.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.018+0000 7f2f296eb700 1 -- 172.21.15.33:0/3308919652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f2406bd10 msgr2=0x7f2f2406c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:01.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.018+0000 7f2f296eb700 1 --2- 172.21.15.33:0/3308919652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f2406bd10 0x7f2f2406c190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2f180099c0 tx=0x7f2f18009cd0 comp rx=0 tx=0).stop 2024-05-15T13:58:01.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.018+0000 7f2f296eb700 1 -- 172.21.15.33:0/3308919652 shutdown_connections 2024-05-15T13:58:01.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.018+0000 7f2f296eb700 1 --2- 172.21.15.33:0/3308919652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f2406c6d0 0x7f2f24110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:01.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.018+0000 7f2f296eb700 1 --2- 172.21.15.33:0/3308919652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f24109630 0x7f2f24109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:01.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.018+0000 7f2f296eb700 1 --2- 172.21.15.33:0/3308919652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f2406bd10 0x7f2f2406c190 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:01.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.018+0000 7f2f296eb700 1 -- 172.21.15.33:0/3308919652 >> 172.21.15.33:0/3308919652 conn(0x7f2f240794e0 msgr2=0x7f2f240798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:01.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.018+0000 7f2f296eb700 1 -- 172.21.15.33:0/3308919652 shutdown_connections 2024-05-15T13:58:01.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.018+0000 7f2f296eb700 1 -- 172.21.15.33:0/3308919652 wait complete. 2024-05-15T13:58:01.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.018+0000 7f2f296eb700 1 Processor -- start 2024-05-15T13:58:01.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.018+0000 7f2f296eb700 1 -- start start 2024-05-15T13:58:01.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.019+0000 7f2f296eb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f2406bd10 0x7f2f2419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:01.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.019+0000 7f2f296eb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f2406c6d0 0x7f2f2419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:01.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.019+0000 7f2f296eb700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f24109630 0x7f2f24199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:01.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.019+0000 7f2f296eb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2f24113790 con 0x7f2f2406c6d0 2024-05-15T13:58:01.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.019+0000 7f2f296eb700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2f24113610 con 0x7f2f2406bd10 2024-05-15T13:58:01.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.019+0000 7f2f296eb700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2f24113910 con 0x7f2f24109630 2024-05-15T13:58:01.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.019+0000 7f2f227fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f2406c6d0 0x7f2f2419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:01.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.019+0000 7f2f227fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f2406c6d0 0x7f2f2419fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52300/0 (socket says 172.21.15.33:52300) 2024-05-15T13:58:01.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.019+0000 7f2f227fc700 1 -- 172.21.15.33:0/624059959 learned_addr learned my addr 172.21.15.33:0/624059959 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:58:01.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.019+0000 7f2f237fe700 1 --2- 172.21.15.33:0/624059959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f24109630 0x7f2f24199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:01.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.019+0000 7f2f22ffd700 1 --2- 172.21.15.33:0/624059959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f2406bd10 0x7f2f2419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:01.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.019+0000 7f2f227fc700 1 -- 172.21.15.33:0/624059959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f24109630 msgr2=0x7f2f24199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:01.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.019+0000 7f2f227fc700 1 --2- 172.21.15.33:0/624059959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f24109630 0x7f2f24199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:01.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.019+0000 7f2f227fc700 1 -- 172.21.15.33:0/624059959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f2406bd10 msgr2=0x7f2f2419f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:01.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.019+0000 7f2f227fc700 1 --2- 172.21.15.33:0/624059959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f2406bd10 0x7f2f2419f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:01.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.019+0000 7f2f227fc700 1 -- 172.21.15.33:0/624059959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f2419a0f0 con 0x7f2f2406c6d0 2024-05-15T13:58:01.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.019+0000 7f2f227fc700 1 --2- 172.21.15.33:0/624059959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f2406c6d0 0x7f2f2419fd10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800b580 tx=0x7f2f180047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:01.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.020+0000 7f2f0bfff700 1 -- 172.21.15.33:0/624059959 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f18027070 con 0x7f2f2406c6d0 2024-05-15T13:58:01.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.020+0000 7f2f296eb700 1 -- 172.21.15.33:0/624059959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f2419a380 con 0x7f2f2406c6d0 2024-05-15T13:58:01.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.020+0000 7f2f0bfff700 1 -- 172.21.15.33:0/624059959 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f180040a0 con 0x7f2f2406c6d0 2024-05-15T13:58:01.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.020+0000 7f2f0bfff700 1 -- 172.21.15.33:0/624059959 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1800f460 con 0x7f2f2406c6d0 2024-05-15T13:58:01.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.020+0000 7f2f296eb700 1 -- 172.21.15.33:0/624059959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f241a66a0 con 0x7f2f2406c6d0 2024-05-15T13:58:01.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.021+0000 7f2f296eb700 1 -- 172.21.15.33:0/624059959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f2406a230 con 0x7f2f2406c6d0 2024-05-15T13:58:01.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.022+0000 7f2f0bfff700 1 -- 172.21.15.33:0/624059959 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f2f1802a070 con 0x7f2f2406c6d0 2024-05-15T13:58:01.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.023+0000 7f2f0bfff700 1 --2- 172.21.15.33:0/624059959 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f2f0c0b2910 0x7f2f0c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:01.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.023+0000 7f2f22ffd700 1 --2- 172.21.15.33:0/624059959 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f2f0c0b2910 0x7f2f0c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:01.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.023+0000 7f2f0bfff700 1 -- 172.21.15.33:0/624059959 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f2f180e3760 con 0x7f2f2406c6d0 2024-05-15T13:58:01.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.023+0000 7f2f22ffd700 1 --2- 172.21.15.33:0/624059959 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f2f0c0b2910 0x7f2f0c0b4dd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2f14005ea0 tx=0x7f2f14005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:01.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.026+0000 7f2f0bfff700 1 -- 172.21.15.33:0/624059959 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2f18031070 con 0x7f2f2406c6d0 2024-05-15T13:58:01.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.237+0000 7f2f296eb700 1 -- 172.21.15.33:0/624059959 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2f2419aec0 con 0x7f2f0c0b2910 2024-05-15T13:58:01.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.245+0000 7f2f0bfff700 1 -- 172.21.15.33:0/624059959 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258678 (secure 0 0 0) 0x7f2f2419aec0 con 0x7f2f0c0b2910 2024-05-15T13:58:01.251 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:58:01.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.249+0000 7f2f296eb700 1 -- 172.21.15.33:0/624059959 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f2f0c0b2910 msgr2=0x7f2f0c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:01.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.249+0000 7f2f296eb700 1 --2- 172.21.15.33:0/624059959 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f2f0c0b2910 0x7f2f0c0b4dd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2f14005ea0 tx=0x7f2f14005e30 comp rx=0 tx=0).stop 2024-05-15T13:58:01.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.249+0000 7f2f296eb700 1 -- 172.21.15.33:0/624059959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f2406c6d0 msgr2=0x7f2f2419fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:01.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.249+0000 7f2f296eb700 1 --2- 172.21.15.33:0/624059959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f2406c6d0 0x7f2f2419fd10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800b580 tx=0x7f2f180047b0 comp rx=0 tx=0).stop 2024-05-15T13:58:01.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.249+0000 7f2f296eb700 1 -- 172.21.15.33:0/624059959 shutdown_connections 2024-05-15T13:58:01.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.249+0000 7f2f296eb700 1 --2- 172.21.15.33:0/624059959 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f2f0c0b2910 0x7f2f0c0b4dd0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:01.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.249+0000 7f2f296eb700 1 --2- 172.21.15.33:0/624059959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f24109630 0x7f2f24199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:01.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.249+0000 7f2f296eb700 1 --2- 172.21.15.33:0/624059959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f2406bd10 0x7f2f2419f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:01.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.249+0000 7f2f296eb700 1 --2- 172.21.15.33:0/624059959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f2406c6d0 0x7f2f2419fd10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:01.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.249+0000 7f2f296eb700 1 -- 172.21.15.33:0/624059959 >> 172.21.15.33:0/624059959 conn(0x7f2f240794e0 msgr2=0x7f2f24100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:01.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.249+0000 7f2f296eb700 1 -- 172.21.15.33:0/624059959 shutdown_connections 2024-05-15T13:58:01.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:01.249+0000 7f2f296eb700 1 -- 172.21.15.33:0/624059959 wait complete. 2024-05-15T13:58:01.254 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-05-15T13:58:01.870 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-05-15T13:58:00.974667+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649946,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":238,"num_read_kb":210,"num_write":284,"num_write_kb":2255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":229,"ondisk_log_size":229,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373824,"kb_used_data":12192,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124568000,"statfs":{"total":1151940427776,"available":1151557632000,"internally_reserved":0,"allocated":12484608,"data_stored":6929730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32739,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":63,"num_read_kb":63,"num_write":21,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002901"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.388808+0000","last_change":"2024-05-15T13:57:17.382728+0000","last_active":"2024-05-15T13:57:19.388808+0000","last_peered":"2024-05-15T13:57:19.388808+0000","last_clean":"2024-05-15T13:57:19.388808+0000","last_became_active":"2024-05-15T13:57:17.382315+0000","last_became_peered":"2024-05-15T13:57:17.382315+0000","last_unstale":"2024-05-15T13:57:19.388808+0000","last_undegraded":"2024-05-15T13:57:19.388808+0000","last_fullsized":"2024-05-15T13:57:19.388808+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:03:53.137916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.386758+0000","last_change":"2024-05-15T13:57:17.390880+0000","last_active":"2024-05-15T13:57:19.386758+0000","last_peered":"2024-05-15T13:57:19.386758+0000","last_clean":"2024-05-15T13:57:19.386758+0000","last_became_active":"2024-05-15T13:57:17.390560+0000","last_became_peered":"2024-05-15T13:57:17.390560+0000","last_unstale":"2024-05-15T13:57:19.386758+0000","last_undegraded":"2024-05-15T13:57:19.386758+0000","last_fullsized":"2024-05-15T13:57:19.386758+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:59:33.615670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.386468+0000","last_change":"2024-05-15T13:57:17.386495+0000","last_active":"2024-05-15T13:57:19.386468+0000","last_peered":"2024-05-15T13:57:19.386468+0000","last_clean":"2024-05-15T13:57:19.386468+0000","last_became_active":"2024-05-15T13:57:17.386183+0000","last_became_peered":"2024-05-15T13:57:17.386183+0000","last_unstale":"2024-05-15T13:57:19.386468+0000","last_undegraded":"2024-05-15T13:57:19.386468+0000","last_fullsized":"2024-05-15T13:57:19.386468+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:36:13.202922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.386591+0000","last_change":"2024-05-15T13:57:17.385437+0000","last_active":"2024-05-15T13:57:19.386591+0000","last_peered":"2024-05-15T13:57:19.386591+0000","last_clean":"2024-05-15T13:57:19.386591+0000","last_became_active":"2024-05-15T13:57:17.384524+0000","last_became_peered":"2024-05-15T13:57:17.384524+0000","last_unstale":"2024-05-15T13:57:19.386591+0000","last_undegraded":"2024-05-15T13:57:19.386591+0000","last_fullsized":"2024-05-15T13:57:19.386591+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:45:17.207819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.387251+0000","last_change":"2024-05-15T13:57:17.387859+0000","last_active":"2024-05-15T13:57:19.387251+0000","last_peered":"2024-05-15T13:57:19.387251+0000","last_clean":"2024-05-15T13:57:19.387251+0000","last_became_active":"2024-05-15T13:57:17.387491+0000","last_became_peered":"2024-05-15T13:57:17.387491+0000","last_unstale":"2024-05-15T13:57:19.387251+0000","last_undegraded":"2024-05-15T13:57:19.387251+0000","last_fullsized":"2024-05-15T13:57:19.387251+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:43:08.212926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.386676+0000","last_change":"2024-05-15T13:57:17.382289+0000","last_active":"2024-05-15T13:57:19.386676+0000","last_peered":"2024-05-15T13:57:19.386676+0000","last_clean":"2024-05-15T13:57:19.386676+0000","last_became_active":"2024-05-15T13:57:17.380494+0000","last_became_peered":"2024-05-15T13:57:17.380494+0000","last_unstale":"2024-05-15T13:57:19.386676+0000","last_undegraded":"2024-05-15T13:57:19.386676+0000","last_fullsized":"2024-05-15T13:57:19.386676+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:11:37.097034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.387417+0000","last_change":"2024-05-15T13:57:17.391924+0000","last_active":"2024-05-15T13:57:19.387417+0000","last_peered":"2024-05-15T13:57:19.387417+0000","last_clean":"2024-05-15T13:57:19.387417+0000","last_became_active":"2024-05-15T13:57:17.391589+0000","last_became_peered":"2024-05-15T13:57:17.391589+0000","last_unstale":"2024-05-15T13:57:19.387417+0000","last_undegraded":"2024-05-15T13:57:19.387417+0000","last_fullsized":"2024-05-15T13:57:19.387417+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:46:30.213590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.375746+0000","last_change":"2024-05-15T13:57:17.380969+0000","last_active":"2024-05-15T13:57:19.375746+0000","last_peered":"2024-05-15T13:57:19.375746+0000","last_clean":"2024-05-15T13:57:19.375746+0000","last_became_active":"2024-05-15T13:57:17.380660+0000","last_became_peered":"2024-05-15T13:57:17.380660+0000","last_unstale":"2024-05-15T13:57:19.375746+0000","last_undegraded":"2024-05-15T13:57:19.375746+0000","last_fullsized":"2024-05-15T13:57:19.375746+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:01:41.790323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.376718+0000","last_change":"2024-05-15T13:57:17.394943+0000","last_active":"2024-05-15T13:57:19.376718+0000","last_peered":"2024-05-15T13:57:19.376718+0000","last_clean":"2024-05-15T13:57:19.376718+0000","last_became_active":"2024-05-15T13:57:17.394571+0000","last_became_peered":"2024-05-15T13:57:17.394571+0000","last_unstale":"2024-05-15T13:57:19.376718+0000","last_undegraded":"2024-05-15T13:57:19.376718+0000","last_fullsized":"2024-05-15T13:57:19.376718+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:39:26.063096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.386159+0000","last_change":"2024-05-15T13:57:17.386012+0000","last_active":"2024-05-15T13:57:19.386159+0000","last_peered":"2024-05-15T13:57:19.386159+0000","last_clean":"2024-05-15T13:57:19.386159+0000","last_became_active":"2024-05-15T13:57:17.385636+0000","last_became_peered":"2024-05-15T13:57:17.385636+0000","last_unstale":"2024-05-15T13:57:19.386159+0000","last_undegraded":"2024-05-15T13:57:19.386159+0000","last_fullsized":"2024-05-15T13:57:19.386159+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:12:08.465959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.387750+0000","last_change":"2024-05-15T13:57:17.389162+0000","last_active":"2024-05-15T13:57:19.387750+0000","last_peered":"2024-05-15T13:57:19.387750+0000","last_clean":"2024-05-15T13:57:19.387750+0000","last_became_active":"2024-05-15T13:57:17.388503+0000","last_became_peered":"2024-05-15T13:57:17.388503+0000","last_unstale":"2024-05-15T13:57:19.387750+0000","last_undegraded":"2024-05-15T13:57:19.387750+0000","last_fullsized":"2024-05-15T13:57:19.387750+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:24:54.354445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.386380+0000","last_change":"2024-05-15T13:57:17.381689+0000","last_active":"2024-05-15T13:57:19.386380+0000","last_peered":"2024-05-15T13:57:19.386380+0000","last_clean":"2024-05-15T13:57:19.386380+0000","last_became_active":"2024-05-15T13:57:17.381376+0000","last_became_peered":"2024-05-15T13:57:17.381376+0000","last_unstale":"2024-05-15T13:57:19.386380+0000","last_undegraded":"2024-05-15T13:57:19.386380+0000","last_fullsized":"2024-05-15T13:57:19.386380+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:03:52.652934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.387845+0000","last_change":"2024-05-15T13:57:17.382214+0000","last_active":"2024-05-15T13:57:19.387845+0000","last_peered":"2024-05-15T13:57:19.387845+0000","last_clean":"2024-05-15T13:57:19.387845+0000","last_became_active":"2024-05-15T13:57:17.381851+0000","last_became_peered":"2024-05-15T13:57:17.381851+0000","last_unstale":"2024-05-15T13:57:19.387845+0000","last_undegraded":"2024-05-15T13:57:19.387845+0000","last_fullsized":"2024-05-15T13:57:19.387845+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:18:39.854918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.386353+0000","last_change":"2024-05-15T13:57:17.390985+0000","last_active":"2024-05-15T13:57:19.386353+0000","last_peered":"2024-05-15T13:57:19.386353+0000","last_clean":"2024-05-15T13:57:19.386353+0000","last_became_active":"2024-05-15T13:57:17.390633+0000","last_became_peered":"2024-05-15T13:57:17.390633+0000","last_unstale":"2024-05-15T13:57:19.386353+0000","last_undegraded":"2024-05-15T13:57:19.386353+0000","last_fullsized":"2024-05-15T13:57:19.386353+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:32:47.177507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.386402+0000","last_change":"2024-05-15T13:57:17.383943+0000","last_active":"2024-05-15T13:57:19.386402+0000","last_peered":"2024-05-15T13:57:19.386402+0000","last_clean":"2024-05-15T13:57:19.386402+0000","last_became_active":"2024-05-15T13:57:17.383622+0000","last_became_peered":"2024-05-15T13:57:17.383622+0000","last_unstale":"2024-05-15T13:57:19.386402+0000","last_undegraded":"2024-05-15T13:57:19.386402+0000","last_fullsized":"2024-05-15T13:57:19.386402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T00:30:23.249813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.375983+0000","last_change":"2024-05-15T13:57:17.390789+0000","last_active":"2024-05-15T13:57:19.375983+0000","last_peered":"2024-05-15T13:57:19.375983+0000","last_clean":"2024-05-15T13:57:19.375983+0000","last_became_active":"2024-05-15T13:57:17.390466+0000","last_became_peered":"2024-05-15T13:57:17.390466+0000","last_unstale":"2024-05-15T13:57:19.375983+0000","last_undegraded":"2024-05-15T13:57:19.375983+0000","last_fullsized":"2024-05-15T13:57:19.375983+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:07:24.746862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.389750+0000","last_change":"2024-05-15T13:57:17.384080+0000","last_active":"2024-05-15T13:57:19.389750+0000","last_peered":"2024-05-15T13:57:19.389750+0000","last_clean":"2024-05-15T13:57:19.389750+0000","last_became_active":"2024-05-15T13:57:17.383731+0000","last_became_peered":"2024-05-15T13:57:17.383731+0000","last_unstale":"2024-05-15T13:57:19.389750+0000","last_undegraded":"2024-05-15T13:57:19.389750+0000","last_fullsized":"2024-05-15T13:57:19.389750+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:36:22.874337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.386008+0000","last_change":"2024-05-15T13:57:17.382209+0000","last_active":"2024-05-15T13:57:19.386008+0000","last_peered":"2024-05-15T13:57:19.386008+0000","last_clean":"2024-05-15T13:57:19.386008+0000","last_became_active":"2024-05-15T13:57:17.381737+0000","last_became_peered":"2024-05-15T13:57:17.381737+0000","last_unstale":"2024-05-15T13:57:19.386008+0000","last_undegraded":"2024-05-15T13:57:19.386008+0000","last_fullsized":"2024-05-15T13:57:19.386008+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:38:53.913745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.386594+0000","last_change":"2024-05-15T13:57:17.384855+0000","last_active":"2024-05-15T13:57:19.386594+0000","last_peered":"2024-05-15T13:57:19.386594+0000","last_clean":"2024-05-15T13:57:19.386594+0000","last_became_active":"2024-05-15T13:57:17.384387+0000","last_became_peered":"2024-05-15T13:57:17.384387+0000","last_unstale":"2024-05-15T13:57:19.386594+0000","last_undegraded":"2024-05-15T13:57:19.386594+0000","last_fullsized":"2024-05-15T13:57:19.386594+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:32:05.363047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:32.493386+0000","last_change":"2024-05-15T13:57:14.256057+0000","last_active":"2024-05-15T13:57:32.493386+0000","last_peered":"2024-05-15T13:57:32.493386+0000","last_clean":"2024-05-15T13:57:32.493386+0000","last_became_active":"2024-05-15T13:57:14.255413+0000","last_became_peered":"2024-05-15T13:57:14.255413+0000","last_unstale":"2024-05-15T13:57:32.493386+0000","last_undegraded":"2024-05-15T13:57:32.493386+0000","last_fullsized":"2024-05-15T13:57:32.493386+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:13:43.433634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.375890+0000","last_change":"2024-05-15T13:57:17.390548+0000","last_active":"2024-05-15T13:57:19.375890+0000","last_peered":"2024-05-15T13:57:19.375890+0000","last_clean":"2024-05-15T13:57:19.375890+0000","last_became_active":"2024-05-15T13:57:17.390144+0000","last_became_peered":"2024-05-15T13:57:17.390144+0000","last_unstale":"2024-05-15T13:57:19.375890+0000","last_undegraded":"2024-05-15T13:57:19.375890+0000","last_fullsized":"2024-05-15T13:57:19.375890+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:28:54.889994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.386622+0000","last_change":"2024-05-15T13:57:14.535704+0000","last_active":"2024-05-15T13:57:19.386622+0000","last_peered":"2024-05-15T13:57:19.386622+0000","last_clean":"2024-05-15T13:57:19.386622+0000","last_became_active":"2024-05-15T13:57:14.535356+0000","last_became_peered":"2024-05-15T13:57:14.535356+0000","last_unstale":"2024-05-15T13:57:19.386622+0000","last_undegraded":"2024-05-15T13:57:19.386622+0000","last_fullsized":"2024-05-15T13:57:19.386622+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:16:21.545507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.387723+0000","last_change":"2024-05-15T13:57:17.383482+0000","last_active":"2024-05-15T13:57:19.387723+0000","last_peered":"2024-05-15T13:57:19.387723+0000","last_clean":"2024-05-15T13:57:19.387723+0000","last_became_active":"2024-05-15T13:57:17.382973+0000","last_became_peered":"2024-05-15T13:57:17.382973+0000","last_unstale":"2024-05-15T13:57:19.387723+0000","last_undegraded":"2024-05-15T13:57:19.387723+0000","last_fullsized":"2024-05-15T13:57:19.387723+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:31:08.937166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:32.492420+0000","last_change":"2024-05-15T13:57:14.259627+0000","last_active":"2024-05-15T13:57:32.492420+0000","last_peered":"2024-05-15T13:57:32.492420+0000","last_clean":"2024-05-15T13:57:32.492420+0000","last_became_active":"2024-05-15T13:57:14.259282+0000","last_became_peered":"2024-05-15T13:57:14.259282+0000","last_unstale":"2024-05-15T13:57:32.492420+0000","last_undegraded":"2024-05-15T13:57:32.492420+0000","last_fullsized":"2024-05-15T13:57:32.492420+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:01:24.304755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.376207+0000","last_change":"2024-05-15T13:57:17.388720+0000","last_active":"2024-05-15T13:57:19.376207+0000","last_peered":"2024-05-15T13:57:19.376207+0000","last_clean":"2024-05-15T13:57:19.376207+0000","last_became_active":"2024-05-15T13:57:17.388386+0000","last_became_peered":"2024-05-15T13:57:17.388386+0000","last_unstale":"2024-05-15T13:57:19.376207+0000","last_undegraded":"2024-05-15T13:57:19.376207+0000","last_fullsized":"2024-05-15T13:57:19.376207+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:02:40.543701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.391755+0000","last_change":"2024-05-15T13:57:14.259064+0000","last_active":"2024-05-15T13:57:19.391755+0000","last_peered":"2024-05-15T13:57:19.391755+0000","last_clean":"2024-05-15T13:57:19.391755+0000","last_became_active":"2024-05-15T13:57:14.258683+0000","last_became_peered":"2024-05-15T13:57:14.258683+0000","last_unstale":"2024-05-15T13:57:19.391755+0000","last_undegraded":"2024-05-15T13:57:19.391755+0000","last_fullsized":"2024-05-15T13:57:19.391755+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:12:43.108066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.375571+0000","last_change":"2024-05-15T13:57:17.380929+0000","last_active":"2024-05-15T13:57:19.375571+0000","last_peered":"2024-05-15T13:57:19.375571+0000","last_clean":"2024-05-15T13:57:19.375571+0000","last_became_active":"2024-05-15T13:57:17.380599+0000","last_became_peered":"2024-05-15T13:57:17.380599+0000","last_unstale":"2024-05-15T13:57:19.375571+0000","last_undegraded":"2024-05-15T13:57:19.375571+0000","last_fullsized":"2024-05-15T13:57:19.375571+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:19:55.368088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":53,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:55.401396+0000","last_change":"2024-05-15T13:57:14.260410+0000","last_active":"2024-05-15T13:57:55.401396+0000","last_peered":"2024-05-15T13:57:55.401396+0000","last_clean":"2024-05-15T13:57:55.401396+0000","last_became_active":"2024-05-15T13:57:14.260076+0000","last_became_peered":"2024-05-15T13:57:14.260076+0000","last_unstale":"2024-05-15T13:57:55.401396+0000","last_undegraded":"2024-05-15T13:57:55.401396+0000","last_fullsized":"2024-05-15T13:57:55.401396+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T00:27:03.436237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.386657+0000","last_change":"2024-05-15T13:57:17.394475+0000","last_active":"2024-05-15T13:57:19.386657+0000","last_peered":"2024-05-15T13:57:19.386657+0000","last_clean":"2024-05-15T13:57:19.386657+0000","last_became_active":"2024-05-15T13:57:17.393733+0000","last_became_peered":"2024-05-15T13:57:17.393733+0000","last_unstale":"2024-05-15T13:57:19.386657+0000","last_undegraded":"2024-05-15T13:57:19.386657+0000","last_fullsized":"2024-05-15T13:57:19.386657+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:55:06.198633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:32.493660+0000","last_change":"2024-05-15T13:57:14.256420+0000","last_active":"2024-05-15T13:57:32.493660+0000","last_peered":"2024-05-15T13:57:32.493660+0000","last_clean":"2024-05-15T13:57:32.493660+0000","last_became_active":"2024-05-15T13:57:14.255722+0000","last_became_peered":"2024-05-15T13:57:14.255722+0000","last_unstale":"2024-05-15T13:57:32.493660+0000","last_undegraded":"2024-05-15T13:57:32.493660+0000","last_fullsized":"2024-05-15T13:57:32.493660+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:21:30.499632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.375599+0000","last_change":"2024-05-15T13:57:17.383858+0000","last_active":"2024-05-15T13:57:19.375599+0000","last_peered":"2024-05-15T13:57:19.375599+0000","last_clean":"2024-05-15T13:57:19.375599+0000","last_became_active":"2024-05-15T13:57:17.383522+0000","last_became_peered":"2024-05-15T13:57:17.383522+0000","last_unstale":"2024-05-15T13:57:19.375599+0000","last_undegraded":"2024-05-15T13:57:19.375599+0000","last_fullsized":"2024-05-15T13:57:19.375599+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:54:59.802226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:32.493377+0000","last_change":"2024-05-15T13:57:14.256192+0000","last_active":"2024-05-15T13:57:32.493377+0000","last_peered":"2024-05-15T13:57:32.493377+0000","last_clean":"2024-05-15T13:57:32.493377+0000","last_became_active":"2024-05-15T13:57:14.255839+0000","last_became_peered":"2024-05-15T13:57:14.255839+0000","last_unstale":"2024-05-15T13:57:32.493377+0000","last_undegraded":"2024-05-15T13:57:32.493377+0000","last_fullsized":"2024-05-15T13:57:32.493377+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:13:26.071403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.375530+0000","last_change":"2024-05-15T13:57:17.386246+0000","last_active":"2024-05-15T13:57:19.375530+0000","last_peered":"2024-05-15T13:57:19.375530+0000","last_clean":"2024-05-15T13:57:19.375530+0000","last_became_active":"2024-05-15T13:57:17.385828+0000","last_became_peered":"2024-05-15T13:57:17.385828+0000","last_unstale":"2024-05-15T13:57:19.375530+0000","last_undegraded":"2024-05-15T13:57:19.375530+0000","last_fullsized":"2024-05-15T13:57:19.375530+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T00:24:38.656469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.375622+0000","last_change":"2024-05-15T13:57:14.259953+0000","last_active":"2024-05-15T13:57:19.375622+0000","last_peered":"2024-05-15T13:57:19.375622+0000","last_clean":"2024-05-15T13:57:19.375622+0000","last_became_active":"2024-05-15T13:57:14.259632+0000","last_became_peered":"2024-05-15T13:57:14.259632+0000","last_unstale":"2024-05-15T13:57:19.375622+0000","last_undegraded":"2024-05-15T13:57:19.375622+0000","last_fullsized":"2024-05-15T13:57:19.375622+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:30:28.361903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.385309+0000","last_change":"2024-05-15T13:57:17.382211+0000","last_active":"2024-05-15T13:57:19.385309+0000","last_peered":"2024-05-15T13:57:19.385309+0000","last_clean":"2024-05-15T13:57:19.385309+0000","last_became_active":"2024-05-15T13:57:17.381904+0000","last_became_peered":"2024-05-15T13:57:17.381904+0000","last_unstale":"2024-05-15T13:57:19.385309+0000","last_undegraded":"2024-05-15T13:57:19.385309+0000","last_fullsized":"2024-05-15T13:57:19.385309+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:53:49.250866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:54.399503+0000","last_change":"2024-05-15T13:57:14.254691+0000","last_active":"2024-05-15T13:57:54.399503+0000","last_peered":"2024-05-15T13:57:54.399503+0000","last_clean":"2024-05-15T13:57:54.399503+0000","last_became_active":"2024-05-15T13:57:14.254233+0000","last_became_peered":"2024-05-15T13:57:14.254233+0000","last_unstale":"2024-05-15T13:57:54.399503+0000","last_undegraded":"2024-05-15T13:57:54.399503+0000","last_fullsized":"2024-05-15T13:57:54.399503+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:41:31.719402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.376549+0000","last_change":"2024-05-15T13:57:17.382251+0000","last_active":"2024-05-15T13:57:19.376549+0000","last_peered":"2024-05-15T13:57:19.376549+0000","last_clean":"2024-05-15T13:57:19.376549+0000","last_became_active":"2024-05-15T13:57:17.381158+0000","last_became_peered":"2024-05-15T13:57:17.381158+0000","last_unstale":"2024-05-15T13:57:19.376549+0000","last_undegraded":"2024-05-15T13:57:19.376549+0000","last_fullsized":"2024-05-15T13:57:19.376549+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:24:55.971079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:31.485285+0000","last_change":"2024-05-15T13:57:14.258350+0000","last_active":"2024-05-15T13:57:31.485285+0000","last_peered":"2024-05-15T13:57:31.485285+0000","last_clean":"2024-05-15T13:57:31.485285+0000","last_became_active":"2024-05-15T13:57:14.257982+0000","last_became_peered":"2024-05-15T13:57:14.257982+0000","last_unstale":"2024-05-15T13:57:31.485285+0000","last_undegraded":"2024-05-15T13:57:31.485285+0000","last_fullsized":"2024-05-15T13:57:31.485285+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:50:07.053401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.387078+0000","last_change":"2024-05-15T13:57:17.392114+0000","last_active":"2024-05-15T13:57:19.387078+0000","last_peered":"2024-05-15T13:57:19.387078+0000","last_clean":"2024-05-15T13:57:19.387078+0000","last_became_active":"2024-05-15T13:57:17.391732+0000","last_became_peered":"2024-05-15T13:57:17.391732+0000","last_unstale":"2024-05-15T13:57:19.387078+0000","last_undegraded":"2024-05-15T13:57:19.387078+0000","last_fullsized":"2024-05-15T13:57:19.387078+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:28:02.304910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:31.484162+0000","last_change":"2024-05-15T13:57:14.535381+0000","last_active":"2024-05-15T13:57:31.484162+0000","last_peered":"2024-05-15T13:57:31.484162+0000","last_clean":"2024-05-15T13:57:31.484162+0000","last_became_active":"2024-05-15T13:57:14.534931+0000","last_became_peered":"2024-05-15T13:57:14.534931+0000","last_unstale":"2024-05-15T13:57:31.484162+0000","last_undegraded":"2024-05-15T13:57:31.484162+0000","last_fullsized":"2024-05-15T13:57:31.484162+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:13:04.785139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.376437+0000","last_change":"2024-05-15T13:57:17.386103+0000","last_active":"2024-05-15T13:57:19.376437+0000","last_peered":"2024-05-15T13:57:19.376437+0000","last_clean":"2024-05-15T13:57:19.376437+0000","last_became_active":"2024-05-15T13:57:17.385738+0000","last_became_peered":"2024-05-15T13:57:17.385738+0000","last_unstale":"2024-05-15T13:57:19.376437+0000","last_undegraded":"2024-05-15T13:57:19.376437+0000","last_fullsized":"2024-05-15T13:57:19.376437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:42:56.136164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'25","reported_seq":96,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:55.401980+0000","last_change":"2024-05-15T13:57:14.258903+0000","last_active":"2024-05-15T13:57:55.401980+0000","last_peered":"2024-05-15T13:57:55.401980+0000","last_clean":"2024-05-15T13:57:55.401980+0000","last_became_active":"2024-05-15T13:57:14.258453+0000","last_became_peered":"2024-05-15T13:57:14.258453+0000","last_unstale":"2024-05-15T13:57:55.401980+0000","last_undegraded":"2024-05-15T13:57:55.401980+0000","last_fullsized":"2024-05-15T13:57:55.401980+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:49:33.004534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54952,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":77,"num_write":26,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.387018+0000","last_change":"2024-05-15T13:57:17.389717+0000","last_active":"2024-05-15T13:57:19.387018+0000","last_peered":"2024-05-15T13:57:19.387018+0000","last_clean":"2024-05-15T13:57:19.387018+0000","last_became_active":"2024-05-15T13:57:17.389188+0000","last_became_peered":"2024-05-15T13:57:17.389188+0000","last_unstale":"2024-05-15T13:57:19.387018+0000","last_undegraded":"2024-05-15T13:57:19.387018+0000","last_fullsized":"2024-05-15T13:57:19.387018+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:40:26.827507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:32.493344+0000","last_change":"2024-05-15T13:57:14.256007+0000","last_active":"2024-05-15T13:57:32.493344+0000","last_peered":"2024-05-15T13:57:32.493344+0000","last_clean":"2024-05-15T13:57:32.493344+0000","last_became_active":"2024-05-15T13:57:14.255656+0000","last_became_peered":"2024-05-15T13:57:14.255656+0000","last_unstale":"2024-05-15T13:57:32.493344+0000","last_undegraded":"2024-05-15T13:57:32.493344+0000","last_fullsized":"2024-05-15T13:57:32.493344+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:14:46.880981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.388234+0000","last_change":"2024-05-15T13:57:17.389971+0000","last_active":"2024-05-15T13:57:19.388234+0000","last_peered":"2024-05-15T13:57:19.388234+0000","last_clean":"2024-05-15T13:57:19.388234+0000","last_became_active":"2024-05-15T13:57:17.389633+0000","last_became_peered":"2024-05-15T13:57:17.389633+0000","last_unstale":"2024-05-15T13:57:19.388234+0000","last_undegraded":"2024-05-15T13:57:19.388234+0000","last_fullsized":"2024-05-15T13:57:19.388234+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:25:10.971086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:31.486543+0000","last_change":"2024-05-15T13:57:14.258086+0000","last_active":"2024-05-15T13:57:31.486543+0000","last_peered":"2024-05-15T13:57:31.486543+0000","last_clean":"2024-05-15T13:57:31.486543+0000","last_became_active":"2024-05-15T13:57:14.257736+0000","last_became_peered":"2024-05-15T13:57:14.257736+0000","last_unstale":"2024-05-15T13:57:31.486543+0000","last_undegraded":"2024-05-15T13:57:31.486543+0000","last_fullsized":"2024-05-15T13:57:31.486543+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T00:51:07.431725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.387924+0000","last_change":"2024-05-15T13:57:17.390804+0000","last_active":"2024-05-15T13:57:19.387924+0000","last_peered":"2024-05-15T13:57:19.387924+0000","last_clean":"2024-05-15T13:57:19.387924+0000","last_became_active":"2024-05-15T13:57:17.390393+0000","last_became_peered":"2024-05-15T13:57:17.390393+0000","last_unstale":"2024-05-15T13:57:19.387924+0000","last_undegraded":"2024-05-15T13:57:19.387924+0000","last_fullsized":"2024-05-15T13:57:19.387924+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:42:56.204275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'4","reported_seq":51,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:55.401341+0000","last_change":"2024-05-15T13:57:14.261911+0000","last_active":"2024-05-15T13:57:55.401341+0000","last_peered":"2024-05-15T13:57:55.401341+0000","last_clean":"2024-05-15T13:57:55.401341+0000","last_became_active":"2024-05-15T13:57:14.261506+0000","last_became_peered":"2024-05-15T13:57:14.261506+0000","last_unstale":"2024-05-15T13:57:55.401341+0000","last_undegraded":"2024-05-15T13:57:55.401341+0000","last_fullsized":"2024-05-15T13:57:55.401341+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:18:21.021651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.388723+0000","last_change":"2024-05-15T13:57:17.395074+0000","last_active":"2024-05-15T13:57:19.388723+0000","last_peered":"2024-05-15T13:57:19.388723+0000","last_clean":"2024-05-15T13:57:19.388723+0000","last_became_active":"2024-05-15T13:57:17.394712+0000","last_became_peered":"2024-05-15T13:57:17.394712+0000","last_unstale":"2024-05-15T13:57:19.388723+0000","last_undegraded":"2024-05-15T13:57:19.388723+0000","last_fullsized":"2024-05-15T13:57:19.388723+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:58:25.385949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:32.494405+0000","last_change":"2024-05-15T13:57:14.535178+0000","last_active":"2024-05-15T13:57:32.494405+0000","last_peered":"2024-05-15T13:57:32.494405+0000","last_clean":"2024-05-15T13:57:32.494405+0000","last_became_active":"2024-05-15T13:57:14.534804+0000","last_became_peered":"2024-05-15T13:57:14.534804+0000","last_unstale":"2024-05-15T13:57:32.494405+0000","last_undegraded":"2024-05-15T13:57:32.494405+0000","last_fullsized":"2024-05-15T13:57:32.494405+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:53:57.558436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.375384+0000","last_change":"2024-05-15T13:57:17.376895+0000","last_active":"2024-05-15T13:57:19.375384+0000","last_peered":"2024-05-15T13:57:19.375384+0000","last_clean":"2024-05-15T13:57:19.375384+0000","last_became_active":"2024-05-15T13:57:17.376499+0000","last_became_peered":"2024-05-15T13:57:17.376499+0000","last_unstale":"2024-05-15T13:57:19.375384+0000","last_undegraded":"2024-05-15T13:57:19.375384+0000","last_fullsized":"2024-05-15T13:57:19.375384+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:39:41.707151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:33.489634+0000","last_change":"2024-05-15T13:57:14.258155+0000","last_active":"2024-05-15T13:57:33.489634+0000","last_peered":"2024-05-15T13:57:33.489634+0000","last_clean":"2024-05-15T13:57:33.489634+0000","last_became_active":"2024-05-15T13:57:14.257733+0000","last_became_peered":"2024-05-15T13:57:14.257733+0000","last_unstale":"2024-05-15T13:57:33.489634+0000","last_undegraded":"2024-05-15T13:57:33.489634+0000","last_fullsized":"2024-05-15T13:57:33.489634+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:33:45.659936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.389281+0000","last_change":"2024-05-15T13:57:17.390640+0000","last_active":"2024-05-15T13:57:19.389281+0000","last_peered":"2024-05-15T13:57:19.389281+0000","last_clean":"2024-05-15T13:57:19.389281+0000","last_became_active":"2024-05-15T13:57:17.390293+0000","last_became_peered":"2024-05-15T13:57:17.390293+0000","last_unstale":"2024-05-15T13:57:19.389281+0000","last_undegraded":"2024-05-15T13:57:19.389281+0000","last_fullsized":"2024-05-15T13:57:19.389281+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:08:18.907391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.385810+0000","last_change":"2024-05-15T13:57:14.535321+0000","last_active":"2024-05-15T13:57:19.385810+0000","last_peered":"2024-05-15T13:57:19.385810+0000","last_clean":"2024-05-15T13:57:19.385810+0000","last_became_active":"2024-05-15T13:57:14.534794+0000","last_became_peered":"2024-05-15T13:57:14.534794+0000","last_unstale":"2024-05-15T13:57:19.385810+0000","last_undegraded":"2024-05-15T13:57:19.385810+0000","last_fullsized":"2024-05-15T13:57:19.385810+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:27:07.087671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:21.352686+0000","last_change":"2024-05-15T13:57:17.381249+0000","last_active":"2024-05-15T13:57:21.352686+0000","last_peered":"2024-05-15T13:57:21.352686+0000","last_clean":"2024-05-15T13:57:21.352686+0000","last_became_active":"2024-05-15T13:57:17.380474+0000","last_became_peered":"2024-05-15T13:57:17.380474+0000","last_unstale":"2024-05-15T13:57:21.352686+0000","last_undegraded":"2024-05-15T13:57:21.352686+0000","last_fullsized":"2024-05-15T13:57:21.352686+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:49:00.341085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.388331+0000","last_change":"2024-05-15T13:57:14.257635+0000","last_active":"2024-05-15T13:57:19.388331+0000","last_peered":"2024-05-15T13:57:19.388331+0000","last_clean":"2024-05-15T13:57:19.388331+0000","last_became_active":"2024-05-15T13:57:14.257156+0000","last_became_peered":"2024-05-15T13:57:14.257156+0000","last_unstale":"2024-05-15T13:57:19.388331+0000","last_undegraded":"2024-05-15T13:57:19.388331+0000","last_fullsized":"2024-05-15T13:57:19.388331+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:11:48.742053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.385643+0000","last_change":"2024-05-15T13:57:17.391067+0000","last_active":"2024-05-15T13:57:19.385643+0000","last_peered":"2024-05-15T13:57:19.385643+0000","last_clean":"2024-05-15T13:57:19.385643+0000","last_became_active":"2024-05-15T13:57:17.390763+0000","last_became_peered":"2024-05-15T13:57:17.390763+0000","last_unstale":"2024-05-15T13:57:19.385643+0000","last_undegraded":"2024-05-15T13:57:19.385643+0000","last_fullsized":"2024-05-15T13:57:19.385643+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:28:46.506307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.375743+0000","last_change":"2024-05-15T13:57:17.391816+0000","last_active":"2024-05-15T13:57:19.375743+0000","last_peered":"2024-05-15T13:57:19.375743+0000","last_clean":"2024-05-15T13:57:19.375743+0000","last_became_active":"2024-05-15T13:57:17.389374+0000","last_became_peered":"2024-05-15T13:57:17.389374+0000","last_unstale":"2024-05-15T13:57:19.375743+0000","last_undegraded":"2024-05-15T13:57:19.375743+0000","last_fullsized":"2024-05-15T13:57:19.375743+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:48:31.756133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":49,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:56.402381+0000","last_change":"2024-05-15T13:57:14.534946+0000","last_active":"2024-05-15T13:57:56.402381+0000","last_peered":"2024-05-15T13:57:56.402381+0000","last_clean":"2024-05-15T13:57:56.402381+0000","last_became_active":"2024-05-15T13:57:14.534535+0000","last_became_peered":"2024-05-15T13:57:14.534535+0000","last_unstale":"2024-05-15T13:57:56.402381+0000","last_undegraded":"2024-05-15T13:57:56.402381+0000","last_fullsized":"2024-05-15T13:57:56.402381+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:27:20.526800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.376133+0000","last_change":"2024-05-15T13:57:17.391637+0000","last_active":"2024-05-15T13:57:19.376133+0000","last_peered":"2024-05-15T13:57:19.376133+0000","last_clean":"2024-05-15T13:57:19.376133+0000","last_became_active":"2024-05-15T13:57:17.391271+0000","last_became_peered":"2024-05-15T13:57:17.391271+0000","last_unstale":"2024-05-15T13:57:19.376133+0000","last_undegraded":"2024-05-15T13:57:19.376133+0000","last_fullsized":"2024-05-15T13:57:19.376133+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:22:06.398387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:35.368235+0000","last_change":"2024-05-15T13:57:14.258510+0000","last_active":"2024-05-15T13:57:35.368235+0000","last_peered":"2024-05-15T13:57:35.368235+0000","last_clean":"2024-05-15T13:57:35.368235+0000","last_became_active":"2024-05-15T13:57:14.257544+0000","last_became_peered":"2024-05-15T13:57:14.257544+0000","last_unstale":"2024-05-15T13:57:35.368235+0000","last_undegraded":"2024-05-15T13:57:35.368235+0000","last_fullsized":"2024-05-15T13:57:35.368235+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:12:27.253931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":95,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.388889+0000","last_change":"2024-05-15T13:55:53.940355+0000","last_active":"2024-05-15T13:57:19.388889+0000","last_peered":"2024-05-15T13:57:19.388889+0000","last_clean":"2024-05-15T13:57:19.388889+0000","last_became_active":"2024-05-15T13:55:53.934638+0000","last_became_peered":"2024-05-15T13:55:53.934638+0000","last_unstale":"2024-05-15T13:57:19.388889+0000","last_undegraded":"2024-05-15T13:57:19.388889+0000","last_fullsized":"2024-05-15T13:57:19.388889+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:52:33.556435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:52:33.556435+0000","last_clean_scrub_stamp":"2024-05-15T13:52:33.556435+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:02:55.332569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.386872+0000","last_change":"2024-05-15T13:57:17.382394+0000","last_active":"2024-05-15T13:57:19.386872+0000","last_peered":"2024-05-15T13:57:19.386872+0000","last_clean":"2024-05-15T13:57:19.386872+0000","last_became_active":"2024-05-15T13:57:17.382090+0000","last_became_peered":"2024-05-15T13:57:17.382090+0000","last_unstale":"2024-05-15T13:57:19.386872+0000","last_undegraded":"2024-05-15T13:57:19.386872+0000","last_fullsized":"2024-05-15T13:57:19.386872+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:01:34.653216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:32.492852+0000","last_change":"2024-05-15T13:57:14.261549+0000","last_active":"2024-05-15T13:57:32.492852+0000","last_peered":"2024-05-15T13:57:32.492852+0000","last_clean":"2024-05-15T13:57:32.492852+0000","last_became_active":"2024-05-15T13:57:14.261165+0000","last_became_peered":"2024-05-15T13:57:14.261165+0000","last_unstale":"2024-05-15T13:57:32.492852+0000","last_undegraded":"2024-05-15T13:57:32.492852+0000","last_fullsized":"2024-05-15T13:57:32.492852+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:04:39.574440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:31.483634+0000","last_change":"2024-05-15T13:57:14.535309+0000","last_active":"2024-05-15T13:57:31.483634+0000","last_peered":"2024-05-15T13:57:31.483634+0000","last_clean":"2024-05-15T13:57:31.483634+0000","last_became_active":"2024-05-15T13:57:14.534938+0000","last_became_peered":"2024-05-15T13:57:14.534938+0000","last_unstale":"2024-05-15T13:57:31.483634+0000","last_undegraded":"2024-05-15T13:57:31.483634+0000","last_fullsized":"2024-05-15T13:57:31.483634+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:34:53.373149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.387317+0000","last_change":"2024-05-15T13:57:17.388272+0000","last_active":"2024-05-15T13:57:19.387317+0000","last_peered":"2024-05-15T13:57:19.387317+0000","last_clean":"2024-05-15T13:57:19.387317+0000","last_became_active":"2024-05-15T13:57:17.387849+0000","last_became_peered":"2024-05-15T13:57:17.387849+0000","last_unstale":"2024-05-15T13:57:19.387317+0000","last_undegraded":"2024-05-15T13:57:19.387317+0000","last_fullsized":"2024-05-15T13:57:19.387317+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:47:47.323566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'10","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:54.387709+0000","last_change":"2024-05-15T13:57:14.256161+0000","last_active":"2024-05-15T13:57:54.387709+0000","last_peered":"2024-05-15T13:57:54.387709+0000","last_clean":"2024-05-15T13:57:54.387709+0000","last_became_active":"2024-05-15T13:57:14.255707+0000","last_became_peered":"2024-05-15T13:57:14.255707+0000","last_unstale":"2024-05-15T13:57:54.387709+0000","last_undegraded":"2024-05-15T13:57:54.387709+0000","last_fullsized":"2024-05-15T13:57:54.387709+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:24:57.200029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.376114+0000","last_change":"2024-05-15T13:57:17.389522+0000","last_active":"2024-05-15T13:57:19.376114+0000","last_peered":"2024-05-15T13:57:19.376114+0000","last_clean":"2024-05-15T13:57:19.376114+0000","last_became_active":"2024-05-15T13:57:17.389177+0000","last_became_peered":"2024-05-15T13:57:17.389177+0000","last_unstale":"2024-05-15T13:57:19.376114+0000","last_undegraded":"2024-05-15T13:57:19.376114+0000","last_fullsized":"2024-05-15T13:57:19.376114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:23:35.716934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:31.485225+0000","last_change":"2024-05-15T13:57:14.254300+0000","last_active":"2024-05-15T13:57:31.485225+0000","last_peered":"2024-05-15T13:57:31.485225+0000","last_clean":"2024-05-15T13:57:31.485225+0000","last_became_active":"2024-05-15T13:57:14.253741+0000","last_became_peered":"2024-05-15T13:57:14.253741+0000","last_unstale":"2024-05-15T13:57:31.485225+0000","last_undegraded":"2024-05-15T13:57:31.485225+0000","last_fullsized":"2024-05-15T13:57:31.485225+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:25:34.018159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.386993+0000","last_change":"2024-05-15T13:57:17.380688+0000","last_active":"2024-05-15T13:57:19.386993+0000","last_peered":"2024-05-15T13:57:19.386993+0000","last_clean":"2024-05-15T13:57:19.386993+0000","last_became_active":"2024-05-15T13:57:17.380379+0000","last_became_peered":"2024-05-15T13:57:17.380379+0000","last_unstale":"2024-05-15T13:57:19.386993+0000","last_undegraded":"2024-05-15T13:57:19.386993+0000","last_fullsized":"2024-05-15T13:57:19.386993+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:32:09.190750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:31.485217+0000","last_change":"2024-05-15T13:57:14.534775+0000","last_active":"2024-05-15T13:57:31.485217+0000","last_peered":"2024-05-15T13:57:31.485217+0000","last_clean":"2024-05-15T13:57:31.485217+0000","last_became_active":"2024-05-15T13:57:14.534440+0000","last_became_peered":"2024-05-15T13:57:14.534440+0000","last_unstale":"2024-05-15T13:57:31.485217+0000","last_undegraded":"2024-05-15T13:57:31.485217+0000","last_fullsized":"2024-05-15T13:57:31.485217+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:21:49.047187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.386481+0000","last_change":"2024-05-15T13:57:17.396252+0000","last_active":"2024-05-15T13:57:19.386481+0000","last_peered":"2024-05-15T13:57:19.386481+0000","last_clean":"2024-05-15T13:57:19.386481+0000","last_became_active":"2024-05-15T13:57:17.395853+0000","last_became_peered":"2024-05-15T13:57:17.395853+0000","last_unstale":"2024-05-15T13:57:19.386481+0000","last_undegraded":"2024-05-15T13:57:19.386481+0000","last_fullsized":"2024-05-15T13:57:19.386481+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:12:54.924053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:31.485720+0000","last_change":"2024-05-15T13:57:14.258411+0000","last_active":"2024-05-15T13:57:31.485720+0000","last_peered":"2024-05-15T13:57:31.485720+0000","last_clean":"2024-05-15T13:57:31.485720+0000","last_became_active":"2024-05-15T13:57:14.257847+0000","last_became_peered":"2024-05-15T13:57:14.257847+0000","last_unstale":"2024-05-15T13:57:31.485720+0000","last_undegraded":"2024-05-15T13:57:31.485720+0000","last_fullsized":"2024-05-15T13:57:31.485720+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:49:32.368351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.388726+0000","last_change":"2024-05-15T13:57:17.388916+0000","last_active":"2024-05-15T13:57:19.388726+0000","last_peered":"2024-05-15T13:57:19.388726+0000","last_clean":"2024-05-15T13:57:19.388726+0000","last_became_active":"2024-05-15T13:57:17.388603+0000","last_became_peered":"2024-05-15T13:57:17.388603+0000","last_unstale":"2024-05-15T13:57:19.388726+0000","last_undegraded":"2024-05-15T13:57:19.388726+0000","last_fullsized":"2024-05-15T13:57:19.388726+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:38:46.650393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.386040+0000","last_change":"2024-05-15T13:57:14.256801+0000","last_active":"2024-05-15T13:57:19.386040+0000","last_peered":"2024-05-15T13:57:19.386040+0000","last_clean":"2024-05-15T13:57:19.386040+0000","last_became_active":"2024-05-15T13:57:14.256476+0000","last_became_peered":"2024-05-15T13:57:14.256476+0000","last_unstale":"2024-05-15T13:57:19.386040+0000","last_undegraded":"2024-05-15T13:57:19.386040+0000","last_fullsized":"2024-05-15T13:57:19.386040+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:22:34.495756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.376000+0000","last_change":"2024-05-15T13:57:17.380927+0000","last_active":"2024-05-15T13:57:19.376000+0000","last_peered":"2024-05-15T13:57:19.376000+0000","last_clean":"2024-05-15T13:57:19.376000+0000","last_became_active":"2024-05-15T13:57:17.380602+0000","last_became_peered":"2024-05-15T13:57:17.380602+0000","last_unstale":"2024-05-15T13:57:19.376000+0000","last_undegraded":"2024-05-15T13:57:19.376000+0000","last_fullsized":"2024-05-15T13:57:19.376000+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:37:13.920968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:31.485300+0000","last_change":"2024-05-15T13:57:14.257954+0000","last_active":"2024-05-15T13:57:31.485300+0000","last_peered":"2024-05-15T13:57:31.485300+0000","last_clean":"2024-05-15T13:57:31.485300+0000","last_became_active":"2024-05-15T13:57:14.257601+0000","last_became_peered":"2024-05-15T13:57:14.257601+0000","last_unstale":"2024-05-15T13:57:31.485300+0000","last_undegraded":"2024-05-15T13:57:31.485300+0000","last_fullsized":"2024-05-15T13:57:31.485300+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:47:14.811264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.388483+0000","last_change":"2024-05-15T13:57:17.385227+0000","last_active":"2024-05-15T13:57:19.388483+0000","last_peered":"2024-05-15T13:57:19.388483+0000","last_clean":"2024-05-15T13:57:19.388483+0000","last_became_active":"2024-05-15T13:57:17.384899+0000","last_became_peered":"2024-05-15T13:57:17.384899+0000","last_unstale":"2024-05-15T13:57:19.388483+0000","last_undegraded":"2024-05-15T13:57:19.388483+0000","last_fullsized":"2024-05-15T13:57:19.388483+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:04:29.162392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:32.494186+0000","last_change":"2024-05-15T13:57:14.535266+0000","last_active":"2024-05-15T13:57:32.494186+0000","last_peered":"2024-05-15T13:57:32.494186+0000","last_clean":"2024-05-15T13:57:32.494186+0000","last_became_active":"2024-05-15T13:57:14.534892+0000","last_became_peered":"2024-05-15T13:57:14.534892+0000","last_unstale":"2024-05-15T13:57:32.494186+0000","last_undegraded":"2024-05-15T13:57:32.494186+0000","last_fullsized":"2024-05-15T13:57:32.494186+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:49:50.934517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.388185+0000","last_change":"2024-05-15T13:57:17.390330+0000","last_active":"2024-05-15T13:57:19.388185+0000","last_peered":"2024-05-15T13:57:19.388185+0000","last_clean":"2024-05-15T13:57:19.388185+0000","last_became_active":"2024-05-15T13:57:17.390000+0000","last_became_peered":"2024-05-15T13:57:17.390000+0000","last_unstale":"2024-05-15T13:57:19.388185+0000","last_undegraded":"2024-05-15T13:57:19.388185+0000","last_fullsized":"2024-05-15T13:57:19.388185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:52:11.361549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.389721+0000","last_change":"2024-05-15T13:57:14.258174+0000","last_active":"2024-05-15T13:57:19.389721+0000","last_peered":"2024-05-15T13:57:19.389721+0000","last_clean":"2024-05-15T13:57:19.389721+0000","last_became_active":"2024-05-15T13:57:14.257763+0000","last_became_peered":"2024-05-15T13:57:14.257763+0000","last_unstale":"2024-05-15T13:57:19.389721+0000","last_undegraded":"2024-05-15T13:57:19.389721+0000","last_fullsized":"2024-05-15T13:57:19.389721+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:34:31.573845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:20.812172+0000","last_change":"2024-05-15T13:57:17.391964+0000","last_active":"2024-05-15T13:57:20.812172+0000","last_peered":"2024-05-15T13:57:20.812172+0000","last_clean":"2024-05-15T13:57:20.812172+0000","last_became_active":"2024-05-15T13:57:17.391517+0000","last_became_peered":"2024-05-15T13:57:17.391517+0000","last_unstale":"2024-05-15T13:57:20.812172+0000","last_undegraded":"2024-05-15T13:57:20.812172+0000","last_fullsized":"2024-05-15T13:57:20.812172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:54:36.883563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.390779+0000","last_change":"2024-05-15T13:57:14.256311+0000","last_active":"2024-05-15T13:57:19.390779+0000","last_peered":"2024-05-15T13:57:19.390779+0000","last_clean":"2024-05-15T13:57:19.390779+0000","last_became_active":"2024-05-15T13:57:14.255798+0000","last_became_peered":"2024-05-15T13:57:14.255798+0000","last_unstale":"2024-05-15T13:57:19.390779+0000","last_undegraded":"2024-05-15T13:57:19.390779+0000","last_fullsized":"2024-05-15T13:57:19.390779+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T00:52:50.996261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.388337+0000","last_change":"2024-05-15T13:57:17.389518+0000","last_active":"2024-05-15T13:57:19.388337+0000","last_peered":"2024-05-15T13:57:19.388337+0000","last_clean":"2024-05-15T13:57:19.388337+0000","last_became_active":"2024-05-15T13:57:17.388905+0000","last_became_peered":"2024-05-15T13:57:17.388905+0000","last_unstale":"2024-05-15T13:57:19.388337+0000","last_undegraded":"2024-05-15T13:57:19.388337+0000","last_fullsized":"2024-05-15T13:57:19.388337+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:00:51.109880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.388878+0000","last_change":"2024-05-15T13:57:17.394127+0000","last_active":"2024-05-15T13:57:19.388878+0000","last_peered":"2024-05-15T13:57:19.388878+0000","last_clean":"2024-05-15T13:57:19.388878+0000","last_became_active":"2024-05-15T13:57:17.393771+0000","last_became_peered":"2024-05-15T13:57:17.393771+0000","last_unstale":"2024-05-15T13:57:19.388878+0000","last_undegraded":"2024-05-15T13:57:19.388878+0000","last_fullsized":"2024-05-15T13:57:19.388878+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:41:33.491001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.386115+0000","last_change":"2024-05-15T13:57:17.392451+0000","last_active":"2024-05-15T13:57:19.386115+0000","last_peered":"2024-05-15T13:57:19.386115+0000","last_clean":"2024-05-15T13:57:19.386115+0000","last_became_active":"2024-05-15T13:57:17.392099+0000","last_became_peered":"2024-05-15T13:57:17.392099+0000","last_unstale":"2024-05-15T13:57:19.386115+0000","last_undegraded":"2024-05-15T13:57:19.386115+0000","last_fullsized":"2024-05-15T13:57:19.386115+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:00:14.774543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.385798+0000","last_change":"2024-05-15T13:57:17.385525+0000","last_active":"2024-05-15T13:57:19.385798+0000","last_peered":"2024-05-15T13:57:19.385798+0000","last_clean":"2024-05-15T13:57:19.385798+0000","last_became_active":"2024-05-15T13:57:17.384497+0000","last_became_peered":"2024-05-15T13:57:17.384497+0000","last_unstale":"2024-05-15T13:57:19.385798+0000","last_undegraded":"2024-05-15T13:57:19.385798+0000","last_fullsized":"2024-05-15T13:57:19.385798+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:50:39.259339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.387360+0000","last_change":"2024-05-15T13:57:17.379673+0000","last_active":"2024-05-15T13:57:19.387360+0000","last_peered":"2024-05-15T13:57:19.387360+0000","last_clean":"2024-05-15T13:57:19.387360+0000","last_became_active":"2024-05-15T13:57:17.379342+0000","last_became_peered":"2024-05-15T13:57:17.379342+0000","last_unstale":"2024-05-15T13:57:19.387360+0000","last_undegraded":"2024-05-15T13:57:19.387360+0000","last_fullsized":"2024-05-15T13:57:19.387360+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:26:43.982972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.387486+0000","last_change":"2024-05-15T13:57:17.376757+0000","last_active":"2024-05-15T13:57:19.387486+0000","last_peered":"2024-05-15T13:57:19.387486+0000","last_clean":"2024-05-15T13:57:19.387486+0000","last_became_active":"2024-05-15T13:57:17.376428+0000","last_became_peered":"2024-05-15T13:57:17.376428+0000","last_unstale":"2024-05-15T13:57:19.387486+0000","last_undegraded":"2024-05-15T13:57:19.387486+0000","last_fullsized":"2024-05-15T13:57:19.387486+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:55:47.739650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.389798+0000","last_change":"2024-05-15T13:57:17.390939+0000","last_active":"2024-05-15T13:57:19.389798+0000","last_peered":"2024-05-15T13:57:19.389798+0000","last_clean":"2024-05-15T13:57:19.389798+0000","last_became_active":"2024-05-15T13:57:17.390281+0000","last_became_peered":"2024-05-15T13:57:17.390281+0000","last_unstale":"2024-05-15T13:57:19.389798+0000","last_undegraded":"2024-05-15T13:57:19.389798+0000","last_fullsized":"2024-05-15T13:57:19.389798+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:02:22.077307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.387077+0000","last_change":"2024-05-15T13:57:17.392009+0000","last_active":"2024-05-15T13:57:19.387077+0000","last_peered":"2024-05-15T13:57:19.387077+0000","last_clean":"2024-05-15T13:57:19.387077+0000","last_became_active":"2024-05-15T13:57:17.391059+0000","last_became_peered":"2024-05-15T13:57:17.391059+0000","last_unstale":"2024-05-15T13:57:19.387077+0000","last_undegraded":"2024-05-15T13:57:19.387077+0000","last_fullsized":"2024-05-15T13:57:19.387077+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:51:46.787790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.375857+0000","last_change":"2024-05-15T13:57:17.383872+0000","last_active":"2024-05-15T13:57:19.375857+0000","last_peered":"2024-05-15T13:57:19.375857+0000","last_clean":"2024-05-15T13:57:19.375857+0000","last_became_active":"2024-05-15T13:57:17.383526+0000","last_became_peered":"2024-05-15T13:57:17.383526+0000","last_unstale":"2024-05-15T13:57:19.375857+0000","last_undegraded":"2024-05-15T13:57:19.375857+0000","last_fullsized":"2024-05-15T13:57:19.375857+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:43:39.343557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.375379+0000","last_change":"2024-05-15T13:57:17.377530+0000","last_active":"2024-05-15T13:57:19.375379+0000","last_peered":"2024-05-15T13:57:19.375379+0000","last_clean":"2024-05-15T13:57:19.375379+0000","last_became_active":"2024-05-15T13:57:17.377237+0000","last_became_peered":"2024-05-15T13:57:17.377237+0000","last_unstale":"2024-05-15T13:57:19.375379+0000","last_undegraded":"2024-05-15T13:57:19.375379+0000","last_fullsized":"2024-05-15T13:57:19.375379+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:29:02.173018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.375535+0000","last_change":"2024-05-15T13:57:17.386208+0000","last_active":"2024-05-15T13:57:19.375535+0000","last_peered":"2024-05-15T13:57:19.375535+0000","last_clean":"2024-05-15T13:57:19.375535+0000","last_became_active":"2024-05-15T13:57:17.385853+0000","last_became_peered":"2024-05-15T13:57:17.385853+0000","last_unstale":"2024-05-15T13:57:19.375535+0000","last_undegraded":"2024-05-15T13:57:19.375535+0000","last_fullsized":"2024-05-15T13:57:19.375535+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:17:06.124653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.376252+0000","last_change":"2024-05-15T13:57:17.390074+0000","last_active":"2024-05-15T13:57:19.376252+0000","last_peered":"2024-05-15T13:57:19.376252+0000","last_clean":"2024-05-15T13:57:19.376252+0000","last_became_active":"2024-05-15T13:57:17.389765+0000","last_became_peered":"2024-05-15T13:57:17.389765+0000","last_unstale":"2024-05-15T13:57:19.376252+0000","last_undegraded":"2024-05-15T13:57:19.376252+0000","last_fullsized":"2024-05-15T13:57:19.376252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:49:51.321180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.388016+0000","last_change":"2024-05-15T13:57:17.389180+0000","last_active":"2024-05-15T13:57:19.388016+0000","last_peered":"2024-05-15T13:57:19.388016+0000","last_clean":"2024-05-15T13:57:19.388016+0000","last_became_active":"2024-05-15T13:57:17.388731+0000","last_became_peered":"2024-05-15T13:57:17.388731+0000","last_unstale":"2024-05-15T13:57:19.388016+0000","last_undegraded":"2024-05-15T13:57:19.388016+0000","last_fullsized":"2024-05-15T13:57:19.388016+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:19:00.655474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.389344+0000","last_change":"2024-05-15T13:57:17.394045+0000","last_active":"2024-05-15T13:57:19.389344+0000","last_peered":"2024-05-15T13:57:19.389344+0000","last_clean":"2024-05-15T13:57:19.389344+0000","last_became_active":"2024-05-15T13:57:17.393699+0000","last_became_peered":"2024-05-15T13:57:17.393699+0000","last_unstale":"2024-05-15T13:57:19.389344+0000","last_undegraded":"2024-05-15T13:57:19.389344+0000","last_fullsized":"2024-05-15T13:57:19.389344+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:05:37.510370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":59344,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":157,"num_read_kb":141,"num_write":159,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":178032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645334,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":860160,"data_stored":415573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841563,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36852,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708300,"statfs":{"total":95995035648,"available":95957299200,"internally_reserved":0,"allocated":1552384,"data_stored":1062397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005088,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":876544,"data_stored":415809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201318,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36192,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708960,"statfs":{"total":95995035648,"available":95957975040,"internally_reserved":0,"allocated":876544,"data_stored":415729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397547,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28452,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716700,"statfs":{"total":95995035648,"available":95965900800,"internally_reserved":0,"allocated":1470464,"data_stored":1008050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561073,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":864256,"data_stored":415198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":469943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36200,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708952,"statfs":{"total":95995035648,"available":95957966848,"internally_reserved":0,"allocated":884736,"data_stored":416469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411395,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":471684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149758,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":880640,"data_stored":415509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953530,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":868352,"data_stored":416026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757301,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32624,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712528,"statfs":{"total":95995035648,"available":95961628672,"internally_reserved":0,"allocated":1482752,"data_stored":1007343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":54952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":56693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":56921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-15T13:58:01.877 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-15T13:58:02.130 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:58:02.190 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:02 smithi033 ceph-mon[31650]: pgmap v308: 97 pgs: 97 active+clean; 635 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-05-15T13:58:02.190 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:02 smithi033 ceph-mon[31650]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-15T13:58:02.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:02 smithi052 ceph-mon[36190]: pgmap v308: 97 pgs: 97 active+clean; 635 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-05-15T13:58:02.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:02 smithi052 ceph-mon[36190]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-15T13:58:02.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:02 smithi081 ceph-mon[36434]: pgmap v308: 97 pgs: 97 active+clean; 635 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-05-15T13:58:02.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:02 smithi081 ceph-mon[36434]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-15T13:58:03.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.806+0000 7f81fd95d700 1 -- 172.21.15.33:0/884889265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81f8102e00 msgr2=0x7f81f810f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:03.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.806+0000 7f81fd95d700 1 --2- 172.21.15.33:0/884889265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81f8102e00 0x7f81f810f2f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f81f000b0a0 tx=0x7f81f000b3b0 comp rx=0 tx=0).stop 2024-05-15T13:58:03.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.806+0000 7f81fd95d700 1 -- 172.21.15.33:0/884889265 shutdown_connections 2024-05-15T13:58:03.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.806+0000 7f81fd95d700 1 --2- 172.21.15.33:0/884889265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81f8102e00 0x7f81f810f2f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:03.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.806+0000 7f81fd95d700 1 --2- 172.21.15.33:0/884889265 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81f8108520 0x7f81f8108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:03.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.806+0000 7f81fd95d700 1 --2- 172.21.15.33:0/884889265 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81f8102440 0x7f81f81028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:03.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.806+0000 7f81fd95d700 1 -- 172.21.15.33:0/884889265 >> 172.21.15.33:0/884889265 conn(0x7f81f80793f0 msgr2=0x7f81f8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:03.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.806+0000 7f81fd95d700 1 -- 172.21.15.33:0/884889265 shutdown_connections 2024-05-15T13:58:03.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.806+0000 7f81fd95d700 1 -- 172.21.15.33:0/884889265 wait complete. 2024-05-15T13:58:03.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.807+0000 7f81fd95d700 1 Processor -- start 2024-05-15T13:58:03.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.807+0000 7f81fd95d700 1 -- start start 2024-05-15T13:58:03.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.807+0000 7f81fd95d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81f8102440 0x7f81f819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:03.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.807+0000 7f81fd95d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81f8102e00 0x7f81f819b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:03.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.807+0000 7f81fd95d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81f8108520 0x7f81f81a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:03.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.807+0000 7f81fd95d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f81f8112650 con 0x7f81f8102440 2024-05-15T13:58:03.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.807+0000 7f81fd95d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f81f81124d0 con 0x7f81f8102e00 2024-05-15T13:58:03.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.807+0000 7f81fd95d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f81f81127d0 con 0x7f81f8108520 2024-05-15T13:58:03.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.807+0000 7f81f6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81f8102440 0x7f81f819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:03.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.807+0000 7f81f6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81f8102440 0x7f81f819aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55476/0 (socket says 172.21.15.33:55476) 2024-05-15T13:58:03.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.807+0000 7f81f6ffd700 1 -- 172.21.15.33:0/1307337860 learned_addr learned my addr 172.21.15.33:0/1307337860 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:58:03.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.807+0000 7f81f67fc700 1 --2- 172.21.15.33:0/1307337860 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81f8102e00 0x7f81f819b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:03.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.808+0000 7f81f77fe700 1 --2- 172.21.15.33:0/1307337860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81f8108520 0x7f81f81a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:03.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.808+0000 7f81f6ffd700 1 -- 172.21.15.33:0/1307337860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81f8108520 msgr2=0x7f81f81a5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:03.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.808+0000 7f81f6ffd700 1 --2- 172.21.15.33:0/1307337860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81f8108520 0x7f81f81a5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:03.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.808+0000 7f81f6ffd700 1 -- 172.21.15.33:0/1307337860 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81f8102e00 msgr2=0x7f81f819b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:03.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.808+0000 7f81f6ffd700 1 --2- 172.21.15.33:0/1307337860 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81f8102e00 0x7f81f819b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:03.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.808+0000 7f81f6ffd700 1 -- 172.21.15.33:0/1307337860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f81a6090 con 0x7f81f8102440 2024-05-15T13:58:03.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.808+0000 7f81f77fe700 1 --2- 172.21.15.33:0/1307337860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81f8108520 0x7f81f81a5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T13:58:03.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.808+0000 7f81f6ffd700 1 --2- 172.21.15.33:0/1307337860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81f8102440 0x7f81f819aad0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f81e800e9a0 tx=0x7f81e800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:03.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.808+0000 7f81fc95b700 1 -- 172.21.15.33:0/1307337860 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e800cbd0 con 0x7f81f8102440 2024-05-15T13:58:03.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.808+0000 7f81fc95b700 1 -- 172.21.15.33:0/1307337860 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81e800cd70 con 0x7f81f8102440 2024-05-15T13:58:03.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.808+0000 7f81fd95d700 1 -- 172.21.15.33:0/1307337860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81f81a6260 con 0x7f81f8102440 2024-05-15T13:58:03.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.808+0000 7f81fc95b700 1 -- 172.21.15.33:0/1307337860 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e8018680 con 0x7f81f8102440 2024-05-15T13:58:03.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.808+0000 7f81fd95d700 1 -- 172.21.15.33:0/1307337860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81f81a67a0 con 0x7f81f8102440 2024-05-15T13:58:03.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.809+0000 7f81fd95d700 1 -- 172.21.15.33:0/1307337860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81f806a230 con 0x7f81f8102440 2024-05-15T13:58:03.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.811+0000 7f81fc95b700 1 -- 172.21.15.33:0/1307337860 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f81e8010430 con 0x7f81f8102440 2024-05-15T13:58:03.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.811+0000 7f81fc95b700 1 --2- 172.21.15.33:0/1307337860 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f81e00b2990 0x7f81e00b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:03.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.811+0000 7f81f67fc700 1 --2- 172.21.15.33:0/1307337860 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f81e00b2990 0x7f81e00b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:03.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.812+0000 7f81fc95b700 1 -- 172.21.15.33:0/1307337860 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f81e80e28f0 con 0x7f81f8102440 2024-05-15T13:58:03.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.812+0000 7f81f67fc700 1 --2- 172.21.15.33:0/1307337860 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f81e00b2990 0x7f81e00b4e50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f81f8079810 tx=0x7f81ec009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:03.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:03.815+0000 7f81fc95b700 1 -- 172.21.15.33:0/1307337860 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f81e80aa660 con 0x7f81f8102440 2024-05-15T13:58:04.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:04.028+0000 7f81fd95d700 1 -- 172.21.15.33:0/1307337860 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f81f8063980 con 0x7f81e00b2990 2024-05-15T13:58:04.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:04.036+0000 7f81fc95b700 1 -- 172.21.15.33:0/1307337860 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258680 (secure 0 0 0) 0x7f81f8063980 con 0x7f81e00b2990 2024-05-15T13:58:04.040 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:58:04.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:04.039+0000 7f81fd95d700 1 -- 172.21.15.33:0/1307337860 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f81e00b2990 msgr2=0x7f81e00b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:04.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:04.039+0000 7f81fd95d700 1 --2- 172.21.15.33:0/1307337860 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f81e00b2990 0x7f81e00b4e50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f81f8079810 tx=0x7f81ec009190 comp rx=0 tx=0).stop 2024-05-15T13:58:04.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:04.039+0000 7f81fd95d700 1 -- 172.21.15.33:0/1307337860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81f8102440 msgr2=0x7f81f819aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:04.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:04.039+0000 7f81fd95d700 1 --2- 172.21.15.33:0/1307337860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81f8102440 0x7f81f819aad0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f81e800e9a0 tx=0x7f81e800ecb0 comp rx=0 tx=0).stop 2024-05-15T13:58:04.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:04.039+0000 7f81fd95d700 1 -- 172.21.15.33:0/1307337860 shutdown_connections 2024-05-15T13:58:04.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:04.039+0000 7f81fd95d700 1 --2- 172.21.15.33:0/1307337860 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f81e00b2990 0x7f81e00b4e50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:04.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:04.039+0000 7f81fd95d700 1 --2- 172.21.15.33:0/1307337860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81f8108520 0x7f81f81a5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:04.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:04.039+0000 7f81fd95d700 1 --2- 172.21.15.33:0/1307337860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81f8102440 0x7f81f819aad0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:04.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:04.039+0000 7f81fd95d700 1 --2- 172.21.15.33:0/1307337860 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81f8102e00 0x7f81f819b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:04.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:04.039+0000 7f81fd95d700 1 -- 172.21.15.33:0/1307337860 >> 172.21.15.33:0/1307337860 conn(0x7f81f80793f0 msgr2=0x7f81f8100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:04.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:04.039+0000 7f81fd95d700 1 -- 172.21.15.33:0/1307337860 shutdown_connections 2024-05-15T13:58:04.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:04.039+0000 7f81fd95d700 1 -- 172.21.15.33:0/1307337860 wait complete. 2024-05-15T13:58:04.043 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-05-15T13:58:04.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:04 smithi052 ceph-mon[36190]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-05-15T13:58:04.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:04 smithi081 ceph-mon[36434]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-05-15T13:58:04.434 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:04 smithi033 ceph-mon[31650]: pgmap v309: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.8 KiB/s wr, 9 op/s 2024-05-15T13:58:04.727 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":309,"stamp":"2024-05-15T13:58:02.975183+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655341,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":264,"num_read_kb":226,"num_write":289,"num_write_kb":2263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":373836,"kb_used_data":12204,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567988,"statfs":{"total":1151940427776,"available":1151557619712,"internally_reserved":0,"allocated":12496896,"data_stored":6945915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":38017,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":89,"num_read_kb":79,"num_write":25,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002948"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.388808+0000","last_change":"2024-05-15T13:57:17.382728+0000","last_active":"2024-05-15T13:57:19.388808+0000","last_peered":"2024-05-15T13:57:19.388808+0000","last_clean":"2024-05-15T13:57:19.388808+0000","last_became_active":"2024-05-15T13:57:17.382315+0000","last_became_peered":"2024-05-15T13:57:17.382315+0000","last_unstale":"2024-05-15T13:57:19.388808+0000","last_undegraded":"2024-05-15T13:57:19.388808+0000","last_fullsized":"2024-05-15T13:57:19.388808+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:03:53.137916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.386758+0000","last_change":"2024-05-15T13:57:17.390880+0000","last_active":"2024-05-15T13:57:19.386758+0000","last_peered":"2024-05-15T13:57:19.386758+0000","last_clean":"2024-05-15T13:57:19.386758+0000","last_became_active":"2024-05-15T13:57:17.390560+0000","last_became_peered":"2024-05-15T13:57:17.390560+0000","last_unstale":"2024-05-15T13:57:19.386758+0000","last_undegraded":"2024-05-15T13:57:19.386758+0000","last_fullsized":"2024-05-15T13:57:19.386758+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:59:33.615670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.386468+0000","last_change":"2024-05-15T13:57:17.386495+0000","last_active":"2024-05-15T13:57:19.386468+0000","last_peered":"2024-05-15T13:57:19.386468+0000","last_clean":"2024-05-15T13:57:19.386468+0000","last_became_active":"2024-05-15T13:57:17.386183+0000","last_became_peered":"2024-05-15T13:57:17.386183+0000","last_unstale":"2024-05-15T13:57:19.386468+0000","last_undegraded":"2024-05-15T13:57:19.386468+0000","last_fullsized":"2024-05-15T13:57:19.386468+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:36:13.202922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.386591+0000","last_change":"2024-05-15T13:57:17.385437+0000","last_active":"2024-05-15T13:57:19.386591+0000","last_peered":"2024-05-15T13:57:19.386591+0000","last_clean":"2024-05-15T13:57:19.386591+0000","last_became_active":"2024-05-15T13:57:17.384524+0000","last_became_peered":"2024-05-15T13:57:17.384524+0000","last_unstale":"2024-05-15T13:57:19.386591+0000","last_undegraded":"2024-05-15T13:57:19.386591+0000","last_fullsized":"2024-05-15T13:57:19.386591+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:45:17.207819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.387251+0000","last_change":"2024-05-15T13:57:17.387859+0000","last_active":"2024-05-15T13:57:19.387251+0000","last_peered":"2024-05-15T13:57:19.387251+0000","last_clean":"2024-05-15T13:57:19.387251+0000","last_became_active":"2024-05-15T13:57:17.387491+0000","last_became_peered":"2024-05-15T13:57:17.387491+0000","last_unstale":"2024-05-15T13:57:19.387251+0000","last_undegraded":"2024-05-15T13:57:19.387251+0000","last_fullsized":"2024-05-15T13:57:19.387251+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:43:08.212926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.386676+0000","last_change":"2024-05-15T13:57:17.382289+0000","last_active":"2024-05-15T13:57:19.386676+0000","last_peered":"2024-05-15T13:57:19.386676+0000","last_clean":"2024-05-15T13:57:19.386676+0000","last_became_active":"2024-05-15T13:57:17.380494+0000","last_became_peered":"2024-05-15T13:57:17.380494+0000","last_unstale":"2024-05-15T13:57:19.386676+0000","last_undegraded":"2024-05-15T13:57:19.386676+0000","last_fullsized":"2024-05-15T13:57:19.386676+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:11:37.097034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.387417+0000","last_change":"2024-05-15T13:57:17.391924+0000","last_active":"2024-05-15T13:57:19.387417+0000","last_peered":"2024-05-15T13:57:19.387417+0000","last_clean":"2024-05-15T13:57:19.387417+0000","last_became_active":"2024-05-15T13:57:17.391589+0000","last_became_peered":"2024-05-15T13:57:17.391589+0000","last_unstale":"2024-05-15T13:57:19.387417+0000","last_undegraded":"2024-05-15T13:57:19.387417+0000","last_fullsized":"2024-05-15T13:57:19.387417+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:46:30.213590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.375746+0000","last_change":"2024-05-15T13:57:17.380969+0000","last_active":"2024-05-15T13:57:19.375746+0000","last_peered":"2024-05-15T13:57:19.375746+0000","last_clean":"2024-05-15T13:57:19.375746+0000","last_became_active":"2024-05-15T13:57:17.380660+0000","last_became_peered":"2024-05-15T13:57:17.380660+0000","last_unstale":"2024-05-15T13:57:19.375746+0000","last_undegraded":"2024-05-15T13:57:19.375746+0000","last_fullsized":"2024-05-15T13:57:19.375746+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:01:41.790323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.376718+0000","last_change":"2024-05-15T13:57:17.394943+0000","last_active":"2024-05-15T13:57:19.376718+0000","last_peered":"2024-05-15T13:57:19.376718+0000","last_clean":"2024-05-15T13:57:19.376718+0000","last_became_active":"2024-05-15T13:57:17.394571+0000","last_became_peered":"2024-05-15T13:57:17.394571+0000","last_unstale":"2024-05-15T13:57:19.376718+0000","last_undegraded":"2024-05-15T13:57:19.376718+0000","last_fullsized":"2024-05-15T13:57:19.376718+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:39:26.063096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.386159+0000","last_change":"2024-05-15T13:57:17.386012+0000","last_active":"2024-05-15T13:57:19.386159+0000","last_peered":"2024-05-15T13:57:19.386159+0000","last_clean":"2024-05-15T13:57:19.386159+0000","last_became_active":"2024-05-15T13:57:17.385636+0000","last_became_peered":"2024-05-15T13:57:17.385636+0000","last_unstale":"2024-05-15T13:57:19.386159+0000","last_undegraded":"2024-05-15T13:57:19.386159+0000","last_fullsized":"2024-05-15T13:57:19.386159+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:12:08.465959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.387750+0000","last_change":"2024-05-15T13:57:17.389162+0000","last_active":"2024-05-15T13:57:19.387750+0000","last_peered":"2024-05-15T13:57:19.387750+0000","last_clean":"2024-05-15T13:57:19.387750+0000","last_became_active":"2024-05-15T13:57:17.388503+0000","last_became_peered":"2024-05-15T13:57:17.388503+0000","last_unstale":"2024-05-15T13:57:19.387750+0000","last_undegraded":"2024-05-15T13:57:19.387750+0000","last_fullsized":"2024-05-15T13:57:19.387750+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:24:54.354445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.386380+0000","last_change":"2024-05-15T13:57:17.381689+0000","last_active":"2024-05-15T13:57:19.386380+0000","last_peered":"2024-05-15T13:57:19.386380+0000","last_clean":"2024-05-15T13:57:19.386380+0000","last_became_active":"2024-05-15T13:57:17.381376+0000","last_became_peered":"2024-05-15T13:57:17.381376+0000","last_unstale":"2024-05-15T13:57:19.386380+0000","last_undegraded":"2024-05-15T13:57:19.386380+0000","last_fullsized":"2024-05-15T13:57:19.386380+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:03:52.652934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.387845+0000","last_change":"2024-05-15T13:57:17.382214+0000","last_active":"2024-05-15T13:57:19.387845+0000","last_peered":"2024-05-15T13:57:19.387845+0000","last_clean":"2024-05-15T13:57:19.387845+0000","last_became_active":"2024-05-15T13:57:17.381851+0000","last_became_peered":"2024-05-15T13:57:17.381851+0000","last_unstale":"2024-05-15T13:57:19.387845+0000","last_undegraded":"2024-05-15T13:57:19.387845+0000","last_fullsized":"2024-05-15T13:57:19.387845+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:18:39.854918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.386353+0000","last_change":"2024-05-15T13:57:17.390985+0000","last_active":"2024-05-15T13:57:19.386353+0000","last_peered":"2024-05-15T13:57:19.386353+0000","last_clean":"2024-05-15T13:57:19.386353+0000","last_became_active":"2024-05-15T13:57:17.390633+0000","last_became_peered":"2024-05-15T13:57:17.390633+0000","last_unstale":"2024-05-15T13:57:19.386353+0000","last_undegraded":"2024-05-15T13:57:19.386353+0000","last_fullsized":"2024-05-15T13:57:19.386353+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:32:47.177507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.386402+0000","last_change":"2024-05-15T13:57:17.383943+0000","last_active":"2024-05-15T13:57:19.386402+0000","last_peered":"2024-05-15T13:57:19.386402+0000","last_clean":"2024-05-15T13:57:19.386402+0000","last_became_active":"2024-05-15T13:57:17.383622+0000","last_became_peered":"2024-05-15T13:57:17.383622+0000","last_unstale":"2024-05-15T13:57:19.386402+0000","last_undegraded":"2024-05-15T13:57:19.386402+0000","last_fullsized":"2024-05-15T13:57:19.386402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T00:30:23.249813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.375983+0000","last_change":"2024-05-15T13:57:17.390789+0000","last_active":"2024-05-15T13:57:19.375983+0000","last_peered":"2024-05-15T13:57:19.375983+0000","last_clean":"2024-05-15T13:57:19.375983+0000","last_became_active":"2024-05-15T13:57:17.390466+0000","last_became_peered":"2024-05-15T13:57:17.390466+0000","last_unstale":"2024-05-15T13:57:19.375983+0000","last_undegraded":"2024-05-15T13:57:19.375983+0000","last_fullsized":"2024-05-15T13:57:19.375983+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:07:24.746862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.389750+0000","last_change":"2024-05-15T13:57:17.384080+0000","last_active":"2024-05-15T13:57:19.389750+0000","last_peered":"2024-05-15T13:57:19.389750+0000","last_clean":"2024-05-15T13:57:19.389750+0000","last_became_active":"2024-05-15T13:57:17.383731+0000","last_became_peered":"2024-05-15T13:57:17.383731+0000","last_unstale":"2024-05-15T13:57:19.389750+0000","last_undegraded":"2024-05-15T13:57:19.389750+0000","last_fullsized":"2024-05-15T13:57:19.389750+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:36:22.874337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.386008+0000","last_change":"2024-05-15T13:57:17.382209+0000","last_active":"2024-05-15T13:57:19.386008+0000","last_peered":"2024-05-15T13:57:19.386008+0000","last_clean":"2024-05-15T13:57:19.386008+0000","last_became_active":"2024-05-15T13:57:17.381737+0000","last_became_peered":"2024-05-15T13:57:17.381737+0000","last_unstale":"2024-05-15T13:57:19.386008+0000","last_undegraded":"2024-05-15T13:57:19.386008+0000","last_fullsized":"2024-05-15T13:57:19.386008+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:38:53.913745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.386594+0000","last_change":"2024-05-15T13:57:17.384855+0000","last_active":"2024-05-15T13:57:19.386594+0000","last_peered":"2024-05-15T13:57:19.386594+0000","last_clean":"2024-05-15T13:57:19.386594+0000","last_became_active":"2024-05-15T13:57:17.384387+0000","last_became_peered":"2024-05-15T13:57:17.384387+0000","last_unstale":"2024-05-15T13:57:19.386594+0000","last_undegraded":"2024-05-15T13:57:19.386594+0000","last_fullsized":"2024-05-15T13:57:19.386594+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:32:05.363047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:32.493386+0000","last_change":"2024-05-15T13:57:14.256057+0000","last_active":"2024-05-15T13:57:32.493386+0000","last_peered":"2024-05-15T13:57:32.493386+0000","last_clean":"2024-05-15T13:57:32.493386+0000","last_became_active":"2024-05-15T13:57:14.255413+0000","last_became_peered":"2024-05-15T13:57:14.255413+0000","last_unstale":"2024-05-15T13:57:32.493386+0000","last_undegraded":"2024-05-15T13:57:32.493386+0000","last_fullsized":"2024-05-15T13:57:32.493386+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:13:43.433634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.375890+0000","last_change":"2024-05-15T13:57:17.390548+0000","last_active":"2024-05-15T13:57:19.375890+0000","last_peered":"2024-05-15T13:57:19.375890+0000","last_clean":"2024-05-15T13:57:19.375890+0000","last_became_active":"2024-05-15T13:57:17.390144+0000","last_became_peered":"2024-05-15T13:57:17.390144+0000","last_unstale":"2024-05-15T13:57:19.375890+0000","last_undegraded":"2024-05-15T13:57:19.375890+0000","last_fullsized":"2024-05-15T13:57:19.375890+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:28:54.889994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.386622+0000","last_change":"2024-05-15T13:57:14.535704+0000","last_active":"2024-05-15T13:57:19.386622+0000","last_peered":"2024-05-15T13:57:19.386622+0000","last_clean":"2024-05-15T13:57:19.386622+0000","last_became_active":"2024-05-15T13:57:14.535356+0000","last_became_peered":"2024-05-15T13:57:14.535356+0000","last_unstale":"2024-05-15T13:57:19.386622+0000","last_undegraded":"2024-05-15T13:57:19.386622+0000","last_fullsized":"2024-05-15T13:57:19.386622+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:16:21.545507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.387723+0000","last_change":"2024-05-15T13:57:17.383482+0000","last_active":"2024-05-15T13:57:19.387723+0000","last_peered":"2024-05-15T13:57:19.387723+0000","last_clean":"2024-05-15T13:57:19.387723+0000","last_became_active":"2024-05-15T13:57:17.382973+0000","last_became_peered":"2024-05-15T13:57:17.382973+0000","last_unstale":"2024-05-15T13:57:19.387723+0000","last_undegraded":"2024-05-15T13:57:19.387723+0000","last_fullsized":"2024-05-15T13:57:19.387723+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:31:08.937166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:32.492420+0000","last_change":"2024-05-15T13:57:14.259627+0000","last_active":"2024-05-15T13:57:32.492420+0000","last_peered":"2024-05-15T13:57:32.492420+0000","last_clean":"2024-05-15T13:57:32.492420+0000","last_became_active":"2024-05-15T13:57:14.259282+0000","last_became_peered":"2024-05-15T13:57:14.259282+0000","last_unstale":"2024-05-15T13:57:32.492420+0000","last_undegraded":"2024-05-15T13:57:32.492420+0000","last_fullsized":"2024-05-15T13:57:32.492420+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:01:24.304755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.376207+0000","last_change":"2024-05-15T13:57:17.388720+0000","last_active":"2024-05-15T13:57:19.376207+0000","last_peered":"2024-05-15T13:57:19.376207+0000","last_clean":"2024-05-15T13:57:19.376207+0000","last_became_active":"2024-05-15T13:57:17.388386+0000","last_became_peered":"2024-05-15T13:57:17.388386+0000","last_unstale":"2024-05-15T13:57:19.376207+0000","last_undegraded":"2024-05-15T13:57:19.376207+0000","last_fullsized":"2024-05-15T13:57:19.376207+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:02:40.543701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.391755+0000","last_change":"2024-05-15T13:57:14.259064+0000","last_active":"2024-05-15T13:57:19.391755+0000","last_peered":"2024-05-15T13:57:19.391755+0000","last_clean":"2024-05-15T13:57:19.391755+0000","last_became_active":"2024-05-15T13:57:14.258683+0000","last_became_peered":"2024-05-15T13:57:14.258683+0000","last_unstale":"2024-05-15T13:57:19.391755+0000","last_undegraded":"2024-05-15T13:57:19.391755+0000","last_fullsized":"2024-05-15T13:57:19.391755+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:12:43.108066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.375571+0000","last_change":"2024-05-15T13:57:17.380929+0000","last_active":"2024-05-15T13:57:19.375571+0000","last_peered":"2024-05-15T13:57:19.375571+0000","last_clean":"2024-05-15T13:57:19.375571+0000","last_became_active":"2024-05-15T13:57:17.380599+0000","last_became_peered":"2024-05-15T13:57:17.380599+0000","last_unstale":"2024-05-15T13:57:19.375571+0000","last_undegraded":"2024-05-15T13:57:19.375571+0000","last_fullsized":"2024-05-15T13:57:19.375571+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:19:55.368088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":58,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:58:00.408881+0000","last_change":"2024-05-15T13:57:14.260410+0000","last_active":"2024-05-15T13:58:00.408881+0000","last_peered":"2024-05-15T13:58:00.408881+0000","last_clean":"2024-05-15T13:58:00.408881+0000","last_became_active":"2024-05-15T13:57:14.260076+0000","last_became_peered":"2024-05-15T13:57:14.260076+0000","last_unstale":"2024-05-15T13:58:00.408881+0000","last_undegraded":"2024-05-15T13:58:00.408881+0000","last_fullsized":"2024-05-15T13:58:00.408881+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T00:27:03.436237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.386657+0000","last_change":"2024-05-15T13:57:17.394475+0000","last_active":"2024-05-15T13:57:19.386657+0000","last_peered":"2024-05-15T13:57:19.386657+0000","last_clean":"2024-05-15T13:57:19.386657+0000","last_became_active":"2024-05-15T13:57:17.393733+0000","last_became_peered":"2024-05-15T13:57:17.393733+0000","last_unstale":"2024-05-15T13:57:19.386657+0000","last_undegraded":"2024-05-15T13:57:19.386657+0000","last_fullsized":"2024-05-15T13:57:19.386657+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:55:06.198633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:32.493660+0000","last_change":"2024-05-15T13:57:14.256420+0000","last_active":"2024-05-15T13:57:32.493660+0000","last_peered":"2024-05-15T13:57:32.493660+0000","last_clean":"2024-05-15T13:57:32.493660+0000","last_became_active":"2024-05-15T13:57:14.255722+0000","last_became_peered":"2024-05-15T13:57:14.255722+0000","last_unstale":"2024-05-15T13:57:32.493660+0000","last_undegraded":"2024-05-15T13:57:32.493660+0000","last_fullsized":"2024-05-15T13:57:32.493660+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:21:30.499632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.375599+0000","last_change":"2024-05-15T13:57:17.383858+0000","last_active":"2024-05-15T13:57:19.375599+0000","last_peered":"2024-05-15T13:57:19.375599+0000","last_clean":"2024-05-15T13:57:19.375599+0000","last_became_active":"2024-05-15T13:57:17.383522+0000","last_became_peered":"2024-05-15T13:57:17.383522+0000","last_unstale":"2024-05-15T13:57:19.375599+0000","last_undegraded":"2024-05-15T13:57:19.375599+0000","last_fullsized":"2024-05-15T13:57:19.375599+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:54:59.802226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:32.493377+0000","last_change":"2024-05-15T13:57:14.256192+0000","last_active":"2024-05-15T13:57:32.493377+0000","last_peered":"2024-05-15T13:57:32.493377+0000","last_clean":"2024-05-15T13:57:32.493377+0000","last_became_active":"2024-05-15T13:57:14.255839+0000","last_became_peered":"2024-05-15T13:57:14.255839+0000","last_unstale":"2024-05-15T13:57:32.493377+0000","last_undegraded":"2024-05-15T13:57:32.493377+0000","last_fullsized":"2024-05-15T13:57:32.493377+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:13:26.071403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.375530+0000","last_change":"2024-05-15T13:57:17.386246+0000","last_active":"2024-05-15T13:57:19.375530+0000","last_peered":"2024-05-15T13:57:19.375530+0000","last_clean":"2024-05-15T13:57:19.375530+0000","last_became_active":"2024-05-15T13:57:17.385828+0000","last_became_peered":"2024-05-15T13:57:17.385828+0000","last_unstale":"2024-05-15T13:57:19.375530+0000","last_undegraded":"2024-05-15T13:57:19.375530+0000","last_fullsized":"2024-05-15T13:57:19.375530+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T00:24:38.656469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.375622+0000","last_change":"2024-05-15T13:57:14.259953+0000","last_active":"2024-05-15T13:57:19.375622+0000","last_peered":"2024-05-15T13:57:19.375622+0000","last_clean":"2024-05-15T13:57:19.375622+0000","last_became_active":"2024-05-15T13:57:14.259632+0000","last_became_peered":"2024-05-15T13:57:14.259632+0000","last_unstale":"2024-05-15T13:57:19.375622+0000","last_undegraded":"2024-05-15T13:57:19.375622+0000","last_fullsized":"2024-05-15T13:57:19.375622+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:30:28.361903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.385309+0000","last_change":"2024-05-15T13:57:17.382211+0000","last_active":"2024-05-15T13:57:19.385309+0000","last_peered":"2024-05-15T13:57:19.385309+0000","last_clean":"2024-05-15T13:57:19.385309+0000","last_became_active":"2024-05-15T13:57:17.381904+0000","last_became_peered":"2024-05-15T13:57:17.381904+0000","last_unstale":"2024-05-15T13:57:19.385309+0000","last_undegraded":"2024-05-15T13:57:19.385309+0000","last_fullsized":"2024-05-15T13:57:19.385309+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:53:49.250866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":51,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:59.406693+0000","last_change":"2024-05-15T13:57:14.254691+0000","last_active":"2024-05-15T13:57:59.406693+0000","last_peered":"2024-05-15T13:57:59.406693+0000","last_clean":"2024-05-15T13:57:59.406693+0000","last_became_active":"2024-05-15T13:57:14.254233+0000","last_became_peered":"2024-05-15T13:57:14.254233+0000","last_unstale":"2024-05-15T13:57:59.406693+0000","last_undegraded":"2024-05-15T13:57:59.406693+0000","last_fullsized":"2024-05-15T13:57:59.406693+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:41:31.719402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.376549+0000","last_change":"2024-05-15T13:57:17.382251+0000","last_active":"2024-05-15T13:57:19.376549+0000","last_peered":"2024-05-15T13:57:19.376549+0000","last_clean":"2024-05-15T13:57:19.376549+0000","last_became_active":"2024-05-15T13:57:17.381158+0000","last_became_peered":"2024-05-15T13:57:17.381158+0000","last_unstale":"2024-05-15T13:57:19.376549+0000","last_undegraded":"2024-05-15T13:57:19.376549+0000","last_fullsized":"2024-05-15T13:57:19.376549+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:24:55.971079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:31.485285+0000","last_change":"2024-05-15T13:57:14.258350+0000","last_active":"2024-05-15T13:57:31.485285+0000","last_peered":"2024-05-15T13:57:31.485285+0000","last_clean":"2024-05-15T13:57:31.485285+0000","last_became_active":"2024-05-15T13:57:14.257982+0000","last_became_peered":"2024-05-15T13:57:14.257982+0000","last_unstale":"2024-05-15T13:57:31.485285+0000","last_undegraded":"2024-05-15T13:57:31.485285+0000","last_fullsized":"2024-05-15T13:57:31.485285+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:50:07.053401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.387078+0000","last_change":"2024-05-15T13:57:17.392114+0000","last_active":"2024-05-15T13:57:19.387078+0000","last_peered":"2024-05-15T13:57:19.387078+0000","last_clean":"2024-05-15T13:57:19.387078+0000","last_became_active":"2024-05-15T13:57:17.391732+0000","last_became_peered":"2024-05-15T13:57:17.391732+0000","last_unstale":"2024-05-15T13:57:19.387078+0000","last_undegraded":"2024-05-15T13:57:19.387078+0000","last_fullsized":"2024-05-15T13:57:19.387078+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:28:02.304910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:31.484162+0000","last_change":"2024-05-15T13:57:14.535381+0000","last_active":"2024-05-15T13:57:31.484162+0000","last_peered":"2024-05-15T13:57:31.484162+0000","last_clean":"2024-05-15T13:57:31.484162+0000","last_became_active":"2024-05-15T13:57:14.534931+0000","last_became_peered":"2024-05-15T13:57:14.534931+0000","last_unstale":"2024-05-15T13:57:31.484162+0000","last_undegraded":"2024-05-15T13:57:31.484162+0000","last_fullsized":"2024-05-15T13:57:31.484162+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:13:04.785139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.376437+0000","last_change":"2024-05-15T13:57:17.386103+0000","last_active":"2024-05-15T13:57:19.376437+0000","last_peered":"2024-05-15T13:57:19.376437+0000","last_clean":"2024-05-15T13:57:19.376437+0000","last_became_active":"2024-05-15T13:57:17.385738+0000","last_became_peered":"2024-05-15T13:57:17.385738+0000","last_unstale":"2024-05-15T13:57:19.376437+0000","last_undegraded":"2024-05-15T13:57:19.376437+0000","last_fullsized":"2024-05-15T13:57:19.376437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:42:56.136164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'28","reported_seq":110,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:58:00.408963+0000","last_change":"2024-05-15T13:57:14.258903+0000","last_active":"2024-05-15T13:58:00.408963+0000","last_peered":"2024-05-15T13:58:00.408963+0000","last_clean":"2024-05-15T13:58:00.408963+0000","last_became_active":"2024-05-15T13:57:14.258453+0000","last_became_peered":"2024-05-15T13:57:14.258453+0000","last_unstale":"2024-05-15T13:58:00.408963+0000","last_undegraded":"2024-05-15T13:58:00.408963+0000","last_fullsized":"2024-05-15T13:58:00.408963+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:49:33.004534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60347,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":60,"num_read_kb":88,"num_write":29,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.387018+0000","last_change":"2024-05-15T13:57:17.389717+0000","last_active":"2024-05-15T13:57:19.387018+0000","last_peered":"2024-05-15T13:57:19.387018+0000","last_clean":"2024-05-15T13:57:19.387018+0000","last_became_active":"2024-05-15T13:57:17.389188+0000","last_became_peered":"2024-05-15T13:57:17.389188+0000","last_unstale":"2024-05-15T13:57:19.387018+0000","last_undegraded":"2024-05-15T13:57:19.387018+0000","last_fullsized":"2024-05-15T13:57:19.387018+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:40:26.827507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:32.493344+0000","last_change":"2024-05-15T13:57:14.256007+0000","last_active":"2024-05-15T13:57:32.493344+0000","last_peered":"2024-05-15T13:57:32.493344+0000","last_clean":"2024-05-15T13:57:32.493344+0000","last_became_active":"2024-05-15T13:57:14.255656+0000","last_became_peered":"2024-05-15T13:57:14.255656+0000","last_unstale":"2024-05-15T13:57:32.493344+0000","last_undegraded":"2024-05-15T13:57:32.493344+0000","last_fullsized":"2024-05-15T13:57:32.493344+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:14:46.880981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.388234+0000","last_change":"2024-05-15T13:57:17.389971+0000","last_active":"2024-05-15T13:57:19.388234+0000","last_peered":"2024-05-15T13:57:19.388234+0000","last_clean":"2024-05-15T13:57:19.388234+0000","last_became_active":"2024-05-15T13:57:17.389633+0000","last_became_peered":"2024-05-15T13:57:17.389633+0000","last_unstale":"2024-05-15T13:57:19.388234+0000","last_undegraded":"2024-05-15T13:57:19.388234+0000","last_fullsized":"2024-05-15T13:57:19.388234+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:25:10.971086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:31.486543+0000","last_change":"2024-05-15T13:57:14.258086+0000","last_active":"2024-05-15T13:57:31.486543+0000","last_peered":"2024-05-15T13:57:31.486543+0000","last_clean":"2024-05-15T13:57:31.486543+0000","last_became_active":"2024-05-15T13:57:14.257736+0000","last_became_peered":"2024-05-15T13:57:14.257736+0000","last_unstale":"2024-05-15T13:57:31.486543+0000","last_undegraded":"2024-05-15T13:57:31.486543+0000","last_fullsized":"2024-05-15T13:57:31.486543+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T00:51:07.431725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.387924+0000","last_change":"2024-05-15T13:57:17.390804+0000","last_active":"2024-05-15T13:57:19.387924+0000","last_peered":"2024-05-15T13:57:19.387924+0000","last_clean":"2024-05-15T13:57:19.387924+0000","last_became_active":"2024-05-15T13:57:17.390393+0000","last_became_peered":"2024-05-15T13:57:17.390393+0000","last_unstale":"2024-05-15T13:57:19.387924+0000","last_undegraded":"2024-05-15T13:57:19.387924+0000","last_fullsized":"2024-05-15T13:57:19.387924+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:42:56.204275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'4","reported_seq":56,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:58:00.409052+0000","last_change":"2024-05-15T13:57:14.261911+0000","last_active":"2024-05-15T13:58:00.409052+0000","last_peered":"2024-05-15T13:58:00.409052+0000","last_clean":"2024-05-15T13:58:00.409052+0000","last_became_active":"2024-05-15T13:57:14.261506+0000","last_became_peered":"2024-05-15T13:57:14.261506+0000","last_unstale":"2024-05-15T13:58:00.409052+0000","last_undegraded":"2024-05-15T13:58:00.409052+0000","last_fullsized":"2024-05-15T13:58:00.409052+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:18:21.021651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":31,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.388723+0000","last_change":"2024-05-15T13:57:17.395074+0000","last_active":"2024-05-15T13:57:19.388723+0000","last_peered":"2024-05-15T13:57:19.388723+0000","last_clean":"2024-05-15T13:57:19.388723+0000","last_became_active":"2024-05-15T13:57:17.394712+0000","last_became_peered":"2024-05-15T13:57:17.394712+0000","last_unstale":"2024-05-15T13:57:19.388723+0000","last_undegraded":"2024-05-15T13:57:19.388723+0000","last_fullsized":"2024-05-15T13:57:19.388723+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:58:25.385949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:32.494405+0000","last_change":"2024-05-15T13:57:14.535178+0000","last_active":"2024-05-15T13:57:32.494405+0000","last_peered":"2024-05-15T13:57:32.494405+0000","last_clean":"2024-05-15T13:57:32.494405+0000","last_became_active":"2024-05-15T13:57:14.534804+0000","last_became_peered":"2024-05-15T13:57:14.534804+0000","last_unstale":"2024-05-15T13:57:32.494405+0000","last_undegraded":"2024-05-15T13:57:32.494405+0000","last_fullsized":"2024-05-15T13:57:32.494405+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:53:57.558436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.375384+0000","last_change":"2024-05-15T13:57:17.376895+0000","last_active":"2024-05-15T13:57:19.375384+0000","last_peered":"2024-05-15T13:57:19.375384+0000","last_clean":"2024-05-15T13:57:19.375384+0000","last_became_active":"2024-05-15T13:57:17.376499+0000","last_became_peered":"2024-05-15T13:57:17.376499+0000","last_unstale":"2024-05-15T13:57:19.375384+0000","last_undegraded":"2024-05-15T13:57:19.375384+0000","last_fullsized":"2024-05-15T13:57:19.375384+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:39:41.707151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:33.489634+0000","last_change":"2024-05-15T13:57:14.258155+0000","last_active":"2024-05-15T13:57:33.489634+0000","last_peered":"2024-05-15T13:57:33.489634+0000","last_clean":"2024-05-15T13:57:33.489634+0000","last_became_active":"2024-05-15T13:57:14.257733+0000","last_became_peered":"2024-05-15T13:57:14.257733+0000","last_unstale":"2024-05-15T13:57:33.489634+0000","last_undegraded":"2024-05-15T13:57:33.489634+0000","last_fullsized":"2024-05-15T13:57:33.489634+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:33:45.659936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.389281+0000","last_change":"2024-05-15T13:57:17.390640+0000","last_active":"2024-05-15T13:57:19.389281+0000","last_peered":"2024-05-15T13:57:19.389281+0000","last_clean":"2024-05-15T13:57:19.389281+0000","last_became_active":"2024-05-15T13:57:17.390293+0000","last_became_peered":"2024-05-15T13:57:17.390293+0000","last_unstale":"2024-05-15T13:57:19.389281+0000","last_undegraded":"2024-05-15T13:57:19.389281+0000","last_fullsized":"2024-05-15T13:57:19.389281+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:08:18.907391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.385810+0000","last_change":"2024-05-15T13:57:14.535321+0000","last_active":"2024-05-15T13:57:19.385810+0000","last_peered":"2024-05-15T13:57:19.385810+0000","last_clean":"2024-05-15T13:57:19.385810+0000","last_became_active":"2024-05-15T13:57:14.534794+0000","last_became_peered":"2024-05-15T13:57:14.534794+0000","last_unstale":"2024-05-15T13:57:19.385810+0000","last_undegraded":"2024-05-15T13:57:19.385810+0000","last_fullsized":"2024-05-15T13:57:19.385810+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:27:07.087671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:21.352686+0000","last_change":"2024-05-15T13:57:17.381249+0000","last_active":"2024-05-15T13:57:21.352686+0000","last_peered":"2024-05-15T13:57:21.352686+0000","last_clean":"2024-05-15T13:57:21.352686+0000","last_became_active":"2024-05-15T13:57:17.380474+0000","last_became_peered":"2024-05-15T13:57:17.380474+0000","last_unstale":"2024-05-15T13:57:21.352686+0000","last_undegraded":"2024-05-15T13:57:21.352686+0000","last_fullsized":"2024-05-15T13:57:21.352686+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:49:00.341085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.388331+0000","last_change":"2024-05-15T13:57:14.257635+0000","last_active":"2024-05-15T13:57:19.388331+0000","last_peered":"2024-05-15T13:57:19.388331+0000","last_clean":"2024-05-15T13:57:19.388331+0000","last_became_active":"2024-05-15T13:57:14.257156+0000","last_became_peered":"2024-05-15T13:57:14.257156+0000","last_unstale":"2024-05-15T13:57:19.388331+0000","last_undegraded":"2024-05-15T13:57:19.388331+0000","last_fullsized":"2024-05-15T13:57:19.388331+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:11:48.742053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.385643+0000","last_change":"2024-05-15T13:57:17.391067+0000","last_active":"2024-05-15T13:57:19.385643+0000","last_peered":"2024-05-15T13:57:19.385643+0000","last_clean":"2024-05-15T13:57:19.385643+0000","last_became_active":"2024-05-15T13:57:17.390763+0000","last_became_peered":"2024-05-15T13:57:17.390763+0000","last_unstale":"2024-05-15T13:57:19.385643+0000","last_undegraded":"2024-05-15T13:57:19.385643+0000","last_fullsized":"2024-05-15T13:57:19.385643+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:28:46.506307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.375743+0000","last_change":"2024-05-15T13:57:17.391816+0000","last_active":"2024-05-15T13:57:19.375743+0000","last_peered":"2024-05-15T13:57:19.375743+0000","last_clean":"2024-05-15T13:57:19.375743+0000","last_became_active":"2024-05-15T13:57:17.389374+0000","last_became_peered":"2024-05-15T13:57:17.389374+0000","last_unstale":"2024-05-15T13:57:19.375743+0000","last_undegraded":"2024-05-15T13:57:19.375743+0000","last_fullsized":"2024-05-15T13:57:19.375743+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:48:31.756133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":49,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:56.402381+0000","last_change":"2024-05-15T13:57:14.534946+0000","last_active":"2024-05-15T13:57:56.402381+0000","last_peered":"2024-05-15T13:57:56.402381+0000","last_clean":"2024-05-15T13:57:56.402381+0000","last_became_active":"2024-05-15T13:57:14.534535+0000","last_became_peered":"2024-05-15T13:57:14.534535+0000","last_unstale":"2024-05-15T13:57:56.402381+0000","last_undegraded":"2024-05-15T13:57:56.402381+0000","last_fullsized":"2024-05-15T13:57:56.402381+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:27:20.526800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.376133+0000","last_change":"2024-05-15T13:57:17.391637+0000","last_active":"2024-05-15T13:57:19.376133+0000","last_peered":"2024-05-15T13:57:19.376133+0000","last_clean":"2024-05-15T13:57:19.376133+0000","last_became_active":"2024-05-15T13:57:17.391271+0000","last_became_peered":"2024-05-15T13:57:17.391271+0000","last_unstale":"2024-05-15T13:57:19.376133+0000","last_undegraded":"2024-05-15T13:57:19.376133+0000","last_fullsized":"2024-05-15T13:57:19.376133+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:22:06.398387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:35.368235+0000","last_change":"2024-05-15T13:57:14.258510+0000","last_active":"2024-05-15T13:57:35.368235+0000","last_peered":"2024-05-15T13:57:35.368235+0000","last_clean":"2024-05-15T13:57:35.368235+0000","last_became_active":"2024-05-15T13:57:14.257544+0000","last_became_peered":"2024-05-15T13:57:14.257544+0000","last_unstale":"2024-05-15T13:57:35.368235+0000","last_undegraded":"2024-05-15T13:57:35.368235+0000","last_fullsized":"2024-05-15T13:57:35.368235+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:12:27.253931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":95,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.388889+0000","last_change":"2024-05-15T13:55:53.940355+0000","last_active":"2024-05-15T13:57:19.388889+0000","last_peered":"2024-05-15T13:57:19.388889+0000","last_clean":"2024-05-15T13:57:19.388889+0000","last_became_active":"2024-05-15T13:55:53.934638+0000","last_became_peered":"2024-05-15T13:55:53.934638+0000","last_unstale":"2024-05-15T13:57:19.388889+0000","last_undegraded":"2024-05-15T13:57:19.388889+0000","last_fullsized":"2024-05-15T13:57:19.388889+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:52:33.556435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:52:33.556435+0000","last_clean_scrub_stamp":"2024-05-15T13:52:33.556435+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:02:55.332569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.386872+0000","last_change":"2024-05-15T13:57:17.382394+0000","last_active":"2024-05-15T13:57:19.386872+0000","last_peered":"2024-05-15T13:57:19.386872+0000","last_clean":"2024-05-15T13:57:19.386872+0000","last_became_active":"2024-05-15T13:57:17.382090+0000","last_became_peered":"2024-05-15T13:57:17.382090+0000","last_unstale":"2024-05-15T13:57:19.386872+0000","last_undegraded":"2024-05-15T13:57:19.386872+0000","last_fullsized":"2024-05-15T13:57:19.386872+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:01:34.653216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:32.492852+0000","last_change":"2024-05-15T13:57:14.261549+0000","last_active":"2024-05-15T13:57:32.492852+0000","last_peered":"2024-05-15T13:57:32.492852+0000","last_clean":"2024-05-15T13:57:32.492852+0000","last_became_active":"2024-05-15T13:57:14.261165+0000","last_became_peered":"2024-05-15T13:57:14.261165+0000","last_unstale":"2024-05-15T13:57:32.492852+0000","last_undegraded":"2024-05-15T13:57:32.492852+0000","last_fullsized":"2024-05-15T13:57:32.492852+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:04:39.574440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:31.483634+0000","last_change":"2024-05-15T13:57:14.535309+0000","last_active":"2024-05-15T13:57:31.483634+0000","last_peered":"2024-05-15T13:57:31.483634+0000","last_clean":"2024-05-15T13:57:31.483634+0000","last_became_active":"2024-05-15T13:57:14.534938+0000","last_became_peered":"2024-05-15T13:57:14.534938+0000","last_unstale":"2024-05-15T13:57:31.483634+0000","last_undegraded":"2024-05-15T13:57:31.483634+0000","last_fullsized":"2024-05-15T13:57:31.483634+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:34:53.373149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.387317+0000","last_change":"2024-05-15T13:57:17.388272+0000","last_active":"2024-05-15T13:57:19.387317+0000","last_peered":"2024-05-15T13:57:19.387317+0000","last_clean":"2024-05-15T13:57:19.387317+0000","last_became_active":"2024-05-15T13:57:17.387849+0000","last_became_peered":"2024-05-15T13:57:17.387849+0000","last_unstale":"2024-05-15T13:57:19.387317+0000","last_undegraded":"2024-05-15T13:57:19.387317+0000","last_fullsized":"2024-05-15T13:57:19.387317+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:47:47.323566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'11","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:55.387907+0000","last_change":"2024-05-15T13:57:14.256161+0000","last_active":"2024-05-15T13:57:55.387907+0000","last_peered":"2024-05-15T13:57:55.387907+0000","last_clean":"2024-05-15T13:57:55.387907+0000","last_became_active":"2024-05-15T13:57:14.255707+0000","last_became_peered":"2024-05-15T13:57:14.255707+0000","last_unstale":"2024-05-15T13:57:55.387907+0000","last_undegraded":"2024-05-15T13:57:55.387907+0000","last_fullsized":"2024-05-15T13:57:55.387907+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:24:57.200029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.376114+0000","last_change":"2024-05-15T13:57:17.389522+0000","last_active":"2024-05-15T13:57:19.376114+0000","last_peered":"2024-05-15T13:57:19.376114+0000","last_clean":"2024-05-15T13:57:19.376114+0000","last_became_active":"2024-05-15T13:57:17.389177+0000","last_became_peered":"2024-05-15T13:57:17.389177+0000","last_unstale":"2024-05-15T13:57:19.376114+0000","last_undegraded":"2024-05-15T13:57:19.376114+0000","last_fullsized":"2024-05-15T13:57:19.376114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:23:35.716934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:31.485225+0000","last_change":"2024-05-15T13:57:14.254300+0000","last_active":"2024-05-15T13:57:31.485225+0000","last_peered":"2024-05-15T13:57:31.485225+0000","last_clean":"2024-05-15T13:57:31.485225+0000","last_became_active":"2024-05-15T13:57:14.253741+0000","last_became_peered":"2024-05-15T13:57:14.253741+0000","last_unstale":"2024-05-15T13:57:31.485225+0000","last_undegraded":"2024-05-15T13:57:31.485225+0000","last_fullsized":"2024-05-15T13:57:31.485225+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:25:34.018159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.386993+0000","last_change":"2024-05-15T13:57:17.380688+0000","last_active":"2024-05-15T13:57:19.386993+0000","last_peered":"2024-05-15T13:57:19.386993+0000","last_clean":"2024-05-15T13:57:19.386993+0000","last_became_active":"2024-05-15T13:57:17.380379+0000","last_became_peered":"2024-05-15T13:57:17.380379+0000","last_unstale":"2024-05-15T13:57:19.386993+0000","last_undegraded":"2024-05-15T13:57:19.386993+0000","last_fullsized":"2024-05-15T13:57:19.386993+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:32:09.190750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:31.485217+0000","last_change":"2024-05-15T13:57:14.534775+0000","last_active":"2024-05-15T13:57:31.485217+0000","last_peered":"2024-05-15T13:57:31.485217+0000","last_clean":"2024-05-15T13:57:31.485217+0000","last_became_active":"2024-05-15T13:57:14.534440+0000","last_became_peered":"2024-05-15T13:57:14.534440+0000","last_unstale":"2024-05-15T13:57:31.485217+0000","last_undegraded":"2024-05-15T13:57:31.485217+0000","last_fullsized":"2024-05-15T13:57:31.485217+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:21:49.047187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.386481+0000","last_change":"2024-05-15T13:57:17.396252+0000","last_active":"2024-05-15T13:57:19.386481+0000","last_peered":"2024-05-15T13:57:19.386481+0000","last_clean":"2024-05-15T13:57:19.386481+0000","last_became_active":"2024-05-15T13:57:17.395853+0000","last_became_peered":"2024-05-15T13:57:17.395853+0000","last_unstale":"2024-05-15T13:57:19.386481+0000","last_undegraded":"2024-05-15T13:57:19.386481+0000","last_fullsized":"2024-05-15T13:57:19.386481+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:12:54.924053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:31.485720+0000","last_change":"2024-05-15T13:57:14.258411+0000","last_active":"2024-05-15T13:57:31.485720+0000","last_peered":"2024-05-15T13:57:31.485720+0000","last_clean":"2024-05-15T13:57:31.485720+0000","last_became_active":"2024-05-15T13:57:14.257847+0000","last_became_peered":"2024-05-15T13:57:14.257847+0000","last_unstale":"2024-05-15T13:57:31.485720+0000","last_undegraded":"2024-05-15T13:57:31.485720+0000","last_fullsized":"2024-05-15T13:57:31.485720+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:49:32.368351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.388726+0000","last_change":"2024-05-15T13:57:17.388916+0000","last_active":"2024-05-15T13:57:19.388726+0000","last_peered":"2024-05-15T13:57:19.388726+0000","last_clean":"2024-05-15T13:57:19.388726+0000","last_became_active":"2024-05-15T13:57:17.388603+0000","last_became_peered":"2024-05-15T13:57:17.388603+0000","last_unstale":"2024-05-15T13:57:19.388726+0000","last_undegraded":"2024-05-15T13:57:19.388726+0000","last_fullsized":"2024-05-15T13:57:19.388726+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:38:46.650393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.386040+0000","last_change":"2024-05-15T13:57:14.256801+0000","last_active":"2024-05-15T13:57:19.386040+0000","last_peered":"2024-05-15T13:57:19.386040+0000","last_clean":"2024-05-15T13:57:19.386040+0000","last_became_active":"2024-05-15T13:57:14.256476+0000","last_became_peered":"2024-05-15T13:57:14.256476+0000","last_unstale":"2024-05-15T13:57:19.386040+0000","last_undegraded":"2024-05-15T13:57:19.386040+0000","last_fullsized":"2024-05-15T13:57:19.386040+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:22:34.495756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.376000+0000","last_change":"2024-05-15T13:57:17.380927+0000","last_active":"2024-05-15T13:57:19.376000+0000","last_peered":"2024-05-15T13:57:19.376000+0000","last_clean":"2024-05-15T13:57:19.376000+0000","last_became_active":"2024-05-15T13:57:17.380602+0000","last_became_peered":"2024-05-15T13:57:17.380602+0000","last_unstale":"2024-05-15T13:57:19.376000+0000","last_undegraded":"2024-05-15T13:57:19.376000+0000","last_fullsized":"2024-05-15T13:57:19.376000+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:37:13.920968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:31.485300+0000","last_change":"2024-05-15T13:57:14.257954+0000","last_active":"2024-05-15T13:57:31.485300+0000","last_peered":"2024-05-15T13:57:31.485300+0000","last_clean":"2024-05-15T13:57:31.485300+0000","last_became_active":"2024-05-15T13:57:14.257601+0000","last_became_peered":"2024-05-15T13:57:14.257601+0000","last_unstale":"2024-05-15T13:57:31.485300+0000","last_undegraded":"2024-05-15T13:57:31.485300+0000","last_fullsized":"2024-05-15T13:57:31.485300+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:47:14.811264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.388483+0000","last_change":"2024-05-15T13:57:17.385227+0000","last_active":"2024-05-15T13:57:19.388483+0000","last_peered":"2024-05-15T13:57:19.388483+0000","last_clean":"2024-05-15T13:57:19.388483+0000","last_became_active":"2024-05-15T13:57:17.384899+0000","last_became_peered":"2024-05-15T13:57:17.384899+0000","last_unstale":"2024-05-15T13:57:19.388483+0000","last_undegraded":"2024-05-15T13:57:19.388483+0000","last_fullsized":"2024-05-15T13:57:19.388483+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:04:29.162392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:32.494186+0000","last_change":"2024-05-15T13:57:14.535266+0000","last_active":"2024-05-15T13:57:32.494186+0000","last_peered":"2024-05-15T13:57:32.494186+0000","last_clean":"2024-05-15T13:57:32.494186+0000","last_became_active":"2024-05-15T13:57:14.534892+0000","last_became_peered":"2024-05-15T13:57:14.534892+0000","last_unstale":"2024-05-15T13:57:32.494186+0000","last_undegraded":"2024-05-15T13:57:32.494186+0000","last_fullsized":"2024-05-15T13:57:32.494186+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:49:50.934517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.388185+0000","last_change":"2024-05-15T13:57:17.390330+0000","last_active":"2024-05-15T13:57:19.388185+0000","last_peered":"2024-05-15T13:57:19.388185+0000","last_clean":"2024-05-15T13:57:19.388185+0000","last_became_active":"2024-05-15T13:57:17.390000+0000","last_became_peered":"2024-05-15T13:57:17.390000+0000","last_unstale":"2024-05-15T13:57:19.388185+0000","last_undegraded":"2024-05-15T13:57:19.388185+0000","last_fullsized":"2024-05-15T13:57:19.388185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:52:11.361549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.389721+0000","last_change":"2024-05-15T13:57:14.258174+0000","last_active":"2024-05-15T13:57:19.389721+0000","last_peered":"2024-05-15T13:57:19.389721+0000","last_clean":"2024-05-15T13:57:19.389721+0000","last_became_active":"2024-05-15T13:57:14.257763+0000","last_became_peered":"2024-05-15T13:57:14.257763+0000","last_unstale":"2024-05-15T13:57:19.389721+0000","last_undegraded":"2024-05-15T13:57:19.389721+0000","last_fullsized":"2024-05-15T13:57:19.389721+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:34:31.573845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:20.812172+0000","last_change":"2024-05-15T13:57:17.391964+0000","last_active":"2024-05-15T13:57:20.812172+0000","last_peered":"2024-05-15T13:57:20.812172+0000","last_clean":"2024-05-15T13:57:20.812172+0000","last_became_active":"2024-05-15T13:57:17.391517+0000","last_became_peered":"2024-05-15T13:57:17.391517+0000","last_unstale":"2024-05-15T13:57:20.812172+0000","last_undegraded":"2024-05-15T13:57:20.812172+0000","last_fullsized":"2024-05-15T13:57:20.812172+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:54:36.883563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.390779+0000","last_change":"2024-05-15T13:57:14.256311+0000","last_active":"2024-05-15T13:57:19.390779+0000","last_peered":"2024-05-15T13:57:19.390779+0000","last_clean":"2024-05-15T13:57:19.390779+0000","last_became_active":"2024-05-15T13:57:14.255798+0000","last_became_peered":"2024-05-15T13:57:14.255798+0000","last_unstale":"2024-05-15T13:57:19.390779+0000","last_undegraded":"2024-05-15T13:57:19.390779+0000","last_fullsized":"2024-05-15T13:57:19.390779+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T00:52:50.996261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.388337+0000","last_change":"2024-05-15T13:57:17.389518+0000","last_active":"2024-05-15T13:57:19.388337+0000","last_peered":"2024-05-15T13:57:19.388337+0000","last_clean":"2024-05-15T13:57:19.388337+0000","last_became_active":"2024-05-15T13:57:17.388905+0000","last_became_peered":"2024-05-15T13:57:17.388905+0000","last_unstale":"2024-05-15T13:57:19.388337+0000","last_undegraded":"2024-05-15T13:57:19.388337+0000","last_fullsized":"2024-05-15T13:57:19.388337+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:00:51.109880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.388878+0000","last_change":"2024-05-15T13:57:17.394127+0000","last_active":"2024-05-15T13:57:19.388878+0000","last_peered":"2024-05-15T13:57:19.388878+0000","last_clean":"2024-05-15T13:57:19.388878+0000","last_became_active":"2024-05-15T13:57:17.393771+0000","last_became_peered":"2024-05-15T13:57:17.393771+0000","last_unstale":"2024-05-15T13:57:19.388878+0000","last_undegraded":"2024-05-15T13:57:19.388878+0000","last_fullsized":"2024-05-15T13:57:19.388878+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:41:33.491001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.386115+0000","last_change":"2024-05-15T13:57:17.392451+0000","last_active":"2024-05-15T13:57:19.386115+0000","last_peered":"2024-05-15T13:57:19.386115+0000","last_clean":"2024-05-15T13:57:19.386115+0000","last_became_active":"2024-05-15T13:57:17.392099+0000","last_became_peered":"2024-05-15T13:57:17.392099+0000","last_unstale":"2024-05-15T13:57:19.386115+0000","last_undegraded":"2024-05-15T13:57:19.386115+0000","last_fullsized":"2024-05-15T13:57:19.386115+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:00:14.774543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.385798+0000","last_change":"2024-05-15T13:57:17.385525+0000","last_active":"2024-05-15T13:57:19.385798+0000","last_peered":"2024-05-15T13:57:19.385798+0000","last_clean":"2024-05-15T13:57:19.385798+0000","last_became_active":"2024-05-15T13:57:17.384497+0000","last_became_peered":"2024-05-15T13:57:17.384497+0000","last_unstale":"2024-05-15T13:57:19.385798+0000","last_undegraded":"2024-05-15T13:57:19.385798+0000","last_fullsized":"2024-05-15T13:57:19.385798+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:50:39.259339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.387360+0000","last_change":"2024-05-15T13:57:17.379673+0000","last_active":"2024-05-15T13:57:19.387360+0000","last_peered":"2024-05-15T13:57:19.387360+0000","last_clean":"2024-05-15T13:57:19.387360+0000","last_became_active":"2024-05-15T13:57:17.379342+0000","last_became_peered":"2024-05-15T13:57:17.379342+0000","last_unstale":"2024-05-15T13:57:19.387360+0000","last_undegraded":"2024-05-15T13:57:19.387360+0000","last_fullsized":"2024-05-15T13:57:19.387360+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:26:43.982972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.387486+0000","last_change":"2024-05-15T13:57:17.376757+0000","last_active":"2024-05-15T13:57:19.387486+0000","last_peered":"2024-05-15T13:57:19.387486+0000","last_clean":"2024-05-15T13:57:19.387486+0000","last_became_active":"2024-05-15T13:57:17.376428+0000","last_became_peered":"2024-05-15T13:57:17.376428+0000","last_unstale":"2024-05-15T13:57:19.387486+0000","last_undegraded":"2024-05-15T13:57:19.387486+0000","last_fullsized":"2024-05-15T13:57:19.387486+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:55:47.739650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.389798+0000","last_change":"2024-05-15T13:57:17.390939+0000","last_active":"2024-05-15T13:57:19.389798+0000","last_peered":"2024-05-15T13:57:19.389798+0000","last_clean":"2024-05-15T13:57:19.389798+0000","last_became_active":"2024-05-15T13:57:17.390281+0000","last_became_peered":"2024-05-15T13:57:17.390281+0000","last_unstale":"2024-05-15T13:57:19.389798+0000","last_undegraded":"2024-05-15T13:57:19.389798+0000","last_fullsized":"2024-05-15T13:57:19.389798+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:02:22.077307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.387077+0000","last_change":"2024-05-15T13:57:17.392009+0000","last_active":"2024-05-15T13:57:19.387077+0000","last_peered":"2024-05-15T13:57:19.387077+0000","last_clean":"2024-05-15T13:57:19.387077+0000","last_became_active":"2024-05-15T13:57:17.391059+0000","last_became_peered":"2024-05-15T13:57:17.391059+0000","last_unstale":"2024-05-15T13:57:19.387077+0000","last_undegraded":"2024-05-15T13:57:19.387077+0000","last_fullsized":"2024-05-15T13:57:19.387077+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:51:46.787790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.375857+0000","last_change":"2024-05-15T13:57:17.383872+0000","last_active":"2024-05-15T13:57:19.375857+0000","last_peered":"2024-05-15T13:57:19.375857+0000","last_clean":"2024-05-15T13:57:19.375857+0000","last_became_active":"2024-05-15T13:57:17.383526+0000","last_became_peered":"2024-05-15T13:57:17.383526+0000","last_unstale":"2024-05-15T13:57:19.375857+0000","last_undegraded":"2024-05-15T13:57:19.375857+0000","last_fullsized":"2024-05-15T13:57:19.375857+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:43:39.343557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.375379+0000","last_change":"2024-05-15T13:57:17.377530+0000","last_active":"2024-05-15T13:57:19.375379+0000","last_peered":"2024-05-15T13:57:19.375379+0000","last_clean":"2024-05-15T13:57:19.375379+0000","last_became_active":"2024-05-15T13:57:17.377237+0000","last_became_peered":"2024-05-15T13:57:17.377237+0000","last_unstale":"2024-05-15T13:57:19.375379+0000","last_undegraded":"2024-05-15T13:57:19.375379+0000","last_fullsized":"2024-05-15T13:57:19.375379+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:29:02.173018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.375535+0000","last_change":"2024-05-15T13:57:17.386208+0000","last_active":"2024-05-15T13:57:19.375535+0000","last_peered":"2024-05-15T13:57:19.375535+0000","last_clean":"2024-05-15T13:57:19.375535+0000","last_became_active":"2024-05-15T13:57:17.385853+0000","last_became_peered":"2024-05-15T13:57:17.385853+0000","last_unstale":"2024-05-15T13:57:19.375535+0000","last_undegraded":"2024-05-15T13:57:19.375535+0000","last_fullsized":"2024-05-15T13:57:19.375535+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:17:06.124653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.376252+0000","last_change":"2024-05-15T13:57:17.390074+0000","last_active":"2024-05-15T13:57:19.376252+0000","last_peered":"2024-05-15T13:57:19.376252+0000","last_clean":"2024-05-15T13:57:19.376252+0000","last_became_active":"2024-05-15T13:57:17.389765+0000","last_became_peered":"2024-05-15T13:57:17.389765+0000","last_unstale":"2024-05-15T13:57:19.376252+0000","last_undegraded":"2024-05-15T13:57:19.376252+0000","last_fullsized":"2024-05-15T13:57:19.376252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:49:51.321180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.388016+0000","last_change":"2024-05-15T13:57:17.389180+0000","last_active":"2024-05-15T13:57:19.388016+0000","last_peered":"2024-05-15T13:57:19.388016+0000","last_clean":"2024-05-15T13:57:19.388016+0000","last_became_active":"2024-05-15T13:57:17.388731+0000","last_became_peered":"2024-05-15T13:57:17.388731+0000","last_unstale":"2024-05-15T13:57:19.388016+0000","last_undegraded":"2024-05-15T13:57:19.388016+0000","last_fullsized":"2024-05-15T13:57:19.388016+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:19:00.655474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-15T13:57:19.389344+0000","last_change":"2024-05-15T13:57:17.394045+0000","last_active":"2024-05-15T13:57:19.389344+0000","last_peered":"2024-05-15T13:57:19.389344+0000","last_clean":"2024-05-15T13:57:19.389344+0000","last_became_active":"2024-05-15T13:57:17.393699+0000","last_became_peered":"2024-05-15T13:57:17.393699+0000","last_unstale":"2024-05-15T13:57:19.389344+0000","last_undegraded":"2024-05-15T13:57:19.389344+0000","last_fullsized":"2024-05-15T13:57:19.389344+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:05:37.510370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64739,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":183,"num_read_kb":157,"num_write":164,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":194217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645335,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":860160,"data_stored":415573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841564,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36856,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708296,"statfs":{"total":95995035648,"available":95957295104,"internally_reserved":0,"allocated":1556480,"data_stored":1067792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005089,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":876544,"data_stored":415809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201318,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36192,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708960,"statfs":{"total":95995035648,"available":95957975040,"internally_reserved":0,"allocated":876544,"data_stored":415729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397547,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28452,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716700,"statfs":{"total":95995035648,"available":95965900800,"internally_reserved":0,"allocated":1470464,"data_stored":1008050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561073,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":864256,"data_stored":415198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":475338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36200,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708952,"statfs":{"total":95995035648,"available":95957966848,"internally_reserved":0,"allocated":884736,"data_stored":416469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":477079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149759,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":880640,"data_stored":415509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953531,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":868352,"data_stored":416026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757302,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32624,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712528,"statfs":{"total":95995035648,"available":95961628672,"internally_reserved":0,"allocated":1482752,"data_stored":1007343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-15T13:58:04.731 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-15T13:58:04.731 INFO:tasks.cephadm:Setting up client nodes... 2024-05-15T13:58:04.732 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-15T13:58:04.990 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:58:05.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:05 smithi052 ceph-mon[36190]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-15T13:58:05.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:05 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-05-15T13:58:05.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:05 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-05-15T13:58:05.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:05 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-05-15T13:58:05.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:05 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-05-15T13:58:05.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:05 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-05-15T13:58:05.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:05 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-05-15T13:58:05.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:05 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-05-15T13:58:05.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:05 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]: dispatch 2024-05-15T13:58:05.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:05 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-05-15T13:58:05.343 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:05 smithi033 ceph-mon[31650]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-15T13:58:05.343 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:05 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-05-15T13:58:05.344 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:05 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-05-15T13:58:05.344 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:05 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-05-15T13:58:05.344 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:05 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-05-15T13:58:05.344 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:05 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-05-15T13:58:05.344 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:05 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-05-15T13:58:05.344 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:05 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-05-15T13:58:05.344 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:05 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]: dispatch 2024-05-15T13:58:05.344 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:05 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-05-15T13:58:05.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:05 smithi081 ceph-mon[36434]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-15T13:58:05.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:05 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-05-15T13:58:05.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:05 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-05-15T13:58:05.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:05 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-05-15T13:58:05.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:05 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-05-15T13:58:05.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:05 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-05-15T13:58:05.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:05 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-05-15T13:58:05.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:05 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-05-15T13:58:05.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:05 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]: dispatch 2024-05-15T13:58:05.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:05 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-05-15T13:58:06.140 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:06 smithi033 ceph-mon[31650]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-05-15T13:58:06.140 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:06 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-05-15T13:58:06.140 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:06 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-05-15T13:58:06.140 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:06 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-05-15T13:58:06.140 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:06 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-05-15T13:58:06.140 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:06 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-05-15T13:58:06.140 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:06 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-05-15T13:58:06.140 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:06 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-05-15T13:58:06.140 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:06 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2024-05-15T13:58:06.141 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:06 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-05-15T13:58:06.141 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:06 smithi033 ceph-mon[31650]: osdmap e81: 12 total, 12 up, 12 in 2024-05-15T13:58:06.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:06 smithi052 ceph-mon[36190]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-05-15T13:58:06.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:06 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-05-15T13:58:06.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:06 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-05-15T13:58:06.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:06 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-05-15T13:58:06.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:06 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-05-15T13:58:06.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:06 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-05-15T13:58:06.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:06 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-05-15T13:58:06.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:06 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-05-15T13:58:06.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:06 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2024-05-15T13:58:06.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:06 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-05-15T13:58:06.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:06 smithi052 ceph-mon[36190]: osdmap e81: 12 total, 12 up, 12 in 2024-05-15T13:58:06.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:06 smithi081 ceph-mon[36434]: pgmap v310: 97 pgs: 97 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-05-15T13:58:06.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:06 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-05-15T13:58:06.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:06 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-05-15T13:58:06.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:06 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-05-15T13:58:06.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:06 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-05-15T13:58:06.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:06 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-05-15T13:58:06.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:06 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-05-15T13:58:06.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:06 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-05-15T13:58:06.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:06 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2024-05-15T13:58:06.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:06 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-05-15T13:58:06.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:06 smithi081 ceph-mon[36434]: osdmap e81: 12 total, 12 up, 12 in 2024-05-15T13:58:06.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.658+0000 7f63ba3d3700 1 -- 172.21.15.33:0/1189070163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63b41035f0 msgr2=0x7f63b4103a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:06.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.658+0000 7f63ba3d3700 1 --2- 172.21.15.33:0/1189070163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63b41035f0 0x7f63b4103a10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f63a40099c0 tx=0x7f63a4009cd0 comp rx=0 tx=0).stop 2024-05-15T13:58:06.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.658+0000 7f63ba3d3700 1 -- 172.21.15.33:0/1189070163 shutdown_connections 2024-05-15T13:58:06.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.658+0000 7f63ba3d3700 1 --2- 172.21.15.33:0/1189070163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63b4105190 0x7f63b410ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:06.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.658+0000 7f63ba3d3700 1 --2- 172.21.15.33:0/1189070163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63b41035f0 0x7f63b4103a10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:06.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.658+0000 7f63ba3d3700 1 --2- 172.21.15.33:0/1189070163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63b41047f0 0x7f63b4104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:06.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.658+0000 7f63ba3d3700 1 -- 172.21.15.33:0/1189070163 >> 172.21.15.33:0/1189070163 conn(0x7f63b40feb90 msgr2=0x7f63b4100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:06.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.658+0000 7f63ba3d3700 1 -- 172.21.15.33:0/1189070163 shutdown_connections 2024-05-15T13:58:06.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.658+0000 7f63ba3d3700 1 -- 172.21.15.33:0/1189070163 wait complete. 2024-05-15T13:58:06.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.659+0000 7f63ba3d3700 1 Processor -- start 2024-05-15T13:58:06.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.659+0000 7f63ba3d3700 1 -- start start 2024-05-15T13:58:06.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.659+0000 7f63ba3d3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63b41035f0 0x7f63b4199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:06.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.660+0000 7f63ba3d3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63b41047f0 0x7f63b4199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:06.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.660+0000 7f63ba3d3700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63b4105190 0x7f63b41a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:06.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.660+0000 7f63ba3d3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f63b410f0b0 con 0x7f63b41047f0 2024-05-15T13:58:06.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.660+0000 7f63ba3d3700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f63b410ef30 con 0x7f63b41035f0 2024-05-15T13:58:06.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.660+0000 7f63ba3d3700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f63b410f230 con 0x7f63b4105190 2024-05-15T13:58:06.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.660+0000 7f63abfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63b41047f0 0x7f63b4199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:06.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.660+0000 7f63abfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63b41047f0 0x7f63b4199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55490/0 (socket says 172.21.15.33:55490) 2024-05-15T13:58:06.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.660+0000 7f63abfff700 1 -- 172.21.15.33:0/2919073911 learned_addr learned my addr 172.21.15.33:0/2919073911 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:58:06.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.660+0000 7f63b3fff700 1 --2- 172.21.15.33:0/2919073911 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63b41035f0 0x7f63b4199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:06.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.660+0000 7f63b8970700 1 --2- 172.21.15.33:0/2919073911 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63b4105190 0x7f63b41a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:06.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.660+0000 7f63abfff700 1 -- 172.21.15.33:0/2919073911 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63b4105190 msgr2=0x7f63b41a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:06.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.660+0000 7f63abfff700 1 --2- 172.21.15.33:0/2919073911 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63b4105190 0x7f63b41a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:06.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.660+0000 7f63abfff700 1 -- 172.21.15.33:0/2919073911 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63b41035f0 msgr2=0x7f63b4199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:06.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.660+0000 7f63abfff700 1 --2- 172.21.15.33:0/2919073911 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63b41035f0 0x7f63b4199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:06.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.660+0000 7f63abfff700 1 -- 172.21.15.33:0/2919073911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b41a12c0 con 0x7f63b41047f0 2024-05-15T13:58:06.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.660+0000 7f63b8970700 1 --2- 172.21.15.33:0/2919073911 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63b4105190 0x7f63b41a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T13:58:06.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.660+0000 7f63abfff700 1 --2- 172.21.15.33:0/2919073911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63b41047f0 0x7f63b4199ba0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f639c00d9b0 tx=0x7f639c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:06.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.660+0000 7f63b1ffb700 1 -- 172.21.15.33:0/2919073911 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639c016070 con 0x7f63b41047f0 2024-05-15T13:58:06.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.660+0000 7f63b1ffb700 1 -- 172.21.15.33:0/2919073911 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f639c004830 con 0x7f63b41047f0 2024-05-15T13:58:06.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.660+0000 7f63ba3d3700 1 -- 172.21.15.33:0/2919073911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b41a15b0 con 0x7f63b41047f0 2024-05-15T13:58:06.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.660+0000 7f63b1ffb700 1 -- 172.21.15.33:0/2919073911 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639c005500 con 0x7f63b41047f0 2024-05-15T13:58:06.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.660+0000 7f63ba3d3700 1 -- 172.21.15.33:0/2919073911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b41a1af0 con 0x7f63b41047f0 2024-05-15T13:58:06.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.662+0000 7f63ba3d3700 1 -- 172.21.15.33:0/2919073911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63b4051a80 con 0x7f63b41047f0 2024-05-15T13:58:06.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.663+0000 7f63b1ffb700 1 -- 172.21.15.33:0/2919073911 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f639c004220 con 0x7f63b41047f0 2024-05-15T13:58:06.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.664+0000 7f63b1ffb700 1 --2- 172.21.15.33:0/2919073911 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f63a00b29b0 0x7f63a00b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:06.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.664+0000 7f63b3fff700 1 --2- 172.21.15.33:0/2919073911 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f63a00b29b0 0x7f63a00b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:06.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.664+0000 7f63b1ffb700 1 -- 172.21.15.33:0/2919073911 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f639c0e2810 con 0x7f63b41047f0 2024-05-15T13:58:06.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.664+0000 7f63b3fff700 1 --2- 172.21.15.33:0/2919073911 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f63a00b29b0 0x7f63a00b4e70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009f50 tx=0x7f63a400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:06.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.667+0000 7f63b1ffb700 1 -- 172.21.15.33:0/2919073911 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f639c01f020 con 0x7f63b41047f0 2024-05-15T13:58:06.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.974+0000 7f63ba3d3700 1 -- 172.21.15.33:0/2919073911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f63b4064150 con 0x7f63b41047f0 2024-05-15T13:58:06.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.977+0000 7f63b1ffb700 1 -- 172.21.15.33:0/2919073911 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f639c0aa390 con 0x7f63b41047f0 2024-05-15T13:58:06.978 INFO:teuthology.orchestra.run.smithi033.stdout:[client.0] 2024-05-15T13:58:06.979 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQBuv0Rm9X0qOhAAufyRUDV4XKzZLJe1zOg51Q== 2024-05-15T13:58:06.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.980+0000 7f63ba3d3700 1 -- 172.21.15.33:0/2919073911 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f63a00b29b0 msgr2=0x7f63a00b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:06.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.980+0000 7f63ba3d3700 1 --2- 172.21.15.33:0/2919073911 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f63a00b29b0 0x7f63a00b4e70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009f50 tx=0x7f63a400b540 comp rx=0 tx=0).stop 2024-05-15T13:58:06.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.980+0000 7f63ba3d3700 1 -- 172.21.15.33:0/2919073911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63b41047f0 msgr2=0x7f63b4199ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:06.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.980+0000 7f63ba3d3700 1 --2- 172.21.15.33:0/2919073911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63b41047f0 0x7f63b4199ba0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f639c00d9b0 tx=0x7f639c00dcc0 comp rx=0 tx=0).stop 2024-05-15T13:58:06.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.980+0000 7f63ba3d3700 1 -- 172.21.15.33:0/2919073911 shutdown_connections 2024-05-15T13:58:06.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.980+0000 7f63ba3d3700 1 --2- 172.21.15.33:0/2919073911 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f63a00b29b0 0x7f63a00b4e70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:06.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.980+0000 7f63ba3d3700 1 --2- 172.21.15.33:0/2919073911 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63b4105190 0x7f63b41a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:06.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.980+0000 7f63ba3d3700 1 --2- 172.21.15.33:0/2919073911 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63b41035f0 0x7f63b4199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:06.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.980+0000 7f63ba3d3700 1 --2- 172.21.15.33:0/2919073911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63b41047f0 0x7f63b4199ba0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:06.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.980+0000 7f63ba3d3700 1 -- 172.21.15.33:0/2919073911 >> 172.21.15.33:0/2919073911 conn(0x7f63b40feb90 msgr2=0x7f63b4107a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:06.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.980+0000 7f63ba3d3700 1 -- 172.21.15.33:0/2919073911 shutdown_connections 2024-05-15T13:58:06.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:06.980+0000 7f63ba3d3700 1 -- 172.21.15.33:0/2919073911 wait complete. 2024-05-15T13:58:07.281 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:07 smithi033 ceph-mon[31650]: osdmap e82: 12 total, 12 up, 12 in 2024-05-15T13:58:07.281 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:07 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/2919073911' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-15T13:58:07.281 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:07 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/2919073911' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-15T13:58:07.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:07 smithi052 ceph-mon[36190]: osdmap e82: 12 total, 12 up, 12 in 2024-05-15T13:58:07.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:07 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/2919073911' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-15T13:58:07.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:07 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/2919073911' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-15T13:58:07.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:07 smithi081 ceph-mon[36434]: osdmap e82: 12 total, 12 up, 12 in 2024-05-15T13:58:07.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:07 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/2919073911' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-15T13:58:07.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:07 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/2919073911' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-15T13:58:07.636 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-15T13:58:07.636 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-15T13:58:07.636 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-15T13:58:07.676 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-15T13:58:07.677 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-15T13:58:07.677 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph mgr dump --format=json 2024-05-15T13:58:07.968 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:58:08.325 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:08 smithi033 ceph-mon[31650]: pgmap v313: 97 pgs: 3 peering, 94 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 6 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-05-15T13:58:08.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:08 smithi052 ceph-mon[36190]: pgmap v313: 97 pgs: 3 peering, 94 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 6 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-05-15T13:58:08.356 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:08 smithi081 ceph-mon[36434]: pgmap v313: 97 pgs: 3 peering, 94 active+clean; 640 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1023 B/s wr, 6 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-05-15T13:58:09.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.616+0000 7f7ec10dc700 1 -- 172.21.15.33:0/2489735991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ebc102e00 msgr2=0x7f7ebc10f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:09.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.616+0000 7f7ec10dc700 1 --2- 172.21.15.33:0/2489735991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ebc102e00 0x7f7ebc10f2f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4009990 tx=0x7f7eb4009ca0 comp rx=0 tx=0).stop 2024-05-15T13:58:09.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.616+0000 7f7ec10dc700 1 -- 172.21.15.33:0/2489735991 shutdown_connections 2024-05-15T13:58:09.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.616+0000 7f7ec10dc700 1 --2- 172.21.15.33:0/2489735991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ebc102e00 0x7f7ebc10f2f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:09.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.616+0000 7f7ec10dc700 1 --2- 172.21.15.33:0/2489735991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ebc108520 0x7f7ebc108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:09.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.616+0000 7f7ec10dc700 1 --2- 172.21.15.33:0/2489735991 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7ebc102440 0x7f7ebc1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:09.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.616+0000 7f7ec10dc700 1 -- 172.21.15.33:0/2489735991 >> 172.21.15.33:0/2489735991 conn(0x7f7ebc0793f0 msgr2=0x7f7ebc079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:09.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.616+0000 7f7ec10dc700 1 -- 172.21.15.33:0/2489735991 shutdown_connections 2024-05-15T13:58:09.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.616+0000 7f7ec10dc700 1 -- 172.21.15.33:0/2489735991 wait complete. 2024-05-15T13:58:09.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.617+0000 7f7ec10dc700 1 Processor -- start 2024-05-15T13:58:09.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.617+0000 7f7ec10dc700 1 -- start start 2024-05-15T13:58:09.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.618+0000 7f7ec10dc700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7ebc102440 0x7f7ebc19bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:09.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.618+0000 7f7ec10dc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ebc102e00 0x7f7ebc19c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:09.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.618+0000 7f7ec10dc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ebc108520 0x7f7ebc1a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:09.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.618+0000 7f7ec10dc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc1125f0 con 0x7f7ebc102e00 2024-05-15T13:58:09.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.618+0000 7f7ec10dc700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc112470 con 0x7f7ebc108520 2024-05-15T13:58:09.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.618+0000 7f7ec10dc700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f7ebc112770 con 0x7f7ebc102440 2024-05-15T13:58:09.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.618+0000 7f7eba59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ebc102e00 0x7f7ebc19c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:09.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.618+0000 7f7eba59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ebc102e00 0x7f7ebc19c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55512/0 (socket says 172.21.15.33:55512) 2024-05-15T13:58:09.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.618+0000 7f7eba59c700 1 -- 172.21.15.33:0/796476114 learned_addr learned my addr 172.21.15.33:0/796476114 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:58:09.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.618+0000 7f7ebb59e700 1 --2- 172.21.15.33:0/796476114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ebc108520 0x7f7ebc1a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:09.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.618+0000 7f7ebad9d700 1 --2- 172.21.15.33:0/796476114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7ebc102440 0x7f7ebc19bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:09.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.618+0000 7f7eba59c700 1 -- 172.21.15.33:0/796476114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7ebc102440 msgr2=0x7f7ebc19bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:09.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.618+0000 7f7eba59c700 1 --2- 172.21.15.33:0/796476114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7ebc102440 0x7f7ebc19bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:09.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.618+0000 7f7eba59c700 1 -- 172.21.15.33:0/796476114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ebc108520 msgr2=0x7f7ebc1a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:09.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.618+0000 7f7eba59c700 1 --2- 172.21.15.33:0/796476114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ebc108520 0x7f7ebc1a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:09.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.618+0000 7f7eba59c700 1 -- 172.21.15.33:0/796476114 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ebc1a0dc0 con 0x7f7ebc102e00 2024-05-15T13:58:09.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.618+0000 7f7ebad9d700 1 --2- 172.21.15.33:0/796476114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7ebc102440 0x7f7ebc19bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T13:58:09.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.618+0000 7f7ebb59e700 1 --2- 172.21.15.33:0/796476114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ebc108520 0x7f7ebc1a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T13:58:09.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.618+0000 7f7eba59c700 1 --2- 172.21.15.33:0/796476114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ebc102e00 0x7f7ebc19c2f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7eb000d980 tx=0x7f7eb000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:09.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.619+0000 7f7ea3fff700 1 -- 172.21.15.33:0/796476114 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0017070 con 0x7f7ebc102e00 2024-05-15T13:58:09.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.619+0000 7f7ec10dc700 1 -- 172.21.15.33:0/796476114 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ebc1a10b0 con 0x7f7ebc102e00 2024-05-15T13:58:09.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.619+0000 7f7ea3fff700 1 -- 172.21.15.33:0/796476114 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7eb0004830 con 0x7f7ebc102e00 2024-05-15T13:58:09.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.619+0000 7f7ea3fff700 1 -- 172.21.15.33:0/796476114 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0005440 con 0x7f7ebc102e00 2024-05-15T13:58:09.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.619+0000 7f7ec10dc700 1 -- 172.21.15.33:0/796476114 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ebc1a8930 con 0x7f7ebc102e00 2024-05-15T13:58:09.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.620+0000 7f7ec10dc700 1 -- 172.21.15.33:0/796476114 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ebc06a230 con 0x7f7ebc102e00 2024-05-15T13:58:09.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.621+0000 7f7ea3fff700 1 -- 172.21.15.33:0/796476114 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f7eb00055e0 con 0x7f7ebc102e00 2024-05-15T13:58:09.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.622+0000 7f7ea3fff700 1 --2- 172.21.15.33:0/796476114 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f7ea40b2990 0x7f7ea40b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:09.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.622+0000 7f7ebad9d700 1 --2- 172.21.15.33:0/796476114 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f7ea40b2990 0x7f7ea40b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:09.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.622+0000 7f7ea3fff700 1 -- 172.21.15.33:0/796476114 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f7eb00e26d0 con 0x7f7ebc102e00 2024-05-15T13:58:09.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.623+0000 7f7ebad9d700 1 --2- 172.21.15.33:0/796476114 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f7ea40b2990 0x7f7ea40b4e50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7eac007700 tx=0x7f7eac007690 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:09.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.625+0000 7f7ea3fff700 1 -- 172.21.15.33:0/796476114 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7eb0020020 con 0x7f7ebc102e00 2024-05-15T13:58:09.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.906+0000 7f7ec10dc700 1 -- 172.21.15.33:0/796476114 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f7ebc1a8bc0 con 0x7f7ebc102e00 2024-05-15T13:58:09.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.912+0000 7f7ea3fff700 1 -- 172.21.15.33:0/796476114 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288646 (secure 0 0 0) 0x7f7eb00aa310 con 0x7f7ebc102e00 2024-05-15T13:58:09.917 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:58:09.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.915+0000 7f7ec10dc700 1 -- 172.21.15.33:0/796476114 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f7ea40b2990 msgr2=0x7f7ea40b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:09.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.915+0000 7f7ec10dc700 1 --2- 172.21.15.33:0/796476114 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f7ea40b2990 0x7f7ea40b4e50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7eac007700 tx=0x7f7eac007690 comp rx=0 tx=0).stop 2024-05-15T13:58:09.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.915+0000 7f7ec10dc700 1 -- 172.21.15.33:0/796476114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ebc102e00 msgr2=0x7f7ebc19c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:09.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.915+0000 7f7ec10dc700 1 --2- 172.21.15.33:0/796476114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ebc102e00 0x7f7ebc19c2f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7eb000d980 tx=0x7f7eb000dd40 comp rx=0 tx=0).stop 2024-05-15T13:58:09.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.915+0000 7f7ec10dc700 1 -- 172.21.15.33:0/796476114 shutdown_connections 2024-05-15T13:58:09.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.915+0000 7f7ec10dc700 1 --2- 172.21.15.33:0/796476114 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f7ea40b2990 0x7f7ea40b4e50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:09.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.915+0000 7f7ec10dc700 1 --2- 172.21.15.33:0/796476114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7ebc108520 0x7f7ebc1a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:09.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.915+0000 7f7ec10dc700 1 --2- 172.21.15.33:0/796476114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7ebc102440 0x7f7ebc19bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:09.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.915+0000 7f7ec10dc700 1 --2- 172.21.15.33:0/796476114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ebc102e00 0x7f7ebc19c2f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:09.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.915+0000 7f7ec10dc700 1 -- 172.21.15.33:0/796476114 >> 172.21.15.33:0/796476114 conn(0x7f7ebc0793f0 msgr2=0x7f7ebc06b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:09.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.915+0000 7f7ec10dc700 1 -- 172.21.15.33:0/796476114 shutdown_connections 2024-05-15T13:58:09.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:09.915+0000 7f7ec10dc700 1 -- 172.21.15.33:0/796476114 wait complete. 2024-05-15T13:58:10.190 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:10 smithi033 ceph-mon[31650]: pgmap v314: 97 pgs: 3 peering, 94 active+clean; 640 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1023 B/s wr, 8 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-05-15T13:58:10.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:10 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/796476114' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-15T13:58:10.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:10 smithi052 ceph-mon[36190]: pgmap v314: 97 pgs: 3 peering, 94 active+clean; 640 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1023 B/s wr, 8 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-05-15T13:58:10.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:10 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/796476114' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-15T13:58:10.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:10 smithi081 ceph-mon[36434]: pgmap v314: 97 pgs: 3 peering, 94 active+clean; 640 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1023 B/s wr, 8 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-05-15T13:58:10.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:10 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/796476114' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-15T13:58:10.611 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1797827152},{"type":"v1","addr":"172.21.15.33:6801","nonce":1797827152}]},"active_addr":"172.21.15.33:6801/1797827152","active_change":"2024-05-15T13:50:04.881770+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.33:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":1564238110}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":2326878092}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":2259371407}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":2647407310}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":757488458}]}]} 2024-05-15T13:58:10.617 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-15T13:58:10.617 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-15T13:58:10.617 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-15T13:58:10.870 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:58:12.283 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:12 smithi033 ceph-mon[31650]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 16 B/s, 1 keys/s, 1 objects/s recovering 2024-05-15T13:58:12.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:12 smithi052 ceph-mon[36190]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 16 B/s, 1 keys/s, 1 objects/s recovering 2024-05-15T13:58:12.356 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:12 smithi081 ceph-mon[36434]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 16 B/s, 1 keys/s, 1 objects/s recovering 2024-05-15T13:58:12.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.507+0000 7fa60f4bf700 1 -- 172.21.15.33:0/1747194047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa608108910 msgr2=0x7fa608110490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:12.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.507+0000 7fa60f4bf700 1 --2- 172.21.15.33:0/1747194047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa608108910 0x7fa608110490 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa60000b0d0 tx=0x7fa60000b3e0 comp rx=0 tx=0).stop 2024-05-15T13:58:12.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.507+0000 7fa60f4bf700 1 -- 172.21.15.33:0/1747194047 shutdown_connections 2024-05-15T13:58:12.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.507+0000 7fa60f4bf700 1 --2- 172.21.15.33:0/1747194047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa608108910 0x7fa608110490 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:12.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.507+0000 7fa60f4bf700 1 --2- 172.21.15.33:0/1747194047 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa608103820 0x7fa608103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:12.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.507+0000 7fa60f4bf700 1 --2- 172.21.15.33:0/1747194047 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6081041d0 0x7fa6081081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:12.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.507+0000 7fa60f4bf700 1 -- 172.21.15.33:0/1747194047 >> 172.21.15.33:0/1747194047 conn(0x7fa60807a6b0 msgr2=0x7fa60807aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:12.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.507+0000 7fa60f4bf700 1 -- 172.21.15.33:0/1747194047 shutdown_connections 2024-05-15T13:58:12.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.507+0000 7fa60f4bf700 1 -- 172.21.15.33:0/1747194047 wait complete. 2024-05-15T13:58:12.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.508+0000 7fa60f4bf700 1 Processor -- start 2024-05-15T13:58:12.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.508+0000 7fa60f4bf700 1 -- start start 2024-05-15T13:58:12.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.508+0000 7fa60f4bf700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa608103820 0x7fa608075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:12.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.508+0000 7fa60f4bf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6081041d0 0x7fa608075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:12.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.508+0000 7fa60f4bf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa608108910 0x7fa6080762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:12.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.508+0000 7fa60f4bf700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa608113700 con 0x7fa6081041d0 2024-05-15T13:58:12.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.508+0000 7fa60f4bf700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa608113580 con 0x7fa608108910 2024-05-15T13:58:12.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.508+0000 7fa60f4bf700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa608113880 con 0x7fa608103820 2024-05-15T13:58:12.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.508+0000 7fa60ca5a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6081041d0 0x7fa608075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:12.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.508+0000 7fa60ca5a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6081041d0 0x7fa608075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55532/0 (socket says 172.21.15.33:55532) 2024-05-15T13:58:12.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.508+0000 7fa60ca5a700 1 -- 172.21.15.33:0/1189411872 learned_addr learned my addr 172.21.15.33:0/1189411872 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:58:12.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.508+0000 7fa60d25b700 1 --2- 172.21.15.33:0/1189411872 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa608103820 0x7fa608075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:12.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.508+0000 7fa60da5c700 1 --2- 172.21.15.33:0/1189411872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa608108910 0x7fa6080762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:12.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.508+0000 7fa60ca5a700 1 -- 172.21.15.33:0/1189411872 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa608103820 msgr2=0x7fa608075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:12.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.508+0000 7fa60ca5a700 1 --2- 172.21.15.33:0/1189411872 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa608103820 0x7fa608075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:12.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.508+0000 7fa60ca5a700 1 -- 172.21.15.33:0/1189411872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa608108910 msgr2=0x7fa6080762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:12.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.508+0000 7fa60ca5a700 1 --2- 172.21.15.33:0/1189411872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa608108910 0x7fa6080762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:12.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.508+0000 7fa60ca5a700 1 -- 172.21.15.33:0/1189411872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6081a8050 con 0x7fa6081041d0 2024-05-15T13:58:12.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.509+0000 7fa60ca5a700 1 --2- 172.21.15.33:0/1189411872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6081041d0 0x7fa608075d80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa5f800d950 tx=0x7fa5f800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:12.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.509+0000 7fa5fe7fc700 1 -- 172.21.15.33:0/1189411872 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8016070 con 0x7fa6081041d0 2024-05-15T13:58:12.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.509+0000 7fa5fe7fc700 1 -- 172.21.15.33:0/1189411872 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5f800fd40 con 0x7fa6081041d0 2024-05-15T13:58:12.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.509+0000 7fa60f4bf700 1 -- 172.21.15.33:0/1189411872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6081a8340 con 0x7fa6081041d0 2024-05-15T13:58:12.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.509+0000 7fa5fe7fc700 1 -- 172.21.15.33:0/1189411872 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f800e3b0 con 0x7fa6081041d0 2024-05-15T13:58:12.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.509+0000 7fa60f4bf700 1 -- 172.21.15.33:0/1189411872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6081a8880 con 0x7fa6081041d0 2024-05-15T13:58:12.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.510+0000 7fa60f4bf700 1 -- 172.21.15.33:0/1189411872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa60810bae0 con 0x7fa6081041d0 2024-05-15T13:58:12.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.511+0000 7fa5fe7fc700 1 -- 172.21.15.33:0/1189411872 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa5f8019070 con 0x7fa6081041d0 2024-05-15T13:58:12.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.512+0000 7fa5fe7fc700 1 --2- 172.21.15.33:0/1189411872 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fa5f40b2910 0x7fa5f40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:12.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.512+0000 7fa60d25b700 1 --2- 172.21.15.33:0/1189411872 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fa5f40b2910 0x7fa5f40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:12.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.512+0000 7fa5fe7fc700 1 -- 172.21.15.33:0/1189411872 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fa5f80e1de0 con 0x7fa6081041d0 2024-05-15T13:58:12.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.512+0000 7fa60d25b700 1 --2- 172.21.15.33:0/1189411872 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fa5f40b2910 0x7fa5f40b4dd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa604005950 tx=0x7fa60400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:12.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.517+0000 7fa5fe7fc700 1 -- 172.21.15.33:0/1189411872 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa5f80a99a0 con 0x7fa6081041d0 2024-05-15T13:58:12.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.735+0000 7fa60f4bf700 1 -- 172.21.15.33:0/1189411872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa60810b4c0 con 0x7fa6081041d0 2024-05-15T13:58:12.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.736+0000 7fa5fe7fc700 1 -- 172.21.15.33:0/1189411872 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22593 (secure 0 0 0) 0x7fa5f80a92a0 con 0x7fa6081041d0 2024-05-15T13:58:12.738 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:58:12.738 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":82,"fsid":"bca78ca4-12c1-11ef-bc98-c7b262605968","created":"2024-05-15T13:49:10.327613+0000","modified":"2024-05-15T13:58:06.058012+0000","last_up_change":"2024-05-15T13:56:16.097711+0000","last_in_change":"2024-05-15T13:55:56.722143+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-15T13:52:32.970844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-15T13:57:12.324802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-15T13:57:15.898406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b434111-810f-450a-b63d-c97080e9f8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6803","nonce":887409130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6805","nonce":887409130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6809","nonce":887409130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6807","nonce":887409130}]},"public_addr":"172.21.15.33:6803/887409130","cluster_addr":"172.21.15.33:6805/887409130","heartbeat_back_addr":"172.21.15.33:6809/887409130","heartbeat_front_addr":"172.21.15.33:6807/887409130","state":["exists","up"]},{"osd":1,"uuid":"0b578d4e-a841-4b27-bc95-d6c3bae580fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6801","nonce":2794180003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6803","nonce":2794180003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6807","nonce":2794180003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6805","nonce":2794180003}]},"public_addr":"172.21.15.52:6801/2794180003","cluster_addr":"172.21.15.52:6803/2794180003","heartbeat_back_addr":"172.21.15.52:6807/2794180003","heartbeat_front_addr":"172.21.15.52:6805/2794180003","state":["exists","up"]},{"osd":2,"uuid":"0e9c9390-ed33-464b-8f39-bca75941e05e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6801","nonce":315610978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6803","nonce":315610978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6807","nonce":315610978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6805","nonce":315610978}]},"public_addr":"172.21.15.81:6801/315610978","cluster_addr":"172.21.15.81:6803/315610978","heartbeat_back_addr":"172.21.15.81:6807/315610978","heartbeat_front_addr":"172.21.15.81:6805/315610978","state":["exists","up"]},{"osd":3,"uuid":"ecdb24c1-fb15-42a5-9bae-436bbae9512a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6811","nonce":818533501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6813","nonce":818533501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6817","nonce":818533501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6815","nonce":818533501}]},"public_addr":"172.21.15.33:6811/818533501","cluster_addr":"172.21.15.33:6813/818533501","heartbeat_back_addr":"172.21.15.33:6817/818533501","heartbeat_front_addr":"172.21.15.33:6815/818533501","state":["exists","up"]},{"osd":4,"uuid":"706752a0-b17f-4563-a7a9-d5078bf0f3b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6809","nonce":3010133776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6811","nonce":3010133776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6815","nonce":3010133776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6813","nonce":3010133776}]},"public_addr":"172.21.15.52:6809/3010133776","cluster_addr":"172.21.15.52:6811/3010133776","heartbeat_back_addr":"172.21.15.52:6815/3010133776","heartbeat_front_addr":"172.21.15.52:6813/3010133776","state":["exists","up"]},{"osd":5,"uuid":"3d45ab97-f364-4903-974d-8acc961363d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6809","nonce":2132616297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6811","nonce":2132616297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6815","nonce":2132616297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6813","nonce":2132616297}]},"public_addr":"172.21.15.81:6809/2132616297","cluster_addr":"172.21.15.81:6811/2132616297","heartbeat_back_addr":"172.21.15.81:6815/2132616297","heartbeat_front_addr":"172.21.15.81:6813/2132616297","state":["exists","up"]},{"osd":6,"uuid":"3be524c5-dcf9-4742-b896-ea9a44a6b391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6819","nonce":1260691801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6821","nonce":1260691801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6825","nonce":1260691801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6823","nonce":1260691801}]},"public_addr":"172.21.15.33:6819/1260691801","cluster_addr":"172.21.15.33:6821/1260691801","heartbeat_back_addr":"172.21.15.33:6825/1260691801","heartbeat_front_addr":"172.21.15.33:6823/1260691801","state":["exists","up"]},{"osd":7,"uuid":"bda2f920-4f2c-4eed-8127-83c1973676e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6817","nonce":1654595813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6819","nonce":1654595813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6823","nonce":1654595813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6821","nonce":1654595813}]},"public_addr":"172.21.15.52:6817/1654595813","cluster_addr":"172.21.15.52:6819/1654595813","heartbeat_back_addr":"172.21.15.52:6823/1654595813","heartbeat_front_addr":"172.21.15.52:6821/1654595813","state":["exists","up"]},{"osd":8,"uuid":"550585b8-0df4-49ee-95c8-3d084744dbd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6817","nonce":3009661748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6819","nonce":3009661748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6823","nonce":3009661748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6821","nonce":3009661748}]},"public_addr":"172.21.15.81:6817/3009661748","cluster_addr":"172.21.15.81:6819/3009661748","heartbeat_back_addr":"172.21.15.81:6823/3009661748","heartbeat_front_addr":"172.21.15.81:6821/3009661748","state":["exists","up"]},{"osd":9,"uuid":"1d781846-26f3-4b23-b22c-c994a8509a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6827","nonce":2843220858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6829","nonce":2843220858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6833","nonce":2843220858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6831","nonce":2843220858}]},"public_addr":"172.21.15.33:6827/2843220858","cluster_addr":"172.21.15.33:6829/2843220858","heartbeat_back_addr":"172.21.15.33:6833/2843220858","heartbeat_front_addr":"172.21.15.33:6831/2843220858","state":["exists","up"]},{"osd":10,"uuid":"af58c940-fdc4-418b-a638-7dc44438b34c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6825","nonce":2422349831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6827","nonce":2422349831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6831","nonce":2422349831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6829","nonce":2422349831}]},"public_addr":"172.21.15.52:6825/2422349831","cluster_addr":"172.21.15.52:6827/2422349831","heartbeat_back_addr":"172.21.15.52:6831/2422349831","heartbeat_front_addr":"172.21.15.52:6829/2422349831","state":["exists","up"]},{"osd":11,"uuid":"ab84aaea-2086-4842-8bf4-628842fce46a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6825","nonce":326313090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6827","nonce":326313090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6831","nonce":326313090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6829","nonce":326313090}]},"public_addr":"172.21.15.81:6825/326313090","cluster_addr":"172.21.15.81:6827/326313090","heartbeat_back_addr":"172.21.15.81:6831/326313090","heartbeat_front_addr":"172.21.15.81:6829/326313090","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:51:41.493136+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:52:05.784773+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:52:29.769429+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:52:54.891620+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:53:19.412922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:53:43.921841+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:54:09.486732+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:54:34.064370+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:54:58.655304+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:55:24.548693+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:55:49.941191+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:56:14.919558+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/436121571":"2024-05-16T13:50:04.881656+0000","172.21.15.33:6801/2680375465":"2024-05-16T13:50:04.881656+0000","172.21.15.33:6800/2680375465":"2024-05-16T13:50:04.881656+0000","172.21.15.33:0/1022787179":"2024-05-16T13:49:37.564643+0000","172.21.15.33:0/3720711369":"2024-05-16T13:50:04.881656+0000","172.21.15.33:0/1282466838":"2024-05-16T13:49:37.564643+0000","172.21.15.33:0/3022248969":"2024-05-16T13:49:37.564643+0000","172.21.15.33:6801/3303495364":"2024-05-16T13:49:37.564643+0000","172.21.15.33:6800/3303495364":"2024-05-16T13:49:37.564643+0000","172.21.15.33:0/4282812692":"2024-05-16T13:50:04.881656+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-15T13:58:12.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.738+0000 7fa60f4bf700 1 -- 172.21.15.33:0/1189411872 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fa5f40b2910 msgr2=0x7fa5f40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:12.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.738+0000 7fa60f4bf700 1 --2- 172.21.15.33:0/1189411872 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fa5f40b2910 0x7fa5f40b4dd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa604005950 tx=0x7fa60400a300 comp rx=0 tx=0).stop 2024-05-15T13:58:12.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.738+0000 7fa60f4bf700 1 -- 172.21.15.33:0/1189411872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6081041d0 msgr2=0x7fa608075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:12.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.738+0000 7fa60f4bf700 1 --2- 172.21.15.33:0/1189411872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6081041d0 0x7fa608075d80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa5f800d950 tx=0x7fa5f800dc60 comp rx=0 tx=0).stop 2024-05-15T13:58:12.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.738+0000 7fa60f4bf700 1 -- 172.21.15.33:0/1189411872 shutdown_connections 2024-05-15T13:58:12.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.738+0000 7fa60f4bf700 1 --2- 172.21.15.33:0/1189411872 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fa5f40b2910 0x7fa5f40b4dd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:12.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.738+0000 7fa60f4bf700 1 --2- 172.21.15.33:0/1189411872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa608108910 0x7fa6080762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:12.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.738+0000 7fa60f4bf700 1 --2- 172.21.15.33:0/1189411872 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa608103820 0x7fa608075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:12.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.738+0000 7fa60f4bf700 1 --2- 172.21.15.33:0/1189411872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6081041d0 0x7fa608075d80 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:12.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.738+0000 7fa60f4bf700 1 -- 172.21.15.33:0/1189411872 >> 172.21.15.33:0/1189411872 conn(0x7fa60807a6b0 msgr2=0x7fa608101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:12.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.739+0000 7fa60f4bf700 1 -- 172.21.15.33:0/1189411872 shutdown_connections 2024-05-15T13:58:12.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:12.739+0000 7fa60f4bf700 1 -- 172.21.15.33:0/1189411872 wait complete. 2024-05-15T13:58:13.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:13 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/1189411872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T13:58:13.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:13 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:58:13.356 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:13 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/1189411872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T13:58:13.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:13 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:58:13.396 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:13 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/1189411872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T13:58:13.397 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:13 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:58:13.425 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-15T13:58:13.425 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-15T13:58:13.694 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:58:14.142 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:14 smithi052 ceph-mon[36190]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 16 B/s, 1 keys/s, 1 objects/s recovering 2024-05-15T13:58:14.190 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:14 smithi033 ceph-mon[31650]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 16 B/s, 1 keys/s, 1 objects/s recovering 2024-05-15T13:58:14.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:14 smithi081 ceph-mon[36434]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 16 B/s, 1 keys/s, 1 objects/s recovering 2024-05-15T13:58:15.762 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:15 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:58:15.762 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:15 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:58:15.762 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:15 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:58:15.762 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:15 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:58:15.762 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:15 smithi033 ceph-mon[31650]: pgmap v317: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2024-05-15T13:58:15.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.866+0000 7f57b9015700 1 -- 172.21.15.33:0/3211608854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b4108910 msgr2=0x7f57b4110490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:15.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.866+0000 7f57b9015700 1 --2- 172.21.15.33:0/3211608854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b4108910 0x7f57b4110490 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f57ac00b0d0 tx=0x7f57ac00b3e0 comp rx=0 tx=0).stop 2024-05-15T13:58:15.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.866+0000 7f57b9015700 1 -- 172.21.15.33:0/3211608854 shutdown_connections 2024-05-15T13:58:15.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.866+0000 7f57b9015700 1 --2- 172.21.15.33:0/3211608854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b4108910 0x7f57b4110490 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:15.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.866+0000 7f57b9015700 1 --2- 172.21.15.33:0/3211608854 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57b4103820 0x7f57b4103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:15.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.866+0000 7f57b9015700 1 --2- 172.21.15.33:0/3211608854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57b41041d0 0x7f57b41081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:15.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.866+0000 7f57b9015700 1 -- 172.21.15.33:0/3211608854 >> 172.21.15.33:0/3211608854 conn(0x7f57b407a6b0 msgr2=0x7f57b407aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:15.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.866+0000 7f57b9015700 1 -- 172.21.15.33:0/3211608854 shutdown_connections 2024-05-15T13:58:15.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.866+0000 7f57b9015700 1 -- 172.21.15.33:0/3211608854 wait complete. 2024-05-15T13:58:15.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.867+0000 7f57b9015700 1 Processor -- start 2024-05-15T13:58:15.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.867+0000 7f57b9015700 1 -- start start 2024-05-15T13:58:15.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.867+0000 7f57b9015700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57b4103820 0x7f57b4075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:15.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.867+0000 7f57b9015700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b41041d0 0x7f57b4075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:15.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.867+0000 7f57b9015700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57b4108910 0x7f57b40762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:15.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.867+0000 7f57b9015700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f57b41136e0 con 0x7f57b41041d0 2024-05-15T13:58:15.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.867+0000 7f57b9015700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f57b4113560 con 0x7f57b4108910 2024-05-15T13:58:15.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.868+0000 7f57b9015700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f57b4113860 con 0x7f57b4103820 2024-05-15T13:58:15.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.868+0000 7f57b259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b41041d0 0x7f57b4075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:15.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.868+0000 7f57b259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b41041d0 0x7f57b4075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56538/0 (socket says 172.21.15.33:56538) 2024-05-15T13:58:15.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.868+0000 7f57b259c700 1 -- 172.21.15.33:0/2567834262 learned_addr learned my addr 172.21.15.33:0/2567834262 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:58:15.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.868+0000 7f57b359e700 1 --2- 172.21.15.33:0/2567834262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57b4108910 0x7f57b40762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:15.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.868+0000 7f57b2d9d700 1 --2- 172.21.15.33:0/2567834262 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57b4103820 0x7f57b4075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:15.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.868+0000 7f57b259c700 1 -- 172.21.15.33:0/2567834262 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57b4103820 msgr2=0x7f57b4075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:15.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.868+0000 7f57b259c700 1 --2- 172.21.15.33:0/2567834262 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57b4103820 0x7f57b4075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:15.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.868+0000 7f57b259c700 1 -- 172.21.15.33:0/2567834262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57b4108910 msgr2=0x7f57b40762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:15.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.868+0000 7f57b259c700 1 --2- 172.21.15.33:0/2567834262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57b4108910 0x7f57b40762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:15.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.868+0000 7f57b259c700 1 -- 172.21.15.33:0/2567834262 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b41a7fd0 con 0x7f57b41041d0 2024-05-15T13:58:15.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.868+0000 7f57b259c700 1 --2- 172.21.15.33:0/2567834262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b41041d0 0x7f57b4075d80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f57a800d950 tx=0x7f57a800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:15.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.868+0000 7f579bfff700 1 -- 172.21.15.33:0/2567834262 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a8016070 con 0x7f57b41041d0 2024-05-15T13:58:15.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.868+0000 7f57b9015700 1 -- 172.21.15.33:0/2567834262 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57b41a82c0 con 0x7f57b41041d0 2024-05-15T13:58:15.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.868+0000 7f579bfff700 1 -- 172.21.15.33:0/2567834262 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57a800fd40 con 0x7f57b41041d0 2024-05-15T13:58:15.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.868+0000 7f579bfff700 1 -- 172.21.15.33:0/2567834262 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a800e3b0 con 0x7f57b41041d0 2024-05-15T13:58:15.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.868+0000 7f57b9015700 1 -- 172.21.15.33:0/2567834262 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57b41a8850 con 0x7f57b41041d0 2024-05-15T13:58:15.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.869+0000 7f57b9015700 1 -- 172.21.15.33:0/2567834262 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57b406a230 con 0x7f57b41041d0 2024-05-15T13:58:15.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.871+0000 7f579bfff700 1 -- 172.21.15.33:0/2567834262 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f57a8019070 con 0x7f57b41041d0 2024-05-15T13:58:15.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.872+0000 7f579bfff700 1 --2- 172.21.15.33:0/2567834262 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f579c0b2910 0x7f579c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:15.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.872+0000 7f57b2d9d700 1 --2- 172.21.15.33:0/2567834262 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f579c0b2910 0x7f579c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:15.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.872+0000 7f579bfff700 1 -- 172.21.15.33:0/2567834262 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f57a80e1da0 con 0x7f57b41041d0 2024-05-15T13:58:15.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.872+0000 7f57b2d9d700 1 --2- 172.21.15.33:0/2567834262 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f579c0b2910 0x7f579c0b4dd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f57a400ac50 tx=0x7f57a400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:15.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:15.875+0000 7f579bfff700 1 -- 172.21.15.33:0/2567834262 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f57a80a9920 con 0x7f57b41041d0 2024-05-15T13:58:16.090 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:15 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:58:16.090 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:15 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:58:16.090 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:15 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:58:16.090 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:15 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:58:16.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:15 smithi052 ceph-mon[36190]: pgmap v317: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2024-05-15T13:58:16.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:15 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:58:16.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:15 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:58:16.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:15 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:58:16.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:15 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:58:16.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:15 smithi081 ceph-mon[36434]: pgmap v317: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 13 B/s, 1 keys/s, 1 objects/s recovering 2024-05-15T13:58:16.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:16.134+0000 7f57b9015700 1 -- 172.21.15.33:0/2567834262 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f57b402e560 con 0x7f57b41041d0 2024-05-15T13:58:16.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:16.136+0000 7f579bfff700 1 -- 172.21.15.33:0/2567834262 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22593 (secure 0 0 0) 0x7f57a80a9220 con 0x7f57b41041d0 2024-05-15T13:58:16.138 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:58:16.138 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":82,"fsid":"bca78ca4-12c1-11ef-bc98-c7b262605968","created":"2024-05-15T13:49:10.327613+0000","modified":"2024-05-15T13:58:06.058012+0000","last_up_change":"2024-05-15T13:56:16.097711+0000","last_in_change":"2024-05-15T13:55:56.722143+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-15T13:52:32.970844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-15T13:57:12.324802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-15T13:57:15.898406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b434111-810f-450a-b63d-c97080e9f8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6803","nonce":887409130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6805","nonce":887409130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6809","nonce":887409130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6807","nonce":887409130}]},"public_addr":"172.21.15.33:6803/887409130","cluster_addr":"172.21.15.33:6805/887409130","heartbeat_back_addr":"172.21.15.33:6809/887409130","heartbeat_front_addr":"172.21.15.33:6807/887409130","state":["exists","up"]},{"osd":1,"uuid":"0b578d4e-a841-4b27-bc95-d6c3bae580fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6801","nonce":2794180003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6803","nonce":2794180003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6807","nonce":2794180003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6805","nonce":2794180003}]},"public_addr":"172.21.15.52:6801/2794180003","cluster_addr":"172.21.15.52:6803/2794180003","heartbeat_back_addr":"172.21.15.52:6807/2794180003","heartbeat_front_addr":"172.21.15.52:6805/2794180003","state":["exists","up"]},{"osd":2,"uuid":"0e9c9390-ed33-464b-8f39-bca75941e05e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6801","nonce":315610978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6803","nonce":315610978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6807","nonce":315610978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6805","nonce":315610978}]},"public_addr":"172.21.15.81:6801/315610978","cluster_addr":"172.21.15.81:6803/315610978","heartbeat_back_addr":"172.21.15.81:6807/315610978","heartbeat_front_addr":"172.21.15.81:6805/315610978","state":["exists","up"]},{"osd":3,"uuid":"ecdb24c1-fb15-42a5-9bae-436bbae9512a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6811","nonce":818533501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6813","nonce":818533501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6817","nonce":818533501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6815","nonce":818533501}]},"public_addr":"172.21.15.33:6811/818533501","cluster_addr":"172.21.15.33:6813/818533501","heartbeat_back_addr":"172.21.15.33:6817/818533501","heartbeat_front_addr":"172.21.15.33:6815/818533501","state":["exists","up"]},{"osd":4,"uuid":"706752a0-b17f-4563-a7a9-d5078bf0f3b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6809","nonce":3010133776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6811","nonce":3010133776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6815","nonce":3010133776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6813","nonce":3010133776}]},"public_addr":"172.21.15.52:6809/3010133776","cluster_addr":"172.21.15.52:6811/3010133776","heartbeat_back_addr":"172.21.15.52:6815/3010133776","heartbeat_front_addr":"172.21.15.52:6813/3010133776","state":["exists","up"]},{"osd":5,"uuid":"3d45ab97-f364-4903-974d-8acc961363d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6809","nonce":2132616297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6811","nonce":2132616297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6815","nonce":2132616297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6813","nonce":2132616297}]},"public_addr":"172.21.15.81:6809/2132616297","cluster_addr":"172.21.15.81:6811/2132616297","heartbeat_back_addr":"172.21.15.81:6815/2132616297","heartbeat_front_addr":"172.21.15.81:6813/2132616297","state":["exists","up"]},{"osd":6,"uuid":"3be524c5-dcf9-4742-b896-ea9a44a6b391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6819","nonce":1260691801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6821","nonce":1260691801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6825","nonce":1260691801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6823","nonce":1260691801}]},"public_addr":"172.21.15.33:6819/1260691801","cluster_addr":"172.21.15.33:6821/1260691801","heartbeat_back_addr":"172.21.15.33:6825/1260691801","heartbeat_front_addr":"172.21.15.33:6823/1260691801","state":["exists","up"]},{"osd":7,"uuid":"bda2f920-4f2c-4eed-8127-83c1973676e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6817","nonce":1654595813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6819","nonce":1654595813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6823","nonce":1654595813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6821","nonce":1654595813}]},"public_addr":"172.21.15.52:6817/1654595813","cluster_addr":"172.21.15.52:6819/1654595813","heartbeat_back_addr":"172.21.15.52:6823/1654595813","heartbeat_front_addr":"172.21.15.52:6821/1654595813","state":["exists","up"]},{"osd":8,"uuid":"550585b8-0df4-49ee-95c8-3d084744dbd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6817","nonce":3009661748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6819","nonce":3009661748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6823","nonce":3009661748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6821","nonce":3009661748}]},"public_addr":"172.21.15.81:6817/3009661748","cluster_addr":"172.21.15.81:6819/3009661748","heartbeat_back_addr":"172.21.15.81:6823/3009661748","heartbeat_front_addr":"172.21.15.81:6821/3009661748","state":["exists","up"]},{"osd":9,"uuid":"1d781846-26f3-4b23-b22c-c994a8509a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6827","nonce":2843220858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6829","nonce":2843220858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6833","nonce":2843220858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6831","nonce":2843220858}]},"public_addr":"172.21.15.33:6827/2843220858","cluster_addr":"172.21.15.33:6829/2843220858","heartbeat_back_addr":"172.21.15.33:6833/2843220858","heartbeat_front_addr":"172.21.15.33:6831/2843220858","state":["exists","up"]},{"osd":10,"uuid":"af58c940-fdc4-418b-a638-7dc44438b34c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6825","nonce":2422349831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6827","nonce":2422349831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6831","nonce":2422349831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6829","nonce":2422349831}]},"public_addr":"172.21.15.52:6825/2422349831","cluster_addr":"172.21.15.52:6827/2422349831","heartbeat_back_addr":"172.21.15.52:6831/2422349831","heartbeat_front_addr":"172.21.15.52:6829/2422349831","state":["exists","up"]},{"osd":11,"uuid":"ab84aaea-2086-4842-8bf4-628842fce46a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6825","nonce":326313090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6827","nonce":326313090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6831","nonce":326313090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6829","nonce":326313090}]},"public_addr":"172.21.15.81:6825/326313090","cluster_addr":"172.21.15.81:6827/326313090","heartbeat_back_addr":"172.21.15.81:6831/326313090","heartbeat_front_addr":"172.21.15.81:6829/326313090","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:51:41.493136+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:52:05.784773+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:52:29.769429+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:52:54.891620+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:53:19.412922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:53:43.921841+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:54:09.486732+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:54:34.064370+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:54:58.655304+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:55:24.548693+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:55:49.941191+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:56:14.919558+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/436121571":"2024-05-16T13:50:04.881656+0000","172.21.15.33:6801/2680375465":"2024-05-16T13:50:04.881656+0000","172.21.15.33:6800/2680375465":"2024-05-16T13:50:04.881656+0000","172.21.15.33:0/1022787179":"2024-05-16T13:49:37.564643+0000","172.21.15.33:0/3720711369":"2024-05-16T13:50:04.881656+0000","172.21.15.33:0/1282466838":"2024-05-16T13:49:37.564643+0000","172.21.15.33:0/3022248969":"2024-05-16T13:49:37.564643+0000","172.21.15.33:6801/3303495364":"2024-05-16T13:49:37.564643+0000","172.21.15.33:6800/3303495364":"2024-05-16T13:49:37.564643+0000","172.21.15.33:0/4282812692":"2024-05-16T13:50:04.881656+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-15T13:58:16.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:16.139+0000 7f57b9015700 1 -- 172.21.15.33:0/2567834262 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f579c0b2910 msgr2=0x7f579c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:16.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:16.139+0000 7f57b9015700 1 --2- 172.21.15.33:0/2567834262 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f579c0b2910 0x7f579c0b4dd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f57a400ac50 tx=0x7f57a400a380 comp rx=0 tx=0).stop 2024-05-15T13:58:16.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:16.139+0000 7f57b9015700 1 -- 172.21.15.33:0/2567834262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b41041d0 msgr2=0x7f57b4075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:16.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:16.139+0000 7f57b9015700 1 --2- 172.21.15.33:0/2567834262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b41041d0 0x7f57b4075d80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f57a800d950 tx=0x7f57a800dc60 comp rx=0 tx=0).stop 2024-05-15T13:58:16.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:16.139+0000 7f57b9015700 1 -- 172.21.15.33:0/2567834262 shutdown_connections 2024-05-15T13:58:16.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:16.139+0000 7f57b9015700 1 --2- 172.21.15.33:0/2567834262 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f579c0b2910 0x7f579c0b4dd0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:16.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:16.139+0000 7f57b9015700 1 --2- 172.21.15.33:0/2567834262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57b4108910 0x7f57b40762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:16.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:16.139+0000 7f57b9015700 1 --2- 172.21.15.33:0/2567834262 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57b4103820 0x7f57b4075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:16.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:16.139+0000 7f57b9015700 1 --2- 172.21.15.33:0/2567834262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b41041d0 0x7f57b4075d80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:16.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:16.139+0000 7f57b9015700 1 -- 172.21.15.33:0/2567834262 >> 172.21.15.33:0/2567834262 conn(0x7f57b407a6b0 msgr2=0x7f57b41015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:16.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:16.139+0000 7f57b9015700 1 -- 172.21.15.33:0/2567834262 shutdown_connections 2024-05-15T13:58:16.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:16.139+0000 7f57b9015700 1 -- 172.21.15.33:0/2567834262 wait complete. 2024-05-15T13:58:16.876 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-15T13:58:16.877 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-15T13:58:16.877 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-15T13:58:16.877 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-15T13:58:16.878 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-15T13:58:16.878 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-15T13:58:16.878 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-15T13:58:16.879 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-15T13:58:16.879 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-15T13:58:16.880 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-15T13:58:16.880 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-15T13:58:16.880 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-15T13:58:17.190 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:16 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/2567834262' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T13:58:17.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:16 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:58:17.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:16 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:58:17.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:16 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/2567834262' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T13:58:17.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:16 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:58:17.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:16 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:58:17.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:16 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/2567834262' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T13:58:17.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:16 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:58:17.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:16 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:58:17.596 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:58:17.611 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:58:17.899 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:58:17.899 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:58:17.903 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:58:17.903 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:58:18.020 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:17 smithi033 ceph-mon[31650]: pgmap v318: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 12 B/s, 0 keys/s, 1 objects/s recovering 2024-05-15T13:58:18.020 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:17 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:58:18.020 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:17 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:58:18.020 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:17 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:58:18.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:17 smithi052 ceph-mon[36190]: pgmap v318: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 12 B/s, 0 keys/s, 1 objects/s recovering 2024-05-15T13:58:18.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:17 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:58:18.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:17 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:58:18.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:17 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:58:18.356 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:17 smithi081 ceph-mon[36434]: pgmap v318: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 12 B/s, 0 keys/s, 1 objects/s recovering 2024-05-15T13:58:18.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:17 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:58:18.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:17 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:58:18.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:17 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:58:18.965 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:58:18.966 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:58:18.967 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:58:18.968 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:58:18.979 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:58:20.078 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:20 smithi033 ceph-mon[31650]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-05-15T13:58:20.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:20 smithi052 ceph-mon[36190]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-05-15T13:58:20.356 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:20 smithi081 ceph-mon[36434]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-05-15T13:58:21.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.679+0000 7f484f015700 1 -- 172.21.15.33:0/1659327818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4848101f70 msgr2=0x7f48481023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:21.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.679+0000 7f484f015700 1 --2- 172.21.15.33:0/1659327818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4848101f70 0x7f48481023f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4844009990 tx=0x7f4844009ca0 comp rx=0 tx=0).stop 2024-05-15T13:58:21.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.679+0000 7f484f015700 1 -- 172.21.15.33:0/1659327818 shutdown_connections 2024-05-15T13:58:21.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.679+0000 7f484f015700 1 --2- 172.21.15.33:0/1659327818 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4848102930 0x7f4848110430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:21.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.679+0000 7f484f015700 1 --2- 172.21.15.33:0/1659327818 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4848109720 0x7f4848109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:21.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.679+0000 7f484f015700 1 --2- 172.21.15.33:0/1659327818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4848101f70 0x7f48481023f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:21.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.679+0000 7f484f015700 1 -- 172.21.15.33:0/1659327818 >> 172.21.15.33:0/1659327818 conn(0x7f48480776b0 msgr2=0x7f4848077ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:21.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.679+0000 7f484f015700 1 -- 172.21.15.33:0/1659327818 shutdown_connections 2024-05-15T13:58:21.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.679+0000 7f484f015700 1 -- 172.21.15.33:0/1659327818 wait complete. 2024-05-15T13:58:21.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.679+0000 7f484f015700 1 Processor -- start 2024-05-15T13:58:21.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.680+0000 7f484f015700 1 -- start start 2024-05-15T13:58:21.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.680+0000 7f484f015700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4848101f70 0x7f484819f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:21.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.680+0000 7f484f015700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4848102930 0x7f484819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:21.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.680+0000 7f484f015700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4848109720 0x7f48481999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:21.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.680+0000 7f484f015700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4848113750 con 0x7f4848101f70 2024-05-15T13:58:21.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.680+0000 7f484f015700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f48481135d0 con 0x7f4848102930 2024-05-15T13:58:21.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.680+0000 7f484f015700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f48481138d0 con 0x7f4848109720 2024-05-15T13:58:21.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.680+0000 7f484cdb1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4848101f70 0x7f484819f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:21.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.680+0000 7f484cdb1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4848101f70 0x7f484819f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56554/0 (socket says 172.21.15.33:56554) 2024-05-15T13:58:21.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.680+0000 7f484cdb1700 1 -- 172.21.15.33:0/3361916069 learned_addr learned my addr 172.21.15.33:0/3361916069 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:58:21.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.680+0000 7f483ffff700 1 --2- 172.21.15.33:0/3361916069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4848102930 0x7f484819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:21.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.680+0000 7f484d5b2700 1 --2- 172.21.15.33:0/3361916069 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4848109720 0x7f48481999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:21.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.680+0000 7f484cdb1700 1 -- 172.21.15.33:0/3361916069 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4848109720 msgr2=0x7f48481999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:21.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.680+0000 7f484cdb1700 1 --2- 172.21.15.33:0/3361916069 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4848109720 0x7f48481999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:21.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.680+0000 7f484cdb1700 1 -- 172.21.15.33:0/3361916069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4848102930 msgr2=0x7f484819fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:21.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.680+0000 7f484cdb1700 1 --2- 172.21.15.33:0/3361916069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4848102930 0x7f484819fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:21.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.680+0000 7f484cdb1700 1 -- 172.21.15.33:0/3361916069 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f484819a150 con 0x7f4848101f70 2024-05-15T13:58:21.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.680+0000 7f483ffff700 1 --2- 172.21.15.33:0/3361916069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4848102930 0x7f484819fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T13:58:21.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.681+0000 7f484cdb1700 1 --2- 172.21.15.33:0/3361916069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4848101f70 0x7f484819f7e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f483800c920 tx=0x7f483800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:21.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.681+0000 7f483dffb700 1 -- 172.21.15.33:0/3361916069 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4838004a30 con 0x7f4848101f70 2024-05-15T13:58:21.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.681+0000 7f484f015700 1 -- 172.21.15.33:0/3361916069 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f484819a440 con 0x7f4848101f70 2024-05-15T13:58:21.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.681+0000 7f483dffb700 1 -- 172.21.15.33:0/3361916069 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48380054d0 con 0x7f4848101f70 2024-05-15T13:58:21.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.681+0000 7f483dffb700 1 -- 172.21.15.33:0/3361916069 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4838007bd0 con 0x7f4848101f70 2024-05-15T13:58:21.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.681+0000 7f484f015700 1 -- 172.21.15.33:0/3361916069 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48481a66b0 con 0x7f4848101f70 2024-05-15T13:58:21.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.682+0000 7f484f015700 1 -- 172.21.15.33:0/3361916069 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4848063cd0 con 0x7f4848101f70 2024-05-15T13:58:21.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.683+0000 7f483dffb700 1 -- 172.21.15.33:0/3361916069 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f4838004bd0 con 0x7f4848101f70 2024-05-15T13:58:21.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.684+0000 7f483dffb700 1 --2- 172.21.15.33:0/3361916069 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f48300b2910 0x7f48300b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:21.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.684+0000 7f483ffff700 1 --2- 172.21.15.33:0/3361916069 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f48300b2910 0x7f48300b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:21.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.684+0000 7f483dffb700 1 -- 172.21.15.33:0/3361916069 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f48380e2300 con 0x7f4848101f70 2024-05-15T13:58:21.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.685+0000 7f483dffb700 1 --2- 172.21.15.33:0/3361916069 >> [v2:172.21.15.81:6808/2132616297,v1:172.21.15.81:6809/2132616297] conn(0x7f48300bc4d0 0x7f48300be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:21.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.685+0000 7f483dffb700 1 -- 172.21.15.33:0/3361916069 --> [v2:172.21.15.81:6808/2132616297,v1:172.21.15.81:6809/2132616297] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f48300bf040 con 0x7f48300bc4d0 2024-05-15T13:58:21.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.685+0000 7f483dffb700 1 -- 172.21.15.33:0/3361916069 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f48380e26b0 con 0x7f4848101f70 2024-05-15T13:58:21.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.685+0000 7f483ffff700 1 --2- 172.21.15.33:0/3361916069 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f48300b2910 0x7f48300b4dd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f484400b5c0 tx=0x7f4844005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:21.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.685+0000 7f484d5b2700 1 --2- 172.21.15.33:0/3361916069 >> [v2:172.21.15.81:6808/2132616297,v1:172.21.15.81:6809/2132616297] conn(0x7f48300bc4d0 0x7f48300be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:21.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.685+0000 7f484d5b2700 1 --2- 172.21.15.33:0/3361916069 >> [v2:172.21.15.81:6808/2132616297,v1:172.21.15.81:6809/2132616297] conn(0x7f48300bc4d0 0x7f48300be970 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:21.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.686+0000 7f483dffb700 1 -- 172.21.15.33:0/3361916069 <== osd.5 v2:172.21.15.81:6808/2132616297 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f48300bf040 con 0x7f48300bc4d0 2024-05-15T13:58:21.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.703+0000 7f484f015700 1 -- 172.21.15.33:0/3361916069 --> [v2:172.21.15.81:6808/2132616297,v1:172.21.15.81:6809/2132616297] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4848064150 con 0x7f48300bc4d0 2024-05-15T13:58:21.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.703+0000 7f483dffb700 1 -- 172.21.15.33:0/3361916069 <== osd.5 v2:172.21.15.81:6808/2132616297 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4848064150 con 0x7f48300bc4d0 2024-05-15T13:58:21.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.704+0000 7f484f015700 1 -- 172.21.15.33:0/3361916069 >> [v2:172.21.15.81:6808/2132616297,v1:172.21.15.81:6809/2132616297] conn(0x7f48300bc4d0 msgr2=0x7f48300be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:21.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.704+0000 7f484f015700 1 --2- 172.21.15.33:0/3361916069 >> [v2:172.21.15.81:6808/2132616297,v1:172.21.15.81:6809/2132616297] conn(0x7f48300bc4d0 0x7f48300be970 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:21.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.704+0000 7f484f015700 1 -- 172.21.15.33:0/3361916069 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f48300b2910 msgr2=0x7f48300b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:21.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.704+0000 7f484f015700 1 --2- 172.21.15.33:0/3361916069 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f48300b2910 0x7f48300b4dd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f484400b5c0 tx=0x7f4844005960 comp rx=0 tx=0).stop 2024-05-15T13:58:21.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.704+0000 7f484f015700 1 -- 172.21.15.33:0/3361916069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4848101f70 msgr2=0x7f484819f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:21.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.704+0000 7f484f015700 1 --2- 172.21.15.33:0/3361916069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4848101f70 0x7f484819f7e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f483800c920 tx=0x7f483800cc30 comp rx=0 tx=0).stop 2024-05-15T13:58:21.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.704+0000 7f484d5b2700 1 -- 172.21.15.33:0/3361916069 reap_dead start 2024-05-15T13:58:21.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.704+0000 7f484f015700 1 -- 172.21.15.33:0/3361916069 shutdown_connections 2024-05-15T13:58:21.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.704+0000 7f484f015700 1 -- 172.21.15.33:0/3361916069 >> 172.21.15.33:0/3361916069 conn(0x7f48480776b0 msgr2=0x7f4848100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:21.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.704+0000 7f484f015700 1 -- 172.21.15.33:0/3361916069 shutdown_connections 2024-05-15T13:58:21.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:21.704+0000 7f484f015700 1 -- 172.21.15.33:0/3361916069 wait complete. 2024-05-15T13:58:22.330 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:22 smithi033 ceph-mon[31650]: pgmap v320: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 9 B/s, 0 objects/s recovering 2024-05-15T13:58:22.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:22 smithi052 ceph-mon[36190]: pgmap v320: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 9 B/s, 0 objects/s recovering 2024-05-15T13:58:22.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:22 smithi081 ceph-mon[36434]: pgmap v320: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 9 B/s, 0 objects/s recovering 2024-05-15T13:58:23.446 INFO:teuthology.orchestra.run.smithi033.stdout:163208757306 2024-05-15T13:58:23.446 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-15T13:58:23.857 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:58:24.142 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:24 smithi033 ceph-mon[31650]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-15T13:58:24.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:24 smithi052 ceph-mon[36190]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-15T13:58:24.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:24 smithi081 ceph-mon[36434]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-15T13:58:26.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.163+0000 7f2f07712700 1 -- 172.21.15.33:0/952638838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f001095e0 msgr2=0x7f2f001099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:26.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.163+0000 7f2f07712700 1 --2- 172.21.15.33:0/952638838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f001095e0 0x7f2f001099c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000b0a0 tx=0x7f2ef000b3b0 comp rx=0 tx=0).stop 2024-05-15T13:58:26.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.163+0000 7f2f07712700 1 -- 172.21.15.33:0/952638838 shutdown_connections 2024-05-15T13:58:26.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.163+0000 7f2f07712700 1 --2- 172.21.15.33:0/952638838 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f0006c670 0x7f2f00110420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:26.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.163+0000 7f2f07712700 1 --2- 172.21.15.33:0/952638838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f001095e0 0x7f2f001099c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:26.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.163+0000 7f2f07712700 1 --2- 172.21.15.33:0/952638838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f0006bcb0 0x7f2f0006c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:26.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.163+0000 7f2f07712700 1 -- 172.21.15.33:0/952638838 >> 172.21.15.33:0/952638838 conn(0x7f2f000784b0 msgr2=0x7f2f000788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:26.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.163+0000 7f2f07712700 1 -- 172.21.15.33:0/952638838 shutdown_connections 2024-05-15T13:58:26.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.163+0000 7f2f07712700 1 -- 172.21.15.33:0/952638838 wait complete. 2024-05-15T13:58:26.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.164+0000 7f2f07712700 1 Processor -- start 2024-05-15T13:58:26.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.164+0000 7f2f07712700 1 -- start start 2024-05-15T13:58:26.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.164+0000 7f2f07712700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f0006bcb0 0x7f2f0019f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:26.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.164+0000 7f2f07712700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f0006c670 0x7f2f0019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:26.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.164+0000 7f2f07712700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f001095e0 0x7f2f00199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:26.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.164+0000 7f2f07712700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2f00113700 con 0x7f2f0006bcb0 2024-05-15T13:58:26.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.164+0000 7f2f07712700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2f00113580 con 0x7f2f0006c670 2024-05-15T13:58:26.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.164+0000 7f2f07712700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2f00113880 con 0x7f2f001095e0 2024-05-15T13:58:26.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.165+0000 7f2f054ae700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f0006bcb0 0x7f2f0019f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:26.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.165+0000 7f2f054ae700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f0006bcb0 0x7f2f0019f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42676/0 (socket says 172.21.15.33:42676) 2024-05-15T13:58:26.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.165+0000 7f2f054ae700 1 -- 172.21.15.33:0/1264567500 learned_addr learned my addr 172.21.15.33:0/1264567500 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:58:26.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.165+0000 7f2f04cad700 1 --2- 172.21.15.33:0/1264567500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f0006c670 0x7f2f0019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:26.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.165+0000 7f2f05caf700 1 --2- 172.21.15.33:0/1264567500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f001095e0 0x7f2f00199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:26.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.165+0000 7f2f054ae700 1 -- 172.21.15.33:0/1264567500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f001095e0 msgr2=0x7f2f00199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:26.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.165+0000 7f2f054ae700 1 --2- 172.21.15.33:0/1264567500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f001095e0 0x7f2f00199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:26.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.165+0000 7f2f054ae700 1 -- 172.21.15.33:0/1264567500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f0006c670 msgr2=0x7f2f0019fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:26.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.165+0000 7f2f054ae700 1 --2- 172.21.15.33:0/1264567500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f0006c670 0x7f2f0019fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:26.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.165+0000 7f2f054ae700 1 -- 172.21.15.33:0/1264567500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f0019a180 con 0x7f2f0006bcb0 2024-05-15T13:58:26.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.165+0000 7f2f04cad700 1 --2- 172.21.15.33:0/1264567500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f0006c670 0x7f2f0019fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T13:58:26.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.165+0000 7f2f054ae700 1 --2- 172.21.15.33:0/1264567500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f0006bcb0 0x7f2f0019f860 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000b070 tx=0x7f2ef0004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:26.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.165+0000 7f2efe7fc700 1 -- 172.21.15.33:0/1264567500 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0021070 con 0x7f2f0006bcb0 2024-05-15T13:58:26.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.165+0000 7f2efe7fc700 1 -- 172.21.15.33:0/1264567500 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ef0003c00 con 0x7f2f0006bcb0 2024-05-15T13:58:26.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.165+0000 7f2f07712700 1 -- 172.21.15.33:0/1264567500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f0019a410 con 0x7f2f0006bcb0 2024-05-15T13:58:26.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.165+0000 7f2efe7fc700 1 -- 172.21.15.33:0/1264567500 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0004880 con 0x7f2f0006bcb0 2024-05-15T13:58:26.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.165+0000 7f2f07712700 1 -- 172.21.15.33:0/1264567500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f001a6730 con 0x7f2f0006bcb0 2024-05-15T13:58:26.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.166+0000 7f2f07712700 1 -- 172.21.15.33:0/1264567500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2f00063cd0 con 0x7f2f0006bcb0 2024-05-15T13:58:26.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.168+0000 7f2efe7fc700 1 -- 172.21.15.33:0/1264567500 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f2ef0004000 con 0x7f2f0006bcb0 2024-05-15T13:58:26.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.169+0000 7f2efe7fc700 1 --2- 172.21.15.33:0/1264567500 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f2ee80b2910 0x7f2ee80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:26.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.169+0000 7f2f04cad700 1 --2- 172.21.15.33:0/1264567500 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f2ee80b2910 0x7f2ee80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.169+0000 7f2efe7fc700 1 -- 172.21.15.33:0/1264567500 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f2ef000f020 con 0x7f2f0006bcb0 2024-05-15T13:58:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.169+0000 7f2efe7fc700 1 --2- 172.21.15.33:0/1264567500 >> [v2:172.21.15.33:6802/887409130,v1:172.21.15.33:6803/887409130] conn(0x7f2ee80bc4d0 0x7f2ee80be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.169+0000 7f2efe7fc700 1 -- 172.21.15.33:0/1264567500 --> [v2:172.21.15.33:6802/887409130,v1:172.21.15.33:6803/887409130] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2ef001fe80 con 0x7f2ee80bc4d0 2024-05-15T13:58:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.169+0000 7f2efe7fc700 1 -- 172.21.15.33:0/1264567500 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f2ef00e3c30 con 0x7f2f0006bcb0 2024-05-15T13:58:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.169+0000 7f2f05caf700 1 --2- 172.21.15.33:0/1264567500 >> [v2:172.21.15.33:6802/887409130,v1:172.21.15.33:6803/887409130] conn(0x7f2ee80bc4d0 0x7f2ee80be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.169+0000 7f2f04cad700 1 --2- 172.21.15.33:0/1264567500 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f2ee80b2910 0x7f2ee80b4dd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4005950 tx=0x7f2ef40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.170+0000 7f2f05caf700 1 --2- 172.21.15.33:0/1264567500 >> [v2:172.21.15.33:6802/887409130,v1:172.21.15.33:6803/887409130] conn(0x7f2ee80bc4d0 0x7f2ee80be970 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.170+0000 7f2efe7fc700 1 -- 172.21.15.33:0/1264567500 <== osd.0 v2:172.21.15.33:6802/887409130 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2ef001fe80 con 0x7f2ee80bc4d0 2024-05-15T13:58:26.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.187+0000 7f2f07712700 1 -- 172.21.15.33:0/1264567500 --> [v2:172.21.15.33:6802/887409130,v1:172.21.15.33:6803/887409130] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2f00064150 con 0x7f2ee80bc4d0 2024-05-15T13:58:26.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.188+0000 7f2efe7fc700 1 -- 172.21.15.33:0/1264567500 <== osd.0 v2:172.21.15.33:6802/887409130 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2f00064150 con 0x7f2ee80bc4d0 2024-05-15T13:58:26.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.188+0000 7f2f07712700 1 -- 172.21.15.33:0/1264567500 >> [v2:172.21.15.33:6802/887409130,v1:172.21.15.33:6803/887409130] conn(0x7f2ee80bc4d0 msgr2=0x7f2ee80be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:26.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.188+0000 7f2f07712700 1 --2- 172.21.15.33:0/1264567500 >> [v2:172.21.15.33:6802/887409130,v1:172.21.15.33:6803/887409130] conn(0x7f2ee80bc4d0 0x7f2ee80be970 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:26.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.188+0000 7f2f07712700 1 -- 172.21.15.33:0/1264567500 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f2ee80b2910 msgr2=0x7f2ee80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:26.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.188+0000 7f2f07712700 1 --2- 172.21.15.33:0/1264567500 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f2ee80b2910 0x7f2ee80b4dd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4005950 tx=0x7f2ef40058e0 comp rx=0 tx=0).stop 2024-05-15T13:58:26.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.188+0000 7f2f07712700 1 -- 172.21.15.33:0/1264567500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f0006bcb0 msgr2=0x7f2f0019f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:26.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.188+0000 7f2f07712700 1 --2- 172.21.15.33:0/1264567500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f0006bcb0 0x7f2f0019f860 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000b070 tx=0x7f2ef0004db0 comp rx=0 tx=0).stop 2024-05-15T13:58:26.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.188+0000 7f2f05caf700 1 -- 172.21.15.33:0/1264567500 reap_dead start 2024-05-15T13:58:26.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.188+0000 7f2f07712700 1 -- 172.21.15.33:0/1264567500 shutdown_connections 2024-05-15T13:58:26.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.188+0000 7f2f07712700 1 -- 172.21.15.33:0/1264567500 >> 172.21.15.33:0/1264567500 conn(0x7f2f000784b0 msgr2=0x7f2f001012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:26.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.188+0000 7f2f07712700 1 -- 172.21.15.33:0/1264567500 shutdown_connections 2024-05-15T13:58:26.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:26.188+0000 7f2f07712700 1 -- 172.21.15.33:0/1264567500 wait complete. 2024-05-15T13:58:26.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:26 smithi052 ceph-mon[36190]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:58:26.341 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:26 smithi033 ceph-mon[31650]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:58:26.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:26 smithi081 ceph-mon[36434]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:58:28.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:28 smithi052 ceph-mon[36190]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-15T13:58:28.348 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:28 smithi033 ceph-mon[31650]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-15T13:58:28.356 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:28 smithi081 ceph-mon[36434]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-15T13:58:28.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.907+0000 7f9ed18ea700 1 -- 172.21.15.33:0/3682164342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ecc103150 msgr2=0x7f9ecc10f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:28.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.907+0000 7f9ed18ea700 1 --2- 172.21.15.33:0/3682164342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ecc103150 0x7f9ecc10f2a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400b0d0 tx=0x7f9ec400b3e0 comp rx=0 tx=0).stop 2024-05-15T13:58:28.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.907+0000 7f9ed18ea700 1 -- 172.21.15.33:0/3682164342 shutdown_connections 2024-05-15T13:58:28.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.907+0000 7f9ed18ea700 1 --2- 172.21.15.33:0/3682164342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ecc103150 0x7f9ecc10f2a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:28.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.907+0000 7f9ed18ea700 1 --2- 172.21.15.33:0/3682164342 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ecc108630 0x7f9ecc108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:28.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.907+0000 7f9ed18ea700 1 --2- 172.21.15.33:0/3682164342 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ecc102790 0x7f9ecc102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:28.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.907+0000 7f9ed18ea700 1 -- 172.21.15.33:0/3682164342 >> 172.21.15.33:0/3682164342 conn(0x7f9ecc0782b0 msgr2=0x7f9ecc0786c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:28.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.907+0000 7f9ed18ea700 1 -- 172.21.15.33:0/3682164342 shutdown_connections 2024-05-15T13:58:28.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.907+0000 7f9ed18ea700 1 -- 172.21.15.33:0/3682164342 wait complete. 2024-05-15T13:58:28.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.907+0000 7f9ed18ea700 1 Processor -- start 2024-05-15T13:58:28.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.908+0000 7f9ed18ea700 1 -- start start 2024-05-15T13:58:28.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.908+0000 7f9ed18ea700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ecc102790 0x7f9ecc19a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:28.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.908+0000 7f9ed18ea700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ecc103150 0x7f9ecc19ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:28.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.908+0000 7f9ed18ea700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ecc108630 0x7f9ecc1a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:28.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.908+0000 7f9ed18ea700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc112680 con 0x7f9ecc103150 2024-05-15T13:58:28.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.908+0000 7f9ed18ea700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc112500 con 0x7f9ecc108630 2024-05-15T13:58:28.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.908+0000 7f9ed18ea700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc112800 con 0x7f9ecc102790 2024-05-15T13:58:28.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.908+0000 7f9eca7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ecc103150 0x7f9ecc19ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:28.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.908+0000 7f9eca7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ecc103150 0x7f9ecc19ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42704/0 (socket says 172.21.15.33:42704) 2024-05-15T13:58:28.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.908+0000 7f9eca7fc700 1 -- 172.21.15.33:0/2028746395 learned_addr learned my addr 172.21.15.33:0/2028746395 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:58:28.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.908+0000 7f9ecaffd700 1 --2- 172.21.15.33:0/2028746395 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ecc102790 0x7f9ecc19a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:28.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.908+0000 7f9ecb7fe700 1 --2- 172.21.15.33:0/2028746395 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ecc108630 0x7f9ecc1a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:28.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.908+0000 7f9eca7fc700 1 -- 172.21.15.33:0/2028746395 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ecc102790 msgr2=0x7f9ecc19a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:28.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.908+0000 7f9eca7fc700 1 --2- 172.21.15.33:0/2028746395 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ecc102790 0x7f9ecc19a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:28.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.908+0000 7f9eca7fc700 1 -- 172.21.15.33:0/2028746395 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ecc108630 msgr2=0x7f9ecc1a59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:28.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.908+0000 7f9eca7fc700 1 --2- 172.21.15.33:0/2028746395 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ecc108630 0x7f9ecc1a59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:28.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.908+0000 7f9eca7fc700 1 -- 172.21.15.33:0/2028746395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ecc1a5f00 con 0x7f9ecc103150 2024-05-15T13:58:28.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.908+0000 7f9ecb7fe700 1 --2- 172.21.15.33:0/2028746395 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ecc108630 0x7f9ecc1a59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T13:58:28.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.908+0000 7f9eca7fc700 1 --2- 172.21.15.33:0/2028746395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ecc103150 0x7f9ecc19ae80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9ec000c9b0 tx=0x7f9ec000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:28.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.909+0000 7f9ed08e8700 1 -- 172.21.15.33:0/2028746395 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec0007b00 con 0x7f9ecc103150 2024-05-15T13:58:28.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.909+0000 7f9ed18ea700 1 -- 172.21.15.33:0/2028746395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ecc1a6130 con 0x7f9ecc103150 2024-05-15T13:58:28.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.909+0000 7f9ed08e8700 1 -- 172.21.15.33:0/2028746395 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ec0007ca0 con 0x7f9ecc103150 2024-05-15T13:58:28.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.909+0000 7f9ed08e8700 1 -- 172.21.15.33:0/2028746395 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec0018730 con 0x7f9ecc103150 2024-05-15T13:58:28.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.909+0000 7f9ed18ea700 1 -- 172.21.15.33:0/2028746395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ecc1a6690 con 0x7f9ecc103150 2024-05-15T13:58:28.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.909+0000 7f9ed18ea700 1 -- 172.21.15.33:0/2028746395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9ecc063c50 con 0x7f9ecc103150 2024-05-15T13:58:28.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.911+0000 7f9ed08e8700 1 -- 172.21.15.33:0/2028746395 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9ec000f450 con 0x7f9ecc103150 2024-05-15T13:58:28.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.912+0000 7f9ed08e8700 1 --2- 172.21.15.33:0/2028746395 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f9eb40b2910 0x7f9eb40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:28.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.912+0000 7f9ecaffd700 1 --2- 172.21.15.33:0/2028746395 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f9eb40b2910 0x7f9eb40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.912+0000 7f9ed08e8700 1 -- 172.21.15.33:0/2028746395 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f9ec00e28f0 con 0x7f9ecc103150 2024-05-15T13:58:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.912+0000 7f9ed08e8700 1 --2- 172.21.15.33:0/2028746395 >> [v2:172.21.15.33:6810/818533501,v1:172.21.15.33:6811/818533501] conn(0x7f9eb40bc4d0 0x7f9eb40be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.912+0000 7f9ed08e8700 1 -- 172.21.15.33:0/2028746395 --> [v2:172.21.15.33:6810/818533501,v1:172.21.15.33:6811/818533501] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9ec001fe50 con 0x7f9eb40bc4d0 2024-05-15T13:58:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.912+0000 7f9ed08e8700 1 -- 172.21.15.33:0/2028746395 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f9ec00e2d20 con 0x7f9ecc103150 2024-05-15T13:58:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.912+0000 7f9ecb7fe700 1 --2- 172.21.15.33:0/2028746395 >> [v2:172.21.15.33:6810/818533501,v1:172.21.15.33:6811/818533501] conn(0x7f9eb40bc4d0 0x7f9eb40be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.912+0000 7f9ecaffd700 1 --2- 172.21.15.33:0/2028746395 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f9eb40b2910 0x7f9eb40b4dd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc005fd0 tx=0x7f9ebc005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.913+0000 7f9ecb7fe700 1 --2- 172.21.15.33:0/2028746395 >> [v2:172.21.15.33:6810/818533501,v1:172.21.15.33:6811/818533501] conn(0x7f9eb40bc4d0 0x7f9eb40be970 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:28.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.913+0000 7f9ed08e8700 1 -- 172.21.15.33:0/2028746395 <== osd.3 v2:172.21.15.33:6810/818533501 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9ec001fe50 con 0x7f9eb40bc4d0 2024-05-15T13:58:28.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.930+0000 7f9ed18ea700 1 -- 172.21.15.33:0/2028746395 --> [v2:172.21.15.33:6810/818533501,v1:172.21.15.33:6811/818533501] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9ecc0640d0 con 0x7f9eb40bc4d0 2024-05-15T13:58:28.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.930+0000 7f9ed08e8700 1 -- 172.21.15.33:0/2028746395 <== osd.3 v2:172.21.15.33:6810/818533501 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9ecc0640d0 con 0x7f9eb40bc4d0 2024-05-15T13:58:28.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.931+0000 7f9ed18ea700 1 -- 172.21.15.33:0/2028746395 >> [v2:172.21.15.33:6810/818533501,v1:172.21.15.33:6811/818533501] conn(0x7f9eb40bc4d0 msgr2=0x7f9eb40be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:28.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.931+0000 7f9ed18ea700 1 --2- 172.21.15.33:0/2028746395 >> [v2:172.21.15.33:6810/818533501,v1:172.21.15.33:6811/818533501] conn(0x7f9eb40bc4d0 0x7f9eb40be970 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:28.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.931+0000 7f9ed18ea700 1 -- 172.21.15.33:0/2028746395 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f9eb40b2910 msgr2=0x7f9eb40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:28.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.931+0000 7f9ed18ea700 1 --2- 172.21.15.33:0/2028746395 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f9eb40b2910 0x7f9eb40b4dd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc005fd0 tx=0x7f9ebc005db0 comp rx=0 tx=0).stop 2024-05-15T13:58:28.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.931+0000 7f9ed18ea700 1 -- 172.21.15.33:0/2028746395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ecc103150 msgr2=0x7f9ecc19ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:28.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.931+0000 7f9ed18ea700 1 --2- 172.21.15.33:0/2028746395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ecc103150 0x7f9ecc19ae80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9ec000c9b0 tx=0x7f9ec000cd70 comp rx=0 tx=0).stop 2024-05-15T13:58:28.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.931+0000 7f9ecb7fe700 1 -- 172.21.15.33:0/2028746395 reap_dead start 2024-05-15T13:58:28.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.931+0000 7f9ed18ea700 1 -- 172.21.15.33:0/2028746395 shutdown_connections 2024-05-15T13:58:28.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.931+0000 7f9ed18ea700 1 -- 172.21.15.33:0/2028746395 >> 172.21.15.33:0/2028746395 conn(0x7f9ecc0782b0 msgr2=0x7f9ecc101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:28.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.931+0000 7f9ed18ea700 1 -- 172.21.15.33:0/2028746395 shutdown_connections 2024-05-15T13:58:28.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:28.931+0000 7f9ed18ea700 1 -- 172.21.15.33:0/2028746395 wait complete. 2024-05-15T13:58:29.743 INFO:teuthology.orchestra.run.smithi033.stdout:34359738452 2024-05-15T13:58:29.743 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-15T13:58:30.328 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:30 smithi033 ceph-mon[31650]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:58:30.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.334+0000 7f49439a3700 1 -- 172.21.15.33:0/688781395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f493c108730 msgr2=0x7f493c108b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:30.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.334+0000 7f49439a3700 1 --2- 172.21.15.33:0/688781395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f493c108730 0x7f493c108b10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f49380099c0 tx=0x7f4938009cd0 comp rx=0 tx=0).stop 2024-05-15T13:58:30.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.334+0000 7f49439a3700 1 -- 172.21.15.33:0/688781395 shutdown_connections 2024-05-15T13:58:30.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.334+0000 7f49439a3700 1 --2- 172.21.15.33:0/688781395 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f493c1039d0 0x7f493c10f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:30.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.334+0000 7f49439a3700 1 --2- 172.21.15.33:0/688781395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f493c108730 0x7f493c108b10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:30.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.334+0000 7f49439a3700 1 --2- 172.21.15.33:0/688781395 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f493c103010 0x7f493c103490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:30.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.334+0000 7f49439a3700 1 -- 172.21.15.33:0/688781395 >> 172.21.15.33:0/688781395 conn(0x7f493c077780 msgr2=0x7f493c077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:30.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.334+0000 7f49439a3700 1 -- 172.21.15.33:0/688781395 shutdown_connections 2024-05-15T13:58:30.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.334+0000 7f49439a3700 1 -- 172.21.15.33:0/688781395 wait complete. 2024-05-15T13:58:30.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.335+0000 7f49439a3700 1 Processor -- start 2024-05-15T13:58:30.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.335+0000 7f49439a3700 1 -- start start 2024-05-15T13:58:30.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.335+0000 7f49439a3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f493c103010 0x7f493c10fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:30.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.335+0000 7f49439a3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f493c1039d0 0x7f493c1103b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:30.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.335+0000 7f49439a3700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f493c108730 0x7f493c1108f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.335+0000 7f49439a3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f493c1138c0 con 0x7f493c103010 2024-05-15T13:58:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.335+0000 7f49439a3700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f493c113740 con 0x7f493c1039d0 2024-05-15T13:58:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.335+0000 7f49439a3700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f493c113a40 con 0x7f493c108730 2024-05-15T13:58:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.335+0000 7f494173f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f493c103010 0x7f493c10fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.336+0000 7f494173f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f493c103010 0x7f493c10fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42726/0 (socket says 172.21.15.33:42726) 2024-05-15T13:58:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.336+0000 7f494173f700 1 -- 172.21.15.33:0/2438068534 learned_addr learned my addr 172.21.15.33:0/2438068534 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:58:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.336+0000 7f4940f3e700 1 --2- 172.21.15.33:0/2438068534 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f493c1039d0 0x7f493c1103b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.336+0000 7f4941f40700 1 --2- 172.21.15.33:0/2438068534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f493c108730 0x7f493c1108f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.336+0000 7f494173f700 1 -- 172.21.15.33:0/2438068534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f493c108730 msgr2=0x7f493c1108f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.336+0000 7f494173f700 1 --2- 172.21.15.33:0/2438068534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f493c108730 0x7f493c1108f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.336+0000 7f494173f700 1 -- 172.21.15.33:0/2438068534 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f493c1039d0 msgr2=0x7f493c1103b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.336+0000 7f494173f700 1 --2- 172.21.15.33:0/2438068534 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f493c1039d0 0x7f493c1103b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.336+0000 7f494173f700 1 -- 172.21.15.33:0/2438068534 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f493c1a82b0 con 0x7f493c103010 2024-05-15T13:58:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.336+0000 7f494173f700 1 --2- 172.21.15.33:0/2438068534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f493c103010 0x7f493c10fe70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4938005910 tx=0x7f49380045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.336+0000 7f49327fc700 1 -- 172.21.15.33:0/2438068534 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4938026070 con 0x7f493c103010 2024-05-15T13:58:30.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.336+0000 7f49439a3700 1 -- 172.21.15.33:0/2438068534 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f493c1a8540 con 0x7f493c103010 2024-05-15T13:58:30.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.336+0000 7f49327fc700 1 -- 172.21.15.33:0/2438068534 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f493800f460 con 0x7f493c103010 2024-05-15T13:58:30.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.336+0000 7f49327fc700 1 -- 172.21.15.33:0/2438068534 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4938009e60 con 0x7f493c103010 2024-05-15T13:58:30.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.336+0000 7f49439a3700 1 -- 172.21.15.33:0/2438068534 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f493c1a89f0 con 0x7f493c103010 2024-05-15T13:58:30.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.337+0000 7f49439a3700 1 -- 172.21.15.33:0/2438068534 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f493c063c50 con 0x7f493c103010 2024-05-15T13:58:30.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:30 smithi052 ceph-mon[36190]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:58:30.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.339+0000 7f49327fc700 1 -- 172.21.15.33:0/2438068534 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f4938029070 con 0x7f493c103010 2024-05-15T13:58:30.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.339+0000 7f49327fc700 1 --2- 172.21.15.33:0/2438068534 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f49280b2910 0x7f49280b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:30.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.340+0000 7f4940f3e700 1 --2- 172.21.15.33:0/2438068534 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f49280b2910 0x7f49280b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:30.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.340+0000 7f49327fc700 1 -- 172.21.15.33:0/2438068534 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f49380e2f20 con 0x7f493c103010 2024-05-15T13:58:30.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.340+0000 7f49327fc700 1 --2- 172.21.15.33:0/2438068534 >> [v2:172.21.15.33:6826/2843220858,v1:172.21.15.33:6827/2843220858] conn(0x7f49280bc4d0 0x7f49280be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:30.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.340+0000 7f49327fc700 1 -- 172.21.15.33:0/2438068534 --> [v2:172.21.15.33:6826/2843220858,v1:172.21.15.33:6827/2843220858] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f493800b0b0 con 0x7f49280bc4d0 2024-05-15T13:58:30.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.340+0000 7f49327fc700 1 -- 172.21.15.33:0/2438068534 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f49380e3350 con 0x7f493c103010 2024-05-15T13:58:30.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.340+0000 7f4941f40700 1 --2- 172.21.15.33:0/2438068534 >> [v2:172.21.15.33:6826/2843220858,v1:172.21.15.33:6827/2843220858] conn(0x7f49280bc4d0 0x7f49280be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:30.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.340+0000 7f4940f3e700 1 --2- 172.21.15.33:0/2438068534 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f49280b2910 0x7f49280b4dd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f493c077d80 tx=0x7f492c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:30.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.340+0000 7f4941f40700 1 --2- 172.21.15.33:0/2438068534 >> [v2:172.21.15.33:6826/2843220858,v1:172.21.15.33:6827/2843220858] conn(0x7f49280bc4d0 0x7f49280be970 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:30.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.341+0000 7f49327fc700 1 -- 172.21.15.33:0/2438068534 <== osd.9 v2:172.21.15.33:6826/2843220858 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f493800b0b0 con 0x7f49280bc4d0 2024-05-15T13:58:30.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:30 smithi081 ceph-mon[36434]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:58:30.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.358+0000 7f49439a3700 1 -- 172.21.15.33:0/2438068534 --> [v2:172.21.15.33:6826/2843220858,v1:172.21.15.33:6827/2843220858] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f493c064050 con 0x7f49280bc4d0 2024-05-15T13:58:30.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.359+0000 7f49327fc700 1 -- 172.21.15.33:0/2438068534 <== osd.9 v2:172.21.15.33:6826/2843220858 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f493c064050 con 0x7f49280bc4d0 2024-05-15T13:58:30.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.359+0000 7f49439a3700 1 -- 172.21.15.33:0/2438068534 >> [v2:172.21.15.33:6826/2843220858,v1:172.21.15.33:6827/2843220858] conn(0x7f49280bc4d0 msgr2=0x7f49280be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:30.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.359+0000 7f49439a3700 1 --2- 172.21.15.33:0/2438068534 >> [v2:172.21.15.33:6826/2843220858,v1:172.21.15.33:6827/2843220858] conn(0x7f49280bc4d0 0x7f49280be970 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:30.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.359+0000 7f49439a3700 1 -- 172.21.15.33:0/2438068534 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f49280b2910 msgr2=0x7f49280b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:30.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.359+0000 7f49439a3700 1 --2- 172.21.15.33:0/2438068534 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f49280b2910 0x7f49280b4dd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f493c077d80 tx=0x7f492c009260 comp rx=0 tx=0).stop 2024-05-15T13:58:30.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.359+0000 7f49439a3700 1 -- 172.21.15.33:0/2438068534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f493c103010 msgr2=0x7f493c10fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:30.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.359+0000 7f49439a3700 1 --2- 172.21.15.33:0/2438068534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f493c103010 0x7f493c10fe70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4938005910 tx=0x7f49380045e0 comp rx=0 tx=0).stop 2024-05-15T13:58:30.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.359+0000 7f4941f40700 1 -- 172.21.15.33:0/2438068534 reap_dead start 2024-05-15T13:58:30.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.359+0000 7f49439a3700 1 -- 172.21.15.33:0/2438068534 shutdown_connections 2024-05-15T13:58:30.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.359+0000 7f49439a3700 1 -- 172.21.15.33:0/2438068534 >> 172.21.15.33:0/2438068534 conn(0x7f493c077780 msgr2=0x7f493c0ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:30.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.359+0000 7f49439a3700 1 -- 172.21.15.33:0/2438068534 shutdown_connections 2024-05-15T13:58:30.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:30.359+0000 7f49439a3700 1 -- 172.21.15.33:0/2438068534 wait complete. 2024-05-15T13:58:31.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.172+0000 7f741c4fe700 1 -- 172.21.15.33:0/2619983987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f741406bb70 msgr2=0x7f741410fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:31.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.172+0000 7f741c4fe700 1 --2- 172.21.15.33:0/2619983987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f741406bb70 0x7f741410fab0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7404009990 tx=0x7f7404009ca0 comp rx=0 tx=0).stop 2024-05-15T13:58:31.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.172+0000 7f741c4fe700 1 -- 172.21.15.33:0/2619983987 shutdown_connections 2024-05-15T13:58:31.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.172+0000 7f741c4fe700 1 --2- 172.21.15.33:0/2619983987 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f741410fff0 0x7f74141123e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:31.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.172+0000 7f741c4fe700 1 --2- 172.21.15.33:0/2619983987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f741406b250 0x7f741406b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:31.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.172+0000 7f741c4fe700 1 --2- 172.21.15.33:0/2619983987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f741406bb70 0x7f741410fab0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:31.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.172+0000 7f741c4fe700 1 -- 172.21.15.33:0/2619983987 >> 172.21.15.33:0/2619983987 conn(0x7f74140793f0 msgr2=0x7f7414079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:31.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.172+0000 7f741c4fe700 1 -- 172.21.15.33:0/2619983987 shutdown_connections 2024-05-15T13:58:31.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.172+0000 7f741c4fe700 1 -- 172.21.15.33:0/2619983987 wait complete. 2024-05-15T13:58:31.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.173+0000 7f741c4fe700 1 Processor -- start 2024-05-15T13:58:31.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.173+0000 7f741c4fe700 1 -- start start 2024-05-15T13:58:31.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.173+0000 7f741c4fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f741406b250 0x7f741419be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:31.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.173+0000 7f741c4fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f741406bb70 0x7f741419c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:31.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.173+0000 7f741c4fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f741410fff0 0x7f74141a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:31.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.173+0000 7f741c4fe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7414115930 con 0x7f741406b250 2024-05-15T13:58:31.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.173+0000 7f741c4fe700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f74141157b0 con 0x7f741406bb70 2024-05-15T13:58:31.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.173+0000 7f741c4fe700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f7414115ab0 con 0x7f741410fff0 2024-05-15T13:58:31.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.173+0000 7f741a29a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f741406b250 0x7f741419be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:31.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.173+0000 7f741a29a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f741406b250 0x7f741419be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42734/0 (socket says 172.21.15.33:42734) 2024-05-15T13:58:31.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.173+0000 7f741a29a700 1 -- 172.21.15.33:0/2810804663 learned_addr learned my addr 172.21.15.33:0/2810804663 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:58:31.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.173+0000 7f7419a99700 1 --2- 172.21.15.33:0/2810804663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f741406bb70 0x7f741419c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:31.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.173+0000 7f741aa9b700 1 --2- 172.21.15.33:0/2810804663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f741410fff0 0x7f74141a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:31.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.174+0000 7f741a29a700 1 -- 172.21.15.33:0/2810804663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f741410fff0 msgr2=0x7f74141a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:31.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.174+0000 7f741a29a700 1 --2- 172.21.15.33:0/2810804663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f741410fff0 0x7f74141a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:31.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.174+0000 7f741a29a700 1 -- 172.21.15.33:0/2810804663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f741406bb70 msgr2=0x7f741419c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:31.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.174+0000 7f741a29a700 1 --2- 172.21.15.33:0/2810804663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f741406bb70 0x7f741419c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:31.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.174+0000 7f741a29a700 1 -- 172.21.15.33:0/2810804663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74141a0e30 con 0x7f741406b250 2024-05-15T13:58:31.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.174+0000 7f741aa9b700 1 --2- 172.21.15.33:0/2810804663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f741410fff0 0x7f74141a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T13:58:31.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.174+0000 7f741a29a700 1 --2- 172.21.15.33:0/2810804663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f741406b250 0x7f741419be20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f74100060b0 tx=0x7f741000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:31.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.174+0000 7f740b7fe700 1 -- 172.21.15.33:0/2810804663 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741001a070 con 0x7f741406b250 2024-05-15T13:58:31.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.174+0000 7f740b7fe700 1 -- 172.21.15.33:0/2810804663 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f741000ddd0 con 0x7f741406b250 2024-05-15T13:58:31.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.174+0000 7f741c4fe700 1 -- 172.21.15.33:0/2810804663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74141a1120 con 0x7f741406b250 2024-05-15T13:58:31.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.174+0000 7f740b7fe700 1 -- 172.21.15.33:0/2810804663 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7410014720 con 0x7f741406b250 2024-05-15T13:58:31.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.174+0000 7f741c4fe700 1 -- 172.21.15.33:0/2810804663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74141a89a0 con 0x7f741406b250 2024-05-15T13:58:31.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.175+0000 7f741c4fe700 1 -- 172.21.15.33:0/2810804663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7414063cd0 con 0x7f741406b250 2024-05-15T13:58:31.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.177+0000 7f740b7fe700 1 -- 172.21.15.33:0/2810804663 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f741001d070 con 0x7f741406b250 2024-05-15T13:58:31.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.177+0000 7f740b7fe700 1 --2- 172.21.15.33:0/2810804663 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f74000b2990 0x7f74000b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:31.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.177+0000 7f7419a99700 1 --2- 172.21.15.33:0/2810804663 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f74000b2990 0x7f74000b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:31.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.177+0000 7f740b7fe700 1 -- 172.21.15.33:0/2810804663 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f74100e23f0 con 0x7f741406b250 2024-05-15T13:58:31.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.178+0000 7f740b7fe700 1 --2- 172.21.15.33:0/2810804663 >> [v2:172.21.15.52:6808/3010133776,v1:172.21.15.52:6809/3010133776] conn(0x7f74000bc550 0x7f74000be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:31.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.178+0000 7f740b7fe700 1 -- 172.21.15.33:0/2810804663 --> [v2:172.21.15.52:6808/3010133776,v1:172.21.15.52:6809/3010133776] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7410019ec0 con 0x7f74000bc550 2024-05-15T13:58:31.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.178+0000 7f740b7fe700 1 -- 172.21.15.33:0/2810804663 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f74100e2820 con 0x7f741406b250 2024-05-15T13:58:31.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.178+0000 7f7419a99700 1 --2- 172.21.15.33:0/2810804663 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f74000b2990 0x7f74000b4e50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f740400b5c0 tx=0x7f7404005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:31.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.178+0000 7f741aa9b700 1 --2- 172.21.15.33:0/2810804663 >> [v2:172.21.15.52:6808/3010133776,v1:172.21.15.52:6809/3010133776] conn(0x7f74000bc550 0x7f74000be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:31.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.178+0000 7f741aa9b700 1 --2- 172.21.15.33:0/2810804663 >> [v2:172.21.15.52:6808/3010133776,v1:172.21.15.52:6809/3010133776] conn(0x7f74000bc550 0x7f74000be9f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:31.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.179+0000 7f740b7fe700 1 -- 172.21.15.33:0/2810804663 <== osd.4 v2:172.21.15.52:6808/3010133776 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7410019ec0 con 0x7f74000bc550 2024-05-15T13:58:31.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.196+0000 7f741c4fe700 1 -- 172.21.15.33:0/2810804663 --> [v2:172.21.15.52:6808/3010133776,v1:172.21.15.52:6809/3010133776] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f74140640d0 con 0x7f74000bc550 2024-05-15T13:58:31.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.197+0000 7f740b7fe700 1 -- 172.21.15.33:0/2810804663 <== osd.4 v2:172.21.15.52:6808/3010133776 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f74140640d0 con 0x7f74000bc550 2024-05-15T13:58:31.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.197+0000 7f741c4fe700 1 -- 172.21.15.33:0/2810804663 >> [v2:172.21.15.52:6808/3010133776,v1:172.21.15.52:6809/3010133776] conn(0x7f74000bc550 msgr2=0x7f74000be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:31.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.197+0000 7f741c4fe700 1 --2- 172.21.15.33:0/2810804663 >> [v2:172.21.15.52:6808/3010133776,v1:172.21.15.52:6809/3010133776] conn(0x7f74000bc550 0x7f74000be9f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:31.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.197+0000 7f741c4fe700 1 -- 172.21.15.33:0/2810804663 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f74000b2990 msgr2=0x7f74000b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:31.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.197+0000 7f741c4fe700 1 --2- 172.21.15.33:0/2810804663 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f74000b2990 0x7f74000b4e50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f740400b5c0 tx=0x7f7404005960 comp rx=0 tx=0).stop 2024-05-15T13:58:31.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.197+0000 7f741c4fe700 1 -- 172.21.15.33:0/2810804663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f741406b250 msgr2=0x7f741419be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:31.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.197+0000 7f741c4fe700 1 --2- 172.21.15.33:0/2810804663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f741406b250 0x7f741419be20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f74100060b0 tx=0x7f741000d3b0 comp rx=0 tx=0).stop 2024-05-15T13:58:31.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.197+0000 7f741aa9b700 1 -- 172.21.15.33:0/2810804663 reap_dead start 2024-05-15T13:58:31.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.197+0000 7f741c4fe700 1 -- 172.21.15.33:0/2810804663 shutdown_connections 2024-05-15T13:58:31.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.197+0000 7f741c4fe700 1 -- 172.21.15.33:0/2810804663 >> 172.21.15.33:0/2810804663 conn(0x7f74140793f0 msgr2=0x7f74141114c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:31.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.197+0000 7f741c4fe700 1 -- 172.21.15.33:0/2810804663 shutdown_connections 2024-05-15T13:58:31.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.197+0000 7f741c4fe700 1 -- 172.21.15.33:0/2810804663 wait complete. 2024-05-15T13:58:31.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.721+0000 7f6b30327700 1 -- 172.21.15.33:0/3794689842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b281088f0 msgr2=0x7f6b28110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:31.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.721+0000 7f6b30327700 1 --2- 172.21.15.33:0/3794689842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b281088f0 0x7f6b28110470 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6b2000b0d0 tx=0x7f6b2000b3e0 comp rx=0 tx=0).stop 2024-05-15T13:58:31.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.721+0000 7f6b30327700 1 -- 172.21.15.33:0/3794689842 shutdown_connections 2024-05-15T13:58:31.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.721+0000 7f6b30327700 1 --2- 172.21.15.33:0/3794689842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b281088f0 0x7f6b28110470 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:31.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.721+0000 7f6b30327700 1 --2- 172.21.15.33:0/3794689842 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6b281037f0 0x7f6b28103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:31.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.721+0000 7f6b30327700 1 --2- 172.21.15.33:0/3794689842 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b281041a0 0x7f6b281081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:31.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.721+0000 7f6b30327700 1 -- 172.21.15.33:0/3794689842 >> 172.21.15.33:0/3794689842 conn(0x7f6b2807a6a0 msgr2=0x7f6b2807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:31.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.721+0000 7f6b30327700 1 -- 172.21.15.33:0/3794689842 shutdown_connections 2024-05-15T13:58:31.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.721+0000 7f6b30327700 1 -- 172.21.15.33:0/3794689842 wait complete. 2024-05-15T13:58:31.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.722+0000 7f6b30327700 1 Processor -- start 2024-05-15T13:58:31.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.722+0000 7f6b30327700 1 -- start start 2024-05-15T13:58:31.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.722+0000 7f6b30327700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6b281037f0 0x7f6b2819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:31.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.722+0000 7f6b30327700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b281041a0 0x7f6b2819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:31.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.722+0000 7f6b30327700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b281088f0 0x7f6b28199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:31.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.722+0000 7f6b30327700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6b281136e0 con 0x7f6b281041a0 2024-05-15T13:58:31.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.722+0000 7f6b30327700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6b28113560 con 0x7f6b281088f0 2024-05-15T13:58:31.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.722+0000 7f6b30327700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f6b28113860 con 0x7f6b281037f0 2024-05-15T13:58:31.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.722+0000 7f6b2d8c2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b281041a0 0x7f6b2819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:31.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.722+0000 7f6b2d8c2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b281041a0 0x7f6b2819fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42752/0 (socket says 172.21.15.33:42752) 2024-05-15T13:58:31.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.722+0000 7f6b2d8c2700 1 -- 172.21.15.33:0/774275018 learned_addr learned my addr 172.21.15.33:0/774275018 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:58:31.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.722+0000 7f6b2e0c3700 1 --2- 172.21.15.33:0/774275018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6b281037f0 0x7f6b2819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:31.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.722+0000 7f6b2e8c4700 1 --2- 172.21.15.33:0/774275018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b281088f0 0x7f6b28199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:31.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.722+0000 7f6b2d8c2700 1 -- 172.21.15.33:0/774275018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6b281037f0 msgr2=0x7f6b2819f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:31.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.722+0000 7f6b2d8c2700 1 --2- 172.21.15.33:0/774275018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6b281037f0 0x7f6b2819f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:31.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.722+0000 7f6b2d8c2700 1 -- 172.21.15.33:0/774275018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b281088f0 msgr2=0x7f6b28199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:31.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.722+0000 7f6b2d8c2700 1 --2- 172.21.15.33:0/774275018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b281088f0 0x7f6b28199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:31.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.722+0000 7f6b2d8c2700 1 -- 172.21.15.33:0/774275018 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b2819a0a0 con 0x7f6b281041a0 2024-05-15T13:58:31.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.722+0000 7f6b2e8c4700 1 --2- 172.21.15.33:0/774275018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b281088f0 0x7f6b28199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T13:58:31.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.722+0000 7f6b2d8c2700 1 --2- 172.21.15.33:0/774275018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b281041a0 0x7f6b2819fcc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800d980 tx=0x7f6b1800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:31.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.723+0000 7f6b1f7fe700 1 -- 172.21.15.33:0/774275018 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b18016070 con 0x7f6b281041a0 2024-05-15T13:58:31.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.723+0000 7f6b30327700 1 -- 172.21.15.33:0/774275018 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b2819a390 con 0x7f6b281041a0 2024-05-15T13:58:31.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.723+0000 7f6b1f7fe700 1 -- 172.21.15.33:0/774275018 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b18004830 con 0x7f6b281041a0 2024-05-15T13:58:31.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.723+0000 7f6b1f7fe700 1 -- 172.21.15.33:0/774275018 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b180054c0 con 0x7f6b281041a0 2024-05-15T13:58:31.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.723+0000 7f6b30327700 1 -- 172.21.15.33:0/774275018 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b281a6650 con 0x7f6b281041a0 2024-05-15T13:58:31.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.723+0000 7f6b30327700 1 -- 172.21.15.33:0/774275018 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6b28063cd0 con 0x7f6b281041a0 2024-05-15T13:58:31.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.725+0000 7f6b1f7fe700 1 -- 172.21.15.33:0/774275018 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f6b18004220 con 0x7f6b281041a0 2024-05-15T13:58:31.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.726+0000 7f6b1f7fe700 1 --2- 172.21.15.33:0/774275018 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f6b140b2910 0x7f6b140b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:31.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.726+0000 7f6b2e0c3700 1 --2- 172.21.15.33:0/774275018 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f6b140b2910 0x7f6b140b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:31.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.726+0000 7f6b1f7fe700 1 -- 172.21.15.33:0/774275018 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f6b180e28a0 con 0x7f6b281041a0 2024-05-15T13:58:31.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.726+0000 7f6b1f7fe700 1 --2- 172.21.15.33:0/774275018 >> [v2:172.21.15.52:6816/1654595813,v1:172.21.15.52:6817/1654595813] conn(0x7f6b140bc4d0 0x7f6b140be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:31.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.726+0000 7f6b1f7fe700 1 -- 172.21.15.33:0/774275018 --> [v2:172.21.15.52:6816/1654595813,v1:172.21.15.52:6817/1654595813] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6b140bf040 con 0x7f6b140bc4d0 2024-05-15T13:58:31.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.726+0000 7f6b1f7fe700 1 -- 172.21.15.33:0/774275018 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f6b180e2cd0 con 0x7f6b281041a0 2024-05-15T13:58:31.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.726+0000 7f6b2e0c3700 1 --2- 172.21.15.33:0/774275018 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f6b140b2910 0x7f6b140b4dd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6b2819aa80 tx=0x7f6b2400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:31.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.727+0000 7f6b2e8c4700 1 --2- 172.21.15.33:0/774275018 >> [v2:172.21.15.52:6816/1654595813,v1:172.21.15.52:6817/1654595813] conn(0x7f6b140bc4d0 0x7f6b140be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:31.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.727+0000 7f6b2e8c4700 1 --2- 172.21.15.33:0/774275018 >> [v2:172.21.15.52:6816/1654595813,v1:172.21.15.52:6817/1654595813] conn(0x7f6b140bc4d0 0x7f6b140be970 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:31.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.728+0000 7f6b1f7fe700 1 -- 172.21.15.33:0/774275018 <== osd.7 v2:172.21.15.52:6816/1654595813 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6b140bf040 con 0x7f6b140bc4d0 2024-05-15T13:58:31.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.744+0000 7f6b30327700 1 -- 172.21.15.33:0/774275018 --> [v2:172.21.15.52:6816/1654595813,v1:172.21.15.52:6817/1654595813] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6b28064150 con 0x7f6b140bc4d0 2024-05-15T13:58:31.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.745+0000 7f6b1f7fe700 1 -- 172.21.15.33:0/774275018 <== osd.7 v2:172.21.15.52:6816/1654595813 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6b28064150 con 0x7f6b140bc4d0 2024-05-15T13:58:31.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.745+0000 7f6b30327700 1 -- 172.21.15.33:0/774275018 >> [v2:172.21.15.52:6816/1654595813,v1:172.21.15.52:6817/1654595813] conn(0x7f6b140bc4d0 msgr2=0x7f6b140be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:31.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.745+0000 7f6b30327700 1 --2- 172.21.15.33:0/774275018 >> [v2:172.21.15.52:6816/1654595813,v1:172.21.15.52:6817/1654595813] conn(0x7f6b140bc4d0 0x7f6b140be970 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:31.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.745+0000 7f6b30327700 1 -- 172.21.15.33:0/774275018 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f6b140b2910 msgr2=0x7f6b140b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:31.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.745+0000 7f6b30327700 1 --2- 172.21.15.33:0/774275018 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f6b140b2910 0x7f6b140b4dd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6b2819aa80 tx=0x7f6b2400b3c0 comp rx=0 tx=0).stop 2024-05-15T13:58:31.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.746+0000 7f6b30327700 1 -- 172.21.15.33:0/774275018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b281041a0 msgr2=0x7f6b2819fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:31.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.746+0000 7f6b30327700 1 --2- 172.21.15.33:0/774275018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b281041a0 0x7f6b2819fcc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800d980 tx=0x7f6b1800dc90 comp rx=0 tx=0).stop 2024-05-15T13:58:31.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.746+0000 7f6b2e8c4700 1 -- 172.21.15.33:0/774275018 reap_dead start 2024-05-15T13:58:31.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.746+0000 7f6b30327700 1 -- 172.21.15.33:0/774275018 shutdown_connections 2024-05-15T13:58:31.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.746+0000 7f6b30327700 1 -- 172.21.15.33:0/774275018 >> 172.21.15.33:0/774275018 conn(0x7f6b2807a6a0 msgr2=0x7f6b28101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:31.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.746+0000 7f6b30327700 1 -- 172.21.15.33:0/774275018 shutdown_connections 2024-05-15T13:58:31.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:31.746+0000 7f6b30327700 1 -- 172.21.15.33:0/774275018 wait complete. 2024-05-15T13:58:32.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.102+0000 7fba3dd72700 1 -- 172.21.15.33:0/2405708862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba38102e00 msgr2=0x7fba3810f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:32.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.102+0000 7fba3dd72700 1 --2- 172.21.15.33:0/2405708862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba38102e00 0x7fba3810f2f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fba3000b0a0 tx=0x7fba3000b3b0 comp rx=0 tx=0).stop 2024-05-15T13:58:32.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.102+0000 7fba3dd72700 1 -- 172.21.15.33:0/2405708862 shutdown_connections 2024-05-15T13:58:32.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.102+0000 7fba3dd72700 1 --2- 172.21.15.33:0/2405708862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba38102e00 0x7fba3810f2f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:32.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.102+0000 7fba3dd72700 1 --2- 172.21.15.33:0/2405708862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba38108520 0x7fba38108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:32.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.102+0000 7fba3dd72700 1 --2- 172.21.15.33:0/2405708862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba38102440 0x7fba381028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:32.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.102+0000 7fba3dd72700 1 -- 172.21.15.33:0/2405708862 >> 172.21.15.33:0/2405708862 conn(0x7fba380793f0 msgr2=0x7fba38079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:32.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.102+0000 7fba3dd72700 1 -- 172.21.15.33:0/2405708862 shutdown_connections 2024-05-15T13:58:32.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.102+0000 7fba3dd72700 1 -- 172.21.15.33:0/2405708862 wait complete. 2024-05-15T13:58:32.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.103+0000 7fba3dd72700 1 Processor -- start 2024-05-15T13:58:32.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.103+0000 7fba3dd72700 1 -- start start 2024-05-15T13:58:32.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.103+0000 7fba3dd72700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba38102440 0x7fba3819f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:32.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.103+0000 7fba3dd72700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba38102e00 0x7fba3819fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:32.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.103+0000 7fba3dd72700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba38108520 0x7fba38199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:32.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.103+0000 7fba3dd72700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fba381125f0 con 0x7fba38102440 2024-05-15T13:58:32.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.103+0000 7fba3dd72700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fba38112470 con 0x7fba38108520 2024-05-15T13:58:32.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.103+0000 7fba3dd72700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fba38112770 con 0x7fba38102e00 2024-05-15T13:58:32.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.103+0000 7fba377fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba38102440 0x7fba3819f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:32.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.103+0000 7fba377fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba38102440 0x7fba3819f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42762/0 (socket says 172.21.15.33:42762) 2024-05-15T13:58:32.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.104+0000 7fba377fe700 1 -- 172.21.15.33:0/1099813915 learned_addr learned my addr 172.21.15.33:0/1099813915 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:58:32.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.104+0000 7fba36ffd700 1 --2- 172.21.15.33:0/1099813915 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba38102e00 0x7fba3819fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:32.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.104+0000 7fba37fff700 1 --2- 172.21.15.33:0/1099813915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba38108520 0x7fba38199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:32.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.104+0000 7fba377fe700 1 -- 172.21.15.33:0/1099813915 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba38102e00 msgr2=0x7fba3819fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:32.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.104+0000 7fba377fe700 1 --2- 172.21.15.33:0/1099813915 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba38102e00 0x7fba3819fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:32.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.104+0000 7fba377fe700 1 -- 172.21.15.33:0/1099813915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba38108520 msgr2=0x7fba38199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:32.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.104+0000 7fba377fe700 1 --2- 172.21.15.33:0/1099813915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba38108520 0x7fba38199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:32.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.104+0000 7fba377fe700 1 -- 172.21.15.33:0/1099813915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba3819a240 con 0x7fba38102440 2024-05-15T13:58:32.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.104+0000 7fba36ffd700 1 --2- 172.21.15.33:0/1099813915 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba38102e00 0x7fba3819fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T13:58:32.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.104+0000 7fba377fe700 1 --2- 172.21.15.33:0/1099813915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba38102440 0x7fba3819f920 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fba2800d950 tx=0x7fba2800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:32.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.104+0000 7fba34ff9700 1 -- 172.21.15.33:0/1099813915 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba28016070 con 0x7fba38102440 2024-05-15T13:58:32.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.104+0000 7fba34ff9700 1 -- 172.21.15.33:0/1099813915 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba28010460 con 0x7fba38102440 2024-05-15T13:58:32.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.104+0000 7fba3dd72700 1 -- 172.21.15.33:0/1099813915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba3819a530 con 0x7fba38102440 2024-05-15T13:58:32.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.104+0000 7fba34ff9700 1 -- 172.21.15.33:0/1099813915 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2800f620 con 0x7fba38102440 2024-05-15T13:58:32.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.104+0000 7fba3dd72700 1 -- 172.21.15.33:0/1099813915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba381a67f0 con 0x7fba38102440 2024-05-15T13:58:32.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.105+0000 7fba3dd72700 1 -- 172.21.15.33:0/1099813915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fba38063c10 con 0x7fba38102440 2024-05-15T13:58:32.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.107+0000 7fba34ff9700 1 -- 172.21.15.33:0/1099813915 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fba28019070 con 0x7fba38102440 2024-05-15T13:58:32.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.107+0000 7fba34ff9700 1 --2- 172.21.15.33:0/1099813915 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fba200b2990 0x7fba200b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:32.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.108+0000 7fba36ffd700 1 --2- 172.21.15.33:0/1099813915 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fba200b2990 0x7fba200b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:32.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.108+0000 7fba34ff9700 1 -- 172.21.15.33:0/1099813915 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fba280e2410 con 0x7fba38102440 2024-05-15T13:58:32.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.108+0000 7fba34ff9700 1 --2- 172.21.15.33:0/1099813915 >> [v2:172.21.15.81:6800/315610978,v1:172.21.15.81:6801/315610978] conn(0x7fba200bc550 0x7fba200be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:32.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.108+0000 7fba34ff9700 1 -- 172.21.15.33:0/1099813915 --> [v2:172.21.15.81:6800/315610978,v1:172.21.15.81:6801/315610978] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fba200bf0c0 con 0x7fba200bc550 2024-05-15T13:58:32.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.108+0000 7fba34ff9700 1 -- 172.21.15.33:0/1099813915 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fba280e2840 con 0x7fba38102440 2024-05-15T13:58:32.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.108+0000 7fba36ffd700 1 --2- 172.21.15.33:0/1099813915 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fba200b2990 0x7fba200b4e50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fba38079bf0 tx=0x7fba2c005f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:32.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.108+0000 7fba37fff700 1 --2- 172.21.15.33:0/1099813915 >> [v2:172.21.15.81:6800/315610978,v1:172.21.15.81:6801/315610978] conn(0x7fba200bc550 0x7fba200be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:32.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.108+0000 7fba37fff700 1 --2- 172.21.15.33:0/1099813915 >> [v2:172.21.15.81:6800/315610978,v1:172.21.15.81:6801/315610978] conn(0x7fba200bc550 0x7fba200be9f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:32.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.109+0000 7fba34ff9700 1 -- 172.21.15.33:0/1099813915 <== osd.2 v2:172.21.15.81:6800/315610978 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fba200bf0c0 con 0x7fba200bc550 2024-05-15T13:58:32.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.126+0000 7fba3dd72700 1 -- 172.21.15.33:0/1099813915 --> [v2:172.21.15.81:6800/315610978,v1:172.21.15.81:6801/315610978] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fba38064010 con 0x7fba200bc550 2024-05-15T13:58:32.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.126+0000 7fba34ff9700 1 -- 172.21.15.33:0/1099813915 <== osd.2 v2:172.21.15.81:6800/315610978 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fba38064010 con 0x7fba200bc550 2024-05-15T13:58:32.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.127+0000 7fba3dd72700 1 -- 172.21.15.33:0/1099813915 >> [v2:172.21.15.81:6800/315610978,v1:172.21.15.81:6801/315610978] conn(0x7fba200bc550 msgr2=0x7fba200be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:32.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.127+0000 7fba3dd72700 1 --2- 172.21.15.33:0/1099813915 >> [v2:172.21.15.81:6800/315610978,v1:172.21.15.81:6801/315610978] conn(0x7fba200bc550 0x7fba200be9f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:32.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.127+0000 7fba3dd72700 1 -- 172.21.15.33:0/1099813915 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fba200b2990 msgr2=0x7fba200b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:32.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.127+0000 7fba3dd72700 1 --2- 172.21.15.33:0/1099813915 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fba200b2990 0x7fba200b4e50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fba38079bf0 tx=0x7fba2c005f30 comp rx=0 tx=0).stop 2024-05-15T13:58:32.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.127+0000 7fba3dd72700 1 -- 172.21.15.33:0/1099813915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba38102440 msgr2=0x7fba3819f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:32.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.127+0000 7fba3dd72700 1 --2- 172.21.15.33:0/1099813915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba38102440 0x7fba3819f920 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fba2800d950 tx=0x7fba2800dd10 comp rx=0 tx=0).stop 2024-05-15T13:58:32.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.127+0000 7fba37fff700 1 -- 172.21.15.33:0/1099813915 reap_dead start 2024-05-15T13:58:32.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.127+0000 7fba3dd72700 1 -- 172.21.15.33:0/1099813915 shutdown_connections 2024-05-15T13:58:32.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.127+0000 7fba3dd72700 1 -- 172.21.15.33:0/1099813915 >> 172.21.15.33:0/1099813915 conn(0x7fba380793f0 msgr2=0x7fba3806b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:32.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.127+0000 7fba3dd72700 1 -- 172.21.15.33:0/1099813915 shutdown_connections 2024-05-15T13:58:32.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:32.127+0000 7fba3dd72700 1 -- 172.21.15.33:0/1099813915 wait complete. 2024-05-15T13:58:32.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:32 smithi052 ceph-mon[36190]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T13:58:32.356 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:32 smithi081 ceph-mon[36434]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T13:58:32.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:32 smithi033 ceph-mon[31650]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T13:58:32.883 INFO:teuthology.orchestra.run.smithi033.stdout:111669149765 2024-05-15T13:58:32.884 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-15T13:58:34.299 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:34 smithi033 ceph-mon[31650]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-15T13:58:34.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.331+0000 7fa1d7131700 1 -- 172.21.15.33:0/1933126418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d006c610 msgr2=0x7fa1d010fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:34.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.331+0000 7fa1d7131700 1 --2- 172.21.15.33:0/1933126418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d006c610 0x7fa1d010fa50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc009990 tx=0x7fa1bc009ca0 comp rx=0 tx=0).stop 2024-05-15T13:58:34.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.331+0000 7fa1d7131700 1 -- 172.21.15.33:0/1933126418 shutdown_connections 2024-05-15T13:58:34.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.331+0000 7fa1d7131700 1 --2- 172.21.15.33:0/1933126418 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1d010ff90 0x7fa1d0112380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:34.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.331+0000 7fa1d7131700 1 --2- 172.21.15.33:0/1933126418 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1d006bcf0 0x7fa1d006c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:34.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.331+0000 7fa1d7131700 1 --2- 172.21.15.33:0/1933126418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d006c610 0x7fa1d010fa50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:34.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.331+0000 7fa1d7131700 1 -- 172.21.15.33:0/1933126418 >> 172.21.15.33:0/1933126418 conn(0x7fa1d00794e0 msgr2=0x7fa1d00798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:34.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.331+0000 7fa1d7131700 1 -- 172.21.15.33:0/1933126418 shutdown_connections 2024-05-15T13:58:34.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.331+0000 7fa1d7131700 1 -- 172.21.15.33:0/1933126418 wait complete. 2024-05-15T13:58:34.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.332+0000 7fa1d7131700 1 Processor -- start 2024-05-15T13:58:34.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.332+0000 7fa1d7131700 1 -- start start 2024-05-15T13:58:34.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.332+0000 7fa1d7131700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1d006bcf0 0x7fa1d019bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:34.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.332+0000 7fa1d7131700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1d006c610 0x7fa1d019c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:34.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.332+0000 7fa1d7131700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d010ff90 0x7fa1d01a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:34.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.332+0000 7fa1d7131700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d01159c0 con 0x7fa1d010ff90 2024-05-15T13:58:34.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.332+0000 7fa1d7131700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d0115840 con 0x7fa1d006c610 2024-05-15T13:58:34.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.332+0000 7fa1d7131700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d0115b40 con 0x7fa1d006bcf0 2024-05-15T13:58:34.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.332+0000 7fa1d56ce700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d010ff90 0x7fa1d01a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:34.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.332+0000 7fa1d56ce700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d010ff90 0x7fa1d01a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56744/0 (socket says 172.21.15.33:56744) 2024-05-15T13:58:34.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.332+0000 7fa1d56ce700 1 -- 172.21.15.33:0/822461646 learned_addr learned my addr 172.21.15.33:0/822461646 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:58:34.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.332+0000 7fa1d4ecd700 1 --2- 172.21.15.33:0/822461646 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1d006bcf0 0x7fa1d019bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:34.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.332+0000 7fa1cffff700 1 --2- 172.21.15.33:0/822461646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1d006c610 0x7fa1d019c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:34.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.332+0000 7fa1d56ce700 1 -- 172.21.15.33:0/822461646 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1d006bcf0 msgr2=0x7fa1d019bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:34.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.332+0000 7fa1d56ce700 1 --2- 172.21.15.33:0/822461646 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1d006bcf0 0x7fa1d019bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:34.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.332+0000 7fa1d56ce700 1 -- 172.21.15.33:0/822461646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1d006c610 msgr2=0x7fa1d019c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:34.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.332+0000 7fa1d56ce700 1 --2- 172.21.15.33:0/822461646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1d006c610 0x7fa1d019c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:34.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.332+0000 7fa1d56ce700 1 -- 172.21.15.33:0/822461646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d01a0f60 con 0x7fa1d010ff90 2024-05-15T13:58:34.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.332+0000 7fa1cffff700 1 --2- 172.21.15.33:0/822461646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1d006c610 0x7fa1d019c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T13:58:34.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.333+0000 7fa1d56ce700 1 --2- 172.21.15.33:0/822461646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d010ff90 0x7fa1d01a0820 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa1c800ada0 tx=0x7fa1c800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:34.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.333+0000 7fa1cdffb700 1 -- 172.21.15.33:0/822461646 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8017020 con 0x7fa1d010ff90 2024-05-15T13:58:34.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.333+0000 7fa1cdffb700 1 -- 172.21.15.33:0/822461646 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1c800ce30 con 0x7fa1d010ff90 2024-05-15T13:58:34.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.333+0000 7fa1cdffb700 1 -- 172.21.15.33:0/822461646 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8005310 con 0x7fa1d010ff90 2024-05-15T13:58:34.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.333+0000 7fa1d7131700 1 -- 172.21.15.33:0/822461646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d01a1250 con 0x7fa1d010ff90 2024-05-15T13:58:34.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.333+0000 7fa1d7131700 1 -- 172.21.15.33:0/822461646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d01a8ad0 con 0x7fa1d010ff90 2024-05-15T13:58:34.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.334+0000 7fa1d7131700 1 -- 172.21.15.33:0/822461646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa1d0063c50 con 0x7fa1d010ff90 2024-05-15T13:58:34.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.335+0000 7fa1cdffb700 1 -- 172.21.15.33:0/822461646 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa1c800c980 con 0x7fa1d010ff90 2024-05-15T13:58:34.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.336+0000 7fa1cdffb700 1 --2- 172.21.15.33:0/822461646 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fa1c00b2910 0x7fa1c00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:34.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.336+0000 7fa1d4ecd700 1 --2- 172.21.15.33:0/822461646 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fa1c00b2910 0x7fa1c00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:34.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.336+0000 7fa1cdffb700 1 -- 172.21.15.33:0/822461646 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fa1c80e2430 con 0x7fa1d010ff90 2024-05-15T13:58:34.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.336+0000 7fa1cdffb700 1 --2- 172.21.15.33:0/822461646 >> [v2:172.21.15.81:6816/3009661748,v1:172.21.15.81:6817/3009661748] conn(0x7fa1c00bc4d0 0x7fa1c00be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:34.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.336+0000 7fa1cdffb700 1 -- 172.21.15.33:0/822461646 --> [v2:172.21.15.81:6816/3009661748,v1:172.21.15.81:6817/3009661748] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa1c8014e50 con 0x7fa1c00bc4d0 2024-05-15T13:58:34.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.336+0000 7fa1cdffb700 1 -- 172.21.15.33:0/822461646 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fa1c80e2860 con 0x7fa1d010ff90 2024-05-15T13:58:34.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.336+0000 7fa1d4ecd700 1 --2- 172.21.15.33:0/822461646 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fa1c00b2910 0x7fa1c00b4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4009a10 tx=0x7fa1c4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:34.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.336+0000 7fa1cffff700 1 --2- 172.21.15.33:0/822461646 >> [v2:172.21.15.81:6816/3009661748,v1:172.21.15.81:6817/3009661748] conn(0x7fa1c00bc4d0 0x7fa1c00be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:34.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.337+0000 7fa1cffff700 1 --2- 172.21.15.33:0/822461646 >> [v2:172.21.15.81:6816/3009661748,v1:172.21.15.81:6817/3009661748] conn(0x7fa1c00bc4d0 0x7fa1c00be970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:34.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:34 smithi052 ceph-mon[36190]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-15T13:58:34.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.338+0000 7fa1cdffb700 1 -- 172.21.15.33:0/822461646 <== osd.8 v2:172.21.15.81:6816/3009661748 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa1c8014e50 con 0x7fa1c00bc4d0 2024-05-15T13:58:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.354+0000 7fa1d7131700 1 -- 172.21.15.33:0/822461646 --> [v2:172.21.15.81:6816/3009661748,v1:172.21.15.81:6817/3009661748] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa1d00640d0 con 0x7fa1c00bc4d0 2024-05-15T13:58:34.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:34 smithi081 ceph-mon[36434]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-15T13:58:34.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.355+0000 7fa1cdffb700 1 -- 172.21.15.33:0/822461646 <== osd.8 v2:172.21.15.81:6816/3009661748 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa1d00640d0 con 0x7fa1c00bc4d0 2024-05-15T13:58:34.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.355+0000 7fa1d7131700 1 -- 172.21.15.33:0/822461646 >> [v2:172.21.15.81:6816/3009661748,v1:172.21.15.81:6817/3009661748] conn(0x7fa1c00bc4d0 msgr2=0x7fa1c00be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:34.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.355+0000 7fa1d7131700 1 --2- 172.21.15.33:0/822461646 >> [v2:172.21.15.81:6816/3009661748,v1:172.21.15.81:6817/3009661748] conn(0x7fa1c00bc4d0 0x7fa1c00be970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:34.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.355+0000 7fa1d7131700 1 -- 172.21.15.33:0/822461646 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fa1c00b2910 msgr2=0x7fa1c00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:34.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.355+0000 7fa1d7131700 1 --2- 172.21.15.33:0/822461646 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fa1c00b2910 0x7fa1c00b4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4009a10 tx=0x7fa1c4009190 comp rx=0 tx=0).stop 2024-05-15T13:58:34.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.355+0000 7fa1d7131700 1 -- 172.21.15.33:0/822461646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d010ff90 msgr2=0x7fa1d01a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:34.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.355+0000 7fa1d7131700 1 --2- 172.21.15.33:0/822461646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1d010ff90 0x7fa1d01a0820 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa1c800ada0 tx=0x7fa1c800c330 comp rx=0 tx=0).stop 2024-05-15T13:58:34.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.355+0000 7fa1d56ce700 1 -- 172.21.15.33:0/822461646 reap_dead start 2024-05-15T13:58:34.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.355+0000 7fa1d7131700 1 -- 172.21.15.33:0/822461646 shutdown_connections 2024-05-15T13:58:34.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.355+0000 7fa1d7131700 1 -- 172.21.15.33:0/822461646 >> 172.21.15.33:0/822461646 conn(0x7fa1d00794e0 msgr2=0x7fa1d0111580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:34.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.355+0000 7fa1d7131700 1 -- 172.21.15.33:0/822461646 shutdown_connections 2024-05-15T13:58:34.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.355+0000 7fa1d7131700 1 -- 172.21.15.33:0/822461646 wait complete. 2024-05-15T13:58:34.789 INFO:teuthology.orchestra.run.smithi033.stdout:137438953537 2024-05-15T13:58:34.926 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-15T13:58:34.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.977+0000 7f07efc35700 1 -- 172.21.15.33:0/3359908766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07e81037f0 msgr2=0x7f07e8103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:34.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.977+0000 7f07efc35700 1 --2- 172.21.15.33:0/3359908766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07e81037f0 0x7f07e8103bd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f07e40099c0 tx=0x7f07e4009cd0 comp rx=0 tx=0).stop 2024-05-15T13:58:34.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.977+0000 7f07efc35700 1 -- 172.21.15.33:0/3359908766 shutdown_connections 2024-05-15T13:58:34.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.977+0000 7f07efc35700 1 --2- 172.21.15.33:0/3359908766 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f07e81088f0 0x7f07e8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:34.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.978+0000 7f07efc35700 1 --2- 172.21.15.33:0/3359908766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07e81037f0 0x7f07e8103bd0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:34.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.978+0000 7f07efc35700 1 --2- 172.21.15.33:0/3359908766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07e81041a0 0x7f07e81081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:34.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.978+0000 7f07efc35700 1 -- 172.21.15.33:0/3359908766 >> 172.21.15.33:0/3359908766 conn(0x7f07e807a6a0 msgr2=0x7f07e807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:34.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.978+0000 7f07efc35700 1 -- 172.21.15.33:0/3359908766 shutdown_connections 2024-05-15T13:58:34.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.978+0000 7f07efc35700 1 -- 172.21.15.33:0/3359908766 wait complete. 2024-05-15T13:58:34.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.978+0000 7f07efc35700 1 Processor -- start 2024-05-15T13:58:34.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.978+0000 7f07efc35700 1 -- start start 2024-05-15T13:58:34.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.979+0000 7f07efc35700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07e81037f0 0x7f07e819f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:34.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.979+0000 7f07efc35700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07e81041a0 0x7f07e819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:34.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.979+0000 7f07efc35700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f07e81088f0 0x7f07e81999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:34.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.979+0000 7f07efc35700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f07e81136e0 con 0x7f07e81037f0 2024-05-15T13:58:34.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.979+0000 7f07efc35700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f07e8113560 con 0x7f07e81041a0 2024-05-15T13:58:34.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.979+0000 7f07efc35700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f07e8113860 con 0x7f07e81088f0 2024-05-15T13:58:34.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.979+0000 7f07ed9d1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07e81037f0 0x7f07e819f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:34.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.979+0000 7f07ed1d0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07e81041a0 0x7f07e819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:34.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.979+0000 7f07ed9d1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07e81037f0 0x7f07e819f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56766/0 (socket says 172.21.15.33:56766) 2024-05-15T13:58:34.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.979+0000 7f07ed9d1700 1 -- 172.21.15.33:0/885902584 learned_addr learned my addr 172.21.15.33:0/885902584 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:58:34.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.979+0000 7f07ee1d2700 1 --2- 172.21.15.33:0/885902584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f07e81088f0 0x7f07e81999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:34.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.979+0000 7f07ed9d1700 1 -- 172.21.15.33:0/885902584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f07e81088f0 msgr2=0x7f07e81999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:34.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.979+0000 7f07ed9d1700 1 --2- 172.21.15.33:0/885902584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f07e81088f0 0x7f07e81999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:34.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.979+0000 7f07ed9d1700 1 -- 172.21.15.33:0/885902584 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07e81041a0 msgr2=0x7f07e819fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:34.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.979+0000 7f07ed9d1700 1 --2- 172.21.15.33:0/885902584 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07e81041a0 0x7f07e819fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:34.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.979+0000 7f07ed9d1700 1 -- 172.21.15.33:0/885902584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e819a100 con 0x7f07e81037f0 2024-05-15T13:58:34.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.980+0000 7f07ed9d1700 1 --2- 172.21.15.33:0/885902584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07e81037f0 0x7f07e819f7e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f07e400b580 tx=0x7f07e40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:34.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.980+0000 7f07deffd700 1 -- 172.21.15.33:0/885902584 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e4027070 con 0x7f07e81037f0 2024-05-15T13:58:34.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.980+0000 7f07efc35700 1 -- 172.21.15.33:0/885902584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07e819a390 con 0x7f07e81037f0 2024-05-15T13:58:34.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.980+0000 7f07deffd700 1 -- 172.21.15.33:0/885902584 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07e4003f20 con 0x7f07e81037f0 2024-05-15T13:58:34.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.980+0000 7f07deffd700 1 -- 172.21.15.33:0/885902584 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e4022d10 con 0x7f07e81037f0 2024-05-15T13:58:34.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.980+0000 7f07efc35700 1 -- 172.21.15.33:0/885902584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07e8075770 con 0x7f07e81037f0 2024-05-15T13:58:34.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.981+0000 7f07efc35700 1 -- 172.21.15.33:0/885902584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f07e8063c90 con 0x7f07e81037f0 2024-05-15T13:58:34.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.983+0000 7f07deffd700 1 -- 172.21.15.33:0/885902584 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f07e4022470 con 0x7f07e81037f0 2024-05-15T13:58:34.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.984+0000 7f07deffd700 1 --2- 172.21.15.33:0/885902584 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f07d40b2910 0x7f07d40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:34.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.984+0000 7f07ed1d0700 1 --2- 172.21.15.33:0/885902584 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f07d40b2910 0x7f07d40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:34.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.984+0000 7f07deffd700 1 -- 172.21.15.33:0/885902584 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f07e40e3580 con 0x7f07e81037f0 2024-05-15T13:58:34.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.984+0000 7f07deffd700 1 --2- 172.21.15.33:0/885902584 >> [v2:172.21.15.52:6824/2422349831,v1:172.21.15.52:6825/2422349831] conn(0x7f07d40bc4d0 0x7f07d40be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:34.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.984+0000 7f07deffd700 1 -- 172.21.15.33:0/885902584 --> [v2:172.21.15.52:6824/2422349831,v1:172.21.15.52:6825/2422349831] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f07e400eea0 con 0x7f07d40bc4d0 2024-05-15T13:58:34.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.984+0000 7f07deffd700 1 -- 172.21.15.33:0/885902584 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f07e40e39b0 con 0x7f07e81037f0 2024-05-15T13:58:34.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.985+0000 7f07ed1d0700 1 --2- 172.21.15.33:0/885902584 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f07d40b2910 0x7f07d40b4dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f07e8103140 tx=0x7f07d8013040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:34.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.985+0000 7f07ee1d2700 1 --2- 172.21.15.33:0/885902584 >> [v2:172.21.15.52:6824/2422349831,v1:172.21.15.52:6825/2422349831] conn(0x7f07d40bc4d0 0x7f07d40be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:34.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.986+0000 7f07ee1d2700 1 --2- 172.21.15.33:0/885902584 >> [v2:172.21.15.52:6824/2422349831,v1:172.21.15.52:6825/2422349831] conn(0x7f07d40bc4d0 0x7f07d40be970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:34.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:34.987+0000 7f07deffd700 1 -- 172.21.15.33:0/885902584 <== osd.10 v2:172.21.15.52:6824/2422349831 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f07e400eea0 con 0x7f07d40bc4d0 2024-05-15T13:58:35.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:35.003+0000 7f07efc35700 1 -- 172.21.15.33:0/885902584 --> [v2:172.21.15.52:6824/2422349831,v1:172.21.15.52:6825/2422349831] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f07e80640c0 con 0x7f07d40bc4d0 2024-05-15T13:58:35.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:35.004+0000 7f07deffd700 1 -- 172.21.15.33:0/885902584 <== osd.10 v2:172.21.15.52:6824/2422349831 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f07e80640c0 con 0x7f07d40bc4d0 2024-05-15T13:58:35.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:35.004+0000 7f07efc35700 1 -- 172.21.15.33:0/885902584 >> [v2:172.21.15.52:6824/2422349831,v1:172.21.15.52:6825/2422349831] conn(0x7f07d40bc4d0 msgr2=0x7f07d40be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:35.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:35.004+0000 7f07efc35700 1 --2- 172.21.15.33:0/885902584 >> [v2:172.21.15.52:6824/2422349831,v1:172.21.15.52:6825/2422349831] conn(0x7f07d40bc4d0 0x7f07d40be970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:35.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:35.004+0000 7f07efc35700 1 -- 172.21.15.33:0/885902584 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f07d40b2910 msgr2=0x7f07d40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:35.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:35.004+0000 7f07efc35700 1 --2- 172.21.15.33:0/885902584 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f07d40b2910 0x7f07d40b4dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f07e8103140 tx=0x7f07d8013040 comp rx=0 tx=0).stop 2024-05-15T13:58:35.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:35.004+0000 7f07efc35700 1 -- 172.21.15.33:0/885902584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07e81037f0 msgr2=0x7f07e819f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:35.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:35.004+0000 7f07efc35700 1 --2- 172.21.15.33:0/885902584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07e81037f0 0x7f07e819f7e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f07e400b580 tx=0x7f07e40047b0 comp rx=0 tx=0).stop 2024-05-15T13:58:35.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:35.004+0000 7f07ee1d2700 1 -- 172.21.15.33:0/885902584 reap_dead start 2024-05-15T13:58:35.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:35.004+0000 7f07efc35700 1 -- 172.21.15.33:0/885902584 shutdown_connections 2024-05-15T13:58:35.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:35.004+0000 7f07efc35700 1 -- 172.21.15.33:0/885902584 >> 172.21.15.33:0/885902584 conn(0x7f07e807a6a0 msgr2=0x7f07e8101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:35.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:35.005+0000 7f07efc35700 1 -- 172.21.15.33:0/885902584 shutdown_connections 2024-05-15T13:58:35.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:35.005+0000 7f07efc35700 1 -- 172.21.15.33:0/885902584 wait complete. 2024-05-15T13:58:36.163 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:36 smithi033 ceph-mon[31650]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:58:36.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:36 smithi052 ceph-mon[36190]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:58:36.356 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:36 smithi081 ceph-mon[36434]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:58:37.408 INFO:teuthology.orchestra.run.smithi033.stdout:210453397554 2024-05-15T13:58:37.408 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-15T13:58:37.985 INFO:teuthology.orchestra.run.smithi033.stdout:236223201325 2024-05-15T13:58:37.985 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-15T13:58:38.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:38 smithi033 ceph-mon[31650]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-15T13:58:38.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:38 smithi052 ceph-mon[36190]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-15T13:58:38.356 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:38 smithi081 ceph-mon[36434]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-15T13:58:38.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.451+0000 7f670f554700 1 -- 172.21.15.33:0/2909510363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6708103600 msgr2=0x7f670810f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:38.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.451+0000 7f670f554700 1 --2- 172.21.15.33:0/2909510363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6708103600 0x7f670810f2e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f670000b0a0 tx=0x7f670000b3b0 comp rx=0 tx=0).stop 2024-05-15T13:58:38.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.452+0000 7f670f554700 1 -- 172.21.15.33:0/2909510363 shutdown_connections 2024-05-15T13:58:38.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.452+0000 7f670f554700 1 --2- 172.21.15.33:0/2909510363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6708103600 0x7f670810f2e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:38.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.452+0000 7f670f554700 1 --2- 172.21.15.33:0/2909510363 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f67081084c0 0x7f67081088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:38.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.452+0000 7f670f554700 1 --2- 172.21.15.33:0/2909510363 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6708102c40 0x7f67081030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:38.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.452+0000 7f670f554700 1 -- 172.21.15.33:0/2909510363 >> 172.21.15.33:0/2909510363 conn(0x7f6708078ab0 msgr2=0x7f67080ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:38.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.452+0000 7f670f554700 1 -- 172.21.15.33:0/2909510363 shutdown_connections 2024-05-15T13:58:38.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.452+0000 7f670f554700 1 -- 172.21.15.33:0/2909510363 wait complete. 2024-05-15T13:58:38.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.452+0000 7f670f554700 1 Processor -- start 2024-05-15T13:58:38.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.453+0000 7f670f554700 1 -- start start 2024-05-15T13:58:38.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.453+0000 7f670f554700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6708102c40 0x7f670819f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:38.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.453+0000 7f670f554700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6708103600 0x7f670819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:38.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.453+0000 7f670f554700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67081084c0 0x7f6708199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:38.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.453+0000 7f670f554700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f67081125f0 con 0x7f67081084c0 2024-05-15T13:58:38.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.453+0000 7f670f554700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6708112470 con 0x7f6708102c40 2024-05-15T13:58:38.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.453+0000 7f670f554700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f6708112770 con 0x7f6708103600 2024-05-15T13:58:38.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.453+0000 7f670daf1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67081084c0 0x7f6708199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:38.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.453+0000 7f670daf1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67081084c0 0x7f6708199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56782/0 (socket says 172.21.15.33:56782) 2024-05-15T13:58:38.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.453+0000 7f670daf1700 1 -- 172.21.15.33:0/4193486965 learned_addr learned my addr 172.21.15.33:0/4193486965 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:58:38.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.453+0000 7f670caef700 1 --2- 172.21.15.33:0/4193486965 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6708103600 0x7f670819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:38.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.453+0000 7f670daf1700 1 -- 172.21.15.33:0/4193486965 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6708103600 msgr2=0x7f670819fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:38.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.453+0000 7f670daf1700 1 --2- 172.21.15.33:0/4193486965 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6708103600 0x7f670819fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:38.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.453+0000 7f670daf1700 1 -- 172.21.15.33:0/4193486965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6708102c40 msgr2=0x7f670819f740 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-15T13:58:38.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.453+0000 7f670daf1700 1 --2- 172.21.15.33:0/4193486965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6708102c40 0x7f670819f740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:38.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.453+0000 7f670daf1700 1 -- 172.21.15.33:0/4193486965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f670819a0b0 con 0x7f67081084c0 2024-05-15T13:58:38.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.454+0000 7f670daf1700 1 --2- 172.21.15.33:0/4193486965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67081084c0 0x7f6708199950 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f67000062a0 tx=0x7f6700004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:38.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.454+0000 7f66fa7fc700 1 -- 172.21.15.33:0/4193486965 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6700021070 con 0x7f67081084c0 2024-05-15T13:58:38.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.454+0000 7f66fa7fc700 1 -- 172.21.15.33:0/4193486965 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6700007c80 con 0x7f67081084c0 2024-05-15T13:58:38.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.454+0000 7f670f554700 1 -- 172.21.15.33:0/4193486965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f670819a340 con 0x7f67081084c0 2024-05-15T13:58:38.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.454+0000 7f66fa7fc700 1 -- 172.21.15.33:0/4193486965 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670001d450 con 0x7f67081084c0 2024-05-15T13:58:38.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.454+0000 7f670f554700 1 -- 172.21.15.33:0/4193486965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67081a6610 con 0x7f67081084c0 2024-05-15T13:58:38.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.455+0000 7f670f554700 1 -- 172.21.15.33:0/4193486965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f67081065a0 con 0x7f67081084c0 2024-05-15T13:58:38.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.457+0000 7f66fa7fc700 1 -- 172.21.15.33:0/4193486965 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f6700004000 con 0x7f67081084c0 2024-05-15T13:58:38.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.458+0000 7f66fa7fc700 1 --2- 172.21.15.33:0/4193486965 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f66f40b2990 0x7f66f40b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:38.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.458+0000 7f670d2f0700 1 --2- 172.21.15.33:0/4193486965 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f66f40b2990 0x7f66f40b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:38.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.458+0000 7f66fa7fc700 1 -- 172.21.15.33:0/4193486965 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f670000f020 con 0x7f67081084c0 2024-05-15T13:58:38.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.458+0000 7f66fa7fc700 1 --2- 172.21.15.33:0/4193486965 >> [v2:172.21.15.81:6824/326313090,v1:172.21.15.81:6825/326313090] conn(0x7f66f40bc550 0x7f66f40be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:38.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.458+0000 7f66fa7fc700 1 -- 172.21.15.33:0/4193486965 --> [v2:172.21.15.81:6824/326313090,v1:172.21.15.81:6825/326313090] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6700009ce0 con 0x7f66f40bc550 2024-05-15T13:58:38.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.458+0000 7f66fa7fc700 1 -- 172.21.15.33:0/4193486965 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f67000e4070 con 0x7f67081084c0 2024-05-15T13:58:38.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.458+0000 7f670d2f0700 1 --2- 172.21.15.33:0/4193486965 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f66f40b2990 0x7f66f40b4e50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6708107610 tx=0x7f66fc005c30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:38.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.459+0000 7f670caef700 1 --2- 172.21.15.33:0/4193486965 >> [v2:172.21.15.81:6824/326313090,v1:172.21.15.81:6825/326313090] conn(0x7f66f40bc550 0x7f66f40be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:38.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.459+0000 7f670caef700 1 --2- 172.21.15.33:0/4193486965 >> [v2:172.21.15.81:6824/326313090,v1:172.21.15.81:6825/326313090] conn(0x7f66f40bc550 0x7f66f40be9f0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:38.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.460+0000 7f66fa7fc700 1 -- 172.21.15.33:0/4193486965 <== osd.11 v2:172.21.15.81:6824/326313090 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6700009ce0 con 0x7f66f40bc550 2024-05-15T13:58:38.476 INFO:teuthology.orchestra.run.smithi033.stdout:283467841571 2024-05-15T13:58:38.476 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-15T13:58:38.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.477+0000 7f670f554700 1 -- 172.21.15.33:0/4193486965 --> [v2:172.21.15.81:6824/326313090,v1:172.21.15.81:6825/326313090] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f670806a230 con 0x7f66f40bc550 2024-05-15T13:58:38.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.478+0000 7f66fa7fc700 1 -- 172.21.15.33:0/4193486965 <== osd.11 v2:172.21.15.81:6824/326313090 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f670806a230 con 0x7f66f40bc550 2024-05-15T13:58:38.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.478+0000 7f670f554700 1 -- 172.21.15.33:0/4193486965 >> [v2:172.21.15.81:6824/326313090,v1:172.21.15.81:6825/326313090] conn(0x7f66f40bc550 msgr2=0x7f66f40be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:38.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.478+0000 7f670f554700 1 --2- 172.21.15.33:0/4193486965 >> [v2:172.21.15.81:6824/326313090,v1:172.21.15.81:6825/326313090] conn(0x7f66f40bc550 0x7f66f40be9f0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:38.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.478+0000 7f670f554700 1 -- 172.21.15.33:0/4193486965 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f66f40b2990 msgr2=0x7f66f40b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:38.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.478+0000 7f670f554700 1 --2- 172.21.15.33:0/4193486965 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f66f40b2990 0x7f66f40b4e50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6708107610 tx=0x7f66fc005c30 comp rx=0 tx=0).stop 2024-05-15T13:58:38.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.478+0000 7f670f554700 1 -- 172.21.15.33:0/4193486965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67081084c0 msgr2=0x7f6708199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:38.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.478+0000 7f670f554700 1 --2- 172.21.15.33:0/4193486965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67081084c0 0x7f6708199950 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f67000062a0 tx=0x7f6700004e20 comp rx=0 tx=0).stop 2024-05-15T13:58:38.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.478+0000 7f670daf1700 1 -- 172.21.15.33:0/4193486965 reap_dead start 2024-05-15T13:58:38.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.478+0000 7f670f554700 1 -- 172.21.15.33:0/4193486965 shutdown_connections 2024-05-15T13:58:38.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.478+0000 7f670f554700 1 -- 172.21.15.33:0/4193486965 >> 172.21.15.33:0/4193486965 conn(0x7f6708078ab0 msgr2=0x7f67081007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:38.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.479+0000 7f670f554700 1 -- 172.21.15.33:0/4193486965 shutdown_connections 2024-05-15T13:58:38.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:38.479+0000 7f670f554700 1 -- 172.21.15.33:0/4193486965 wait complete. 2024-05-15T13:58:38.969 INFO:teuthology.orchestra.run.smithi033.stdout:77309411402 2024-05-15T13:58:38.969 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-15T13:58:40.313 INFO:teuthology.orchestra.run.smithi033.stdout:309237645343 2024-05-15T13:58:40.313 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-15T13:58:40.337 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:40 smithi033 ceph-mon[31650]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:58:40.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:40 smithi052 ceph-mon[36190]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:58:40.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:40 smithi081 ceph-mon[36434]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:58:40.719 INFO:teuthology.orchestra.run.smithi033.stdout:261993005095 2024-05-15T13:58:40.719 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-15T13:58:41.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.379+0000 7efd96c1d700 1 -- 172.21.15.33:0/911363345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd9010ffe0 msgr2=0x7efd901123d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:41.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.379+0000 7efd96c1d700 1 --2- 172.21.15.33:0/911363345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd9010ffe0 0x7efd901123d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7efd8800b0a0 tx=0x7efd8800b3b0 comp rx=0 tx=0).stop 2024-05-15T13:58:41.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.379+0000 7efd96c1d700 1 -- 172.21.15.33:0/911363345 shutdown_connections 2024-05-15T13:58:41.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.379+0000 7efd96c1d700 1 --2- 172.21.15.33:0/911363345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd9010ffe0 0x7efd901123d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:41.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.379+0000 7efd96c1d700 1 --2- 172.21.15.33:0/911363345 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efd9006bae0 0x7efd9006bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:41.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.379+0000 7efd96c1d700 1 --2- 172.21.15.33:0/911363345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd9006c400 0x7efd9010faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:41.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.379+0000 7efd96c1d700 1 -- 172.21.15.33:0/911363345 >> 172.21.15.33:0/911363345 conn(0x7efd900793b0 msgr2=0x7efd900797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:41.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.379+0000 7efd96c1d700 1 -- 172.21.15.33:0/911363345 shutdown_connections 2024-05-15T13:58:41.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.379+0000 7efd96c1d700 1 -- 172.21.15.33:0/911363345 wait complete. 2024-05-15T13:58:41.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.379+0000 7efd96c1d700 1 Processor -- start 2024-05-15T13:58:41.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.379+0000 7efd96c1d700 1 -- start start 2024-05-15T13:58:41.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.380+0000 7efd96c1d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd9006bae0 0x7efd9019be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:41.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.380+0000 7efd96c1d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd9006c400 0x7efd9019c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:41.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.380+0000 7efd96c1d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efd9010ffe0 0x7efd901a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:41.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.380+0000 7efd96c1d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efd90115940 con 0x7efd9006bae0 2024-05-15T13:58:41.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.380+0000 7efd96c1d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efd901157c0 con 0x7efd9006c400 2024-05-15T13:58:41.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.380+0000 7efd96c1d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7efd90115ac0 con 0x7efd9010ffe0 2024-05-15T13:58:41.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.380+0000 7efd949b9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd9006bae0 0x7efd9019be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:41.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.380+0000 7efd949b9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd9006bae0 0x7efd9019be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56796/0 (socket says 172.21.15.33:56796) 2024-05-15T13:58:41.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.380+0000 7efd949b9700 1 -- 172.21.15.33:0/1435355737 learned_addr learned my addr 172.21.15.33:0/1435355737 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:58:41.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.380+0000 7efd8ffff700 1 --2- 172.21.15.33:0/1435355737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd9006c400 0x7efd9019c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:41.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.380+0000 7efd951ba700 1 --2- 172.21.15.33:0/1435355737 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efd9010ffe0 0x7efd901a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:41.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.380+0000 7efd949b9700 1 -- 172.21.15.33:0/1435355737 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efd9010ffe0 msgr2=0x7efd901a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:41.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.380+0000 7efd949b9700 1 --2- 172.21.15.33:0/1435355737 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efd9010ffe0 0x7efd901a0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:41.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.380+0000 7efd949b9700 1 -- 172.21.15.33:0/1435355737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd9006c400 msgr2=0x7efd9019c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:41.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.380+0000 7efd949b9700 1 --2- 172.21.15.33:0/1435355737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efd9006c400 0x7efd9019c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:41.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.380+0000 7efd949b9700 1 -- 172.21.15.33:0/1435355737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd901a0e80 con 0x7efd9006bae0 2024-05-15T13:58:41.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.380+0000 7efd949b9700 1 --2- 172.21.15.33:0/1435355737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd9006bae0 0x7efd9019be70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efd8400b530 tx=0x7efd8400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:41.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.380+0000 7efd8dffb700 1 -- 172.21.15.33:0/1435355737 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd84015020 con 0x7efd9006bae0 2024-05-15T13:58:41.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.380+0000 7efd96c1d700 1 -- 172.21.15.33:0/1435355737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd901a1170 con 0x7efd9006bae0 2024-05-15T13:58:41.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.380+0000 7efd8dffb700 1 -- 172.21.15.33:0/1435355737 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd84011ce0 con 0x7efd9006bae0 2024-05-15T13:58:41.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.380+0000 7efd8dffb700 1 -- 172.21.15.33:0/1435355737 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd84010d20 con 0x7efd9006bae0 2024-05-15T13:58:41.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.381+0000 7efd96c1d700 1 -- 172.21.15.33:0/1435355737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd901a89f0 con 0x7efd9006bae0 2024-05-15T13:58:41.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.381+0000 7efd96c1d700 1 -- 172.21.15.33:0/1435355737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efd90063cd0 con 0x7efd9006bae0 2024-05-15T13:58:41.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.383+0000 7efd8dffb700 1 -- 172.21.15.33:0/1435355737 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7efd84018020 con 0x7efd9006bae0 2024-05-15T13:58:41.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.384+0000 7efd8dffb700 1 --2- 172.21.15.33:0/1435355737 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7efd800b2910 0x7efd800b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:41.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.384+0000 7efd8ffff700 1 --2- 172.21.15.33:0/1435355737 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7efd800b2910 0x7efd800b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:41.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.384+0000 7efd8dffb700 1 -- 172.21.15.33:0/1435355737 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7efd840e2280 con 0x7efd9006bae0 2024-05-15T13:58:41.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.384+0000 7efd8dffb700 1 --2- 172.21.15.33:0/1435355737 >> [v2:172.21.15.52:6800/2794180003,v1:172.21.15.52:6801/2794180003] conn(0x7efd800bc4d0 0x7efd800be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:41.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.384+0000 7efd8dffb700 1 -- 172.21.15.33:0/1435355737 --> [v2:172.21.15.52:6800/2794180003,v1:172.21.15.52:6801/2794180003] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efd8401be50 con 0x7efd800bc4d0 2024-05-15T13:58:41.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.384+0000 7efd8dffb700 1 -- 172.21.15.33:0/1435355737 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7efd840e26b0 con 0x7efd9006bae0 2024-05-15T13:58:41.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.384+0000 7efd8ffff700 1 --2- 172.21.15.33:0/1435355737 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7efd800b2910 0x7efd800b4dd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efd90103900 tx=0x7efd7c005cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:41.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.384+0000 7efd951ba700 1 --2- 172.21.15.33:0/1435355737 >> [v2:172.21.15.52:6800/2794180003,v1:172.21.15.52:6801/2794180003] conn(0x7efd800bc4d0 0x7efd800be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:41.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.385+0000 7efd951ba700 1 --2- 172.21.15.33:0/1435355737 >> [v2:172.21.15.52:6800/2794180003,v1:172.21.15.52:6801/2794180003] conn(0x7efd800bc4d0 0x7efd800be970 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:41.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.386+0000 7efd8dffb700 1 -- 172.21.15.33:0/1435355737 <== osd.1 v2:172.21.15.52:6800/2794180003 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efd8401be50 con 0x7efd800bc4d0 2024-05-15T13:58:41.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.402+0000 7efd96c1d700 1 -- 172.21.15.33:0/1435355737 --> [v2:172.21.15.52:6800/2794180003,v1:172.21.15.52:6801/2794180003] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efd90064150 con 0x7efd800bc4d0 2024-05-15T13:58:41.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.403+0000 7efd8dffb700 1 -- 172.21.15.33:0/1435355737 <== osd.1 v2:172.21.15.52:6800/2794180003 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7efd90064150 con 0x7efd800bc4d0 2024-05-15T13:58:41.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.403+0000 7efd96c1d700 1 -- 172.21.15.33:0/1435355737 >> [v2:172.21.15.52:6800/2794180003,v1:172.21.15.52:6801/2794180003] conn(0x7efd800bc4d0 msgr2=0x7efd800be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:41.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.403+0000 7efd96c1d700 1 --2- 172.21.15.33:0/1435355737 >> [v2:172.21.15.52:6800/2794180003,v1:172.21.15.52:6801/2794180003] conn(0x7efd800bc4d0 0x7efd800be970 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:41.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.403+0000 7efd96c1d700 1 -- 172.21.15.33:0/1435355737 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7efd800b2910 msgr2=0x7efd800b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:41.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.403+0000 7efd96c1d700 1 --2- 172.21.15.33:0/1435355737 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7efd800b2910 0x7efd800b4dd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efd90103900 tx=0x7efd7c005cf0 comp rx=0 tx=0).stop 2024-05-15T13:58:41.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.403+0000 7efd96c1d700 1 -- 172.21.15.33:0/1435355737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd9006bae0 msgr2=0x7efd9019be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:41.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.403+0000 7efd96c1d700 1 --2- 172.21.15.33:0/1435355737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd9006bae0 0x7efd9019be70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efd8400b530 tx=0x7efd8400b8f0 comp rx=0 tx=0).stop 2024-05-15T13:58:41.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.403+0000 7efd951ba700 1 -- 172.21.15.33:0/1435355737 reap_dead start 2024-05-15T13:58:41.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.404+0000 7efd96c1d700 1 -- 172.21.15.33:0/1435355737 shutdown_connections 2024-05-15T13:58:41.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.404+0000 7efd96c1d700 1 -- 172.21.15.33:0/1435355737 >> 172.21.15.33:0/1435355737 conn(0x7efd900793b0 msgr2=0x7efd90111490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:41.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.404+0000 7efd96c1d700 1 -- 172.21.15.33:0/1435355737 shutdown_connections 2024-05-15T13:58:41.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.404+0000 7efd96c1d700 1 -- 172.21.15.33:0/1435355737 wait complete. 2024-05-15T13:58:41.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.670+0000 7fa50d5ca700 1 -- 172.21.15.33:0/3524428686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa50806c470 msgr2=0x7fa5081103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:41.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.670+0000 7fa50d5ca700 1 --2- 172.21.15.33:0/3524428686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa50806c470 0x7fa5081103c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa500009990 tx=0x7fa500009ca0 comp rx=0 tx=0).stop 2024-05-15T13:58:41.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.670+0000 7fa50d5ca700 1 -- 172.21.15.33:0/3524428686 shutdown_connections 2024-05-15T13:58:41.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.670+0000 7fa50d5ca700 1 --2- 172.21.15.33:0/3524428686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa50806c470 0x7fa5081103c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:41.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.670+0000 7fa50d5ca700 1 --2- 172.21.15.33:0/3524428686 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5081095a0 0x7fa508109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:41.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.670+0000 7fa50d5ca700 1 --2- 172.21.15.33:0/3524428686 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa50806bab0 0x7fa50806bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:41.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.670+0000 7fa50d5ca700 1 -- 172.21.15.33:0/3524428686 >> 172.21.15.33:0/3524428686 conn(0x7fa5080793b0 msgr2=0x7fa5080797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:41.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.670+0000 7fa50d5ca700 1 -- 172.21.15.33:0/3524428686 shutdown_connections 2024-05-15T13:58:41.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.670+0000 7fa50d5ca700 1 -- 172.21.15.33:0/3524428686 wait complete. 2024-05-15T13:58:41.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.671+0000 7fa50d5ca700 1 Processor -- start 2024-05-15T13:58:41.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.671+0000 7fa50d5ca700 1 -- start start 2024-05-15T13:58:41.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.671+0000 7fa50d5ca700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa50806bab0 0x7fa50819f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:41.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.671+0000 7fa50d5ca700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa50806c470 0x7fa50819fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:41.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.671+0000 7fa50d5ca700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5081095a0 0x7fa508199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:41.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.671+0000 7fa50d5ca700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa508113730 con 0x7fa50806c470 2024-05-15T13:58:41.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.671+0000 7fa50d5ca700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa5081135b0 con 0x7fa5081095a0 2024-05-15T13:58:41.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.671+0000 7fa50d5ca700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa5081138b0 con 0x7fa50806bab0 2024-05-15T13:58:41.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.671+0000 7fa5067fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa50806c470 0x7fa50819fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:41.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.671+0000 7fa5067fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa50806c470 0x7fa50819fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56814/0 (socket says 172.21.15.33:56814) 2024-05-15T13:58:41.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.671+0000 7fa5067fc700 1 -- 172.21.15.33:0/702735998 learned_addr learned my addr 172.21.15.33:0/702735998 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:58:41.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.672+0000 7fa506ffd700 1 --2- 172.21.15.33:0/702735998 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa50806bab0 0x7fa50819f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:41.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.672+0000 7fa5077fe700 1 --2- 172.21.15.33:0/702735998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5081095a0 0x7fa508199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:41.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.672+0000 7fa5067fc700 1 -- 172.21.15.33:0/702735998 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa50806bab0 msgr2=0x7fa50819f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:41.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.672+0000 7fa5067fc700 1 --2- 172.21.15.33:0/702735998 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa50806bab0 0x7fa50819f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:41.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.672+0000 7fa5067fc700 1 -- 172.21.15.33:0/702735998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5081095a0 msgr2=0x7fa508199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:41.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.672+0000 7fa5067fc700 1 --2- 172.21.15.33:0/702735998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5081095a0 0x7fa508199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:41.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.672+0000 7fa5067fc700 1 -- 172.21.15.33:0/702735998 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa50819a1d0 con 0x7fa50806c470 2024-05-15T13:58:41.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.672+0000 7fa5077fe700 1 --2- 172.21.15.33:0/702735998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5081095a0 0x7fa508199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T13:58:41.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.672+0000 7fa506ffd700 1 --2- 172.21.15.33:0/702735998 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa50806bab0 0x7fa50819f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T13:58:41.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.672+0000 7fa5067fc700 1 --2- 172.21.15.33:0/702735998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa50806c470 0x7fa50819fdf0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00b530 tx=0x7fa4fc00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:41.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.672+0000 7fa4effff700 1 -- 172.21.15.33:0/702735998 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc015020 con 0x7fa50806c470 2024-05-15T13:58:41.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.672+0000 7fa50d5ca700 1 -- 172.21.15.33:0/702735998 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa50819a4c0 con 0x7fa50806c470 2024-05-15T13:58:41.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.672+0000 7fa4effff700 1 -- 172.21.15.33:0/702735998 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4fc011ce0 con 0x7fa50806c470 2024-05-15T13:58:41.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.672+0000 7fa4effff700 1 -- 172.21.15.33:0/702735998 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc010d20 con 0x7fa50806c470 2024-05-15T13:58:41.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.672+0000 7fa50d5ca700 1 -- 172.21.15.33:0/702735998 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5081a6780 con 0x7fa50806c470 2024-05-15T13:58:41.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.673+0000 7fa50d5ca700 1 -- 172.21.15.33:0/702735998 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa508063c50 con 0x7fa50806c470 2024-05-15T13:58:41.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.675+0000 7fa4effff700 1 -- 172.21.15.33:0/702735998 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa4fc018020 con 0x7fa50806c470 2024-05-15T13:58:41.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.675+0000 7fa4effff700 1 --2- 172.21.15.33:0/702735998 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fa4f00b2910 0x7fa4f00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:41.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.676+0000 7fa506ffd700 1 --2- 172.21.15.33:0/702735998 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fa4f00b2910 0x7fa4f00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:41.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.676+0000 7fa4effff700 1 -- 172.21.15.33:0/702735998 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fa4fc0e2260 con 0x7fa50806c470 2024-05-15T13:58:41.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.676+0000 7fa4effff700 1 --2- 172.21.15.33:0/702735998 >> [v2:172.21.15.33:6818/1260691801,v1:172.21.15.33:6819/1260691801] conn(0x7fa4f00bc4d0 0x7fa4f00be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:41.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.676+0000 7fa4effff700 1 -- 172.21.15.33:0/702735998 --> [v2:172.21.15.33:6818/1260691801,v1:172.21.15.33:6819/1260691801] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa4fc00fec0 con 0x7fa4f00bc4d0 2024-05-15T13:58:41.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.676+0000 7fa4effff700 1 -- 172.21.15.33:0/702735998 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fa4fc0e2690 con 0x7fa50806c470 2024-05-15T13:58:41.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.676+0000 7fa5077fe700 1 --2- 172.21.15.33:0/702735998 >> [v2:172.21.15.33:6818/1260691801,v1:172.21.15.33:6819/1260691801] conn(0x7fa4f00bc4d0 0x7fa4f00be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:41.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.676+0000 7fa506ffd700 1 --2- 172.21.15.33:0/702735998 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fa4f00b2910 0x7fa4f00b4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa50819abb0 tx=0x7fa4f8009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:41.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.676+0000 7fa5077fe700 1 --2- 172.21.15.33:0/702735998 >> [v2:172.21.15.33:6818/1260691801,v1:172.21.15.33:6819/1260691801] conn(0x7fa4f00bc4d0 0x7fa4f00be970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:41.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.677+0000 7fa4effff700 1 -- 172.21.15.33:0/702735998 <== osd.6 v2:172.21.15.33:6818/1260691801 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa4fc00fec0 con 0x7fa4f00bc4d0 2024-05-15T13:58:41.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.694+0000 7fa50d5ca700 1 -- 172.21.15.33:0/702735998 --> [v2:172.21.15.33:6818/1260691801,v1:172.21.15.33:6819/1260691801] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa5080640d0 con 0x7fa4f00bc4d0 2024-05-15T13:58:41.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.694+0000 7fa4effff700 1 -- 172.21.15.33:0/702735998 <== osd.6 v2:172.21.15.33:6818/1260691801 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa5080640d0 con 0x7fa4f00bc4d0 2024-05-15T13:58:41.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.694+0000 7fa50d5ca700 1 -- 172.21.15.33:0/702735998 >> [v2:172.21.15.33:6818/1260691801,v1:172.21.15.33:6819/1260691801] conn(0x7fa4f00bc4d0 msgr2=0x7fa4f00be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:41.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.694+0000 7fa50d5ca700 1 --2- 172.21.15.33:0/702735998 >> [v2:172.21.15.33:6818/1260691801,v1:172.21.15.33:6819/1260691801] conn(0x7fa4f00bc4d0 0x7fa4f00be970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:41.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.694+0000 7fa50d5ca700 1 -- 172.21.15.33:0/702735998 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fa4f00b2910 msgr2=0x7fa4f00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:41.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.694+0000 7fa50d5ca700 1 --2- 172.21.15.33:0/702735998 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fa4f00b2910 0x7fa4f00b4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa50819abb0 tx=0x7fa4f8009040 comp rx=0 tx=0).stop 2024-05-15T13:58:41.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.694+0000 7fa50d5ca700 1 -- 172.21.15.33:0/702735998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa50806c470 msgr2=0x7fa50819fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:41.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.694+0000 7fa50d5ca700 1 --2- 172.21.15.33:0/702735998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa50806c470 0x7fa50819fdf0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00b530 tx=0x7fa4fc00b8f0 comp rx=0 tx=0).stop 2024-05-15T13:58:41.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.694+0000 7fa5077fe700 1 -- 172.21.15.33:0/702735998 reap_dead start 2024-05-15T13:58:41.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.694+0000 7fa50d5ca700 1 -- 172.21.15.33:0/702735998 shutdown_connections 2024-05-15T13:58:41.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.695+0000 7fa50d5ca700 1 -- 172.21.15.33:0/702735998 >> 172.21.15.33:0/702735998 conn(0x7fa5080793b0 msgr2=0x7fa508101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:41.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.695+0000 7fa50d5ca700 1 -- 172.21.15.33:0/702735998 shutdown_connections 2024-05-15T13:58:41.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:41.695+0000 7fa50d5ca700 1 -- 172.21.15.33:0/702735998 wait complete. 2024-05-15T13:58:42.307 INFO:teuthology.orchestra.run.smithi033.stdout:188978561082 2024-05-15T13:58:42.307 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-15T13:58:42.331 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:42 smithi033 ceph-mon[31650]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T13:58:42.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:42 smithi052 ceph-mon[36190]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T13:58:42.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:42 smithi081 ceph-mon[36434]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T13:58:42.928 INFO:teuthology.orchestra.run.smithi033.stdout:55834574929 2024-05-15T13:58:42.928 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-15T13:58:43.067 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:58:43.301 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:58:43.302 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:58:43.302 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:58:43.303 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:58:43.319 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:58:43.321 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:58:43.326 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:58:43.326 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:58:43.326 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:58:43.335 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:58:44.190 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:44 smithi033 ceph-mon[31650]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-15T13:58:44.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:44 smithi052 ceph-mon[36190]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-15T13:58:44.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:44 smithi081 ceph-mon[36434]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-15T13:58:46.331 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:46 smithi033 ceph-mon[31650]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:58:46.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:46 smithi052 ceph-mon[36190]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:58:46.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:46 smithi081 ceph-mon[36434]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:58:48.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:48 smithi052 ceph-mon[36190]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-15T13:58:48.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:48 smithi081 ceph-mon[36434]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-15T13:58:48.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:48 smithi033 ceph-mon[31650]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-15T13:58:49.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.857+0000 7f4a21d70700 1 -- 172.21.15.33:0/66008322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a1c075770 msgr2=0x7f4a1c075b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:49.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.857+0000 7f4a21d70700 1 --2- 172.21.15.33:0/66008322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a1c075770 0x7f4a1c075b50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c009990 tx=0x7f4a0c009ca0 comp rx=0 tx=0).stop 2024-05-15T13:58:49.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.857+0000 7f4a21d70700 1 -- 172.21.15.33:0/66008322 shutdown_connections 2024-05-15T13:58:49.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.857+0000 7f4a21d70700 1 --2- 172.21.15.33:0/66008322 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4a1c114710 0x7f4a1c116b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:49.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.857+0000 7f4a21d70700 1 --2- 172.21.15.33:0/66008322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a1c075770 0x7f4a1c075b50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:49.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.857+0000 7f4a21d70700 1 --2- 172.21.15.33:0/66008322 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a1c076090 0x7f4a1c1141d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:49.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.857+0000 7f4a21d70700 1 -- 172.21.15.33:0/66008322 >> 172.21.15.33:0/66008322 conn(0x7f4a1c07a6f0 msgr2=0x7f4a1c07ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:49.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.857+0000 7f4a21d70700 1 -- 172.21.15.33:0/66008322 shutdown_connections 2024-05-15T13:58:49.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.857+0000 7f4a21d70700 1 -- 172.21.15.33:0/66008322 wait complete. 2024-05-15T13:58:49.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.857+0000 7f4a21d70700 1 Processor -- start 2024-05-15T13:58:49.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.858+0000 7f4a21d70700 1 -- start start 2024-05-15T13:58:49.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.858+0000 7f4a21d70700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4a1c075770 0x7f4a1c1a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:49.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.858+0000 7f4a21d70700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a1c076090 0x7f4a1c1a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:49.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.858+0000 7f4a21d70700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a1c114710 0x7f4a1c1a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:49.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.858+0000 7f4a21d70700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1c119d70 con 0x7f4a1c114710 2024-05-15T13:58:49.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.858+0000 7f4a21d70700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1c119bf0 con 0x7f4a1c076090 2024-05-15T13:58:49.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.858+0000 7f4a21d70700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1c119ef0 con 0x7f4a1c075770 2024-05-15T13:58:49.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.858+0000 7f4a1bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a1c114710 0x7f4a1c1a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:49.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.858+0000 7f4a1bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a1c114710 0x7f4a1c1a4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57358/0 (socket says 172.21.15.33:57358) 2024-05-15T13:58:49.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.858+0000 7f4a1bfff700 1 -- 172.21.15.33:0/1838912326 learned_addr learned my addr 172.21.15.33:0/1838912326 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:58:49.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.858+0000 7f4a1b7fe700 1 --2- 172.21.15.33:0/1838912326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4a1c075770 0x7f4a1c1a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:49.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.858+0000 7f4a1affd700 1 --2- 172.21.15.33:0/1838912326 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a1c076090 0x7f4a1c1a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:49.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.858+0000 7f4a1bfff700 1 -- 172.21.15.33:0/1838912326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4a1c075770 msgr2=0x7f4a1c1a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:49.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.858+0000 7f4a1bfff700 1 --2- 172.21.15.33:0/1838912326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4a1c075770 0x7f4a1c1a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:49.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.858+0000 7f4a1bfff700 1 -- 172.21.15.33:0/1838912326 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a1c076090 msgr2=0x7f4a1c1a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:49.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.858+0000 7f4a1bfff700 1 --2- 172.21.15.33:0/1838912326 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a1c076090 0x7f4a1c1a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:49.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.858+0000 7f4a1bfff700 1 -- 172.21.15.33:0/1838912326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a1c1a5290 con 0x7f4a1c114710 2024-05-15T13:58:49.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.858+0000 7f4a1bfff700 1 --2- 172.21.15.33:0/1838912326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a1c114710 0x7f4a1c1a4b50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4a14007f10 tx=0x7f4a1400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:49.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.859+0000 7f4a18ff9700 1 -- 172.21.15.33:0/1838912326 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a14009e20 con 0x7f4a1c114710 2024-05-15T13:58:49.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.859+0000 7f4a18ff9700 1 -- 172.21.15.33:0/1838912326 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a14004bb0 con 0x7f4a1c114710 2024-05-15T13:58:49.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.859+0000 7f4a21d70700 1 -- 172.21.15.33:0/1838912326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a1c1a5580 con 0x7f4a1c114710 2024-05-15T13:58:49.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.859+0000 7f4a18ff9700 1 -- 172.21.15.33:0/1838912326 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a140058e0 con 0x7f4a1c114710 2024-05-15T13:58:49.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.859+0000 7f4a21d70700 1 -- 172.21.15.33:0/1838912326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a1c1ace00 con 0x7f4a1c114710 2024-05-15T13:58:49.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.860+0000 7f4a21d70700 1 -- 172.21.15.33:0/1838912326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a1c06a230 con 0x7f4a1c114710 2024-05-15T13:58:49.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.862+0000 7f4a18ff9700 1 -- 172.21.15.33:0/1838912326 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f4a1401a020 con 0x7f4a1c114710 2024-05-15T13:58:49.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.863+0000 7f4a18ff9700 1 --2- 172.21.15.33:0/1838912326 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f4a040b2910 0x7f4a040b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:49.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.863+0000 7f4a1b7fe700 1 --2- 172.21.15.33:0/1838912326 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f4a040b2910 0x7f4a040b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:49.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.863+0000 7f4a18ff9700 1 -- 172.21.15.33:0/1838912326 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f4a140e2060 con 0x7f4a1c114710 2024-05-15T13:58:49.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.863+0000 7f4a1b7fe700 1 --2- 172.21.15.33:0/1838912326 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f4a040b2910 0x7f4a040b4dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00b5c0 tx=0x7f4a0c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:49.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:48.866+0000 7f4a18ff9700 1 -- 172.21.15.33:0/1838912326 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4a14010070 con 0x7f4a1c114710 2024-05-15T13:58:49.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:49.084+0000 7f4a21d70700 1 -- 172.21.15.33:0/1838912326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f4a1c1acfd0 con 0x7f4a1c114710 2024-05-15T13:58:49.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:49.084+0000 7f4a18ff9700 1 -- 172.21.15.33:0/1838912326 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4a14004d60 con 0x7f4a1c114710 2024-05-15T13:58:49.376 INFO:teuthology.orchestra.run.smithi033.stdout:188978561084 2024-05-15T13:58:49.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:49.087+0000 7f4a21d70700 1 -- 172.21.15.33:0/1838912326 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f4a040b2910 msgr2=0x7f4a040b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:49.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:49.087+0000 7f4a21d70700 1 --2- 172.21.15.33:0/1838912326 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f4a040b2910 0x7f4a040b4dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00b5c0 tx=0x7f4a0c005960 comp rx=0 tx=0).stop 2024-05-15T13:58:49.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:49.087+0000 7f4a21d70700 1 -- 172.21.15.33:0/1838912326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a1c114710 msgr2=0x7f4a1c1a4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:49.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:49.087+0000 7f4a21d70700 1 --2- 172.21.15.33:0/1838912326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a1c114710 0x7f4a1c1a4b50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4a14007f10 tx=0x7f4a1400d3b0 comp rx=0 tx=0).stop 2024-05-15T13:58:49.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:49.087+0000 7f4a21d70700 1 -- 172.21.15.33:0/1838912326 shutdown_connections 2024-05-15T13:58:49.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:49.087+0000 7f4a21d70700 1 --2- 172.21.15.33:0/1838912326 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f4a040b2910 0x7f4a040b4dd0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:49.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:49.087+0000 7f4a21d70700 1 --2- 172.21.15.33:0/1838912326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a1c114710 0x7f4a1c1a4b50 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:49.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:49.087+0000 7f4a21d70700 1 --2- 172.21.15.33:0/1838912326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4a1c075770 0x7f4a1c1a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:49.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:49.087+0000 7f4a21d70700 1 --2- 172.21.15.33:0/1838912326 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a1c076090 0x7f4a1c1a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:49.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:49.087+0000 7f4a21d70700 1 -- 172.21.15.33:0/1838912326 >> 172.21.15.33:0/1838912326 conn(0x7f4a1c07a6f0 msgr2=0x7f4a1c1135b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:49.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:49.087+0000 7f4a21d70700 1 -- 172.21.15.33:0/1838912326 shutdown_connections 2024-05-15T13:58:49.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:49.087+0000 7f4a21d70700 1 -- 172.21.15.33:0/1838912326 wait complete. 2024-05-15T13:58:50.285 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:50 smithi033 ceph-mon[31650]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:58:50.285 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:50 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/1838912326' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-15T13:58:50.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:50 smithi052 ceph-mon[36190]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:58:50.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:50 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/1838912326' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-15T13:58:50.356 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:50 smithi081 ceph-mon[36434]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:58:50.356 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:50 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/1838912326' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-15T13:58:50.998 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561082 got 188978561084 for osd.6 2024-05-15T13:58:50.999 DEBUG:teuthology.parallel:result is None 2024-05-15T13:58:52.323 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:52 smithi033 ceph-mon[31650]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T13:58:52.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:52 smithi052 ceph-mon[36190]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T13:58:52.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:52 smithi081 ceph-mon[36434]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T13:58:53.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.434+0000 7f8bfc863700 1 -- 172.21.15.33:0/3681100524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bf806c6d0 msgr2=0x7f8bf8110400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:53.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.434+0000 7f8bfc863700 1 --2- 172.21.15.33:0/3681100524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bf806c6d0 0x7f8bf8110400 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8bf000b0d0 tx=0x7f8bf000b3e0 comp rx=0 tx=0).stop 2024-05-15T13:58:53.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.434+0000 7f8bfc863700 1 -- 172.21.15.33:0/3681100524 shutdown_connections 2024-05-15T13:58:53.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.434+0000 7f8bfc863700 1 --2- 172.21.15.33:0/3681100524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bf806c6d0 0x7f8bf8110400 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:53.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.434+0000 7f8bfc863700 1 --2- 172.21.15.33:0/3681100524 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8bf8109630 0x7f8bf8109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:53.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.434+0000 7f8bfc863700 1 --2- 172.21.15.33:0/3681100524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8bf806bd10 0x7f8bf806c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:53.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.434+0000 7f8bfc863700 1 -- 172.21.15.33:0/3681100524 >> 172.21.15.33:0/3681100524 conn(0x7f8bf80794e0 msgr2=0x7f8bf80798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:53.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.434+0000 7f8bfc863700 1 -- 172.21.15.33:0/3681100524 shutdown_connections 2024-05-15T13:58:53.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.434+0000 7f8bfc863700 1 -- 172.21.15.33:0/3681100524 wait complete. 2024-05-15T13:58:53.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.435+0000 7f8bfc863700 1 Processor -- start 2024-05-15T13:58:53.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.435+0000 7f8bfc863700 1 -- start start 2024-05-15T13:58:53.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.435+0000 7f8bfc863700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8bf806bd10 0x7f8bf819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:53.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.435+0000 7f8bfc863700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bf806c6d0 0x7f8bf819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:53.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.435+0000 7f8bfc863700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8bf8109630 0x7f8bf81999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:53.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.435+0000 7f8bfc863700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf8113790 con 0x7f8bf806c6d0 2024-05-15T13:58:53.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.435+0000 7f8bfc863700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf8113610 con 0x7f8bf806bd10 2024-05-15T13:58:53.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.435+0000 7f8bfc863700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf8113910 con 0x7f8bf8109630 2024-05-15T13:58:53.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.435+0000 7f8bedd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bf806c6d0 0x7f8bf819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:53.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.435+0000 7f8bedd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bf806c6d0 0x7f8bf819fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51330/0 (socket says 172.21.15.33:51330) 2024-05-15T13:58:53.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.435+0000 7f8bedd9b700 1 -- 172.21.15.33:0/2962755390 learned_addr learned my addr 172.21.15.33:0/2962755390 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:58:53.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.435+0000 7f8bf659c700 1 --2- 172.21.15.33:0/2962755390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8bf806bd10 0x7f8bf819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:53.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.435+0000 7f8bf6d9d700 1 --2- 172.21.15.33:0/2962755390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8bf8109630 0x7f8bf81999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:53.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.436+0000 7f8bedd9b700 1 -- 172.21.15.33:0/2962755390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8bf8109630 msgr2=0x7f8bf81999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:53.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.436+0000 7f8bedd9b700 1 --2- 172.21.15.33:0/2962755390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8bf8109630 0x7f8bf81999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:53.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.436+0000 7f8bedd9b700 1 -- 172.21.15.33:0/2962755390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8bf806bd10 msgr2=0x7f8bf819f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:53.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.436+0000 7f8bedd9b700 1 --2- 172.21.15.33:0/2962755390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8bf806bd10 0x7f8bf819f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:53.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.436+0000 7f8bedd9b700 1 -- 172.21.15.33:0/2962755390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf819a110 con 0x7f8bf806c6d0 2024-05-15T13:58:53.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.436+0000 7f8bf659c700 1 --2- 172.21.15.33:0/2962755390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8bf806bd10 0x7f8bf819f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T13:58:53.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.436+0000 7f8bedd9b700 1 --2- 172.21.15.33:0/2962755390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bf806c6d0 0x7f8bf819fd30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8be000d950 tx=0x7f8be000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:53.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.436+0000 7f8beffff700 1 -- 172.21.15.33:0/2962755390 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be0016070 con 0x7f8bf806c6d0 2024-05-15T13:58:53.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.436+0000 7f8bfc863700 1 -- 172.21.15.33:0/2962755390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bf819a400 con 0x7f8bf806c6d0 2024-05-15T13:58:53.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.436+0000 7f8beffff700 1 -- 172.21.15.33:0/2962755390 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8be000fd40 con 0x7f8bf806c6d0 2024-05-15T13:58:53.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.436+0000 7f8beffff700 1 -- 172.21.15.33:0/2962755390 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be000e3b0 con 0x7f8bf806c6d0 2024-05-15T13:58:53.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.436+0000 7f8bfc863700 1 -- 172.21.15.33:0/2962755390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bf81a66c0 con 0x7f8bf806c6d0 2024-05-15T13:58:53.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.437+0000 7f8bfc863700 1 -- 172.21.15.33:0/2962755390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bf810bac0 con 0x7f8bf806c6d0 2024-05-15T13:58:53.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.439+0000 7f8beffff700 1 -- 172.21.15.33:0/2962755390 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f8be0019070 con 0x7f8bf806c6d0 2024-05-15T13:58:53.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.439+0000 7f8beffff700 1 --2- 172.21.15.33:0/2962755390 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f8be40b2910 0x7f8be40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:53.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.439+0000 7f8bf659c700 1 --2- 172.21.15.33:0/2962755390 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f8be40b2910 0x7f8be40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:53.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.439+0000 7f8beffff700 1 -- 172.21.15.33:0/2962755390 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f8be00e1e00 con 0x7f8bf806c6d0 2024-05-15T13:58:53.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.440+0000 7f8bf659c700 1 --2- 172.21.15.33:0/2962755390 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f8be40b2910 0x7f8be40b4dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8bf819aaf0 tx=0x7f8be800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:53.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.443+0000 7f8beffff700 1 -- 172.21.15.33:0/2962755390 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8be00a99c0 con 0x7f8bf806c6d0 2024-05-15T13:58:53.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.660+0000 7f8bfc863700 1 -- 172.21.15.33:0/2962755390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f8bf819aee0 con 0x7f8bf806c6d0 2024-05-15T13:58:53.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.660+0000 7f8beffff700 1 -- 172.21.15.33:0/2962755390 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8be00a92c0 con 0x7f8bf806c6d0 2024-05-15T13:58:53.662 INFO:teuthology.orchestra.run.smithi033.stdout:210453397558 2024-05-15T13:58:53.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.663+0000 7f8bfc863700 1 -- 172.21.15.33:0/2962755390 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f8be40b2910 msgr2=0x7f8be40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:53.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.663+0000 7f8bfc863700 1 --2- 172.21.15.33:0/2962755390 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f8be40b2910 0x7f8be40b4dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8bf819aaf0 tx=0x7f8be800b3c0 comp rx=0 tx=0).stop 2024-05-15T13:58:53.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.663+0000 7f8bfc863700 1 -- 172.21.15.33:0/2962755390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bf806c6d0 msgr2=0x7f8bf819fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:53.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.663+0000 7f8bfc863700 1 --2- 172.21.15.33:0/2962755390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bf806c6d0 0x7f8bf819fd30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8be000d950 tx=0x7f8be000dc60 comp rx=0 tx=0).stop 2024-05-15T13:58:53.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.663+0000 7f8bfc863700 1 -- 172.21.15.33:0/2962755390 shutdown_connections 2024-05-15T13:58:53.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.663+0000 7f8bfc863700 1 --2- 172.21.15.33:0/2962755390 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f8be40b2910 0x7f8be40b4dd0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:53.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.663+0000 7f8bfc863700 1 --2- 172.21.15.33:0/2962755390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8bf8109630 0x7f8bf81999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:53.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.663+0000 7f8bfc863700 1 --2- 172.21.15.33:0/2962755390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8bf806bd10 0x7f8bf819f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:53.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.663+0000 7f8bfc863700 1 --2- 172.21.15.33:0/2962755390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bf806c6d0 0x7f8bf819fd30 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:53.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.663+0000 7f8bfc863700 1 -- 172.21.15.33:0/2962755390 >> 172.21.15.33:0/2962755390 conn(0x7f8bf80794e0 msgr2=0x7f8bf8100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:53.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.663+0000 7f8bfc863700 1 -- 172.21.15.33:0/2962755390 shutdown_connections 2024-05-15T13:58:53.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:53.663+0000 7f8bfc863700 1 -- 172.21.15.33:0/2962755390 wait complete. 2024-05-15T13:58:54.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:54 smithi052 ceph-mon[36190]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-15T13:58:54.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:54 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/2962755390' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-15T13:58:54.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:54 smithi081 ceph-mon[36434]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-15T13:58:54.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:54 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/2962755390' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-15T13:58:54.360 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:54 smithi033 ceph-mon[31650]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-15T13:58:54.360 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:54 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/2962755390' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-15T13:58:55.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.555+0000 7f463cd15700 1 -- 172.21.15.33:0/3347673896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4638103820 msgr2=0x7f4638103c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:55.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.555+0000 7f463cd15700 1 --2- 172.21.15.33:0/3347673896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4638103820 0x7f4638103c00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f46280099c0 tx=0x7f4628009cd0 comp rx=0 tx=0).stop 2024-05-15T13:58:55.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.555+0000 7f463cd15700 1 -- 172.21.15.33:0/3347673896 shutdown_connections 2024-05-15T13:58:55.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.555+0000 7f463cd15700 1 --2- 172.21.15.33:0/3347673896 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4638108910 0x7f4638110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:55.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.555+0000 7f463cd15700 1 --2- 172.21.15.33:0/3347673896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4638103820 0x7f4638103c00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:55.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.555+0000 7f463cd15700 1 --2- 172.21.15.33:0/3347673896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46381041d0 0x7f46381081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:55.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.555+0000 7f463cd15700 1 -- 172.21.15.33:0/3347673896 >> 172.21.15.33:0/3347673896 conn(0x7f463807a6b0 msgr2=0x7f463807aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:55.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.555+0000 7f463cd15700 1 -- 172.21.15.33:0/3347673896 shutdown_connections 2024-05-15T13:58:55.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.555+0000 7f463cd15700 1 -- 172.21.15.33:0/3347673896 wait complete. 2024-05-15T13:58:55.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.555+0000 7f463cd15700 1 Processor -- start 2024-05-15T13:58:55.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.556+0000 7f463cd15700 1 -- start start 2024-05-15T13:58:55.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.556+0000 7f463cd15700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4638103820 0x7f463819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:55.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.556+0000 7f463cd15700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46381041d0 0x7f463819fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:55.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.556+0000 7f463cd15700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4638108910 0x7f4638199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:55.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.556+0000 7f463cd15700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f46381136f0 con 0x7f4638108910 2024-05-15T13:58:55.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.556+0000 7f463cd15700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4638113570 con 0x7f4638103820 2024-05-15T13:58:55.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.556+0000 7f463cd15700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4638113870 con 0x7f46381041d0 2024-05-15T13:58:55.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.556+0000 7f4636d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4638108910 0x7f4638199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:55.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.556+0000 7f4636d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4638108910 0x7f4638199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51340/0 (socket says 172.21.15.33:51340) 2024-05-15T13:58:55.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.556+0000 7f4636d9d700 1 -- 172.21.15.33:0/3176823271 learned_addr learned my addr 172.21.15.33:0/3176823271 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:58:55.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.556+0000 7f463659c700 1 --2- 172.21.15.33:0/3176823271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4638103820 0x7f463819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:55.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.556+0000 7f4635d9b700 1 --2- 172.21.15.33:0/3176823271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46381041d0 0x7f463819fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:55.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.556+0000 7f4636d9d700 1 -- 172.21.15.33:0/3176823271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46381041d0 msgr2=0x7f463819fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:55.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.556+0000 7f4636d9d700 1 --2- 172.21.15.33:0/3176823271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46381041d0 0x7f463819fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:55.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.556+0000 7f4636d9d700 1 -- 172.21.15.33:0/3176823271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4638103820 msgr2=0x7f463819f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:55.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.556+0000 7f4636d9d700 1 --2- 172.21.15.33:0/3176823271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4638103820 0x7f463819f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:55.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.556+0000 7f4636d9d700 1 -- 172.21.15.33:0/3176823271 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f463819a070 con 0x7f4638108910 2024-05-15T13:58:55.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.556+0000 7f463659c700 1 --2- 172.21.15.33:0/3176823271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4638103820 0x7f463819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T13:58:55.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.556+0000 7f4635d9b700 1 --2- 172.21.15.33:0/3176823271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46381041d0 0x7f463819fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T13:58:55.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.557+0000 7f4636d9d700 1 --2- 172.21.15.33:0/3176823271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4638108910 0x7f4638199910 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4630007d70 tx=0x7f463000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:55.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.557+0000 7f46277fe700 1 -- 172.21.15.33:0/3176823271 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4630016020 con 0x7f4638108910 2024-05-15T13:58:55.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.557+0000 7f46277fe700 1 -- 172.21.15.33:0/3176823271 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f463000ad40 con 0x7f4638108910 2024-05-15T13:58:55.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.557+0000 7f463cd15700 1 -- 172.21.15.33:0/3176823271 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f463819a360 con 0x7f4638108910 2024-05-15T13:58:55.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.557+0000 7f46277fe700 1 -- 172.21.15.33:0/3176823271 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46300115e0 con 0x7f4638108910 2024-05-15T13:58:55.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.557+0000 7f463cd15700 1 -- 172.21.15.33:0/3176823271 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46381a6620 con 0x7f4638108910 2024-05-15T13:58:55.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.558+0000 7f463cd15700 1 -- 172.21.15.33:0/3176823271 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f463810bb50 con 0x7f4638108910 2024-05-15T13:58:55.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.559+0000 7f46277fe700 1 -- 172.21.15.33:0/3176823271 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f4630019020 con 0x7f4638108910 2024-05-15T13:58:55.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.560+0000 7f46277fe700 1 --2- 172.21.15.33:0/3176823271 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f46200b2990 0x7f46200b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:55.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.560+0000 7f463659c700 1 --2- 172.21.15.33:0/3176823271 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f46200b2990 0x7f46200b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:55.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.560+0000 7f46277fe700 1 -- 172.21.15.33:0/3176823271 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f46300e21e0 con 0x7f4638108910 2024-05-15T13:58:55.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.561+0000 7f463659c700 1 --2- 172.21.15.33:0/3176823271 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f46200b2990 0x7f46200b4e50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f463819aa50 tx=0x7f4628000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:55.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.565+0000 7f46277fe700 1 -- 172.21.15.33:0/3176823271 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4630020020 con 0x7f4638108910 2024-05-15T13:58:55.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.784+0000 7f463cd15700 1 -- 172.21.15.33:0/3176823271 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f463810b530 con 0x7f4638108910 2024-05-15T13:58:55.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.784+0000 7f46277fe700 1 -- 172.21.15.33:0/3176823271 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f46300a9eb0 con 0x7f4638108910 2024-05-15T13:58:55.787 INFO:teuthology.orchestra.run.smithi033.stdout:309237645346 2024-05-15T13:58:55.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.787+0000 7f463cd15700 1 -- 172.21.15.33:0/3176823271 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f46200b2990 msgr2=0x7f46200b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:55.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.787+0000 7f463cd15700 1 --2- 172.21.15.33:0/3176823271 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f46200b2990 0x7f46200b4e50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f463819aa50 tx=0x7f4628000bc0 comp rx=0 tx=0).stop 2024-05-15T13:58:55.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.787+0000 7f463cd15700 1 -- 172.21.15.33:0/3176823271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4638108910 msgr2=0x7f4638199910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:55.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.787+0000 7f463cd15700 1 --2- 172.21.15.33:0/3176823271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4638108910 0x7f4638199910 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4630007d70 tx=0x7f463000a340 comp rx=0 tx=0).stop 2024-05-15T13:58:55.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.787+0000 7f463cd15700 1 -- 172.21.15.33:0/3176823271 shutdown_connections 2024-05-15T13:58:55.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.787+0000 7f463cd15700 1 --2- 172.21.15.33:0/3176823271 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f46200b2990 0x7f46200b4e50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:55.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.787+0000 7f463cd15700 1 --2- 172.21.15.33:0/3176823271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4638108910 0x7f4638199910 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:55.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.787+0000 7f463cd15700 1 --2- 172.21.15.33:0/3176823271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4638103820 0x7f463819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:55.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.787+0000 7f463cd15700 1 --2- 172.21.15.33:0/3176823271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46381041d0 0x7f463819fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:55.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.787+0000 7f463cd15700 1 -- 172.21.15.33:0/3176823271 >> 172.21.15.33:0/3176823271 conn(0x7f463807a6b0 msgr2=0x7f4638102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:55.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.787+0000 7f463cd15700 1 -- 172.21.15.33:0/3176823271 shutdown_connections 2024-05-15T13:58:55.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.787+0000 7f463cd15700 1 -- 172.21.15.33:0/3176823271 wait complete. 2024-05-15T13:58:56.186 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:56 smithi033 ceph-mon[31650]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-15T13:58:56.187 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:56 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/3176823271' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-15T13:58:56.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.333+0000 7fea4cdd5700 1 -- 172.21.15.33:0/2823832494 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea48102440 msgr2=0x7fea481028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:56.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.333+0000 7fea4cdd5700 1 --2- 172.21.15.33:0/2823832494 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea48102440 0x7fea481028c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009990 tx=0x7fea3c009ca0 comp rx=0 tx=0).stop 2024-05-15T13:58:56.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.333+0000 7fea4cdd5700 1 -- 172.21.15.33:0/2823832494 shutdown_connections 2024-05-15T13:58:56.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.333+0000 7fea4cdd5700 1 --2- 172.21.15.33:0/2823832494 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea48102e00 0x7fea4810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:56.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.333+0000 7fea4cdd5700 1 --2- 172.21.15.33:0/2823832494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea48108520 0x7fea48108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:56.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.333+0000 7fea4cdd5700 1 --2- 172.21.15.33:0/2823832494 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea48102440 0x7fea481028c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:56.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.333+0000 7fea4cdd5700 1 -- 172.21.15.33:0/2823832494 >> 172.21.15.33:0/2823832494 conn(0x7fea480793f0 msgr2=0x7fea48079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:56.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.333+0000 7fea4cdd5700 1 -- 172.21.15.33:0/2823832494 shutdown_connections 2024-05-15T13:58:56.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.333+0000 7fea4cdd5700 1 -- 172.21.15.33:0/2823832494 wait complete. 2024-05-15T13:58:56.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.334+0000 7fea4cdd5700 1 Processor -- start 2024-05-15T13:58:56.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.335+0000 7fea4cdd5700 1 -- start start 2024-05-15T13:58:56.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.335+0000 7fea4cdd5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea48102440 0x7fea4819f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:56.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.335+0000 7fea4cdd5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea48102e00 0x7fea4819fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:56.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.335+0000 7fea4cdd5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea48108520 0x7fea48199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:56.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.335+0000 7fea4cdd5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fea481125f0 con 0x7fea48102440 2024-05-15T13:58:56.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.335+0000 7fea4cdd5700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fea48112470 con 0x7fea48108520 2024-05-15T13:58:56.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.335+0000 7fea4cdd5700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fea48112770 con 0x7fea48102e00 2024-05-15T13:58:56.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.335+0000 7fea4659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea48102440 0x7fea4819f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:56.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.335+0000 7fea4659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea48102440 0x7fea4819f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51382/0 (socket says 172.21.15.33:51382) 2024-05-15T13:58:56.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.335+0000 7fea4659c700 1 -- 172.21.15.33:0/2452460027 learned_addr learned my addr 172.21.15.33:0/2452460027 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:58:56.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.335+0000 7fea45d9b700 1 --2- 172.21.15.33:0/2452460027 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea48102e00 0x7fea4819fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:56.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.335+0000 7fea46d9d700 1 --2- 172.21.15.33:0/2452460027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea48108520 0x7fea48199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:56.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.336+0000 7fea4659c700 1 -- 172.21.15.33:0/2452460027 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea48102e00 msgr2=0x7fea4819fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:56.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.336+0000 7fea4659c700 1 --2- 172.21.15.33:0/2452460027 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea48102e00 0x7fea4819fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:56.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.336+0000 7fea4659c700 1 -- 172.21.15.33:0/2452460027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea48108520 msgr2=0x7fea48199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:56.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.336+0000 7fea4659c700 1 --2- 172.21.15.33:0/2452460027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea48108520 0x7fea48199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:56.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.336+0000 7fea4659c700 1 -- 172.21.15.33:0/2452460027 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea4819a1b0 con 0x7fea48102440 2024-05-15T13:58:56.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.336+0000 7fea45d9b700 1 --2- 172.21.15.33:0/2452460027 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea48102e00 0x7fea4819fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T13:58:56.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.336+0000 7fea4659c700 1 --2- 172.21.15.33:0/2452460027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea48102440 0x7fea4819f890 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fea3800d950 tx=0x7fea3800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:56.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.336+0000 7fea377fe700 1 -- 172.21.15.33:0/2452460027 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38016070 con 0x7fea48102440 2024-05-15T13:58:56.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.336+0000 7fea377fe700 1 -- 172.21.15.33:0/2452460027 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea3800fd40 con 0x7fea48102440 2024-05-15T13:58:56.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.336+0000 7fea4cdd5700 1 -- 172.21.15.33:0/2452460027 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea4819a4a0 con 0x7fea48102440 2024-05-15T13:58:56.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.336+0000 7fea377fe700 1 -- 172.21.15.33:0/2452460027 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3800e3b0 con 0x7fea48102440 2024-05-15T13:58:56.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.336+0000 7fea4cdd5700 1 -- 172.21.15.33:0/2452460027 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea481a6760 con 0x7fea48102440 2024-05-15T13:58:56.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.338+0000 7fea4cdd5700 1 -- 172.21.15.33:0/2452460027 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea4806a230 con 0x7fea48102440 2024-05-15T13:58:56.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:56 smithi052 ceph-mon[36190]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-15T13:58:56.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:56 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/3176823271' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-15T13:58:56.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.340+0000 7fea377fe700 1 -- 172.21.15.33:0/2452460027 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fea38019070 con 0x7fea48102440 2024-05-15T13:58:56.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.341+0000 7fea377fe700 1 --2- 172.21.15.33:0/2452460027 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fea300b2a10 0x7fea300b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:56.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.341+0000 7fea377fe700 1 -- 172.21.15.33:0/2452460027 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fea380e1da0 con 0x7fea48102440 2024-05-15T13:58:56.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.344+0000 7fea45d9b700 1 --2- 172.21.15.33:0/2452460027 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fea300b2a10 0x7fea300b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:56.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.346+0000 7fea45d9b700 1 --2- 172.21.15.33:0/2452460027 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fea300b2a10 0x7fea300b4ed0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00b2e0 tx=0x7fea3c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:56.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.349+0000 7fea377fe700 1 -- 172.21.15.33:0/2452460027 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fea380a9920 con 0x7fea48102440 2024-05-15T13:58:56.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:56 smithi081 ceph-mon[36434]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-15T13:58:56.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:56 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/3176823271' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-15T13:58:56.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.576+0000 7fea4cdd5700 1 -- 172.21.15.33:0/2452460027 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fea4802e560 con 0x7fea48102440 2024-05-15T13:58:56.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.576+0000 7fea377fe700 1 -- 172.21.15.33:0/2452460027 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fea380a9220 con 0x7fea48102440 2024-05-15T13:58:56.578 INFO:teuthology.orchestra.run.smithi033.stdout:236223201329 2024-05-15T13:58:56.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.579+0000 7fea4cdd5700 1 -- 172.21.15.33:0/2452460027 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fea300b2a10 msgr2=0x7fea300b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:56.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.579+0000 7fea4cdd5700 1 --2- 172.21.15.33:0/2452460027 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fea300b2a10 0x7fea300b4ed0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00b2e0 tx=0x7fea3c01a040 comp rx=0 tx=0).stop 2024-05-15T13:58:56.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.579+0000 7fea4cdd5700 1 -- 172.21.15.33:0/2452460027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea48102440 msgr2=0x7fea4819f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:56.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.579+0000 7fea4cdd5700 1 --2- 172.21.15.33:0/2452460027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea48102440 0x7fea4819f890 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fea3800d950 tx=0x7fea3800dc60 comp rx=0 tx=0).stop 2024-05-15T13:58:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.579+0000 7fea4cdd5700 1 -- 172.21.15.33:0/2452460027 shutdown_connections 2024-05-15T13:58:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.579+0000 7fea4cdd5700 1 --2- 172.21.15.33:0/2452460027 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fea300b2a10 0x7fea300b4ed0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.579+0000 7fea4cdd5700 1 --2- 172.21.15.33:0/2452460027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea48108520 0x7fea48199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.579+0000 7fea4cdd5700 1 --2- 172.21.15.33:0/2452460027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea48102440 0x7fea4819f890 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.579+0000 7fea4cdd5700 1 --2- 172.21.15.33:0/2452460027 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea48102e00 0x7fea4819fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.579+0000 7fea4cdd5700 1 -- 172.21.15.33:0/2452460027 >> 172.21.15.33:0/2452460027 conn(0x7fea480793f0 msgr2=0x7fea4806b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.579+0000 7fea4cdd5700 1 -- 172.21.15.33:0/2452460027 shutdown_connections 2024-05-15T13:58:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.579+0000 7fea4cdd5700 1 -- 172.21.15.33:0/2452460027 wait complete. 2024-05-15T13:58:56.883 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397554 got 210453397558 for osd.7 2024-05-15T13:58:56.883 DEBUG:teuthology.parallel:result is None 2024-05-15T13:58:56.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.934+0000 7fbe0cf44700 1 -- 172.21.15.33:0/2793264739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08108520 msgr2=0x7fbe08108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:56.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.934+0000 7fbe0cf44700 1 --2- 172.21.15.33:0/2793264739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08108520 0x7fbe08108900 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8009990 tx=0x7fbdf8009ca0 comp rx=0 tx=0).stop 2024-05-15T13:58:56.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.934+0000 7fbe0cf44700 1 -- 172.21.15.33:0/2793264739 shutdown_connections 2024-05-15T13:58:56.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.934+0000 7fbe0cf44700 1 --2- 172.21.15.33:0/2793264739 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe08102e00 0x7fbe0810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:56.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.934+0000 7fbe0cf44700 1 --2- 172.21.15.33:0/2793264739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08108520 0x7fbe08108900 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:56.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.934+0000 7fbe0cf44700 1 --2- 172.21.15.33:0/2793264739 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe08102440 0x7fbe081028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:56.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.934+0000 7fbe0cf44700 1 -- 172.21.15.33:0/2793264739 >> 172.21.15.33:0/2793264739 conn(0x7fbe080793f0 msgr2=0x7fbe08079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:56.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.934+0000 7fbe0cf44700 1 -- 172.21.15.33:0/2793264739 shutdown_connections 2024-05-15T13:58:56.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.934+0000 7fbe0cf44700 1 -- 172.21.15.33:0/2793264739 wait complete. 2024-05-15T13:58:56.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.935+0000 7fbe0cf44700 1 Processor -- start 2024-05-15T13:58:56.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.935+0000 7fbe0cf44700 1 -- start start 2024-05-15T13:58:56.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.935+0000 7fbe0cf44700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe08102440 0x7fbe0819ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:56.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.935+0000 7fbe0cf44700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe08102e00 0x7fbe0819b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:56.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.935+0000 7fbe0cf44700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08108520 0x7fbe081a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:56.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.935+0000 7fbe0cf44700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbe08112720 con 0x7fbe08108520 2024-05-15T13:58:56.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.935+0000 7fbe0cf44700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbe081125a0 con 0x7fbe08102440 2024-05-15T13:58:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.935+0000 7fbe0cf44700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbe081128a0 con 0x7fbe08102e00 2024-05-15T13:58:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.935+0000 7fbe06d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08108520 0x7fbe081a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.935+0000 7fbe06d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08108520 0x7fbe081a5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51364/0 (socket says 172.21.15.33:51364) 2024-05-15T13:58:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.935+0000 7fbe06d9d700 1 -- 172.21.15.33:0/3497141121 learned_addr learned my addr 172.21.15.33:0/3497141121 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:58:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.935+0000 7fbdfdd9b700 1 --2- 172.21.15.33:0/3497141121 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe08102e00 0x7fbe0819b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.935+0000 7fbe0659c700 1 --2- 172.21.15.33:0/3497141121 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe08102440 0x7fbe0819ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.936+0000 7fbe06d9d700 1 -- 172.21.15.33:0/3497141121 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe08102e00 msgr2=0x7fbe0819b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.936+0000 7fbe06d9d700 1 --2- 172.21.15.33:0/3497141121 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe08102e00 0x7fbe0819b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.936+0000 7fbe06d9d700 1 -- 172.21.15.33:0/3497141121 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe08102440 msgr2=0x7fbe0819ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.936+0000 7fbe06d9d700 1 --2- 172.21.15.33:0/3497141121 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe08102440 0x7fbe0819ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.936+0000 7fbe06d9d700 1 -- 172.21.15.33:0/3497141121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe081a6220 con 0x7fbe08108520 2024-05-15T13:58:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.936+0000 7fbe06d9d700 1 --2- 172.21.15.33:0/3497141121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08108520 0x7fbe081a5ce0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000b970 tx=0x7fbe0000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.936+0000 7fbdfffff700 1 -- 172.21.15.33:0/3497141121 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0000d860 con 0x7fbe08108520 2024-05-15T13:58:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.936+0000 7fbdfffff700 1 -- 172.21.15.33:0/3497141121 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe0000e430 con 0x7fbe08108520 2024-05-15T13:58:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.936+0000 7fbe0cf44700 1 -- 172.21.15.33:0/3497141121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe081a63f0 con 0x7fbe08108520 2024-05-15T13:58:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.936+0000 7fbdfffff700 1 -- 172.21.15.33:0/3497141121 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe00017660 con 0x7fbe08108520 2024-05-15T13:58:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.936+0000 7fbe0cf44700 1 -- 172.21.15.33:0/3497141121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe081a68e0 con 0x7fbe08108520 2024-05-15T13:58:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.937+0000 7fbe0cf44700 1 -- 172.21.15.33:0/3497141121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe0810a9b0 con 0x7fbe08108520 2024-05-15T13:58:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.939+0000 7fbdfffff700 1 -- 172.21.15.33:0/3497141121 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fbe0000da00 con 0x7fbe08108520 2024-05-15T13:58:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.940+0000 7fbdfffff700 1 --2- 172.21.15.33:0/3497141121 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbdf40b2910 0x7fbdf40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:58:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.940+0000 7fbe0659c700 1 --2- 172.21.15.33:0/3497141121 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbdf40b2910 0x7fbdf40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:58:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.940+0000 7fbdfffff700 1 -- 172.21.15.33:0/3497141121 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fbe000e1ff0 con 0x7fbe08108520 2024-05-15T13:58:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.940+0000 7fbe0659c700 1 --2- 172.21.15.33:0/3497141121 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbdf40b2910 0x7fbdf40b4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8009750 tx=0x7fbdf8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:58:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:55.944+0000 7fbdfffff700 1 -- 172.21.15.33:0/3497141121 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbe00013030 con 0x7fbe08108520 2024-05-15T13:58:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.171+0000 7fbe0cf44700 1 -- 172.21.15.33:0/3497141121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fbe0810a390 con 0x7fbe08108520 2024-05-15T13:58:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.171+0000 7fbdfffff700 1 -- 172.21.15.33:0/3497141121 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbe000a9bb0 con 0x7fbe08108520 2024-05-15T13:58:56.966 INFO:teuthology.orchestra.run.smithi033.stdout:34359738458 2024-05-15T13:58:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.174+0000 7fbe0cf44700 1 -- 172.21.15.33:0/3497141121 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbdf40b2910 msgr2=0x7fbdf40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.174+0000 7fbe0cf44700 1 --2- 172.21.15.33:0/3497141121 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbdf40b2910 0x7fbdf40b4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8009750 tx=0x7fbdf8005fd0 comp rx=0 tx=0).stop 2024-05-15T13:58:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.174+0000 7fbe0cf44700 1 -- 172.21.15.33:0/3497141121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08108520 msgr2=0x7fbe081a5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:58:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.174+0000 7fbe0cf44700 1 --2- 172.21.15.33:0/3497141121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08108520 0x7fbe081a5ce0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000b970 tx=0x7fbe0000bc80 comp rx=0 tx=0).stop 2024-05-15T13:58:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.174+0000 7fbe0cf44700 1 -- 172.21.15.33:0/3497141121 shutdown_connections 2024-05-15T13:58:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.174+0000 7fbe0cf44700 1 --2- 172.21.15.33:0/3497141121 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbdf40b2910 0x7fbdf40b4dd0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.174+0000 7fbe0cf44700 1 --2- 172.21.15.33:0/3497141121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08108520 0x7fbe081a5ce0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.174+0000 7fbe0cf44700 1 --2- 172.21.15.33:0/3497141121 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe08102440 0x7fbe0819ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:56.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.174+0000 7fbe0cf44700 1 --2- 172.21.15.33:0/3497141121 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe08102e00 0x7fbe0819b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:58:56.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.174+0000 7fbe0cf44700 1 -- 172.21.15.33:0/3497141121 >> 172.21.15.33:0/3497141121 conn(0x7fbe080793f0 msgr2=0x7fbe08101420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:58:56.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.174+0000 7fbe0cf44700 1 -- 172.21.15.33:0/3497141121 shutdown_connections 2024-05-15T13:58:56.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:56.174+0000 7fbe0cf44700 1 -- 172.21.15.33:0/3497141121 wait complete. 2024-05-15T13:58:57.190 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:57 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/3497141121' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-15T13:58:57.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:57 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/2452460027' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-15T13:58:57.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:57 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/3497141121' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-15T13:58:57.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:57 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/2452460027' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-15T13:58:57.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:57 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/3497141121' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-15T13:58:57.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:57 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/2452460027' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-15T13:58:58.356 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:58 smithi081 ceph-mon[36434]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-15T13:58:58.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:58 smithi033 ceph-mon[31650]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-15T13:58:58.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:58 smithi052 ceph-mon[36190]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-15T13:58:59.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:58:59 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/1863206726' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-15T13:58:59.418 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:58:59 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/1863206726' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-15T13:58:59.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:58:59 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/1863206726' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-15T13:59:00.216 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201325 got 236223201329 for osd.8 2024-05-15T13:59:00.216 DEBUG:teuthology.parallel:result is None 2024-05-15T13:59:00.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:00 smithi081 ceph-mon[36434]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:59:00.388 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:00 smithi033 ceph-mon[31650]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:59:00.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.414+0000 7f2833e61700 1 -- 172.21.15.33:0/508146270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f282c104450 msgr2=0x7f282c06bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:00.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.414+0000 7f2833e61700 1 --2- 172.21.15.33:0/508146270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f282c104450 0x7f282c06bc10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2828009990 tx=0x7f2828009ca0 comp rx=0 tx=0).stop 2024-05-15T13:59:00.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.414+0000 7f2833e61700 1 -- 172.21.15.33:0/508146270 shutdown_connections 2024-05-15T13:59:00.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.414+0000 7f2833e61700 1 --2- 172.21.15.33:0/508146270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f282c10fe90 0x7f282c112280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:00.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.414+0000 7f2833e61700 1 --2- 172.21.15.33:0/508146270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f282c104450 0x7f282c06bc10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:00.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.414+0000 7f2833e61700 1 --2- 172.21.15.33:0/508146270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f282c06c150 0x7f282c06c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:00.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.414+0000 7f2833e61700 1 -- 172.21.15.33:0/508146270 >> 172.21.15.33:0/508146270 conn(0x7f282c0793f0 msgr2=0x7f282c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:00.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.414+0000 7f2833e61700 1 -- 172.21.15.33:0/508146270 shutdown_connections 2024-05-15T13:59:00.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.414+0000 7f2833e61700 1 -- 172.21.15.33:0/508146270 wait complete. 2024-05-15T13:59:00.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.415+0000 7f2833e61700 1 Processor -- start 2024-05-15T13:59:00.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.415+0000 7f2833e61700 1 -- start start 2024-05-15T13:59:00.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.415+0000 7f2833e61700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f282c06c150 0x7f282c19beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:00.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.415+0000 7f2833e61700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f282c104450 0x7f282c19c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:00.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.415+0000 7f2833e61700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f282c10fe90 0x7f282c1a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:00.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.415+0000 7f2833e61700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f282c1159e0 con 0x7f282c10fe90 2024-05-15T13:59:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.415+0000 7f2833e61700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f282c115860 con 0x7f282c06c150 2024-05-15T13:59:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.415+0000 7f2833e61700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f282c115b60 con 0x7f282c104450 2024-05-15T13:59:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.415+0000 7f28323fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f282c10fe90 0x7f282c1a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.415+0000 7f28323fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f282c10fe90 0x7f282c1a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51404/0 (socket says 172.21.15.33:51404) 2024-05-15T13:59:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.415+0000 7f28323fe700 1 -- 172.21.15.33:0/665610472 learned_addr learned my addr 172.21.15.33:0/665610472 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.415+0000 7f2831bfd700 1 --2- 172.21.15.33:0/665610472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f282c06c150 0x7f282c19beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.415+0000 7f28313fc700 1 --2- 172.21.15.33:0/665610472 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f282c104450 0x7f282c19c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.415+0000 7f28323fe700 1 -- 172.21.15.33:0/665610472 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f282c104450 msgr2=0x7f282c19c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.415+0000 7f28323fe700 1 --2- 172.21.15.33:0/665610472 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f282c104450 0x7f282c19c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.415+0000 7f28323fe700 1 -- 172.21.15.33:0/665610472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f282c06c150 msgr2=0x7f282c19beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.415+0000 7f28323fe700 1 --2- 172.21.15.33:0/665610472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f282c06c150 0x7f282c19beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.415+0000 7f28323fe700 1 -- 172.21.15.33:0/665610472 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f282c1a0ec0 con 0x7f282c10fe90 2024-05-15T13:59:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.415+0000 7f28323fe700 1 --2- 172.21.15.33:0/665610472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f282c10fe90 0x7f282c1a0780 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f282400b9e0 tx=0x7f282400bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.416+0000 7f2822ffd700 1 -- 172.21.15.33:0/665610472 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282400d8e0 con 0x7f282c10fe90 2024-05-15T13:59:00.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.416+0000 7f2833e61700 1 -- 172.21.15.33:0/665610472 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f282c1a11b0 con 0x7f282c10fe90 2024-05-15T13:59:00.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.416+0000 7f2822ffd700 1 -- 172.21.15.33:0/665610472 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2824015420 con 0x7f282c10fe90 2024-05-15T13:59:00.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.416+0000 7f2822ffd700 1 -- 172.21.15.33:0/665610472 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2824014630 con 0x7f282c10fe90 2024-05-15T13:59:00.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.416+0000 7f2833e61700 1 -- 172.21.15.33:0/665610472 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f282c1a8a30 con 0x7f282c10fe90 2024-05-15T13:59:00.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.417+0000 7f2833e61700 1 -- 172.21.15.33:0/665610472 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f282c102a80 con 0x7f282c10fe90 2024-05-15T13:59:00.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.418+0000 7f2822ffd700 1 -- 172.21.15.33:0/665610472 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f28240155d0 con 0x7f282c10fe90 2024-05-15T13:59:00.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.419+0000 7f2822ffd700 1 --2- 172.21.15.33:0/665610472 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f28180b2910 0x7f28180b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:00.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.419+0000 7f2831bfd700 1 --2- 172.21.15.33:0/665610472 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f28180b2910 0x7f28180b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:00.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.419+0000 7f2822ffd700 1 -- 172.21.15.33:0/665610472 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f28240e1f10 con 0x7f282c10fe90 2024-05-15T13:59:00.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.420+0000 7f2831bfd700 1 --2- 172.21.15.33:0/665610472 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f28180b2910 0x7f28180b4dd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2828009750 tx=0x7f2828005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:00.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.423+0000 7f2822ffd700 1 -- 172.21.15.33:0/665610472 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f28240a9a90 con 0x7f282c10fe90 2024-05-15T13:59:00.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:00 smithi052 ceph-mon[36190]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:59:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.639+0000 7f2833e61700 1 -- 172.21.15.33:0/665610472 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f282c1a8c60 con 0x7f282c10fe90 2024-05-15T13:59:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.639+0000 7f2822ffd700 1 -- 172.21.15.33:0/665610472 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f28240a9390 con 0x7f282c10fe90 2024-05-15T13:59:00.641 INFO:teuthology.orchestra.run.smithi033.stdout:77309411408 2024-05-15T13:59:00.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.642+0000 7f2833e61700 1 -- 172.21.15.33:0/665610472 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f28180b2910 msgr2=0x7f28180b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:00.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.642+0000 7f2833e61700 1 --2- 172.21.15.33:0/665610472 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f28180b2910 0x7f28180b4dd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2828009750 tx=0x7f2828005fd0 comp rx=0 tx=0).stop 2024-05-15T13:59:00.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.642+0000 7f2833e61700 1 -- 172.21.15.33:0/665610472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f282c10fe90 msgr2=0x7f282c1a0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:00.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.642+0000 7f2833e61700 1 --2- 172.21.15.33:0/665610472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f282c10fe90 0x7f282c1a0780 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f282400b9e0 tx=0x7f282400bcf0 comp rx=0 tx=0).stop 2024-05-15T13:59:00.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.642+0000 7f2833e61700 1 -- 172.21.15.33:0/665610472 shutdown_connections 2024-05-15T13:59:00.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.642+0000 7f2833e61700 1 --2- 172.21.15.33:0/665610472 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f28180b2910 0x7f28180b4dd0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:00.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.642+0000 7f2833e61700 1 --2- 172.21.15.33:0/665610472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f282c10fe90 0x7f282c1a0780 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:00.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.642+0000 7f2833e61700 1 --2- 172.21.15.33:0/665610472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f282c06c150 0x7f282c19beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:00.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.642+0000 7f2833e61700 1 --2- 172.21.15.33:0/665610472 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f282c104450 0x7f282c19c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:00.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.642+0000 7f2833e61700 1 -- 172.21.15.33:0/665610472 >> 172.21.15.33:0/665610472 conn(0x7f282c0793f0 msgr2=0x7f282c101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:00.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.642+0000 7f2833e61700 1 -- 172.21.15.33:0/665610472 shutdown_connections 2024-05-15T13:59:00.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:00.642+0000 7f2833e61700 1 -- 172.21.15.33:0/665610472 wait complete. 2024-05-15T13:59:01.062 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645343 got 309237645346 for osd.11 2024-05-15T13:59:01.062 DEBUG:teuthology.parallel:result is None 2024-05-15T13:59:01.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.829+0000 7f74ed2d9700 1 -- 172.21.15.33:0/4195961243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74e81088f0 msgr2=0x7f74e8110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:01.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.829+0000 7f74ed2d9700 1 --2- 172.21.15.33:0/4195961243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74e81088f0 0x7f74e8110470 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f74e000b0d0 tx=0x7f74e000b3e0 comp rx=0 tx=0).stop 2024-05-15T13:59:01.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.829+0000 7f74ed2d9700 1 -- 172.21.15.33:0/4195961243 shutdown_connections 2024-05-15T13:59:01.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.829+0000 7f74ed2d9700 1 --2- 172.21.15.33:0/4195961243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74e81088f0 0x7f74e8110470 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:01.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.829+0000 7f74ed2d9700 1 --2- 172.21.15.33:0/4195961243 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74e81037f0 0x7f74e8103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:01.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.829+0000 7f74ed2d9700 1 --2- 172.21.15.33:0/4195961243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74e81041a0 0x7f74e81081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:01.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.829+0000 7f74ed2d9700 1 -- 172.21.15.33:0/4195961243 >> 172.21.15.33:0/4195961243 conn(0x7f74e807a6a0 msgr2=0x7f74e807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:01.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.829+0000 7f74ed2d9700 1 -- 172.21.15.33:0/4195961243 shutdown_connections 2024-05-15T13:59:01.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.829+0000 7f74ed2d9700 1 -- 172.21.15.33:0/4195961243 wait complete. 2024-05-15T13:59:01.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.829+0000 7f74ed2d9700 1 Processor -- start 2024-05-15T13:59:01.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.829+0000 7f74ed2d9700 1 -- start start 2024-05-15T13:59:01.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.830+0000 7f74ed2d9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74e81037f0 0x7f74e819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:01.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.830+0000 7f74ed2d9700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74e81041a0 0x7f74e819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:01.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.830+0000 7f74ed2d9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74e81088f0 0x7f74e8199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.830+0000 7f74ed2d9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f74e81136e0 con 0x7f74e81088f0 2024-05-15T13:59:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.830+0000 7f74ed2d9700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f74e8113560 con 0x7f74e81037f0 2024-05-15T13:59:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.830+0000 7f74ed2d9700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f74e8113860 con 0x7f74e81041a0 2024-05-15T13:59:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.830+0000 7f74e77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74e81088f0 0x7f74e8199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.830+0000 7f74e77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74e81088f0 0x7f74e8199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51396/0 (socket says 172.21.15.33:51396) 2024-05-15T13:59:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.830+0000 7f74e77fe700 1 -- 172.21.15.33:0/1863206726 learned_addr learned my addr 172.21.15.33:0/1863206726 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.830+0000 7f74e67fc700 1 --2- 172.21.15.33:0/1863206726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74e81041a0 0x7f74e819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.830+0000 7f74e6ffd700 1 --2- 172.21.15.33:0/1863206726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74e81037f0 0x7f74e819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:01.347 INFO:teuthology.orchestra.run.smithi033.stdout:163208757314 2024-05-15T13:59:01.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.830+0000 7f74e77fe700 1 -- 172.21.15.33:0/1863206726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74e81041a0 msgr2=0x7f74e819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:01.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.830+0000 7f74e77fe700 1 --2- 172.21.15.33:0/1863206726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74e81041a0 0x7f74e819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:01.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.830+0000 7f74e77fe700 1 -- 172.21.15.33:0/1863206726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74e81037f0 msgr2=0x7f74e819f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:01.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.830+0000 7f74e77fe700 1 --2- 172.21.15.33:0/1863206726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74e81037f0 0x7f74e819f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:01.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.830+0000 7f74e77fe700 1 -- 172.21.15.33:0/1863206726 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74e819a0c0 con 0x7f74e81088f0 2024-05-15T13:59:01.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.830+0000 7f74e6ffd700 1 --2- 172.21.15.33:0/1863206726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74e81037f0 0x7f74e819f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T13:59:01.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.830+0000 7f74e77fe700 1 --2- 172.21.15.33:0/1863206726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74e81088f0 0x7f74e8199960 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f74e0007a00 tx=0x7f74e0007a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:01.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.830+0000 7f74cffff700 1 -- 172.21.15.33:0/1863206726 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e001f070 con 0x7f74e81088f0 2024-05-15T13:59:01.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.831+0000 7f74ed2d9700 1 -- 172.21.15.33:0/1863206726 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74e819a350 con 0x7f74e81088f0 2024-05-15T13:59:01.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.831+0000 7f74cffff700 1 -- 172.21.15.33:0/1863206726 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74e0004c50 con 0x7f74e81088f0 2024-05-15T13:59:01.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.831+0000 7f74cffff700 1 -- 172.21.15.33:0/1863206726 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e001b690 con 0x7f74e81088f0 2024-05-15T13:59:01.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.831+0000 7f74ed2d9700 1 -- 172.21.15.33:0/1863206726 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74e81a6670 con 0x7f74e81088f0 2024-05-15T13:59:01.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.832+0000 7f74ed2d9700 1 -- 172.21.15.33:0/1863206726 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74e810bac0 con 0x7f74e81088f0 2024-05-15T13:59:01.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.833+0000 7f74cffff700 1 -- 172.21.15.33:0/1863206726 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f74e0022070 con 0x7f74e81088f0 2024-05-15T13:59:01.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.834+0000 7f74cffff700 1 --2- 172.21.15.33:0/1863206726 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f74d00b2910 0x7f74d00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:01.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.834+0000 7f74e6ffd700 1 --2- 172.21.15.33:0/1863206726 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f74d00b2910 0x7f74d00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:01.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.835+0000 7f74cffff700 1 -- 172.21.15.33:0/1863206726 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f74e000f020 con 0x7f74e81088f0 2024-05-15T13:59:01.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.835+0000 7f74e6ffd700 1 --2- 172.21.15.33:0/1863206726 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f74d00b2910 0x7f74d00b4dd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f74e819aaa0 tx=0x7f74d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:01.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:58.838+0000 7f74cffff700 1 -- 172.21.15.33:0/1863206726 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f74e0029020 con 0x7f74e81088f0 2024-05-15T13:59:01.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:59.054+0000 7f74ed2d9700 1 -- 172.21.15.33:0/1863206726 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f74e819ae90 con 0x7f74e81088f0 2024-05-15T13:59:01.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:59.054+0000 7f74cffff700 1 -- 172.21.15.33:0/1863206726 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f74e00ab4f0 con 0x7f74e81088f0 2024-05-15T13:59:01.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:59.056+0000 7f74ed2d9700 1 -- 172.21.15.33:0/1863206726 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f74d00b2910 msgr2=0x7f74d00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:01.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:59.056+0000 7f74ed2d9700 1 --2- 172.21.15.33:0/1863206726 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f74d00b2910 0x7f74d00b4dd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f74e819aaa0 tx=0x7f74d8009260 comp rx=0 tx=0).stop 2024-05-15T13:59:01.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:59.056+0000 7f74ed2d9700 1 -- 172.21.15.33:0/1863206726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74e81088f0 msgr2=0x7f74e8199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:01.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:59.056+0000 7f74ed2d9700 1 --2- 172.21.15.33:0/1863206726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74e81088f0 0x7f74e8199960 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f74e0007a00 tx=0x7f74e0007a30 comp rx=0 tx=0).stop 2024-05-15T13:59:01.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:59.056+0000 7f74ed2d9700 1 -- 172.21.15.33:0/1863206726 shutdown_connections 2024-05-15T13:59:01.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:59.056+0000 7f74ed2d9700 1 --2- 172.21.15.33:0/1863206726 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f74d00b2910 0x7f74d00b4dd0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:01.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:59.056+0000 7f74ed2d9700 1 --2- 172.21.15.33:0/1863206726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74e81088f0 0x7f74e8199960 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:01.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:59.056+0000 7f74ed2d9700 1 --2- 172.21.15.33:0/1863206726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74e81037f0 0x7f74e819f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:01.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:59.056+0000 7f74ed2d9700 1 --2- 172.21.15.33:0/1863206726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74e81041a0 0x7f74e819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:01.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:59.056+0000 7f74ed2d9700 1 -- 172.21.15.33:0/1863206726 >> 172.21.15.33:0/1863206726 conn(0x7f74e807a6a0 msgr2=0x7f74e8101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:01.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:59.057+0000 7f74ed2d9700 1 -- 172.21.15.33:0/1863206726 shutdown_connections 2024-05-15T13:59:01.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:58:59.057+0000 7f74ed2d9700 1 -- 172.21.15.33:0/1863206726 wait complete. 2024-05-15T13:59:01.350 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:01 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/665610472' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-15T13:59:01.356 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:01 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/665610472' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-15T13:59:01.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:01 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/665610472' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-15T13:59:02.232 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:02 smithi033 ceph-mon[31650]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T13:59:02.234 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738452 got 34359738458 for osd.0 2024-05-15T13:59:02.234 DEBUG:teuthology.parallel:result is None 2024-05-15T13:59:02.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:02 smithi081 ceph-mon[36434]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T13:59:02.427 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757306 got 163208757314 for osd.5 2024-05-15T13:59:02.427 DEBUG:teuthology.parallel:result is None 2024-05-15T13:59:02.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:02 smithi052 ceph-mon[36190]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T13:59:02.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.609+0000 7ff81d904700 1 -- 172.21.15.33:0/1882118347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff81806b220 msgr2=0x7ff81806b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:02.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.609+0000 7ff81d904700 1 --2- 172.21.15.33:0/1882118347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff81806b220 0x7ff81806b6a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff80c009990 tx=0x7ff80c009ca0 comp rx=0 tx=0).stop 2024-05-15T13:59:02.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.609+0000 7ff81d904700 1 -- 172.21.15.33:0/1882118347 shutdown_connections 2024-05-15T13:59:02.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.609+0000 7ff81d904700 1 --2- 172.21.15.33:0/1882118347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff81806bbe0 0x7ff8181103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:02.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.609+0000 7ff81d904700 1 --2- 172.21.15.33:0/1882118347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8181095b0 0x7ff818109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:02.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.609+0000 7ff81d904700 1 --2- 172.21.15.33:0/1882118347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff81806b220 0x7ff81806b6a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:02.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.609+0000 7ff81d904700 1 -- 172.21.15.33:0/1882118347 >> 172.21.15.33:0/1882118347 conn(0x7ff8180793f0 msgr2=0x7ff818079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:02.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.609+0000 7ff81d904700 1 -- 172.21.15.33:0/1882118347 shutdown_connections 2024-05-15T13:59:02.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.609+0000 7ff81d904700 1 -- 172.21.15.33:0/1882118347 wait complete. 2024-05-15T13:59:02.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.610+0000 7ff81d904700 1 Processor -- start 2024-05-15T13:59:02.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.610+0000 7ff81d904700 1 -- start start 2024-05-15T13:59:02.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.610+0000 7ff81d904700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff81806b220 0x7ff818102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:02.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.610+0000 7ff81d904700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff81806bbe0 0x7ff818102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:02.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.610+0000 7ff81d904700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8181095b0 0x7ff818102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:02.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.611+0000 7ff81d904700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff8181136e0 con 0x7ff8181095b0 2024-05-15T13:59:02.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.611+0000 7ff81d904700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff818113560 con 0x7ff81806bbe0 2024-05-15T13:59:02.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.611+0000 7ff81d904700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ff818113860 con 0x7ff81806b220 2024-05-15T13:59:02.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.611+0000 7ff8177fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8181095b0 0x7ff818102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:02.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.611+0000 7ff8177fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8181095b0 0x7ff818102f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51432/0 (socket says 172.21.15.33:51432) 2024-05-15T13:59:02.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.611+0000 7ff8177fe700 1 -- 172.21.15.33:0/2906638188 learned_addr learned my addr 172.21.15.33:0/2906638188 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:02.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.611+0000 7ff8167fc700 1 --2- 172.21.15.33:0/2906638188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff81806bbe0 0x7ff818102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:02.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.611+0000 7ff816ffd700 1 --2- 172.21.15.33:0/2906638188 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff81806b220 0x7ff818102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:02.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.611+0000 7ff8177fe700 1 -- 172.21.15.33:0/2906638188 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff81806b220 msgr2=0x7ff818102510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:02.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.611+0000 7ff8177fe700 1 --2- 172.21.15.33:0/2906638188 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff81806b220 0x7ff818102510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:02.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.611+0000 7ff8177fe700 1 -- 172.21.15.33:0/2906638188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff81806bbe0 msgr2=0x7ff818102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:02.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.611+0000 7ff8177fe700 1 --2- 172.21.15.33:0/2906638188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff81806bbe0 0x7ff818102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:02.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.611+0000 7ff8177fe700 1 -- 172.21.15.33:0/2906638188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8181a8140 con 0x7ff8181095b0 2024-05-15T13:59:02.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.611+0000 7ff816ffd700 1 --2- 172.21.15.33:0/2906638188 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff81806b220 0x7ff818102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T13:59:02.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.611+0000 7ff8167fc700 1 --2- 172.21.15.33:0/2906638188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff81806bbe0 0x7ff818102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T13:59:02.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.611+0000 7ff8177fe700 1 --2- 172.21.15.33:0/2906638188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8181095b0 0x7ff818102f90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff81000ef30 tx=0x7ff81000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:02.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.611+0000 7ff81c902700 1 -- 172.21.15.33:0/2906638188 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8100092e0 con 0x7ff8181095b0 2024-05-15T13:59:02.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.611+0000 7ff81c902700 1 -- 172.21.15.33:0/2906638188 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff810004bb0 con 0x7ff8181095b0 2024-05-15T13:59:02.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.611+0000 7ff81d904700 1 -- 172.21.15.33:0/2906638188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8181a83d0 con 0x7ff8181095b0 2024-05-15T13:59:02.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.612+0000 7ff81c902700 1 -- 172.21.15.33:0/2906638188 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff810004280 con 0x7ff8181095b0 2024-05-15T13:59:02.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.612+0000 7ff81d904700 1 -- 172.21.15.33:0/2906638188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8181a88e0 con 0x7ff8181095b0 2024-05-15T13:59:02.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.612+0000 7ff81d904700 1 -- 172.21.15.33:0/2906638188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff81810ba90 con 0x7ff8181095b0 2024-05-15T13:59:02.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.614+0000 7ff81c902700 1 -- 172.21.15.33:0/2906638188 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff81001b030 con 0x7ff8181095b0 2024-05-15T13:59:02.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.615+0000 7ff81c902700 1 --2- 172.21.15.33:0/2906638188 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff8000b2990 0x7ff8000b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:02.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.615+0000 7ff816ffd700 1 --2- 172.21.15.33:0/2906638188 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff8000b2990 0x7ff8000b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:02.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.615+0000 7ff81c902700 1 -- 172.21.15.33:0/2906638188 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7ff8100e2840 con 0x7ff8181095b0 2024-05-15T13:59:02.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.616+0000 7ff816ffd700 1 --2- 172.21.15.33:0/2906638188 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff8000b2990 0x7ff8000b4e50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff808005e10 tx=0x7ff808005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:02.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.619+0000 7ff81c902700 1 -- 172.21.15.33:0/2906638188 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff810022020 con 0x7ff8181095b0 2024-05-15T13:59:02.640 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:59:02.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.841+0000 7ff81d904700 1 -- 172.21.15.33:0/2906638188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7ff818104c70 con 0x7ff8181095b0 2024-05-15T13:59:02.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.841+0000 7ff81c902700 1 -- 172.21.15.33:0/2906638188 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff818104c70 con 0x7ff8181095b0 2024-05-15T13:59:02.843 INFO:teuthology.orchestra.run.smithi033.stdout:261993005101 2024-05-15T13:59:02.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.844+0000 7ff81d904700 1 -- 172.21.15.33:0/2906638188 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff8000b2990 msgr2=0x7ff8000b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:02.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.844+0000 7ff81d904700 1 --2- 172.21.15.33:0/2906638188 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff8000b2990 0x7ff8000b4e50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff808005e10 tx=0x7ff808005d80 comp rx=0 tx=0).stop 2024-05-15T13:59:02.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.844+0000 7ff81d904700 1 -- 172.21.15.33:0/2906638188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8181095b0 msgr2=0x7ff818102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:02.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.844+0000 7ff81d904700 1 --2- 172.21.15.33:0/2906638188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8181095b0 0x7ff818102f90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff81000ef30 tx=0x7ff81000c3a0 comp rx=0 tx=0).stop 2024-05-15T13:59:02.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.844+0000 7ff81d904700 1 -- 172.21.15.33:0/2906638188 shutdown_connections 2024-05-15T13:59:02.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.844+0000 7ff81d904700 1 --2- 172.21.15.33:0/2906638188 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff8000b2990 0x7ff8000b4e50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:02.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.844+0000 7ff81d904700 1 --2- 172.21.15.33:0/2906638188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8181095b0 0x7ff818102f90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:02.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.844+0000 7ff81d904700 1 --2- 172.21.15.33:0/2906638188 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff81806b220 0x7ff818102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:02.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.844+0000 7ff81d904700 1 --2- 172.21.15.33:0/2906638188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff81806bbe0 0x7ff818102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:02.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.844+0000 7ff81d904700 1 -- 172.21.15.33:0/2906638188 >> 172.21.15.33:0/2906638188 conn(0x7ff8180793f0 msgr2=0x7ff8181011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:02.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.844+0000 7ff81d904700 1 -- 172.21.15.33:0/2906638188 shutdown_connections 2024-05-15T13:59:02.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:02.844+0000 7ff81d904700 1 -- 172.21.15.33:0/2906638188 wait complete. 2024-05-15T13:59:03.101 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411402 got 77309411408 for osd.2 2024-05-15T13:59:03.101 DEBUG:teuthology.parallel:result is None 2024-05-15T13:59:03.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:03 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/2906638188' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-15T13:59:03.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.578+0000 7f63c51d2700 1 -- 172.21.15.33:0/2522282825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63c006bbe0 msgr2=0x7f63c01103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:03.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.578+0000 7f63c51d2700 1 --2- 172.21.15.33:0/2522282825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63c006bbe0 0x7f63c01103d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f63b800b0a0 tx=0x7f63b800b3b0 comp rx=0 tx=0).stop 2024-05-15T13:59:03.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.579+0000 7f63c51d2700 1 -- 172.21.15.33:0/2522282825 shutdown_connections 2024-05-15T13:59:03.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.579+0000 7f63c51d2700 1 --2- 172.21.15.33:0/2522282825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63c006bbe0 0x7f63c01103d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:03.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.579+0000 7f63c51d2700 1 --2- 172.21.15.33:0/2522282825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63c01095b0 0x7f63c0109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:03.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.579+0000 7f63c51d2700 1 --2- 172.21.15.33:0/2522282825 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63c006b220 0x7f63c006b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:03.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.579+0000 7f63c51d2700 1 -- 172.21.15.33:0/2522282825 >> 172.21.15.33:0/2522282825 conn(0x7f63c00793f0 msgr2=0x7f63c0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:03.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.579+0000 7f63c51d2700 1 -- 172.21.15.33:0/2522282825 shutdown_connections 2024-05-15T13:59:03.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.579+0000 7f63c51d2700 1 -- 172.21.15.33:0/2522282825 wait complete. 2024-05-15T13:59:03.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.579+0000 7f63c51d2700 1 Processor -- start 2024-05-15T13:59:03.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.580+0000 7f63c51d2700 1 -- start start 2024-05-15T13:59:03.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.580+0000 7f63c51d2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63c006b220 0x7f63c019f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:03.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.580+0000 7f63c51d2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63c006bbe0 0x7f63c019fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:03.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.580+0000 7f63c51d2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63c01095b0 0x7f63c0199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:03.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.580+0000 7f63c51d2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f63c01136d0 con 0x7f63c006b220 2024-05-15T13:59:03.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.580+0000 7f63c51d2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f63c0113550 con 0x7f63c01095b0 2024-05-15T13:59:03.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.580+0000 7f63c51d2700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f63c0113850 con 0x7f63c006bbe0 2024-05-15T13:59:03.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.580+0000 7f63bed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63c006b220 0x7f63c019f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:03.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.580+0000 7f63bed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63c006b220 0x7f63c019f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48966/0 (socket says 172.21.15.33:48966) 2024-05-15T13:59:03.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.580+0000 7f63bed9d700 1 -- 172.21.15.33:0/204162115 learned_addr learned my addr 172.21.15.33:0/204162115 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:03.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.580+0000 7f63bf59e700 1 --2- 172.21.15.33:0/204162115 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63c01095b0 0x7f63c0199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:03.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.580+0000 7f63be59c700 1 --2- 172.21.15.33:0/204162115 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63c006bbe0 0x7f63c019fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:03.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.580+0000 7f63bed9d700 1 -- 172.21.15.33:0/204162115 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63c006bbe0 msgr2=0x7f63c019fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:03.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.580+0000 7f63bed9d700 1 --2- 172.21.15.33:0/204162115 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63c006bbe0 0x7f63c019fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:03.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.580+0000 7f63bed9d700 1 -- 172.21.15.33:0/204162115 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63c01095b0 msgr2=0x7f63c0199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:03.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.580+0000 7f63bed9d700 1 --2- 172.21.15.33:0/204162115 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63c01095b0 0x7f63c0199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:03.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.581+0000 7f63bed9d700 1 -- 172.21.15.33:0/204162115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c019a190 con 0x7f63c006b220 2024-05-15T13:59:03.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.581+0000 7f63bed9d700 1 --2- 172.21.15.33:0/204162115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63c006b220 0x7f63c019f870 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f63b000d950 tx=0x7f63b000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:03.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.581+0000 7f63a7fff700 1 -- 172.21.15.33:0/204162115 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b0016070 con 0x7f63c006b220 2024-05-15T13:59:03.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.581+0000 7f63a7fff700 1 -- 172.21.15.33:0/204162115 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63b000fd40 con 0x7f63c006b220 2024-05-15T13:59:03.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.581+0000 7f63c51d2700 1 -- 172.21.15.33:0/204162115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63c019a480 con 0x7f63c006b220 2024-05-15T13:59:03.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.581+0000 7f63a7fff700 1 -- 172.21.15.33:0/204162115 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b000e3b0 con 0x7f63c006b220 2024-05-15T13:59:03.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.581+0000 7f63c51d2700 1 -- 172.21.15.33:0/204162115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63c01a6740 con 0x7f63c006b220 2024-05-15T13:59:03.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.582+0000 7f63c51d2700 1 -- 172.21.15.33:0/204162115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63c006a230 con 0x7f63c006b220 2024-05-15T13:59:03.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.583+0000 7f63a7fff700 1 -- 172.21.15.33:0/204162115 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f63b0019070 con 0x7f63c006b220 2024-05-15T13:59:03.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.584+0000 7f63a7fff700 1 --2- 172.21.15.33:0/204162115 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f63a80b2910 0x7f63a80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:03.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.584+0000 7f63be59c700 1 --2- 172.21.15.33:0/204162115 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f63a80b2910 0x7f63a80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:03.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.584+0000 7f63a7fff700 1 -- 172.21.15.33:0/204162115 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f63b00e1e00 con 0x7f63c006b220 2024-05-15T13:59:03.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.585+0000 7f63be59c700 1 --2- 172.21.15.33:0/204162115 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f63a80b2910 0x7f63a80b4dd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f63c0079aa0 tx=0x7f63b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:03.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.589+0000 7f63a7fff700 1 -- 172.21.15.33:0/204162115 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f63b00a99c0 con 0x7f63c006b220 2024-05-15T13:59:03.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:03 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/2906638188' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-15T13:59:03.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:03 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/2906638188' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-15T13:59:03.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.812+0000 7f63c51d2700 1 -- 172.21.15.33:0/204162115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f63c002e560 con 0x7f63c006b220 2024-05-15T13:59:03.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.812+0000 7f63a7fff700 1 -- 172.21.15.33:0/204162115 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f63b00a92c0 con 0x7f63c006b220 2024-05-15T13:59:03.814 INFO:teuthology.orchestra.run.smithi033.stdout:137438953544 2024-05-15T13:59:03.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.815+0000 7f63c51d2700 1 -- 172.21.15.33:0/204162115 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f63a80b2910 msgr2=0x7f63a80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:03.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.815+0000 7f63c51d2700 1 --2- 172.21.15.33:0/204162115 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f63a80b2910 0x7f63a80b4dd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f63c0079aa0 tx=0x7f63b4009260 comp rx=0 tx=0).stop 2024-05-15T13:59:03.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.815+0000 7f63c51d2700 1 -- 172.21.15.33:0/204162115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63c006b220 msgr2=0x7f63c019f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:03.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.815+0000 7f63c51d2700 1 --2- 172.21.15.33:0/204162115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63c006b220 0x7f63c019f870 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f63b000d950 tx=0x7f63b000dc60 comp rx=0 tx=0).stop 2024-05-15T13:59:03.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.815+0000 7f63c51d2700 1 -- 172.21.15.33:0/204162115 shutdown_connections 2024-05-15T13:59:03.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.815+0000 7f63c51d2700 1 --2- 172.21.15.33:0/204162115 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f63a80b2910 0x7f63a80b4dd0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:03.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.815+0000 7f63c51d2700 1 --2- 172.21.15.33:0/204162115 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63c01095b0 0x7f63c0199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:03.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.815+0000 7f63c51d2700 1 --2- 172.21.15.33:0/204162115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63c006b220 0x7f63c019f870 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:03.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.815+0000 7f63c51d2700 1 --2- 172.21.15.33:0/204162115 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63c006bbe0 0x7f63c019fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:03.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.815+0000 7f63c51d2700 1 -- 172.21.15.33:0/204162115 >> 172.21.15.33:0/204162115 conn(0x7f63c00793f0 msgr2=0x7f63c0101140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:03.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.815+0000 7f63c51d2700 1 -- 172.21.15.33:0/204162115 shutdown_connections 2024-05-15T13:59:03.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.815+0000 7f63c51d2700 1 -- 172.21.15.33:0/204162115 wait complete. 2024-05-15T13:59:03.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.288+0000 7fbeec5ba700 1 -- 172.21.15.33:0/1497053650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbee406bc10 msgr2=0x7fbee406c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:03.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.288+0000 7fbeec5ba700 1 --2- 172.21.15.33:0/1497053650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbee406bc10 0x7fbee406c090 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbed4009990 tx=0x7fbed4009ca0 comp rx=0 tx=0).stop 2024-05-15T13:59:03.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.288+0000 7fbeec5ba700 1 -- 172.21.15.33:0/1497053650 shutdown_connections 2024-05-15T13:59:03.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.288+0000 7fbeec5ba700 1 --2- 172.21.15.33:0/1497053650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbee406c5d0 0x7fbee41103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:03.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.288+0000 7fbeec5ba700 1 --2- 172.21.15.33:0/1497053650 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbee41095c0 0x7fbee41099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:03.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.288+0000 7fbeec5ba700 1 --2- 172.21.15.33:0/1497053650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbee406bc10 0x7fbee406c090 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:03.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.288+0000 7fbeec5ba700 1 -- 172.21.15.33:0/1497053650 >> 172.21.15.33:0/1497053650 conn(0x7fbee40793f0 msgr2=0x7fbee4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:03.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.289+0000 7fbeec5ba700 1 -- 172.21.15.33:0/1497053650 shutdown_connections 2024-05-15T13:59:03.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.289+0000 7fbeec5ba700 1 -- 172.21.15.33:0/1497053650 wait complete. 2024-05-15T13:59:03.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.289+0000 7fbeec5ba700 1 Processor -- start 2024-05-15T13:59:03.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.289+0000 7fbeec5ba700 1 -- start start 2024-05-15T13:59:03.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.290+0000 7fbeec5ba700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbee406bc10 0x7fbee41028f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:03.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.290+0000 7fbeec5ba700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbee406c5d0 0x7fbee4102e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:03.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.290+0000 7fbeec5ba700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbee41095c0 0x7fbee4103370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:03.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.290+0000 7fbeec5ba700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbee41136f0 con 0x7fbee406c5d0 2024-05-15T13:59:03.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.290+0000 7fbeec5ba700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbee4113570 con 0x7fbee41095c0 2024-05-15T13:59:03.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.290+0000 7fbeec5ba700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbee4113870 con 0x7fbee406bc10 2024-05-15T13:59:03.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.290+0000 7fbee9b55700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbee406c5d0 0x7fbee4102e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:03.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.290+0000 7fbee9b55700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbee406c5d0 0x7fbee4102e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48936/0 (socket says 172.21.15.33:48936) 2024-05-15T13:59:03.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.290+0000 7fbee9b55700 1 -- 172.21.15.33:0/4285190412 learned_addr learned my addr 172.21.15.33:0/4285190412 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:03.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.290+0000 7fbeea356700 1 --2- 172.21.15.33:0/4285190412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbee406bc10 0x7fbee41028f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:03.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.290+0000 7fbeeab57700 1 --2- 172.21.15.33:0/4285190412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbee41095c0 0x7fbee4103370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:03.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.290+0000 7fbee9b55700 1 -- 172.21.15.33:0/4285190412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbee406bc10 msgr2=0x7fbee41028f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:03.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.290+0000 7fbee9b55700 1 --2- 172.21.15.33:0/4285190412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbee406bc10 0x7fbee41028f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:03.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.290+0000 7fbee9b55700 1 -- 172.21.15.33:0/4285190412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbee41095c0 msgr2=0x7fbee4103370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:03.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.290+0000 7fbee9b55700 1 --2- 172.21.15.33:0/4285190412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbee41095c0 0x7fbee4103370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:03.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.290+0000 7fbee9b55700 1 -- 172.21.15.33:0/4285190412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee41a8190 con 0x7fbee406c5d0 2024-05-15T13:59:03.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.290+0000 7fbeea356700 1 --2- 172.21.15.33:0/4285190412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbee406bc10 0x7fbee41028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T13:59:03.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.290+0000 7fbeeab57700 1 --2- 172.21.15.33:0/4285190412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbee41095c0 0x7fbee4103370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T13:59:03.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.290+0000 7fbee9b55700 1 --2- 172.21.15.33:0/4285190412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbee406c5d0 0x7fbee4102e30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbed40057f0 tx=0x7fbed4003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:03.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.290+0000 7fbedb7fe700 1 -- 172.21.15.33:0/4285190412 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4027070 con 0x7fbee406c5d0 2024-05-15T13:59:03.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.291+0000 7fbedb7fe700 1 -- 172.21.15.33:0/4285190412 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbed40041c0 con 0x7fbee406c5d0 2024-05-15T13:59:03.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.291+0000 7fbedb7fe700 1 -- 172.21.15.33:0/4285190412 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4022e60 con 0x7fbee406c5d0 2024-05-15T13:59:03.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.291+0000 7fbeec5ba700 1 -- 172.21.15.33:0/4285190412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee41a8420 con 0x7fbee406c5d0 2024-05-15T13:59:03.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.291+0000 7fbeec5ba700 1 -- 172.21.15.33:0/4285190412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee41a8950 con 0x7fbee406c5d0 2024-05-15T13:59:03.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.292+0000 7fbeec5ba700 1 -- 172.21.15.33:0/4285190412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbee410baa0 con 0x7fbee406c5d0 2024-05-15T13:59:03.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.293+0000 7fbedb7fe700 1 -- 172.21.15.33:0/4285190412 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fbed402a070 con 0x7fbee406c5d0 2024-05-15T13:59:03.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.294+0000 7fbedb7fe700 1 --2- 172.21.15.33:0/4285190412 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbed00b2990 0x7fbed00b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:03.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.294+0000 7fbeea356700 1 --2- 172.21.15.33:0/4285190412 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbed00b2990 0x7fbed00b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:03.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.294+0000 7fbedb7fe700 1 -- 172.21.15.33:0/4285190412 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fbed40e3630 con 0x7fbee406c5d0 2024-05-15T13:59:03.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.294+0000 7fbeea356700 1 --2- 172.21.15.33:0/4285190412 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbed00b2990 0x7fbed00b4e50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbee00076e0 tx=0x7fbee0007670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:03.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.297+0000 7fbedb7fe700 1 -- 172.21.15.33:0/4285190412 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbed4013070 con 0x7fbee406c5d0 2024-05-15T13:59:03.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.513+0000 7fbeec5ba700 1 -- 172.21.15.33:0/4285190412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fbee410b480 con 0x7fbee406c5d0 2024-05-15T13:59:03.834 INFO:teuthology.orchestra.run.smithi033.stdout:283467841577 2024-05-15T13:59:03.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.514+0000 7fbedb7fe700 1 -- 172.21.15.33:0/4285190412 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fbed40ab280 con 0x7fbee406c5d0 2024-05-15T13:59:03.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.516+0000 7fbeec5ba700 1 -- 172.21.15.33:0/4285190412 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbed00b2990 msgr2=0x7fbed00b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:03.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.516+0000 7fbeec5ba700 1 --2- 172.21.15.33:0/4285190412 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbed00b2990 0x7fbed00b4e50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbee00076e0 tx=0x7fbee0007670 comp rx=0 tx=0).stop 2024-05-15T13:59:03.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.516+0000 7fbeec5ba700 1 -- 172.21.15.33:0/4285190412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbee406c5d0 msgr2=0x7fbee4102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:03.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.516+0000 7fbeec5ba700 1 --2- 172.21.15.33:0/4285190412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbee406c5d0 0x7fbee4102e30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbed40057f0 tx=0x7fbed4003730 comp rx=0 tx=0).stop 2024-05-15T13:59:03.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.516+0000 7fbeec5ba700 1 -- 172.21.15.33:0/4285190412 shutdown_connections 2024-05-15T13:59:03.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.516+0000 7fbeec5ba700 1 --2- 172.21.15.33:0/4285190412 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbed00b2990 0x7fbed00b4e50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:03.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.516+0000 7fbeec5ba700 1 --2- 172.21.15.33:0/4285190412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbee41095c0 0x7fbee4103370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:03.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.516+0000 7fbeec5ba700 1 --2- 172.21.15.33:0/4285190412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbee406bc10 0x7fbee41028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:03.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.516+0000 7fbeec5ba700 1 --2- 172.21.15.33:0/4285190412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbee406c5d0 0x7fbee4102e30 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:03.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.516+0000 7fbeec5ba700 1 -- 172.21.15.33:0/4285190412 >> 172.21.15.33:0/4285190412 conn(0x7fbee40793f0 msgr2=0x7fbee4101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:03.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.516+0000 7fbeec5ba700 1 -- 172.21.15.33:0/4285190412 shutdown_connections 2024-05-15T13:59:03.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:03.516+0000 7fbeec5ba700 1 -- 172.21.15.33:0/4285190412 wait complete. 2024-05-15T13:59:04.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:04 smithi033 ceph-mon[31650]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-15T13:59:04.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:04 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/4285190412' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-15T13:59:04.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:04 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/204162115' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-15T13:59:04.540 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953537 got 137438953544 for osd.4 2024-05-15T13:59:04.540 DEBUG:teuthology.parallel:result is None 2024-05-15T13:59:04.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:04 smithi052 ceph-mon[36190]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-15T13:59:04.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:04 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/4285190412' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-15T13:59:04.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:04 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/204162115' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-15T13:59:04.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:04 smithi081 ceph-mon[36434]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-15T13:59:04.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:04 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/4285190412' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-15T13:59:04.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:04 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/204162115' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-15T13:59:05.111 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841571 got 283467841577 for osd.10 2024-05-15T13:59:05.112 DEBUG:teuthology.parallel:result is None 2024-05-15T13:59:05.653 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005095 got 261993005101 for osd.9 2024-05-15T13:59:05.653 DEBUG:teuthology.parallel:result is None 2024-05-15T13:59:05.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.282+0000 7fb03da6b700 1 -- 172.21.15.33:0/2394184996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb038101b30 msgr2=0x7fb038101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:05.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.282+0000 7fb03da6b700 1 --2- 172.21.15.33:0/2394184996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb038101b30 0x7fb038101fb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb02c0099c0 tx=0x7fb02c009cd0 comp rx=0 tx=0).stop 2024-05-15T13:59:05.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.282+0000 7fb03da6b700 1 -- 172.21.15.33:0/2394184996 shutdown_connections 2024-05-15T13:59:05.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.282+0000 7fb03da6b700 1 --2- 172.21.15.33:0/2394184996 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0381024f0 0x7fb0381103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:05.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.282+0000 7fb03da6b700 1 --2- 172.21.15.33:0/2394184996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb038109520 0x7fb038109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:05.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.282+0000 7fb03da6b700 1 --2- 172.21.15.33:0/2394184996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb038101b30 0x7fb038101fb0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:05.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.282+0000 7fb03da6b700 1 -- 172.21.15.33:0/2394184996 >> 172.21.15.33:0/2394184996 conn(0x7fb038078ab0 msgr2=0x7fb0380ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:05.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.282+0000 7fb03da6b700 1 -- 172.21.15.33:0/2394184996 shutdown_connections 2024-05-15T13:59:05.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.282+0000 7fb03da6b700 1 -- 172.21.15.33:0/2394184996 wait complete. 2024-05-15T13:59:05.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.283+0000 7fb03da6b700 1 Processor -- start 2024-05-15T13:59:05.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.283+0000 7fb03da6b700 1 -- start start 2024-05-15T13:59:05.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.283+0000 7fb03da6b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb038101b30 0x7fb03819fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:05.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.283+0000 7fb03da6b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb0381024f0 0x7fb03819ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:05.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.283+0000 7fb03da6b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb038109520 0x7fb038199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:05.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.283+0000 7fb03da6b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb0381137b0 con 0x7fb038109520 2024-05-15T13:59:05.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.283+0000 7fb03da6b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb038113630 con 0x7fb0381024f0 2024-05-15T13:59:05.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.283+0000 7fb03da6b700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb038113930 con 0x7fb038101b30 2024-05-15T13:59:05.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.283+0000 7fb037fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb038109520 0x7fb038199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:05.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.283+0000 7fb037fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb038109520 0x7fb038199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48990/0 (socket says 172.21.15.33:48990) 2024-05-15T13:59:05.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.283+0000 7fb037fff700 1 -- 172.21.15.33:0/3835156039 learned_addr learned my addr 172.21.15.33:0/3835156039 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:05.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.283+0000 7fb0377fe700 1 --2- 172.21.15.33:0/3835156039 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb038101b30 0x7fb03819fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:05.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.284+0000 7fb036ffd700 1 --2- 172.21.15.33:0/3835156039 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb0381024f0 0x7fb03819ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:05.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.284+0000 7fb037fff700 1 -- 172.21.15.33:0/3835156039 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb038101b30 msgr2=0x7fb03819fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:05.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.284+0000 7fb037fff700 1 --2- 172.21.15.33:0/3835156039 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb038101b30 0x7fb03819fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:05.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.284+0000 7fb037fff700 1 -- 172.21.15.33:0/3835156039 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb0381024f0 msgr2=0x7fb03819ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:05.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.284+0000 7fb037fff700 1 --2- 172.21.15.33:0/3835156039 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb0381024f0 0x7fb03819ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:05.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.284+0000 7fb037fff700 1 -- 172.21.15.33:0/3835156039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb03819a3f0 con 0x7fb038109520 2024-05-15T13:59:05.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.284+0000 7fb037fff700 1 --2- 172.21.15.33:0/3835156039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb038109520 0x7fb038199b10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb03000b550 tx=0x7fb03000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:05.789 INFO:teuthology.orchestra.run.smithi033.stdout:111669149772 2024-05-15T13:59:05.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.284+0000 7fb034ff9700 1 -- 172.21.15.33:0/3835156039 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb030015020 con 0x7fb038109520 2024-05-15T13:59:05.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.284+0000 7fb034ff9700 1 -- 172.21.15.33:0/3835156039 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb030011ce0 con 0x7fb038109520 2024-05-15T13:59:05.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.284+0000 7fb03da6b700 1 -- 172.21.15.33:0/3835156039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb03819a6e0 con 0x7fb038109520 2024-05-15T13:59:05.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.284+0000 7fb034ff9700 1 -- 172.21.15.33:0/3835156039 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb030010d40 con 0x7fb038109520 2024-05-15T13:59:05.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.284+0000 7fb03da6b700 1 -- 172.21.15.33:0/3835156039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0381a69c0 con 0x7fb038109520 2024-05-15T13:59:05.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.285+0000 7fb03da6b700 1 -- 172.21.15.33:0/3835156039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb038104e50 con 0x7fb038109520 2024-05-15T13:59:05.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.286+0000 7fb034ff9700 1 -- 172.21.15.33:0/3835156039 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb030018020 con 0x7fb038109520 2024-05-15T13:59:05.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.287+0000 7fb034ff9700 1 --2- 172.21.15.33:0/3835156039 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fb0200b2880 0x7fb0200b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:05.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.287+0000 7fb0377fe700 1 --2- 172.21.15.33:0/3835156039 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fb0200b2880 0x7fb0200b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:05.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.288+0000 7fb034ff9700 1 -- 172.21.15.33:0/3835156039 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fb0300e1a40 con 0x7fb038109520 2024-05-15T13:59:05.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.288+0000 7fb0377fe700 1 --2- 172.21.15.33:0/3835156039 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fb0200b2880 0x7fb0200b4d40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb02800e270 tx=0x7fb028009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:05.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.292+0000 7fb034ff9700 1 -- 172.21.15.33:0/3835156039 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb03001f020 con 0x7fb038109520 2024-05-15T13:59:05.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.509+0000 7fb03da6b700 1 -- 172.21.15.33:0/3835156039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fb03819afd0 con 0x7fb038109520 2024-05-15T13:59:05.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.509+0000 7fb034ff9700 1 -- 172.21.15.33:0/3835156039 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb0300a9720 con 0x7fb038109520 2024-05-15T13:59:05.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.511+0000 7fb03da6b700 1 -- 172.21.15.33:0/3835156039 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fb0200b2880 msgr2=0x7fb0200b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:05.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.511+0000 7fb03da6b700 1 --2- 172.21.15.33:0/3835156039 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fb0200b2880 0x7fb0200b4d40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb02800e270 tx=0x7fb028009040 comp rx=0 tx=0).stop 2024-05-15T13:59:05.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.511+0000 7fb03da6b700 1 -- 172.21.15.33:0/3835156039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb038109520 msgr2=0x7fb038199b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:05.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.511+0000 7fb03da6b700 1 --2- 172.21.15.33:0/3835156039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb038109520 0x7fb038199b10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb03000b550 tx=0x7fb03000b910 comp rx=0 tx=0).stop 2024-05-15T13:59:05.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.511+0000 7fb03da6b700 1 -- 172.21.15.33:0/3835156039 shutdown_connections 2024-05-15T13:59:05.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.511+0000 7fb03da6b700 1 --2- 172.21.15.33:0/3835156039 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fb0200b2880 0x7fb0200b4d40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:05.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.511+0000 7fb03da6b700 1 --2- 172.21.15.33:0/3835156039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb038109520 0x7fb038199b10 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:05.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.511+0000 7fb03da6b700 1 --2- 172.21.15.33:0/3835156039 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb038101b30 0x7fb03819fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:05.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.511+0000 7fb03da6b700 1 --2- 172.21.15.33:0/3835156039 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb0381024f0 0x7fb03819ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:05.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.511+0000 7fb03da6b700 1 -- 172.21.15.33:0/3835156039 >> 172.21.15.33:0/3835156039 conn(0x7fb038078ab0 msgr2=0x7fb038100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:05.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.511+0000 7fb03da6b700 1 -- 172.21.15.33:0/3835156039 shutdown_connections 2024-05-15T13:59:05.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:05.512+0000 7fb03da6b700 1 -- 172.21.15.33:0/3835156039 wait complete. 2024-05-15T13:59:06.332 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:06 smithi033 ceph-mon[31650]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:59:06.333 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:06 smithi033 ceph-mon[31650]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-15T13:59:06.333 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:06 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/3835156039' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-15T13:59:06.361 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149765 got 111669149772 for osd.3 2024-05-15T13:59:06.361 DEBUG:teuthology.parallel:result is None 2024-05-15T13:59:06.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:06 smithi052 ceph-mon[36190]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:59:06.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:06 smithi052 ceph-mon[36190]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-15T13:59:06.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:06 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/3835156039' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-15T13:59:06.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:06 smithi081 ceph-mon[36434]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:59:06.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:06 smithi081 ceph-mon[36434]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-15T13:59:06.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:06 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/3835156039' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-15T13:59:08.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.027+0000 7f272115f700 1 -- 172.21.15.33:0/790921783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f271c102e30 msgr2=0x7f271c10f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:08.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.027+0000 7f272115f700 1 --2- 172.21.15.33:0/790921783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f271c102e30 0x7f271c10f2f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f271400b1a0 tx=0x7f271400b4b0 comp rx=0 tx=0).stop 2024-05-15T13:59:08.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.027+0000 7f272115f700 1 -- 172.21.15.33:0/790921783 shutdown_connections 2024-05-15T13:59:08.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.027+0000 7f272115f700 1 --2- 172.21.15.33:0/790921783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f271c102e30 0x7f271c10f2f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:08.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.027+0000 7f272115f700 1 --2- 172.21.15.33:0/790921783 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f271c1084b0 0x7f271c108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:08.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.027+0000 7f272115f700 1 --2- 172.21.15.33:0/790921783 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f271c102470 0x7f271c1028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:08.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.027+0000 7f272115f700 1 -- 172.21.15.33:0/790921783 >> 172.21.15.33:0/790921783 conn(0x7f271c0793f0 msgr2=0x7f271c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:08.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.027+0000 7f272115f700 1 -- 172.21.15.33:0/790921783 shutdown_connections 2024-05-15T13:59:08.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.027+0000 7f272115f700 1 -- 172.21.15.33:0/790921783 wait complete. 2024-05-15T13:59:08.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.028+0000 7f272115f700 1 Processor -- start 2024-05-15T13:59:08.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.028+0000 7f272115f700 1 -- start start 2024-05-15T13:59:08.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.028+0000 7f272115f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f271c102470 0x7f271c19bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:08.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.028+0000 7f272115f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f271c102e30 0x7f271c19c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:08.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.028+0000 7f272115f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f271c1084b0 0x7f271c1a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:08.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.028+0000 7f272115f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f271c111570 con 0x7f271c102e30 2024-05-15T13:59:08.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.028+0000 7f272115f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f271c1113f0 con 0x7f271c1084b0 2024-05-15T13:59:08.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.028+0000 7f272115f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f271c1116f0 con 0x7f271c102470 2024-05-15T13:59:08.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.029+0000 7f271ad9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f271c102470 0x7f271c19bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:08.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.029+0000 7f271ad9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f271c102470 0x7f271c19bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:37464/0 (socket says 172.21.15.33:37464) 2024-05-15T13:59:08.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.029+0000 7f271ad9d700 1 -- 172.21.15.33:0/1877624956 learned_addr learned my addr 172.21.15.33:0/1877624956 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:08.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.029+0000 7f271b59e700 1 --2- 172.21.15.33:0/1877624956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f271c1084b0 0x7f271c1a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:08.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.029+0000 7f271a59c700 1 --2- 172.21.15.33:0/1877624956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f271c102e30 0x7f271c19c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:08.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.029+0000 7f271ad9d700 1 -- 172.21.15.33:0/1877624956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f271c1084b0 msgr2=0x7f271c1a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:08.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.029+0000 7f271ad9d700 1 --2- 172.21.15.33:0/1877624956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f271c1084b0 0x7f271c1a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:08.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.029+0000 7f271ad9d700 1 -- 172.21.15.33:0/1877624956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f271c102e30 msgr2=0x7f271c19c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:08.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.029+0000 7f271ad9d700 1 --2- 172.21.15.33:0/1877624956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f271c102e30 0x7f271c19c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:08.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.029+0000 7f271ad9d700 1 -- 172.21.15.33:0/1877624956 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f271c1a0d70 con 0x7f271c102470 2024-05-15T13:59:08.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.029+0000 7f271b59e700 1 --2- 172.21.15.33:0/1877624956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f271c1084b0 0x7f271c1a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T13:59:08.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.029+0000 7f271ad9d700 1 --2- 172.21.15.33:0/1877624956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f271c102470 0x7f271c19bdb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f270c00d980 tx=0x7f270c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:08.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.029+0000 7f2703fff700 1 -- 172.21.15.33:0/1877624956 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c017070 con 0x7f271c102470 2024-05-15T13:59:08.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.029+0000 7f2703fff700 1 -- 172.21.15.33:0/1877624956 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f270c005020 con 0x7f271c102470 2024-05-15T13:59:08.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.029+0000 7f272115f700 1 -- 172.21.15.33:0/1877624956 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f271c1a1060 con 0x7f271c102470 2024-05-15T13:59:08.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.030+0000 7f2703fff700 1 -- 172.21.15.33:0/1877624956 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c00fa10 con 0x7f271c102470 2024-05-15T13:59:08.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.030+0000 7f272115f700 1 -- 172.21.15.33:0/1877624956 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f271c1a88e0 con 0x7f271c102470 2024-05-15T13:59:08.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.031+0000 7f272115f700 1 -- 172.21.15.33:0/1877624956 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f271c06a230 con 0x7f271c102470 2024-05-15T13:59:08.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.032+0000 7f2703fff700 1 -- 172.21.15.33:0/1877624956 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f270c004a30 con 0x7f271c102470 2024-05-15T13:59:08.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.033+0000 7f2703fff700 1 --2- 172.21.15.33:0/1877624956 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f27040b2880 0x7f27040b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:08.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.033+0000 7f271a59c700 1 --2- 172.21.15.33:0/1877624956 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f27040b2880 0x7f27040b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:08.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.033+0000 7f2703fff700 1 -- 172.21.15.33:0/1877624956 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f270c0e2570 con 0x7f271c102470 2024-05-15T13:59:08.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.034+0000 7f271a59c700 1 --2- 172.21.15.33:0/1877624956 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f27040b2880 0x7f27040b4d40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f271c079960 tx=0x7f271000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:08.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.036+0000 7f2703fff700 1 -- 172.21.15.33:0/1877624956 <== mon.1 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f270c020020 con 0x7f271c102470 2024-05-15T13:59:08.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.256+0000 7f272115f700 1 -- 172.21.15.33:0/1877624956 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f271c02e560 con 0x7f271c102470 2024-05-15T13:59:08.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.257+0000 7f2703fff700 1 -- 172.21.15.33:0/1877624956 <== mon.1 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f270c0aa250 con 0x7f271c102470 2024-05-15T13:59:08.259 INFO:teuthology.orchestra.run.smithi033.stdout:55834574934 2024-05-15T13:59:08.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.260+0000 7f272115f700 1 -- 172.21.15.33:0/1877624956 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f27040b2880 msgr2=0x7f27040b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:08.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.260+0000 7f272115f700 1 --2- 172.21.15.33:0/1877624956 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f27040b2880 0x7f27040b4d40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f271c079960 tx=0x7f271000a380 comp rx=0 tx=0).stop 2024-05-15T13:59:08.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.260+0000 7f272115f700 1 -- 172.21.15.33:0/1877624956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f271c102470 msgr2=0x7f271c19bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:08.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.260+0000 7f272115f700 1 --2- 172.21.15.33:0/1877624956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f271c102470 0x7f271c19bdb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f270c00d980 tx=0x7f270c00dc90 comp rx=0 tx=0).stop 2024-05-15T13:59:08.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.260+0000 7f272115f700 1 -- 172.21.15.33:0/1877624956 shutdown_connections 2024-05-15T13:59:08.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.260+0000 7f272115f700 1 --2- 172.21.15.33:0/1877624956 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f27040b2880 0x7f27040b4d40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:08.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.260+0000 7f272115f700 1 --2- 172.21.15.33:0/1877624956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f271c1084b0 0x7f271c1a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:08.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.260+0000 7f272115f700 1 --2- 172.21.15.33:0/1877624956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f271c102470 0x7f271c19bdb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:08.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.260+0000 7f272115f700 1 --2- 172.21.15.33:0/1877624956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f271c102e30 0x7f271c19c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:08.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.260+0000 7f272115f700 1 -- 172.21.15.33:0/1877624956 >> 172.21.15.33:0/1877624956 conn(0x7f271c0793f0 msgr2=0x7f271c100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:08.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.260+0000 7f272115f700 1 -- 172.21.15.33:0/1877624956 shutdown_connections 2024-05-15T13:59:08.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:08.260+0000 7f272115f700 1 -- 172.21.15.33:0/1877624956 wait complete. 2024-05-15T13:59:08.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:08 smithi033 ceph-mon[31650]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-15T13:59:08.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:08 smithi052 ceph-mon[36190]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-15T13:59:08.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:08 smithi081 ceph-mon[36434]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-15T13:59:09.016 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574929 got 55834574934 for osd.1 2024-05-15T13:59:09.016 DEBUG:teuthology.parallel:result is None 2024-05-15T13:59:09.016 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-15T13:59:09.016 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-15T13:59:09.271 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:59:09.395 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:09 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/1877624956' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-15T13:59:09.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:09 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/1877624956' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-15T13:59:09.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:09 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/1877624956' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-15T13:59:10.401 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:10 smithi033 ceph-mon[31650]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-15T13:59:10.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:10 smithi052 ceph-mon[36190]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-15T13:59:10.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:10 smithi081 ceph-mon[36434]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-15T13:59:10.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.873+0000 7f2d90662700 1 -- 172.21.15.33:0/748791853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d88102e00 msgr2=0x7f2d8810f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:10.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.873+0000 7f2d90662700 1 --2- 172.21.15.33:0/748791853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d88102e00 0x7f2d8810f2f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2d8000b0a0 tx=0x7f2d8000b3b0 comp rx=0 tx=0).stop 2024-05-15T13:59:10.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.873+0000 7f2d90662700 1 -- 172.21.15.33:0/748791853 shutdown_connections 2024-05-15T13:59:10.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.873+0000 7f2d90662700 1 --2- 172.21.15.33:0/748791853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d88102e00 0x7f2d8810f2f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:10.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.873+0000 7f2d90662700 1 --2- 172.21.15.33:0/748791853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d88108520 0x7f2d88108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:10.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.873+0000 7f2d90662700 1 --2- 172.21.15.33:0/748791853 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d88102440 0x7f2d881028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:10.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.873+0000 7f2d90662700 1 -- 172.21.15.33:0/748791853 >> 172.21.15.33:0/748791853 conn(0x7f2d880793f0 msgr2=0x7f2d88079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:10.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.873+0000 7f2d90662700 1 -- 172.21.15.33:0/748791853 shutdown_connections 2024-05-15T13:59:10.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.873+0000 7f2d90662700 1 -- 172.21.15.33:0/748791853 wait complete. 2024-05-15T13:59:10.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.873+0000 7f2d90662700 1 Processor -- start 2024-05-15T13:59:10.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.873+0000 7f2d90662700 1 -- start start 2024-05-15T13:59:10.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.874+0000 7f2d90662700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d88102440 0x7f2d8819f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:10.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.874+0000 7f2d90662700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d88102e00 0x7f2d8819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:10.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.874+0000 7f2d90662700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d88108520 0x7f2d88199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:10.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.874+0000 7f2d90662700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2d881125d0 con 0x7f2d88102e00 2024-05-15T13:59:10.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.874+0000 7f2d90662700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2d88112450 con 0x7f2d88102440 2024-05-15T13:59:10.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.874+0000 7f2d90662700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2d88112750 con 0x7f2d88108520 2024-05-15T13:59:10.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.874+0000 7f2d8dbfd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d88102e00 0x7f2d8819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:10.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.874+0000 7f2d8dbfd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d88102e00 0x7f2d8819fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49034/0 (socket says 172.21.15.33:49034) 2024-05-15T13:59:10.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.874+0000 7f2d8dbfd700 1 -- 172.21.15.33:0/2126817086 learned_addr learned my addr 172.21.15.33:0/2126817086 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:10.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.874+0000 7f2d8e3fe700 1 --2- 172.21.15.33:0/2126817086 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d88102440 0x7f2d8819f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:10.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.874+0000 7f2d8ebff700 1 --2- 172.21.15.33:0/2126817086 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d88108520 0x7f2d88199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:10.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.874+0000 7f2d8dbfd700 1 -- 172.21.15.33:0/2126817086 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d88108520 msgr2=0x7f2d88199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.874+0000 7f2d8dbfd700 1 --2- 172.21.15.33:0/2126817086 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d88108520 0x7f2d88199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.874+0000 7f2d8dbfd700 1 -- 172.21.15.33:0/2126817086 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d88102440 msgr2=0x7f2d8819f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.874+0000 7f2d8dbfd700 1 --2- 172.21.15.33:0/2126817086 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d88102440 0x7f2d8819f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.874+0000 7f2d8dbfd700 1 -- 172.21.15.33:0/2126817086 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d8819a170 con 0x7f2d88102e00 2024-05-15T13:59:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.874+0000 7f2d8dbfd700 1 --2- 172.21.15.33:0/2126817086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d88102e00 0x7f2d8819fd40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2d7800d950 tx=0x7f2d7800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.874+0000 7f2d7f7fe700 1 -- 172.21.15.33:0/2126817086 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d78016070 con 0x7f2d88102e00 2024-05-15T13:59:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.875+0000 7f2d90662700 1 -- 172.21.15.33:0/2126817086 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d8819a460 con 0x7f2d88102e00 2024-05-15T13:59:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.875+0000 7f2d7f7fe700 1 -- 172.21.15.33:0/2126817086 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d7800fd40 con 0x7f2d88102e00 2024-05-15T13:59:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.875+0000 7f2d7f7fe700 1 -- 172.21.15.33:0/2126817086 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7800e3b0 con 0x7f2d88102e00 2024-05-15T13:59:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.875+0000 7f2d90662700 1 -- 172.21.15.33:0/2126817086 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d881a66d0 con 0x7f2d88102e00 2024-05-15T13:59:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.875+0000 7f2d90662700 1 -- 172.21.15.33:0/2126817086 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d8806a230 con 0x7f2d88102e00 2024-05-15T13:59:10.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.877+0000 7f2d7f7fe700 1 -- 172.21.15.33:0/2126817086 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2d78019070 con 0x7f2d88102e00 2024-05-15T13:59:10.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.878+0000 7f2d7f7fe700 1 --2- 172.21.15.33:0/2126817086 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f2d740b2900 0x7f2d740b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:10.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.878+0000 7f2d8e3fe700 1 --2- 172.21.15.33:0/2126817086 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f2d740b2900 0x7f2d740b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:10.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.878+0000 7f2d7f7fe700 1 -- 172.21.15.33:0/2126817086 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f2d780e1c20 con 0x7f2d88102e00 2024-05-15T13:59:10.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.879+0000 7f2d8e3fe700 1 --2- 172.21.15.33:0/2126817086 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f2d740b2900 0x7f2d740b4dc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2d881078b0 tx=0x7f2d840073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:10.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:10.882+0000 7f2d7f7fe700 1 -- 172.21.15.33:0/2126817086 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2d780a9900 con 0x7f2d88102e00 2024-05-15T13:59:11.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:11.093+0000 7f2d90662700 1 -- 172.21.15.33:0/2126817086 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2d88063980 con 0x7f2d740b2900 2024-05-15T13:59:11.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:11.100+0000 7f2d7f7fe700 1 -- 172.21.15.33:0/2126817086 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258930 (secure 0 0 0) 0x7f2d88063980 con 0x7f2d740b2900 2024-05-15T13:59:11.106 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:59:11.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:11.103+0000 7f2d90662700 1 -- 172.21.15.33:0/2126817086 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f2d740b2900 msgr2=0x7f2d740b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:11.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:11.103+0000 7f2d90662700 1 --2- 172.21.15.33:0/2126817086 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f2d740b2900 0x7f2d740b4dc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2d881078b0 tx=0x7f2d840073d0 comp rx=0 tx=0).stop 2024-05-15T13:59:11.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:11.103+0000 7f2d90662700 1 -- 172.21.15.33:0/2126817086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d88102e00 msgr2=0x7f2d8819fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:11.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:11.103+0000 7f2d90662700 1 --2- 172.21.15.33:0/2126817086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d88102e00 0x7f2d8819fd40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2d7800d950 tx=0x7f2d7800dc60 comp rx=0 tx=0).stop 2024-05-15T13:59:11.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:11.103+0000 7f2d90662700 1 -- 172.21.15.33:0/2126817086 shutdown_connections 2024-05-15T13:59:11.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:11.103+0000 7f2d90662700 1 --2- 172.21.15.33:0/2126817086 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f2d740b2900 0x7f2d740b4dc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:11.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:11.103+0000 7f2d90662700 1 --2- 172.21.15.33:0/2126817086 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2d88108520 0x7f2d88199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:11.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:11.103+0000 7f2d90662700 1 --2- 172.21.15.33:0/2126817086 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d88102440 0x7f2d8819f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:11.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:11.103+0000 7f2d90662700 1 --2- 172.21.15.33:0/2126817086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d88102e00 0x7f2d8819fd40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:11.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:11.103+0000 7f2d90662700 1 -- 172.21.15.33:0/2126817086 >> 172.21.15.33:0/2126817086 conn(0x7f2d880793f0 msgr2=0x7f2d8806b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:11.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:11.104+0000 7f2d90662700 1 -- 172.21.15.33:0/2126817086 shutdown_connections 2024-05-15T13:59:11.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:11.104+0000 7f2d90662700 1 -- 172.21.15.33:0/2126817086 wait complete. 2024-05-15T13:59:11.109 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-05-15T13:59:11.829 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-05-15T13:59:10.993332+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655872,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":661,"num_read_kb":428,"num_write":298,"num_write_kb":2270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2951974,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":741,"ondisk_log_size":741,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382540,"kb_used_data":12588,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559284,"statfs":{"total":1151940427776,"available":1151548706816,"internally_reserved":0,"allocated":12890112,"data_stored":7153776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":531,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":33,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003003"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.010666+0000","last_change":"2024-05-15T13:57:17.382728+0000","last_active":"2024-05-15T13:59:05.010666+0000","last_peered":"2024-05-15T13:59:05.010666+0000","last_clean":"2024-05-15T13:59:05.010666+0000","last_became_active":"2024-05-15T13:57:17.382315+0000","last_became_peered":"2024-05-15T13:57:17.382315+0000","last_unstale":"2024-05-15T13:59:05.010666+0000","last_undegraded":"2024-05-15T13:59:05.010666+0000","last_fullsized":"2024-05-15T13:59:05.010666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:03:53.137916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.009033+0000","last_change":"2024-05-15T13:57:17.390880+0000","last_active":"2024-05-15T13:59:05.009033+0000","last_peered":"2024-05-15T13:59:05.009033+0000","last_clean":"2024-05-15T13:59:05.009033+0000","last_became_active":"2024-05-15T13:57:17.390560+0000","last_became_peered":"2024-05-15T13:57:17.390560+0000","last_unstale":"2024-05-15T13:59:05.009033+0000","last_undegraded":"2024-05-15T13:59:05.009033+0000","last_fullsized":"2024-05-15T13:59:05.009033+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:59:33.615670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.008118+0000","last_change":"2024-05-15T13:57:17.386495+0000","last_active":"2024-05-15T13:59:05.008118+0000","last_peered":"2024-05-15T13:59:05.008118+0000","last_clean":"2024-05-15T13:59:05.008118+0000","last_became_active":"2024-05-15T13:57:17.386183+0000","last_became_peered":"2024-05-15T13:57:17.386183+0000","last_unstale":"2024-05-15T13:59:05.008118+0000","last_undegraded":"2024-05-15T13:59:05.008118+0000","last_fullsized":"2024-05-15T13:59:05.008118+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:36:13.202922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.988632+0000","last_change":"2024-05-15T13:57:17.385437+0000","last_active":"2024-05-15T13:59:04.988632+0000","last_peered":"2024-05-15T13:59:04.988632+0000","last_clean":"2024-05-15T13:59:04.988632+0000","last_became_active":"2024-05-15T13:57:17.384524+0000","last_became_peered":"2024-05-15T13:57:17.384524+0000","last_unstale":"2024-05-15T13:59:04.988632+0000","last_undegraded":"2024-05-15T13:59:04.988632+0000","last_fullsized":"2024-05-15T13:59:04.988632+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:45:17.207819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.990830+0000","last_change":"2024-05-15T13:57:17.387859+0000","last_active":"2024-05-15T13:59:04.990830+0000","last_peered":"2024-05-15T13:59:04.990830+0000","last_clean":"2024-05-15T13:59:04.990830+0000","last_became_active":"2024-05-15T13:57:17.387491+0000","last_became_peered":"2024-05-15T13:57:17.387491+0000","last_unstale":"2024-05-15T13:59:04.990830+0000","last_undegraded":"2024-05-15T13:59:04.990830+0000","last_fullsized":"2024-05-15T13:59:04.990830+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:43:08.212926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.992494+0000","last_change":"2024-05-15T13:57:17.382289+0000","last_active":"2024-05-15T13:59:04.992494+0000","last_peered":"2024-05-15T13:59:04.992494+0000","last_clean":"2024-05-15T13:59:04.992494+0000","last_became_active":"2024-05-15T13:57:17.380494+0000","last_became_peered":"2024-05-15T13:57:17.380494+0000","last_unstale":"2024-05-15T13:59:04.992494+0000","last_undegraded":"2024-05-15T13:59:04.992494+0000","last_fullsized":"2024-05-15T13:59:04.992494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:11:37.097034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.991272+0000","last_change":"2024-05-15T13:58:06.335776+0000","last_active":"2024-05-15T13:59:04.991272+0000","last_peered":"2024-05-15T13:59:04.991272+0000","last_clean":"2024-05-15T13:59:04.991272+0000","last_became_active":"2024-05-15T13:58:06.335239+0000","last_became_peered":"2024-05-15T13:58:06.335239+0000","last_unstale":"2024-05-15T13:59:04.991272+0000","last_undegraded":"2024-05-15T13:59:04.991272+0000","last_fullsized":"2024-05-15T13:59:04.991272+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:53:54.469082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.003919+0000","last_change":"2024-05-15T13:57:17.380969+0000","last_active":"2024-05-15T13:59:05.003919+0000","last_peered":"2024-05-15T13:59:05.003919+0000","last_clean":"2024-05-15T13:59:05.003919+0000","last_became_active":"2024-05-15T13:57:17.380660+0000","last_became_peered":"2024-05-15T13:57:17.380660+0000","last_unstale":"2024-05-15T13:59:05.003919+0000","last_undegraded":"2024-05-15T13:59:05.003919+0000","last_fullsized":"2024-05-15T13:59:05.003919+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:01:41.790323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.008419+0000","last_change":"2024-05-15T13:57:17.394943+0000","last_active":"2024-05-15T13:59:05.008419+0000","last_peered":"2024-05-15T13:59:05.008419+0000","last_clean":"2024-05-15T13:59:05.008419+0000","last_became_active":"2024-05-15T13:57:17.394571+0000","last_became_peered":"2024-05-15T13:57:17.394571+0000","last_unstale":"2024-05-15T13:59:05.008419+0000","last_undegraded":"2024-05-15T13:59:05.008419+0000","last_fullsized":"2024-05-15T13:59:05.008419+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:39:26.063096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.992735+0000","last_change":"2024-05-15T13:57:17.386012+0000","last_active":"2024-05-15T13:59:04.992735+0000","last_peered":"2024-05-15T13:59:04.992735+0000","last_clean":"2024-05-15T13:59:04.992735+0000","last_became_active":"2024-05-15T13:57:17.385636+0000","last_became_peered":"2024-05-15T13:57:17.385636+0000","last_unstale":"2024-05-15T13:59:04.992735+0000","last_undegraded":"2024-05-15T13:59:04.992735+0000","last_fullsized":"2024-05-15T13:59:04.992735+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:12:08.465959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.996962+0000","last_change":"2024-05-15T13:57:17.389162+0000","last_active":"2024-05-15T13:59:04.996962+0000","last_peered":"2024-05-15T13:59:04.996962+0000","last_clean":"2024-05-15T13:59:04.996962+0000","last_became_active":"2024-05-15T13:57:17.388503+0000","last_became_peered":"2024-05-15T13:57:17.388503+0000","last_unstale":"2024-05-15T13:59:04.996962+0000","last_undegraded":"2024-05-15T13:59:04.996962+0000","last_fullsized":"2024-05-15T13:59:04.996962+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:24:54.354445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.987834+0000","last_change":"2024-05-15T13:57:17.381689+0000","last_active":"2024-05-15T13:59:04.987834+0000","last_peered":"2024-05-15T13:59:04.987834+0000","last_clean":"2024-05-15T13:59:04.987834+0000","last_became_active":"2024-05-15T13:57:17.381376+0000","last_became_peered":"2024-05-15T13:57:17.381376+0000","last_unstale":"2024-05-15T13:59:04.987834+0000","last_undegraded":"2024-05-15T13:59:04.987834+0000","last_fullsized":"2024-05-15T13:59:04.987834+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:03:52.652934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.000206+0000","last_change":"2024-05-15T13:57:17.382214+0000","last_active":"2024-05-15T13:59:05.000206+0000","last_peered":"2024-05-15T13:59:05.000206+0000","last_clean":"2024-05-15T13:59:05.000206+0000","last_became_active":"2024-05-15T13:57:17.381851+0000","last_became_peered":"2024-05-15T13:57:17.381851+0000","last_unstale":"2024-05-15T13:59:05.000206+0000","last_undegraded":"2024-05-15T13:59:05.000206+0000","last_fullsized":"2024-05-15T13:59:05.000206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:18:39.854918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.009394+0000","last_change":"2024-05-15T13:57:17.390985+0000","last_active":"2024-05-15T13:59:05.009394+0000","last_peered":"2024-05-15T13:59:05.009394+0000","last_clean":"2024-05-15T13:59:05.009394+0000","last_became_active":"2024-05-15T13:57:17.390633+0000","last_became_peered":"2024-05-15T13:57:17.390633+0000","last_unstale":"2024-05-15T13:59:05.009394+0000","last_undegraded":"2024-05-15T13:59:05.009394+0000","last_fullsized":"2024-05-15T13:59:05.009394+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:32:47.177507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.010250+0000","last_change":"2024-05-15T13:57:17.383943+0000","last_active":"2024-05-15T13:59:05.010250+0000","last_peered":"2024-05-15T13:59:05.010250+0000","last_clean":"2024-05-15T13:59:05.010250+0000","last_became_active":"2024-05-15T13:57:17.383622+0000","last_became_peered":"2024-05-15T13:57:17.383622+0000","last_unstale":"2024-05-15T13:59:05.010250+0000","last_undegraded":"2024-05-15T13:59:05.010250+0000","last_fullsized":"2024-05-15T13:59:05.010250+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T00:30:23.249813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.007282+0000","last_change":"2024-05-15T13:57:17.390789+0000","last_active":"2024-05-15T13:59:05.007282+0000","last_peered":"2024-05-15T13:59:05.007282+0000","last_clean":"2024-05-15T13:59:05.007282+0000","last_became_active":"2024-05-15T13:57:17.390466+0000","last_became_peered":"2024-05-15T13:57:17.390466+0000","last_unstale":"2024-05-15T13:59:05.007282+0000","last_undegraded":"2024-05-15T13:59:05.007282+0000","last_fullsized":"2024-05-15T13:59:05.007282+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:07:24.746862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.010932+0000","last_change":"2024-05-15T13:57:17.384080+0000","last_active":"2024-05-15T13:59:05.010932+0000","last_peered":"2024-05-15T13:59:05.010932+0000","last_clean":"2024-05-15T13:59:05.010932+0000","last_became_active":"2024-05-15T13:57:17.383731+0000","last_became_peered":"2024-05-15T13:57:17.383731+0000","last_unstale":"2024-05-15T13:59:05.010932+0000","last_undegraded":"2024-05-15T13:59:05.010932+0000","last_fullsized":"2024-05-15T13:59:05.010932+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:36:22.874337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.991181+0000","last_change":"2024-05-15T13:57:17.382209+0000","last_active":"2024-05-15T13:59:04.991181+0000","last_peered":"2024-05-15T13:59:04.991181+0000","last_clean":"2024-05-15T13:59:04.991181+0000","last_became_active":"2024-05-15T13:57:17.381737+0000","last_became_peered":"2024-05-15T13:57:17.381737+0000","last_unstale":"2024-05-15T13:59:04.991181+0000","last_undegraded":"2024-05-15T13:59:04.991181+0000","last_fullsized":"2024-05-15T13:59:04.991181+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:38:53.913745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.990783+0000","last_change":"2024-05-15T13:58:06.335718+0000","last_active":"2024-05-15T13:59:04.990783+0000","last_peered":"2024-05-15T13:59:04.990783+0000","last_clean":"2024-05-15T13:59:04.990783+0000","last_became_active":"2024-05-15T13:58:06.335383+0000","last_became_peered":"2024-05-15T13:58:06.335383+0000","last_unstale":"2024-05-15T13:59:04.990783+0000","last_undegraded":"2024-05-15T13:59:04.990783+0000","last_fullsized":"2024-05-15T13:59:04.990783+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T00:19:47.658973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.334275+0000","last_change":"2024-05-15T13:57:14.256057+0000","last_active":"2024-05-15T13:58:06.334275+0000","last_peered":"2024-05-15T13:58:06.334275+0000","last_clean":"2024-05-15T13:58:06.334275+0000","last_became_active":"2024-05-15T13:57:14.255413+0000","last_became_peered":"2024-05-15T13:57:14.255413+0000","last_unstale":"2024-05-15T13:58:06.334275+0000","last_undegraded":"2024-05-15T13:58:06.334275+0000","last_fullsized":"2024-05-15T13:58:06.334275+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:13:43.433634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.005212+0000","last_change":"2024-05-15T13:57:17.390548+0000","last_active":"2024-05-15T13:59:05.005212+0000","last_peered":"2024-05-15T13:59:05.005212+0000","last_clean":"2024-05-15T13:59:05.005212+0000","last_became_active":"2024-05-15T13:57:17.390144+0000","last_became_peered":"2024-05-15T13:57:17.390144+0000","last_unstale":"2024-05-15T13:59:05.005212+0000","last_undegraded":"2024-05-15T13:59:05.005212+0000","last_fullsized":"2024-05-15T13:59:05.005212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:28:54.889994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.072265+0000","last_change":"2024-05-15T13:57:14.535704+0000","last_active":"2024-05-15T13:58:06.072265+0000","last_peered":"2024-05-15T13:58:06.072265+0000","last_clean":"2024-05-15T13:58:06.072265+0000","last_became_active":"2024-05-15T13:57:14.535356+0000","last_became_peered":"2024-05-15T13:57:14.535356+0000","last_unstale":"2024-05-15T13:58:06.072265+0000","last_undegraded":"2024-05-15T13:58:06.072265+0000","last_fullsized":"2024-05-15T13:58:06.072265+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:16:21.545507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.010399+0000","last_change":"2024-05-15T13:57:17.383482+0000","last_active":"2024-05-15T13:59:05.010399+0000","last_peered":"2024-05-15T13:59:05.010399+0000","last_clean":"2024-05-15T13:59:05.010399+0000","last_became_active":"2024-05-15T13:57:17.382973+0000","last_became_peered":"2024-05-15T13:57:17.382973+0000","last_unstale":"2024-05-15T13:59:05.010399+0000","last_undegraded":"2024-05-15T13:59:05.010399+0000","last_fullsized":"2024-05-15T13:59:05.010399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:31:08.937166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.079377+0000","last_change":"2024-05-15T13:58:06.078918+0000","last_active":"2024-05-15T13:58:06.079377+0000","last_peered":"2024-05-15T13:58:06.079377+0000","last_clean":"2024-05-15T13:58:06.079377+0000","last_became_active":"2024-05-15T13:58:06.073173+0000","last_became_peered":"2024-05-15T13:58:06.073173+0000","last_unstale":"2024-05-15T13:58:06.079377+0000","last_undegraded":"2024-05-15T13:58:06.079377+0000","last_fullsized":"2024-05-15T13:58:06.079377+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:21:10.582131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.008204+0000","last_change":"2024-05-15T13:57:17.388720+0000","last_active":"2024-05-15T13:59:05.008204+0000","last_peered":"2024-05-15T13:59:05.008204+0000","last_clean":"2024-05-15T13:59:05.008204+0000","last_became_active":"2024-05-15T13:57:17.388386+0000","last_became_peered":"2024-05-15T13:57:17.388386+0000","last_unstale":"2024-05-15T13:59:05.008204+0000","last_undegraded":"2024-05-15T13:59:05.008204+0000","last_fullsized":"2024-05-15T13:59:05.008204+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:02:40.543701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.070118+0000","last_change":"2024-05-15T13:57:14.259064+0000","last_active":"2024-05-15T13:58:06.070118+0000","last_peered":"2024-05-15T13:58:06.070118+0000","last_clean":"2024-05-15T13:58:06.070118+0000","last_became_active":"2024-05-15T13:57:14.258683+0000","last_became_peered":"2024-05-15T13:57:14.258683+0000","last_unstale":"2024-05-15T13:58:06.070118+0000","last_undegraded":"2024-05-15T13:58:06.070118+0000","last_fullsized":"2024-05-15T13:58:06.070118+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:12:43.108066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.992118+0000","last_change":"2024-05-15T13:57:17.380929+0000","last_active":"2024-05-15T13:59:04.992118+0000","last_peered":"2024-05-15T13:59:04.992118+0000","last_clean":"2024-05-15T13:59:04.992118+0000","last_became_active":"2024-05-15T13:57:17.380599+0000","last_became_peered":"2024-05-15T13:57:17.380599+0000","last_unstale":"2024-05-15T13:59:04.992118+0000","last_undegraded":"2024-05-15T13:59:04.992118+0000","last_fullsized":"2024-05-15T13:59:04.992118+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:19:55.368088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":129,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.503680+0000","last_change":"2024-05-15T13:57:14.260410+0000","last_active":"2024-05-15T13:59:05.503680+0000","last_peered":"2024-05-15T13:59:05.503680+0000","last_clean":"2024-05-15T13:59:05.503680+0000","last_became_active":"2024-05-15T13:57:14.260076+0000","last_became_peered":"2024-05-15T13:57:14.260076+0000","last_unstale":"2024-05-15T13:59:05.503680+0000","last_undegraded":"2024-05-15T13:59:05.503680+0000","last_fullsized":"2024-05-15T13:59:05.503680+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T00:27:03.436237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.989258+0000","last_change":"2024-05-15T13:57:17.394475+0000","last_active":"2024-05-15T13:59:04.989258+0000","last_peered":"2024-05-15T13:59:04.989258+0000","last_clean":"2024-05-15T13:59:04.989258+0000","last_became_active":"2024-05-15T13:57:17.393733+0000","last_became_peered":"2024-05-15T13:57:17.393733+0000","last_unstale":"2024-05-15T13:59:04.989258+0000","last_undegraded":"2024-05-15T13:59:04.989258+0000","last_fullsized":"2024-05-15T13:59:04.989258+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:55:06.198633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.067357+0000","last_change":"2024-05-15T13:57:14.256420+0000","last_active":"2024-05-15T13:58:06.067357+0000","last_peered":"2024-05-15T13:58:06.067357+0000","last_clean":"2024-05-15T13:58:06.067357+0000","last_became_active":"2024-05-15T13:57:14.255722+0000","last_became_peered":"2024-05-15T13:57:14.255722+0000","last_unstale":"2024-05-15T13:58:06.067357+0000","last_undegraded":"2024-05-15T13:58:06.067357+0000","last_fullsized":"2024-05-15T13:58:06.067357+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:21:30.499632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.994805+0000","last_change":"2024-05-15T13:58:06.074623+0000","last_active":"2024-05-15T13:59:04.994805+0000","last_peered":"2024-05-15T13:59:04.994805+0000","last_clean":"2024-05-15T13:59:04.994805+0000","last_became_active":"2024-05-15T13:58:06.074153+0000","last_became_peered":"2024-05-15T13:58:06.074153+0000","last_unstale":"2024-05-15T13:59:04.994805+0000","last_undegraded":"2024-05-15T13:59:04.994805+0000","last_fullsized":"2024-05-15T13:59:04.994805+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:08:22.922239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.069329+0000","last_change":"2024-05-15T13:57:14.256192+0000","last_active":"2024-05-15T13:58:06.069329+0000","last_peered":"2024-05-15T13:58:06.069329+0000","last_clean":"2024-05-15T13:58:06.069329+0000","last_became_active":"2024-05-15T13:57:14.255839+0000","last_became_peered":"2024-05-15T13:57:14.255839+0000","last_unstale":"2024-05-15T13:58:06.069329+0000","last_undegraded":"2024-05-15T13:58:06.069329+0000","last_fullsized":"2024-05-15T13:58:06.069329+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:13:26.071403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.008304+0000","last_change":"2024-05-15T13:57:17.386246+0000","last_active":"2024-05-15T13:59:05.008304+0000","last_peered":"2024-05-15T13:59:05.008304+0000","last_clean":"2024-05-15T13:59:05.008304+0000","last_became_active":"2024-05-15T13:57:17.385828+0000","last_became_peered":"2024-05-15T13:57:17.385828+0000","last_unstale":"2024-05-15T13:59:05.008304+0000","last_undegraded":"2024-05-15T13:59:05.008304+0000","last_fullsized":"2024-05-15T13:59:05.008304+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T00:24:38.656469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.071379+0000","last_change":"2024-05-15T13:57:14.259953+0000","last_active":"2024-05-15T13:58:06.071379+0000","last_peered":"2024-05-15T13:58:06.071379+0000","last_clean":"2024-05-15T13:58:06.071379+0000","last_became_active":"2024-05-15T13:57:14.259632+0000","last_became_peered":"2024-05-15T13:57:14.259632+0000","last_unstale":"2024-05-15T13:58:06.071379+0000","last_undegraded":"2024-05-15T13:58:06.071379+0000","last_fullsized":"2024-05-15T13:58:06.071379+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:30:28.361903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.984863+0000","last_change":"2024-05-15T13:57:17.382211+0000","last_active":"2024-05-15T13:59:04.984863+0000","last_peered":"2024-05-15T13:59:04.984863+0000","last_clean":"2024-05-15T13:59:04.984863+0000","last_became_active":"2024-05-15T13:57:17.381904+0000","last_became_peered":"2024-05-15T13:57:17.381904+0000","last_unstale":"2024-05-15T13:59:04.984863+0000","last_undegraded":"2024-05-15T13:59:04.984863+0000","last_fullsized":"2024-05-15T13:59:04.984863+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:53:49.250866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":121,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.956825+0000","last_change":"2024-05-15T13:57:14.254691+0000","last_active":"2024-05-15T13:59:04.956825+0000","last_peered":"2024-05-15T13:59:04.956825+0000","last_clean":"2024-05-15T13:59:04.956825+0000","last_became_active":"2024-05-15T13:57:14.254233+0000","last_became_peered":"2024-05-15T13:57:14.254233+0000","last_unstale":"2024-05-15T13:59:04.956825+0000","last_undegraded":"2024-05-15T13:59:04.956825+0000","last_fullsized":"2024-05-15T13:59:04.956825+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:41:31.719402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":91,"num_read_kb":91,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.008485+0000","last_change":"2024-05-15T13:57:17.382251+0000","last_active":"2024-05-15T13:59:05.008485+0000","last_peered":"2024-05-15T13:59:05.008485+0000","last_clean":"2024-05-15T13:59:05.008485+0000","last_became_active":"2024-05-15T13:57:17.381158+0000","last_became_peered":"2024-05-15T13:57:17.381158+0000","last_unstale":"2024-05-15T13:59:05.008485+0000","last_undegraded":"2024-05-15T13:59:05.008485+0000","last_fullsized":"2024-05-15T13:59:05.008485+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:24:55.971079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.070401+0000","last_change":"2024-05-15T13:57:14.258350+0000","last_active":"2024-05-15T13:58:06.070401+0000","last_peered":"2024-05-15T13:58:06.070401+0000","last_clean":"2024-05-15T13:58:06.070401+0000","last_became_active":"2024-05-15T13:57:14.257982+0000","last_became_peered":"2024-05-15T13:57:14.257982+0000","last_unstale":"2024-05-15T13:58:06.070401+0000","last_undegraded":"2024-05-15T13:58:06.070401+0000","last_fullsized":"2024-05-15T13:58:06.070401+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:50:07.053401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.010292+0000","last_change":"2024-05-15T13:57:17.392114+0000","last_active":"2024-05-15T13:59:05.010292+0000","last_peered":"2024-05-15T13:59:05.010292+0000","last_clean":"2024-05-15T13:59:05.010292+0000","last_became_active":"2024-05-15T13:57:17.391732+0000","last_became_peered":"2024-05-15T13:57:17.391732+0000","last_unstale":"2024-05-15T13:59:05.010292+0000","last_undegraded":"2024-05-15T13:59:05.010292+0000","last_fullsized":"2024-05-15T13:59:05.010292+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:28:02.304910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.070876+0000","last_change":"2024-05-15T13:57:14.535381+0000","last_active":"2024-05-15T13:58:06.070876+0000","last_peered":"2024-05-15T13:58:06.070876+0000","last_clean":"2024-05-15T13:58:06.070876+0000","last_became_active":"2024-05-15T13:57:14.534931+0000","last_became_peered":"2024-05-15T13:57:14.534931+0000","last_unstale":"2024-05-15T13:58:06.070876+0000","last_undegraded":"2024-05-15T13:58:06.070876+0000","last_fullsized":"2024-05-15T13:58:06.070876+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:13:04.785139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.008919+0000","last_change":"2024-05-15T13:57:17.386103+0000","last_active":"2024-05-15T13:59:05.008919+0000","last_peered":"2024-05-15T13:59:05.008919+0000","last_clean":"2024-05-15T13:59:05.008919+0000","last_became_active":"2024-05-15T13:57:17.385738+0000","last_became_peered":"2024-05-15T13:57:17.385738+0000","last_unstale":"2024-05-15T13:59:05.008919+0000","last_undegraded":"2024-05-15T13:59:05.008919+0000","last_fullsized":"2024-05-15T13:59:05.008919+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:42:56.136164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'31","reported_seq":248,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.502376+0000","last_change":"2024-05-15T13:57:14.258903+0000","last_active":"2024-05-15T13:59:05.502376+0000","last_peered":"2024-05-15T13:59:05.502376+0000","last_clean":"2024-05-15T13:59:05.502376+0000","last_became_active":"2024-05-15T13:57:14.258453+0000","last_became_peered":"2024-05-15T13:57:14.258453+0000","last_unstale":"2024-05-15T13:59:05.502376+0000","last_undegraded":"2024-05-15T13:59:05.502376+0000","last_fullsized":"2024-05-15T13:59:05.502376+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:49:33.004534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60596,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":191,"num_read_kb":154,"num_write":32,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.002777+0000","last_change":"2024-05-15T13:57:17.389717+0000","last_active":"2024-05-15T13:59:05.002777+0000","last_peered":"2024-05-15T13:59:05.002777+0000","last_clean":"2024-05-15T13:59:05.002777+0000","last_became_active":"2024-05-15T13:57:17.389188+0000","last_became_peered":"2024-05-15T13:57:17.389188+0000","last_unstale":"2024-05-15T13:59:05.002777+0000","last_undegraded":"2024-05-15T13:59:05.002777+0000","last_fullsized":"2024-05-15T13:59:05.002777+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:40:26.827507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.070290+0000","last_change":"2024-05-15T13:57:14.256007+0000","last_active":"2024-05-15T13:58:06.070290+0000","last_peered":"2024-05-15T13:58:06.070290+0000","last_clean":"2024-05-15T13:58:06.070290+0000","last_became_active":"2024-05-15T13:57:14.255656+0000","last_became_peered":"2024-05-15T13:57:14.255656+0000","last_unstale":"2024-05-15T13:58:06.070290+0000","last_undegraded":"2024-05-15T13:58:06.070290+0000","last_fullsized":"2024-05-15T13:58:06.070290+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:14:46.880981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.999058+0000","last_change":"2024-05-15T13:57:17.389971+0000","last_active":"2024-05-15T13:59:04.999058+0000","last_peered":"2024-05-15T13:59:04.999058+0000","last_clean":"2024-05-15T13:59:04.999058+0000","last_became_active":"2024-05-15T13:57:17.389633+0000","last_became_peered":"2024-05-15T13:57:17.389633+0000","last_unstale":"2024-05-15T13:59:04.999058+0000","last_undegraded":"2024-05-15T13:59:04.999058+0000","last_fullsized":"2024-05-15T13:59:04.999058+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:25:10.971086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.070528+0000","last_change":"2024-05-15T13:57:14.258086+0000","last_active":"2024-05-15T13:58:06.070528+0000","last_peered":"2024-05-15T13:58:06.070528+0000","last_clean":"2024-05-15T13:58:06.070528+0000","last_became_active":"2024-05-15T13:57:14.257736+0000","last_became_peered":"2024-05-15T13:57:14.257736+0000","last_unstale":"2024-05-15T13:58:06.070528+0000","last_undegraded":"2024-05-15T13:58:06.070528+0000","last_fullsized":"2024-05-15T13:58:06.070528+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T00:51:07.431725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.998308+0000","last_change":"2024-05-15T13:58:06.335760+0000","last_active":"2024-05-15T13:59:04.998308+0000","last_peered":"2024-05-15T13:59:04.998308+0000","last_clean":"2024-05-15T13:59:04.998308+0000","last_became_active":"2024-05-15T13:58:06.335236+0000","last_became_peered":"2024-05-15T13:58:06.335236+0000","last_unstale":"2024-05-15T13:59:04.998308+0000","last_undegraded":"2024-05-15T13:59:04.998308+0000","last_fullsized":"2024-05-15T13:59:04.998308+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:04:47.007886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":127,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.503262+0000","last_change":"2024-05-15T13:57:14.261911+0000","last_active":"2024-05-15T13:59:05.503262+0000","last_peered":"2024-05-15T13:59:05.503262+0000","last_clean":"2024-05-15T13:59:05.503262+0000","last_became_active":"2024-05-15T13:57:14.261506+0000","last_became_peered":"2024-05-15T13:57:14.261506+0000","last_unstale":"2024-05-15T13:59:05.503262+0000","last_undegraded":"2024-05-15T13:59:05.503262+0000","last_fullsized":"2024-05-15T13:59:05.503262+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:18:21.021651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.997726+0000","last_change":"2024-05-15T13:57:17.395074+0000","last_active":"2024-05-15T13:59:04.997726+0000","last_peered":"2024-05-15T13:59:04.997726+0000","last_clean":"2024-05-15T13:59:04.997726+0000","last_became_active":"2024-05-15T13:57:17.394712+0000","last_became_peered":"2024-05-15T13:57:17.394712+0000","last_unstale":"2024-05-15T13:59:04.997726+0000","last_undegraded":"2024-05-15T13:59:04.997726+0000","last_fullsized":"2024-05-15T13:59:04.997726+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:58:25.385949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":66,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.086231+0000","last_change":"2024-05-15T13:58:06.086231+0000","last_active":"2024-05-15T13:58:06.086231+0000","last_peered":"2024-05-15T13:58:06.086231+0000","last_clean":"2024-05-15T13:58:06.086231+0000","last_became_active":"2024-05-15T13:58:06.074640+0000","last_became_peered":"2024-05-15T13:58:06.074640+0000","last_unstale":"2024-05-15T13:58:06.086231+0000","last_undegraded":"2024-05-15T13:58:06.086231+0000","last_fullsized":"2024-05-15T13:58:06.086231+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:41:54.747228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.991517+0000","last_change":"2024-05-15T13:57:17.376895+0000","last_active":"2024-05-15T13:59:04.991517+0000","last_peered":"2024-05-15T13:59:04.991517+0000","last_clean":"2024-05-15T13:59:04.991517+0000","last_became_active":"2024-05-15T13:57:17.376499+0000","last_became_peered":"2024-05-15T13:57:17.376499+0000","last_unstale":"2024-05-15T13:59:04.991517+0000","last_undegraded":"2024-05-15T13:59:04.991517+0000","last_fullsized":"2024-05-15T13:59:04.991517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:39:41.707151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.090458+0000","last_change":"2024-05-15T13:58:06.090051+0000","last_active":"2024-05-15T13:58:06.090458+0000","last_peered":"2024-05-15T13:58:06.090458+0000","last_clean":"2024-05-15T13:58:06.090458+0000","last_became_active":"2024-05-15T13:58:06.074177+0000","last_became_peered":"2024-05-15T13:58:06.074177+0000","last_unstale":"2024-05-15T13:58:06.090458+0000","last_undegraded":"2024-05-15T13:58:06.090458+0000","last_fullsized":"2024-05-15T13:58:06.090458+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:07:11.090910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.005415+0000","last_change":"2024-05-15T13:57:17.390640+0000","last_active":"2024-05-15T13:59:05.005415+0000","last_peered":"2024-05-15T13:59:05.005415+0000","last_clean":"2024-05-15T13:59:05.005415+0000","last_became_active":"2024-05-15T13:57:17.390293+0000","last_became_peered":"2024-05-15T13:57:17.390293+0000","last_unstale":"2024-05-15T13:59:05.005415+0000","last_undegraded":"2024-05-15T13:59:05.005415+0000","last_fullsized":"2024-05-15T13:59:05.005415+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:08:18.907391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.334128+0000","last_change":"2024-05-15T13:57:14.535321+0000","last_active":"2024-05-15T13:58:06.334128+0000","last_peered":"2024-05-15T13:58:06.334128+0000","last_clean":"2024-05-15T13:58:06.334128+0000","last_became_active":"2024-05-15T13:57:14.534794+0000","last_became_peered":"2024-05-15T13:57:14.534794+0000","last_unstale":"2024-05-15T13:58:06.334128+0000","last_undegraded":"2024-05-15T13:58:06.334128+0000","last_fullsized":"2024-05-15T13:58:06.334128+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:27:07.087671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.000076+0000","last_change":"2024-05-15T13:57:17.381249+0000","last_active":"2024-05-15T13:59:05.000076+0000","last_peered":"2024-05-15T13:59:05.000076+0000","last_clean":"2024-05-15T13:59:05.000076+0000","last_became_active":"2024-05-15T13:57:17.380474+0000","last_became_peered":"2024-05-15T13:57:17.380474+0000","last_unstale":"2024-05-15T13:59:05.000076+0000","last_undegraded":"2024-05-15T13:59:05.000076+0000","last_fullsized":"2024-05-15T13:59:05.000076+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:49:00.341085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.070193+0000","last_change":"2024-05-15T13:57:14.257635+0000","last_active":"2024-05-15T13:58:06.070193+0000","last_peered":"2024-05-15T13:58:06.070193+0000","last_clean":"2024-05-15T13:58:06.070193+0000","last_became_active":"2024-05-15T13:57:14.257156+0000","last_became_peered":"2024-05-15T13:57:14.257156+0000","last_unstale":"2024-05-15T13:58:06.070193+0000","last_undegraded":"2024-05-15T13:58:06.070193+0000","last_fullsized":"2024-05-15T13:58:06.070193+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:11:48.742053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.010544+0000","last_change":"2024-05-15T13:57:17.391067+0000","last_active":"2024-05-15T13:59:05.010544+0000","last_peered":"2024-05-15T13:59:05.010544+0000","last_clean":"2024-05-15T13:59:05.010544+0000","last_became_active":"2024-05-15T13:57:17.390763+0000","last_became_peered":"2024-05-15T13:57:17.390763+0000","last_unstale":"2024-05-15T13:59:05.010544+0000","last_undegraded":"2024-05-15T13:59:05.010544+0000","last_fullsized":"2024-05-15T13:59:05.010544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:28:46.506307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.994098+0000","last_change":"2024-05-15T13:57:17.391816+0000","last_active":"2024-05-15T13:59:04.994098+0000","last_peered":"2024-05-15T13:59:04.994098+0000","last_clean":"2024-05-15T13:59:04.994098+0000","last_became_active":"2024-05-15T13:57:17.389374+0000","last_became_peered":"2024-05-15T13:57:17.389374+0000","last_unstale":"2024-05-15T13:59:04.994098+0000","last_undegraded":"2024-05-15T13:59:04.994098+0000","last_fullsized":"2024-05-15T13:59:04.994098+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:48:31.756133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":124,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:06.504060+0000","last_change":"2024-05-15T13:57:14.534946+0000","last_active":"2024-05-15T13:59:06.504060+0000","last_peered":"2024-05-15T13:59:06.504060+0000","last_clean":"2024-05-15T13:59:06.504060+0000","last_became_active":"2024-05-15T13:57:14.534535+0000","last_became_peered":"2024-05-15T13:57:14.534535+0000","last_unstale":"2024-05-15T13:59:06.504060+0000","last_undegraded":"2024-05-15T13:59:06.504060+0000","last_fullsized":"2024-05-15T13:59:06.504060+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:27:20.526800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":93,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.001208+0000","last_change":"2024-05-15T13:57:17.391637+0000","last_active":"2024-05-15T13:59:05.001208+0000","last_peered":"2024-05-15T13:59:05.001208+0000","last_clean":"2024-05-15T13:59:05.001208+0000","last_became_active":"2024-05-15T13:57:17.391271+0000","last_became_peered":"2024-05-15T13:57:17.391271+0000","last_unstale":"2024-05-15T13:59:05.001208+0000","last_undegraded":"2024-05-15T13:59:05.001208+0000","last_fullsized":"2024-05-15T13:59:05.001208+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:22:06.398387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.070436+0000","last_change":"2024-05-15T13:57:14.258510+0000","last_active":"2024-05-15T13:58:06.070436+0000","last_peered":"2024-05-15T13:58:06.070436+0000","last_clean":"2024-05-15T13:58:06.070436+0000","last_became_active":"2024-05-15T13:57:14.257544+0000","last_became_peered":"2024-05-15T13:57:14.257544+0000","last_unstale":"2024-05-15T13:58:06.070436+0000","last_undegraded":"2024-05-15T13:58:06.070436+0000","last_fullsized":"2024-05-15T13:58:06.070436+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:12:27.253931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":100,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.070866+0000","last_change":"2024-05-15T13:55:53.940355+0000","last_active":"2024-05-15T13:58:06.070866+0000","last_peered":"2024-05-15T13:58:06.070866+0000","last_clean":"2024-05-15T13:58:06.070866+0000","last_became_active":"2024-05-15T13:55:53.934638+0000","last_became_peered":"2024-05-15T13:55:53.934638+0000","last_unstale":"2024-05-15T13:58:06.070866+0000","last_undegraded":"2024-05-15T13:58:06.070866+0000","last_fullsized":"2024-05-15T13:58:06.070866+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:52:33.556435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:52:33.556435+0000","last_clean_scrub_stamp":"2024-05-15T13:52:33.556435+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:02:55.332569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.994765+0000","last_change":"2024-05-15T13:57:17.382394+0000","last_active":"2024-05-15T13:59:04.994765+0000","last_peered":"2024-05-15T13:59:04.994765+0000","last_clean":"2024-05-15T13:59:04.994765+0000","last_became_active":"2024-05-15T13:57:17.382090+0000","last_became_peered":"2024-05-15T13:57:17.382090+0000","last_unstale":"2024-05-15T13:59:04.994765+0000","last_undegraded":"2024-05-15T13:59:04.994765+0000","last_fullsized":"2024-05-15T13:59:04.994765+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:01:34.653216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.071031+0000","last_change":"2024-05-15T13:57:14.261549+0000","last_active":"2024-05-15T13:58:06.071031+0000","last_peered":"2024-05-15T13:58:06.071031+0000","last_clean":"2024-05-15T13:58:06.071031+0000","last_became_active":"2024-05-15T13:57:14.261165+0000","last_became_peered":"2024-05-15T13:57:14.261165+0000","last_unstale":"2024-05-15T13:58:06.071031+0000","last_undegraded":"2024-05-15T13:58:06.071031+0000","last_fullsized":"2024-05-15T13:58:06.071031+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:04:39.574440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.071070+0000","last_change":"2024-05-15T13:57:14.535309+0000","last_active":"2024-05-15T13:58:06.071070+0000","last_peered":"2024-05-15T13:58:06.071070+0000","last_clean":"2024-05-15T13:58:06.071070+0000","last_became_active":"2024-05-15T13:57:14.534938+0000","last_became_peered":"2024-05-15T13:57:14.534938+0000","last_unstale":"2024-05-15T13:58:06.071070+0000","last_undegraded":"2024-05-15T13:58:06.071070+0000","last_fullsized":"2024-05-15T13:58:06.071070+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:34:53.373149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.993876+0000","last_change":"2024-05-15T13:57:17.388272+0000","last_active":"2024-05-15T13:59:04.993876+0000","last_peered":"2024-05-15T13:59:04.993876+0000","last_clean":"2024-05-15T13:59:04.993876+0000","last_became_active":"2024-05-15T13:57:17.387849+0000","last_became_peered":"2024-05-15T13:57:17.387849+0000","last_unstale":"2024-05-15T13:59:04.993876+0000","last_undegraded":"2024-05-15T13:59:04.993876+0000","last_fullsized":"2024-05-15T13:59:04.993876+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:47:47.323566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'12","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.334307+0000","last_change":"2024-05-15T13:57:14.256161+0000","last_active":"2024-05-15T13:58:06.334307+0000","last_peered":"2024-05-15T13:58:06.334307+0000","last_clean":"2024-05-15T13:58:06.334307+0000","last_became_active":"2024-05-15T13:57:14.255707+0000","last_became_peered":"2024-05-15T13:57:14.255707+0000","last_unstale":"2024-05-15T13:58:06.334307+0000","last_undegraded":"2024-05-15T13:58:06.334307+0000","last_fullsized":"2024-05-15T13:58:06.334307+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:24:57.200029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.007271+0000","last_change":"2024-05-15T13:57:17.389522+0000","last_active":"2024-05-15T13:59:05.007271+0000","last_peered":"2024-05-15T13:59:05.007271+0000","last_clean":"2024-05-15T13:59:05.007271+0000","last_became_active":"2024-05-15T13:57:17.389177+0000","last_became_peered":"2024-05-15T13:57:17.389177+0000","last_unstale":"2024-05-15T13:59:05.007271+0000","last_undegraded":"2024-05-15T13:59:05.007271+0000","last_fullsized":"2024-05-15T13:59:05.007271+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:23:35.716934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.070146+0000","last_change":"2024-05-15T13:57:14.254300+0000","last_active":"2024-05-15T13:58:06.070146+0000","last_peered":"2024-05-15T13:58:06.070146+0000","last_clean":"2024-05-15T13:58:06.070146+0000","last_became_active":"2024-05-15T13:57:14.253741+0000","last_became_peered":"2024-05-15T13:57:14.253741+0000","last_unstale":"2024-05-15T13:58:06.070146+0000","last_undegraded":"2024-05-15T13:58:06.070146+0000","last_fullsized":"2024-05-15T13:58:06.070146+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:25:34.018159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.009993+0000","last_change":"2024-05-15T13:57:17.380688+0000","last_active":"2024-05-15T13:59:05.009993+0000","last_peered":"2024-05-15T13:59:05.009993+0000","last_clean":"2024-05-15T13:59:05.009993+0000","last_became_active":"2024-05-15T13:57:17.380379+0000","last_became_peered":"2024-05-15T13:57:17.380379+0000","last_unstale":"2024-05-15T13:59:05.009993+0000","last_undegraded":"2024-05-15T13:59:05.009993+0000","last_fullsized":"2024-05-15T13:59:05.009993+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:32:09.190750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.251776+0000","last_change":"2024-05-15T13:57:14.534775+0000","last_active":"2024-05-15T13:58:06.251776+0000","last_peered":"2024-05-15T13:58:06.251776+0000","last_clean":"2024-05-15T13:58:06.251776+0000","last_became_active":"2024-05-15T13:57:14.534440+0000","last_became_peered":"2024-05-15T13:57:14.534440+0000","last_unstale":"2024-05-15T13:58:06.251776+0000","last_undegraded":"2024-05-15T13:58:06.251776+0000","last_fullsized":"2024-05-15T13:58:06.251776+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:21:49.047187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.961744+0000","last_change":"2024-05-15T13:57:17.396252+0000","last_active":"2024-05-15T13:59:04.961744+0000","last_peered":"2024-05-15T13:59:04.961744+0000","last_clean":"2024-05-15T13:59:04.961744+0000","last_became_active":"2024-05-15T13:57:17.395853+0000","last_became_peered":"2024-05-15T13:57:17.395853+0000","last_unstale":"2024-05-15T13:59:04.961744+0000","last_undegraded":"2024-05-15T13:59:04.961744+0000","last_fullsized":"2024-05-15T13:59:04.961744+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:12:54.924053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.071745+0000","last_change":"2024-05-15T13:57:14.258411+0000","last_active":"2024-05-15T13:58:06.071745+0000","last_peered":"2024-05-15T13:58:06.071745+0000","last_clean":"2024-05-15T13:58:06.071745+0000","last_became_active":"2024-05-15T13:57:14.257847+0000","last_became_peered":"2024-05-15T13:57:14.257847+0000","last_unstale":"2024-05-15T13:58:06.071745+0000","last_undegraded":"2024-05-15T13:58:06.071745+0000","last_fullsized":"2024-05-15T13:58:06.071745+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:49:32.368351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.010085+0000","last_change":"2024-05-15T13:57:17.388916+0000","last_active":"2024-05-15T13:59:05.010085+0000","last_peered":"2024-05-15T13:59:05.010085+0000","last_clean":"2024-05-15T13:59:05.010085+0000","last_became_active":"2024-05-15T13:57:17.388603+0000","last_became_peered":"2024-05-15T13:57:17.388603+0000","last_unstale":"2024-05-15T13:59:05.010085+0000","last_undegraded":"2024-05-15T13:59:05.010085+0000","last_fullsized":"2024-05-15T13:59:05.010085+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:38:46.650393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.070010+0000","last_change":"2024-05-15T13:57:14.256801+0000","last_active":"2024-05-15T13:58:06.070010+0000","last_peered":"2024-05-15T13:58:06.070010+0000","last_clean":"2024-05-15T13:58:06.070010+0000","last_became_active":"2024-05-15T13:57:14.256476+0000","last_became_peered":"2024-05-15T13:57:14.256476+0000","last_unstale":"2024-05-15T13:58:06.070010+0000","last_undegraded":"2024-05-15T13:58:06.070010+0000","last_fullsized":"2024-05-15T13:58:06.070010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:22:34.495756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.004392+0000","last_change":"2024-05-15T13:57:17.380927+0000","last_active":"2024-05-15T13:59:05.004392+0000","last_peered":"2024-05-15T13:59:05.004392+0000","last_clean":"2024-05-15T13:59:05.004392+0000","last_became_active":"2024-05-15T13:57:17.380602+0000","last_became_peered":"2024-05-15T13:57:17.380602+0000","last_unstale":"2024-05-15T13:59:05.004392+0000","last_undegraded":"2024-05-15T13:59:05.004392+0000","last_fullsized":"2024-05-15T13:59:05.004392+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:37:13.920968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.069626+0000","last_change":"2024-05-15T13:57:14.257954+0000","last_active":"2024-05-15T13:58:06.069626+0000","last_peered":"2024-05-15T13:58:06.069626+0000","last_clean":"2024-05-15T13:58:06.069626+0000","last_became_active":"2024-05-15T13:57:14.257601+0000","last_became_peered":"2024-05-15T13:57:14.257601+0000","last_unstale":"2024-05-15T13:58:06.069626+0000","last_undegraded":"2024-05-15T13:58:06.069626+0000","last_fullsized":"2024-05-15T13:58:06.069626+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:47:14.811264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.998424+0000","last_change":"2024-05-15T13:57:17.385227+0000","last_active":"2024-05-15T13:59:04.998424+0000","last_peered":"2024-05-15T13:59:04.998424+0000","last_clean":"2024-05-15T13:59:04.998424+0000","last_became_active":"2024-05-15T13:57:17.384899+0000","last_became_peered":"2024-05-15T13:57:17.384899+0000","last_unstale":"2024-05-15T13:59:04.998424+0000","last_undegraded":"2024-05-15T13:59:04.998424+0000","last_fullsized":"2024-05-15T13:59:04.998424+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:04:29.162392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.070663+0000","last_change":"2024-05-15T13:57:14.535266+0000","last_active":"2024-05-15T13:58:06.070663+0000","last_peered":"2024-05-15T13:58:06.070663+0000","last_clean":"2024-05-15T13:58:06.070663+0000","last_became_active":"2024-05-15T13:57:14.534892+0000","last_became_peered":"2024-05-15T13:57:14.534892+0000","last_unstale":"2024-05-15T13:58:06.070663+0000","last_undegraded":"2024-05-15T13:58:06.070663+0000","last_fullsized":"2024-05-15T13:58:06.070663+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:49:50.934517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.004659+0000","last_change":"2024-05-15T13:57:17.390330+0000","last_active":"2024-05-15T13:59:05.004659+0000","last_peered":"2024-05-15T13:59:05.004659+0000","last_clean":"2024-05-15T13:59:05.004659+0000","last_became_active":"2024-05-15T13:57:17.390000+0000","last_became_peered":"2024-05-15T13:57:17.390000+0000","last_unstale":"2024-05-15T13:59:05.004659+0000","last_undegraded":"2024-05-15T13:59:05.004659+0000","last_fullsized":"2024-05-15T13:59:05.004659+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:52:11.361549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.070763+0000","last_change":"2024-05-15T13:57:14.258174+0000","last_active":"2024-05-15T13:58:06.070763+0000","last_peered":"2024-05-15T13:58:06.070763+0000","last_clean":"2024-05-15T13:58:06.070763+0000","last_became_active":"2024-05-15T13:57:14.257763+0000","last_became_peered":"2024-05-15T13:57:14.257763+0000","last_unstale":"2024-05-15T13:58:06.070763+0000","last_undegraded":"2024-05-15T13:58:06.070763+0000","last_fullsized":"2024-05-15T13:58:06.070763+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:34:31.573845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.001456+0000","last_change":"2024-05-15T13:57:17.391964+0000","last_active":"2024-05-15T13:59:05.001456+0000","last_peered":"2024-05-15T13:59:05.001456+0000","last_clean":"2024-05-15T13:59:05.001456+0000","last_became_active":"2024-05-15T13:57:17.391517+0000","last_became_peered":"2024-05-15T13:57:17.391517+0000","last_unstale":"2024-05-15T13:59:05.001456+0000","last_undegraded":"2024-05-15T13:59:05.001456+0000","last_fullsized":"2024-05-15T13:59:05.001456+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:54:36.883563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.256119+0000","last_change":"2024-05-15T13:58:06.256119+0000","last_active":"2024-05-15T13:58:06.256119+0000","last_peered":"2024-05-15T13:58:06.256119+0000","last_clean":"2024-05-15T13:58:06.256119+0000","last_became_active":"2024-05-15T13:58:06.253495+0000","last_became_peered":"2024-05-15T13:58:06.253495+0000","last_unstale":"2024-05-15T13:58:06.256119+0000","last_undegraded":"2024-05-15T13:58:06.256119+0000","last_fullsized":"2024-05-15T13:58:06.256119+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:41:06.585181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.005403+0000","last_change":"2024-05-15T13:57:17.389518+0000","last_active":"2024-05-15T13:59:05.005403+0000","last_peered":"2024-05-15T13:59:05.005403+0000","last_clean":"2024-05-15T13:59:05.005403+0000","last_became_active":"2024-05-15T13:57:17.388905+0000","last_became_peered":"2024-05-15T13:57:17.388905+0000","last_unstale":"2024-05-15T13:59:05.005403+0000","last_undegraded":"2024-05-15T13:59:05.005403+0000","last_fullsized":"2024-05-15T13:59:05.005403+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:00:51.109880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.007113+0000","last_change":"2024-05-15T13:57:17.394127+0000","last_active":"2024-05-15T13:59:05.007113+0000","last_peered":"2024-05-15T13:59:05.007113+0000","last_clean":"2024-05-15T13:59:05.007113+0000","last_became_active":"2024-05-15T13:57:17.393771+0000","last_became_peered":"2024-05-15T13:57:17.393771+0000","last_unstale":"2024-05-15T13:59:05.007113+0000","last_undegraded":"2024-05-15T13:59:05.007113+0000","last_fullsized":"2024-05-15T13:59:05.007113+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:41:33.491001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.986850+0000","last_change":"2024-05-15T13:57:17.392451+0000","last_active":"2024-05-15T13:59:04.986850+0000","last_peered":"2024-05-15T13:59:04.986850+0000","last_clean":"2024-05-15T13:59:04.986850+0000","last_became_active":"2024-05-15T13:57:17.392099+0000","last_became_peered":"2024-05-15T13:57:17.392099+0000","last_unstale":"2024-05-15T13:59:04.986850+0000","last_undegraded":"2024-05-15T13:59:04.986850+0000","last_fullsized":"2024-05-15T13:59:04.986850+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:00:14.774543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.984896+0000","last_change":"2024-05-15T13:57:17.385525+0000","last_active":"2024-05-15T13:59:04.984896+0000","last_peered":"2024-05-15T13:59:04.984896+0000","last_clean":"2024-05-15T13:59:04.984896+0000","last_became_active":"2024-05-15T13:57:17.384497+0000","last_became_peered":"2024-05-15T13:57:17.384497+0000","last_unstale":"2024-05-15T13:59:04.984896+0000","last_undegraded":"2024-05-15T13:59:04.984896+0000","last_fullsized":"2024-05-15T13:59:04.984896+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:50:39.259339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.010099+0000","last_change":"2024-05-15T13:57:17.379673+0000","last_active":"2024-05-15T13:59:05.010099+0000","last_peered":"2024-05-15T13:59:05.010099+0000","last_clean":"2024-05-15T13:59:05.010099+0000","last_became_active":"2024-05-15T13:57:17.379342+0000","last_became_peered":"2024-05-15T13:57:17.379342+0000","last_unstale":"2024-05-15T13:59:05.010099+0000","last_undegraded":"2024-05-15T13:59:05.010099+0000","last_fullsized":"2024-05-15T13:59:05.010099+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:26:43.982972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.990507+0000","last_change":"2024-05-15T13:58:06.335360+0000","last_active":"2024-05-15T13:59:04.990507+0000","last_peered":"2024-05-15T13:59:04.990507+0000","last_clean":"2024-05-15T13:59:04.990507+0000","last_became_active":"2024-05-15T13:58:06.334899+0000","last_became_peered":"2024-05-15T13:58:06.334899+0000","last_unstale":"2024-05-15T13:59:04.990507+0000","last_undegraded":"2024-05-15T13:59:04.990507+0000","last_fullsized":"2024-05-15T13:59:04.990507+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:34:27.081929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.005225+0000","last_change":"2024-05-15T13:57:17.390939+0000","last_active":"2024-05-15T13:59:05.005225+0000","last_peered":"2024-05-15T13:59:05.005225+0000","last_clean":"2024-05-15T13:59:05.005225+0000","last_became_active":"2024-05-15T13:57:17.390281+0000","last_became_peered":"2024-05-15T13:57:17.390281+0000","last_unstale":"2024-05-15T13:59:05.005225+0000","last_undegraded":"2024-05-15T13:59:05.005225+0000","last_fullsized":"2024-05-15T13:59:05.005225+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:02:22.077307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.989944+0000","last_change":"2024-05-15T13:57:17.392009+0000","last_active":"2024-05-15T13:59:04.989944+0000","last_peered":"2024-05-15T13:59:04.989944+0000","last_clean":"2024-05-15T13:59:04.989944+0000","last_became_active":"2024-05-15T13:57:17.391059+0000","last_became_peered":"2024-05-15T13:57:17.391059+0000","last_unstale":"2024-05-15T13:59:04.989944+0000","last_undegraded":"2024-05-15T13:59:04.989944+0000","last_fullsized":"2024-05-15T13:59:04.989944+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:51:46.787790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.007690+0000","last_change":"2024-05-15T13:57:17.383872+0000","last_active":"2024-05-15T13:59:05.007690+0000","last_peered":"2024-05-15T13:59:05.007690+0000","last_clean":"2024-05-15T13:59:05.007690+0000","last_became_active":"2024-05-15T13:57:17.383526+0000","last_became_peered":"2024-05-15T13:57:17.383526+0000","last_unstale":"2024-05-15T13:59:05.007690+0000","last_undegraded":"2024-05-15T13:59:05.007690+0000","last_fullsized":"2024-05-15T13:59:05.007690+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:43:39.343557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.000454+0000","last_change":"2024-05-15T13:57:17.377530+0000","last_active":"2024-05-15T13:59:05.000454+0000","last_peered":"2024-05-15T13:59:05.000454+0000","last_clean":"2024-05-15T13:59:05.000454+0000","last_became_active":"2024-05-15T13:57:17.377237+0000","last_became_peered":"2024-05-15T13:57:17.377237+0000","last_unstale":"2024-05-15T13:59:05.000454+0000","last_undegraded":"2024-05-15T13:59:05.000454+0000","last_fullsized":"2024-05-15T13:59:05.000454+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:29:02.173018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.007984+0000","last_change":"2024-05-15T13:57:17.386208+0000","last_active":"2024-05-15T13:59:05.007984+0000","last_peered":"2024-05-15T13:59:05.007984+0000","last_clean":"2024-05-15T13:59:05.007984+0000","last_became_active":"2024-05-15T13:57:17.385853+0000","last_became_peered":"2024-05-15T13:57:17.385853+0000","last_unstale":"2024-05-15T13:59:05.007984+0000","last_undegraded":"2024-05-15T13:59:05.007984+0000","last_fullsized":"2024-05-15T13:59:05.007984+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:17:06.124653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.006241+0000","last_change":"2024-05-15T13:57:17.390074+0000","last_active":"2024-05-15T13:59:05.006241+0000","last_peered":"2024-05-15T13:59:05.006241+0000","last_clean":"2024-05-15T13:59:05.006241+0000","last_became_active":"2024-05-15T13:57:17.389765+0000","last_became_peered":"2024-05-15T13:57:17.389765+0000","last_unstale":"2024-05-15T13:59:05.006241+0000","last_undegraded":"2024-05-15T13:59:05.006241+0000","last_fullsized":"2024-05-15T13:59:05.006241+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:49:51.321180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.999666+0000","last_change":"2024-05-15T13:57:17.389180+0000","last_active":"2024-05-15T13:59:04.999666+0000","last_peered":"2024-05-15T13:59:04.999666+0000","last_clean":"2024-05-15T13:59:04.999666+0000","last_became_active":"2024-05-15T13:57:17.388731+0000","last_became_peered":"2024-05-15T13:57:17.388731+0000","last_unstale":"2024-05-15T13:59:04.999666+0000","last_undegraded":"2024-05-15T13:59:04.999666+0000","last_fullsized":"2024-05-15T13:59:04.999666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:19:00.655474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.004125+0000","last_change":"2024-05-15T13:57:17.394045+0000","last_active":"2024-05-15T13:59:05.004125+0000","last_peered":"2024-05-15T13:59:05.004125+0000","last_clean":"2024-05-15T13:59:05.004125+0000","last_became_active":"2024-05-15T13:57:17.393699+0000","last_became_peered":"2024-05-15T13:57:17.393699+0000","last_unstale":"2024-05-15T13:59:05.004125+0000","last_undegraded":"2024-05-15T13:59:05.004125+0000","last_fullsized":"2024-05-15T13:59:05.004125+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:05:37.510370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65270,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":580,"num_read_kb":359,"num_write":173,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":134,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":195810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645349,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":897024,"data_stored":432784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841578,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36888,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708264,"statfs":{"total":95995035648,"available":95957262336,"internally_reserved":0,"allocated":1589248,"data_stored":1085371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005103,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":909312,"data_stored":432998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201333,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":905216,"data_stored":432896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397562,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28484,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716668,"statfs":{"total":95995035648,"available":95965868032,"internally_reserved":0,"allocated":1503232,"data_stored":1025521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561088,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36212,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708940,"statfs":{"total":95995035648,"available":95957954560,"internally_reserved":0,"allocated":897024,"data_stored":432387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":492776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36232,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708920,"statfs":{"total":95995035648,"available":95957934080,"internally_reserved":0,"allocated":917504,"data_stored":433658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":494546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149773,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":432698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953545,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":433468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757316,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36816,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708336,"statfs":{"total":95995035648,"available":95957336064,"internally_reserved":0,"allocated":1515520,"data_stored":1024673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":62366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-15T13:59:11.833 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-15T13:59:12.087 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:59:12.386 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:12 smithi033 ceph-mon[31650]: pgmap v345: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-15T13:59:12.387 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:12 smithi033 ceph-mon[31650]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-15T13:59:12.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:12 smithi052 ceph-mon[36190]: pgmap v345: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-15T13:59:12.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:12 smithi052 ceph-mon[36190]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-15T13:59:12.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:12 smithi081 ceph-mon[36434]: pgmap v345: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-15T13:59:12.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:12 smithi081 ceph-mon[36434]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-15T13:59:13.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.645+0000 7f9a7d076700 1 -- 172.21.15.33:0/1471115847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a781084f0 msgr2=0x7f9a781088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:13.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.645+0000 7f9a7d076700 1 --2- 172.21.15.33:0/1471115847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a781084f0 0x7f9a781088d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9a68009990 tx=0x7f9a68009ca0 comp rx=0 tx=0).stop 2024-05-15T13:59:13.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.645+0000 7f9a7d076700 1 -- 172.21.15.33:0/1471115847 shutdown_connections 2024-05-15T13:59:13.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.645+0000 7f9a7d076700 1 --2- 172.21.15.33:0/1471115847 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a78102e20 0x7f9a7810f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:13.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.645+0000 7f9a7d076700 1 --2- 172.21.15.33:0/1471115847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a781084f0 0x7f9a781088d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:13.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.645+0000 7f9a7d076700 1 --2- 172.21.15.33:0/1471115847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a78102460 0x7f9a781028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:13.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.645+0000 7f9a7d076700 1 -- 172.21.15.33:0/1471115847 >> 172.21.15.33:0/1471115847 conn(0x7f9a780794e0 msgr2=0x7f9a780798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:13.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.645+0000 7f9a7d076700 1 -- 172.21.15.33:0/1471115847 shutdown_connections 2024-05-15T13:59:13.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.645+0000 7f9a7d076700 1 -- 172.21.15.33:0/1471115847 wait complete. 2024-05-15T13:59:13.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.646+0000 7f9a7d076700 1 Processor -- start 2024-05-15T13:59:13.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.646+0000 7f9a7d076700 1 -- start start 2024-05-15T13:59:13.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.646+0000 7f9a7d076700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a78102460 0x7f9a781a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:13.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.646+0000 7f9a7d076700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a78102e20 0x7f9a781a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:13.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.646+0000 7f9a7d076700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a781084f0 0x7f9a781a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:13.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.646+0000 7f9a7d076700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9a78112700 con 0x7f9a78102e20 2024-05-15T13:59:13.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.646+0000 7f9a7d076700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9a78112580 con 0x7f9a781084f0 2024-05-15T13:59:13.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.646+0000 7f9a7d076700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9a78112880 con 0x7f9a78102460 2024-05-15T13:59:13.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.646+0000 7f9a7659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a78102e20 0x7f9a781a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:13.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.646+0000 7f9a7659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a78102e20 0x7f9a781a0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55814/0 (socket says 172.21.15.33:55814) 2024-05-15T13:59:13.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.647+0000 7f9a7659c700 1 -- 172.21.15.33:0/482269793 learned_addr learned my addr 172.21.15.33:0/482269793 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:13.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.647+0000 7f9a76d9d700 1 --2- 172.21.15.33:0/482269793 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a78102460 0x7f9a781a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:13.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.647+0000 7f9a7759e700 1 --2- 172.21.15.33:0/482269793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a781084f0 0x7f9a781a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:13.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.647+0000 7f9a7659c700 1 -- 172.21.15.33:0/482269793 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a78102460 msgr2=0x7f9a781a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:13.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.647+0000 7f9a7659c700 1 --2- 172.21.15.33:0/482269793 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a78102460 0x7f9a781a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:13.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.647+0000 7f9a7659c700 1 -- 172.21.15.33:0/482269793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a781084f0 msgr2=0x7f9a781a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:13.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.647+0000 7f9a7659c700 1 --2- 172.21.15.33:0/482269793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a781084f0 0x7f9a781a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:13.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.647+0000 7f9a7659c700 1 -- 172.21.15.33:0/482269793 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a781a5360 con 0x7f9a78102e20 2024-05-15T13:59:13.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.647+0000 7f9a7759e700 1 --2- 172.21.15.33:0/482269793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a781084f0 0x7f9a781a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T13:59:13.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.647+0000 7f9a7659c700 1 --2- 172.21.15.33:0/482269793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a78102e20 0x7f9a781a0af0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00b530 tx=0x7f9a6c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:13.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.647+0000 7f9a5ffff700 1 -- 172.21.15.33:0/482269793 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c015020 con 0x7f9a78102e20 2024-05-15T13:59:13.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.647+0000 7f9a7d076700 1 -- 172.21.15.33:0/482269793 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a781a5650 con 0x7f9a78102e20 2024-05-15T13:59:13.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.647+0000 7f9a5ffff700 1 -- 172.21.15.33:0/482269793 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a6c011ce0 con 0x7f9a78102e20 2024-05-15T13:59:13.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.648+0000 7f9a5ffff700 1 -- 172.21.15.33:0/482269793 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c010e40 con 0x7f9a78102e20 2024-05-15T13:59:13.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.648+0000 7f9a7d076700 1 -- 172.21.15.33:0/482269793 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a781acf00 con 0x7f9a78102e20 2024-05-15T13:59:13.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.649+0000 7f9a7d076700 1 -- 172.21.15.33:0/482269793 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a7806a230 con 0x7f9a78102e20 2024-05-15T13:59:13.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.650+0000 7f9a5ffff700 1 -- 172.21.15.33:0/482269793 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9a6c018020 con 0x7f9a78102e20 2024-05-15T13:59:13.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.651+0000 7f9a5ffff700 1 --2- 172.21.15.33:0/482269793 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f9a600b2880 0x7f9a600b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:13.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.651+0000 7f9a76d9d700 1 --2- 172.21.15.33:0/482269793 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f9a600b2880 0x7f9a600b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:13.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.651+0000 7f9a5ffff700 1 -- 172.21.15.33:0/482269793 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f9a6c0e1a70 con 0x7f9a78102e20 2024-05-15T13:59:13.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.651+0000 7f9a76d9d700 1 --2- 172.21.15.33:0/482269793 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f9a600b2880 0x7f9a600b4d40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9a6800b5c0 tx=0x7f9a68005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:13.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.655+0000 7f9a5ffff700 1 -- 172.21.15.33:0/482269793 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9a6c01f020 con 0x7f9a78102e20 2024-05-15T13:59:13.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.864+0000 7f9a7d076700 1 -- 172.21.15.33:0/482269793 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9a7806c4e0 con 0x7f9a600b2880 2024-05-15T13:59:13.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.871+0000 7f9a5ffff700 1 -- 172.21.15.33:0/482269793 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258844 (secure 0 0 0) 0x7f9a7806c4e0 con 0x7f9a600b2880 2024-05-15T13:59:13.876 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:59:13.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.874+0000 7f9a7d076700 1 -- 172.21.15.33:0/482269793 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f9a600b2880 msgr2=0x7f9a600b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:13.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.874+0000 7f9a7d076700 1 --2- 172.21.15.33:0/482269793 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f9a600b2880 0x7f9a600b4d40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9a6800b5c0 tx=0x7f9a68005960 comp rx=0 tx=0).stop 2024-05-15T13:59:13.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.874+0000 7f9a7d076700 1 -- 172.21.15.33:0/482269793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a78102e20 msgr2=0x7f9a781a0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:13.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.874+0000 7f9a7d076700 1 --2- 172.21.15.33:0/482269793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a78102e20 0x7f9a781a0af0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00b530 tx=0x7f9a6c00b840 comp rx=0 tx=0).stop 2024-05-15T13:59:13.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.874+0000 7f9a7d076700 1 -- 172.21.15.33:0/482269793 shutdown_connections 2024-05-15T13:59:13.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.874+0000 7f9a7d076700 1 --2- 172.21.15.33:0/482269793 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f9a600b2880 0x7f9a600b4d40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:13.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.874+0000 7f9a7d076700 1 --2- 172.21.15.33:0/482269793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a781084f0 0x7f9a781a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:13.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.874+0000 7f9a7d076700 1 --2- 172.21.15.33:0/482269793 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a78102460 0x7f9a781a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:13.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.874+0000 7f9a7d076700 1 --2- 172.21.15.33:0/482269793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a78102e20 0x7f9a781a0af0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:13.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.874+0000 7f9a7d076700 1 -- 172.21.15.33:0/482269793 >> 172.21.15.33:0/482269793 conn(0x7f9a780794e0 msgr2=0x7f9a7806bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:13.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.874+0000 7f9a7d076700 1 -- 172.21.15.33:0/482269793 shutdown_connections 2024-05-15T13:59:13.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:13.874+0000 7f9a7d076700 1 -- 172.21.15.33:0/482269793 wait complete. 2024-05-15T13:59:13.879 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-05-15T13:59:14.219 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:14 smithi033 ceph-mon[31650]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-15T13:59:14.501 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-05-15T13:59:12.993845+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655872,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":686,"num_read_kb":438,"num_write":298,"num_write_kb":2270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2951974,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":741,"ondisk_log_size":741,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382540,"kb_used_data":12588,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559284,"statfs":{"total":1151940427776,"available":1151548706816,"internally_reserved":0,"allocated":12890112,"data_stored":7153776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":531,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":43,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003091"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.010666+0000","last_change":"2024-05-15T13:57:17.382728+0000","last_active":"2024-05-15T13:59:05.010666+0000","last_peered":"2024-05-15T13:59:05.010666+0000","last_clean":"2024-05-15T13:59:05.010666+0000","last_became_active":"2024-05-15T13:57:17.382315+0000","last_became_peered":"2024-05-15T13:57:17.382315+0000","last_unstale":"2024-05-15T13:59:05.010666+0000","last_undegraded":"2024-05-15T13:59:05.010666+0000","last_fullsized":"2024-05-15T13:59:05.010666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:03:53.137916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.009033+0000","last_change":"2024-05-15T13:57:17.390880+0000","last_active":"2024-05-15T13:59:05.009033+0000","last_peered":"2024-05-15T13:59:05.009033+0000","last_clean":"2024-05-15T13:59:05.009033+0000","last_became_active":"2024-05-15T13:57:17.390560+0000","last_became_peered":"2024-05-15T13:57:17.390560+0000","last_unstale":"2024-05-15T13:59:05.009033+0000","last_undegraded":"2024-05-15T13:59:05.009033+0000","last_fullsized":"2024-05-15T13:59:05.009033+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:59:33.615670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.008118+0000","last_change":"2024-05-15T13:57:17.386495+0000","last_active":"2024-05-15T13:59:05.008118+0000","last_peered":"2024-05-15T13:59:05.008118+0000","last_clean":"2024-05-15T13:59:05.008118+0000","last_became_active":"2024-05-15T13:57:17.386183+0000","last_became_peered":"2024-05-15T13:57:17.386183+0000","last_unstale":"2024-05-15T13:59:05.008118+0000","last_undegraded":"2024-05-15T13:59:05.008118+0000","last_fullsized":"2024-05-15T13:59:05.008118+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:36:13.202922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.988632+0000","last_change":"2024-05-15T13:57:17.385437+0000","last_active":"2024-05-15T13:59:04.988632+0000","last_peered":"2024-05-15T13:59:04.988632+0000","last_clean":"2024-05-15T13:59:04.988632+0000","last_became_active":"2024-05-15T13:57:17.384524+0000","last_became_peered":"2024-05-15T13:57:17.384524+0000","last_unstale":"2024-05-15T13:59:04.988632+0000","last_undegraded":"2024-05-15T13:59:04.988632+0000","last_fullsized":"2024-05-15T13:59:04.988632+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:45:17.207819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.990830+0000","last_change":"2024-05-15T13:57:17.387859+0000","last_active":"2024-05-15T13:59:04.990830+0000","last_peered":"2024-05-15T13:59:04.990830+0000","last_clean":"2024-05-15T13:59:04.990830+0000","last_became_active":"2024-05-15T13:57:17.387491+0000","last_became_peered":"2024-05-15T13:57:17.387491+0000","last_unstale":"2024-05-15T13:59:04.990830+0000","last_undegraded":"2024-05-15T13:59:04.990830+0000","last_fullsized":"2024-05-15T13:59:04.990830+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:43:08.212926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.992494+0000","last_change":"2024-05-15T13:57:17.382289+0000","last_active":"2024-05-15T13:59:04.992494+0000","last_peered":"2024-05-15T13:59:04.992494+0000","last_clean":"2024-05-15T13:59:04.992494+0000","last_became_active":"2024-05-15T13:57:17.380494+0000","last_became_peered":"2024-05-15T13:57:17.380494+0000","last_unstale":"2024-05-15T13:59:04.992494+0000","last_undegraded":"2024-05-15T13:59:04.992494+0000","last_fullsized":"2024-05-15T13:59:04.992494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:11:37.097034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.991272+0000","last_change":"2024-05-15T13:58:06.335776+0000","last_active":"2024-05-15T13:59:04.991272+0000","last_peered":"2024-05-15T13:59:04.991272+0000","last_clean":"2024-05-15T13:59:04.991272+0000","last_became_active":"2024-05-15T13:58:06.335239+0000","last_became_peered":"2024-05-15T13:58:06.335239+0000","last_unstale":"2024-05-15T13:59:04.991272+0000","last_undegraded":"2024-05-15T13:59:04.991272+0000","last_fullsized":"2024-05-15T13:59:04.991272+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:53:54.469082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.003919+0000","last_change":"2024-05-15T13:57:17.380969+0000","last_active":"2024-05-15T13:59:05.003919+0000","last_peered":"2024-05-15T13:59:05.003919+0000","last_clean":"2024-05-15T13:59:05.003919+0000","last_became_active":"2024-05-15T13:57:17.380660+0000","last_became_peered":"2024-05-15T13:57:17.380660+0000","last_unstale":"2024-05-15T13:59:05.003919+0000","last_undegraded":"2024-05-15T13:59:05.003919+0000","last_fullsized":"2024-05-15T13:59:05.003919+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:01:41.790323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.008419+0000","last_change":"2024-05-15T13:57:17.394943+0000","last_active":"2024-05-15T13:59:05.008419+0000","last_peered":"2024-05-15T13:59:05.008419+0000","last_clean":"2024-05-15T13:59:05.008419+0000","last_became_active":"2024-05-15T13:57:17.394571+0000","last_became_peered":"2024-05-15T13:57:17.394571+0000","last_unstale":"2024-05-15T13:59:05.008419+0000","last_undegraded":"2024-05-15T13:59:05.008419+0000","last_fullsized":"2024-05-15T13:59:05.008419+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:39:26.063096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.992735+0000","last_change":"2024-05-15T13:57:17.386012+0000","last_active":"2024-05-15T13:59:04.992735+0000","last_peered":"2024-05-15T13:59:04.992735+0000","last_clean":"2024-05-15T13:59:04.992735+0000","last_became_active":"2024-05-15T13:57:17.385636+0000","last_became_peered":"2024-05-15T13:57:17.385636+0000","last_unstale":"2024-05-15T13:59:04.992735+0000","last_undegraded":"2024-05-15T13:59:04.992735+0000","last_fullsized":"2024-05-15T13:59:04.992735+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:12:08.465959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.996962+0000","last_change":"2024-05-15T13:57:17.389162+0000","last_active":"2024-05-15T13:59:04.996962+0000","last_peered":"2024-05-15T13:59:04.996962+0000","last_clean":"2024-05-15T13:59:04.996962+0000","last_became_active":"2024-05-15T13:57:17.388503+0000","last_became_peered":"2024-05-15T13:57:17.388503+0000","last_unstale":"2024-05-15T13:59:04.996962+0000","last_undegraded":"2024-05-15T13:59:04.996962+0000","last_fullsized":"2024-05-15T13:59:04.996962+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:24:54.354445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.987834+0000","last_change":"2024-05-15T13:57:17.381689+0000","last_active":"2024-05-15T13:59:04.987834+0000","last_peered":"2024-05-15T13:59:04.987834+0000","last_clean":"2024-05-15T13:59:04.987834+0000","last_became_active":"2024-05-15T13:57:17.381376+0000","last_became_peered":"2024-05-15T13:57:17.381376+0000","last_unstale":"2024-05-15T13:59:04.987834+0000","last_undegraded":"2024-05-15T13:59:04.987834+0000","last_fullsized":"2024-05-15T13:59:04.987834+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:03:52.652934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.000206+0000","last_change":"2024-05-15T13:57:17.382214+0000","last_active":"2024-05-15T13:59:05.000206+0000","last_peered":"2024-05-15T13:59:05.000206+0000","last_clean":"2024-05-15T13:59:05.000206+0000","last_became_active":"2024-05-15T13:57:17.381851+0000","last_became_peered":"2024-05-15T13:57:17.381851+0000","last_unstale":"2024-05-15T13:59:05.000206+0000","last_undegraded":"2024-05-15T13:59:05.000206+0000","last_fullsized":"2024-05-15T13:59:05.000206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:18:39.854918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.009394+0000","last_change":"2024-05-15T13:57:17.390985+0000","last_active":"2024-05-15T13:59:05.009394+0000","last_peered":"2024-05-15T13:59:05.009394+0000","last_clean":"2024-05-15T13:59:05.009394+0000","last_became_active":"2024-05-15T13:57:17.390633+0000","last_became_peered":"2024-05-15T13:57:17.390633+0000","last_unstale":"2024-05-15T13:59:05.009394+0000","last_undegraded":"2024-05-15T13:59:05.009394+0000","last_fullsized":"2024-05-15T13:59:05.009394+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:32:47.177507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.010250+0000","last_change":"2024-05-15T13:57:17.383943+0000","last_active":"2024-05-15T13:59:05.010250+0000","last_peered":"2024-05-15T13:59:05.010250+0000","last_clean":"2024-05-15T13:59:05.010250+0000","last_became_active":"2024-05-15T13:57:17.383622+0000","last_became_peered":"2024-05-15T13:57:17.383622+0000","last_unstale":"2024-05-15T13:59:05.010250+0000","last_undegraded":"2024-05-15T13:59:05.010250+0000","last_fullsized":"2024-05-15T13:59:05.010250+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T00:30:23.249813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.007282+0000","last_change":"2024-05-15T13:57:17.390789+0000","last_active":"2024-05-15T13:59:05.007282+0000","last_peered":"2024-05-15T13:59:05.007282+0000","last_clean":"2024-05-15T13:59:05.007282+0000","last_became_active":"2024-05-15T13:57:17.390466+0000","last_became_peered":"2024-05-15T13:57:17.390466+0000","last_unstale":"2024-05-15T13:59:05.007282+0000","last_undegraded":"2024-05-15T13:59:05.007282+0000","last_fullsized":"2024-05-15T13:59:05.007282+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:07:24.746862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.010932+0000","last_change":"2024-05-15T13:57:17.384080+0000","last_active":"2024-05-15T13:59:05.010932+0000","last_peered":"2024-05-15T13:59:05.010932+0000","last_clean":"2024-05-15T13:59:05.010932+0000","last_became_active":"2024-05-15T13:57:17.383731+0000","last_became_peered":"2024-05-15T13:57:17.383731+0000","last_unstale":"2024-05-15T13:59:05.010932+0000","last_undegraded":"2024-05-15T13:59:05.010932+0000","last_fullsized":"2024-05-15T13:59:05.010932+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:36:22.874337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.991181+0000","last_change":"2024-05-15T13:57:17.382209+0000","last_active":"2024-05-15T13:59:04.991181+0000","last_peered":"2024-05-15T13:59:04.991181+0000","last_clean":"2024-05-15T13:59:04.991181+0000","last_became_active":"2024-05-15T13:57:17.381737+0000","last_became_peered":"2024-05-15T13:57:17.381737+0000","last_unstale":"2024-05-15T13:59:04.991181+0000","last_undegraded":"2024-05-15T13:59:04.991181+0000","last_fullsized":"2024-05-15T13:59:04.991181+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:38:53.913745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.990783+0000","last_change":"2024-05-15T13:58:06.335718+0000","last_active":"2024-05-15T13:59:04.990783+0000","last_peered":"2024-05-15T13:59:04.990783+0000","last_clean":"2024-05-15T13:59:04.990783+0000","last_became_active":"2024-05-15T13:58:06.335383+0000","last_became_peered":"2024-05-15T13:58:06.335383+0000","last_unstale":"2024-05-15T13:59:04.990783+0000","last_undegraded":"2024-05-15T13:59:04.990783+0000","last_fullsized":"2024-05-15T13:59:04.990783+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T00:19:47.658973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.334275+0000","last_change":"2024-05-15T13:57:14.256057+0000","last_active":"2024-05-15T13:58:06.334275+0000","last_peered":"2024-05-15T13:58:06.334275+0000","last_clean":"2024-05-15T13:58:06.334275+0000","last_became_active":"2024-05-15T13:57:14.255413+0000","last_became_peered":"2024-05-15T13:57:14.255413+0000","last_unstale":"2024-05-15T13:58:06.334275+0000","last_undegraded":"2024-05-15T13:58:06.334275+0000","last_fullsized":"2024-05-15T13:58:06.334275+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:13:43.433634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.005212+0000","last_change":"2024-05-15T13:57:17.390548+0000","last_active":"2024-05-15T13:59:05.005212+0000","last_peered":"2024-05-15T13:59:05.005212+0000","last_clean":"2024-05-15T13:59:05.005212+0000","last_became_active":"2024-05-15T13:57:17.390144+0000","last_became_peered":"2024-05-15T13:57:17.390144+0000","last_unstale":"2024-05-15T13:59:05.005212+0000","last_undegraded":"2024-05-15T13:59:05.005212+0000","last_fullsized":"2024-05-15T13:59:05.005212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:28:54.889994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.072265+0000","last_change":"2024-05-15T13:57:14.535704+0000","last_active":"2024-05-15T13:58:06.072265+0000","last_peered":"2024-05-15T13:58:06.072265+0000","last_clean":"2024-05-15T13:58:06.072265+0000","last_became_active":"2024-05-15T13:57:14.535356+0000","last_became_peered":"2024-05-15T13:57:14.535356+0000","last_unstale":"2024-05-15T13:58:06.072265+0000","last_undegraded":"2024-05-15T13:58:06.072265+0000","last_fullsized":"2024-05-15T13:58:06.072265+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:16:21.545507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.010399+0000","last_change":"2024-05-15T13:57:17.383482+0000","last_active":"2024-05-15T13:59:05.010399+0000","last_peered":"2024-05-15T13:59:05.010399+0000","last_clean":"2024-05-15T13:59:05.010399+0000","last_became_active":"2024-05-15T13:57:17.382973+0000","last_became_peered":"2024-05-15T13:57:17.382973+0000","last_unstale":"2024-05-15T13:59:05.010399+0000","last_undegraded":"2024-05-15T13:59:05.010399+0000","last_fullsized":"2024-05-15T13:59:05.010399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:31:08.937166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.079377+0000","last_change":"2024-05-15T13:58:06.078918+0000","last_active":"2024-05-15T13:58:06.079377+0000","last_peered":"2024-05-15T13:58:06.079377+0000","last_clean":"2024-05-15T13:58:06.079377+0000","last_became_active":"2024-05-15T13:58:06.073173+0000","last_became_peered":"2024-05-15T13:58:06.073173+0000","last_unstale":"2024-05-15T13:58:06.079377+0000","last_undegraded":"2024-05-15T13:58:06.079377+0000","last_fullsized":"2024-05-15T13:58:06.079377+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:21:10.582131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.008204+0000","last_change":"2024-05-15T13:57:17.388720+0000","last_active":"2024-05-15T13:59:05.008204+0000","last_peered":"2024-05-15T13:59:05.008204+0000","last_clean":"2024-05-15T13:59:05.008204+0000","last_became_active":"2024-05-15T13:57:17.388386+0000","last_became_peered":"2024-05-15T13:57:17.388386+0000","last_unstale":"2024-05-15T13:59:05.008204+0000","last_undegraded":"2024-05-15T13:59:05.008204+0000","last_fullsized":"2024-05-15T13:59:05.008204+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:02:40.543701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.070118+0000","last_change":"2024-05-15T13:57:14.259064+0000","last_active":"2024-05-15T13:58:06.070118+0000","last_peered":"2024-05-15T13:58:06.070118+0000","last_clean":"2024-05-15T13:58:06.070118+0000","last_became_active":"2024-05-15T13:57:14.258683+0000","last_became_peered":"2024-05-15T13:57:14.258683+0000","last_unstale":"2024-05-15T13:58:06.070118+0000","last_undegraded":"2024-05-15T13:58:06.070118+0000","last_fullsized":"2024-05-15T13:58:06.070118+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:12:43.108066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.992118+0000","last_change":"2024-05-15T13:57:17.380929+0000","last_active":"2024-05-15T13:59:04.992118+0000","last_peered":"2024-05-15T13:59:04.992118+0000","last_clean":"2024-05-15T13:59:04.992118+0000","last_became_active":"2024-05-15T13:57:17.380599+0000","last_became_peered":"2024-05-15T13:57:17.380599+0000","last_unstale":"2024-05-15T13:59:04.992118+0000","last_undegraded":"2024-05-15T13:59:04.992118+0000","last_fullsized":"2024-05-15T13:59:04.992118+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:19:55.368088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":134,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:10.510818+0000","last_change":"2024-05-15T13:57:14.260410+0000","last_active":"2024-05-15T13:59:10.510818+0000","last_peered":"2024-05-15T13:59:10.510818+0000","last_clean":"2024-05-15T13:59:10.510818+0000","last_became_active":"2024-05-15T13:57:14.260076+0000","last_became_peered":"2024-05-15T13:57:14.260076+0000","last_unstale":"2024-05-15T13:59:10.510818+0000","last_undegraded":"2024-05-15T13:59:10.510818+0000","last_fullsized":"2024-05-15T13:59:10.510818+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T00:27:03.436237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.989258+0000","last_change":"2024-05-15T13:57:17.394475+0000","last_active":"2024-05-15T13:59:04.989258+0000","last_peered":"2024-05-15T13:59:04.989258+0000","last_clean":"2024-05-15T13:59:04.989258+0000","last_became_active":"2024-05-15T13:57:17.393733+0000","last_became_peered":"2024-05-15T13:57:17.393733+0000","last_unstale":"2024-05-15T13:59:04.989258+0000","last_undegraded":"2024-05-15T13:59:04.989258+0000","last_fullsized":"2024-05-15T13:59:04.989258+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:55:06.198633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.067357+0000","last_change":"2024-05-15T13:57:14.256420+0000","last_active":"2024-05-15T13:58:06.067357+0000","last_peered":"2024-05-15T13:58:06.067357+0000","last_clean":"2024-05-15T13:58:06.067357+0000","last_became_active":"2024-05-15T13:57:14.255722+0000","last_became_peered":"2024-05-15T13:57:14.255722+0000","last_unstale":"2024-05-15T13:58:06.067357+0000","last_undegraded":"2024-05-15T13:58:06.067357+0000","last_fullsized":"2024-05-15T13:58:06.067357+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:21:30.499632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.994805+0000","last_change":"2024-05-15T13:58:06.074623+0000","last_active":"2024-05-15T13:59:04.994805+0000","last_peered":"2024-05-15T13:59:04.994805+0000","last_clean":"2024-05-15T13:59:04.994805+0000","last_became_active":"2024-05-15T13:58:06.074153+0000","last_became_peered":"2024-05-15T13:58:06.074153+0000","last_unstale":"2024-05-15T13:59:04.994805+0000","last_undegraded":"2024-05-15T13:59:04.994805+0000","last_fullsized":"2024-05-15T13:59:04.994805+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:08:22.922239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.069329+0000","last_change":"2024-05-15T13:57:14.256192+0000","last_active":"2024-05-15T13:58:06.069329+0000","last_peered":"2024-05-15T13:58:06.069329+0000","last_clean":"2024-05-15T13:58:06.069329+0000","last_became_active":"2024-05-15T13:57:14.255839+0000","last_became_peered":"2024-05-15T13:57:14.255839+0000","last_unstale":"2024-05-15T13:58:06.069329+0000","last_undegraded":"2024-05-15T13:58:06.069329+0000","last_fullsized":"2024-05-15T13:58:06.069329+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:13:26.071403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.008304+0000","last_change":"2024-05-15T13:57:17.386246+0000","last_active":"2024-05-15T13:59:05.008304+0000","last_peered":"2024-05-15T13:59:05.008304+0000","last_clean":"2024-05-15T13:59:05.008304+0000","last_became_active":"2024-05-15T13:57:17.385828+0000","last_became_peered":"2024-05-15T13:57:17.385828+0000","last_unstale":"2024-05-15T13:59:05.008304+0000","last_undegraded":"2024-05-15T13:59:05.008304+0000","last_fullsized":"2024-05-15T13:59:05.008304+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T00:24:38.656469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.071379+0000","last_change":"2024-05-15T13:57:14.259953+0000","last_active":"2024-05-15T13:58:06.071379+0000","last_peered":"2024-05-15T13:58:06.071379+0000","last_clean":"2024-05-15T13:58:06.071379+0000","last_became_active":"2024-05-15T13:57:14.259632+0000","last_became_peered":"2024-05-15T13:57:14.259632+0000","last_unstale":"2024-05-15T13:58:06.071379+0000","last_undegraded":"2024-05-15T13:58:06.071379+0000","last_fullsized":"2024-05-15T13:58:06.071379+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:30:28.361903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.984863+0000","last_change":"2024-05-15T13:57:17.382211+0000","last_active":"2024-05-15T13:59:04.984863+0000","last_peered":"2024-05-15T13:59:04.984863+0000","last_clean":"2024-05-15T13:59:04.984863+0000","last_became_active":"2024-05-15T13:57:17.381904+0000","last_became_peered":"2024-05-15T13:57:17.381904+0000","last_unstale":"2024-05-15T13:59:04.984863+0000","last_undegraded":"2024-05-15T13:59:04.984863+0000","last_fullsized":"2024-05-15T13:59:04.984863+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:53:49.250866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'4","reported_seq":126,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:09.508754+0000","last_change":"2024-05-15T13:57:14.254691+0000","last_active":"2024-05-15T13:59:09.508754+0000","last_peered":"2024-05-15T13:59:09.508754+0000","last_clean":"2024-05-15T13:59:09.508754+0000","last_became_active":"2024-05-15T13:57:14.254233+0000","last_became_peered":"2024-05-15T13:57:14.254233+0000","last_unstale":"2024-05-15T13:59:09.508754+0000","last_undegraded":"2024-05-15T13:59:09.508754+0000","last_fullsized":"2024-05-15T13:59:09.508754+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:41:31.719402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.008485+0000","last_change":"2024-05-15T13:57:17.382251+0000","last_active":"2024-05-15T13:59:05.008485+0000","last_peered":"2024-05-15T13:59:05.008485+0000","last_clean":"2024-05-15T13:59:05.008485+0000","last_became_active":"2024-05-15T13:57:17.381158+0000","last_became_peered":"2024-05-15T13:57:17.381158+0000","last_unstale":"2024-05-15T13:59:05.008485+0000","last_undegraded":"2024-05-15T13:59:05.008485+0000","last_fullsized":"2024-05-15T13:59:05.008485+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:24:55.971079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.070401+0000","last_change":"2024-05-15T13:57:14.258350+0000","last_active":"2024-05-15T13:58:06.070401+0000","last_peered":"2024-05-15T13:58:06.070401+0000","last_clean":"2024-05-15T13:58:06.070401+0000","last_became_active":"2024-05-15T13:57:14.257982+0000","last_became_peered":"2024-05-15T13:57:14.257982+0000","last_unstale":"2024-05-15T13:58:06.070401+0000","last_undegraded":"2024-05-15T13:58:06.070401+0000","last_fullsized":"2024-05-15T13:58:06.070401+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:50:07.053401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.010292+0000","last_change":"2024-05-15T13:57:17.392114+0000","last_active":"2024-05-15T13:59:05.010292+0000","last_peered":"2024-05-15T13:59:05.010292+0000","last_clean":"2024-05-15T13:59:05.010292+0000","last_became_active":"2024-05-15T13:57:17.391732+0000","last_became_peered":"2024-05-15T13:57:17.391732+0000","last_unstale":"2024-05-15T13:59:05.010292+0000","last_undegraded":"2024-05-15T13:59:05.010292+0000","last_fullsized":"2024-05-15T13:59:05.010292+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:28:02.304910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.070876+0000","last_change":"2024-05-15T13:57:14.535381+0000","last_active":"2024-05-15T13:58:06.070876+0000","last_peered":"2024-05-15T13:58:06.070876+0000","last_clean":"2024-05-15T13:58:06.070876+0000","last_became_active":"2024-05-15T13:57:14.534931+0000","last_became_peered":"2024-05-15T13:57:14.534931+0000","last_unstale":"2024-05-15T13:58:06.070876+0000","last_undegraded":"2024-05-15T13:58:06.070876+0000","last_fullsized":"2024-05-15T13:58:06.070876+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:13:04.785139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.008919+0000","last_change":"2024-05-15T13:57:17.386103+0000","last_active":"2024-05-15T13:59:05.008919+0000","last_peered":"2024-05-15T13:59:05.008919+0000","last_clean":"2024-05-15T13:59:05.008919+0000","last_became_active":"2024-05-15T13:57:17.385738+0000","last_became_peered":"2024-05-15T13:57:17.385738+0000","last_unstale":"2024-05-15T13:59:05.008919+0000","last_undegraded":"2024-05-15T13:59:05.008919+0000","last_fullsized":"2024-05-15T13:59:05.008919+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:42:56.136164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'31","reported_seq":258,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:10.510217+0000","last_change":"2024-05-15T13:57:14.258903+0000","last_active":"2024-05-15T13:59:10.510217+0000","last_peered":"2024-05-15T13:59:10.510217+0000","last_clean":"2024-05-15T13:59:10.510217+0000","last_became_active":"2024-05-15T13:57:14.258453+0000","last_became_peered":"2024-05-15T13:57:14.258453+0000","last_unstale":"2024-05-15T13:59:10.510217+0000","last_undegraded":"2024-05-15T13:59:10.510217+0000","last_fullsized":"2024-05-15T13:59:10.510217+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:49:33.004534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60596,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":201,"num_read_kb":159,"num_write":32,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.002777+0000","last_change":"2024-05-15T13:57:17.389717+0000","last_active":"2024-05-15T13:59:05.002777+0000","last_peered":"2024-05-15T13:59:05.002777+0000","last_clean":"2024-05-15T13:59:05.002777+0000","last_became_active":"2024-05-15T13:57:17.389188+0000","last_became_peered":"2024-05-15T13:57:17.389188+0000","last_unstale":"2024-05-15T13:59:05.002777+0000","last_undegraded":"2024-05-15T13:59:05.002777+0000","last_fullsized":"2024-05-15T13:59:05.002777+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:40:26.827507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.070290+0000","last_change":"2024-05-15T13:57:14.256007+0000","last_active":"2024-05-15T13:58:06.070290+0000","last_peered":"2024-05-15T13:58:06.070290+0000","last_clean":"2024-05-15T13:58:06.070290+0000","last_became_active":"2024-05-15T13:57:14.255656+0000","last_became_peered":"2024-05-15T13:57:14.255656+0000","last_unstale":"2024-05-15T13:58:06.070290+0000","last_undegraded":"2024-05-15T13:58:06.070290+0000","last_fullsized":"2024-05-15T13:58:06.070290+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:14:46.880981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.999058+0000","last_change":"2024-05-15T13:57:17.389971+0000","last_active":"2024-05-15T13:59:04.999058+0000","last_peered":"2024-05-15T13:59:04.999058+0000","last_clean":"2024-05-15T13:59:04.999058+0000","last_became_active":"2024-05-15T13:57:17.389633+0000","last_became_peered":"2024-05-15T13:57:17.389633+0000","last_unstale":"2024-05-15T13:59:04.999058+0000","last_undegraded":"2024-05-15T13:59:04.999058+0000","last_fullsized":"2024-05-15T13:59:04.999058+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:25:10.971086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.070528+0000","last_change":"2024-05-15T13:57:14.258086+0000","last_active":"2024-05-15T13:58:06.070528+0000","last_peered":"2024-05-15T13:58:06.070528+0000","last_clean":"2024-05-15T13:58:06.070528+0000","last_became_active":"2024-05-15T13:57:14.257736+0000","last_became_peered":"2024-05-15T13:57:14.257736+0000","last_unstale":"2024-05-15T13:58:06.070528+0000","last_undegraded":"2024-05-15T13:58:06.070528+0000","last_fullsized":"2024-05-15T13:58:06.070528+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T00:51:07.431725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.998308+0000","last_change":"2024-05-15T13:58:06.335760+0000","last_active":"2024-05-15T13:59:04.998308+0000","last_peered":"2024-05-15T13:59:04.998308+0000","last_clean":"2024-05-15T13:59:04.998308+0000","last_became_active":"2024-05-15T13:58:06.335236+0000","last_became_peered":"2024-05-15T13:58:06.335236+0000","last_unstale":"2024-05-15T13:59:04.998308+0000","last_undegraded":"2024-05-15T13:59:04.998308+0000","last_fullsized":"2024-05-15T13:59:04.998308+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:04:47.007886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'5","reported_seq":132,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:10.510829+0000","last_change":"2024-05-15T13:57:14.261911+0000","last_active":"2024-05-15T13:59:10.510829+0000","last_peered":"2024-05-15T13:59:10.510829+0000","last_clean":"2024-05-15T13:59:10.510829+0000","last_became_active":"2024-05-15T13:57:14.261506+0000","last_became_peered":"2024-05-15T13:57:14.261506+0000","last_unstale":"2024-05-15T13:59:10.510829+0000","last_undegraded":"2024-05-15T13:59:10.510829+0000","last_fullsized":"2024-05-15T13:59:10.510829+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:18:21.021651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.997726+0000","last_change":"2024-05-15T13:57:17.395074+0000","last_active":"2024-05-15T13:59:04.997726+0000","last_peered":"2024-05-15T13:59:04.997726+0000","last_clean":"2024-05-15T13:59:04.997726+0000","last_became_active":"2024-05-15T13:57:17.394712+0000","last_became_peered":"2024-05-15T13:57:17.394712+0000","last_unstale":"2024-05-15T13:59:04.997726+0000","last_undegraded":"2024-05-15T13:59:04.997726+0000","last_fullsized":"2024-05-15T13:59:04.997726+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:58:25.385949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":66,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.086231+0000","last_change":"2024-05-15T13:58:06.086231+0000","last_active":"2024-05-15T13:58:06.086231+0000","last_peered":"2024-05-15T13:58:06.086231+0000","last_clean":"2024-05-15T13:58:06.086231+0000","last_became_active":"2024-05-15T13:58:06.074640+0000","last_became_peered":"2024-05-15T13:58:06.074640+0000","last_unstale":"2024-05-15T13:58:06.086231+0000","last_undegraded":"2024-05-15T13:58:06.086231+0000","last_fullsized":"2024-05-15T13:58:06.086231+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:41:54.747228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.991517+0000","last_change":"2024-05-15T13:57:17.376895+0000","last_active":"2024-05-15T13:59:04.991517+0000","last_peered":"2024-05-15T13:59:04.991517+0000","last_clean":"2024-05-15T13:59:04.991517+0000","last_became_active":"2024-05-15T13:57:17.376499+0000","last_became_peered":"2024-05-15T13:57:17.376499+0000","last_unstale":"2024-05-15T13:59:04.991517+0000","last_undegraded":"2024-05-15T13:59:04.991517+0000","last_fullsized":"2024-05-15T13:59:04.991517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:39:41.707151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.090458+0000","last_change":"2024-05-15T13:58:06.090051+0000","last_active":"2024-05-15T13:58:06.090458+0000","last_peered":"2024-05-15T13:58:06.090458+0000","last_clean":"2024-05-15T13:58:06.090458+0000","last_became_active":"2024-05-15T13:58:06.074177+0000","last_became_peered":"2024-05-15T13:58:06.074177+0000","last_unstale":"2024-05-15T13:58:06.090458+0000","last_undegraded":"2024-05-15T13:58:06.090458+0000","last_fullsized":"2024-05-15T13:58:06.090458+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:07:11.090910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.005415+0000","last_change":"2024-05-15T13:57:17.390640+0000","last_active":"2024-05-15T13:59:05.005415+0000","last_peered":"2024-05-15T13:59:05.005415+0000","last_clean":"2024-05-15T13:59:05.005415+0000","last_became_active":"2024-05-15T13:57:17.390293+0000","last_became_peered":"2024-05-15T13:57:17.390293+0000","last_unstale":"2024-05-15T13:59:05.005415+0000","last_undegraded":"2024-05-15T13:59:05.005415+0000","last_fullsized":"2024-05-15T13:59:05.005415+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:08:18.907391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.334128+0000","last_change":"2024-05-15T13:57:14.535321+0000","last_active":"2024-05-15T13:58:06.334128+0000","last_peered":"2024-05-15T13:58:06.334128+0000","last_clean":"2024-05-15T13:58:06.334128+0000","last_became_active":"2024-05-15T13:57:14.534794+0000","last_became_peered":"2024-05-15T13:57:14.534794+0000","last_unstale":"2024-05-15T13:58:06.334128+0000","last_undegraded":"2024-05-15T13:58:06.334128+0000","last_fullsized":"2024-05-15T13:58:06.334128+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:27:07.087671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.000076+0000","last_change":"2024-05-15T13:57:17.381249+0000","last_active":"2024-05-15T13:59:05.000076+0000","last_peered":"2024-05-15T13:59:05.000076+0000","last_clean":"2024-05-15T13:59:05.000076+0000","last_became_active":"2024-05-15T13:57:17.380474+0000","last_became_peered":"2024-05-15T13:57:17.380474+0000","last_unstale":"2024-05-15T13:59:05.000076+0000","last_undegraded":"2024-05-15T13:59:05.000076+0000","last_fullsized":"2024-05-15T13:59:05.000076+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:49:00.341085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.070193+0000","last_change":"2024-05-15T13:57:14.257635+0000","last_active":"2024-05-15T13:58:06.070193+0000","last_peered":"2024-05-15T13:58:06.070193+0000","last_clean":"2024-05-15T13:58:06.070193+0000","last_became_active":"2024-05-15T13:57:14.257156+0000","last_became_peered":"2024-05-15T13:57:14.257156+0000","last_unstale":"2024-05-15T13:58:06.070193+0000","last_undegraded":"2024-05-15T13:58:06.070193+0000","last_fullsized":"2024-05-15T13:58:06.070193+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:11:48.742053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.010544+0000","last_change":"2024-05-15T13:57:17.391067+0000","last_active":"2024-05-15T13:59:05.010544+0000","last_peered":"2024-05-15T13:59:05.010544+0000","last_clean":"2024-05-15T13:59:05.010544+0000","last_became_active":"2024-05-15T13:57:17.390763+0000","last_became_peered":"2024-05-15T13:57:17.390763+0000","last_unstale":"2024-05-15T13:59:05.010544+0000","last_undegraded":"2024-05-15T13:59:05.010544+0000","last_fullsized":"2024-05-15T13:59:05.010544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:28:46.506307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.994098+0000","last_change":"2024-05-15T13:57:17.391816+0000","last_active":"2024-05-15T13:59:04.994098+0000","last_peered":"2024-05-15T13:59:04.994098+0000","last_clean":"2024-05-15T13:59:04.994098+0000","last_became_active":"2024-05-15T13:57:17.389374+0000","last_became_peered":"2024-05-15T13:57:17.389374+0000","last_unstale":"2024-05-15T13:59:04.994098+0000","last_undegraded":"2024-05-15T13:59:04.994098+0000","last_fullsized":"2024-05-15T13:59:04.994098+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:48:31.756133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'5","reported_seq":124,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:06.504060+0000","last_change":"2024-05-15T13:57:14.534946+0000","last_active":"2024-05-15T13:59:06.504060+0000","last_peered":"2024-05-15T13:59:06.504060+0000","last_clean":"2024-05-15T13:59:06.504060+0000","last_became_active":"2024-05-15T13:57:14.534535+0000","last_became_peered":"2024-05-15T13:57:14.534535+0000","last_unstale":"2024-05-15T13:59:06.504060+0000","last_undegraded":"2024-05-15T13:59:06.504060+0000","last_fullsized":"2024-05-15T13:59:06.504060+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:27:20.526800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":93,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.001208+0000","last_change":"2024-05-15T13:57:17.391637+0000","last_active":"2024-05-15T13:59:05.001208+0000","last_peered":"2024-05-15T13:59:05.001208+0000","last_clean":"2024-05-15T13:59:05.001208+0000","last_became_active":"2024-05-15T13:57:17.391271+0000","last_became_peered":"2024-05-15T13:57:17.391271+0000","last_unstale":"2024-05-15T13:59:05.001208+0000","last_undegraded":"2024-05-15T13:59:05.001208+0000","last_fullsized":"2024-05-15T13:59:05.001208+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:22:06.398387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.070436+0000","last_change":"2024-05-15T13:57:14.258510+0000","last_active":"2024-05-15T13:58:06.070436+0000","last_peered":"2024-05-15T13:58:06.070436+0000","last_clean":"2024-05-15T13:58:06.070436+0000","last_became_active":"2024-05-15T13:57:14.257544+0000","last_became_peered":"2024-05-15T13:57:14.257544+0000","last_unstale":"2024-05-15T13:58:06.070436+0000","last_undegraded":"2024-05-15T13:58:06.070436+0000","last_fullsized":"2024-05-15T13:58:06.070436+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:12:27.253931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":100,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.070866+0000","last_change":"2024-05-15T13:55:53.940355+0000","last_active":"2024-05-15T13:58:06.070866+0000","last_peered":"2024-05-15T13:58:06.070866+0000","last_clean":"2024-05-15T13:58:06.070866+0000","last_became_active":"2024-05-15T13:55:53.934638+0000","last_became_peered":"2024-05-15T13:55:53.934638+0000","last_unstale":"2024-05-15T13:58:06.070866+0000","last_undegraded":"2024-05-15T13:58:06.070866+0000","last_fullsized":"2024-05-15T13:58:06.070866+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:52:33.556435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:52:33.556435+0000","last_clean_scrub_stamp":"2024-05-15T13:52:33.556435+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:02:55.332569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.994765+0000","last_change":"2024-05-15T13:57:17.382394+0000","last_active":"2024-05-15T13:59:04.994765+0000","last_peered":"2024-05-15T13:59:04.994765+0000","last_clean":"2024-05-15T13:59:04.994765+0000","last_became_active":"2024-05-15T13:57:17.382090+0000","last_became_peered":"2024-05-15T13:57:17.382090+0000","last_unstale":"2024-05-15T13:59:04.994765+0000","last_undegraded":"2024-05-15T13:59:04.994765+0000","last_fullsized":"2024-05-15T13:59:04.994765+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:01:34.653216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.071031+0000","last_change":"2024-05-15T13:57:14.261549+0000","last_active":"2024-05-15T13:58:06.071031+0000","last_peered":"2024-05-15T13:58:06.071031+0000","last_clean":"2024-05-15T13:58:06.071031+0000","last_became_active":"2024-05-15T13:57:14.261165+0000","last_became_peered":"2024-05-15T13:57:14.261165+0000","last_unstale":"2024-05-15T13:58:06.071031+0000","last_undegraded":"2024-05-15T13:58:06.071031+0000","last_fullsized":"2024-05-15T13:58:06.071031+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:04:39.574440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.071070+0000","last_change":"2024-05-15T13:57:14.535309+0000","last_active":"2024-05-15T13:58:06.071070+0000","last_peered":"2024-05-15T13:58:06.071070+0000","last_clean":"2024-05-15T13:58:06.071070+0000","last_became_active":"2024-05-15T13:57:14.534938+0000","last_became_peered":"2024-05-15T13:57:14.534938+0000","last_unstale":"2024-05-15T13:58:06.071070+0000","last_undegraded":"2024-05-15T13:58:06.071070+0000","last_fullsized":"2024-05-15T13:58:06.071070+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:34:53.373149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.993876+0000","last_change":"2024-05-15T13:57:17.388272+0000","last_active":"2024-05-15T13:59:04.993876+0000","last_peered":"2024-05-15T13:59:04.993876+0000","last_clean":"2024-05-15T13:59:04.993876+0000","last_became_active":"2024-05-15T13:57:17.387849+0000","last_became_peered":"2024-05-15T13:57:17.387849+0000","last_unstale":"2024-05-15T13:59:04.993876+0000","last_undegraded":"2024-05-15T13:59:04.993876+0000","last_fullsized":"2024-05-15T13:59:04.993876+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:47:47.323566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'12","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.334307+0000","last_change":"2024-05-15T13:57:14.256161+0000","last_active":"2024-05-15T13:58:06.334307+0000","last_peered":"2024-05-15T13:58:06.334307+0000","last_clean":"2024-05-15T13:58:06.334307+0000","last_became_active":"2024-05-15T13:57:14.255707+0000","last_became_peered":"2024-05-15T13:57:14.255707+0000","last_unstale":"2024-05-15T13:58:06.334307+0000","last_undegraded":"2024-05-15T13:58:06.334307+0000","last_fullsized":"2024-05-15T13:58:06.334307+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:24:57.200029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.007271+0000","last_change":"2024-05-15T13:57:17.389522+0000","last_active":"2024-05-15T13:59:05.007271+0000","last_peered":"2024-05-15T13:59:05.007271+0000","last_clean":"2024-05-15T13:59:05.007271+0000","last_became_active":"2024-05-15T13:57:17.389177+0000","last_became_peered":"2024-05-15T13:57:17.389177+0000","last_unstale":"2024-05-15T13:59:05.007271+0000","last_undegraded":"2024-05-15T13:59:05.007271+0000","last_fullsized":"2024-05-15T13:59:05.007271+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:23:35.716934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.070146+0000","last_change":"2024-05-15T13:57:14.254300+0000","last_active":"2024-05-15T13:58:06.070146+0000","last_peered":"2024-05-15T13:58:06.070146+0000","last_clean":"2024-05-15T13:58:06.070146+0000","last_became_active":"2024-05-15T13:57:14.253741+0000","last_became_peered":"2024-05-15T13:57:14.253741+0000","last_unstale":"2024-05-15T13:58:06.070146+0000","last_undegraded":"2024-05-15T13:58:06.070146+0000","last_fullsized":"2024-05-15T13:58:06.070146+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:25:34.018159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.009993+0000","last_change":"2024-05-15T13:57:17.380688+0000","last_active":"2024-05-15T13:59:05.009993+0000","last_peered":"2024-05-15T13:59:05.009993+0000","last_clean":"2024-05-15T13:59:05.009993+0000","last_became_active":"2024-05-15T13:57:17.380379+0000","last_became_peered":"2024-05-15T13:57:17.380379+0000","last_unstale":"2024-05-15T13:59:05.009993+0000","last_undegraded":"2024-05-15T13:59:05.009993+0000","last_fullsized":"2024-05-15T13:59:05.009993+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:32:09.190750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.251776+0000","last_change":"2024-05-15T13:57:14.534775+0000","last_active":"2024-05-15T13:58:06.251776+0000","last_peered":"2024-05-15T13:58:06.251776+0000","last_clean":"2024-05-15T13:58:06.251776+0000","last_became_active":"2024-05-15T13:57:14.534440+0000","last_became_peered":"2024-05-15T13:57:14.534440+0000","last_unstale":"2024-05-15T13:58:06.251776+0000","last_undegraded":"2024-05-15T13:58:06.251776+0000","last_fullsized":"2024-05-15T13:58:06.251776+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:21:49.047187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.961744+0000","last_change":"2024-05-15T13:57:17.396252+0000","last_active":"2024-05-15T13:59:04.961744+0000","last_peered":"2024-05-15T13:59:04.961744+0000","last_clean":"2024-05-15T13:59:04.961744+0000","last_became_active":"2024-05-15T13:57:17.395853+0000","last_became_peered":"2024-05-15T13:57:17.395853+0000","last_unstale":"2024-05-15T13:59:04.961744+0000","last_undegraded":"2024-05-15T13:59:04.961744+0000","last_fullsized":"2024-05-15T13:59:04.961744+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:12:54.924053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.071745+0000","last_change":"2024-05-15T13:57:14.258411+0000","last_active":"2024-05-15T13:58:06.071745+0000","last_peered":"2024-05-15T13:58:06.071745+0000","last_clean":"2024-05-15T13:58:06.071745+0000","last_became_active":"2024-05-15T13:57:14.257847+0000","last_became_peered":"2024-05-15T13:57:14.257847+0000","last_unstale":"2024-05-15T13:58:06.071745+0000","last_undegraded":"2024-05-15T13:58:06.071745+0000","last_fullsized":"2024-05-15T13:58:06.071745+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:49:32.368351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.010085+0000","last_change":"2024-05-15T13:57:17.388916+0000","last_active":"2024-05-15T13:59:05.010085+0000","last_peered":"2024-05-15T13:59:05.010085+0000","last_clean":"2024-05-15T13:59:05.010085+0000","last_became_active":"2024-05-15T13:57:17.388603+0000","last_became_peered":"2024-05-15T13:57:17.388603+0000","last_unstale":"2024-05-15T13:59:05.010085+0000","last_undegraded":"2024-05-15T13:59:05.010085+0000","last_fullsized":"2024-05-15T13:59:05.010085+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:38:46.650393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.070010+0000","last_change":"2024-05-15T13:57:14.256801+0000","last_active":"2024-05-15T13:58:06.070010+0000","last_peered":"2024-05-15T13:58:06.070010+0000","last_clean":"2024-05-15T13:58:06.070010+0000","last_became_active":"2024-05-15T13:57:14.256476+0000","last_became_peered":"2024-05-15T13:57:14.256476+0000","last_unstale":"2024-05-15T13:58:06.070010+0000","last_undegraded":"2024-05-15T13:58:06.070010+0000","last_fullsized":"2024-05-15T13:58:06.070010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:22:34.495756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.004392+0000","last_change":"2024-05-15T13:57:17.380927+0000","last_active":"2024-05-15T13:59:05.004392+0000","last_peered":"2024-05-15T13:59:05.004392+0000","last_clean":"2024-05-15T13:59:05.004392+0000","last_became_active":"2024-05-15T13:57:17.380602+0000","last_became_peered":"2024-05-15T13:57:17.380602+0000","last_unstale":"2024-05-15T13:59:05.004392+0000","last_undegraded":"2024-05-15T13:59:05.004392+0000","last_fullsized":"2024-05-15T13:59:05.004392+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:37:13.920968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.069626+0000","last_change":"2024-05-15T13:57:14.257954+0000","last_active":"2024-05-15T13:58:06.069626+0000","last_peered":"2024-05-15T13:58:06.069626+0000","last_clean":"2024-05-15T13:58:06.069626+0000","last_became_active":"2024-05-15T13:57:14.257601+0000","last_became_peered":"2024-05-15T13:57:14.257601+0000","last_unstale":"2024-05-15T13:58:06.069626+0000","last_undegraded":"2024-05-15T13:58:06.069626+0000","last_fullsized":"2024-05-15T13:58:06.069626+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:47:14.811264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.998424+0000","last_change":"2024-05-15T13:57:17.385227+0000","last_active":"2024-05-15T13:59:04.998424+0000","last_peered":"2024-05-15T13:59:04.998424+0000","last_clean":"2024-05-15T13:59:04.998424+0000","last_became_active":"2024-05-15T13:57:17.384899+0000","last_became_peered":"2024-05-15T13:57:17.384899+0000","last_unstale":"2024-05-15T13:59:04.998424+0000","last_undegraded":"2024-05-15T13:59:04.998424+0000","last_fullsized":"2024-05-15T13:59:04.998424+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:04:29.162392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.070663+0000","last_change":"2024-05-15T13:57:14.535266+0000","last_active":"2024-05-15T13:58:06.070663+0000","last_peered":"2024-05-15T13:58:06.070663+0000","last_clean":"2024-05-15T13:58:06.070663+0000","last_became_active":"2024-05-15T13:57:14.534892+0000","last_became_peered":"2024-05-15T13:57:14.534892+0000","last_unstale":"2024-05-15T13:58:06.070663+0000","last_undegraded":"2024-05-15T13:58:06.070663+0000","last_fullsized":"2024-05-15T13:58:06.070663+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:49:50.934517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.004659+0000","last_change":"2024-05-15T13:57:17.390330+0000","last_active":"2024-05-15T13:59:05.004659+0000","last_peered":"2024-05-15T13:59:05.004659+0000","last_clean":"2024-05-15T13:59:05.004659+0000","last_became_active":"2024-05-15T13:57:17.390000+0000","last_became_peered":"2024-05-15T13:57:17.390000+0000","last_unstale":"2024-05-15T13:59:05.004659+0000","last_undegraded":"2024-05-15T13:59:05.004659+0000","last_fullsized":"2024-05-15T13:59:05.004659+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:52:11.361549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.070763+0000","last_change":"2024-05-15T13:57:14.258174+0000","last_active":"2024-05-15T13:58:06.070763+0000","last_peered":"2024-05-15T13:58:06.070763+0000","last_clean":"2024-05-15T13:58:06.070763+0000","last_became_active":"2024-05-15T13:57:14.257763+0000","last_became_peered":"2024-05-15T13:57:14.257763+0000","last_unstale":"2024-05-15T13:58:06.070763+0000","last_undegraded":"2024-05-15T13:58:06.070763+0000","last_fullsized":"2024-05-15T13:58:06.070763+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:34:31.573845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.001456+0000","last_change":"2024-05-15T13:57:17.391964+0000","last_active":"2024-05-15T13:59:05.001456+0000","last_peered":"2024-05-15T13:59:05.001456+0000","last_clean":"2024-05-15T13:59:05.001456+0000","last_became_active":"2024-05-15T13:57:17.391517+0000","last_became_peered":"2024-05-15T13:57:17.391517+0000","last_unstale":"2024-05-15T13:59:05.001456+0000","last_undegraded":"2024-05-15T13:59:05.001456+0000","last_fullsized":"2024-05-15T13:59:05.001456+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:54:36.883563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:58:06.256119+0000","last_change":"2024-05-15T13:58:06.256119+0000","last_active":"2024-05-15T13:58:06.256119+0000","last_peered":"2024-05-15T13:58:06.256119+0000","last_clean":"2024-05-15T13:58:06.256119+0000","last_became_active":"2024-05-15T13:58:06.253495+0000","last_became_peered":"2024-05-15T13:58:06.253495+0000","last_unstale":"2024-05-15T13:58:06.256119+0000","last_undegraded":"2024-05-15T13:58:06.256119+0000","last_fullsized":"2024-05-15T13:58:06.256119+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:41:06.585181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.005403+0000","last_change":"2024-05-15T13:57:17.389518+0000","last_active":"2024-05-15T13:59:05.005403+0000","last_peered":"2024-05-15T13:59:05.005403+0000","last_clean":"2024-05-15T13:59:05.005403+0000","last_became_active":"2024-05-15T13:57:17.388905+0000","last_became_peered":"2024-05-15T13:57:17.388905+0000","last_unstale":"2024-05-15T13:59:05.005403+0000","last_undegraded":"2024-05-15T13:59:05.005403+0000","last_fullsized":"2024-05-15T13:59:05.005403+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:00:51.109880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.007113+0000","last_change":"2024-05-15T13:57:17.394127+0000","last_active":"2024-05-15T13:59:05.007113+0000","last_peered":"2024-05-15T13:59:05.007113+0000","last_clean":"2024-05-15T13:59:05.007113+0000","last_became_active":"2024-05-15T13:57:17.393771+0000","last_became_peered":"2024-05-15T13:57:17.393771+0000","last_unstale":"2024-05-15T13:59:05.007113+0000","last_undegraded":"2024-05-15T13:59:05.007113+0000","last_fullsized":"2024-05-15T13:59:05.007113+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:41:33.491001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.986850+0000","last_change":"2024-05-15T13:57:17.392451+0000","last_active":"2024-05-15T13:59:04.986850+0000","last_peered":"2024-05-15T13:59:04.986850+0000","last_clean":"2024-05-15T13:59:04.986850+0000","last_became_active":"2024-05-15T13:57:17.392099+0000","last_became_peered":"2024-05-15T13:57:17.392099+0000","last_unstale":"2024-05-15T13:59:04.986850+0000","last_undegraded":"2024-05-15T13:59:04.986850+0000","last_fullsized":"2024-05-15T13:59:04.986850+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:00:14.774543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.984896+0000","last_change":"2024-05-15T13:57:17.385525+0000","last_active":"2024-05-15T13:59:04.984896+0000","last_peered":"2024-05-15T13:59:04.984896+0000","last_clean":"2024-05-15T13:59:04.984896+0000","last_became_active":"2024-05-15T13:57:17.384497+0000","last_became_peered":"2024-05-15T13:57:17.384497+0000","last_unstale":"2024-05-15T13:59:04.984896+0000","last_undegraded":"2024-05-15T13:59:04.984896+0000","last_fullsized":"2024-05-15T13:59:04.984896+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:50:39.259339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.010099+0000","last_change":"2024-05-15T13:57:17.379673+0000","last_active":"2024-05-15T13:59:05.010099+0000","last_peered":"2024-05-15T13:59:05.010099+0000","last_clean":"2024-05-15T13:59:05.010099+0000","last_became_active":"2024-05-15T13:57:17.379342+0000","last_became_peered":"2024-05-15T13:57:17.379342+0000","last_unstale":"2024-05-15T13:59:05.010099+0000","last_undegraded":"2024-05-15T13:59:05.010099+0000","last_fullsized":"2024-05-15T13:59:05.010099+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:26:43.982972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.990507+0000","last_change":"2024-05-15T13:58:06.335360+0000","last_active":"2024-05-15T13:59:04.990507+0000","last_peered":"2024-05-15T13:59:04.990507+0000","last_clean":"2024-05-15T13:59:04.990507+0000","last_became_active":"2024-05-15T13:58:06.334899+0000","last_became_peered":"2024-05-15T13:58:06.334899+0000","last_unstale":"2024-05-15T13:59:04.990507+0000","last_undegraded":"2024-05-15T13:59:04.990507+0000","last_fullsized":"2024-05-15T13:59:04.990507+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:34:27.081929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.005225+0000","last_change":"2024-05-15T13:57:17.390939+0000","last_active":"2024-05-15T13:59:05.005225+0000","last_peered":"2024-05-15T13:59:05.005225+0000","last_clean":"2024-05-15T13:59:05.005225+0000","last_became_active":"2024-05-15T13:57:17.390281+0000","last_became_peered":"2024-05-15T13:57:17.390281+0000","last_unstale":"2024-05-15T13:59:05.005225+0000","last_undegraded":"2024-05-15T13:59:05.005225+0000","last_fullsized":"2024-05-15T13:59:05.005225+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:02:22.077307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.989944+0000","last_change":"2024-05-15T13:57:17.392009+0000","last_active":"2024-05-15T13:59:04.989944+0000","last_peered":"2024-05-15T13:59:04.989944+0000","last_clean":"2024-05-15T13:59:04.989944+0000","last_became_active":"2024-05-15T13:57:17.391059+0000","last_became_peered":"2024-05-15T13:57:17.391059+0000","last_unstale":"2024-05-15T13:59:04.989944+0000","last_undegraded":"2024-05-15T13:59:04.989944+0000","last_fullsized":"2024-05-15T13:59:04.989944+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:51:46.787790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.007690+0000","last_change":"2024-05-15T13:57:17.383872+0000","last_active":"2024-05-15T13:59:05.007690+0000","last_peered":"2024-05-15T13:59:05.007690+0000","last_clean":"2024-05-15T13:59:05.007690+0000","last_became_active":"2024-05-15T13:57:17.383526+0000","last_became_peered":"2024-05-15T13:57:17.383526+0000","last_unstale":"2024-05-15T13:59:05.007690+0000","last_undegraded":"2024-05-15T13:59:05.007690+0000","last_fullsized":"2024-05-15T13:59:05.007690+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:43:39.343557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.000454+0000","last_change":"2024-05-15T13:57:17.377530+0000","last_active":"2024-05-15T13:59:05.000454+0000","last_peered":"2024-05-15T13:59:05.000454+0000","last_clean":"2024-05-15T13:59:05.000454+0000","last_became_active":"2024-05-15T13:57:17.377237+0000","last_became_peered":"2024-05-15T13:57:17.377237+0000","last_unstale":"2024-05-15T13:59:05.000454+0000","last_undegraded":"2024-05-15T13:59:05.000454+0000","last_fullsized":"2024-05-15T13:59:05.000454+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:29:02.173018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.007984+0000","last_change":"2024-05-15T13:57:17.386208+0000","last_active":"2024-05-15T13:59:05.007984+0000","last_peered":"2024-05-15T13:59:05.007984+0000","last_clean":"2024-05-15T13:59:05.007984+0000","last_became_active":"2024-05-15T13:57:17.385853+0000","last_became_peered":"2024-05-15T13:57:17.385853+0000","last_unstale":"2024-05-15T13:59:05.007984+0000","last_undegraded":"2024-05-15T13:59:05.007984+0000","last_fullsized":"2024-05-15T13:59:05.007984+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:17:06.124653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.006241+0000","last_change":"2024-05-15T13:57:17.390074+0000","last_active":"2024-05-15T13:59:05.006241+0000","last_peered":"2024-05-15T13:59:05.006241+0000","last_clean":"2024-05-15T13:59:05.006241+0000","last_became_active":"2024-05-15T13:57:17.389765+0000","last_became_peered":"2024-05-15T13:57:17.389765+0000","last_unstale":"2024-05-15T13:59:05.006241+0000","last_undegraded":"2024-05-15T13:59:05.006241+0000","last_fullsized":"2024-05-15T13:59:05.006241+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:49:51.321180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:04.999666+0000","last_change":"2024-05-15T13:57:17.389180+0000","last_active":"2024-05-15T13:59:04.999666+0000","last_peered":"2024-05-15T13:59:04.999666+0000","last_clean":"2024-05-15T13:59:04.999666+0000","last_became_active":"2024-05-15T13:57:17.388731+0000","last_became_peered":"2024-05-15T13:57:17.388731+0000","last_unstale":"2024-05-15T13:59:04.999666+0000","last_undegraded":"2024-05-15T13:59:04.999666+0000","last_fullsized":"2024-05-15T13:59:04.999666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:19:00.655474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-15T13:59:05.004125+0000","last_change":"2024-05-15T13:57:17.394045+0000","last_active":"2024-05-15T13:59:05.004125+0000","last_peered":"2024-05-15T13:59:05.004125+0000","last_clean":"2024-05-15T13:59:05.004125+0000","last_became_active":"2024-05-15T13:57:17.393699+0000","last_became_peered":"2024-05-15T13:57:17.393699+0000","last_unstale":"2024-05-15T13:59:05.004125+0000","last_undegraded":"2024-05-15T13:59:05.004125+0000","last_fullsized":"2024-05-15T13:59:05.004125+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:05:37.510370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65270,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":605,"num_read_kb":369,"num_write":173,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":134,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":195810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645350,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":897024,"data_stored":432784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841579,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36888,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708264,"statfs":{"total":95995035648,"available":95957262336,"internally_reserved":0,"allocated":1589248,"data_stored":1085371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005104,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":909312,"data_stored":432998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201333,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":905216,"data_stored":432896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397562,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28484,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716668,"statfs":{"total":95995035648,"available":95965868032,"internally_reserved":0,"allocated":1503232,"data_stored":1025521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561088,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36212,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708940,"statfs":{"total":95995035648,"available":95957954560,"internally_reserved":0,"allocated":897024,"data_stored":432387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":492776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36232,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708920,"statfs":{"total":95995035648,"available":95957934080,"internally_reserved":0,"allocated":917504,"data_stored":433658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":494546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149774,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":432698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953546,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":433468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757317,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36816,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708336,"statfs":{"total":95995035648,"available":95957336064,"internally_reserved":0,"allocated":1515520,"data_stored":1024673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":62366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-15T13:59:14.505 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-15T13:59:14.505 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-15T13:59:14.505 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-15T13:59:14.505 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph health --format=json 2024-05-15T13:59:14.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:14 smithi052 ceph-mon[36190]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-15T13:59:14.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:14 smithi081 ceph-mon[36434]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-15T13:59:14.759 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:59:15.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:15 smithi033 ceph-mon[31650]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-15T13:59:15.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:15 smithi052 ceph-mon[36190]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-15T13:59:15.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:15 smithi081 ceph-mon[36434]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-15T13:59:16.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.370+0000 7f6983098700 1 -- 172.21.15.33:0/427417250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c1041a0 msgr2=0x7f697c1081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:16.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.370+0000 7f6983098700 1 --2- 172.21.15.33:0/427417250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c1041a0 0x7f697c1081c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f69680099c0 tx=0x7f6968009cd0 comp rx=0 tx=0).stop 2024-05-15T13:59:16.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.370+0000 7f6983098700 1 -- 172.21.15.33:0/427417250 shutdown_connections 2024-05-15T13:59:16.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.370+0000 7f6983098700 1 --2- 172.21.15.33:0/427417250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f697c1088f0 0x7f697c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:16.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.370+0000 7f6983098700 1 --2- 172.21.15.33:0/427417250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f697c1037f0 0x7f697c103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:16.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.370+0000 7f6983098700 1 --2- 172.21.15.33:0/427417250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c1041a0 0x7f697c1081c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:16.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.370+0000 7f6983098700 1 -- 172.21.15.33:0/427417250 >> 172.21.15.33:0/427417250 conn(0x7f697c07a6a0 msgr2=0x7f697c07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:16.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.370+0000 7f6983098700 1 -- 172.21.15.33:0/427417250 shutdown_connections 2024-05-15T13:59:16.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.370+0000 7f6983098700 1 -- 172.21.15.33:0/427417250 wait complete. 2024-05-15T13:59:16.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.371+0000 7f6983098700 1 Processor -- start 2024-05-15T13:59:16.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.371+0000 7f6983098700 1 -- start start 2024-05-15T13:59:16.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.371+0000 7f6983098700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f697c1037f0 0x7f697c075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:16.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.371+0000 7f6983098700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f697c1041a0 0x7f697c075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:16.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.371+0000 7f6983098700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c1088f0 0x7f697c0762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:16.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.371+0000 7f6983098700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f697c113730 con 0x7f697c1088f0 2024-05-15T13:59:16.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.371+0000 7f6983098700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f697c1135b0 con 0x7f697c1041a0 2024-05-15T13:59:16.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.371+0000 7f6983098700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f697c1138b0 con 0x7f697c1037f0 2024-05-15T13:59:16.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.371+0000 7f6981635700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c1088f0 0x7f697c0762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:16.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.371+0000 7f6981635700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c1088f0 0x7f697c0762c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55828/0 (socket says 172.21.15.33:55828) 2024-05-15T13:59:16.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.371+0000 7f6981635700 1 -- 172.21.15.33:0/2997053606 learned_addr learned my addr 172.21.15.33:0/2997053606 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:16.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.371+0000 7f6980e34700 1 --2- 172.21.15.33:0/2997053606 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f697c1037f0 0x7f697c075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:16.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.371+0000 7f697bfff700 1 --2- 172.21.15.33:0/2997053606 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f697c1041a0 0x7f697c075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:16.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.371+0000 7f6981635700 1 -- 172.21.15.33:0/2997053606 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f697c1037f0 msgr2=0x7f697c075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:16.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.371+0000 7f6981635700 1 --2- 172.21.15.33:0/2997053606 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f697c1037f0 0x7f697c075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:16.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.371+0000 7f6981635700 1 -- 172.21.15.33:0/2997053606 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f697c1041a0 msgr2=0x7f697c075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:16.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.371+0000 7f6981635700 1 --2- 172.21.15.33:0/2997053606 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f697c1041a0 0x7f697c075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:16.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.372+0000 7f6981635700 1 -- 172.21.15.33:0/2997053606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f697c1a80b0 con 0x7f697c1088f0 2024-05-15T13:59:16.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.372+0000 7f697bfff700 1 --2- 172.21.15.33:0/2997053606 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f697c1041a0 0x7f697c075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T13:59:16.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.372+0000 7f6981635700 1 --2- 172.21.15.33:0/2997053606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c1088f0 0x7f697c0762c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f697400b970 tx=0x7f697400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:16.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.372+0000 7f6979ffb700 1 -- 172.21.15.33:0/2997053606 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974019070 con 0x7f697c1088f0 2024-05-15T13:59:16.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.372+0000 7f6979ffb700 1 -- 172.21.15.33:0/2997053606 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f697400cde0 con 0x7f697c1088f0 2024-05-15T13:59:16.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.372+0000 7f6983098700 1 -- 172.21.15.33:0/2997053606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f697c1a83a0 con 0x7f697c1088f0 2024-05-15T13:59:16.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.372+0000 7f6979ffb700 1 -- 172.21.15.33:0/2997053606 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974007370 con 0x7f697c1088f0 2024-05-15T13:59:16.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.372+0000 7f6983098700 1 -- 172.21.15.33:0/2997053606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f697c1a88b0 con 0x7f697c1088f0 2024-05-15T13:59:16.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.373+0000 7f6983098700 1 -- 172.21.15.33:0/2997053606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f697c06a230 con 0x7f697c1088f0 2024-05-15T13:59:16.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.374+0000 7f6979ffb700 1 -- 172.21.15.33:0/2997053606 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f697401c070 con 0x7f697c1088f0 2024-05-15T13:59:16.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.375+0000 7f6979ffb700 1 --2- 172.21.15.33:0/2997053606 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f696c0b2900 0x7f696c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:16.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.375+0000 7f6980e34700 1 --2- 172.21.15.33:0/2997053606 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f696c0b2900 0x7f696c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:16.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.375+0000 7f6979ffb700 1 -- 172.21.15.33:0/2997053606 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f69740e16c0 con 0x7f697c1088f0 2024-05-15T13:59:16.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.375+0000 7f6980e34700 1 --2- 172.21.15.33:0/2997053606 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f696c0b2900 0x7f696c0b4dc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f697000ac50 tx=0x7f697000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:16.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.380+0000 7f6979ffb700 1 -- 172.21.15.33:0/2997053606 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6974051df0 con 0x7f697c1088f0 2024-05-15T13:59:16.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:16 smithi033 ceph-mon[31650]: pgmap v347: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-15T13:59:16.492 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:16 smithi052 ceph-mon[36190]: pgmap v347: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-15T13:59:16.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:16 smithi081 ceph-mon[36434]: pgmap v347: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-15T13:59:16.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.640+0000 7f6983098700 1 -- 172.21.15.33:0/2997053606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f697c02e560 con 0x7f697c1088f0 2024-05-15T13:59:16.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.640+0000 7f6979ffb700 1 -- 172.21.15.33:0/2997053606 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f69740a93e0 con 0x7f697c1088f0 2024-05-15T13:59:16.642 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:59:16.642 INFO:teuthology.orchestra.run.smithi033.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-15T13:59:16.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.642+0000 7f6983098700 1 -- 172.21.15.33:0/2997053606 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f696c0b2900 msgr2=0x7f696c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:16.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.642+0000 7f6983098700 1 --2- 172.21.15.33:0/2997053606 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f696c0b2900 0x7f696c0b4dc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f697000ac50 tx=0x7f697000a380 comp rx=0 tx=0).stop 2024-05-15T13:59:16.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.643+0000 7f6983098700 1 -- 172.21.15.33:0/2997053606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c1088f0 msgr2=0x7f697c0762c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:16.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.643+0000 7f6983098700 1 --2- 172.21.15.33:0/2997053606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c1088f0 0x7f697c0762c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f697400b970 tx=0x7f697400bc80 comp rx=0 tx=0).stop 2024-05-15T13:59:16.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.643+0000 7f6983098700 1 -- 172.21.15.33:0/2997053606 shutdown_connections 2024-05-15T13:59:16.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.643+0000 7f6983098700 1 --2- 172.21.15.33:0/2997053606 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f696c0b2900 0x7f696c0b4dc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:16.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.643+0000 7f6983098700 1 --2- 172.21.15.33:0/2997053606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f697c1088f0 0x7f697c0762c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:16.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.643+0000 7f6983098700 1 --2- 172.21.15.33:0/2997053606 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f697c1037f0 0x7f697c075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:16.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.643+0000 7f6983098700 1 --2- 172.21.15.33:0/2997053606 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f697c1041a0 0x7f697c075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:16.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.643+0000 7f6983098700 1 -- 172.21.15.33:0/2997053606 >> 172.21.15.33:0/2997053606 conn(0x7f697c07a6a0 msgr2=0x7f697c102d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:16.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.643+0000 7f6983098700 1 -- 172.21.15.33:0/2997053606 shutdown_connections 2024-05-15T13:59:16.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:16.643+0000 7f6983098700 1 -- 172.21.15.33:0/2997053606 wait complete. 2024-05-15T13:59:17.237 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:17 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/2997053606' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-15T13:59:17.265 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-15T13:59:17.266 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T13:59:17.518 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:59:17.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:17 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/2997053606' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-15T13:59:17.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:17 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/2997053606' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-15T13:59:18.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:18 smithi033 ceph-mon[31650]: pgmap v348: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-15T13:59:18.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:18 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:59:18.484 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:18 smithi081 ceph-mon[36434]: pgmap v348: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-15T13:59:18.484 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:18 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:59:18.515 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:18 smithi052 ceph-mon[36190]: pgmap v348: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-15T13:59:18.516 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:18 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T13:59:18.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:18.996+0000 7ff160382700 1 -- 172.21.15.33:0/1689834748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff158102440 msgr2=0x7ff1581028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:18.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:18.996+0000 7ff160382700 1 --2- 172.21.15.33:0/1689834748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff158102440 0x7ff1581028c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff148009990 tx=0x7ff148009ca0 comp rx=0 tx=0).stop 2024-05-15T13:59:18.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:18.996+0000 7ff160382700 1 -- 172.21.15.33:0/1689834748 shutdown_connections 2024-05-15T13:59:18.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:18.996+0000 7ff160382700 1 --2- 172.21.15.33:0/1689834748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff158102e00 0x7ff15810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:18.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:18.996+0000 7ff160382700 1 --2- 172.21.15.33:0/1689834748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff158108520 0x7ff158108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:18.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:18.996+0000 7ff160382700 1 --2- 172.21.15.33:0/1689834748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff158102440 0x7ff1581028c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:18.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:18.996+0000 7ff160382700 1 -- 172.21.15.33:0/1689834748 >> 172.21.15.33:0/1689834748 conn(0x7ff1580793f0 msgr2=0x7ff158079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:18.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:18.996+0000 7ff160382700 1 -- 172.21.15.33:0/1689834748 shutdown_connections 2024-05-15T13:59:18.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:18.996+0000 7ff160382700 1 -- 172.21.15.33:0/1689834748 wait complete. 2024-05-15T13:59:19.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:18.997+0000 7ff160382700 1 Processor -- start 2024-05-15T13:59:19.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:18.997+0000 7ff160382700 1 -- start start 2024-05-15T13:59:19.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:18.998+0000 7ff160382700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff158102440 0x7ff15819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:19.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:18.998+0000 7ff160382700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff158102e00 0x7ff15819dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:19.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:18.998+0000 7ff160382700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff158108520 0x7ff158197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:19.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:18.998+0000 7ff160382700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff158111560 con 0x7ff158102440 2024-05-15T13:59:19.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:18.998+0000 7ff160382700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff1581113e0 con 0x7ff158108520 2024-05-15T13:59:19.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:18.998+0000 7ff160382700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ff1581116e0 con 0x7ff158102e00 2024-05-15T13:59:19.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:18.998+0000 7ff15e11e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff158102440 0x7ff15819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:19.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:18.998+0000 7ff15e11e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff158102440 0x7ff15819d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55856/0 (socket says 172.21.15.33:55856) 2024-05-15T13:59:19.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:18.998+0000 7ff15e11e700 1 -- 172.21.15.33:0/2044590869 learned_addr learned my addr 172.21.15.33:0/2044590869 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:19.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:18.998+0000 7ff15d91d700 1 --2- 172.21.15.33:0/2044590869 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff158102e00 0x7ff15819dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:19.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:18.998+0000 7ff15e91f700 1 --2- 172.21.15.33:0/2044590869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff158108520 0x7ff158197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:19.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:18.998+0000 7ff15e11e700 1 -- 172.21.15.33:0/2044590869 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff158102e00 msgr2=0x7ff15819dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:19.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:18.998+0000 7ff15e11e700 1 --2- 172.21.15.33:0/2044590869 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff158102e00 0x7ff15819dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:19.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:18.998+0000 7ff15e11e700 1 -- 172.21.15.33:0/2044590869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff158108520 msgr2=0x7ff158197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:19.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:18.998+0000 7ff15e11e700 1 --2- 172.21.15.33:0/2044590869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff158108520 0x7ff158197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:19.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:18.998+0000 7ff15e11e700 1 -- 172.21.15.33:0/2044590869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff158198060 con 0x7ff158102440 2024-05-15T13:59:19.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:18.998+0000 7ff15e11e700 1 --2- 172.21.15.33:0/2044590869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff158102440 0x7ff15819d660 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff154009a20 tx=0x7ff154009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:19.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:18.999+0000 7ff14f7fe700 1 -- 172.21.15.33:0/2044590869 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff154004830 con 0x7ff158102440 2024-05-15T13:59:19.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:18.999+0000 7ff14f7fe700 1 -- 172.21.15.33:0/2044590869 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff154005020 con 0x7ff158102440 2024-05-15T13:59:19.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:18.999+0000 7ff14f7fe700 1 -- 172.21.15.33:0/2044590869 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15400c9c0 con 0x7ff158102440 2024-05-15T13:59:19.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:18.999+0000 7ff160382700 1 -- 172.21.15.33:0/2044590869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff158198350 con 0x7ff158102440 2024-05-15T13:59:19.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:18.999+0000 7ff160382700 1 -- 172.21.15.33:0/2044590869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1581a45e0 con 0x7ff158102440 2024-05-15T13:59:19.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:19.000+0000 7ff160382700 1 -- 172.21.15.33:0/2044590869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff15810a9b0 con 0x7ff158102440 2024-05-15T13:59:19.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:19.002+0000 7ff14f7fe700 1 -- 172.21.15.33:0/2044590869 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff1540054d0 con 0x7ff158102440 2024-05-15T13:59:19.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:19.003+0000 7ff14f7fe700 1 --2- 172.21.15.33:0/2044590869 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff1440b2900 0x7ff1440b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:19.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:19.003+0000 7ff15d91d700 1 --2- 172.21.15.33:0/2044590869 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff1440b2900 0x7ff1440b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:19.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:19.003+0000 7ff14f7fe700 1 -- 172.21.15.33:0/2044590869 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7ff1540e1dd0 con 0x7ff158102440 2024-05-15T13:59:19.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:19.003+0000 7ff15d91d700 1 --2- 172.21.15.33:0/2044590869 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff1440b2900 0x7ff1440b4dc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff158198f80 tx=0x7ff148005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:19.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:19.006+0000 7ff14f7fe700 1 -- 172.21.15.33:0/2044590869 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff1540a9af0 con 0x7ff158102440 2024-05-15T13:59:19.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:19.305+0000 7ff160382700 1 -- 172.21.15.33:0/2044590869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff158103b60 con 0x7ff158102440 2024-05-15T13:59:19.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:19.306+0000 7ff14f7fe700 1 -- 172.21.15.33:0/2044590869 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11858 (secure 0 0 0) 0x7ff1540a93f0 con 0x7ff158102440 2024-05-15T13:59:19.308 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:59:19.308 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2030117633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2030117633},{"type":"v1","addr":"172.21.15.33:6839","nonce":2030117633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1918979061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1918979061},{"type":"v1","addr":"172.21.15.33:6841","nonce":1918979061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T13:57:19.056417+0000","modified":"2024-05-15T13:57:38.201184+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24463,"mds_2":24461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/1629697841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1629697841},{"type":"v1","addr":"172.21.15.52:6837","nonce":1629697841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/3773839758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3773839758},{"type":"v1","addr":"172.21.15.81:6835","nonce":3773839758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.52:6839/2696044471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2696044471},{"type":"v1","addr":"172.21.15.52:6839","nonce":2696044471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.81:6839/1554395940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":1554395940},{"type":"v1","addr":"172.21.15.81:6839","nonce":1554395940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24478,24461,24463]},"id":1}]} 2024-05-15T13:59:19.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:19.309+0000 7ff160382700 1 -- 172.21.15.33:0/2044590869 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff1440b2900 msgr2=0x7ff1440b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:19.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:19.309+0000 7ff160382700 1 --2- 172.21.15.33:0/2044590869 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff1440b2900 0x7ff1440b4dc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff158198f80 tx=0x7ff148005fb0 comp rx=0 tx=0).stop 2024-05-15T13:59:19.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:19.309+0000 7ff160382700 1 -- 172.21.15.33:0/2044590869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff158102440 msgr2=0x7ff15819d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:19.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:19.309+0000 7ff160382700 1 --2- 172.21.15.33:0/2044590869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff158102440 0x7ff15819d660 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff154009a20 tx=0x7ff154009d30 comp rx=0 tx=0).stop 2024-05-15T13:59:19.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:19.309+0000 7ff160382700 1 -- 172.21.15.33:0/2044590869 shutdown_connections 2024-05-15T13:59:19.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:19.309+0000 7ff160382700 1 --2- 172.21.15.33:0/2044590869 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7ff1440b2900 0x7ff1440b4dc0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:19.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:19.309+0000 7ff160382700 1 --2- 172.21.15.33:0/2044590869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff158108520 0x7ff158197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:19.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:19.309+0000 7ff160382700 1 --2- 172.21.15.33:0/2044590869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff158102440 0x7ff15819d660 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:19.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:19.309+0000 7ff160382700 1 --2- 172.21.15.33:0/2044590869 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff158102e00 0x7ff15819dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:19.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:19.309+0000 7ff160382700 1 -- 172.21.15.33:0/2044590869 >> 172.21.15.33:0/2044590869 conn(0x7ff1580793f0 msgr2=0x7ff158100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:19.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:19.309+0000 7ff160382700 1 -- 172.21.15.33:0/2044590869 shutdown_connections 2024-05-15T13:59:19.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:19.309+0000 7ff160382700 1 -- 172.21.15.33:0/2044590869 wait complete. 2024-05-15T13:59:19.311 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 23 2024-05-15T13:59:20.340 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T13:59:20.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:20 smithi052 ceph-mon[36190]: pgmap v349: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-15T13:59:20.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:20 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:59:20.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:20 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/2044590869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:59:20.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:20 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:59:20.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:20 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:59:20.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:20 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:59:20.606 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:59:20.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:20 smithi081 ceph-mon[36434]: pgmap v349: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-15T13:59:20.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:20 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:59:20.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:20 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/2044590869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:59:20.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:20 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:59:20.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:20 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:59:20.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:20 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:59:20.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:20 smithi033 ceph-mon[31650]: pgmap v349: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-15T13:59:20.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:20 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:59:20.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:20 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/2044590869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:59:20.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:20 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:59:20.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:20 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:59:20.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:20 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:59:22.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.219+0000 7f59d12dd700 1 -- 172.21.15.33:0/2601551374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59cc1088f0 msgr2=0x7f59cc110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:22.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.219+0000 7f59d12dd700 1 --2- 172.21.15.33:0/2601551374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59cc1088f0 0x7f59cc110470 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f59c400b0d0 tx=0x7f59c400b3e0 comp rx=0 tx=0).stop 2024-05-15T13:59:22.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.219+0000 7f59d12dd700 1 -- 172.21.15.33:0/2601551374 shutdown_connections 2024-05-15T13:59:22.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.219+0000 7f59d12dd700 1 --2- 172.21.15.33:0/2601551374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59cc1088f0 0x7f59cc110470 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:22.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.219+0000 7f59d12dd700 1 --2- 172.21.15.33:0/2601551374 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59cc1037f0 0x7f59cc103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:22.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.219+0000 7f59d12dd700 1 --2- 172.21.15.33:0/2601551374 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59cc1041a0 0x7f59cc1081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:22.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.219+0000 7f59d12dd700 1 -- 172.21.15.33:0/2601551374 >> 172.21.15.33:0/2601551374 conn(0x7f59cc07a6a0 msgr2=0x7f59cc07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:22.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.219+0000 7f59d12dd700 1 -- 172.21.15.33:0/2601551374 shutdown_connections 2024-05-15T13:59:22.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.219+0000 7f59d12dd700 1 -- 172.21.15.33:0/2601551374 wait complete. 2024-05-15T13:59:22.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.220+0000 7f59d12dd700 1 Processor -- start 2024-05-15T13:59:22.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.220+0000 7f59d12dd700 1 -- start start 2024-05-15T13:59:22.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.220+0000 7f59d12dd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59cc1037f0 0x7f59cc19f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:22.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.220+0000 7f59d12dd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59cc1041a0 0x7f59cc19fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:22.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.220+0000 7f59d12dd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59cc1088f0 0x7f59cc199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:22.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.221+0000 7f59d12dd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc113730 con 0x7f59cc1041a0 2024-05-15T13:59:22.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.221+0000 7f59d12dd700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc1135b0 con 0x7f59cc1088f0 2024-05-15T13:59:22.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.221+0000 7f59d12dd700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc1138b0 con 0x7f59cc1037f0 2024-05-15T13:59:22.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.221+0000 7f59ca7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59cc1041a0 0x7f59cc19fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:22.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.221+0000 7f59caffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59cc1037f0 0x7f59cc19f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:22.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.221+0000 7f59ca7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59cc1041a0 0x7f59cc19fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55874/0 (socket says 172.21.15.33:55874) 2024-05-15T13:59:22.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.221+0000 7f59ca7fc700 1 -- 172.21.15.33:0/1117345369 learned_addr learned my addr 172.21.15.33:0/1117345369 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:22.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.221+0000 7f59cb7fe700 1 --2- 172.21.15.33:0/1117345369 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59cc1088f0 0x7f59cc199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:22.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.221+0000 7f59ca7fc700 1 -- 172.21.15.33:0/1117345369 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59cc1037f0 msgr2=0x7f59cc19f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:22.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.221+0000 7f59ca7fc700 1 --2- 172.21.15.33:0/1117345369 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59cc1037f0 0x7f59cc19f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:22.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.221+0000 7f59ca7fc700 1 -- 172.21.15.33:0/1117345369 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59cc1088f0 msgr2=0x7f59cc199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:22.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.221+0000 7f59ca7fc700 1 --2- 172.21.15.33:0/1117345369 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59cc1088f0 0x7f59cc199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:22.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.221+0000 7f59ca7fc700 1 -- 172.21.15.33:0/1117345369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59cc19a0d0 con 0x7f59cc1041a0 2024-05-15T13:59:22.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.221+0000 7f59caffd700 1 --2- 172.21.15.33:0/1117345369 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59cc1037f0 0x7f59cc19f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T13:59:22.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.221+0000 7f59ca7fc700 1 --2- 172.21.15.33:0/1117345369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59cc1041a0 0x7f59cc19fcf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f59c000b530 tx=0x7f59c000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:22.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.221+0000 7f59b3fff700 1 -- 172.21.15.33:0/1117345369 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c0015020 con 0x7f59cc1041a0 2024-05-15T13:59:22.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.222+0000 7f59d12dd700 1 -- 172.21.15.33:0/1117345369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59cc19a3c0 con 0x7f59cc1041a0 2024-05-15T13:59:22.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.222+0000 7f59b3fff700 1 -- 172.21.15.33:0/1117345369 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59c0011ce0 con 0x7f59cc1041a0 2024-05-15T13:59:22.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.222+0000 7f59b3fff700 1 -- 172.21.15.33:0/1117345369 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c0010dd0 con 0x7f59cc1041a0 2024-05-15T13:59:22.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.222+0000 7f59d12dd700 1 -- 172.21.15.33:0/1117345369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59cc1a6680 con 0x7f59cc1041a0 2024-05-15T13:59:22.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.223+0000 7f59d12dd700 1 -- 172.21.15.33:0/1117345369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59cc06a230 con 0x7f59cc1041a0 2024-05-15T13:59:22.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.224+0000 7f59b3fff700 1 -- 172.21.15.33:0/1117345369 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f59c0018020 con 0x7f59cc1041a0 2024-05-15T13:59:22.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.225+0000 7f59b3fff700 1 --2- 172.21.15.33:0/1117345369 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f59b40b2900 0x7f59b40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:22.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.225+0000 7f59caffd700 1 --2- 172.21.15.33:0/1117345369 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f59b40b2900 0x7f59b40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:22.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.225+0000 7f59b3fff700 1 -- 172.21.15.33:0/1117345369 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f59c00e1ce0 con 0x7f59cc1041a0 2024-05-15T13:59:22.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.226+0000 7f59caffd700 1 --2- 172.21.15.33:0/1117345369 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f59b40b2900 0x7f59b40b4dc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f59cc19aab0 tx=0x7f59bc0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:22.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.228+0000 7f59b3fff700 1 -- 172.21.15.33:0/1117345369 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f59c001f030 con 0x7f59cc1041a0 2024-05-15T13:59:22.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.493+0000 7f59d12dd700 1 -- 172.21.15.33:0/1117345369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59cc02e560 con 0x7f59cc1041a0 2024-05-15T13:59:22.495 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:22 smithi033 ceph-mon[31650]: pgmap v350: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T13:59:22.496 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:22 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:59:22.496 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:22 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:59:22.496 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:22 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:59:22.496 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:22 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:59:22.496 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:22 smithi033 ceph-mon[31650]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:59:22.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.493+0000 7f59b3fff700 1 -- 172.21.15.33:0/1117345369 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11858 (secure 0 0 0) 0x7f59c00a99c0 con 0x7f59cc1041a0 2024-05-15T13:59:22.496 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:59:22.497 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2030117633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2030117633},{"type":"v1","addr":"172.21.15.33:6839","nonce":2030117633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1918979061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1918979061},{"type":"v1","addr":"172.21.15.33:6841","nonce":1918979061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T13:57:19.056417+0000","modified":"2024-05-15T13:57:38.201184+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24463,"mds_2":24461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/1629697841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1629697841},{"type":"v1","addr":"172.21.15.52:6837","nonce":1629697841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/3773839758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3773839758},{"type":"v1","addr":"172.21.15.81:6835","nonce":3773839758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.52:6839/2696044471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2696044471},{"type":"v1","addr":"172.21.15.52:6839","nonce":2696044471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.81:6839/1554395940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":1554395940},{"type":"v1","addr":"172.21.15.81:6839","nonce":1554395940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24478,24461,24463]},"id":1}]} 2024-05-15T13:59:22.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.496+0000 7f59d12dd700 1 -- 172.21.15.33:0/1117345369 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f59b40b2900 msgr2=0x7f59b40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:22.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.496+0000 7f59d12dd700 1 --2- 172.21.15.33:0/1117345369 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f59b40b2900 0x7f59b40b4dc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f59cc19aab0 tx=0x7f59bc0073d0 comp rx=0 tx=0).stop 2024-05-15T13:59:22.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.496+0000 7f59d12dd700 1 -- 172.21.15.33:0/1117345369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59cc1041a0 msgr2=0x7f59cc19fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:22.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.496+0000 7f59d12dd700 1 --2- 172.21.15.33:0/1117345369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59cc1041a0 0x7f59cc19fcf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f59c000b530 tx=0x7f59c000b840 comp rx=0 tx=0).stop 2024-05-15T13:59:22.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.496+0000 7f59d12dd700 1 -- 172.21.15.33:0/1117345369 shutdown_connections 2024-05-15T13:59:22.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.496+0000 7f59d12dd700 1 --2- 172.21.15.33:0/1117345369 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f59b40b2900 0x7f59b40b4dc0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:22.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.496+0000 7f59d12dd700 1 --2- 172.21.15.33:0/1117345369 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59cc1088f0 0x7f59cc199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:22.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.496+0000 7f59d12dd700 1 --2- 172.21.15.33:0/1117345369 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59cc1037f0 0x7f59cc19f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:22.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.496+0000 7f59d12dd700 1 --2- 172.21.15.33:0/1117345369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59cc1041a0 0x7f59cc19fcf0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:22.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.496+0000 7f59d12dd700 1 -- 172.21.15.33:0/1117345369 >> 172.21.15.33:0/1117345369 conn(0x7f59cc07a6a0 msgr2=0x7f59cc1016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:22.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.496+0000 7f59d12dd700 1 -- 172.21.15.33:0/1117345369 shutdown_connections 2024-05-15T13:59:22.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:22.496+0000 7f59d12dd700 1 -- 172.21.15.33:0/1117345369 wait complete. 2024-05-15T13:59:22.499 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 23 2024-05-15T13:59:22.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:22 smithi052 ceph-mon[36190]: pgmap v350: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T13:59:22.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:22 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:59:22.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:22 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:59:22.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:22 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:59:22.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:22 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:59:22.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:22 smithi052 ceph-mon[36190]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:59:22.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:22 smithi081 ceph-mon[36434]: pgmap v350: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T13:59:22.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:22 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:59:22.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:22 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:59:22.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:22 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T13:59:22.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:22 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T13:59:22.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:22 smithi081 ceph-mon[36434]: from='mgr.14152 172.21.15.33:0/3365776795' entity='mgr.x' 2024-05-15T13:59:23.198 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-15T13:59:23.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:23 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/1117345369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:59:23.461 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:59:23.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:23 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/1117345369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:59:23.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:23 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/1117345369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:59:24.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:24 smithi052 ceph-mon[36190]: pgmap v351: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-15T13:59:24.599 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:24 smithi033 ceph-mon[31650]: pgmap v351: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-15T13:59:24.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:24 smithi081 ceph-mon[36434]: pgmap v351: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-15T13:59:25.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.111+0000 7fca5451b700 1 -- 172.21.15.33:0/3561045942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca4c1028a0 msgr2=0x7fca4c110310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:25.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.111+0000 7fca5451b700 1 --2- 172.21.15.33:0/3561045942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca4c1028a0 0x7fca4c110310 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fca4400b0a0 tx=0x7fca4400b3b0 comp rx=0 tx=0).stop 2024-05-15T13:59:25.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.111+0000 7fca5451b700 1 -- 172.21.15.33:0/3561045942 shutdown_connections 2024-05-15T13:59:25.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.111+0000 7fca5451b700 1 --2- 172.21.15.33:0/3561045942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca4c1028a0 0x7fca4c110310 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:25.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.111+0000 7fca5451b700 1 --2- 172.21.15.33:0/3561045942 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca4c108600 0x7fca4c1089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:25.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.111+0000 7fca5451b700 1 --2- 172.21.15.33:0/3561045942 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca4c101ee0 0x7fca4c102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:25.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.111+0000 7fca5451b700 1 -- 172.21.15.33:0/3561045942 >> 172.21.15.33:0/3561045942 conn(0x7fca4c077760 msgr2=0x7fca4c077b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:25.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.111+0000 7fca5451b700 1 -- 172.21.15.33:0/3561045942 shutdown_connections 2024-05-15T13:59:25.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.111+0000 7fca5451b700 1 -- 172.21.15.33:0/3561045942 wait complete. 2024-05-15T13:59:25.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.112+0000 7fca5451b700 1 Processor -- start 2024-05-15T13:59:25.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.112+0000 7fca5451b700 1 -- start start 2024-05-15T13:59:25.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.112+0000 7fca5451b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca4c101ee0 0x7fca4c199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:25.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.112+0000 7fca5451b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca4c1028a0 0x7fca4c19a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:25.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.112+0000 7fca5451b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca4c108600 0x7fca4c19e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:25.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.112+0000 7fca5451b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fca4c113810 con 0x7fca4c1028a0 2024-05-15T13:59:25.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.112+0000 7fca5451b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fca4c113690 con 0x7fca4c108600 2024-05-15T13:59:25.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.112+0000 7fca5451b700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fca4c113990 con 0x7fca4c101ee0 2024-05-15T13:59:25.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.112+0000 7fca51ab6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca4c1028a0 0x7fca4c19a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:25.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.112+0000 7fca51ab6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca4c1028a0 0x7fca4c19a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57732/0 (socket says 172.21.15.33:57732) 2024-05-15T13:59:25.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.112+0000 7fca51ab6700 1 -- 172.21.15.33:0/620540625 learned_addr learned my addr 172.21.15.33:0/620540625 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:25.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.113+0000 7fca522b7700 1 --2- 172.21.15.33:0/620540625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca4c101ee0 0x7fca4c199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:25.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.113+0000 7fca51ab6700 1 -- 172.21.15.33:0/620540625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca4c101ee0 msgr2=0x7fca4c199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:25.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.113+0000 7fca51ab6700 1 --2- 172.21.15.33:0/620540625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca4c101ee0 0x7fca4c199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:25.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.113+0000 7fca51ab6700 1 -- 172.21.15.33:0/620540625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca4c108600 msgr2=0x7fca4c19e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:25.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.113+0000 7fca52ab8700 1 --2- 172.21.15.33:0/620540625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca4c108600 0x7fca4c19e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:25.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.113+0000 7fca51ab6700 1 --2- 172.21.15.33:0/620540625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca4c108600 0x7fca4c19e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:25.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.113+0000 7fca51ab6700 1 -- 172.21.15.33:0/620540625 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca4c19ed30 con 0x7fca4c1028a0 2024-05-15T13:59:25.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.113+0000 7fca51ab6700 1 --2- 172.21.15.33:0/620540625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca4c1028a0 0x7fca4c19a260 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fca3c00b530 tx=0x7fca3c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:25.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.113+0000 7fca437fe700 1 -- 172.21.15.33:0/620540625 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c015020 con 0x7fca4c1028a0 2024-05-15T13:59:25.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.113+0000 7fca5451b700 1 -- 172.21.15.33:0/620540625 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca4c19f020 con 0x7fca4c1028a0 2024-05-15T13:59:25.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.113+0000 7fca437fe700 1 -- 172.21.15.33:0/620540625 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca3c011ce0 con 0x7fca4c1028a0 2024-05-15T13:59:25.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.113+0000 7fca437fe700 1 -- 172.21.15.33:0/620540625 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c010e40 con 0x7fca4c1028a0 2024-05-15T13:59:25.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.113+0000 7fca5451b700 1 -- 172.21.15.33:0/620540625 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca4c1a68a0 con 0x7fca4c1028a0 2024-05-15T13:59:25.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.114+0000 7fca5451b700 1 -- 172.21.15.33:0/620540625 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca4c06a230 con 0x7fca4c1028a0 2024-05-15T13:59:25.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.116+0000 7fca437fe700 1 -- 172.21.15.33:0/620540625 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fca3c018020 con 0x7fca4c1028a0 2024-05-15T13:59:25.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.116+0000 7fca437fe700 1 --2- 172.21.15.33:0/620540625 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fca380b2880 0x7fca380b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:25.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.116+0000 7fca522b7700 1 --2- 172.21.15.33:0/620540625 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fca380b2880 0x7fca380b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:25.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.116+0000 7fca437fe700 1 -- 172.21.15.33:0/620540625 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fca3c0e2000 con 0x7fca4c1028a0 2024-05-15T13:59:25.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.117+0000 7fca522b7700 1 --2- 172.21.15.33:0/620540625 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fca380b2880 0x7fca380b4d40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fca4c078ce0 tx=0x7fca48009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:25.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.120+0000 7fca437fe700 1 -- 172.21.15.33:0/620540625 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fca3c01f020 con 0x7fca4c1028a0 2024-05-15T13:59:25.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.336+0000 7fca5451b700 1 -- 172.21.15.33:0/620540625 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca4c02e560 con 0x7fca4c1028a0 2024-05-15T13:59:25.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.338+0000 7fca437fe700 1 -- 172.21.15.33:0/620540625 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22593 (secure 0 0 0) 0x7fca3c0a9d20 con 0x7fca4c1028a0 2024-05-15T13:59:25.341 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:59:25.341 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":82,"fsid":"bca78ca4-12c1-11ef-bc98-c7b262605968","created":"2024-05-15T13:49:10.327613+0000","modified":"2024-05-15T13:58:06.058012+0000","last_up_change":"2024-05-15T13:56:16.097711+0000","last_in_change":"2024-05-15T13:55:56.722143+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-15T13:52:32.970844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-15T13:57:12.324802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-15T13:57:15.898406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b434111-810f-450a-b63d-c97080e9f8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6803","nonce":887409130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6805","nonce":887409130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6809","nonce":887409130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6807","nonce":887409130}]},"public_addr":"172.21.15.33:6803/887409130","cluster_addr":"172.21.15.33:6805/887409130","heartbeat_back_addr":"172.21.15.33:6809/887409130","heartbeat_front_addr":"172.21.15.33:6807/887409130","state":["exists","up"]},{"osd":1,"uuid":"0b578d4e-a841-4b27-bc95-d6c3bae580fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6801","nonce":2794180003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6803","nonce":2794180003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6807","nonce":2794180003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6805","nonce":2794180003}]},"public_addr":"172.21.15.52:6801/2794180003","cluster_addr":"172.21.15.52:6803/2794180003","heartbeat_back_addr":"172.21.15.52:6807/2794180003","heartbeat_front_addr":"172.21.15.52:6805/2794180003","state":["exists","up"]},{"osd":2,"uuid":"0e9c9390-ed33-464b-8f39-bca75941e05e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6801","nonce":315610978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6803","nonce":315610978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6807","nonce":315610978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6805","nonce":315610978}]},"public_addr":"172.21.15.81:6801/315610978","cluster_addr":"172.21.15.81:6803/315610978","heartbeat_back_addr":"172.21.15.81:6807/315610978","heartbeat_front_addr":"172.21.15.81:6805/315610978","state":["exists","up"]},{"osd":3,"uuid":"ecdb24c1-fb15-42a5-9bae-436bbae9512a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6811","nonce":818533501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6813","nonce":818533501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6817","nonce":818533501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6815","nonce":818533501}]},"public_addr":"172.21.15.33:6811/818533501","cluster_addr":"172.21.15.33:6813/818533501","heartbeat_back_addr":"172.21.15.33:6817/818533501","heartbeat_front_addr":"172.21.15.33:6815/818533501","state":["exists","up"]},{"osd":4,"uuid":"706752a0-b17f-4563-a7a9-d5078bf0f3b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6809","nonce":3010133776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6811","nonce":3010133776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6815","nonce":3010133776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6813","nonce":3010133776}]},"public_addr":"172.21.15.52:6809/3010133776","cluster_addr":"172.21.15.52:6811/3010133776","heartbeat_back_addr":"172.21.15.52:6815/3010133776","heartbeat_front_addr":"172.21.15.52:6813/3010133776","state":["exists","up"]},{"osd":5,"uuid":"3d45ab97-f364-4903-974d-8acc961363d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6809","nonce":2132616297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6811","nonce":2132616297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6815","nonce":2132616297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6813","nonce":2132616297}]},"public_addr":"172.21.15.81:6809/2132616297","cluster_addr":"172.21.15.81:6811/2132616297","heartbeat_back_addr":"172.21.15.81:6815/2132616297","heartbeat_front_addr":"172.21.15.81:6813/2132616297","state":["exists","up"]},{"osd":6,"uuid":"3be524c5-dcf9-4742-b896-ea9a44a6b391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6819","nonce":1260691801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6821","nonce":1260691801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6825","nonce":1260691801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6823","nonce":1260691801}]},"public_addr":"172.21.15.33:6819/1260691801","cluster_addr":"172.21.15.33:6821/1260691801","heartbeat_back_addr":"172.21.15.33:6825/1260691801","heartbeat_front_addr":"172.21.15.33:6823/1260691801","state":["exists","up"]},{"osd":7,"uuid":"bda2f920-4f2c-4eed-8127-83c1973676e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6817","nonce":1654595813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6819","nonce":1654595813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6823","nonce":1654595813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6821","nonce":1654595813}]},"public_addr":"172.21.15.52:6817/1654595813","cluster_addr":"172.21.15.52:6819/1654595813","heartbeat_back_addr":"172.21.15.52:6823/1654595813","heartbeat_front_addr":"172.21.15.52:6821/1654595813","state":["exists","up"]},{"osd":8,"uuid":"550585b8-0df4-49ee-95c8-3d084744dbd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6817","nonce":3009661748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6819","nonce":3009661748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6823","nonce":3009661748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6821","nonce":3009661748}]},"public_addr":"172.21.15.81:6817/3009661748","cluster_addr":"172.21.15.81:6819/3009661748","heartbeat_back_addr":"172.21.15.81:6823/3009661748","heartbeat_front_addr":"172.21.15.81:6821/3009661748","state":["exists","up"]},{"osd":9,"uuid":"1d781846-26f3-4b23-b22c-c994a8509a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6827","nonce":2843220858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6829","nonce":2843220858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6833","nonce":2843220858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6831","nonce":2843220858}]},"public_addr":"172.21.15.33:6827/2843220858","cluster_addr":"172.21.15.33:6829/2843220858","heartbeat_back_addr":"172.21.15.33:6833/2843220858","heartbeat_front_addr":"172.21.15.33:6831/2843220858","state":["exists","up"]},{"osd":10,"uuid":"af58c940-fdc4-418b-a638-7dc44438b34c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6825","nonce":2422349831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6827","nonce":2422349831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6831","nonce":2422349831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6829","nonce":2422349831}]},"public_addr":"172.21.15.52:6825/2422349831","cluster_addr":"172.21.15.52:6827/2422349831","heartbeat_back_addr":"172.21.15.52:6831/2422349831","heartbeat_front_addr":"172.21.15.52:6829/2422349831","state":["exists","up"]},{"osd":11,"uuid":"ab84aaea-2086-4842-8bf4-628842fce46a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6825","nonce":326313090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6827","nonce":326313090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6831","nonce":326313090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6829","nonce":326313090}]},"public_addr":"172.21.15.81:6825/326313090","cluster_addr":"172.21.15.81:6827/326313090","heartbeat_back_addr":"172.21.15.81:6831/326313090","heartbeat_front_addr":"172.21.15.81:6829/326313090","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:51:41.493136+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:52:05.784773+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:52:29.769429+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:52:54.891620+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:53:19.412922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:53:43.921841+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:54:09.486732+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:54:34.064370+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:54:58.655304+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:55:24.548693+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:55:49.941191+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:56:14.919558+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/436121571":"2024-05-16T13:50:04.881656+0000","172.21.15.33:6801/2680375465":"2024-05-16T13:50:04.881656+0000","172.21.15.33:6800/2680375465":"2024-05-16T13:50:04.881656+0000","172.21.15.33:0/1022787179":"2024-05-16T13:49:37.564643+0000","172.21.15.33:0/3720711369":"2024-05-16T13:50:04.881656+0000","172.21.15.33:0/1282466838":"2024-05-16T13:49:37.564643+0000","172.21.15.33:0/3022248969":"2024-05-16T13:49:37.564643+0000","172.21.15.33:6801/3303495364":"2024-05-16T13:49:37.564643+0000","172.21.15.33:6800/3303495364":"2024-05-16T13:49:37.564643+0000","172.21.15.33:0/4282812692":"2024-05-16T13:50:04.881656+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-15T13:59:25.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.340+0000 7fca5451b700 1 -- 172.21.15.33:0/620540625 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fca380b2880 msgr2=0x7fca380b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:25.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.340+0000 7fca5451b700 1 --2- 172.21.15.33:0/620540625 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fca380b2880 0x7fca380b4d40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fca4c078ce0 tx=0x7fca48009190 comp rx=0 tx=0).stop 2024-05-15T13:59:25.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.341+0000 7fca5451b700 1 -- 172.21.15.33:0/620540625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca4c1028a0 msgr2=0x7fca4c19a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:25.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.341+0000 7fca5451b700 1 --2- 172.21.15.33:0/620540625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca4c1028a0 0x7fca4c19a260 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fca3c00b530 tx=0x7fca3c00b840 comp rx=0 tx=0).stop 2024-05-15T13:59:25.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.341+0000 7fca5451b700 1 -- 172.21.15.33:0/620540625 shutdown_connections 2024-05-15T13:59:25.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.341+0000 7fca5451b700 1 --2- 172.21.15.33:0/620540625 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fca380b2880 0x7fca380b4d40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:25.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.341+0000 7fca5451b700 1 --2- 172.21.15.33:0/620540625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca4c108600 0x7fca4c19e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:25.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.341+0000 7fca5451b700 1 --2- 172.21.15.33:0/620540625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca4c101ee0 0x7fca4c199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:25.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.341+0000 7fca5451b700 1 --2- 172.21.15.33:0/620540625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca4c1028a0 0x7fca4c19a260 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:25.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.341+0000 7fca5451b700 1 -- 172.21.15.33:0/620540625 >> 172.21.15.33:0/620540625 conn(0x7fca4c077760 msgr2=0x7fca4c100ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:25.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.341+0000 7fca5451b700 1 -- 172.21.15.33:0/620540625 shutdown_connections 2024-05-15T13:59:25.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:25.341+0000 7fca5451b700 1 -- 172.21.15.33:0/620540625 wait complete. 2024-05-15T13:59:25.998 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T13:59:26.254 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:59:26.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:26 smithi052 ceph-mon[36190]: pgmap v352: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:59:26.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:26 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/620540625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T13:59:26.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:26 smithi081 ceph-mon[36434]: pgmap v352: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:59:26.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:26 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/620540625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T13:59:26.649 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:26 smithi033 ceph-mon[31650]: pgmap v352: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:59:26.649 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:26 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/620540625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T13:59:27.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.840+0000 7fc2fe1d7700 1 -- 172.21.15.33:0/1784270851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f81041a0 msgr2=0x7fc2f81081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:27.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.840+0000 7fc2fe1d7700 1 --2- 172.21.15.33:0/1784270851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f81041a0 0x7fc2f81081c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0099c0 tx=0x7fc2ec009cd0 comp rx=0 tx=0).stop 2024-05-15T13:59:27.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.840+0000 7fc2fe1d7700 1 -- 172.21.15.33:0/1784270851 shutdown_connections 2024-05-15T13:59:27.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.840+0000 7fc2fe1d7700 1 --2- 172.21.15.33:0/1784270851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2f81088f0 0x7fc2f8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:27.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.840+0000 7fc2fe1d7700 1 --2- 172.21.15.33:0/1784270851 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc2f81037f0 0x7fc2f8103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:27.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.840+0000 7fc2fe1d7700 1 --2- 172.21.15.33:0/1784270851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f81041a0 0x7fc2f81081c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:27.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.840+0000 7fc2fe1d7700 1 -- 172.21.15.33:0/1784270851 >> 172.21.15.33:0/1784270851 conn(0x7fc2f807a6a0 msgr2=0x7fc2f807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:27.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.840+0000 7fc2fe1d7700 1 -- 172.21.15.33:0/1784270851 shutdown_connections 2024-05-15T13:59:27.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.840+0000 7fc2fe1d7700 1 -- 172.21.15.33:0/1784270851 wait complete. 2024-05-15T13:59:27.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.841+0000 7fc2fe1d7700 1 Processor -- start 2024-05-15T13:59:27.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.841+0000 7fc2fe1d7700 1 -- start start 2024-05-15T13:59:27.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.841+0000 7fc2fe1d7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f81037f0 0x7fc2f8075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:27.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.841+0000 7fc2fe1d7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2f81041a0 0x7fc2f8075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:27.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.841+0000 7fc2fe1d7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc2f81088f0 0x7fc2f80762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:27.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.841+0000 7fc2fe1d7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f81137b0 con 0x7fc2f81037f0 2024-05-15T13:59:27.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.841+0000 7fc2fe1d7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f8113630 con 0x7fc2f81041a0 2024-05-15T13:59:27.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.841+0000 7fc2fe1d7700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f8113930 con 0x7fc2f81088f0 2024-05-15T13:59:27.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.841+0000 7fc2f77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f81037f0 0x7fc2f8075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:27.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.841+0000 7fc2f77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f81037f0 0x7fc2f8075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57750/0 (socket says 172.21.15.33:57750) 2024-05-15T13:59:27.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.841+0000 7fc2f77fe700 1 -- 172.21.15.33:0/4217279655 learned_addr learned my addr 172.21.15.33:0/4217279655 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:27.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.841+0000 7fc2f7fff700 1 --2- 172.21.15.33:0/4217279655 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc2f81088f0 0x7fc2f80762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:27.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.841+0000 7fc2f6ffd700 1 --2- 172.21.15.33:0/4217279655 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2f81041a0 0x7fc2f8075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:27.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.841+0000 7fc2f77fe700 1 -- 172.21.15.33:0/4217279655 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc2f81088f0 msgr2=0x7fc2f80762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:27.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.841+0000 7fc2f77fe700 1 --2- 172.21.15.33:0/4217279655 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc2f81088f0 0x7fc2f80762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:27.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.841+0000 7fc2f77fe700 1 -- 172.21.15.33:0/4217279655 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2f81041a0 msgr2=0x7fc2f8075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:27.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.842+0000 7fc2f77fe700 1 --2- 172.21.15.33:0/4217279655 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2f81041a0 0x7fc2f8075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:27.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.842+0000 7fc2f77fe700 1 -- 172.21.15.33:0/4217279655 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f81a8190 con 0x7fc2f81037f0 2024-05-15T13:59:27.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.842+0000 7fc2f6ffd700 1 --2- 172.21.15.33:0/4217279655 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2f81041a0 0x7fc2f8075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T13:59:27.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.842+0000 7fc2f7fff700 1 --2- 172.21.15.33:0/4217279655 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc2f81088f0 0x7fc2f80762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T13:59:27.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.842+0000 7fc2f77fe700 1 --2- 172.21.15.33:0/4217279655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f81037f0 0x7fc2f8075840 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800b530 tx=0x7fc2e800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:27.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.842+0000 7fc2f4ff9700 1 -- 172.21.15.33:0/4217279655 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8015020 con 0x7fc2f81037f0 2024-05-15T13:59:27.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.842+0000 7fc2f4ff9700 1 -- 172.21.15.33:0/4217279655 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2e8011ce0 con 0x7fc2f81037f0 2024-05-15T13:59:27.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.842+0000 7fc2f4ff9700 1 -- 172.21.15.33:0/4217279655 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8010d20 con 0x7fc2f81037f0 2024-05-15T13:59:27.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.842+0000 7fc2fe1d7700 1 -- 172.21.15.33:0/4217279655 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2f81a8480 con 0x7fc2f81037f0 2024-05-15T13:59:27.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.842+0000 7fc2fe1d7700 1 -- 172.21.15.33:0/4217279655 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2f81a8990 con 0x7fc2f81037f0 2024-05-15T13:59:27.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.843+0000 7fc2fe1d7700 1 -- 172.21.15.33:0/4217279655 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2f810bb30 con 0x7fc2f81037f0 2024-05-15T13:59:27.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.845+0000 7fc2f4ff9700 1 -- 172.21.15.33:0/4217279655 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc2e8018020 con 0x7fc2f81037f0 2024-05-15T13:59:27.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.846+0000 7fc2f4ff9700 1 --2- 172.21.15.33:0/4217279655 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fc2e00b2880 0x7fc2e00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:27.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.846+0000 7fc2f6ffd700 1 --2- 172.21.15.33:0/4217279655 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fc2e00b2880 0x7fc2e00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:27.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.846+0000 7fc2f4ff9700 1 -- 172.21.15.33:0/4217279655 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fc2e80e1a50 con 0x7fc2f81037f0 2024-05-15T13:59:27.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.846+0000 7fc2f6ffd700 1 --2- 172.21.15.33:0/4217279655 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fc2e00b2880 0x7fc2e00b4d40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80772d0 tx=0x7fc2ec005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:27.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:27.850+0000 7fc2f4ff9700 1 -- 172.21.15.33:0/4217279655 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc2e801f020 con 0x7fc2f81037f0 2024-05-15T13:59:28.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:28.110+0000 7fc2fe1d7700 1 -- 172.21.15.33:0/4217279655 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2f810b510 con 0x7fc2f81037f0 2024-05-15T13:59:28.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:28.111+0000 7fc2f4ff9700 1 -- 172.21.15.33:0/4217279655 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11858 (secure 0 0 0) 0x7fc2e80a9770 con 0x7fc2f81037f0 2024-05-15T13:59:28.113 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:59:28.114 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2030117633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2030117633},{"type":"v1","addr":"172.21.15.33:6839","nonce":2030117633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1918979061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1918979061},{"type":"v1","addr":"172.21.15.33:6841","nonce":1918979061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T13:57:19.056417+0000","modified":"2024-05-15T13:57:38.201184+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24463,"mds_2":24461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/1629697841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1629697841},{"type":"v1","addr":"172.21.15.52:6837","nonce":1629697841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/3773839758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3773839758},{"type":"v1","addr":"172.21.15.81:6835","nonce":3773839758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.52:6839/2696044471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2696044471},{"type":"v1","addr":"172.21.15.52:6839","nonce":2696044471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.81:6839/1554395940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":1554395940},{"type":"v1","addr":"172.21.15.81:6839","nonce":1554395940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24478,24461,24463]},"id":1}]} 2024-05-15T13:59:28.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:28.114+0000 7fc2fe1d7700 1 -- 172.21.15.33:0/4217279655 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fc2e00b2880 msgr2=0x7fc2e00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:28.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:28.114+0000 7fc2fe1d7700 1 --2- 172.21.15.33:0/4217279655 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fc2e00b2880 0x7fc2e00b4d40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80772d0 tx=0x7fc2ec005fb0 comp rx=0 tx=0).stop 2024-05-15T13:59:28.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:28.114+0000 7fc2fe1d7700 1 -- 172.21.15.33:0/4217279655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f81037f0 msgr2=0x7fc2f8075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:28.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:28.114+0000 7fc2fe1d7700 1 --2- 172.21.15.33:0/4217279655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f81037f0 0x7fc2f8075840 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800b530 tx=0x7fc2e800b8f0 comp rx=0 tx=0).stop 2024-05-15T13:59:28.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:28.114+0000 7fc2fe1d7700 1 -- 172.21.15.33:0/4217279655 shutdown_connections 2024-05-15T13:59:28.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:28.114+0000 7fc2fe1d7700 1 --2- 172.21.15.33:0/4217279655 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fc2e00b2880 0x7fc2e00b4d40 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:28.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:28.114+0000 7fc2fe1d7700 1 --2- 172.21.15.33:0/4217279655 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc2f81088f0 0x7fc2f80762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:28.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:28.114+0000 7fc2fe1d7700 1 --2- 172.21.15.33:0/4217279655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f81037f0 0x7fc2f8075840 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:28.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:28.114+0000 7fc2fe1d7700 1 --2- 172.21.15.33:0/4217279655 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2f81041a0 0x7fc2f8075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:28.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:28.114+0000 7fc2fe1d7700 1 -- 172.21.15.33:0/4217279655 >> 172.21.15.33:0/4217279655 conn(0x7fc2f807a6a0 msgr2=0x7fc2f81018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:28.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:28.115+0000 7fc2fe1d7700 1 -- 172.21.15.33:0/4217279655 shutdown_connections 2024-05-15T13:59:28.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:28.115+0000 7fc2fe1d7700 1 -- 172.21.15.33:0/4217279655 wait complete. 2024-05-15T13:59:28.118 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 23 2024-05-15T13:59:28.432 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:28 smithi033 ceph-mon[31650]: pgmap v353: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-15T13:59:28.432 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:28 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/4217279655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:59:28.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:28 smithi052 ceph-mon[36190]: pgmap v353: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-15T13:59:28.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:28 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/4217279655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:59:28.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:28 smithi081 ceph-mon[36434]: pgmap v353: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-15T13:59:28.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:28 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/4217279655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:59:28.746 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-15T13:57:19.056417+0000', 'modified': '2024-05-15T13:57:38.201184+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24478, 'mds_1': 24463, 'mds_2': 24461}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24424': {'gid': 24424, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.81:6833/1872861431', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6832', 'nonce': 1872861431}, {'type': 'v1', 'addr': '172.21.15.81:6833', 'nonce': 1872861431}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.52:6837/1629697841', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6836', 'nonce': 1629697841}, {'type': 'v1', 'addr': '172.21.15.52:6837', 'nonce': 1629697841}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.81:6835/3773839758', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6834', 'nonce': 3773839758}, {'type': 'v1', 'addr': '172.21.15.81:6835', 'nonce': 3773839758}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24461': {'gid': 24461, 'name': 'b', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.52:6839/2696044471', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6838', 'nonce': 2696044471}, {'type': 'v1', 'addr': '172.21.15.52:6839', 'nonce': 2696044471}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'c', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.81:6837/744100239', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6836', 'nonce': 744100239}, {'type': 'v1', 'addr': '172.21.15.81:6837', 'nonce': 744100239}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.81:6839/1554395940', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6838', 'nonce': 1554395940}, {'type': 'v1', 'addr': '172.21.15.81:6839', 'nonce': 1554395940}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24478, 'qdb_cluster': [24478, 24461, 24463]} 2024-05-15T13:59:28.746 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-15T13:59:28.746 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph tell mds.b status 2024-05-15T13:59:28.998 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:59:30.585 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:30 smithi033 ceph-mon[31650]: pgmap v354: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:59:30.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:30 smithi052 ceph-mon[36190]: pgmap v354: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:59:30.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:30 smithi081 ceph-mon[36434]: pgmap v354: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:59:30.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.620+0000 7f62b4726700 1 -- 172.21.15.33:0/540396754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62ac1028c0 msgr2=0x7f62ac110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:30.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.620+0000 7f62b4726700 1 --2- 172.21.15.33:0/540396754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62ac1028c0 0x7f62ac110330 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f62a400b0a0 tx=0x7f62a400b3b0 comp rx=0 tx=0).stop 2024-05-15T13:59:30.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.620+0000 7f62b4726700 1 -- 172.21.15.33:0/540396754 shutdown_connections 2024-05-15T13:59:30.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.620+0000 7f62b4726700 1 --2- 172.21.15.33:0/540396754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62ac1028c0 0x7f62ac110330 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:30.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.620+0000 7f62b4726700 1 --2- 172.21.15.33:0/540396754 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62ac108620 0x7f62ac108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:30.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.620+0000 7f62b4726700 1 --2- 172.21.15.33:0/540396754 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62ac101f00 0x7f62ac102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:30.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.620+0000 7f62b4726700 1 -- 172.21.15.33:0/540396754 >> 172.21.15.33:0/540396754 conn(0x7f62ac077780 msgr2=0x7f62ac077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:30.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.620+0000 7f62b4726700 1 -- 172.21.15.33:0/540396754 shutdown_connections 2024-05-15T13:59:30.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.620+0000 7f62b4726700 1 -- 172.21.15.33:0/540396754 wait complete. 2024-05-15T13:59:30.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.621+0000 7f62b4726700 1 Processor -- start 2024-05-15T13:59:30.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.621+0000 7f62b4726700 1 -- start start 2024-05-15T13:59:30.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.621+0000 7f62b4726700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62ac101f00 0x7f62ac1030e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:30.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.621+0000 7f62b4726700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62ac1028c0 0x7f62ac103620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:30.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.621+0000 7f62b4726700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62ac108620 0x7f62ac103b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:30.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.621+0000 7f62b4726700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f62ac1137a0 con 0x7f62ac101f00 2024-05-15T13:59:30.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.621+0000 7f62b4726700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f62ac113620 con 0x7f62ac108620 2024-05-15T13:59:30.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.621+0000 7f62b4726700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f62ac113920 con 0x7f62ac1028c0 2024-05-15T13:59:30.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.621+0000 7f62b24c2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62ac101f00 0x7f62ac1030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:30.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.621+0000 7f62b24c2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62ac101f00 0x7f62ac1030e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57768/0 (socket says 172.21.15.33:57768) 2024-05-15T13:59:30.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.621+0000 7f62b24c2700 1 -- 172.21.15.33:0/4166501548 learned_addr learned my addr 172.21.15.33:0/4166501548 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:30.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.621+0000 7f62b1cc1700 1 --2- 172.21.15.33:0/4166501548 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62ac1028c0 0x7f62ac103620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:30.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.621+0000 7f62b2cc3700 1 --2- 172.21.15.33:0/4166501548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62ac108620 0x7f62ac103b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:30.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.621+0000 7f62b24c2700 1 -- 172.21.15.33:0/4166501548 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62ac1028c0 msgr2=0x7f62ac103620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:30.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.621+0000 7f62b24c2700 1 --2- 172.21.15.33:0/4166501548 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62ac1028c0 0x7f62ac103620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:30.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.621+0000 7f62b24c2700 1 -- 172.21.15.33:0/4166501548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62ac108620 msgr2=0x7f62ac103b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:30.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.621+0000 7f62b24c2700 1 --2- 172.21.15.33:0/4166501548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62ac108620 0x7f62ac103b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:30.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.621+0000 7f62b24c2700 1 -- 172.21.15.33:0/4166501548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62ac1a8090 con 0x7f62ac101f00 2024-05-15T13:59:30.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.621+0000 7f62b2cc3700 1 --2- 172.21.15.33:0/4166501548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62ac108620 0x7f62ac103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T13:59:30.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.622+0000 7f62b24c2700 1 --2- 172.21.15.33:0/4166501548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62ac101f00 0x7f62ac1030e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f62a800c990 tx=0x7f62a800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:30.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.622+0000 7f62a37fe700 1 -- 172.21.15.33:0/4166501548 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a8007b00 con 0x7f62ac101f00 2024-05-15T13:59:30.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.622+0000 7f62a37fe700 1 -- 172.21.15.33:0/4166501548 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62a8007ca0 con 0x7f62ac101f00 2024-05-15T13:59:30.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.622+0000 7f62b4726700 1 -- 172.21.15.33:0/4166501548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62ac1a8380 con 0x7f62ac101f00 2024-05-15T13:59:30.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.622+0000 7f62a37fe700 1 -- 172.21.15.33:0/4166501548 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a8018660 con 0x7f62ac101f00 2024-05-15T13:59:30.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.622+0000 7f62b4726700 1 -- 172.21.15.33:0/4166501548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62ac1a88c0 con 0x7f62ac101f00 2024-05-15T13:59:30.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.625+0000 7f62a37fe700 1 -- 172.21.15.33:0/4166501548 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f62a800f450 con 0x7f62ac101f00 2024-05-15T13:59:30.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.626+0000 7f62a37fe700 1 --2- 172.21.15.33:0/4166501548 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f62980b2880 0x7f62980b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:30.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.626+0000 7f62b1cc1700 1 --2- 172.21.15.33:0/4166501548 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f62980b2880 0x7f62980b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:30.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.626+0000 7f62a37fe700 1 -- 172.21.15.33:0/4166501548 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f62a80e21f0 con 0x7f62ac101f00 2024-05-15T13:59:30.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.626+0000 7f62befbcb80 1 Processor -- start 2024-05-15T13:59:30.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.626+0000 7f62befbcb80 1 -- start start 2024-05-15T13:59:30.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.627+0000 7f62b1cc1700 1 --2- 172.21.15.33:0/4166501548 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f62980b2880 0x7f62980b4d40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f629c005fd0 tx=0x7f629c00a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:30.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.627+0000 7f62befbcb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a0ed9e2f0 0x556a0ed9e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:30.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.627+0000 7f62befbcb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a0ed9ec10 0x556a0eda23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:30.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.627+0000 7f62befbcb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a0eda2910 0x556a0eda2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:30.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.627+0000 7f62befbcb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556a0ecfcc40 con 0x556a0ed9ec10 2024-05-15T13:59:30.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.627+0000 7f62befbcb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556a0ecec7e0 con 0x556a0ed9e2f0 2024-05-15T13:59:30.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.627+0000 7f62befbcb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x556a0ecf0820 con 0x556a0eda2910 2024-05-15T13:59:30.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.627+0000 7f62b1cc1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a0ed9ec10 0x556a0eda23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:30.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.627+0000 7f62b1cc1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a0ed9ec10 0x556a0eda23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57782/0 (socket says 172.21.15.33:57782) 2024-05-15T13:59:30.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.627+0000 7f62b1cc1700 1 -- 172.21.15.33:0/4289801343 learned_addr learned my addr 172.21.15.33:0/4289801343 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:30.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.627+0000 7f62b24c2700 1 --2- 172.21.15.33:0/4289801343 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a0ed9e2f0 0x556a0ed9e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:30.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.627+0000 7f62b2cc3700 1 --2- 172.21.15.33:0/4289801343 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a0eda2910 0x556a0eda2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:30.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.627+0000 7f62b1cc1700 1 -- 172.21.15.33:0/4289801343 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a0eda2910 msgr2=0x556a0eda2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T13:59:30.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.627+0000 7f62b1cc1700 1 --2- 172.21.15.33:0/4289801343 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a0eda2910 0x556a0eda2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:30.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.627+0000 7f62b1cc1700 1 -- 172.21.15.33:0/4289801343 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a0ed9e2f0 msgr2=0x556a0ed9e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T13:59:30.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.627+0000 7f62b1cc1700 1 --2- 172.21.15.33:0/4289801343 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a0ed9e2f0 0x556a0ed9e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:30.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.627+0000 7f62b1cc1700 1 -- 172.21.15.33:0/4289801343 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a0ec3c880 con 0x556a0ed9ec10 2024-05-15T13:59:30.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.627+0000 7f62b1cc1700 1 --2- 172.21.15.33:0/4289801343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a0ed9ec10 0x556a0eda23d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f629c00ec00 tx=0x7f629c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3723b628b68a5eb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:30.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.627+0000 7f62a0ff9700 1 -- 172.21.15.33:0/4289801343 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c021070 con 0x556a0ed9ec10 2024-05-15T13:59:30.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.628+0000 7f62a0ff9700 1 -- 172.21.15.33:0/4289801343 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f629c010c50 con 0x556a0ed9ec10 2024-05-15T13:59:30.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.628+0000 7f62a0ff9700 1 -- 172.21.15.33:0/4289801343 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c00f620 con 0x556a0ed9ec10 2024-05-15T13:59:30.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.628+0000 7f62befbcb80 1 -- 172.21.15.33:0/4289801343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a0ed9ec10 msgr2=0x556a0eda23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:30.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.628+0000 7f62befbcb80 1 --2- 172.21.15.33:0/4289801343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a0ed9ec10 0x556a0eda23d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f629c00ec00 tx=0x7f629c011000 comp rx=0 tx=0).stop 2024-05-15T13:59:30.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.628+0000 7f62befbcb80 1 -- 172.21.15.33:0/4289801343 shutdown_connections 2024-05-15T13:59:30.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.628+0000 7f62befbcb80 1 --2- 172.21.15.33:0/4289801343 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a0eda2910 0x556a0eda2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:30.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.628+0000 7f62befbcb80 1 --2- 172.21.15.33:0/4289801343 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a0ed9e2f0 0x556a0ed9e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:30.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.628+0000 7f62befbcb80 1 --2- 172.21.15.33:0/4289801343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a0ed9ec10 0x556a0eda23d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:30.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.628+0000 7f62befbcb80 1 -- 172.21.15.33:0/4289801343 >> 172.21.15.33:0/4289801343 conn(0x556a0ecc9200 msgr2=0x556a0ed98930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:30.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.628+0000 7f62befbcb80 1 -- 172.21.15.33:0/4289801343 shutdown_connections 2024-05-15T13:59:30.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.628+0000 7f62befbcb80 1 -- 172.21.15.33:0/4289801343 wait complete. 2024-05-15T13:59:30.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.629+0000 7f62befbcb80 1 -- start start 2024-05-15T13:59:30.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.629+0000 7f62befbcb80 1 Processor -- start 2024-05-15T13:59:30.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.629+0000 7f62befbcb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a0ed9e2f0 0x556a0eda9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:30.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.629+0000 7f62befbcb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a0ed9ec10 0x556a0eda71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:30.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.629+0000 7f62befbcb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a0eda2910 0x556a0eda7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:30.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.629+0000 7f62befbcb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556a0ebf2f50 con 0x556a0eda2910 2024-05-15T13:59:30.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.629+0000 7f62befbcb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556a0ec8e770 con 0x556a0ed9ec10 2024-05-15T13:59:30.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.629+0000 7f62befbcb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x556a0edaab70 con 0x556a0ed9e2f0 2024-05-15T13:59:30.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.629+0000 7f62befbcb80 10 client.0 _ll_register_callbacks cb 0x556a0ec83530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T13:59:30.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.629+0000 7f62b2cc3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a0eda2910 0x556a0eda7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:30.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.629+0000 7f62b2cc3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a0eda2910 0x556a0eda7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57784/0 (socket says 172.21.15.33:57784) 2024-05-15T13:59:30.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.629+0000 7f62b24c2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a0ed9e2f0 0x556a0eda9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:30.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.629+0000 7f62b2cc3700 1 -- 172.21.15.33:0/4048776920 learned_addr learned my addr 172.21.15.33:0/4048776920 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:30.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.629+0000 7f62b1cc1700 1 --2- 172.21.15.33:0/4048776920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a0ed9ec10 0x556a0eda71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:30.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.630+0000 7f62b2cc3700 1 -- 172.21.15.33:0/4048776920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a0ed9e2f0 msgr2=0x556a0eda9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T13:59:30.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.630+0000 7f62b2cc3700 1 --2- 172.21.15.33:0/4048776920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a0ed9e2f0 0x556a0eda9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:30.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.630+0000 7f62b2cc3700 1 -- 172.21.15.33:0/4048776920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a0ed9ec10 msgr2=0x556a0eda71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T13:59:30.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.630+0000 7f62b2cc3700 1 --2- 172.21.15.33:0/4048776920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a0ed9ec10 0x556a0eda71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:30.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.630+0000 7f62b2cc3700 1 -- 172.21.15.33:0/4048776920 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a0ec8b640 con 0x556a0eda2910 2024-05-15T13:59:30.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.630+0000 7f62b2cc3700 1 --2- 172.21.15.33:0/4048776920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a0eda2910 0x556a0eda7730 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f62a4004cb0 tx=0x7f62a4004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb872d3001360122 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:30.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.630+0000 7f6295ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T13:59:30.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.630+0000 7f6295ffb700 1 -- 172.21.15.33:0/4048776920 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a40078f0 con 0x556a0eda2910 2024-05-15T13:59:30.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.630+0000 7f62befbcb80 1 -- 172.21.15.33:0/4048776920 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f629c010c50 con 0x556a0eda2910 2024-05-15T13:59:30.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.630+0000 7f6295ffb700 1 -- 172.21.15.33:0/4048776920 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f62a4007a90 con 0x556a0eda2910 2024-05-15T13:59:30.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.630+0000 7f6295ffb700 1 -- 172.21.15.33:0/4048776920 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f62a4007c90 con 0x556a0eda2910 2024-05-15T13:59:30.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.630+0000 7f62befbcb80 10 client.14964 fetch_fsmap learned FSMap version 23 2024-05-15T13:59:30.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.630+0000 7f62befbcb80 1 -- 172.21.15.33:0/4048776920 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x556a0edb5340 con 0x556a0eda2910 2024-05-15T13:59:30.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.630+0000 7f6295ffb700 1 -- 172.21.15.33:0/4048776920 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f629c010c50 con 0x556a0eda2910 2024-05-15T13:59:30.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.631+0000 7f6295ffb700 1 -- 172.21.15.33:0/4048776920 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f62a4011070 con 0x556a0eda2910 2024-05-15T13:59:30.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.631+0000 7f62befbcb80 10 client.14964 fetch_fsmap finished waiting for FSMap version 23 2024-05-15T13:59:30.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.631+0000 7f62befbcb80 10 client.14964 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-15T13:59:30.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.631+0000 7f62befbcb80 20 client.14964 populate_metadata read hostname 'smithi033' 2024-05-15T13:59:30.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.631+0000 7f62befbcb80 1 --2- 172.21.15.33:0/4048776920 >> [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] conn(0x556a0edb7930 0x556a0edb8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:30.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.631+0000 7f62befbcb80 4 client.14964 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T13:59:30.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.631+0000 7f62befbcb80 1 -- 172.21.15.33:0/4048776920 --> [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a0ed6a310 con 0x556a0edb7930 2024-05-15T13:59:30.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.631+0000 7f62b24c2700 1 --2- 172.21.15.33:0/4048776920 >> [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] conn(0x556a0edb7930 0x556a0edb8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:30.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.631+0000 7f6295ffb700 10 client.14964 ms_handle_connect on v2:172.21.15.52:6838/2696044471 2024-05-15T13:59:30.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.631+0000 7f62b24c2700 1 --2- 172.21.15.33:0/4048776920 >> [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] conn(0x556a0edb7930 0x556a0edb8da0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=54bb92b07a4a1608 server_cookie=1b980d91c8e60029 in_seq=0 out_seq=0 2024-05-15T13:59:30.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.632+0000 7f6295ffb700 1 -- 172.21.15.33:0/4048776920 <== mds.2 v2:172.21.15.52:6838/2696044471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556a0ed6a310 con 0x556a0edb7930 2024-05-15T13:59:30.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.632+0000 7f6295ffb700 10 client.14964 handle_command_reply: tid=0 2024-05-15T13:59:30.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.632+0000 7f62befbcb80 1 client.14964 shutdown 2024-05-15T13:59:30.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.632+0000 7f62befbcb80 1 -- 172.21.15.33:0/4048776920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a0eda2910 msgr2=0x556a0eda7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:30.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.632+0000 7f62befbcb80 1 --2- 172.21.15.33:0/4048776920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a0eda2910 0x556a0eda7730 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f62a4004cb0 tx=0x7f62a4004d90 comp rx=0 tx=0).stop 2024-05-15T13:59:30.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.632+0000 7f62befbcb80 1 -- 172.21.15.33:0/4048776920 shutdown_connections 2024-05-15T13:59:30.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.632+0000 7f62befbcb80 1 --2- 172.21.15.33:0/4048776920 >> [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] conn(0x556a0edb7930 0x556a0edb8da0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:30.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.632+0000 7f62befbcb80 1 --2- 172.21.15.33:0/4048776920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a0eda2910 0x556a0eda7730 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:30.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.632+0000 7f62befbcb80 1 --2- 172.21.15.33:0/4048776920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a0ed9e2f0 0x556a0eda9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:30.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.632+0000 7f6295ffb700 0 client.14964 ms_handle_reset on v2:172.21.15.52:6838/2696044471 2024-05-15T13:59:30.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.632+0000 7f62befbcb80 1 --2- 172.21.15.33:0/4048776920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a0ed9ec10 0x556a0eda71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:30.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.632+0000 7f62befbcb80 1 -- 172.21.15.33:0/4048776920 >> 172.21.15.33:0/4048776920 conn(0x556a0ecc9200 msgr2=0x556a0eda3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:30.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.633+0000 7f62befbcb80 1 -- 172.21.15.33:0/4048776920 shutdown_connections 2024-05-15T13:59:30.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.633+0000 7f62befbcb80 1 -- 172.21.15.33:0/4048776920 wait complete. 2024-05-15T13:59:30.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.633+0000 7f62befbcb80 20 client.14964 trim_cache size 0 max 16384 2024-05-15T13:59:30.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.654+0000 7f62befbcb80 1 Processor -- start 2024-05-15T13:59:30.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.654+0000 7f62befbcb80 1 -- start start 2024-05-15T13:59:30.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.654+0000 7f62befbcb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a0ed9e2f0 0x556a0eda7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:30.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.654+0000 7f62befbcb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a0ed9ec10 0x556a0eda0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:30.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.654+0000 7f62befbcb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a0eda2910 0x556a0eda0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:30.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.654+0000 7f62befbcb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556a0edb70e0 con 0x556a0ed9ec10 2024-05-15T13:59:30.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.654+0000 7f62befbcb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556a0edb77b0 con 0x556a0ed9e2f0 2024-05-15T13:59:30.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.654+0000 7f62befbcb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x556a0edb7e80 con 0x556a0eda2910 2024-05-15T13:59:30.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.654+0000 7f62b1cc1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a0ed9ec10 0x556a0eda0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:30.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.654+0000 7f62b1cc1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a0ed9ec10 0x556a0eda0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57788/0 (socket says 172.21.15.33:57788) 2024-05-15T13:59:30.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.654+0000 7f62b1cc1700 1 -- 172.21.15.33:0/3910043159 learned_addr learned my addr 172.21.15.33:0/3910043159 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:30.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.654+0000 7f62b1cc1700 1 -- 172.21.15.33:0/3910043159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a0eda2910 msgr2=0x556a0eda0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T13:59:30.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.654+0000 7f62b1cc1700 1 --2- 172.21.15.33:0/3910043159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a0eda2910 0x556a0eda0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:30.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.654+0000 7f62b1cc1700 1 -- 172.21.15.33:0/3910043159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a0ed9e2f0 msgr2=0x556a0eda7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T13:59:30.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.654+0000 7f62b24c2700 1 --2- 172.21.15.33:0/3910043159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a0ed9e2f0 0x556a0eda7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:30.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.654+0000 7f62b1cc1700 1 --2- 172.21.15.33:0/3910043159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a0ed9e2f0 0x556a0eda7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:30.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.654+0000 7f62b1cc1700 1 -- 172.21.15.33:0/3910043159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a0ed9a160 con 0x556a0ed9ec10 2024-05-15T13:59:30.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.655+0000 7f62b1cc1700 1 --2- 172.21.15.33:0/3910043159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a0ed9ec10 0x556a0eda0340 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f629c010710 tx=0x7f629c010be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e614d88dacdabf8b server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:30.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.655+0000 7f6297fff700 1 -- 172.21.15.33:0/3910043159 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c035070 con 0x556a0ed9ec10 2024-05-15T13:59:30.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.655+0000 7f6297fff700 1 -- 172.21.15.33:0/3910043159 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f629c01de50 con 0x556a0ed9ec10 2024-05-15T13:59:30.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.655+0000 7f6297fff700 1 -- 172.21.15.33:0/3910043159 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c021050 con 0x556a0ed9ec10 2024-05-15T13:59:30.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.655+0000 7f62befbcb80 1 -- 172.21.15.33:0/3910043159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a0ed9ec10 msgr2=0x556a0eda0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:30.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.655+0000 7f62befbcb80 1 --2- 172.21.15.33:0/3910043159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a0ed9ec10 0x556a0eda0340 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f629c010710 tx=0x7f629c010be0 comp rx=0 tx=0).stop 2024-05-15T13:59:30.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.655+0000 7f62befbcb80 1 -- 172.21.15.33:0/3910043159 shutdown_connections 2024-05-15T13:59:30.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.655+0000 7f62befbcb80 1 --2- 172.21.15.33:0/3910043159 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a0eda2910 0x556a0eda0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:30.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.655+0000 7f62befbcb80 1 --2- 172.21.15.33:0/3910043159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a0ed9e2f0 0x556a0eda7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:30.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.655+0000 7f62befbcb80 1 --2- 172.21.15.33:0/3910043159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a0ed9ec10 0x556a0eda0340 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:30.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.655+0000 7f62befbcb80 1 -- 172.21.15.33:0/3910043159 >> 172.21.15.33:0/3910043159 conn(0x556a0ecc9200 msgr2=0x556a0ed92dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:30.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.655+0000 7f62befbcb80 1 -- 172.21.15.33:0/3910043159 shutdown_connections 2024-05-15T13:59:30.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.655+0000 7f62befbcb80 1 -- 172.21.15.33:0/3910043159 wait complete. 2024-05-15T13:59:30.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.656+0000 7f62befbcb80 1 -- start start 2024-05-15T13:59:30.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.656+0000 7f62befbcb80 1 Processor -- start 2024-05-15T13:59:30.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.656+0000 7f62befbcb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a0ed9e2f0 0x556a0edbd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:30.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.656+0000 7f62befbcb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a0ed9ec10 0x556a0edbd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:30.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.656+0000 7f62befbcb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a0eda2910 0x556a0edbdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:30.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.656+0000 7f62befbcb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556a0edb8c20 con 0x556a0eda2910 2024-05-15T13:59:30.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.656+0000 7f62befbcb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556a0edb8550 con 0x556a0ed9e2f0 2024-05-15T13:59:30.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.656+0000 7f62befbcb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x556a0edb92f0 con 0x556a0ed9ec10 2024-05-15T13:59:30.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.656+0000 7f62befbcb80 10 client.0 _ll_register_callbacks cb 0x556a0ec83530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T13:59:30.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.656+0000 7f62b2cc3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a0eda2910 0x556a0edbdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:30.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.656+0000 7f62b2cc3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a0eda2910 0x556a0edbdc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57802/0 (socket says 172.21.15.33:57802) 2024-05-15T13:59:30.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.656+0000 7f62b2cc3700 1 -- 172.21.15.33:0/3475918543 learned_addr learned my addr 172.21.15.33:0/3475918543 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:30.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.656+0000 7f62b1cc1700 1 --2- 172.21.15.33:0/3475918543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a0ed9ec10 0x556a0edbd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:30.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.656+0000 7f62b24c2700 1 --2- 172.21.15.33:0/3475918543 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a0ed9e2f0 0x556a0edbd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:30.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.657+0000 7f62b2cc3700 1 -- 172.21.15.33:0/3475918543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a0ed9ec10 msgr2=0x556a0edbd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T13:59:30.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.657+0000 7f62b2cc3700 1 --2- 172.21.15.33:0/3475918543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a0ed9ec10 0x556a0edbd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:30.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.657+0000 7f62b2cc3700 1 -- 172.21.15.33:0/3475918543 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a0ed9e2f0 msgr2=0x556a0edbd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T13:59:30.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.657+0000 7f62b2cc3700 1 --2- 172.21.15.33:0/3475918543 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a0ed9e2f0 0x556a0edbd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:30.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.657+0000 7f62b2cc3700 1 -- 172.21.15.33:0/3475918543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a0ee22da0 con 0x556a0eda2910 2024-05-15T13:59:30.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.657+0000 7f62b2cc3700 1 --2- 172.21.15.33:0/3475918543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a0eda2910 0x556a0edbdc50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f62a40263e0 tx=0x7f62a4006260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e8eb7d25d7c081 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:30.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.657+0000 7f6295ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T13:59:30.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.657+0000 7f6295ffb700 1 -- 172.21.15.33:0/3475918543 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a4009c90 con 0x556a0eda2910 2024-05-15T13:59:30.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.657+0000 7f6295ffb700 1 -- 172.21.15.33:0/3475918543 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f62a400f040 con 0x556a0eda2910 2024-05-15T13:59:30.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.657+0000 7f62befbcb80 1 -- 172.21.15.33:0/3475918543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f629c01de50 con 0x556a0eda2910 2024-05-15T13:59:30.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.657+0000 7f6295ffb700 1 -- 172.21.15.33:0/3475918543 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f62a4012d90 con 0x556a0eda2910 2024-05-15T13:59:30.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.657+0000 7f62befbcb80 10 client.14970 fetch_fsmap learned FSMap version 23 2024-05-15T13:59:30.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.657+0000 7f62befbcb80 1 -- 172.21.15.33:0/3475918543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x556a0ee23fc0 con 0x556a0eda2910 2024-05-15T13:59:30.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.657+0000 7f6295ffb700 1 -- 172.21.15.33:0/3475918543 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f629c01de50 con 0x556a0eda2910 2024-05-15T13:59:30.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.657+0000 7f6295ffb700 1 -- 172.21.15.33:0/3475918543 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f62a4011070 con 0x556a0eda2910 2024-05-15T13:59:30.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.658+0000 7f62befbcb80 10 client.14970 fetch_fsmap finished waiting for FSMap version 23 2024-05-15T13:59:30.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.658+0000 7f62befbcb80 10 client.14970 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-15T13:59:30.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.658+0000 7f62befbcb80 20 client.14970 populate_metadata read hostname 'smithi033' 2024-05-15T13:59:30.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.658+0000 7f62befbcb80 1 --2- 172.21.15.33:0/3475918543 >> [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] conn(0x556a0ee289a0 0x556a0ee276b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:30.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.658+0000 7f62befbcb80 4 client.14970 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-15T13:59:30.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.658+0000 7f62befbcb80 1 -- 172.21.15.33:0/3475918543 --> [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] -- command(tid 0: {"prefix": "status"}) -- 0x556a0ecfeb90 con 0x556a0ee289a0 2024-05-15T13:59:30.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.658+0000 7f62b24c2700 1 --2- 172.21.15.33:0/3475918543 >> [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] conn(0x556a0ee289a0 0x556a0ee276b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:30.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.658+0000 7f62b24c2700 1 --2- 172.21.15.33:0/3475918543 >> [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] conn(0x556a0ee289a0 0x556a0ee276b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=36332e285b6acc1a server_cookie=c59511fce54ec3a5 in_seq=0 out_seq=0 2024-05-15T13:59:30.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.658+0000 7f6295ffb700 10 client.14970 ms_handle_connect on v2:172.21.15.52:6838/2696044471 2024-05-15T13:59:30.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.658+0000 7f6295ffb700 1 -- 172.21.15.33:0/3475918543 <== mds.2 v2:172.21.15.52:6838/2696044471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x556a0ecfeb90 con 0x556a0ee289a0 2024-05-15T13:59:30.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.658+0000 7f6295ffb700 10 client.14970 handle_command_reply: tid=0 2024-05-15T13:59:30.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.658+0000 7f62befbcb80 1 client.14970 shutdown 2024-05-15T13:59:30.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.659+0000 7f62befbcb80 1 -- 172.21.15.33:0/3475918543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a0eda2910 msgr2=0x556a0edbdc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:30.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.659+0000 7f62befbcb80 1 --2- 172.21.15.33:0/3475918543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a0eda2910 0x556a0edbdc50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f62a40263e0 tx=0x7f62a4006260 comp rx=0 tx=0).stop 2024-05-15T13:59:30.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.659+0000 7f62befbcb80 1 -- 172.21.15.33:0/3475918543 shutdown_connections 2024-05-15T13:59:30.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.659+0000 7f62befbcb80 1 --2- 172.21.15.33:0/3475918543 >> [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] conn(0x556a0ee289a0 0x556a0ee276b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:30.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.659+0000 7f62befbcb80 1 --2- 172.21.15.33:0/3475918543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556a0eda2910 0x556a0edbdc50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:30.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.659+0000 7f62befbcb80 1 --2- 172.21.15.33:0/3475918543 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556a0ed9e2f0 0x556a0edbd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:30.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.659+0000 7f62befbcb80 1 --2- 172.21.15.33:0/3475918543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556a0ed9ec10 0x556a0edbd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:30.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.659+0000 7f62befbcb80 1 -- 172.21.15.33:0/3475918543 >> 172.21.15.33:0/3475918543 conn(0x556a0ecc9200 msgr2=0x556a0eda17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:30.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.659+0000 7f6295ffb700 0 client.14970 ms_handle_reset on v2:172.21.15.52:6838/2696044471 2024-05-15T13:59:30.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.659+0000 7f62befbcb80 1 -- 172.21.15.33:0/3475918543 shutdown_connections 2024-05-15T13:59:30.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.659+0000 7f62befbcb80 1 -- 172.21.15.33:0/3475918543 wait complete. 2024-05-15T13:59:30.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.659+0000 7f62befbcb80 20 client.14970 trim_cache size 0 max 16384 2024-05-15T13:59:30.667 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T13:59:30.667 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "bca78ca4-12c1-11ef-bc98-c7b262605968", 2024-05-15T13:59:30.667 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 2, 2024-05-15T13:59:30.667 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24461, 2024-05-15T13:59:30.667 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-05-15T13:59:30.667 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-05-15T13:59:30.667 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-05-15T13:59:30.667 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 118.17611597699999, 2024-05-15T13:59:30.667 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 23, 2024-05-15T13:59:30.667 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 82, 2024-05-15T13:59:30.667 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 80, 2024-05-15T13:59:30.667 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 146.86777855899999 2024-05-15T13:59:30.667 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T13:59:30.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.659+0000 7f62b4726700 1 -- 172.21.15.33:0/4166501548 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f62980b2880 msgr2=0x7f62980b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:30.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.659+0000 7f62b4726700 1 --2- 172.21.15.33:0/4166501548 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f62980b2880 0x7f62980b4d40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f629c005fd0 tx=0x7f629c00a560 comp rx=0 tx=0).stop 2024-05-15T13:59:30.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.659+0000 7f62b4726700 1 -- 172.21.15.33:0/4166501548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62ac101f00 msgr2=0x7f62ac1030e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:30.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.659+0000 7f62b4726700 1 --2- 172.21.15.33:0/4166501548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62ac101f00 0x7f62ac1030e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f62a800c990 tx=0x7f62a800cd50 comp rx=0 tx=0).stop 2024-05-15T13:59:30.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.659+0000 7f62b4726700 1 -- 172.21.15.33:0/4166501548 shutdown_connections 2024-05-15T13:59:30.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.659+0000 7f62b4726700 1 --2- 172.21.15.33:0/4166501548 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f62980b2880 0x7f62980b4d40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:30.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.660+0000 7f62b4726700 1 --2- 172.21.15.33:0/4166501548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62ac108620 0x7f62ac103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:30.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.660+0000 7f62b4726700 1 --2- 172.21.15.33:0/4166501548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62ac101f00 0x7f62ac1030e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:30.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.660+0000 7f62b4726700 1 --2- 172.21.15.33:0/4166501548 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62ac1028c0 0x7f62ac103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:30.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.660+0000 7f62b4726700 1 -- 172.21.15.33:0/4166501548 >> 172.21.15.33:0/4166501548 conn(0x7f62ac077780 msgr2=0x7f62ac1008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:30.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.660+0000 7f62b4726700 1 -- 172.21.15.33:0/4166501548 shutdown_connections 2024-05-15T13:59:30.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:30.660+0000 7f62b4726700 1 -- 172.21.15.33:0/4166501548 wait complete. 2024-05-15T13:59:31.362 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph tell mds.c status 2024-05-15T13:59:31.612 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:59:32.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:32 smithi052 ceph-mon[36190]: pgmap v355: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T13:59:32.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:32 smithi081 ceph-mon[36434]: pgmap v355: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T13:59:32.647 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:32 smithi033 ceph-mon[31650]: pgmap v355: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T13:59:33.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.156+0000 7f1887334700 1 -- 172.21.15.33:0/1729020508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18801095b0 msgr2=0x7f1880109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:33.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.156+0000 7f1887334700 1 --2- 172.21.15.33:0/1729020508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18801095b0 0x7f1880109990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f187000b0a0 tx=0x7f187000b3b0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.156+0000 7f1887334700 1 -- 172.21.15.33:0/1729020508 shutdown_connections 2024-05-15T13:59:33.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.156+0000 7f1887334700 1 --2- 172.21.15.33:0/1729020508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f188006bbe0 0x7f18801103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.156+0000 7f1887334700 1 --2- 172.21.15.33:0/1729020508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18801095b0 0x7f1880109990 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.156+0000 7f1887334700 1 --2- 172.21.15.33:0/1729020508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f188006b220 0x7f188006b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.156+0000 7f1887334700 1 -- 172.21.15.33:0/1729020508 >> 172.21.15.33:0/1729020508 conn(0x7f18800793f0 msgr2=0x7f1880079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:33.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.156+0000 7f1887334700 1 -- 172.21.15.33:0/1729020508 shutdown_connections 2024-05-15T13:59:33.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.156+0000 7f1887334700 1 -- 172.21.15.33:0/1729020508 wait complete. 2024-05-15T13:59:33.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.157+0000 7f1887334700 1 Processor -- start 2024-05-15T13:59:33.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.157+0000 7f1887334700 1 -- start start 2024-05-15T13:59:33.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.157+0000 7f1887334700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f188006b220 0x7f188019f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:33.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.157+0000 7f1887334700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f188006bbe0 0x7f188019fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:33.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.157+0000 7f1887334700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18801095b0 0x7f18801999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:33.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.157+0000 7f1887334700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1880113740 con 0x7f188006bbe0 2024-05-15T13:59:33.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.157+0000 7f1887334700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f18801135c0 con 0x7f188006b220 2024-05-15T13:59:33.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.157+0000 7f1887334700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f18801138c0 con 0x7f18801095b0 2024-05-15T13:59:33.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.157+0000 7f18848cf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f188006bbe0 0x7f188019fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:33.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.157+0000 7f18848cf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f188006bbe0 0x7f188019fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40922/0 (socket says 172.21.15.33:40922) 2024-05-15T13:59:33.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.157+0000 7f18848cf700 1 -- 172.21.15.33:0/3769670627 learned_addr learned my addr 172.21.15.33:0/3769670627 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:33.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.157+0000 7f18858d1700 1 --2- 172.21.15.33:0/3769670627 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18801095b0 0x7f18801999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:33.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.157+0000 7f18850d0700 1 --2- 172.21.15.33:0/3769670627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f188006b220 0x7f188019f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:33.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.158+0000 7f18848cf700 1 -- 172.21.15.33:0/3769670627 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18801095b0 msgr2=0x7f18801999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:33.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.158+0000 7f18848cf700 1 --2- 172.21.15.33:0/3769670627 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18801095b0 0x7f18801999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.158+0000 7f18848cf700 1 -- 172.21.15.33:0/3769670627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f188006b220 msgr2=0x7f188019f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:33.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.158+0000 7f18848cf700 1 --2- 172.21.15.33:0/3769670627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f188006b220 0x7f188019f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.158+0000 7f18848cf700 1 -- 172.21.15.33:0/3769670627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188019a140 con 0x7f188006bbe0 2024-05-15T13:59:33.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.158+0000 7f18848cf700 1 --2- 172.21.15.33:0/3769670627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f188006bbe0 0x7f188019fd60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f187400c9b0 tx=0x7f187400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:33.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.158+0000 7f187e7fc700 1 -- 172.21.15.33:0/3769670627 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1874007b00 con 0x7f188006bbe0 2024-05-15T13:59:33.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.158+0000 7f1887334700 1 -- 172.21.15.33:0/3769670627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f188019a430 con 0x7f188006bbe0 2024-05-15T13:59:33.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.158+0000 7f187e7fc700 1 -- 172.21.15.33:0/3769670627 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f187400ce50 con 0x7f188006bbe0 2024-05-15T13:59:33.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.158+0000 7f187e7fc700 1 -- 172.21.15.33:0/3769670627 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1874018660 con 0x7f188006bbe0 2024-05-15T13:59:33.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.158+0000 7f1887334700 1 -- 172.21.15.33:0/3769670627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18801a66f0 con 0x7f188006bbe0 2024-05-15T13:59:33.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.161+0000 7f187e7fc700 1 -- 172.21.15.33:0/3769670627 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f187400f450 con 0x7f188006bbe0 2024-05-15T13:59:33.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.161+0000 7f187e7fc700 1 --2- 172.21.15.33:0/3769670627 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f18680b2880 0x7f18680b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:33.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.161+0000 7f18850d0700 1 --2- 172.21.15.33:0/3769670627 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f18680b2880 0x7f18680b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:33.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.161+0000 7f187e7fc700 1 -- 172.21.15.33:0/3769670627 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f18740e2140 con 0x7f188006bbe0 2024-05-15T13:59:33.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.162+0000 7f1891bcab80 1 Processor -- start 2024-05-15T13:59:33.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.162+0000 7f1891bcab80 1 -- start start 2024-05-15T13:59:33.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.162+0000 7f18850d0700 1 --2- 172.21.15.33:0/3769670627 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f18680b2880 0x7f18680b4d40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1870009fd0 tx=0x7f187000b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:33.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.162+0000 7f1891bcab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fc938bfa0 0x559fc938c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:33.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.162+0000 7f1891bcab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559fc938c8c0 0x559fc93867a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:33.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.162+0000 7f1891bcab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fc9386dc0 0x559fc9387220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:33.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.162+0000 7f1891bcab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559fc92e4d20 con 0x559fc938bfa0 2024-05-15T13:59:33.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.162+0000 7f1891bcab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559fc92d48c0 con 0x559fc9386dc0 2024-05-15T13:59:33.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.162+0000 7f1891bcab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x559fc92d8900 con 0x559fc938c8c0 2024-05-15T13:59:33.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.162+0000 7f18850d0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fc938bfa0 0x559fc938c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:33.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.162+0000 7f18850d0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fc938bfa0 0x559fc938c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40926/0 (socket says 172.21.15.33:40926) 2024-05-15T13:59:33.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.162+0000 7f18850d0700 1 -- 172.21.15.33:0/3218213211 learned_addr learned my addr 172.21.15.33:0/3218213211 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:33.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.162+0000 7f18858d1700 1 --2- 172.21.15.33:0/3218213211 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fc9386dc0 0x559fc9387220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:33.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.162+0000 7f18848cf700 1 --2- 172.21.15.33:0/3218213211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559fc938c8c0 0x559fc93867a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:33.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.162+0000 7f18850d0700 1 -- 172.21.15.33:0/3218213211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559fc938c8c0 msgr2=0x559fc93867a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T13:59:33.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.162+0000 7f18850d0700 1 --2- 172.21.15.33:0/3218213211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559fc938c8c0 0x559fc93867a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.162+0000 7f18850d0700 1 -- 172.21.15.33:0/3218213211 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fc9386dc0 msgr2=0x559fc9387220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T13:59:33.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.162+0000 7f18850d0700 1 --2- 172.21.15.33:0/3218213211 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fc9386dc0 0x559fc9387220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.162+0000 7f18850d0700 1 -- 172.21.15.33:0/3218213211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fc927df60 con 0x559fc938bfa0 2024-05-15T13:59:33.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.162+0000 7f18858d1700 1 --2- 172.21.15.33:0/3218213211 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fc9386dc0 0x559fc9387220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T13:59:33.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.163+0000 7f18850d0700 1 --2- 172.21.15.33:0/3218213211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fc938bfa0 0x559fc938c380 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f187000b0a0 tx=0x7f187001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e00363337cc17da8 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:33.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.163+0000 7f18677fe700 1 -- 172.21.15.33:0/3218213211 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187002c030 con 0x559fc938bfa0 2024-05-15T13:59:33.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.163+0000 7f18677fe700 1 -- 172.21.15.33:0/3218213211 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1870007df0 con 0x559fc938bfa0 2024-05-15T13:59:33.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.163+0000 7f18677fe700 1 -- 172.21.15.33:0/3218213211 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1870028c60 con 0x559fc938bfa0 2024-05-15T13:59:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.164+0000 7f1891bcab80 1 -- 172.21.15.33:0/3218213211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fc938bfa0 msgr2=0x559fc938c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.164+0000 7f1891bcab80 1 --2- 172.21.15.33:0/3218213211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fc938bfa0 0x559fc938c380 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f187000b0a0 tx=0x7f187001d000 comp rx=0 tx=0).stop 2024-05-15T13:59:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.164+0000 7f1891bcab80 1 -- 172.21.15.33:0/3218213211 shutdown_connections 2024-05-15T13:59:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.164+0000 7f1891bcab80 1 --2- 172.21.15.33:0/3218213211 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fc9386dc0 0x559fc9387220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.164+0000 7f1891bcab80 1 --2- 172.21.15.33:0/3218213211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fc938bfa0 0x559fc938c380 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.164+0000 7f1891bcab80 1 --2- 172.21.15.33:0/3218213211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559fc938c8c0 0x559fc93867a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.164+0000 7f1891bcab80 1 -- 172.21.15.33:0/3218213211 >> 172.21.15.33:0/3218213211 conn(0x559fc92b3d60 msgr2=0x559fc9380e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.164+0000 7f1891bcab80 1 -- 172.21.15.33:0/3218213211 shutdown_connections 2024-05-15T13:59:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.164+0000 7f1891bcab80 1 -- 172.21.15.33:0/3218213211 wait complete. 2024-05-15T13:59:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.164+0000 7f1891bcab80 1 -- start start 2024-05-15T13:59:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.164+0000 7f1891bcab80 1 Processor -- start 2024-05-15T13:59:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.165+0000 7f1891bcab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fc9386dc0 0x559fc93925f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.165+0000 7f1891bcab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fc938bfa0 0x559fc938f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:33.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.165+0000 7f1891bcab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559fc938c8c0 0x559fc938fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:33.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.165+0000 7f1891bcab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559fc91daf50 con 0x559fc938bfa0 2024-05-15T13:59:33.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.165+0000 7f1891bcab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559fc923cd50 con 0x559fc9386dc0 2024-05-15T13:59:33.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.165+0000 7f1891bcab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x559fc9393270 con 0x559fc938c8c0 2024-05-15T13:59:33.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.165+0000 7f1891bcab80 10 client.0 _ll_register_callbacks cb 0x559fc926f900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T13:59:33.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.165+0000 7f18848cf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fc938bfa0 0x559fc938f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:33.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.165+0000 7f18848cf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fc938bfa0 0x559fc938f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40942/0 (socket says 172.21.15.33:40942) 2024-05-15T13:59:33.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.165+0000 7f18848cf700 1 -- 172.21.15.33:0/4033637675 learned_addr learned my addr 172.21.15.33:0/4033637675 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:33.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.165+0000 7f18850d0700 1 --2- 172.21.15.33:0/4033637675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fc9386dc0 0x559fc93925f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:33.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.165+0000 7f18858d1700 1 --2- 172.21.15.33:0/4033637675 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559fc938c8c0 0x559fc938fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:33.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.165+0000 7f18848cf700 1 -- 172.21.15.33:0/4033637675 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559fc938c8c0 msgr2=0x559fc938fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T13:59:33.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.165+0000 7f18848cf700 1 --2- 172.21.15.33:0/4033637675 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559fc938c8c0 0x559fc938fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.165+0000 7f18848cf700 1 -- 172.21.15.33:0/4033637675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fc9386dc0 msgr2=0x559fc93925f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T13:59:33.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.165+0000 7f18848cf700 1 --2- 172.21.15.33:0/4033637675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fc9386dc0 0x559fc93925f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.165+0000 7f18848cf700 1 -- 172.21.15.33:0/4033637675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fc9224880 con 0x559fc938bfa0 2024-05-15T13:59:33.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.165+0000 7f18850d0700 1 --2- 172.21.15.33:0/4033637675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fc9386dc0 0x559fc93925f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T13:59:33.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.165+0000 7f18848cf700 1 --2- 172.21.15.33:0/4033637675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fc938bfa0 0x559fc938f990 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f18740a96f0 tx=0x7f18740a8ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd172f36c5e872f server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:33.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.165+0000 7f1864ff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T13:59:33.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.165+0000 7f1864ff9700 1 -- 172.21.15.33:0/4033637675 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18740a9760 con 0x559fc938bfa0 2024-05-15T13:59:33.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.165+0000 7f1864ff9700 1 -- 172.21.15.33:0/4033637675 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f1874007ca0 con 0x559fc938bfa0 2024-05-15T13:59:33.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.165+0000 7f1891bcab80 1 -- 172.21.15.33:0/4033637675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1870007df0 con 0x559fc938bfa0 2024-05-15T13:59:33.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.166+0000 7f1864ff9700 1 -- 172.21.15.33:0/4033637675 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f18740adb10 con 0x559fc938bfa0 2024-05-15T13:59:33.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.166+0000 7f1891bcab80 10 client.14982 fetch_fsmap learned FSMap version 23 2024-05-15T13:59:33.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.166+0000 7f1891bcab80 1 -- 172.21.15.33:0/4033637675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x559fc928e710 con 0x559fc938bfa0 2024-05-15T13:59:33.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.166+0000 7f1864ff9700 1 -- 172.21.15.33:0/4033637675 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1870007df0 con 0x559fc938bfa0 2024-05-15T13:59:33.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.166+0000 7f1864ff9700 1 -- 172.21.15.33:0/4033637675 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f18740becd0 con 0x559fc938bfa0 2024-05-15T13:59:33.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.166+0000 7f1891bcab80 10 client.14982 fetch_fsmap finished waiting for FSMap version 23 2024-05-15T13:59:33.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.166+0000 7f1891bcab80 10 client.14982 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-15T13:59:33.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.166+0000 7f1891bcab80 20 client.14982 populate_metadata read hostname 'smithi033' 2024-05-15T13:59:33.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.166+0000 7f1891bcab80 1 --2- 172.21.15.33:0/4033637675 >> [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] conn(0x559fc939fdc0 0x559fc93a1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:33.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.166+0000 7f1891bcab80 4 client.14982 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T13:59:33.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.166+0000 7f1891bcab80 1 -- 172.21.15.33:0/4033637675 --> [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fc92e6c70 con 0x559fc939fdc0 2024-05-15T13:59:33.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.166+0000 7f18850d0700 1 --2- 172.21.15.33:0/4033637675 >> [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] conn(0x559fc939fdc0 0x559fc93a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:33.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.167+0000 7f1864ff9700 10 client.14982 ms_handle_connect on v2:172.21.15.81:6836/744100239 2024-05-15T13:59:33.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.167+0000 7f18850d0700 1 --2- 172.21.15.33:0/4033637675 >> [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] conn(0x559fc939fdc0 0x559fc93a1230 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f6770cbea9f33b29 server_cookie=73dbd9c2fdfd7a74 in_seq=0 out_seq=0 2024-05-15T13:59:33.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.168+0000 7f1864ff9700 1 -- 172.21.15.33:0/4033637675 <== mds.1 v2:172.21.15.81:6836/744100239 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559fc92e6c70 con 0x559fc939fdc0 2024-05-15T13:59:33.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.168+0000 7f1864ff9700 10 client.14982 handle_command_reply: tid=0 2024-05-15T13:59:33.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.168+0000 7f1891bcab80 1 client.14982 shutdown 2024-05-15T13:59:33.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.168+0000 7f1891bcab80 1 -- 172.21.15.33:0/4033637675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fc938bfa0 msgr2=0x559fc938f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:33.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.168+0000 7f1891bcab80 1 --2- 172.21.15.33:0/4033637675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fc938bfa0 0x559fc938f990 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f18740a96f0 tx=0x7f18740a8ed0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.168+0000 7f1891bcab80 1 -- 172.21.15.33:0/4033637675 shutdown_connections 2024-05-15T13:59:33.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.168+0000 7f1891bcab80 1 --2- 172.21.15.33:0/4033637675 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559fc938c8c0 0x559fc938fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.168+0000 7f1891bcab80 1 --2- 172.21.15.33:0/4033637675 >> [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] conn(0x559fc939fdc0 0x559fc93a1230 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.168+0000 7f1891bcab80 1 --2- 172.21.15.33:0/4033637675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fc9386dc0 0x559fc93925f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.168+0000 7f1891bcab80 1 --2- 172.21.15.33:0/4033637675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fc938bfa0 0x559fc938f990 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.168+0000 7f1864ff9700 0 client.14982 ms_handle_reset on v2:172.21.15.81:6836/744100239 2024-05-15T13:59:33.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.168+0000 7f1891bcab80 1 -- 172.21.15.33:0/4033637675 >> 172.21.15.33:0/4033637675 conn(0x559fc92b3d60 msgr2=0x559fc9380630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:33.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.168+0000 7f1891bcab80 1 -- 172.21.15.33:0/4033637675 shutdown_connections 2024-05-15T13:59:33.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.168+0000 7f1891bcab80 1 -- 172.21.15.33:0/4033637675 wait complete. 2024-05-15T13:59:33.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.168+0000 7f1891bcab80 20 client.14982 trim_cache size 0 max 16384 2024-05-15T13:59:33.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.189+0000 7f1891bcab80 1 Processor -- start 2024-05-15T13:59:33.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.189+0000 7f1891bcab80 1 -- start start 2024-05-15T13:59:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.189+0000 7f1891bcab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fc9386dc0 0x559fc93871a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.190+0000 7f1891bcab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559fc938bfa0 0x559fc9380db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.190+0000 7f1891bcab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fc938c8c0 0x559fc93aa690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.190+0000 7f1891bcab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559fc939f570 con 0x559fc9386dc0 2024-05-15T13:59:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.190+0000 7f1891bcab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559fc939fc40 con 0x559fc938c8c0 2024-05-15T13:59:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.190+0000 7f1891bcab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x559fc93a0310 con 0x559fc938bfa0 2024-05-15T13:59:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.190+0000 7f18850d0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fc9386dc0 0x559fc93871a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.190+0000 7f18850d0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fc9386dc0 0x559fc93871a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40952/0 (socket says 172.21.15.33:40952) 2024-05-15T13:59:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.190+0000 7f18850d0700 1 -- 172.21.15.33:0/3655704138 learned_addr learned my addr 172.21.15.33:0/3655704138 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.190+0000 7f18848cf700 1 --2- 172.21.15.33:0/3655704138 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559fc938bfa0 0x559fc9380db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.190+0000 7f18850d0700 1 -- 172.21.15.33:0/3655704138 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559fc938bfa0 msgr2=0x559fc9380db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T13:59:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.190+0000 7f18850d0700 1 --2- 172.21.15.33:0/3655704138 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559fc938bfa0 0x559fc9380db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.190+0000 7f18850d0700 1 -- 172.21.15.33:0/3655704138 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fc938c8c0 msgr2=0x559fc93aa690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T13:59:33.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.190+0000 7f18850d0700 1 --2- 172.21.15.33:0/3655704138 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fc938c8c0 0x559fc93aa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.190+0000 7f18850d0700 1 -- 172.21.15.33:0/3655704138 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fc9392230 con 0x559fc9386dc0 2024-05-15T13:59:33.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.190+0000 7f18848cf700 1 --2- 172.21.15.33:0/3655704138 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559fc938bfa0 0x559fc9380db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T13:59:33.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.190+0000 7f18850d0700 1 --2- 172.21.15.33:0/3655704138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fc9386dc0 0x559fc93871a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1870003c60 tx=0x7f1870009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee738c208e1ee99f server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:33.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.190+0000 7f1866ffd700 1 -- 172.21.15.33:0/3655704138 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1870020040 con 0x559fc9386dc0 2024-05-15T13:59:33.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.190+0000 7f1866ffd700 1 -- 172.21.15.33:0/3655704138 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1870007db0 con 0x559fc9386dc0 2024-05-15T13:59:33.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.190+0000 7f1866ffd700 1 -- 172.21.15.33:0/3655704138 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187002c040 con 0x559fc9386dc0 2024-05-15T13:59:33.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.191+0000 7f1891bcab80 1 -- 172.21.15.33:0/3655704138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fc9386dc0 msgr2=0x559fc93871a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:33.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.191+0000 7f1891bcab80 1 --2- 172.21.15.33:0/3655704138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fc9386dc0 0x559fc93871a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1870003c60 tx=0x7f1870009de0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.191+0000 7f1891bcab80 1 -- 172.21.15.33:0/3655704138 shutdown_connections 2024-05-15T13:59:33.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.191+0000 7f1891bcab80 1 --2- 172.21.15.33:0/3655704138 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fc938c8c0 0x559fc93aa690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.191+0000 7f1891bcab80 1 --2- 172.21.15.33:0/3655704138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fc9386dc0 0x559fc93871a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.191+0000 7f1891bcab80 1 --2- 172.21.15.33:0/3655704138 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559fc938bfa0 0x559fc9380db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.191+0000 7f1891bcab80 1 -- 172.21.15.33:0/3655704138 >> 172.21.15.33:0/3655704138 conn(0x559fc92b3d60 msgr2=0x559fc9389a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:33.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.191+0000 7f1891bcab80 1 -- 172.21.15.33:0/3655704138 shutdown_connections 2024-05-15T13:59:33.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.191+0000 7f1891bcab80 1 -- 172.21.15.33:0/3655704138 wait complete. 2024-05-15T13:59:33.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.191+0000 7f1891bcab80 1 -- start start 2024-05-15T13:59:33.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.191+0000 7f1891bcab80 1 Processor -- start 2024-05-15T13:59:33.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.192+0000 7f1891bcab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fc9386dc0 0x559fc93aad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:33.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.192+0000 7f1891bcab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559fc938bfa0 0x559fc9380180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:33.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.192+0000 7f1891bcab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fc938c8c0 0x559fc9380870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:33.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.192+0000 7f1891bcab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559fc93a10b0 con 0x559fc9386dc0 2024-05-15T13:59:33.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.192+0000 7f1891bcab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559fc93a09e0 con 0x559fc938c8c0 2024-05-15T13:59:33.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.192+0000 7f1891bcab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x559fc93a1780 con 0x559fc938bfa0 2024-05-15T13:59:33.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.192+0000 7f1891bcab80 10 client.0 _ll_register_callbacks cb 0x559fc926f900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T13:59:33.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.192+0000 7f18850d0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fc9386dc0 0x559fc93aad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:33.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.192+0000 7f18850d0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fc9386dc0 0x559fc93aad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40964/0 (socket says 172.21.15.33:40964) 2024-05-15T13:59:33.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.192+0000 7f18850d0700 1 -- 172.21.15.33:0/4243303881 learned_addr learned my addr 172.21.15.33:0/4243303881 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:33.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.192+0000 7f18848cf700 1 --2- 172.21.15.33:0/4243303881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559fc938bfa0 0x559fc9380180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:33.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.192+0000 7f18858d1700 1 --2- 172.21.15.33:0/4243303881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fc938c8c0 0x559fc9380870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.192+0000 7f18850d0700 1 -- 172.21.15.33:0/4243303881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559fc938bfa0 msgr2=0x559fc9380180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T13:59:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.192+0000 7f18850d0700 1 --2- 172.21.15.33:0/4243303881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559fc938bfa0 0x559fc9380180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.192+0000 7f18850d0700 1 -- 172.21.15.33:0/4243303881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fc938c8c0 msgr2=0x559fc9380870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T13:59:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.192+0000 7f18850d0700 1 --2- 172.21.15.33:0/4243303881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fc938c8c0 0x559fc9380870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.192+0000 7f18850d0700 1 -- 172.21.15.33:0/4243303881 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fc93a8920 con 0x559fc9386dc0 2024-05-15T13:59:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.192+0000 7f18858d1700 1 --2- 172.21.15.33:0/4243303881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fc938c8c0 0x559fc9380870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T13:59:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.192+0000 7f18848cf700 1 --2- 172.21.15.33:0/4243303881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559fc938bfa0 0x559fc9380180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T13:59:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.193+0000 7f18850d0700 1 --2- 172.21.15.33:0/4243303881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fc9386dc0 0x559fc93aad80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f187001f040 tx=0x7f1870007bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b7e55017829d37 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.193+0000 7f1864ff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T13:59:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.193+0000 7f1864ff9700 1 -- 172.21.15.33:0/4243303881 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187002c030 con 0x559fc9386dc0 2024-05-15T13:59:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.193+0000 7f1864ff9700 1 -- 172.21.15.33:0/4243303881 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f1870028a50 con 0x559fc9386dc0 2024-05-15T13:59:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.193+0000 7f1891bcab80 1 -- 172.21.15.33:0/4243303881 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1870007db0 con 0x559fc9386dc0 2024-05-15T13:59:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.193+0000 7f1864ff9700 1 -- 172.21.15.33:0/4243303881 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f187004d050 con 0x559fc9386dc0 2024-05-15T13:59:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.193+0000 7f1891bcab80 10 client.14988 fetch_fsmap learned FSMap version 23 2024-05-15T13:59:33.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.193+0000 7f1891bcab80 1 -- 172.21.15.33:0/4243303881 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x559fc9413bc0 con 0x559fc9386dc0 2024-05-15T13:59:33.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.193+0000 7f1864ff9700 1 -- 172.21.15.33:0/4243303881 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1870007db0 con 0x559fc9386dc0 2024-05-15T13:59:33.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.193+0000 7f1864ff9700 1 -- 172.21.15.33:0/4243303881 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f1870011070 con 0x559fc9386dc0 2024-05-15T13:59:33.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.193+0000 7f1891bcab80 10 client.14988 fetch_fsmap finished waiting for FSMap version 23 2024-05-15T13:59:33.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.194+0000 7f1891bcab80 10 client.14988 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-15T13:59:33.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.194+0000 7f1891bcab80 20 client.14988 populate_metadata read hostname 'smithi033' 2024-05-15T13:59:33.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.194+0000 7f1891bcab80 1 --2- 172.21.15.33:0/4243303881 >> [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] conn(0x559fc9418090 0x559fc9416da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:33.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.194+0000 7f1891bcab80 4 client.14988 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-15T13:59:33.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.194+0000 7f1891bcab80 1 -- 172.21.15.33:0/4243303881 --> [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] -- command(tid 0: {"prefix": "status"}) -- 0x559fc92e0450 con 0x559fc9418090 2024-05-15T13:59:33.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.194+0000 7f18848cf700 1 --2- 172.21.15.33:0/4243303881 >> [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] conn(0x559fc9418090 0x559fc9416da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:33.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.194+0000 7f1864ff9700 10 client.14988 ms_handle_connect on v2:172.21.15.81:6836/744100239 2024-05-15T13:59:33.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.194+0000 7f18848cf700 1 --2- 172.21.15.33:0/4243303881 >> [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] conn(0x559fc9418090 0x559fc9416da0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=48beb8969f97b038 server_cookie=7f65ee7ffc843bec in_seq=0 out_seq=0 2024-05-15T13:59:33.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.194+0000 7f1864ff9700 1 -- 172.21.15.33:0/4243303881 <== mds.1 v2:172.21.15.81:6836/744100239 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x559fc92e0450 con 0x559fc9418090 2024-05-15T13:59:33.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.194+0000 7f1864ff9700 10 client.14988 handle_command_reply: tid=0 2024-05-15T13:59:33.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.194+0000 7f1891bcab80 1 client.14988 shutdown 2024-05-15T13:59:33.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.195+0000 7f1891bcab80 1 -- 172.21.15.33:0/4243303881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fc9386dc0 msgr2=0x559fc93aad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:33.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.195+0000 7f1891bcab80 1 --2- 172.21.15.33:0/4243303881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fc9386dc0 0x559fc93aad80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f187001f040 tx=0x7f1870007bb0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.195+0000 7f1891bcab80 1 -- 172.21.15.33:0/4243303881 shutdown_connections 2024-05-15T13:59:33.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.195+0000 7f1891bcab80 1 --2- 172.21.15.33:0/4243303881 >> [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] conn(0x559fc9418090 0x559fc9416da0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.195+0000 7f1891bcab80 1 --2- 172.21.15.33:0/4243303881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fc938c8c0 0x559fc9380870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.195+0000 7f1891bcab80 1 --2- 172.21.15.33:0/4243303881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fc9386dc0 0x559fc93aad80 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.195+0000 7f1864ff9700 0 client.14988 ms_handle_reset on v2:172.21.15.81:6836/744100239 2024-05-15T13:59:33.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.195+0000 7f1891bcab80 1 --2- 172.21.15.33:0/4243303881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559fc938bfa0 0x559fc9380180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.195+0000 7f1891bcab80 1 -- 172.21.15.33:0/4243303881 >> 172.21.15.33:0/4243303881 conn(0x559fc92b3d60 msgr2=0x559fc938f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:33.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.195+0000 7f1891bcab80 1 -- 172.21.15.33:0/4243303881 shutdown_connections 2024-05-15T13:59:33.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.195+0000 7f1891bcab80 1 -- 172.21.15.33:0/4243303881 wait complete. 2024-05-15T13:59:33.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.195+0000 7f1891bcab80 20 client.14988 trim_cache size 0 max 16384 2024-05-15T13:59:33.203 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T13:59:33.203 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "bca78ca4-12c1-11ef-bc98-c7b262605968", 2024-05-15T13:59:33.203 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 1, 2024-05-15T13:59:33.203 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24463, 2024-05-15T13:59:33.203 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-05-15T13:59:33.203 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-05-15T13:59:33.203 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-05-15T13:59:33.203 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 121.72038470699999, 2024-05-15T13:59:33.203 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 23, 2024-05-15T13:59:33.203 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 82, 2024-05-15T13:59:33.203 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 80, 2024-05-15T13:59:33.203 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 164.944323309 2024-05-15T13:59:33.203 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T13:59:33.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.195+0000 7f1887334700 1 -- 172.21.15.33:0/3769670627 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f18680b2880 msgr2=0x7f18680b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:33.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.195+0000 7f1887334700 1 --2- 172.21.15.33:0/3769670627 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f18680b2880 0x7f18680b4d40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1870009fd0 tx=0x7f187000b7b0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.195+0000 7f1887334700 1 -- 172.21.15.33:0/3769670627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f188006bbe0 msgr2=0x7f188019fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:33.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.195+0000 7f1887334700 1 --2- 172.21.15.33:0/3769670627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f188006bbe0 0x7f188019fd60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f187400c9b0 tx=0x7f187400ccc0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.196+0000 7f1887334700 1 -- 172.21.15.33:0/3769670627 shutdown_connections 2024-05-15T13:59:33.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.196+0000 7f1887334700 1 --2- 172.21.15.33:0/3769670627 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f18680b2880 0x7f18680b4d40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.196+0000 7f1887334700 1 --2- 172.21.15.33:0/3769670627 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18801095b0 0x7f18801999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.196+0000 7f1887334700 1 --2- 172.21.15.33:0/3769670627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f188006b220 0x7f188019f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.196+0000 7f1887334700 1 --2- 172.21.15.33:0/3769670627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f188006bbe0 0x7f188019fd60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:33.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.196+0000 7f1887334700 1 -- 172.21.15.33:0/3769670627 >> 172.21.15.33:0/3769670627 conn(0x7f18800793f0 msgr2=0x7f1880101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:33.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.196+0000 7f1887334700 1 -- 172.21.15.33:0/3769670627 shutdown_connections 2024-05-15T13:59:33.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:33.196+0000 7f1887334700 1 -- 172.21.15.33:0/3769670627 wait complete. 2024-05-15T13:59:33.835 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph tell mds.i status 2024-05-15T13:59:34.086 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:59:34.491 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:34 smithi033 ceph-mon[31650]: pgmap v356: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-15T13:59:34.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:34 smithi052 ceph-mon[36190]: pgmap v356: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-15T13:59:34.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:34 smithi081 ceph-mon[36434]: pgmap v356: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-15T13:59:35.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.696+0000 7f1658c72700 1 -- 172.21.15.33:0/494141269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16541088f0 msgr2=0x7f1654110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:35.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.696+0000 7f1658c72700 1 --2- 172.21.15.33:0/494141269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16541088f0 0x7f1654110470 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f164c00b0d0 tx=0x7f164c00b3e0 comp rx=0 tx=0).stop 2024-05-15T13:59:35.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.697+0000 7f1658c72700 1 -- 172.21.15.33:0/494141269 shutdown_connections 2024-05-15T13:59:35.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.697+0000 7f1658c72700 1 --2- 172.21.15.33:0/494141269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16541088f0 0x7f1654110470 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:35.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.697+0000 7f1658c72700 1 --2- 172.21.15.33:0/494141269 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16541037f0 0x7f1654103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:35.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.697+0000 7f1658c72700 1 --2- 172.21.15.33:0/494141269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16541041a0 0x7f16541081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:35.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.697+0000 7f1658c72700 1 -- 172.21.15.33:0/494141269 >> 172.21.15.33:0/494141269 conn(0x7f165407a6a0 msgr2=0x7f165407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:35.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.697+0000 7f1658c72700 1 -- 172.21.15.33:0/494141269 shutdown_connections 2024-05-15T13:59:35.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.697+0000 7f1658c72700 1 -- 172.21.15.33:0/494141269 wait complete. 2024-05-15T13:59:35.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.697+0000 7f1658c72700 1 Processor -- start 2024-05-15T13:59:35.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.697+0000 7f1658c72700 1 -- start start 2024-05-15T13:59:35.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.698+0000 7f1658c72700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16541037f0 0x7f165419be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:35.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.698+0000 7f1658c72700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16541041a0 0x7f165419c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:35.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.698+0000 7f1658c72700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16541088f0 0x7f16541a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:35.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.698+0000 7f1658c72700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1654113730 con 0x7f16541037f0 2024-05-15T13:59:35.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.698+0000 7f1658c72700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f16541135b0 con 0x7f16541041a0 2024-05-15T13:59:35.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.698+0000 7f1658c72700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f16541138b0 con 0x7f16541088f0 2024-05-15T13:59:35.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.698+0000 7f165259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16541037f0 0x7f165419be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:35.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.698+0000 7f165259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16541037f0 0x7f165419be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40994/0 (socket says 172.21.15.33:40994) 2024-05-15T13:59:35.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.698+0000 7f165259c700 1 -- 172.21.15.33:0/4235180683 learned_addr learned my addr 172.21.15.33:0/4235180683 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:35.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.698+0000 7f1651d9b700 1 --2- 172.21.15.33:0/4235180683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16541041a0 0x7f165419c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:35.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.698+0000 7f1652d9d700 1 --2- 172.21.15.33:0/4235180683 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16541088f0 0x7f16541a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:35.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.698+0000 7f165259c700 1 -- 172.21.15.33:0/4235180683 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16541088f0 msgr2=0x7f16541a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:35.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.698+0000 7f165259c700 1 --2- 172.21.15.33:0/4235180683 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16541088f0 0x7f16541a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:35.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.698+0000 7f165259c700 1 -- 172.21.15.33:0/4235180683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16541041a0 msgr2=0x7f165419c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:35.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.698+0000 7f165259c700 1 --2- 172.21.15.33:0/4235180683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16541041a0 0x7f165419c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:35.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.698+0000 7f165259c700 1 -- 172.21.15.33:0/4235180683 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16541a0e10 con 0x7f16541037f0 2024-05-15T13:59:35.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.698+0000 7f165259c700 1 --2- 172.21.15.33:0/4235180683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16541037f0 0x7f165419be00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f164400c970 tx=0x7f164400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:35.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.699+0000 7f16437fe700 1 -- 172.21.15.33:0/4235180683 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1644007aa0 con 0x7f16541037f0 2024-05-15T13:59:35.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.699+0000 7f1658c72700 1 -- 172.21.15.33:0/4235180683 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16541a1100 con 0x7f16541037f0 2024-05-15T13:59:35.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.699+0000 7f16437fe700 1 -- 172.21.15.33:0/4235180683 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1644004ce0 con 0x7f16541037f0 2024-05-15T13:59:35.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.699+0000 7f16437fe700 1 -- 172.21.15.33:0/4235180683 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1644005510 con 0x7f16541037f0 2024-05-15T13:59:35.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.699+0000 7f1658c72700 1 -- 172.21.15.33:0/4235180683 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16541a8980 con 0x7f16541037f0 2024-05-15T13:59:35.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.701+0000 7f16437fe700 1 -- 172.21.15.33:0/4235180683 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1644007c40 con 0x7f16541037f0 2024-05-15T13:59:35.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.702+0000 7f16437fe700 1 --2- 172.21.15.33:0/4235180683 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f163c0b2880 0x7f163c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:35.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.702+0000 7f1651d9b700 1 --2- 172.21.15.33:0/4235180683 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f163c0b2880 0x7f163c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:35.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.702+0000 7f16437fe700 1 -- 172.21.15.33:0/4235180683 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f16440e1ff0 con 0x7f16541037f0 2024-05-15T13:59:35.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.703+0000 7f1651d9b700 1 --2- 172.21.15.33:0/4235180683 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f163c0b2880 0x7f163c0b4d40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f16541034d0 tx=0x7f164800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:35.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.703+0000 7f1663508b80 1 Processor -- start 2024-05-15T13:59:35.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.703+0000 7f1663508b80 1 -- start start 2024-05-15T13:59:35.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.703+0000 7f1663508b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635ac4adfa0 0x5635ac4ae380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:35.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.703+0000 7f1663508b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635ac4ae8c0 0x5635ac4a87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:35.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.703+0000 7f1663508b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5635ac4a8dc0 0x5635ac4a9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:35.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.703+0000 7f1663508b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5635ac406d20 con 0x5635ac4ae8c0 2024-05-15T13:59:35.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.703+0000 7f1663508b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5635ac3f68c0 con 0x5635ac4adfa0 2024-05-15T13:59:35.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.703+0000 7f1663508b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5635ac3fa900 con 0x5635ac4a8dc0 2024-05-15T13:59:35.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.703+0000 7f1651d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635ac4ae8c0 0x5635ac4a87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:35.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.703+0000 7f1651d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635ac4ae8c0 0x5635ac4a87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41002/0 (socket says 172.21.15.33:41002) 2024-05-15T13:59:35.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.703+0000 7f1651d9b700 1 -- 172.21.15.33:0/25998920 learned_addr learned my addr 172.21.15.33:0/25998920 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:35.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.704+0000 7f1651d9b700 1 -- 172.21.15.33:0/25998920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5635ac4a8dc0 msgr2=0x5635ac4a9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T13:59:35.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.704+0000 7f1652d9d700 1 --2- 172.21.15.33:0/25998920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5635ac4a8dc0 0x5635ac4a9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:35.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.704+0000 7f165259c700 1 --2- 172.21.15.33:0/25998920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635ac4adfa0 0x5635ac4ae380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:35.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.704+0000 7f1651d9b700 1 --2- 172.21.15.33:0/25998920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5635ac4a8dc0 0x5635ac4a9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:35.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.704+0000 7f1651d9b700 1 -- 172.21.15.33:0/25998920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635ac4adfa0 msgr2=0x5635ac4ae380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T13:59:35.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.704+0000 7f1651d9b700 1 --2- 172.21.15.33:0/25998920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635ac4adfa0 0x5635ac4ae380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:35.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.704+0000 7f1651d9b700 1 -- 172.21.15.33:0/25998920 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635ac39ff60 con 0x5635ac4ae8c0 2024-05-15T13:59:35.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.704+0000 7f165259c700 1 --2- 172.21.15.33:0/25998920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635ac4adfa0 0x5635ac4ae380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T13:59:35.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.704+0000 7f1651d9b700 1 --2- 172.21.15.33:0/25998920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635ac4ae8c0 0x5635ac4a87a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1648010e30 tx=0x7f1648013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b21cf3084ce0fc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:35.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.704+0000 7f1640ff9700 1 -- 172.21.15.33:0/25998920 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164801e620 con 0x5635ac4ae8c0 2024-05-15T13:59:35.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.704+0000 7f1640ff9700 1 -- 172.21.15.33:0/25998920 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1648010e60 con 0x5635ac4ae8c0 2024-05-15T13:59:35.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.704+0000 7f1640ff9700 1 -- 172.21.15.33:0/25998920 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f164801cba0 con 0x5635ac4ae8c0 2024-05-15T13:59:35.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.705+0000 7f1663508b80 1 -- 172.21.15.33:0/25998920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635ac4ae8c0 msgr2=0x5635ac4a87a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:35.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.705+0000 7f1663508b80 1 --2- 172.21.15.33:0/25998920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635ac4ae8c0 0x5635ac4a87a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1648010e30 tx=0x7f1648013000 comp rx=0 tx=0).stop 2024-05-15T13:59:35.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.705+0000 7f1663508b80 1 -- 172.21.15.33:0/25998920 shutdown_connections 2024-05-15T13:59:35.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.705+0000 7f1663508b80 1 --2- 172.21.15.33:0/25998920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5635ac4a8dc0 0x5635ac4a9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:35.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.705+0000 7f1663508b80 1 --2- 172.21.15.33:0/25998920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635ac4adfa0 0x5635ac4ae380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:35.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.705+0000 7f1663508b80 1 --2- 172.21.15.33:0/25998920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635ac4ae8c0 0x5635ac4a87a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:35.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.705+0000 7f1663508b80 1 -- 172.21.15.33:0/25998920 >> 172.21.15.33:0/25998920 conn(0x5635ac3d5d60 msgr2=0x5635ac4a2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:35.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.705+0000 7f1663508b80 1 -- 172.21.15.33:0/25998920 shutdown_connections 2024-05-15T13:59:35.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.705+0000 7f1663508b80 1 -- 172.21.15.33:0/25998920 wait complete. 2024-05-15T13:59:35.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.705+0000 7f1663508b80 1 -- start start 2024-05-15T13:59:35.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.705+0000 7f1663508b80 1 Processor -- start 2024-05-15T13:59:35.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.706+0000 7f1663508b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635ac4a8dc0 0x5635ac4b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:35.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.706+0000 7f1663508b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635ac4adfa0 0x5635ac4b1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:35.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.706+0000 7f1663508b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5635ac4ae8c0 0x5635ac4b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:35.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.706+0000 7f1663508b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5635ac2fcf50 con 0x5635ac4adfa0 2024-05-15T13:59:35.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.706+0000 7f1663508b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5635ac35ed50 con 0x5635ac4a8dc0 2024-05-15T13:59:35.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.706+0000 7f1663508b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5635ac4b52d0 con 0x5635ac4ae8c0 2024-05-15T13:59:35.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.706+0000 7f1663508b80 10 client.0 _ll_register_callbacks cb 0x5635ac391900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T13:59:35.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.706+0000 7f1651d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635ac4adfa0 0x5635ac4b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:35.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.706+0000 7f1651d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635ac4adfa0 0x5635ac4b1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41010/0 (socket says 172.21.15.33:41010) 2024-05-15T13:59:35.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.706+0000 7f1651d9b700 1 -- 172.21.15.33:0/1171014863 learned_addr learned my addr 172.21.15.33:0/1171014863 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:35.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.706+0000 7f1652d9d700 1 --2- 172.21.15.33:0/1171014863 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5635ac4ae8c0 0x5635ac4b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:35.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.706+0000 7f165259c700 1 --2- 172.21.15.33:0/1171014863 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635ac4a8dc0 0x5635ac4b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:35.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.706+0000 7f1651d9b700 1 -- 172.21.15.33:0/1171014863 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5635ac4ae8c0 msgr2=0x5635ac4b1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T13:59:35.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.706+0000 7f1651d9b700 1 --2- 172.21.15.33:0/1171014863 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5635ac4ae8c0 0x5635ac4b1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:35.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.706+0000 7f1651d9b700 1 -- 172.21.15.33:0/1171014863 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635ac4a8dc0 msgr2=0x5635ac4b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T13:59:35.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.706+0000 7f1651d9b700 1 --2- 172.21.15.33:0/1171014863 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635ac4a8dc0 0x5635ac4b45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:35.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.706+0000 7f1651d9b700 1 -- 172.21.15.33:0/1171014863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635ac346880 con 0x5635ac4adfa0 2024-05-15T13:59:35.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.706+0000 7f165259c700 1 --2- 172.21.15.33:0/1171014863 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635ac4a8dc0 0x5635ac4b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T13:59:35.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.706+0000 7f1651d9b700 1 --2- 172.21.15.33:0/1171014863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635ac4adfa0 0x5635ac4b1970 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f164801c6e0 tx=0x7f164801c5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668b015545de92ff server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:35.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.706+0000 7f1639ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T13:59:35.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.706+0000 7f1639ffb700 1 -- 172.21.15.33:0/1171014863 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1648036060 con 0x5635ac4adfa0 2024-05-15T13:59:35.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.707+0000 7f1639ffb700 1 -- 172.21.15.33:0/1171014863 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f164801cd60 con 0x5635ac4adfa0 2024-05-15T13:59:35.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.707+0000 7f1663508b80 1 -- 172.21.15.33:0/1171014863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1648010e60 con 0x5635ac4adfa0 2024-05-15T13:59:35.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.707+0000 7f1639ffb700 1 -- 172.21.15.33:0/1171014863 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f1648031470 con 0x5635ac4adfa0 2024-05-15T13:59:35.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.707+0000 7f1663508b80 10 client.15000 fetch_fsmap learned FSMap version 23 2024-05-15T13:59:35.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.707+0000 7f1663508b80 1 -- 172.21.15.33:0/1171014863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5635ac3b0710 con 0x5635ac4adfa0 2024-05-15T13:59:35.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.707+0000 7f1639ffb700 1 -- 172.21.15.33:0/1171014863 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1648010e60 con 0x5635ac4adfa0 2024-05-15T13:59:35.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.707+0000 7f1639ffb700 1 -- 172.21.15.33:0/1171014863 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f164803b070 con 0x5635ac4adfa0 2024-05-15T13:59:35.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.707+0000 7f1663508b80 10 client.15000 fetch_fsmap finished waiting for FSMap version 23 2024-05-15T13:59:35.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.707+0000 7f1663508b80 10 client.15000 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-15T13:59:35.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.707+0000 7f1663508b80 20 client.15000 populate_metadata read hostname 'smithi033' 2024-05-15T13:59:35.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.707+0000 7f1663508b80 1 --2- 172.21.15.33:0/1171014863 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5635ac4c19b0 0x5635ac4c2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:35.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.707+0000 7f1663508b80 4 client.15000 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T13:59:35.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.707+0000 7f1663508b80 1 -- 172.21.15.33:0/1171014863 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635ac408c70 con 0x5635ac4c19b0 2024-05-15T13:59:35.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.707+0000 7f165259c700 1 --2- 172.21.15.33:0/1171014863 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5635ac4c19b0 0x5635ac4c2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:35.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.708+0000 7f1639ffb700 10 client.15000 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T13:59:35.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.708+0000 7f165259c700 1 --2- 172.21.15.33:0/1171014863 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5635ac4c19b0 0x5635ac4c2e20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acb445ecabe6fa34 server_cookie=e51f2b94ae535395 in_seq=0 out_seq=0 2024-05-15T13:59:35.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.709+0000 7f1639ffb700 1 -- 172.21.15.33:0/1171014863 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5635ac408c70 con 0x5635ac4c19b0 2024-05-15T13:59:35.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.709+0000 7f1639ffb700 10 client.15000 handle_command_reply: tid=0 2024-05-15T13:59:35.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.709+0000 7f1663508b80 1 client.15000 shutdown 2024-05-15T13:59:35.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.709+0000 7f1663508b80 1 -- 172.21.15.33:0/1171014863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635ac4adfa0 msgr2=0x5635ac4b1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:35.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.709+0000 7f1663508b80 1 --2- 172.21.15.33:0/1171014863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635ac4adfa0 0x5635ac4b1970 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f164801c6e0 tx=0x7f164801c5e0 comp rx=0 tx=0).stop 2024-05-15T13:59:35.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.709+0000 7f1663508b80 1 -- 172.21.15.33:0/1171014863 shutdown_connections 2024-05-15T13:59:35.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.709+0000 7f1663508b80 1 --2- 172.21.15.33:0/1171014863 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5635ac4ae8c0 0x5635ac4b1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:35.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.709+0000 7f1663508b80 1 --2- 172.21.15.33:0/1171014863 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635ac4a8dc0 0x5635ac4b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:35.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.709+0000 7f1663508b80 1 --2- 172.21.15.33:0/1171014863 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5635ac4c19b0 0x5635ac4c2e20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:35.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.709+0000 7f1663508b80 1 --2- 172.21.15.33:0/1171014863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635ac4adfa0 0x5635ac4b1970 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:35.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.709+0000 7f1663508b80 1 -- 172.21.15.33:0/1171014863 >> 172.21.15.33:0/1171014863 conn(0x5635ac3d5d60 msgr2=0x5635ac4a2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:35.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.709+0000 7f1639ffb700 0 client.15000 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T13:59:35.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.709+0000 7f1663508b80 1 -- 172.21.15.33:0/1171014863 shutdown_connections 2024-05-15T13:59:35.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.709+0000 7f1663508b80 1 -- 172.21.15.33:0/1171014863 wait complete. 2024-05-15T13:59:35.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.709+0000 7f1663508b80 20 client.15000 trim_cache size 0 max 16384 2024-05-15T13:59:35.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.730+0000 7f1663508b80 1 Processor -- start 2024-05-15T13:59:35.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.730+0000 7f1663508b80 1 -- start start 2024-05-15T13:59:35.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.730+0000 7f1663508b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635ac4a8dc0 0x5635ac4b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:35.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.730+0000 7f1663508b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635ac4adfa0 0x5635ac4b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:35.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.730+0000 7f1663508b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5635ac4ae8c0 0x5635ac4a2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:35.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.730+0000 7f1663508b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5635ac4c1160 con 0x5635ac4adfa0 2024-05-15T13:59:35.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.730+0000 7f1663508b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5635ac4c1830 con 0x5635ac4a8dc0 2024-05-15T13:59:35.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.730+0000 7f1663508b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5635ac4c1f00 con 0x5635ac4ae8c0 2024-05-15T13:59:35.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.730+0000 7f1651d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635ac4adfa0 0x5635ac4b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:35.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.730+0000 7f1651d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635ac4adfa0 0x5635ac4b1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41012/0 (socket says 172.21.15.33:41012) 2024-05-15T13:59:35.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.730+0000 7f1651d9b700 1 -- 172.21.15.33:0/588009339 learned_addr learned my addr 172.21.15.33:0/588009339 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:35.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.731+0000 7f165259c700 1 --2- 172.21.15.33:0/588009339 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635ac4a8dc0 0x5635ac4b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:35.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.731+0000 7f1651d9b700 1 -- 172.21.15.33:0/588009339 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5635ac4ae8c0 msgr2=0x5635ac4a2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T13:59:35.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.731+0000 7f1651d9b700 1 --2- 172.21.15.33:0/588009339 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5635ac4ae8c0 0x5635ac4a2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:35.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.731+0000 7f1651d9b700 1 -- 172.21.15.33:0/588009339 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635ac4a8dc0 msgr2=0x5635ac4b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T13:59:35.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.731+0000 7f1651d9b700 1 --2- 172.21.15.33:0/588009339 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635ac4a8dc0 0x5635ac4b0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:35.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.731+0000 7f1651d9b700 1 -- 172.21.15.33:0/588009339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635ac4a4f30 con 0x5635ac4adfa0 2024-05-15T13:59:35.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.731+0000 7f1651d9b700 1 --2- 172.21.15.33:0/588009339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635ac4adfa0 0x5635ac4b1e60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1648005850 tx=0x7f164801ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3e756940266fd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:35.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.731+0000 7f163bfff700 1 -- 172.21.15.33:0/588009339 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1648041070 con 0x5635ac4adfa0 2024-05-15T13:59:35.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.731+0000 7f163bfff700 1 -- 172.21.15.33:0/588009339 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f164801e420 con 0x5635ac4adfa0 2024-05-15T13:59:35.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.731+0000 7f163bfff700 1 -- 172.21.15.33:0/588009339 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1648036040 con 0x5635ac4adfa0 2024-05-15T13:59:35.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.732+0000 7f1663508b80 1 -- 172.21.15.33:0/588009339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635ac4adfa0 msgr2=0x5635ac4b1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:35.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.732+0000 7f1663508b80 1 --2- 172.21.15.33:0/588009339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635ac4adfa0 0x5635ac4b1e60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1648005850 tx=0x7f164801ca20 comp rx=0 tx=0).stop 2024-05-15T13:59:35.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.732+0000 7f1663508b80 1 -- 172.21.15.33:0/588009339 shutdown_connections 2024-05-15T13:59:35.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.732+0000 7f1663508b80 1 --2- 172.21.15.33:0/588009339 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5635ac4ae8c0 0x5635ac4a2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:35.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.732+0000 7f1663508b80 1 --2- 172.21.15.33:0/588009339 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635ac4a8dc0 0x5635ac4b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:35.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.732+0000 7f1663508b80 1 --2- 172.21.15.33:0/588009339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635ac4adfa0 0x5635ac4b1e60 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:35.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.732+0000 7f1663508b80 1 -- 172.21.15.33:0/588009339 >> 172.21.15.33:0/588009339 conn(0x5635ac3d5d60 msgr2=0x5635ac4b1750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:35.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.732+0000 7f1663508b80 1 -- 172.21.15.33:0/588009339 shutdown_connections 2024-05-15T13:59:35.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.732+0000 7f1663508b80 1 -- 172.21.15.33:0/588009339 wait complete. 2024-05-15T13:59:35.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.732+0000 7f1663508b80 1 -- start start 2024-05-15T13:59:35.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.732+0000 7f1663508b80 1 Processor -- start 2024-05-15T13:59:35.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.732+0000 7f1663508b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635ac4a8dc0 0x5635ac4cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:35.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.732+0000 7f1663508b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5635ac4adfa0 0x5635ac4cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:35.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.732+0000 7f1663508b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635ac4ae8c0 0x5635ac4c7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:35.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.732+0000 7f1663508b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5635ac4c2ca0 con 0x5635ac4ae8c0 2024-05-15T13:59:35.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.732+0000 7f1663508b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5635ac4c25d0 con 0x5635ac4a8dc0 2024-05-15T13:59:35.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.732+0000 7f1663508b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5635ac4c3370 con 0x5635ac4adfa0 2024-05-15T13:59:35.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.733+0000 7f1663508b80 10 client.0 _ll_register_callbacks cb 0x5635ac391900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T13:59:35.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.733+0000 7f1652d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635ac4ae8c0 0x5635ac4c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:35.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.733+0000 7f1651d9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5635ac4adfa0 0x5635ac4cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:35.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.733+0000 7f1652d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635ac4ae8c0 0x5635ac4c7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41028/0 (socket says 172.21.15.33:41028) 2024-05-15T13:59:35.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.733+0000 7f1652d9d700 1 -- 172.21.15.33:0/3253763264 learned_addr learned my addr 172.21.15.33:0/3253763264 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:35.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.733+0000 7f165259c700 1 --2- 172.21.15.33:0/3253763264 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635ac4a8dc0 0x5635ac4cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:35.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.733+0000 7f1651d9b700 1 -- 172.21.15.33:0/3253763264 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635ac4a8dc0 msgr2=0x5635ac4cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T13:59:35.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.733+0000 7f1651d9b700 1 --2- 172.21.15.33:0/3253763264 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635ac4a8dc0 0x5635ac4cc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:35.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.733+0000 7f1651d9b700 1 -- 172.21.15.33:0/3253763264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635ac4ae8c0 msgr2=0x5635ac4c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T13:59:35.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.733+0000 7f1651d9b700 1 --2- 172.21.15.33:0/3253763264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635ac4ae8c0 0x5635ac4c7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:35.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.733+0000 7f1651d9b700 1 -- 172.21.15.33:0/3253763264 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635ac4c7a90 con 0x5635ac4adfa0 2024-05-15T13:59:35.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.733+0000 7f1652d9d700 1 --2- 172.21.15.33:0/3253763264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635ac4ae8c0 0x5635ac4c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T13:59:35.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.733+0000 7f1651d9b700 1 --2- 172.21.15.33:0/3253763264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5635ac4adfa0 0x5635ac4cc7e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1648010ab0 tx=0x7f164801ef30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e352973279424769 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:35.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.733+0000 7f1639ffb700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-05-15T13:59:35.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.734+0000 7f1639ffb700 1 -- 172.21.15.33:0/3253763264 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1648041030 con 0x5635ac4adfa0 2024-05-15T13:59:35.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.734+0000 7f1663508b80 1 -- 172.21.15.33:0/3253763264 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f164801e420 con 0x5635ac4adfa0 2024-05-15T13:59:35.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.734+0000 7f1639ffb700 1 -- 172.21.15.33:0/3253763264 <== mon.1 v2:172.21.15.81:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f164802d050 con 0x5635ac4adfa0 2024-05-15T13:59:35.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.734+0000 7f1639ffb700 1 -- 172.21.15.33:0/3253763264 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f1648036040 con 0x5635ac4adfa0 2024-05-15T13:59:35.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.734+0000 7f1663508b80 10 client.24763 fetch_fsmap learned FSMap version 23 2024-05-15T13:59:35.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.734+0000 7f1663508b80 1 -- 172.21.15.33:0/3253763264 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5635ac52d900 con 0x5635ac4adfa0 2024-05-15T13:59:35.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.734+0000 7f1639ffb700 1 -- 172.21.15.33:0/3253763264 <== mon.1 v2:172.21.15.81:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f164801e420 con 0x5635ac4adfa0 2024-05-15T13:59:35.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.734+0000 7f1639ffb700 1 -- 172.21.15.33:0/3253763264 <== mon.1 v2:172.21.15.81:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f164803b040 con 0x5635ac4adfa0 2024-05-15T13:59:35.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.734+0000 7f1663508b80 10 client.24763 fetch_fsmap finished waiting for FSMap version 23 2024-05-15T13:59:35.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.734+0000 7f1663508b80 10 client.24763 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-15T13:59:35.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.734+0000 7f1663508b80 20 client.24763 populate_metadata read hostname 'smithi033' 2024-05-15T13:59:35.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.734+0000 7f1663508b80 1 --2- 172.21.15.33:0/3253763264 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5635ac5321f0 0x5635ac530f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:35.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.734+0000 7f1663508b80 4 client.24763 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-15T13:59:35.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.734+0000 7f1663508b80 1 -- 172.21.15.33:0/3253763264 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "status"}) -- 0x5635ac402450 con 0x5635ac5321f0 2024-05-15T13:59:35.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.735+0000 7f165259c700 1 --2- 172.21.15.33:0/3253763264 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5635ac5321f0 0x5635ac530f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:35.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.735+0000 7f1639ffb700 10 client.24763 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T13:59:35.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.735+0000 7f165259c700 1 --2- 172.21.15.33:0/3253763264 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5635ac5321f0 0x5635ac530f00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7d91fcee00d3aac server_cookie=c0ed962f588accf8 in_seq=0 out_seq=0 2024-05-15T13:59:35.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.735+0000 7f1639ffb700 1 -- 172.21.15.33:0/3253763264 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5635ac402450 con 0x5635ac5321f0 2024-05-15T13:59:35.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.735+0000 7f1639ffb700 10 client.24763 handle_command_reply: tid=0 2024-05-15T13:59:35.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.735+0000 7f1663508b80 1 client.24763 shutdown 2024-05-15T13:59:35.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.735+0000 7f1663508b80 1 -- 172.21.15.33:0/3253763264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5635ac4adfa0 msgr2=0x5635ac4cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:35.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.735+0000 7f1663508b80 1 --2- 172.21.15.33:0/3253763264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5635ac4adfa0 0x5635ac4cc7e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1648010ab0 tx=0x7f164801ef30 comp rx=0 tx=0).stop 2024-05-15T13:59:35.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.735+0000 7f1663508b80 1 -- 172.21.15.33:0/3253763264 shutdown_connections 2024-05-15T13:59:35.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.735+0000 7f1663508b80 1 --2- 172.21.15.33:0/3253763264 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5635ac5321f0 0x5635ac530f00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:35.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.735+0000 7f1663508b80 1 --2- 172.21.15.33:0/3253763264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635ac4ae8c0 0x5635ac4c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:35.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.735+0000 7f1663508b80 1 --2- 172.21.15.33:0/3253763264 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5635ac4a8dc0 0x5635ac4cc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:35.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.735+0000 7f1639ffb700 0 client.24763 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T13:59:35.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.735+0000 7f1663508b80 1 --2- 172.21.15.33:0/3253763264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5635ac4adfa0 0x5635ac4cc7e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:35.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.735+0000 7f1663508b80 1 -- 172.21.15.33:0/3253763264 >> 172.21.15.33:0/3253763264 conn(0x5635ac3d5d60 msgr2=0x5635ac4aeca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:35.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.736+0000 7f1663508b80 1 -- 172.21.15.33:0/3253763264 shutdown_connections 2024-05-15T13:59:35.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.736+0000 7f1663508b80 1 -- 172.21.15.33:0/3253763264 wait complete. 2024-05-15T13:59:35.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.736+0000 7f1663508b80 20 client.24763 trim_cache size 0 max 16384 2024-05-15T13:59:35.742 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T13:59:35.742 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "bca78ca4-12c1-11ef-bc98-c7b262605968", 2024-05-15T13:59:35.742 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2024-05-15T13:59:35.743 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24478, 2024-05-15T13:59:35.743 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-05-15T13:59:35.743 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-05-15T13:59:35.743 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-05-15T13:59:35.743 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 136.35763236099999, 2024-05-15T13:59:35.743 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 23, 2024-05-15T13:59:35.743 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 82, 2024-05-15T13:59:35.743 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 80, 2024-05-15T13:59:35.743 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 162.07272535999999 2024-05-15T13:59:35.743 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T13:59:35.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.736+0000 7f1658c72700 1 -- 172.21.15.33:0/4235180683 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f163c0b2880 msgr2=0x7f163c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:35.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.736+0000 7f1658c72700 1 --2- 172.21.15.33:0/4235180683 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f163c0b2880 0x7f163c0b4d40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f16541034d0 tx=0x7f164800b410 comp rx=0 tx=0).stop 2024-05-15T13:59:35.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.736+0000 7f1658c72700 1 -- 172.21.15.33:0/4235180683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16541037f0 msgr2=0x7f165419be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:35.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.736+0000 7f1658c72700 1 --2- 172.21.15.33:0/4235180683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16541037f0 0x7f165419be00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f164400c970 tx=0x7f164400cc80 comp rx=0 tx=0).stop 2024-05-15T13:59:35.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.736+0000 7f1658c72700 1 -- 172.21.15.33:0/4235180683 shutdown_connections 2024-05-15T13:59:35.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.736+0000 7f1658c72700 1 --2- 172.21.15.33:0/4235180683 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f163c0b2880 0x7f163c0b4d40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:35.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.736+0000 7f1658c72700 1 --2- 172.21.15.33:0/4235180683 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16541088f0 0x7f16541a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:35.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.736+0000 7f1658c72700 1 --2- 172.21.15.33:0/4235180683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16541037f0 0x7f165419be00 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:35.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.736+0000 7f1658c72700 1 --2- 172.21.15.33:0/4235180683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16541041a0 0x7f165419c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:35.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.736+0000 7f1658c72700 1 -- 172.21.15.33:0/4235180683 >> 172.21.15.33:0/4235180683 conn(0x7f165407a6a0 msgr2=0x7f1654101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:35.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.736+0000 7f1658c72700 1 -- 172.21.15.33:0/4235180683 shutdown_connections 2024-05-15T13:59:35.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:35.736+0000 7f1658c72700 1 -- 172.21.15.33:0/4235180683 wait complete. 2024-05-15T13:59:36.333 INFO:tasks.cephadm:Setup complete, yielding 2024-05-15T13:59:36.334 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-15T13:59:36.346 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-05-15T13:59:36.347 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- bash -c 'ceph orch status' 2024-05-15T13:59:36.366 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:36 smithi033 ceph-mon[31650]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:59:36.598 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:59:36.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:36 smithi081 ceph-mon[36434]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:59:36.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:36 smithi052 ceph-mon[36190]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:59:38.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.294+0000 7f0be5b0d700 1 -- 172.21.15.33:0/2890310133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0be01047a0 msgr2=0x7f0be0104c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:38.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.294+0000 7f0be5b0d700 1 --2- 172.21.15.33:0/2890310133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0be01047a0 0x7f0be0104c00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0bc80099c0 tx=0x7f0bc8009cd0 comp rx=0 tx=0).stop 2024-05-15T13:59:38.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.294+0000 7f0be5b0d700 1 -- 172.21.15.33:0/2890310133 shutdown_connections 2024-05-15T13:59:38.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.294+0000 7f0be5b0d700 1 --2- 172.21.15.33:0/2890310133 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0be0105140 0x7f0be010b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:38.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.294+0000 7f0be5b0d700 1 --2- 172.21.15.33:0/2890310133 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0be01035a0 0x7f0be01039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:38.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.294+0000 7f0be5b0d700 1 --2- 172.21.15.33:0/2890310133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0be01047a0 0x7f0be0104c00 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:38.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.294+0000 7f0be5b0d700 1 -- 172.21.15.33:0/2890310133 >> 172.21.15.33:0/2890310133 conn(0x7f0be00feb40 msgr2=0x7f0be0100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:38.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.294+0000 7f0be5b0d700 1 -- 172.21.15.33:0/2890310133 shutdown_connections 2024-05-15T13:59:38.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.294+0000 7f0be5b0d700 1 -- 172.21.15.33:0/2890310133 wait complete. 2024-05-15T13:59:38.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.295+0000 7f0be5b0d700 1 Processor -- start 2024-05-15T13:59:38.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.295+0000 7f0be5b0d700 1 -- start start 2024-05-15T13:59:38.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.295+0000 7f0be5b0d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0be01035a0 0x7f0be00772e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:38.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.295+0000 7f0be5b0d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0be01047a0 0x7f0be0075930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:38.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.295+0000 7f0be5b0d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0be0105140 0x7f0be0075e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:38.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.295+0000 7f0be5b0d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0be010f010 con 0x7f0be01035a0 2024-05-15T13:59:38.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.295+0000 7f0be5b0d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0be010ee90 con 0x7f0be0105140 2024-05-15T13:59:38.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.295+0000 7f0be5b0d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0be010f190 con 0x7f0be01047a0 2024-05-15T13:59:38.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.295+0000 7f0bdf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0be01035a0 0x7f0be00772e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:38.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.295+0000 7f0bdf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0be01035a0 0x7f0be00772e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41058/0 (socket says 172.21.15.33:41058) 2024-05-15T13:59:38.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.295+0000 7f0bdf7fe700 1 -- 172.21.15.33:0/781830245 learned_addr learned my addr 172.21.15.33:0/781830245 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:38.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.295+0000 7f0bdeffd700 1 --2- 172.21.15.33:0/781830245 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0be01047a0 0x7f0be0075930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:38.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.295+0000 7f0bdffff700 1 --2- 172.21.15.33:0/781830245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0be0105140 0x7f0be0075e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:38.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.296+0000 7f0bdf7fe700 1 -- 172.21.15.33:0/781830245 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0be01047a0 msgr2=0x7f0be0075930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:38.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.296+0000 7f0bdf7fe700 1 --2- 172.21.15.33:0/781830245 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0be01047a0 0x7f0be0075930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:38.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.296+0000 7f0bdf7fe700 1 -- 172.21.15.33:0/781830245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0be0105140 msgr2=0x7f0be0075e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:38.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.296+0000 7f0bdf7fe700 1 --2- 172.21.15.33:0/781830245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0be0105140 0x7f0be0075e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:38.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.296+0000 7f0bdf7fe700 1 -- 172.21.15.33:0/781830245 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be0076720 con 0x7f0be01035a0 2024-05-15T13:59:38.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.296+0000 7f0bdf7fe700 1 --2- 172.21.15.33:0/781830245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0be01035a0 0x7f0be00772e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000d980 tx=0x7f0bd000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:38.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.296+0000 7f0bdcff9700 1 -- 172.21.15.33:0/781830245 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0017030 con 0x7f0be01035a0 2024-05-15T13:59:38.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.296+0000 7f0be5b0d700 1 -- 172.21.15.33:0/781830245 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be01a37c0 con 0x7f0be01035a0 2024-05-15T13:59:38.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.296+0000 7f0be5b0d700 1 -- 172.21.15.33:0/781830245 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be01a3d00 con 0x7f0be01035a0 2024-05-15T13:59:38.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.296+0000 7f0bdcff9700 1 -- 172.21.15.33:0/781830245 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bd00044d0 con 0x7f0be01035a0 2024-05-15T13:59:38.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.296+0000 7f0bdcff9700 1 -- 172.21.15.33:0/781830245 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd00039e0 con 0x7f0be01035a0 2024-05-15T13:59:38.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.297+0000 7f0be5b0d700 1 -- 172.21.15.33:0/781830245 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bc0005360 con 0x7f0be01035a0 2024-05-15T13:59:38.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.298+0000 7f0bdcff9700 1 -- 172.21.15.33:0/781830245 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0bd001a070 con 0x7f0be01035a0 2024-05-15T13:59:38.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.299+0000 7f0bdcff9700 1 --2- 172.21.15.33:0/781830245 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f0bcc0b2900 0x7f0bcc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:38.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.299+0000 7f0bdeffd700 1 --2- 172.21.15.33:0/781830245 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f0bcc0b2900 0x7f0bcc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:38.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.299+0000 7f0bdcff9700 1 -- 172.21.15.33:0/781830245 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f0bd00e1d80 con 0x7f0be01035a0 2024-05-15T13:59:38.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.300+0000 7f0bdeffd700 1 --2- 172.21.15.33:0/781830245 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f0bcc0b2900 0x7f0bcc0b4dc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8009990 tx=0x7f0bc800b870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:38.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.307+0000 7f0bdcff9700 1 -- 172.21.15.33:0/781830245 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0bd00a9b20 con 0x7f0be01035a0 2024-05-15T13:59:38.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:38 smithi033 ceph-mon[31650]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-15T13:59:38.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.562+0000 7f0be5b0d700 1 -- 172.21.15.33:0/781830245 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f0bc0000bf0 con 0x7f0bcc0b2900 2024-05-15T13:59:38.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.563+0000 7f0bdcff9700 1 -- 172.21.15.33:0/781830245 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f0bc0000bf0 con 0x7f0bcc0b2900 2024-05-15T13:59:38.566 INFO:teuthology.orchestra.run.smithi033.stdout:Backend: cephadm 2024-05-15T13:59:38.566 INFO:teuthology.orchestra.run.smithi033.stdout:Available: Yes 2024-05-15T13:59:38.566 INFO:teuthology.orchestra.run.smithi033.stdout:Paused: No 2024-05-15T13:59:38.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.565+0000 7f0be5b0d700 1 -- 172.21.15.33:0/781830245 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f0bcc0b2900 msgr2=0x7f0bcc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:38.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.565+0000 7f0be5b0d700 1 --2- 172.21.15.33:0/781830245 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f0bcc0b2900 0x7f0bcc0b4dc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8009990 tx=0x7f0bc800b870 comp rx=0 tx=0).stop 2024-05-15T13:59:38.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.565+0000 7f0be5b0d700 1 -- 172.21.15.33:0/781830245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0be01035a0 msgr2=0x7f0be00772e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:38.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.565+0000 7f0be5b0d700 1 --2- 172.21.15.33:0/781830245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0be01035a0 0x7f0be00772e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000d980 tx=0x7f0bd000dc90 comp rx=0 tx=0).stop 2024-05-15T13:59:38.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.565+0000 7f0be5b0d700 1 -- 172.21.15.33:0/781830245 shutdown_connections 2024-05-15T13:59:38.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.565+0000 7f0be5b0d700 1 --2- 172.21.15.33:0/781830245 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f0bcc0b2900 0x7f0bcc0b4dc0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:38.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.565+0000 7f0be5b0d700 1 --2- 172.21.15.33:0/781830245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0be0105140 0x7f0be0075e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:38.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.565+0000 7f0be5b0d700 1 --2- 172.21.15.33:0/781830245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0be01035a0 0x7f0be00772e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:38.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.565+0000 7f0be5b0d700 1 --2- 172.21.15.33:0/781830245 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0be01047a0 0x7f0be0075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:38.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.565+0000 7f0be5b0d700 1 -- 172.21.15.33:0/781830245 >> 172.21.15.33:0/781830245 conn(0x7f0be00feb40 msgr2=0x7f0be0107ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:38.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.565+0000 7f0be5b0d700 1 -- 172.21.15.33:0/781830245 shutdown_connections 2024-05-15T13:59:38.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:38.565+0000 7f0be5b0d700 1 -- 172.21.15.33:0/781830245 wait complete. 2024-05-15T13:59:38.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:38 smithi081 ceph-mon[36434]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-15T13:59:38.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:38 smithi052 ceph-mon[36190]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-15T13:59:39.331 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-15T13:59:39.624 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:59:40.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:40 smithi081 ceph-mon[36434]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:59:40.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:40 smithi081 ceph-mon[36434]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:59:40.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:40 smithi033 ceph-mon[31650]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:59:40.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:40 smithi033 ceph-mon[31650]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:59:40.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:40 smithi052 ceph-mon[36190]: from='client.15012 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:59:40.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:40 smithi052 ceph-mon[36190]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:59:41.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.300+0000 7f52146a4700 1 -- 172.21.15.33:0/2638499384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f520c104790 msgr2=0x7f520c104bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:41.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.300+0000 7f52146a4700 1 --2- 172.21.15.33:0/2638499384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f520c104790 0x7f520c104bf0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f51fc0099c0 tx=0x7f51fc009cd0 comp rx=0 tx=0).stop 2024-05-15T13:59:41.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.300+0000 7f52146a4700 1 -- 172.21.15.33:0/2638499384 shutdown_connections 2024-05-15T13:59:41.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.300+0000 7f52146a4700 1 --2- 172.21.15.33:0/2638499384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f520c105130 0x7f520c10b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:41.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.300+0000 7f52146a4700 1 --2- 172.21.15.33:0/2638499384 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f520c103590 0x7f520c1039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:41.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.300+0000 7f52146a4700 1 --2- 172.21.15.33:0/2638499384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f520c104790 0x7f520c104bf0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:41.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.300+0000 7f52146a4700 1 -- 172.21.15.33:0/2638499384 >> 172.21.15.33:0/2638499384 conn(0x7f520c0feb30 msgr2=0x7f520c100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:41.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.300+0000 7f52146a4700 1 -- 172.21.15.33:0/2638499384 shutdown_connections 2024-05-15T13:59:41.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.300+0000 7f52146a4700 1 -- 172.21.15.33:0/2638499384 wait complete. 2024-05-15T13:59:41.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.301+0000 7f52146a4700 1 Processor -- start 2024-05-15T13:59:41.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.301+0000 7f52146a4700 1 -- start start 2024-05-15T13:59:41.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.301+0000 7f52146a4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f520c103590 0x7f520c0772e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:41.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.301+0000 7f52146a4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f520c104790 0x7f520c075930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:41.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.301+0000 7f52146a4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f520c105130 0x7f520c075e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:41.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.301+0000 7f52146a4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f520c10f020 con 0x7f520c103590 2024-05-15T13:59:41.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.301+0000 7f52146a4700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f520c10eea0 con 0x7f520c104790 2024-05-15T13:59:41.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.301+0000 7f52146a4700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f520c10f1a0 con 0x7f520c105130 2024-05-15T13:59:41.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.301+0000 7f5212440700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f520c103590 0x7f520c0772e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:41.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.301+0000 7f5212440700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f520c103590 0x7f520c0772e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41078/0 (socket says 172.21.15.33:41078) 2024-05-15T13:59:41.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.301+0000 7f5212440700 1 -- 172.21.15.33:0/2424699162 learned_addr learned my addr 172.21.15.33:0/2424699162 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:41.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.301+0000 7f5211c3f700 1 --2- 172.21.15.33:0/2424699162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f520c104790 0x7f520c075930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:41.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.301+0000 7f5212c41700 1 --2- 172.21.15.33:0/2424699162 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f520c105130 0x7f520c075e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:41.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.301+0000 7f5212440700 1 -- 172.21.15.33:0/2424699162 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f520c105130 msgr2=0x7f520c075e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:41.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.301+0000 7f5212440700 1 --2- 172.21.15.33:0/2424699162 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f520c105130 0x7f520c075e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:41.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.301+0000 7f5212440700 1 -- 172.21.15.33:0/2424699162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f520c104790 msgr2=0x7f520c075930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:41.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.301+0000 7f5212440700 1 --2- 172.21.15.33:0/2424699162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f520c104790 0x7f520c075930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:41.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.301+0000 7f5212440700 1 -- 172.21.15.33:0/2424699162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520c076720 con 0x7f520c103590 2024-05-15T13:59:41.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.302+0000 7f5212c41700 1 --2- 172.21.15.33:0/2424699162 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f520c105130 0x7f520c075e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T13:59:41.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.302+0000 7f5212440700 1 --2- 172.21.15.33:0/2424699162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f520c103590 0x7f520c0772e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f520800d980 tx=0x7f520800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:41.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.302+0000 7f52037fe700 1 -- 172.21.15.33:0/2424699162 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5208017070 con 0x7f520c103590 2024-05-15T13:59:41.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.302+0000 7f52037fe700 1 -- 172.21.15.33:0/2424699162 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5208009de0 con 0x7f520c103590 2024-05-15T13:59:41.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.302+0000 7f52037fe700 1 -- 172.21.15.33:0/2424699162 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520800f3c0 con 0x7f520c103590 2024-05-15T13:59:41.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.302+0000 7f52146a4700 1 -- 172.21.15.33:0/2424699162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f520c1a3830 con 0x7f520c103590 2024-05-15T13:59:41.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.302+0000 7f52146a4700 1 -- 172.21.15.33:0/2424699162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f520c1a3d40 con 0x7f520c103590 2024-05-15T13:59:41.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.303+0000 7f52146a4700 1 -- 172.21.15.33:0/2424699162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f520c06a230 con 0x7f520c103590 2024-05-15T13:59:41.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.304+0000 7f52037fe700 1 -- 172.21.15.33:0/2424699162 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5208009930 con 0x7f520c103590 2024-05-15T13:59:41.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.305+0000 7f52037fe700 1 --2- 172.21.15.33:0/2424699162 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f51f80b2900 0x7f51f80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:41.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.305+0000 7f52037fe700 1 -- 172.21.15.33:0/2424699162 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f52080e1ce0 con 0x7f520c103590 2024-05-15T13:59:41.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.305+0000 7f5211c3f700 1 --2- 172.21.15.33:0/2424699162 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f51f80b2900 0x7f51f80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:41.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.306+0000 7f5211c3f700 1 --2- 172.21.15.33:0/2424699162 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f51f80b2900 0x7f51f80b4dc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f51fc009990 tx=0x7f51fc0096b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:41.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.309+0000 7f52037fe700 1 -- 172.21.15.33:0/2424699162 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f52080a9a80 con 0x7f520c103590 2024-05-15T13:59:41.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.535+0000 7f52146a4700 1 -- 172.21.15.33:0/2424699162 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f520c063980 con 0x7f51f80b2900 2024-05-15T13:59:41.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.549+0000 7f52037fe700 1 -- 172.21.15.33:0/2424699162 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f520c063980 con 0x7f51f80b2900 2024-05-15T13:59:41.551 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-15T13:59:41.551 INFO:teuthology.orchestra.run.smithi033.stdout:mds.a smithi033 running (2m) 20s ago 2m 14.7M - 19.0.0-3687-g9b638668 c922e207469e 4e9df59fc51d 2024-05-15T13:59:41.551 INFO:teuthology.orchestra.run.smithi033.stdout:mds.b smithi052 running (2m) 22s ago 2m 19.7M - 19.0.0-3687-g9b638668 c922e207469e da7651bf28af 2024-05-15T13:59:41.551 INFO:teuthology.orchestra.run.smithi033.stdout:mds.c smithi081 running (2m) 22s ago 2m 22.7M - 19.0.0-3687-g9b638668 c922e207469e b722e0a0be65 2024-05-15T13:59:41.552 INFO:teuthology.orchestra.run.smithi033.stdout:mds.d smithi033 running (3m) 20s ago 3m 17.3M - 19.0.0-3687-g9b638668 c922e207469e e29496fc34f2 2024-05-15T13:59:41.552 INFO:teuthology.orchestra.run.smithi033.stdout:mds.e smithi052 running (2m) 22s ago 2m 17.3M - 19.0.0-3687-g9b638668 c922e207469e ce9ab971fa68 2024-05-15T13:59:41.552 INFO:teuthology.orchestra.run.smithi033.stdout:mds.f smithi081 running (2m) 22s ago 2m 22.2M - 19.0.0-3687-g9b638668 c922e207469e 031e85e53545 2024-05-15T13:59:41.552 INFO:teuthology.orchestra.run.smithi033.stdout:mds.g smithi033 running (2m) 20s ago 2m 17.1M - 19.0.0-3687-g9b638668 c922e207469e a1133de4c956 2024-05-15T13:59:41.552 INFO:teuthology.orchestra.run.smithi033.stdout:mds.h smithi052 running (2m) 22s ago 2m 20.9M - 19.0.0-3687-g9b638668 c922e207469e ac645799e591 2024-05-15T13:59:41.552 INFO:teuthology.orchestra.run.smithi033.stdout:mds.i smithi081 running (2m) 22s ago 2m 29.6M - 19.0.0-3687-g9b638668 c922e207469e 63576537a4ee 2024-05-15T13:59:41.552 INFO:teuthology.orchestra.run.smithi033.stdout:mds.j smithi033 running (3m) 20s ago 3m 17.0M - 19.0.0-3687-g9b638668 c922e207469e 8e2339d6cdaa 2024-05-15T13:59:41.552 INFO:teuthology.orchestra.run.smithi033.stdout:mds.k smithi052 running (3m) 22s ago 3m 17.1M - 19.0.0-3687-g9b638668 c922e207469e 3504755753aa 2024-05-15T13:59:41.552 INFO:teuthology.orchestra.run.smithi033.stdout:mds.l smithi081 running (3m) 22s ago 3m 21.0M - 19.0.0-3687-g9b638668 c922e207469e e49dda6b9f58 2024-05-15T13:59:41.552 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.x smithi033 *:9283,8765 running (10m) 20s ago 10m 529M - 19.0.0-3687-g9b638668 c922e207469e cfcd834cab7f 2024-05-15T13:59:41.552 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.y smithi052 *:8443,8765 running (8m) 22s ago 8m 469M - 19.0.0-3687-g9b638668 c922e207469e c0b639a4f899 2024-05-15T13:59:41.552 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.z smithi081 *:8443,8765 running (8m) 22s ago 8m 466M - 19.0.0-3687-g9b638668 c922e207469e 9f81e1ae812c 2024-05-15T13:59:41.552 INFO:teuthology.orchestra.run.smithi033.stdout:mon.a smithi033 running (10m) 20s ago 10m 55.0M 2048M 19.0.0-3687-g9b638668 c922e207469e 40d78556a6be 2024-05-15T13:59:41.552 INFO:teuthology.orchestra.run.smithi033.stdout:mon.b smithi052 running (8m) 22s ago 8m 40.5M 2048M 19.0.0-3687-g9b638668 c922e207469e 9ec142c69ed6 2024-05-15T13:59:41.552 INFO:teuthology.orchestra.run.smithi033.stdout:mon.c smithi081 running (8m) 22s ago 8m 44.0M 2048M 19.0.0-3687-g9b638668 c922e207469e c2480a957329 2024-05-15T13:59:41.553 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (8m) 20s ago 8m 59.8M 4273M 19.0.0-3687-g9b638668 c922e207469e 5adae181e337 2024-05-15T13:59:41.553 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi052 running (7m) 22s ago 7m 50.6M 4273M 19.0.0-3687-g9b638668 c922e207469e 7da3988ca00c 2024-05-15T13:59:41.553 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi081 running (7m) 22s ago 7m 50.8M 4272M 19.0.0-3687-g9b638668 c922e207469e 8a6fc4cb89da 2024-05-15T13:59:41.553 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi033 running (6m) 20s ago 6m 47.9M 4273M 19.0.0-3687-g9b638668 c922e207469e f25bac3320a5 2024-05-15T13:59:41.553 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi052 running (6m) 22s ago 6m 48.9M 4273M 19.0.0-3687-g9b638668 c922e207469e a0a6fec869be 2024-05-15T13:59:41.553 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi081 running (6m) 22s ago 6m 58.2M 4272M 19.0.0-3687-g9b638668 c922e207469e e3626aba0ece 2024-05-15T13:59:41.553 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6 smithi033 running (5m) 20s ago 5m 54.0M 4273M 19.0.0-3687-g9b638668 c922e207469e 3d29df46e4d5 2024-05-15T13:59:41.553 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7 smithi052 running (5m) 22s ago 5m 48.8M 4273M 19.0.0-3687-g9b638668 c922e207469e c0e3cc5f40b9 2024-05-15T13:59:41.553 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8 smithi081 running (4m) 22s ago 4m 56.5M 4272M 19.0.0-3687-g9b638668 c922e207469e 321f32b34f2b 2024-05-15T13:59:41.553 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9 smithi033 running (4m) 20s ago 4m 53.2M 4273M 19.0.0-3687-g9b638668 c922e207469e 529998a6cd60 2024-05-15T13:59:41.553 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10 smithi052 running (3m) 22s ago 3m 54.1M 4273M 19.0.0-3687-g9b638668 c922e207469e 1b4978797d63 2024-05-15T13:59:41.553 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11 smithi081 running (3m) 22s ago 3m 46.1M 4272M 19.0.0-3687-g9b638668 c922e207469e e9570a0c3fab 2024-05-15T13:59:41.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.552+0000 7f52146a4700 1 -- 172.21.15.33:0/2424699162 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f51f80b2900 msgr2=0x7f51f80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:41.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.552+0000 7f52146a4700 1 --2- 172.21.15.33:0/2424699162 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f51f80b2900 0x7f51f80b4dc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f51fc009990 tx=0x7f51fc0096b0 comp rx=0 tx=0).stop 2024-05-15T13:59:41.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.552+0000 7f52146a4700 1 -- 172.21.15.33:0/2424699162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f520c103590 msgr2=0x7f520c0772e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:41.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.552+0000 7f52146a4700 1 --2- 172.21.15.33:0/2424699162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f520c103590 0x7f520c0772e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f520800d980 tx=0x7f520800dc90 comp rx=0 tx=0).stop 2024-05-15T13:59:41.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.553+0000 7f52146a4700 1 -- 172.21.15.33:0/2424699162 shutdown_connections 2024-05-15T13:59:41.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.553+0000 7f52146a4700 1 --2- 172.21.15.33:0/2424699162 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f51f80b2900 0x7f51f80b4dc0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:41.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.553+0000 7f52146a4700 1 --2- 172.21.15.33:0/2424699162 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f520c105130 0x7f520c075e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:41.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.553+0000 7f52146a4700 1 --2- 172.21.15.33:0/2424699162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f520c103590 0x7f520c0772e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:41.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.553+0000 7f52146a4700 1 --2- 172.21.15.33:0/2424699162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f520c104790 0x7f520c075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:41.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.553+0000 7f52146a4700 1 -- 172.21.15.33:0/2424699162 >> 172.21.15.33:0/2424699162 conn(0x7f520c0feb30 msgr2=0x7f520c107b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:41.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.553+0000 7f52146a4700 1 -- 172.21.15.33:0/2424699162 shutdown_connections 2024-05-15T13:59:41.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:41.553+0000 7f52146a4700 1 -- 172.21.15.33:0/2424699162 wait complete. 2024-05-15T13:59:42.184 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-15T13:59:42.442 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:59:42.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:42 smithi033 ceph-mon[31650]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T13:59:42.443 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:42 smithi033 ceph-mon[31650]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:59:42.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:42 smithi081 ceph-mon[36434]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T13:59:42.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:42 smithi081 ceph-mon[36434]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:59:42.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:42 smithi052 ceph-mon[36190]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T13:59:42.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:42 smithi052 ceph-mon[36190]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:59:44.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.115+0000 7fbf9aeee700 1 -- 172.21.15.33:0/537811682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf941047c0 msgr2=0x7fbf94104c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:44.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.115+0000 7fbf9aeee700 1 --2- 172.21.15.33:0/537811682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf941047c0 0x7fbf94104c20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbf800099c0 tx=0x7fbf80009cd0 comp rx=0 tx=0).stop 2024-05-15T13:59:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.116+0000 7fbf9aeee700 1 -- 172.21.15.33:0/537811682 shutdown_connections 2024-05-15T13:59:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.116+0000 7fbf9aeee700 1 --2- 172.21.15.33:0/537811682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf94105160 0x7fbf9410ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.116+0000 7fbf9aeee700 1 --2- 172.21.15.33:0/537811682 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf941035c0 0x7fbf941039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.116+0000 7fbf9aeee700 1 --2- 172.21.15.33:0/537811682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf941047c0 0x7fbf94104c20 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.116+0000 7fbf9aeee700 1 -- 172.21.15.33:0/537811682 >> 172.21.15.33:0/537811682 conn(0x7fbf940feb60 msgr2=0x7fbf94100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.116+0000 7fbf9aeee700 1 -- 172.21.15.33:0/537811682 shutdown_connections 2024-05-15T13:59:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.116+0000 7fbf9aeee700 1 -- 172.21.15.33:0/537811682 wait complete. 2024-05-15T13:59:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.116+0000 7fbf9aeee700 1 Processor -- start 2024-05-15T13:59:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.116+0000 7fbf9aeee700 1 -- start start 2024-05-15T13:59:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.117+0000 7fbf9aeee700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf941035c0 0x7fbf941996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.117+0000 7fbf9aeee700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf941047c0 0x7fbf94199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.117+0000 7fbf9aeee700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf94105160 0x7fbf941a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.117+0000 7fbf9aeee700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9410f060 con 0x7fbf94105160 2024-05-15T13:59:44.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.117+0000 7fbf9aeee700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9410eee0 con 0x7fbf941035c0 2024-05-15T13:59:44.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.117+0000 7fbf9aeee700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9410f1e0 con 0x7fbf941047c0 2024-05-15T13:59:44.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.117+0000 7fbf9948b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf94105160 0x7fbf941a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:44.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.117+0000 7fbf9948b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf94105160 0x7fbf941a0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55518/0 (socket says 172.21.15.33:55518) 2024-05-15T13:59:44.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.117+0000 7fbf9948b700 1 -- 172.21.15.33:0/1603423192 learned_addr learned my addr 172.21.15.33:0/1603423192 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:44.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.117+0000 7fbf93fff700 1 --2- 172.21.15.33:0/1603423192 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf941047c0 0x7fbf94199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:44.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.117+0000 7fbf98c8a700 1 --2- 172.21.15.33:0/1603423192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf941035c0 0x7fbf941996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:44.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.117+0000 7fbf9948b700 1 -- 172.21.15.33:0/1603423192 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf941047c0 msgr2=0x7fbf94199c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:44.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.117+0000 7fbf9948b700 1 --2- 172.21.15.33:0/1603423192 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf941047c0 0x7fbf94199c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:44.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.117+0000 7fbf9948b700 1 -- 172.21.15.33:0/1603423192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf941035c0 msgr2=0x7fbf941996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:44.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.117+0000 7fbf9948b700 1 --2- 172.21.15.33:0/1603423192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf941035c0 0x7fbf941996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:44.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.117+0000 7fbf9948b700 1 -- 172.21.15.33:0/1603423192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf941a1330 con 0x7fbf94105160 2024-05-15T13:59:44.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.117+0000 7fbf9948b700 1 --2- 172.21.15.33:0/1603423192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf94105160 0x7fbf941a0c90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c007f10 tx=0x7fbf8c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:44.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.118+0000 7fbf91ffb700 1 -- 172.21.15.33:0/1603423192 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c004bb0 con 0x7fbf94105160 2024-05-15T13:59:44.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.118+0000 7fbf91ffb700 1 -- 172.21.15.33:0/1603423192 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf8c004d50 con 0x7fbf94105160 2024-05-15T13:59:44.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.118+0000 7fbf9aeee700 1 -- 172.21.15.33:0/1603423192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf941a1620 con 0x7fbf94105160 2024-05-15T13:59:44.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.118+0000 7fbf91ffb700 1 -- 172.21.15.33:0/1603423192 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c0058e0 con 0x7fbf94105160 2024-05-15T13:59:44.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.118+0000 7fbf9aeee700 1 -- 172.21.15.33:0/1603423192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf941a1b60 con 0x7fbf94105160 2024-05-15T13:59:44.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.119+0000 7fbf9aeee700 1 -- 172.21.15.33:0/1603423192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf9406a230 con 0x7fbf94105160 2024-05-15T13:59:44.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.121+0000 7fbf91ffb700 1 -- 172.21.15.33:0/1603423192 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbf8c01a020 con 0x7fbf94105160 2024-05-15T13:59:44.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.121+0000 7fbf91ffb700 1 --2- 172.21.15.33:0/1603423192 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbf840b2880 0x7fbf840b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:44.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.122+0000 7fbf98c8a700 1 --2- 172.21.15.33:0/1603423192 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbf840b2880 0x7fbf840b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:44.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.122+0000 7fbf91ffb700 1 -- 172.21.15.33:0/1603423192 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fbf8c0e1940 con 0x7fbf94105160 2024-05-15T13:59:44.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.122+0000 7fbf98c8a700 1 --2- 172.21.15.33:0/1603423192 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbf840b2880 0x7fbf840b4d40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbf94104620 tx=0x7fbf88009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.126+0000 7fbf91ffb700 1 -- 172.21.15.33:0/1603423192 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbf8c010070 con 0x7fbf94105160 2024-05-15T13:59:44.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.350+0000 7fbf9aeee700 1 -- 172.21.15.33:0/1603423192 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fbf94063980 con 0x7fbf840b2880 2024-05-15T13:59:44.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.356+0000 7fbf91ffb700 1 -- 172.21.15.33:0/1603423192 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fbf94063980 con 0x7fbf840b2880 2024-05-15T13:59:44.358 INFO:teuthology.orchestra.run.smithi033.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-15T13:59:44.359 INFO:teuthology.orchestra.run.smithi033.stdout:mds.all 12/12 25s ago 3m smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi052=b;smithi052=e;smithi052=h;smithi052=k;smithi081=c;smithi081=f;smithi081=i;smithi081=l;count:12 2024-05-15T13:59:44.359 INFO:teuthology.orchestra.run.smithi033.stdout:mgr 3/3 25s ago 8m smithi033=x;smithi052=y;smithi081=z;count:3 2024-05-15T13:59:44.359 INFO:teuthology.orchestra.run.smithi033.stdout:mon 3/3 25s ago 8m smithi033:172.21.15.33=a;smithi052:172.21.15.52=b;smithi081:172.21.15.81=c;count:3 2024-05-15T13:59:44.359 INFO:teuthology.orchestra.run.smithi033.stdout:osd 12 25s ago - 2024-05-15T13:59:44.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.359+0000 7fbf9aeee700 1 -- 172.21.15.33:0/1603423192 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbf840b2880 msgr2=0x7fbf840b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:44.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.359+0000 7fbf9aeee700 1 --2- 172.21.15.33:0/1603423192 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbf840b2880 0x7fbf840b4d40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbf94104620 tx=0x7fbf88009260 comp rx=0 tx=0).stop 2024-05-15T13:59:44.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.359+0000 7fbf9aeee700 1 -- 172.21.15.33:0/1603423192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf94105160 msgr2=0x7fbf941a0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:44.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.359+0000 7fbf9aeee700 1 --2- 172.21.15.33:0/1603423192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf94105160 0x7fbf941a0c90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c007f10 tx=0x7fbf8c00d3b0 comp rx=0 tx=0).stop 2024-05-15T13:59:44.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.359+0000 7fbf9aeee700 1 -- 172.21.15.33:0/1603423192 shutdown_connections 2024-05-15T13:59:44.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.359+0000 7fbf9aeee700 1 --2- 172.21.15.33:0/1603423192 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fbf840b2880 0x7fbf840b4d40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:44.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.359+0000 7fbf9aeee700 1 --2- 172.21.15.33:0/1603423192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf94105160 0x7fbf941a0c90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:44.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.359+0000 7fbf9aeee700 1 --2- 172.21.15.33:0/1603423192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf941035c0 0x7fbf941996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:44.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.359+0000 7fbf9aeee700 1 --2- 172.21.15.33:0/1603423192 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf941047c0 0x7fbf94199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:44.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.359+0000 7fbf9aeee700 1 -- 172.21.15.33:0/1603423192 >> 172.21.15.33:0/1603423192 conn(0x7fbf940feb60 msgr2=0x7fbf94107b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:44.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.359+0000 7fbf9aeee700 1 -- 172.21.15.33:0/1603423192 shutdown_connections 2024-05-15T13:59:44.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:44.359+0000 7fbf9aeee700 1 -- 172.21.15.33:0/1603423192 wait complete. 2024-05-15T13:59:44.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:44 smithi081 ceph-mon[36434]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-15T13:59:44.676 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:44 smithi033 ceph-mon[31650]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-15T13:59:44.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:44 smithi052 ceph-mon[36190]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-15T13:59:44.983 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-15T13:59:45.236 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:59:45.631 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:45 smithi033 ceph-mon[31650]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:59:45.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:45 smithi052 ceph-mon[36190]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:59:45.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:45 smithi081 ceph-mon[36434]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:59:46.514 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:46 smithi033 ceph-mon[31650]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:59:46.741 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:46 smithi052 ceph-mon[36190]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:59:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.762+0000 7f3a0a872700 1 -- 172.21.15.33:0/2685761373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a041092a0 msgr2=0x7f3a0410b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.762+0000 7f3a0a872700 1 --2- 172.21.15.33:0/2685761373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a041092a0 0x7f3a0410b710 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f39fc00b0a0 tx=0x7f39fc00b3b0 comp rx=0 tx=0).stop 2024-05-15T13:59:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.762+0000 7f3a0a872700 1 -- 172.21.15.33:0/2685761373 shutdown_connections 2024-05-15T13:59:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.762+0000 7f3a0a872700 1 --2- 172.21.15.33:0/2685761373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a041092a0 0x7f3a0410b710 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.762+0000 7f3a0a872700 1 --2- 172.21.15.33:0/2685761373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a04104040 0x7f3a04106430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:46.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.762+0000 7f3a0a872700 1 --2- 172.21.15.33:0/2685761373 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a04106970 0x7f3a04108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:46.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.762+0000 7f3a0a872700 1 -- 172.21.15.33:0/2685761373 >> 172.21.15.33:0/2685761373 conn(0x7f3a040fda70 msgr2=0x7f3a040ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:46.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.762+0000 7f3a0a872700 1 -- 172.21.15.33:0/2685761373 shutdown_connections 2024-05-15T13:59:46.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.762+0000 7f3a0a872700 1 -- 172.21.15.33:0/2685761373 wait complete. 2024-05-15T13:59:46.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.762+0000 7f3a0a872700 1 Processor -- start 2024-05-15T13:59:46.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.763+0000 7f3a0a872700 1 -- start start 2024-05-15T13:59:46.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.763+0000 7f3a0a872700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a04104040 0x7f3a0419ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:46.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.763+0000 7f3a0a872700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a04106970 0x7f3a0419bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:46.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.763+0000 7f3a0a872700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a041092a0 0x7f3a041a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:46.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.763+0000 7f3a0a872700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0410f100 con 0x7f3a041092a0 2024-05-15T13:59:46.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.763+0000 7f3a0a872700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0410ef80 con 0x7f3a04106970 2024-05-15T13:59:46.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.763+0000 7f3a0a872700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0410f280 con 0x7f3a04104040 2024-05-15T13:59:46.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.763+0000 7f3a08e0f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a041092a0 0x7f3a041a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:46.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.763+0000 7f3a08e0f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a041092a0 0x7f3a041a2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55526/0 (socket says 172.21.15.33:55526) 2024-05-15T13:59:46.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.763+0000 7f3a08e0f700 1 -- 172.21.15.33:0/2733706463 learned_addr learned my addr 172.21.15.33:0/2733706463 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:46.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.763+0000 7f39fbfff700 1 --2- 172.21.15.33:0/2733706463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a04106970 0x7f3a0419bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:46.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.763+0000 7f3a03fff700 1 --2- 172.21.15.33:0/2733706463 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a04104040 0x7f3a0419ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:46.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.763+0000 7f3a08e0f700 1 -- 172.21.15.33:0/2733706463 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a04104040 msgr2=0x7f3a0419ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:46.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.763+0000 7f3a08e0f700 1 --2- 172.21.15.33:0/2733706463 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a04104040 0x7f3a0419ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:46.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.763+0000 7f3a08e0f700 1 -- 172.21.15.33:0/2733706463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a04106970 msgr2=0x7f3a0419bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:46.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.763+0000 7f3a08e0f700 1 --2- 172.21.15.33:0/2733706463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a04106970 0x7f3a0419bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:46.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.763+0000 7f3a08e0f700 1 -- 172.21.15.33:0/2733706463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a041a3680 con 0x7f3a041092a0 2024-05-15T13:59:46.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.763+0000 7f3a08e0f700 1 --2- 172.21.15.33:0/2733706463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a041092a0 0x7f3a041a2fe0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f39fc006070 tx=0x7f39fc00ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:46.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.764+0000 7f3a01ffb700 1 -- 172.21.15.33:0/2733706463 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39fc021070 con 0x7f3a041092a0 2024-05-15T13:59:46.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.764+0000 7f3a01ffb700 1 -- 172.21.15.33:0/2733706463 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39fc007e30 con 0x7f3a041092a0 2024-05-15T13:59:46.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.764+0000 7f3a0a872700 1 -- 172.21.15.33:0/2733706463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a041a3910 con 0x7f3a041092a0 2024-05-15T13:59:46.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.764+0000 7f3a01ffb700 1 -- 172.21.15.33:0/2733706463 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39fc0047d0 con 0x7f3a041092a0 2024-05-15T13:59:46.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.764+0000 7f3a0a872700 1 -- 172.21.15.33:0/2733706463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a041a3df0 con 0x7f3a041092a0 2024-05-15T13:59:46.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.765+0000 7f3a0a872700 1 -- 172.21.15.33:0/2733706463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a0406a230 con 0x7f3a041092a0 2024-05-15T13:59:46.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.766+0000 7f3a01ffb700 1 -- 172.21.15.33:0/2733706463 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f39fc007980 con 0x7f3a041092a0 2024-05-15T13:59:46.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.767+0000 7f3a01ffb700 1 --2- 172.21.15.33:0/2733706463 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f39f00b2880 0x7f39f00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:46.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.767+0000 7f3a03fff700 1 --2- 172.21.15.33:0/2733706463 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f39f00b2880 0x7f39f00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:46.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.768+0000 7f3a01ffb700 1 -- 172.21.15.33:0/2733706463 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f39fc00f020 con 0x7f3a041092a0 2024-05-15T13:59:46.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.768+0000 7f3a03fff700 1 --2- 172.21.15.33:0/2733706463 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f39f00b2880 0x7f39f00b4d40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f39f4004630 tx=0x7f39f400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:46.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.771+0000 7f3a01ffb700 1 -- 172.21.15.33:0/2733706463 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f39fc02a020 con 0x7f3a041092a0 2024-05-15T13:59:46.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:46 smithi081 ceph-mon[36434]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:59:46.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.991+0000 7f3a0a872700 1 -- 172.21.15.33:0/2733706463 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f3a04063980 con 0x7f39f00b2880 2024-05-15T13:59:46.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.992+0000 7f3a01ffb700 1 -- 172.21.15.33:0/2733706463 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+187 (secure 0 0 0) 0x7f3a04063980 con 0x7f39f00b2880 2024-05-15T13:59:46.994 INFO:teuthology.orchestra.run.smithi033.stdout:HOST ADDR LABELS STATUS 2024-05-15T13:59:46.995 INFO:teuthology.orchestra.run.smithi033.stdout:smithi033 172.21.15.33 2024-05-15T13:59:46.995 INFO:teuthology.orchestra.run.smithi033.stdout:smithi052 172.21.15.52 2024-05-15T13:59:46.995 INFO:teuthology.orchestra.run.smithi033.stdout:smithi081 172.21.15.81 2024-05-15T13:59:46.995 INFO:teuthology.orchestra.run.smithi033.stdout:3 hosts in cluster 2024-05-15T13:59:46.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.996+0000 7f3a0a872700 1 -- 172.21.15.33:0/2733706463 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f39f00b2880 msgr2=0x7f39f00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:46.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.996+0000 7f3a0a872700 1 --2- 172.21.15.33:0/2733706463 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f39f00b2880 0x7f39f00b4d40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f39f4004630 tx=0x7f39f400b3c0 comp rx=0 tx=0).stop 2024-05-15T13:59:46.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.996+0000 7f3a0a872700 1 -- 172.21.15.33:0/2733706463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a041092a0 msgr2=0x7f3a041a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:46.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.996+0000 7f3a0a872700 1 --2- 172.21.15.33:0/2733706463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a041092a0 0x7f3a041a2fe0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f39fc006070 tx=0x7f39fc00ce50 comp rx=0 tx=0).stop 2024-05-15T13:59:46.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.996+0000 7f3a0a872700 1 -- 172.21.15.33:0/2733706463 shutdown_connections 2024-05-15T13:59:46.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.996+0000 7f3a0a872700 1 --2- 172.21.15.33:0/2733706463 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f39f00b2880 0x7f39f00b4d40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:46.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.996+0000 7f3a0a872700 1 --2- 172.21.15.33:0/2733706463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a041092a0 0x7f3a041a2fe0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:46.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.996+0000 7f3a0a872700 1 --2- 172.21.15.33:0/2733706463 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3a04104040 0x7f3a0419ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:46.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.996+0000 7f3a0a872700 1 --2- 172.21.15.33:0/2733706463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a04106970 0x7f3a0419bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:46.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.996+0000 7f3a0a872700 1 -- 172.21.15.33:0/2733706463 >> 172.21.15.33:0/2733706463 conn(0x7f3a040fda70 msgr2=0x7f3a04103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:46.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.996+0000 7f3a0a872700 1 -- 172.21.15.33:0/2733706463 shutdown_connections 2024-05-15T13:59:46.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:46.996+0000 7f3a0a872700 1 -- 172.21.15.33:0/2733706463 wait complete. 2024-05-15T13:59:47.572 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-15T13:59:47.822 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:59:48.673 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:48 smithi033 ceph-mon[31650]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:59:48.673 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:48 smithi033 ceph-mon[31650]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-15T13:59:48.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:48 smithi052 ceph-mon[36190]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:59:48.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:48 smithi052 ceph-mon[36190]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-15T13:59:48.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:48 smithi081 ceph-mon[36434]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:59:48.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:48 smithi081 ceph-mon[36434]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-15T13:59:49.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.412+0000 7f86dca88700 1 -- 172.21.15.33:0/3445993952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86d8105140 msgr2=0x7f86d810b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:49.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.412+0000 7f86dca88700 1 --2- 172.21.15.33:0/3445993952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86d8105140 0x7f86d810b9f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f86d000b0d0 tx=0x7f86d000b3e0 comp rx=0 tx=0).stop 2024-05-15T13:59:49.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.413+0000 7f86dca88700 1 -- 172.21.15.33:0/3445993952 shutdown_connections 2024-05-15T13:59:49.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.413+0000 7f86dca88700 1 --2- 172.21.15.33:0/3445993952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86d8105140 0x7f86d810b9f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:49.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.413+0000 7f86dca88700 1 --2- 172.21.15.33:0/3445993952 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f86d81035a0 0x7f86d81039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:49.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.413+0000 7f86dca88700 1 --2- 172.21.15.33:0/3445993952 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86d81047a0 0x7f86d8104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:49.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.413+0000 7f86dca88700 1 -- 172.21.15.33:0/3445993952 >> 172.21.15.33:0/3445993952 conn(0x7f86d80feb40 msgr2=0x7f86d8100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:49.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.413+0000 7f86dca88700 1 -- 172.21.15.33:0/3445993952 shutdown_connections 2024-05-15T13:59:49.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.413+0000 7f86dca88700 1 -- 172.21.15.33:0/3445993952 wait complete. 2024-05-15T13:59:49.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.413+0000 7f86dca88700 1 Processor -- start 2024-05-15T13:59:49.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.413+0000 7f86dca88700 1 -- start start 2024-05-15T13:59:49.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.413+0000 7f86dca88700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86d81035a0 0x7f86d819b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:49.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.414+0000 7f86dca88700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f86d81047a0 0x7f86d819bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:49.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.414+0000 7f86dca88700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86d8105140 0x7f86d81a2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:49.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.414+0000 7f86dca88700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f86d810f040 con 0x7f86d81035a0 2024-05-15T13:59:49.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.414+0000 7f86dca88700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f86d810eec0 con 0x7f86d8105140 2024-05-15T13:59:49.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.414+0000 7f86dca88700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f86d810f1c0 con 0x7f86d81047a0 2024-05-15T13:59:49.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.414+0000 7f86d659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86d81035a0 0x7f86d819b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:49.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.414+0000 7f86d659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86d81035a0 0x7f86d819b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55544/0 (socket says 172.21.15.33:55544) 2024-05-15T13:59:49.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.414+0000 7f86d659c700 1 -- 172.21.15.33:0/746855208 learned_addr learned my addr 172.21.15.33:0/746855208 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:49.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.414+0000 7f86d6d9d700 1 --2- 172.21.15.33:0/746855208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86d8105140 0x7f86d81a2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:49.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.414+0000 7f86d5d9b700 1 --2- 172.21.15.33:0/746855208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f86d81047a0 0x7f86d819bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:49.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.414+0000 7f86d659c700 1 -- 172.21.15.33:0/746855208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f86d81047a0 msgr2=0x7f86d819bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:49.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.414+0000 7f86d659c700 1 --2- 172.21.15.33:0/746855208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f86d81047a0 0x7f86d819bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:49.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.414+0000 7f86d659c700 1 -- 172.21.15.33:0/746855208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86d8105140 msgr2=0x7f86d81a2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:49.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.414+0000 7f86d659c700 1 --2- 172.21.15.33:0/746855208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86d8105140 0x7f86d81a2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:49.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.414+0000 7f86d659c700 1 -- 172.21.15.33:0/746855208 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d81a3470 con 0x7f86d81035a0 2024-05-15T13:59:49.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.414+0000 7f86d659c700 1 --2- 172.21.15.33:0/746855208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86d81035a0 0x7f86d819b810 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f86c800d9b0 tx=0x7f86c800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:49.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.414+0000 7f86c77fe700 1 -- 172.21.15.33:0/746855208 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c8017070 con 0x7f86d81035a0 2024-05-15T13:59:49.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.414+0000 7f86c77fe700 1 -- 172.21.15.33:0/746855208 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86c8009de0 con 0x7f86d81035a0 2024-05-15T13:59:49.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.414+0000 7f86dca88700 1 -- 172.21.15.33:0/746855208 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86d81a3760 con 0x7f86d81035a0 2024-05-15T13:59:49.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.414+0000 7f86c77fe700 1 -- 172.21.15.33:0/746855208 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c800f3c0 con 0x7f86d81035a0 2024-05-15T13:59:49.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.414+0000 7f86dca88700 1 -- 172.21.15.33:0/746855208 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86d81a3c50 con 0x7f86d81035a0 2024-05-15T13:59:49.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.415+0000 7f86dca88700 1 -- 172.21.15.33:0/746855208 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86d806a230 con 0x7f86d81035a0 2024-05-15T13:59:49.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.417+0000 7f86c77fe700 1 -- 172.21.15.33:0/746855208 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f86c8009930 con 0x7f86d81035a0 2024-05-15T13:59:49.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.417+0000 7f86c77fe700 1 --2- 172.21.15.33:0/746855208 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f86c00b2880 0x7f86c00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:49.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.418+0000 7f86d5d9b700 1 --2- 172.21.15.33:0/746855208 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f86c00b2880 0x7f86c00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:49.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.418+0000 7f86c77fe700 1 -- 172.21.15.33:0/746855208 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f86c80e1d70 con 0x7f86d81035a0 2024-05-15T13:59:49.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.418+0000 7f86d5d9b700 1 --2- 172.21.15.33:0/746855208 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f86c00b2880 0x7f86c00b4d40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f86cc005ea0 tx=0x7f86cc005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:49.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.421+0000 7f86c77fe700 1 -- 172.21.15.33:0/746855208 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f86c80a9a90 con 0x7f86d81035a0 2024-05-15T13:59:49.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.643+0000 7f86dca88700 1 -- 172.21.15.33:0/746855208 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f86d8076e90 con 0x7f86c00b2880 2024-05-15T13:59:49.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.645+0000 7f86c77fe700 1 -- 172.21.15.33:0/746855208 <== mgr.14152 v2:172.21.15.33:6800/1797827152 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f86d8076e90 con 0x7f86c00b2880 2024-05-15T13:59:49.647 INFO:teuthology.orchestra.run.smithi033.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-15T13:59:49.647 INFO:teuthology.orchestra.run.smithi033.stdout:smithi033 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330008Q400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-05-15T13:59:49.647 INFO:teuthology.orchestra.run.smithi033.stdout:smithi052 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513010N480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-05-15T13:59:49.647 INFO:teuthology.orchestra.run.smithi033.stdout:smithi081 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400N7400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-05-15T13:59:49.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.648+0000 7f86dca88700 1 -- 172.21.15.33:0/746855208 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f86c00b2880 msgr2=0x7f86c00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:49.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.648+0000 7f86dca88700 1 --2- 172.21.15.33:0/746855208 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f86c00b2880 0x7f86c00b4d40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f86cc005ea0 tx=0x7f86cc005e30 comp rx=0 tx=0).stop 2024-05-15T13:59:49.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.648+0000 7f86dca88700 1 -- 172.21.15.33:0/746855208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86d81035a0 msgr2=0x7f86d819b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:49.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.648+0000 7f86dca88700 1 --2- 172.21.15.33:0/746855208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86d81035a0 0x7f86d819b810 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f86c800d9b0 tx=0x7f86c800dcc0 comp rx=0 tx=0).stop 2024-05-15T13:59:49.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.648+0000 7f86dca88700 1 -- 172.21.15.33:0/746855208 shutdown_connections 2024-05-15T13:59:49.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.648+0000 7f86dca88700 1 --2- 172.21.15.33:0/746855208 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f86c00b2880 0x7f86c00b4d40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:49.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.648+0000 7f86dca88700 1 --2- 172.21.15.33:0/746855208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86d8105140 0x7f86d81a2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:49.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.648+0000 7f86dca88700 1 --2- 172.21.15.33:0/746855208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86d81035a0 0x7f86d819b810 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:49.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.648+0000 7f86dca88700 1 --2- 172.21.15.33:0/746855208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f86d81047a0 0x7f86d819bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:49.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.648+0000 7f86dca88700 1 -- 172.21.15.33:0/746855208 >> 172.21.15.33:0/746855208 conn(0x7f86d80feb40 msgr2=0x7f86d8075770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:49.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.648+0000 7f86dca88700 1 -- 172.21.15.33:0/746855208 shutdown_connections 2024-05-15T13:59:49.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:49.648+0000 7f86dca88700 1 -- 172.21.15.33:0/746855208 wait complete. 2024-05-15T13:59:50.306 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-15T13:59:50.315 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-05-15T13:59:50.316 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-15T13:59:50.571 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:59:50.659 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:50 smithi081 ceph-mon[36434]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:59:50.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:50 smithi033 ceph-mon[31650]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:59:50.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:50 smithi052 ceph-mon[36190]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:59:51.591 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:51 smithi033 ceph-mon[31650]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:59:51.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:51 smithi052 ceph-mon[36190]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:59:51.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:51 smithi081 ceph-mon[36434]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T13:59:52.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.136+0000 7fe56ce42700 1 -- 172.21.15.33:0/258056180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe568104040 msgr2=0x7fe56810a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:52.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.136+0000 7fe56ce42700 1 --2- 172.21.15.33:0/258056180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe568104040 0x7fe56810a8f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe56000b0d0 tx=0x7fe56000b3e0 comp rx=0 tx=0).stop 2024-05-15T13:59:52.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.136+0000 7fe56ce42700 1 -- 172.21.15.33:0/258056180 shutdown_connections 2024-05-15T13:59:52.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.136+0000 7fe56ce42700 1 --2- 172.21.15.33:0/258056180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe568104040 0x7fe56810a8f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:52.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.136+0000 7fe56ce42700 1 --2- 172.21.15.33:0/258056180 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe5681024a0 0x7fe5681028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:52.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.136+0000 7fe56ce42700 1 --2- 172.21.15.33:0/258056180 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5681036a0 0x7fe568103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:52.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.136+0000 7fe56ce42700 1 -- 172.21.15.33:0/258056180 >> 172.21.15.33:0/258056180 conn(0x7fe5680fda40 msgr2=0x7fe5680ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:52.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.136+0000 7fe56ce42700 1 -- 172.21.15.33:0/258056180 shutdown_connections 2024-05-15T13:59:52.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.137+0000 7fe56ce42700 1 -- 172.21.15.33:0/258056180 wait complete. 2024-05-15T13:59:52.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.137+0000 7fe56ce42700 1 Processor -- start 2024-05-15T13:59:52.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.137+0000 7fe56ce42700 1 -- start start 2024-05-15T13:59:52.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.138+0000 7fe56ce42700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5681024a0 0x7fe568106430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:52.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.138+0000 7fe56ce42700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5681036a0 0x7fe568104a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:52.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.138+0000 7fe56ce42700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe568104040 0x7fe568104fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:52.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.138+0000 7fe56ce42700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe56810df90 con 0x7fe5681024a0 2024-05-15T13:59:52.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.138+0000 7fe56ce42700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe56810de10 con 0x7fe568104040 2024-05-15T13:59:52.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.138+0000 7fe56ce42700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe56810e110 con 0x7fe5681036a0 2024-05-15T13:59:52.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.138+0000 7fe56659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5681024a0 0x7fe568106430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:52.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.138+0000 7fe56659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5681024a0 0x7fe568106430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55574/0 (socket says 172.21.15.33:55574) 2024-05-15T13:59:52.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.138+0000 7fe56659c700 1 -- 172.21.15.33:0/451938585 learned_addr learned my addr 172.21.15.33:0/451938585 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:52.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.138+0000 7fe566d9d700 1 --2- 172.21.15.33:0/451938585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe568104040 0x7fe568104fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:52.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.138+0000 7fe565d9b700 1 --2- 172.21.15.33:0/451938585 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5681036a0 0x7fe568104a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:52.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.138+0000 7fe56659c700 1 -- 172.21.15.33:0/451938585 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5681036a0 msgr2=0x7fe568104a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:52.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.138+0000 7fe56659c700 1 --2- 172.21.15.33:0/451938585 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5681036a0 0x7fe568104a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:52.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.138+0000 7fe56659c700 1 -- 172.21.15.33:0/451938585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe568104040 msgr2=0x7fe568104fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:52.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.138+0000 7fe56659c700 1 --2- 172.21.15.33:0/451938585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe568104040 0x7fe568104fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:52.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.138+0000 7fe56659c700 1 -- 172.21.15.33:0/451938585 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe568105870 con 0x7fe5681024a0 2024-05-15T13:59:52.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.138+0000 7fe56659c700 1 --2- 172.21.15.33:0/451938585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5681024a0 0x7fe568106430 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe55800d980 tx=0x7fe55800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:52.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.139+0000 7fe5577fe700 1 -- 172.21.15.33:0/451938585 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe558017030 con 0x7fe5681024a0 2024-05-15T13:59:52.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.139+0000 7fe56ce42700 1 -- 172.21.15.33:0/451938585 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5681a38d0 con 0x7fe5681024a0 2024-05-15T13:59:52.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.139+0000 7fe5577fe700 1 -- 172.21.15.33:0/451938585 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5580044d0 con 0x7fe5681024a0 2024-05-15T13:59:52.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.139+0000 7fe5577fe700 1 -- 172.21.15.33:0/451938585 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe558003970 con 0x7fe5681024a0 2024-05-15T13:59:52.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.139+0000 7fe56ce42700 1 -- 172.21.15.33:0/451938585 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5681a3e10 con 0x7fe5681024a0 2024-05-15T13:59:52.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.140+0000 7fe56ce42700 1 -- 172.21.15.33:0/451938585 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe56806a230 con 0x7fe5681024a0 2024-05-15T13:59:52.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.141+0000 7fe5577fe700 1 -- 172.21.15.33:0/451938585 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe55801a070 con 0x7fe5681024a0 2024-05-15T13:59:52.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.142+0000 7fe5577fe700 1 --2- 172.21.15.33:0/451938585 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fe5500b2900 0x7fe5500b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:52.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.142+0000 7fe565d9b700 1 --2- 172.21.15.33:0/451938585 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fe5500b2900 0x7fe5500b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:52.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.142+0000 7fe5577fe700 1 -- 172.21.15.33:0/451938585 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fe5580e1dd0 con 0x7fe5681024a0 2024-05-15T13:59:52.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.143+0000 7fe565d9b700 1 --2- 172.21.15.33:0/451938585 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fe5500b2900 0x7fe5500b4dc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe55c004630 tx=0x7fe55c00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:52.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.146+0000 7fe5577fe700 1 -- 172.21.15.33:0/451938585 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe5580a9b70 con 0x7fe5681024a0 2024-05-15T13:59:52.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.407+0000 7fe56ce42700 1 -- 172.21.15.33:0/451938585 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fe5681a40a0 con 0x7fe5681024a0 2024-05-15T13:59:52.409 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:52 smithi033 ceph-mon[31650]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T13:59:52.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.408+0000 7fe5577fe700 1 -- 172.21.15.33:0/451938585 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) ==== 76+0+3041 (secure 0 0 0) 0x7fe5580a9470 con 0x7fe5681024a0 2024-05-15T13:59:52.409 INFO:teuthology.orchestra.run.smithi033.stdout:e23 2024-05-15T13:59:52.410 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-15T13:59:52.410 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-15T13:59:52.410 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-05-15T13:59:52.410 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:59:52.410 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-05-15T13:59:52.410 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-05-15T13:59:52.410 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 23 2024-05-15T13:59:52.410 INFO:teuthology.orchestra.run.smithi033.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-15T13:59:52.410 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-05-15T13:57:19.056417+0000 2024-05-15T13:59:52.410 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-05-15T13:57:38.201184+0000 2024-05-15T13:59:52.410 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-05-15T13:59:52.411 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-05-15T13:59:52.411 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 300 2024-05-15T13:59:52.411 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-05-15T13:59:52.411 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-05-15T13:59:52.411 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-05-15T13:59:52.411 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-05-15T13:59:52.411 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-05-15T13:59:52.411 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 0 2024-05-15T13:59:52.412 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-15T13:59:52.412 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 3 2024-05-15T13:59:52.412 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1,2 2024-05-15T13:59:52.412 INFO:teuthology.orchestra.run.smithi033.stdout:up {0=24478,1=24463,2=24461} 2024-05-15T13:59:52.412 INFO:teuthology.orchestra.run.smithi033.stdout:failed 2024-05-15T13:59:52.412 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-05-15T13:59:52.412 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-05-15T13:59:52.413 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-05-15T13:59:52.413 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-05-15T13:59:52.413 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-05-15T13:59:52.414 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-05-15T13:59:52.414 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-05-15T13:59:52.414 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 0 2024-05-15T13:59:52.414 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 24478 members: 24478,24461,24463 2024-05-15T13:59:52.415 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.i{0:24478} state up:active seq 8 addr [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T13:59:52.415 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.f{0:24457} state up:standby-replay seq 1 addr [v2:172.21.15.81:6834/3773839758,v1:172.21.15.81:6835/3773839758] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T13:59:52.415 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.c{1:24463} state up:active seq 12 addr [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T13:59:52.415 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.h{1:24440} state up:standby-replay seq 1 addr [v2:172.21.15.52:6836/1629697841,v1:172.21.15.52:6837/1629697841] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T13:59:52.415 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.b{2:24461} state up:active seq 9 addr [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T13:59:52.415 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.l{2:24424} state up:standby-replay seq 1 addr [v2:172.21.15.81:6832/1872861431,v1:172.21.15.81:6833/1872861431] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T13:59:52.416 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:59:52.416 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:59:52.416 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-05-15T13:59:52.416 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:59:52.416 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.j{-1:14535} state up:standby seq 1 addr [v2:172.21.15.33:6834/3975341851,v1:172.21.15.33:6835/3975341851] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T13:59:52.416 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.d{-1:14547} state up:standby seq 1 addr [v2:172.21.15.33:6836/3558077615,v1:172.21.15.33:6837/3558077615] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T13:59:52.416 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.g{-1:14598} state up:standby seq 1 addr [v2:172.21.15.33:6838/2030117633,v1:172.21.15.33:6839/2030117633] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T13:59:52.417 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.a{-1:14610} state up:standby seq 1 addr [v2:172.21.15.33:6840/1918979061,v1:172.21.15.33:6841/1918979061] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T13:59:52.417 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.k{-1:24395} state up:standby seq 1 addr [v2:172.21.15.52:6832/2083190067,v1:172.21.15.52:6833/2083190067] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T13:59:52.417 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.e{-1:24416} state up:standby seq 1 addr [v2:172.21.15.52:6834/3191964924,v1:172.21.15.52:6835/3191964924] compat {c=[1],r=[1],i=[1fff]}] 2024-05-15T13:59:52.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.410+0000 7fe56ce42700 1 -- 172.21.15.33:0/451938585 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fe5500b2900 msgr2=0x7fe5500b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:52.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.410+0000 7fe56ce42700 1 --2- 172.21.15.33:0/451938585 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fe5500b2900 0x7fe5500b4dc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe55c004630 tx=0x7fe55c00b3c0 comp rx=0 tx=0).stop 2024-05-15T13:59:52.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.410+0000 7fe56ce42700 1 -- 172.21.15.33:0/451938585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5681024a0 msgr2=0x7fe568106430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:52.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.410+0000 7fe56ce42700 1 --2- 172.21.15.33:0/451938585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5681024a0 0x7fe568106430 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe55800d980 tx=0x7fe55800dc90 comp rx=0 tx=0).stop 2024-05-15T13:59:52.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.410+0000 7fe56ce42700 1 -- 172.21.15.33:0/451938585 shutdown_connections 2024-05-15T13:59:52.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.410+0000 7fe56ce42700 1 --2- 172.21.15.33:0/451938585 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fe5500b2900 0x7fe5500b4dc0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:52.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.410+0000 7fe56ce42700 1 --2- 172.21.15.33:0/451938585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe568104040 0x7fe568104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:52.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.410+0000 7fe56ce42700 1 --2- 172.21.15.33:0/451938585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5681024a0 0x7fe568106430 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:52.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.410+0000 7fe56ce42700 1 --2- 172.21.15.33:0/451938585 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5681036a0 0x7fe568104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:52.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.410+0000 7fe56ce42700 1 -- 172.21.15.33:0/451938585 >> 172.21.15.33:0/451938585 conn(0x7fe5680fda40 msgr2=0x7fe5680ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:52.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.410+0000 7fe56ce42700 1 -- 172.21.15.33:0/451938585 shutdown_connections 2024-05-15T13:59:52.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:52.410+0000 7fe56ce42700 1 -- 172.21.15.33:0/451938585 wait complete. 2024-05-15T13:59:52.419 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 23 2024-05-15T13:59:52.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:52 smithi052 ceph-mon[36190]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T13:59:52.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:52 smithi081 ceph-mon[36434]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T13:59:53.022 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-15T13:59:53.273 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:59:53.676 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:53 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/451938585' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T13:59:53.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:53 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/451938585' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T13:59:53.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:53 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/451938585' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-15T13:59:54.649 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:54 smithi033 ceph-mon[31650]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-15T13:59:54.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:54 smithi052 ceph-mon[36190]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-15T13:59:54.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:54 smithi081 ceph-mon[36434]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-15T13:59:54.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.868+0000 7f26b359d700 1 -- 172.21.15.33:0/805903736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26ac1036a0 msgr2=0x7f26ac103b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:54.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.868+0000 7f26b359d700 1 --2- 172.21.15.33:0/805903736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26ac1036a0 0x7f26ac103b00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f26a80099c0 tx=0x7f26a8009cd0 comp rx=0 tx=0).stop 2024-05-15T13:59:54.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.868+0000 7f26b359d700 1 -- 172.21.15.33:0/805903736 shutdown_connections 2024-05-15T13:59:54.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.868+0000 7f26b359d700 1 --2- 172.21.15.33:0/805903736 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26ac104040 0x7f26ac10a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:54.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.868+0000 7f26b359d700 1 --2- 172.21.15.33:0/805903736 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26ac1024a0 0x7f26ac1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:54.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.868+0000 7f26b359d700 1 --2- 172.21.15.33:0/805903736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26ac1036a0 0x7f26ac103b00 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:54.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.868+0000 7f26b359d700 1 -- 172.21.15.33:0/805903736 >> 172.21.15.33:0/805903736 conn(0x7f26ac0fda40 msgr2=0x7f26ac0ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:54.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.868+0000 7f26b359d700 1 -- 172.21.15.33:0/805903736 shutdown_connections 2024-05-15T13:59:54.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.868+0000 7f26b359d700 1 -- 172.21.15.33:0/805903736 wait complete. 2024-05-15T13:59:54.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.869+0000 7f26b359d700 1 Processor -- start 2024-05-15T13:59:54.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.869+0000 7f26b359d700 1 -- start start 2024-05-15T13:59:54.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.869+0000 7f26b359d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26ac1024a0 0x7f26ac1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:54.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.869+0000 7f26b359d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26ac1036a0 0x7f26ac198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:54.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.869+0000 7f26b359d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26ac104040 0x7f26ac19fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:54.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.869+0000 7f26b359d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f26ac10df20 con 0x7f26ac1036a0 2024-05-15T13:59:54.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.869+0000 7f26b359d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f26ac10dda0 con 0x7f26ac1024a0 2024-05-15T13:59:54.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.869+0000 7f26b359d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f26ac10e0a0 con 0x7f26ac104040 2024-05-15T13:59:54.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.870+0000 7f26b0b38700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26ac1036a0 0x7f26ac198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:54.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.870+0000 7f26b0b38700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26ac1036a0 0x7f26ac198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46470/0 (socket says 172.21.15.33:46470) 2024-05-15T13:59:54.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.870+0000 7f26b0b38700 1 -- 172.21.15.33:0/591828914 learned_addr learned my addr 172.21.15.33:0/591828914 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:54.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.870+0000 7f26b1339700 1 --2- 172.21.15.33:0/591828914 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26ac1024a0 0x7f26ac1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:54.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.870+0000 7f26b1b3a700 1 --2- 172.21.15.33:0/591828914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26ac104040 0x7f26ac19fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:54.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.870+0000 7f26b0b38700 1 -- 172.21.15.33:0/591828914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26ac104040 msgr2=0x7f26ac19fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:54.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.870+0000 7f26b0b38700 1 --2- 172.21.15.33:0/591828914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26ac104040 0x7f26ac19fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:54.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.870+0000 7f26b0b38700 1 -- 172.21.15.33:0/591828914 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26ac1024a0 msgr2=0x7f26ac1985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:54.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.870+0000 7f26b0b38700 1 --2- 172.21.15.33:0/591828914 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26ac1024a0 0x7f26ac1985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:54.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.870+0000 7f26b0b38700 1 -- 172.21.15.33:0/591828914 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26ac1a0250 con 0x7f26ac1036a0 2024-05-15T13:59:54.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.870+0000 7f26b0b38700 1 --2- 172.21.15.33:0/591828914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26ac1036a0 0x7f26ac198b30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f26a8005910 tx=0x7f26a8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:54.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.870+0000 7f269e7fc700 1 -- 172.21.15.33:0/591828914 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a8027070 con 0x7f26ac1036a0 2024-05-15T13:59:54.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.870+0000 7f269e7fc700 1 -- 172.21.15.33:0/591828914 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26a80040a0 con 0x7f26ac1036a0 2024-05-15T13:59:54.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.870+0000 7f26b359d700 1 -- 172.21.15.33:0/591828914 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26ac1a04e0 con 0x7f26ac1036a0 2024-05-15T13:59:54.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.870+0000 7f269e7fc700 1 -- 172.21.15.33:0/591828914 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a8022d10 con 0x7f26ac1036a0 2024-05-15T13:59:54.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.870+0000 7f26b359d700 1 -- 172.21.15.33:0/591828914 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26ac1a0a10 con 0x7f26ac1036a0 2024-05-15T13:59:54.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.871+0000 7f26b359d700 1 -- 172.21.15.33:0/591828914 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26ac06a230 con 0x7f26ac1036a0 2024-05-15T13:59:54.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.873+0000 7f269e7fc700 1 -- 172.21.15.33:0/591828914 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f26a802a070 con 0x7f26ac1036a0 2024-05-15T13:59:54.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.874+0000 7f269e7fc700 1 --2- 172.21.15.33:0/591828914 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f26980b2900 0x7f26980b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:54.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.874+0000 7f26b1339700 1 --2- 172.21.15.33:0/591828914 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f26980b2900 0x7f26980b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:54.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.874+0000 7f269e7fc700 1 -- 172.21.15.33:0/591828914 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f26a80e3740 con 0x7f26ac1036a0 2024-05-15T13:59:54.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.874+0000 7f26b1339700 1 --2- 172.21.15.33:0/591828914 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f26980b2900 0x7f26980b4dc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f26ac103500 tx=0x7f26a000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:54.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:54.877+0000 7f269e7fc700 1 -- 172.21.15.33:0/591828914 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f26a8031070 con 0x7f26ac1036a0 2024-05-15T13:59:55.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:55.093+0000 7f26b359d700 1 -- 172.21.15.33:0/591828914 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f26ac02e590 con 0x7f26ac1036a0 2024-05-15T13:59:55.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:55.094+0000 7f269e7fc700 1 -- 172.21.15.33:0/591828914 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v82) ==== 56+0+5275 (secure 0 0 0) 0x7f26a8013070 con 0x7f26ac1036a0 2024-05-15T13:59:55.095 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 82 2024-05-15T13:59:55.095 INFO:teuthology.orchestra.run.smithi033.stdout:fsid bca78ca4-12c1-11ef-bc98-c7b262605968 2024-05-15T13:59:55.095 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-05-15T13:49:10.327613+0000 2024-05-15T13:59:55.095 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-05-15T13:58:06.058012+0000 2024-05-15T13:59:55.096 INFO:teuthology.orchestra.run.smithi033.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-15T13:59:55.096 INFO:teuthology.orchestra.run.smithi033.stdout:crush_version 26 2024-05-15T13:59:55.096 INFO:teuthology.orchestra.run.smithi033.stdout:full_ratio 0.95 2024-05-15T13:59:55.096 INFO:teuthology.orchestra.run.smithi033.stdout:backfillfull_ratio 0.9 2024-05-15T13:59:55.096 INFO:teuthology.orchestra.run.smithi033.stdout:nearfull_ratio 0.85 2024-05-15T13:59:55.096 INFO:teuthology.orchestra.run.smithi033.stdout:require_min_compat_client luminous 2024-05-15T13:59:55.096 INFO:teuthology.orchestra.run.smithi033.stdout:min_compat_client luminous 2024-05-15T13:59:55.096 INFO:teuthology.orchestra.run.smithi033.stdout:require_osd_release squid 2024-05-15T13:59:55.096 INFO:teuthology.orchestra.run.smithi033.stdout:stretch_mode_enabled false 2024-05-15T13:59:55.096 INFO:teuthology.orchestra.run.smithi033.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-15T13:59:55.096 INFO:teuthology.orchestra.run.smithi033.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-15T13:59:55.096 INFO:teuthology.orchestra.run.smithi033.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-05-15T13:59:55.097 INFO:teuthology.orchestra.run.smithi033.stdout:max_osd 12 2024-05-15T13:59:55.097 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 up in weight 1 up_from 8 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6802/887409130,v1:172.21.15.33:6803/887409130] [v2:172.21.15.33:6804/887409130,v1:172.21.15.33:6805/887409130] exists,up 1b434111-810f-450a-b63d-c97080e9f8fc 2024-05-15T13:59:55.097 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 up in weight 1 up_from 13 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.52:6800/2794180003,v1:172.21.15.52:6801/2794180003] [v2:172.21.15.52:6802/2794180003,v1:172.21.15.52:6803/2794180003] exists,up 0b578d4e-a841-4b27-bc95-d6c3bae580fd 2024-05-15T13:59:55.097 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 up in weight 1 up_from 18 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.81:6800/315610978,v1:172.21.15.81:6801/315610978] [v2:172.21.15.81:6802/315610978,v1:172.21.15.81:6803/315610978] exists,up 0e9c9390-ed33-464b-8f39-bca75941e05e 2024-05-15T13:59:55.097 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 up in weight 1 up_from 26 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6810/818533501,v1:172.21.15.33:6811/818533501] [v2:172.21.15.33:6812/818533501,v1:172.21.15.33:6813/818533501] exists,up ecdb24c1-fb15-42a5-9bae-436bbae9512a 2024-05-15T13:59:55.097 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 up in weight 1 up_from 32 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.52:6808/3010133776,v1:172.21.15.52:6809/3010133776] [v2:172.21.15.52:6810/3010133776,v1:172.21.15.52:6811/3010133776] exists,up 706752a0-b17f-4563-a7a9-d5078bf0f3b8 2024-05-15T13:59:55.097 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 up in weight 1 up_from 38 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.81:6808/2132616297,v1:172.21.15.81:6809/2132616297] [v2:172.21.15.81:6810/2132616297,v1:172.21.15.81:6811/2132616297] exists,up 3d45ab97-f364-4903-974d-8acc961363d4 2024-05-15T13:59:55.097 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6 up in weight 1 up_from 44 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6818/1260691801,v1:172.21.15.33:6819/1260691801] [v2:172.21.15.33:6820/1260691801,v1:172.21.15.33:6821/1260691801] exists,up 3be524c5-dcf9-4742-b896-ea9a44a6b391 2024-05-15T13:59:55.097 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7 up in weight 1 up_from 49 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.52:6816/1654595813,v1:172.21.15.52:6817/1654595813] [v2:172.21.15.52:6818/1654595813,v1:172.21.15.52:6819/1654595813] exists,up bda2f920-4f2c-4eed-8127-83c1973676e8 2024-05-15T13:59:55.097 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8 up in weight 1 up_from 55 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.81:6816/3009661748,v1:172.21.15.81:6817/3009661748] [v2:172.21.15.81:6818/3009661748,v1:172.21.15.81:6819/3009661748] exists,up 550585b8-0df4-49ee-95c8-3d084744dbd4 2024-05-15T13:59:55.097 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9 up in weight 1 up_from 61 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6826/2843220858,v1:172.21.15.33:6827/2843220858] [v2:172.21.15.33:6828/2843220858,v1:172.21.15.33:6829/2843220858] exists,up 1d781846-26f3-4b23-b22c-c994a8509a71 2024-05-15T13:59:55.098 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10 up in weight 1 up_from 66 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.52:6824/2422349831,v1:172.21.15.52:6825/2422349831] [v2:172.21.15.52:6826/2422349831,v1:172.21.15.52:6827/2422349831] exists,up af58c940-fdc4-418b-a638-7dc44438b34c 2024-05-15T13:59:55.098 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11 up in weight 1 up_from 72 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.81:6824/326313090,v1:172.21.15.81:6825/326313090] [v2:172.21.15.81:6826/326313090,v1:172.21.15.81:6827/326313090] exists,up ab84aaea-2086-4842-8bf4-628842fce46a 2024-05-15T13:59:55.098 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 2.e [2,1] 2024-05-15T13:59:55.098 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 2.f [8,11] 2024-05-15T13:59:55.098 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 2.10 [2,4] 2024-05-15T13:59:55.098 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 2.1d [8,11] 2024-05-15T13:59:55.098 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.13 [4,9] 2024-05-15T13:59:55.098 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.1b [4,8] 2024-05-15T13:59:55.098 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.24 [4,9] 2024-05-15T13:59:55.098 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.2d [3,9] 2024-05-15T13:59:55.098 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.39 [5,6] 2024-05-15T13:59:55.098 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/436121571 expires 2024-05-16T13:50:04.881656+0000 2024-05-15T13:59:55.098 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:6801/2680375465 expires 2024-05-16T13:50:04.881656+0000 2024-05-15T13:59:55.098 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:6800/2680375465 expires 2024-05-16T13:50:04.881656+0000 2024-05-15T13:59:55.098 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/1022787179 expires 2024-05-16T13:49:37.564643+0000 2024-05-15T13:59:55.099 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/3720711369 expires 2024-05-16T13:50:04.881656+0000 2024-05-15T13:59:55.099 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/1282466838 expires 2024-05-16T13:49:37.564643+0000 2024-05-15T13:59:55.099 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/3022248969 expires 2024-05-16T13:49:37.564643+0000 2024-05-15T13:59:55.099 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:6801/3303495364 expires 2024-05-16T13:49:37.564643+0000 2024-05-15T13:59:55.099 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:6800/3303495364 expires 2024-05-16T13:49:37.564643+0000 2024-05-15T13:59:55.099 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/4282812692 expires 2024-05-16T13:50:04.881656+0000 2024-05-15T13:59:55.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:55.096+0000 7f26b359d700 1 -- 172.21.15.33:0/591828914 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f26980b2900 msgr2=0x7f26980b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:55.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:55.096+0000 7f26b359d700 1 --2- 172.21.15.33:0/591828914 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f26980b2900 0x7f26980b4dc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f26ac103500 tx=0x7f26a000b410 comp rx=0 tx=0).stop 2024-05-15T13:59:55.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:55.096+0000 7f26b359d700 1 -- 172.21.15.33:0/591828914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26ac1036a0 msgr2=0x7f26ac198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:55.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:55.096+0000 7f26b359d700 1 --2- 172.21.15.33:0/591828914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26ac1036a0 0x7f26ac198b30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f26a8005910 tx=0x7f26a8009580 comp rx=0 tx=0).stop 2024-05-15T13:59:55.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:55.096+0000 7f26b359d700 1 -- 172.21.15.33:0/591828914 shutdown_connections 2024-05-15T13:59:55.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:55.096+0000 7f26b359d700 1 --2- 172.21.15.33:0/591828914 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f26980b2900 0x7f26980b4dc0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:55.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:55.096+0000 7f26b359d700 1 --2- 172.21.15.33:0/591828914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26ac104040 0x7f26ac19fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:55.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:55.096+0000 7f26b359d700 1 --2- 172.21.15.33:0/591828914 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26ac1024a0 0x7f26ac1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:55.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:55.096+0000 7f26b359d700 1 --2- 172.21.15.33:0/591828914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26ac1036a0 0x7f26ac198b30 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:55.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:55.096+0000 7f26b359d700 1 -- 172.21.15.33:0/591828914 >> 172.21.15.33:0/591828914 conn(0x7f26ac0fda40 msgr2=0x7f26ac106ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:55.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:55.096+0000 7f26b359d700 1 -- 172.21.15.33:0/591828914 shutdown_connections 2024-05-15T13:59:55.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:55.096+0000 7f26b359d700 1 -- 172.21.15.33:0/591828914 wait complete. 2024-05-15T13:59:55.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:55 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/591828914' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-15T13:59:55.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:55 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/591828914' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-15T13:59:55.842 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-15T13:59:55.854 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T13:59:55.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:55 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/591828914' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-15T13:59:56.110 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T13:59:56.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:56 smithi033 ceph-mon[31650]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:59:56.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:56 smithi052 ceph-mon[36190]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:59:56.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:56 smithi081 ceph-mon[36434]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T13:59:57.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.773+0000 7f79a6a47700 1 -- 172.21.15.33:0/2021557102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a01088f0 msgr2=0x7f79a0110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:57.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.773+0000 7f79a6a47700 1 --2- 172.21.15.33:0/2021557102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a01088f0 0x7f79a0110470 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f799800b0d0 tx=0x7f799800b3e0 comp rx=0 tx=0).stop 2024-05-15T13:59:57.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.773+0000 7f79a6a47700 1 -- 172.21.15.33:0/2021557102 shutdown_connections 2024-05-15T13:59:57.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.773+0000 7f79a6a47700 1 --2- 172.21.15.33:0/2021557102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a01088f0 0x7f79a0110470 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:57.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.773+0000 7f79a6a47700 1 --2- 172.21.15.33:0/2021557102 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79a01037f0 0x7f79a0103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:57.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.773+0000 7f79a6a47700 1 --2- 172.21.15.33:0/2021557102 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79a01041a0 0x7f79a01081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:57.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.773+0000 7f79a6a47700 1 -- 172.21.15.33:0/2021557102 >> 172.21.15.33:0/2021557102 conn(0x7f79a007a6a0 msgr2=0x7f79a007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:57.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.773+0000 7f79a6a47700 1 -- 172.21.15.33:0/2021557102 shutdown_connections 2024-05-15T13:59:57.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.773+0000 7f79a6a47700 1 -- 172.21.15.33:0/2021557102 wait complete. 2024-05-15T13:59:57.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.774+0000 7f79a6a47700 1 Processor -- start 2024-05-15T13:59:57.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.774+0000 7f79a6a47700 1 -- start start 2024-05-15T13:59:57.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.774+0000 7f79a6a47700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79a01037f0 0x7f79a0075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:57.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.774+0000 7f79a6a47700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79a01041a0 0x7f79a0075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:57.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.774+0000 7f79a6a47700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a01088f0 0x7f79a00762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:57.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.774+0000 7f79a6a47700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f79a01136c0 con 0x7f79a01088f0 2024-05-15T13:59:57.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.774+0000 7f79a6a47700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f79a0113540 con 0x7f79a01037f0 2024-05-15T13:59:57.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.774+0000 7f79a6a47700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f79a0113840 con 0x7f79a01041a0 2024-05-15T13:59:57.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.774+0000 7f79a4fe4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a01088f0 0x7f79a00762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:57.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.774+0000 7f79a4fe4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a01088f0 0x7f79a00762c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46500/0 (socket says 172.21.15.33:46500) 2024-05-15T13:59:57.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.774+0000 7f79a4fe4700 1 -- 172.21.15.33:0/659035608 learned_addr learned my addr 172.21.15.33:0/659035608 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T13:59:57.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.774+0000 7f799ffff700 1 --2- 172.21.15.33:0/659035608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79a01037f0 0x7f79a0075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:57.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.774+0000 7f7997fff700 1 --2- 172.21.15.33:0/659035608 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79a01041a0 0x7f79a0075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:57.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.775+0000 7f79a4fe4700 1 -- 172.21.15.33:0/659035608 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79a01041a0 msgr2=0x7f79a0075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:57.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.775+0000 7f79a4fe4700 1 --2- 172.21.15.33:0/659035608 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79a01041a0 0x7f79a0075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:57.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.775+0000 7f79a4fe4700 1 -- 172.21.15.33:0/659035608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79a01037f0 msgr2=0x7f79a0075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:57.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.775+0000 7f79a4fe4700 1 --2- 172.21.15.33:0/659035608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79a01037f0 0x7f79a0075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:57.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.775+0000 7f79a4fe4700 1 -- 172.21.15.33:0/659035608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a01a7f40 con 0x7f79a01088f0 2024-05-15T13:59:57.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.775+0000 7f79a4fe4700 1 --2- 172.21.15.33:0/659035608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a01088f0 0x7f79a00762c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7998009fd0 tx=0x7f7998007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:57.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.775+0000 7f799dffb700 1 -- 172.21.15.33:0/659035608 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799801f070 con 0x7f79a01088f0 2024-05-15T13:59:57.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.775+0000 7f79a6a47700 1 -- 172.21.15.33:0/659035608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79a01a81d0 con 0x7f79a01088f0 2024-05-15T13:59:57.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.775+0000 7f799dffb700 1 -- 172.21.15.33:0/659035608 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7998003c00 con 0x7f79a01088f0 2024-05-15T13:59:57.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.775+0000 7f799dffb700 1 -- 172.21.15.33:0/659035608 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799801b7b0 con 0x7f79a01088f0 2024-05-15T13:59:57.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.775+0000 7f79a6a47700 1 -- 172.21.15.33:0/659035608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a01a8620 con 0x7f79a01088f0 2024-05-15T13:59:57.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.776+0000 7f79a6a47700 1 -- 172.21.15.33:0/659035608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79a006a230 con 0x7f79a01088f0 2024-05-15T13:59:57.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.778+0000 7f799dffb700 1 -- 172.21.15.33:0/659035608 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7998022070 con 0x7f79a01088f0 2024-05-15T13:59:57.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.778+0000 7f799dffb700 1 --2- 172.21.15.33:0/659035608 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f798c0b2880 0x7f798c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T13:59:57.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.778+0000 7f799ffff700 1 --2- 172.21.15.33:0/659035608 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f798c0b2880 0x7f798c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T13:59:57.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.778+0000 7f799dffb700 1 -- 172.21.15.33:0/659035608 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f799800f020 con 0x7f79a01088f0 2024-05-15T13:59:57.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.779+0000 7f799ffff700 1 --2- 172.21.15.33:0/659035608 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f798c0b2880 0x7f798c0b4d40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f799000ac50 tx=0x7f799000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T13:59:57.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:57.782+0000 7f799dffb700 1 -- 172.21.15.33:0/659035608 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7998029020 con 0x7f79a01088f0 2024-05-15T13:59:58.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:58.043+0000 7f79a6a47700 1 -- 172.21.15.33:0/659035608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79a01a8a80 con 0x7f79a01088f0 2024-05-15T13:59:58.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:58.043+0000 7f799dffb700 1 -- 172.21.15.33:0/659035608 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11858 (secure 0 0 0) 0x7f79980ab6c0 con 0x7f79a01088f0 2024-05-15T13:59:58.045 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T13:59:58.045 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2030117633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2030117633},{"type":"v1","addr":"172.21.15.33:6839","nonce":2030117633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1918979061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1918979061},{"type":"v1","addr":"172.21.15.33:6841","nonce":1918979061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T13:57:19.056417+0000","modified":"2024-05-15T13:57:38.201184+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24463,"mds_2":24461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/1629697841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1629697841},{"type":"v1","addr":"172.21.15.52:6837","nonce":1629697841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/3773839758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3773839758},{"type":"v1","addr":"172.21.15.81:6835","nonce":3773839758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.52:6839/2696044471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2696044471},{"type":"v1","addr":"172.21.15.52:6839","nonce":2696044471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.81:6839/1554395940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":1554395940},{"type":"v1","addr":"172.21.15.81:6839","nonce":1554395940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24478,24461,24463]},"id":1}]} 2024-05-15T13:59:58.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:58.045+0000 7f79a6a47700 1 -- 172.21.15.33:0/659035608 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f798c0b2880 msgr2=0x7f798c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:58.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:58.045+0000 7f79a6a47700 1 --2- 172.21.15.33:0/659035608 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f798c0b2880 0x7f798c0b4d40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f799000ac50 tx=0x7f799000a380 comp rx=0 tx=0).stop 2024-05-15T13:59:58.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:58.045+0000 7f79a6a47700 1 -- 172.21.15.33:0/659035608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a01088f0 msgr2=0x7f79a00762c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T13:59:58.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:58.045+0000 7f79a6a47700 1 --2- 172.21.15.33:0/659035608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a01088f0 0x7f79a00762c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7998009fd0 tx=0x7f7998007a20 comp rx=0 tx=0).stop 2024-05-15T13:59:58.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:58.045+0000 7f79a6a47700 1 -- 172.21.15.33:0/659035608 shutdown_connections 2024-05-15T13:59:58.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:58.045+0000 7f79a6a47700 1 --2- 172.21.15.33:0/659035608 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f798c0b2880 0x7f798c0b4d40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:58.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:58.046+0000 7f79a6a47700 1 --2- 172.21.15.33:0/659035608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a01088f0 0x7f79a00762c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:58.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:58.046+0000 7f79a6a47700 1 --2- 172.21.15.33:0/659035608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79a01037f0 0x7f79a0075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:58.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:58.046+0000 7f79a6a47700 1 --2- 172.21.15.33:0/659035608 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79a01041a0 0x7f79a0075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T13:59:58.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:58.046+0000 7f79a6a47700 1 -- 172.21.15.33:0/659035608 >> 172.21.15.33:0/659035608 conn(0x7f79a007a6a0 msgr2=0x7f79a0101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T13:59:58.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:58.046+0000 7f79a6a47700 1 -- 172.21.15.33:0/659035608 shutdown_connections 2024-05-15T13:59:58.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T13:59:58.046+0000 7f79a6a47700 1 -- 172.21.15.33:0/659035608 wait complete. 2024-05-15T13:59:58.048 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 23 2024-05-15T13:59:58.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:58 smithi033 ceph-mon[31650]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-15T13:59:58.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 13:59:58 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/659035608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:59:58.756 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T13:59:58.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:58 smithi052 ceph-mon[36190]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-15T13:59:58.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 13:59:58 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/659035608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:59:58.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:58 smithi081 ceph-mon[36434]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-15T13:59:58.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 13:59:58 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/659035608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T13:59:59.013 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:00:00.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.563+0000 7f601f5b2700 1 -- 172.21.15.33:0/459110236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6018102e20 msgr2=0x7f601810f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:00.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.563+0000 7f601f5b2700 1 --2- 172.21.15.33:0/459110236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6018102e20 0x7f601810f330 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f601000b0a0 tx=0x7f601000b3b0 comp rx=0 tx=0).stop 2024-05-15T14:00:00.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.563+0000 7f601f5b2700 1 -- 172.21.15.33:0/459110236 shutdown_connections 2024-05-15T14:00:00.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.563+0000 7f601f5b2700 1 --2- 172.21.15.33:0/459110236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6018102e20 0x7f601810f330 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:00.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.563+0000 7f601f5b2700 1 --2- 172.21.15.33:0/459110236 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60181084f0 0x7f60181088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:00.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.563+0000 7f601f5b2700 1 --2- 172.21.15.33:0/459110236 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6018102460 0x7f60181028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:00.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.563+0000 7f601f5b2700 1 -- 172.21.15.33:0/459110236 >> 172.21.15.33:0/459110236 conn(0x7f60180794e0 msgr2=0x7f60180798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:00.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.563+0000 7f601f5b2700 1 -- 172.21.15.33:0/459110236 shutdown_connections 2024-05-15T14:00:00.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.563+0000 7f601f5b2700 1 -- 172.21.15.33:0/459110236 wait complete. 2024-05-15T14:00:00.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.564+0000 7f601f5b2700 1 Processor -- start 2024-05-15T14:00:00.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.564+0000 7f601f5b2700 1 -- start start 2024-05-15T14:00:00.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.564+0000 7f601f5b2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6018102460 0x7f601819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:00.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.564+0000 7f601f5b2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6018102e20 0x7f601819afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:00.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.564+0000 7f601f5b2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60181084f0 0x7f60181a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:00.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.564+0000 7f601f5b2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6018112620 con 0x7f6018102e20 2024-05-15T14:00:00.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.564+0000 7f601f5b2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f60181124a0 con 0x7f60181084f0 2024-05-15T14:00:00.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.564+0000 7f601f5b2700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f60181127a0 con 0x7f6018102460 2024-05-15T14:00:00.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.564+0000 7f601cb4d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6018102e20 0x7f601819afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:00.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.564+0000 7f601cb4d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6018102e20 0x7f601819afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46518/0 (socket says 172.21.15.33:46518) 2024-05-15T14:00:00.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.564+0000 7f601cb4d700 1 -- 172.21.15.33:0/611189105 learned_addr learned my addr 172.21.15.33:0/611189105 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:00.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.564+0000 7f601d34e700 1 --2- 172.21.15.33:0/611189105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6018102460 0x7f601819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:00.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.564+0000 7f601db4f700 1 --2- 172.21.15.33:0/611189105 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60181084f0 0x7f60181a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:00.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.565+0000 7f601cb4d700 1 -- 172.21.15.33:0/611189105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6018102460 msgr2=0x7f601819aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:00.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.565+0000 7f601cb4d700 1 --2- 172.21.15.33:0/611189105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6018102460 0x7f601819aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:00.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.565+0000 7f601cb4d700 1 -- 172.21.15.33:0/611189105 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60181084f0 msgr2=0x7f60181a5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:00.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.565+0000 7f601cb4d700 1 --2- 172.21.15.33:0/611189105 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60181084f0 0x7f60181a5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:00.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.565+0000 7f601cb4d700 1 -- 172.21.15.33:0/611189105 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60181a6020 con 0x7f6018102e20 2024-05-15T14:00:00.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.565+0000 7f601cb4d700 1 --2- 172.21.15.33:0/611189105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6018102e20 0x7f601819afa0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f600800d950 tx=0x7f600800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:00.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.565+0000 7f600e7fc700 1 -- 172.21.15.33:0/611189105 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008016070 con 0x7f6018102e20 2024-05-15T14:00:00.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.565+0000 7f601f5b2700 1 -- 172.21.15.33:0/611189105 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60181a6250 con 0x7f6018102e20 2024-05-15T14:00:00.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.565+0000 7f600e7fc700 1 -- 172.21.15.33:0/611189105 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f600800fd40 con 0x7f6018102e20 2024-05-15T14:00:00.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.565+0000 7f600e7fc700 1 -- 172.21.15.33:0/611189105 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600800e3b0 con 0x7f6018102e20 2024-05-15T14:00:00.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.565+0000 7f601f5b2700 1 -- 172.21.15.33:0/611189105 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60181a67b0 con 0x7f6018102e20 2024-05-15T14:00:00.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.566+0000 7f601f5b2700 1 -- 172.21.15.33:0/611189105 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f601806a230 con 0x7f6018102e20 2024-05-15T14:00:00.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.567+0000 7f600e7fc700 1 -- 172.21.15.33:0/611189105 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6008019070 con 0x7f6018102e20 2024-05-15T14:00:00.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.568+0000 7f600e7fc700 1 --2- 172.21.15.33:0/611189105 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f60040b2880 0x7f60040b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:00.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.568+0000 7f601d34e700 1 --2- 172.21.15.33:0/611189105 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f60040b2880 0x7f60040b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:00.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.568+0000 7f600e7fc700 1 -- 172.21.15.33:0/611189105 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f60080e1d40 con 0x7f6018102e20 2024-05-15T14:00:00.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.569+0000 7f601d34e700 1 --2- 172.21.15.33:0/611189105 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f60040b2880 0x7f60040b4d40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f60140045b0 tx=0x7f601400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:00.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.573+0000 7f600e7fc700 1 -- 172.21.15.33:0/611189105 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f60080a9a20 con 0x7f6018102e20 2024-05-15T14:00:00.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:00 smithi033 ceph-mon[31650]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T14:00:00.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:00 smithi033 ceph-mon[31650]: overall HEALTH_OK 2024-05-15T14:00:00.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.837+0000 7f601f5b2700 1 -- 172.21.15.33:0/611189105 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f601802e560 con 0x7f6018102e20 2024-05-15T14:00:00.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.837+0000 7f600e7fc700 1 -- 172.21.15.33:0/611189105 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11858 (secure 0 0 0) 0x7f60080a9320 con 0x7f6018102e20 2024-05-15T14:00:00.839 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T14:00:00.839 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2030117633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2030117633},{"type":"v1","addr":"172.21.15.33:6839","nonce":2030117633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1918979061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1918979061},{"type":"v1","addr":"172.21.15.33:6841","nonce":1918979061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T13:57:19.056417+0000","modified":"2024-05-15T13:57:38.201184+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24463,"mds_2":24461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/1629697841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1629697841},{"type":"v1","addr":"172.21.15.52:6837","nonce":1629697841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/3773839758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3773839758},{"type":"v1","addr":"172.21.15.81:6835","nonce":3773839758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.52:6839/2696044471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2696044471},{"type":"v1","addr":"172.21.15.52:6839","nonce":2696044471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.81:6839/1554395940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":1554395940},{"type":"v1","addr":"172.21.15.81:6839","nonce":1554395940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24478,24461,24463]},"id":1}]} 2024-05-15T14:00:00.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:00 smithi052 ceph-mon[36190]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T14:00:00.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:00 smithi052 ceph-mon[36190]: overall HEALTH_OK 2024-05-15T14:00:00.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.840+0000 7f601f5b2700 1 -- 172.21.15.33:0/611189105 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f60040b2880 msgr2=0x7f60040b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:00.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.840+0000 7f601f5b2700 1 --2- 172.21.15.33:0/611189105 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f60040b2880 0x7f60040b4d40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f60140045b0 tx=0x7f601400b3c0 comp rx=0 tx=0).stop 2024-05-15T14:00:00.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.840+0000 7f601f5b2700 1 -- 172.21.15.33:0/611189105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6018102e20 msgr2=0x7f601819afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:00.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.840+0000 7f601f5b2700 1 --2- 172.21.15.33:0/611189105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6018102e20 0x7f601819afa0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f600800d950 tx=0x7f600800dc60 comp rx=0 tx=0).stop 2024-05-15T14:00:00.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.840+0000 7f601f5b2700 1 -- 172.21.15.33:0/611189105 shutdown_connections 2024-05-15T14:00:00.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.840+0000 7f601f5b2700 1 --2- 172.21.15.33:0/611189105 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f60040b2880 0x7f60040b4d40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:00.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.840+0000 7f601f5b2700 1 --2- 172.21.15.33:0/611189105 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f60181084f0 0x7f60181a5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:00.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.840+0000 7f601f5b2700 1 --2- 172.21.15.33:0/611189105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6018102460 0x7f601819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:00.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.840+0000 7f601f5b2700 1 --2- 172.21.15.33:0/611189105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6018102e20 0x7f601819afa0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:00.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.840+0000 7f601f5b2700 1 -- 172.21.15.33:0/611189105 >> 172.21.15.33:0/611189105 conn(0x7f60180794e0 msgr2=0x7f60181050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:00.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.840+0000 7f601f5b2700 1 -- 172.21.15.33:0/611189105 shutdown_connections 2024-05-15T14:00:00.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:00.840+0000 7f601f5b2700 1 -- 172.21.15.33:0/611189105 wait complete. 2024-05-15T14:00:00.842 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 23 2024-05-15T14:00:00.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:00 smithi081 ceph-mon[36434]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T14:00:00.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:00 smithi081 ceph-mon[36434]: overall HEALTH_OK 2024-05-15T14:00:01.542 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-15T14:00:01.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:01 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/611189105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:00:01.793 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:00:01.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:01 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/611189105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:00:01.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:01 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/611189105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:00:02.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:02 smithi033 ceph-mon[31650]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T14:00:02.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:02 smithi052 ceph-mon[36190]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T14:00:02.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:02 smithi081 ceph-mon[36434]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T14:00:03.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.305+0000 7f88a6c37700 1 -- 172.21.15.33:0/2650541142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88a006b220 msgr2=0x7f88a006b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:03.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.305+0000 7f88a6c37700 1 --2- 172.21.15.33:0/2650541142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88a006b220 0x7f88a006b6a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8888009990 tx=0x7f8888009ca0 comp rx=0 tx=0).stop 2024-05-15T14:00:03.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.305+0000 7f88a6c37700 1 -- 172.21.15.33:0/2650541142 shutdown_connections 2024-05-15T14:00:03.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.305+0000 7f88a6c37700 1 --2- 172.21.15.33:0/2650541142 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88a006bbe0 0x7f88a01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:03.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.305+0000 7f88a6c37700 1 --2- 172.21.15.33:0/2650541142 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88a01095b0 0x7f88a0109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:03.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.305+0000 7f88a6c37700 1 --2- 172.21.15.33:0/2650541142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88a006b220 0x7f88a006b6a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:03.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.305+0000 7f88a6c37700 1 -- 172.21.15.33:0/2650541142 >> 172.21.15.33:0/2650541142 conn(0x7f88a00793f0 msgr2=0x7f88a0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:03.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.305+0000 7f88a6c37700 1 -- 172.21.15.33:0/2650541142 shutdown_connections 2024-05-15T14:00:03.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.305+0000 7f88a6c37700 1 -- 172.21.15.33:0/2650541142 wait complete. 2024-05-15T14:00:03.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.306+0000 7f88a6c37700 1 Processor -- start 2024-05-15T14:00:03.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.306+0000 7f88a6c37700 1 -- start start 2024-05-15T14:00:03.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.306+0000 7f88a6c37700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88a006b220 0x7f88a019f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:03.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.306+0000 7f88a6c37700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88a006bbe0 0x7f88a019fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:03.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.306+0000 7f88a6c37700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88a01095b0 0x7f88a0199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:03.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.306+0000 7f88a6c37700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f88a01136c0 con 0x7f88a006b220 2024-05-15T14:00:03.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.306+0000 7f88a6c37700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f88a0113540 con 0x7f88a006bbe0 2024-05-15T14:00:03.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.306+0000 7f88a6c37700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f88a0113840 con 0x7f88a01095b0 2024-05-15T14:00:03.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.306+0000 7f88a49d3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88a006b220 0x7f88a019f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:03.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.306+0000 7f88a49d3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88a006b220 0x7f88a019f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51844/0 (socket says 172.21.15.33:51844) 2024-05-15T14:00:03.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.306+0000 7f88a49d3700 1 -- 172.21.15.33:0/602246270 learned_addr learned my addr 172.21.15.33:0/602246270 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:03.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.306+0000 7f889ffff700 1 --2- 172.21.15.33:0/602246270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88a006bbe0 0x7f88a019fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:03.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.306+0000 7f88a51d4700 1 --2- 172.21.15.33:0/602246270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88a01095b0 0x7f88a0199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:03.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.307+0000 7f88a49d3700 1 -- 172.21.15.33:0/602246270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88a01095b0 msgr2=0x7f88a0199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:03.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.307+0000 7f88a49d3700 1 --2- 172.21.15.33:0/602246270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88a01095b0 0x7f88a0199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:03.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.307+0000 7f88a49d3700 1 -- 172.21.15.33:0/602246270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88a006bbe0 msgr2=0x7f88a019fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:03.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.307+0000 7f88a49d3700 1 --2- 172.21.15.33:0/602246270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88a006bbe0 0x7f88a019fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:03.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.307+0000 7f88a49d3700 1 -- 172.21.15.33:0/602246270 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a019a1a0 con 0x7f88a006b220 2024-05-15T14:00:03.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.307+0000 7f88a49d3700 1 --2- 172.21.15.33:0/602246270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88a006b220 0x7f88a019f880 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f889000d980 tx=0x7f889000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:03.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.307+0000 7f889dffb700 1 -- 172.21.15.33:0/602246270 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8890016070 con 0x7f88a006b220 2024-05-15T14:00:03.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.307+0000 7f889dffb700 1 -- 172.21.15.33:0/602246270 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8890004830 con 0x7f88a006b220 2024-05-15T14:00:03.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.307+0000 7f889dffb700 1 -- 172.21.15.33:0/602246270 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8890005540 con 0x7f88a006b220 2024-05-15T14:00:03.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.307+0000 7f88a6c37700 1 -- 172.21.15.33:0/602246270 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88a019a490 con 0x7f88a006b220 2024-05-15T14:00:03.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.307+0000 7f88a6c37700 1 -- 172.21.15.33:0/602246270 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88a01a6750 con 0x7f88a006b220 2024-05-15T14:00:03.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.308+0000 7f88a6c37700 1 -- 172.21.15.33:0/602246270 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88a006a230 con 0x7f88a006b220 2024-05-15T14:00:03.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.309+0000 7f889dffb700 1 -- 172.21.15.33:0/602246270 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8890004220 con 0x7f88a006b220 2024-05-15T14:00:03.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.310+0000 7f889dffb700 1 --2- 172.21.15.33:0/602246270 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f888c0b2880 0x7f888c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:03.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.310+0000 7f889ffff700 1 --2- 172.21.15.33:0/602246270 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f888c0b2880 0x7f888c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:03.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.310+0000 7f889dffb700 1 -- 172.21.15.33:0/602246270 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f88900e1f40 con 0x7f88a006b220 2024-05-15T14:00:03.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.311+0000 7f889ffff700 1 --2- 172.21.15.33:0/602246270 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f888c0b2880 0x7f888c0b4d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f88a0079ad0 tx=0x7f888800b560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:03.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.315+0000 7f889dffb700 1 -- 172.21.15.33:0/602246270 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f889001f020 con 0x7f88a006b220 2024-05-15T14:00:03.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.532+0000 7f88a6c37700 1 -- 172.21.15.33:0/602246270 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f88a002e560 con 0x7f88a006b220 2024-05-15T14:00:03.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.534+0000 7f889dffb700 1 -- 172.21.15.33:0/602246270 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22593 (secure 0 0 0) 0x7f88900a9c60 con 0x7f88a006b220 2024-05-15T14:00:03.535 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T14:00:03.535 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":82,"fsid":"bca78ca4-12c1-11ef-bc98-c7b262605968","created":"2024-05-15T13:49:10.327613+0000","modified":"2024-05-15T13:58:06.058012+0000","last_up_change":"2024-05-15T13:56:16.097711+0000","last_in_change":"2024-05-15T13:55:56.722143+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-15T13:52:32.970844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-15T13:57:12.324802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-15T13:57:15.898406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b434111-810f-450a-b63d-c97080e9f8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6803","nonce":887409130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6805","nonce":887409130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6809","nonce":887409130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6807","nonce":887409130}]},"public_addr":"172.21.15.33:6803/887409130","cluster_addr":"172.21.15.33:6805/887409130","heartbeat_back_addr":"172.21.15.33:6809/887409130","heartbeat_front_addr":"172.21.15.33:6807/887409130","state":["exists","up"]},{"osd":1,"uuid":"0b578d4e-a841-4b27-bc95-d6c3bae580fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6801","nonce":2794180003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6803","nonce":2794180003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6807","nonce":2794180003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6805","nonce":2794180003}]},"public_addr":"172.21.15.52:6801/2794180003","cluster_addr":"172.21.15.52:6803/2794180003","heartbeat_back_addr":"172.21.15.52:6807/2794180003","heartbeat_front_addr":"172.21.15.52:6805/2794180003","state":["exists","up"]},{"osd":2,"uuid":"0e9c9390-ed33-464b-8f39-bca75941e05e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6801","nonce":315610978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6803","nonce":315610978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6807","nonce":315610978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6805","nonce":315610978}]},"public_addr":"172.21.15.81:6801/315610978","cluster_addr":"172.21.15.81:6803/315610978","heartbeat_back_addr":"172.21.15.81:6807/315610978","heartbeat_front_addr":"172.21.15.81:6805/315610978","state":["exists","up"]},{"osd":3,"uuid":"ecdb24c1-fb15-42a5-9bae-436bbae9512a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6811","nonce":818533501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6813","nonce":818533501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6817","nonce":818533501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6815","nonce":818533501}]},"public_addr":"172.21.15.33:6811/818533501","cluster_addr":"172.21.15.33:6813/818533501","heartbeat_back_addr":"172.21.15.33:6817/818533501","heartbeat_front_addr":"172.21.15.33:6815/818533501","state":["exists","up"]},{"osd":4,"uuid":"706752a0-b17f-4563-a7a9-d5078bf0f3b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6809","nonce":3010133776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6811","nonce":3010133776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6815","nonce":3010133776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6813","nonce":3010133776}]},"public_addr":"172.21.15.52:6809/3010133776","cluster_addr":"172.21.15.52:6811/3010133776","heartbeat_back_addr":"172.21.15.52:6815/3010133776","heartbeat_front_addr":"172.21.15.52:6813/3010133776","state":["exists","up"]},{"osd":5,"uuid":"3d45ab97-f364-4903-974d-8acc961363d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6809","nonce":2132616297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6811","nonce":2132616297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6815","nonce":2132616297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6813","nonce":2132616297}]},"public_addr":"172.21.15.81:6809/2132616297","cluster_addr":"172.21.15.81:6811/2132616297","heartbeat_back_addr":"172.21.15.81:6815/2132616297","heartbeat_front_addr":"172.21.15.81:6813/2132616297","state":["exists","up"]},{"osd":6,"uuid":"3be524c5-dcf9-4742-b896-ea9a44a6b391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6819","nonce":1260691801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6821","nonce":1260691801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6825","nonce":1260691801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6823","nonce":1260691801}]},"public_addr":"172.21.15.33:6819/1260691801","cluster_addr":"172.21.15.33:6821/1260691801","heartbeat_back_addr":"172.21.15.33:6825/1260691801","heartbeat_front_addr":"172.21.15.33:6823/1260691801","state":["exists","up"]},{"osd":7,"uuid":"bda2f920-4f2c-4eed-8127-83c1973676e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6817","nonce":1654595813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6819","nonce":1654595813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6823","nonce":1654595813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6821","nonce":1654595813}]},"public_addr":"172.21.15.52:6817/1654595813","cluster_addr":"172.21.15.52:6819/1654595813","heartbeat_back_addr":"172.21.15.52:6823/1654595813","heartbeat_front_addr":"172.21.15.52:6821/1654595813","state":["exists","up"]},{"osd":8,"uuid":"550585b8-0df4-49ee-95c8-3d084744dbd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6817","nonce":3009661748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6819","nonce":3009661748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6823","nonce":3009661748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6821","nonce":3009661748}]},"public_addr":"172.21.15.81:6817/3009661748","cluster_addr":"172.21.15.81:6819/3009661748","heartbeat_back_addr":"172.21.15.81:6823/3009661748","heartbeat_front_addr":"172.21.15.81:6821/3009661748","state":["exists","up"]},{"osd":9,"uuid":"1d781846-26f3-4b23-b22c-c994a8509a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6827","nonce":2843220858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6829","nonce":2843220858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6833","nonce":2843220858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6831","nonce":2843220858}]},"public_addr":"172.21.15.33:6827/2843220858","cluster_addr":"172.21.15.33:6829/2843220858","heartbeat_back_addr":"172.21.15.33:6833/2843220858","heartbeat_front_addr":"172.21.15.33:6831/2843220858","state":["exists","up"]},{"osd":10,"uuid":"af58c940-fdc4-418b-a638-7dc44438b34c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6825","nonce":2422349831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6827","nonce":2422349831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6831","nonce":2422349831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6829","nonce":2422349831}]},"public_addr":"172.21.15.52:6825/2422349831","cluster_addr":"172.21.15.52:6827/2422349831","heartbeat_back_addr":"172.21.15.52:6831/2422349831","heartbeat_front_addr":"172.21.15.52:6829/2422349831","state":["exists","up"]},{"osd":11,"uuid":"ab84aaea-2086-4842-8bf4-628842fce46a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6825","nonce":326313090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6827","nonce":326313090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6831","nonce":326313090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6829","nonce":326313090}]},"public_addr":"172.21.15.81:6825/326313090","cluster_addr":"172.21.15.81:6827/326313090","heartbeat_back_addr":"172.21.15.81:6831/326313090","heartbeat_front_addr":"172.21.15.81:6829/326313090","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:51:41.493136+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:52:05.784773+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:52:29.769429+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:52:54.891620+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:53:19.412922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:53:43.921841+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:54:09.486732+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:54:34.064370+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:54:58.655304+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:55:24.548693+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:55:49.941191+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:56:14.919558+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/436121571":"2024-05-16T13:50:04.881656+0000","172.21.15.33:6801/2680375465":"2024-05-16T13:50:04.881656+0000","172.21.15.33:6800/2680375465":"2024-05-16T13:50:04.881656+0000","172.21.15.33:0/1022787179":"2024-05-16T13:49:37.564643+0000","172.21.15.33:0/3720711369":"2024-05-16T13:50:04.881656+0000","172.21.15.33:0/1282466838":"2024-05-16T13:49:37.564643+0000","172.21.15.33:0/3022248969":"2024-05-16T13:49:37.564643+0000","172.21.15.33:6801/3303495364":"2024-05-16T13:49:37.564643+0000","172.21.15.33:6800/3303495364":"2024-05-16T13:49:37.564643+0000","172.21.15.33:0/4282812692":"2024-05-16T13:50:04.881656+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-15T14:00:03.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.536+0000 7f88a6c37700 1 -- 172.21.15.33:0/602246270 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f888c0b2880 msgr2=0x7f888c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:03.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.536+0000 7f88a6c37700 1 --2- 172.21.15.33:0/602246270 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f888c0b2880 0x7f888c0b4d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f88a0079ad0 tx=0x7f888800b560 comp rx=0 tx=0).stop 2024-05-15T14:00:03.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.536+0000 7f88a6c37700 1 -- 172.21.15.33:0/602246270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88a006b220 msgr2=0x7f88a019f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:03.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.536+0000 7f88a6c37700 1 --2- 172.21.15.33:0/602246270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88a006b220 0x7f88a019f880 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f889000d980 tx=0x7f889000dc90 comp rx=0 tx=0).stop 2024-05-15T14:00:03.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.536+0000 7f88a6c37700 1 -- 172.21.15.33:0/602246270 shutdown_connections 2024-05-15T14:00:03.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.536+0000 7f88a6c37700 1 --2- 172.21.15.33:0/602246270 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f888c0b2880 0x7f888c0b4d40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:03.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.536+0000 7f88a6c37700 1 --2- 172.21.15.33:0/602246270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88a01095b0 0x7f88a0199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:03.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.536+0000 7f88a6c37700 1 --2- 172.21.15.33:0/602246270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88a006b220 0x7f88a019f880 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:03.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.536+0000 7f88a6c37700 1 --2- 172.21.15.33:0/602246270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88a006bbe0 0x7f88a019fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:03.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.536+0000 7f88a6c37700 1 -- 172.21.15.33:0/602246270 >> 172.21.15.33:0/602246270 conn(0x7f88a00793f0 msgr2=0x7f88a0101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:03.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.536+0000 7f88a6c37700 1 -- 172.21.15.33:0/602246270 shutdown_connections 2024-05-15T14:00:03.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:03.536+0000 7f88a6c37700 1 -- 172.21.15.33:0/602246270 wait complete. 2024-05-15T14:00:04.155 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-15T13:57:19.056417+0000', 'modified': '2024-05-15T13:57:38.201184+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24478, 'mds_1': 24463, 'mds_2': 24461}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24424': {'gid': 24424, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.81:6833/1872861431', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6832', 'nonce': 1872861431}, {'type': 'v1', 'addr': '172.21.15.81:6833', 'nonce': 1872861431}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.52:6837/1629697841', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6836', 'nonce': 1629697841}, {'type': 'v1', 'addr': '172.21.15.52:6837', 'nonce': 1629697841}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.81:6835/3773839758', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6834', 'nonce': 3773839758}, {'type': 'v1', 'addr': '172.21.15.81:6835', 'nonce': 3773839758}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24461': {'gid': 24461, 'name': 'b', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.52:6839/2696044471', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6838', 'nonce': 2696044471}, {'type': 'v1', 'addr': '172.21.15.52:6839', 'nonce': 2696044471}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'c', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.81:6837/744100239', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6836', 'nonce': 744100239}, {'type': 'v1', 'addr': '172.21.15.81:6837', 'nonce': 744100239}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.81:6839/1554395940', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6838', 'nonce': 1554395940}, {'type': 'v1', 'addr': '172.21.15.81:6839', 'nonce': 1554395940}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24478, 'qdb_cluster': [24478, 24461, 24463]} 2024-05-15T14:00:04.156 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-15T14:00:04.156 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph tell mds.b status 2024-05-15T14:00:04.413 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:00:04.655 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:04 smithi033 ceph-mon[31650]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-15T14:00:04.655 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:04 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/602246270' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T14:00:04.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:04 smithi052 ceph-mon[36190]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-15T14:00:04.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:04 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/602246270' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T14:00:04.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:04 smithi081 ceph-mon[36434]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-15T14:00:04.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:04 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/602246270' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T14:00:06.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.137+0000 7fa9217bd700 1 -- 172.21.15.33:0/1313995549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa91c102810 msgr2=0x7fa91c106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:06.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.137+0000 7fa9217bd700 1 --2- 172.21.15.33:0/1313995549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa91c102810 0x7fa91c106810 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa9100099c0 tx=0x7fa910009cd0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.137+0000 7fa9217bd700 1 -- 172.21.15.33:0/1313995549 shutdown_connections 2024-05-15T14:00:06.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.137+0000 7fa9217bd700 1 --2- 172.21.15.33:0/1313995549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa91c106ef0 0x7fa91c10eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.137+0000 7fa9217bd700 1 --2- 172.21.15.33:0/1313995549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa91c101e60 0x7fa91c102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.137+0000 7fa9217bd700 1 --2- 172.21.15.33:0/1313995549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa91c102810 0x7fa91c106810 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.137+0000 7fa9217bd700 1 -- 172.21.15.33:0/1313995549 >> 172.21.15.33:0/1313995549 conn(0x7fa91c078ab0 msgr2=0x7fa91c0ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:06.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.137+0000 7fa9217bd700 1 -- 172.21.15.33:0/1313995549 shutdown_connections 2024-05-15T14:00:06.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.137+0000 7fa9217bd700 1 -- 172.21.15.33:0/1313995549 wait complete. 2024-05-15T14:00:06.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.138+0000 7fa9217bd700 1 Processor -- start 2024-05-15T14:00:06.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.138+0000 7fa9217bd700 1 -- start start 2024-05-15T14:00:06.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.138+0000 7fa9217bd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa91c101e60 0x7fa91c19f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:06.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.138+0000 7fa9217bd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa91c102810 0x7fa91c19fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:06.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.138+0000 7fa9217bd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa91c106ef0 0x7fa91c199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:06.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.138+0000 7fa9217bd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c111df0 con 0x7fa91c102810 2024-05-15T14:00:06.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.138+0000 7fa9217bd700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c111c70 con 0x7fa91c101e60 2024-05-15T14:00:06.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.138+0000 7fa9217bd700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c111f70 con 0x7fa91c106ef0 2024-05-15T14:00:06.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.138+0000 7fa91a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa91c102810 0x7fa91c19fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:06.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.139+0000 7fa91a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa91c102810 0x7fa91c19fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51866/0 (socket says 172.21.15.33:51866) 2024-05-15T14:00:06.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.139+0000 7fa91a7fc700 1 -- 172.21.15.33:0/464197426 learned_addr learned my addr 172.21.15.33:0/464197426 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:06.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.139+0000 7fa91affd700 1 --2- 172.21.15.33:0/464197426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa91c101e60 0x7fa91c19f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:06.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.139+0000 7fa91b7fe700 1 --2- 172.21.15.33:0/464197426 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa91c106ef0 0x7fa91c199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:06.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.139+0000 7fa91a7fc700 1 -- 172.21.15.33:0/464197426 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa91c106ef0 msgr2=0x7fa91c199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:06.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.139+0000 7fa91a7fc700 1 --2- 172.21.15.33:0/464197426 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa91c106ef0 0x7fa91c199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.139+0000 7fa91a7fc700 1 -- 172.21.15.33:0/464197426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa91c101e60 msgr2=0x7fa91c19f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:06.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.139+0000 7fa91a7fc700 1 --2- 172.21.15.33:0/464197426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa91c101e60 0x7fa91c19f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.139+0000 7fa91a7fc700 1 -- 172.21.15.33:0/464197426 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91c19a1e0 con 0x7fa91c102810 2024-05-15T14:00:06.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.139+0000 7fa91a7fc700 1 --2- 172.21.15.33:0/464197426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa91c102810 0x7fa91c19fe00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa910009f50 tx=0x7fa9100037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:06.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.139+0000 7fa903fff700 1 -- 172.21.15.33:0/464197426 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910027070 con 0x7fa91c102810 2024-05-15T14:00:06.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.139+0000 7fa9217bd700 1 -- 172.21.15.33:0/464197426 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa91c19a470 con 0x7fa91c102810 2024-05-15T14:00:06.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.139+0000 7fa903fff700 1 -- 172.21.15.33:0/464197426 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa910004140 con 0x7fa91c102810 2024-05-15T14:00:06.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.139+0000 7fa903fff700 1 -- 172.21.15.33:0/464197426 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91000f460 con 0x7fa91c102810 2024-05-15T14:00:06.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.140+0000 7fa9217bd700 1 -- 172.21.15.33:0/464197426 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa91c1a6790 con 0x7fa91c102810 2024-05-15T14:00:06.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.142+0000 7fa903fff700 1 -- 172.21.15.33:0/464197426 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa91002a070 con 0x7fa91c102810 2024-05-15T14:00:06.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.143+0000 7fa903fff700 1 --2- 172.21.15.33:0/464197426 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fa9040b2880 0x7fa9040b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:06.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.143+0000 7fa91affd700 1 --2- 172.21.15.33:0/464197426 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fa9040b2880 0x7fa9040b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:06.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.144+0000 7fa903fff700 1 -- 172.21.15.33:0/464197426 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fa9100e3150 con 0x7fa91c102810 2024-05-15T14:00:06.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.144+0000 7fa92c053b80 1 Processor -- start 2024-05-15T14:00:06.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.144+0000 7fa92c053b80 1 -- start start 2024-05-15T14:00:06.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.144+0000 7fa91affd700 1 --2- 172.21.15.33:0/464197426 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fa9040b2880 0x7fa9040b4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa91c19abc0 tx=0x7fa90c0094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:06.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.144+0000 7fa92c053b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617682d32f0 0x5617682d36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:06.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.144+0000 7fa92c053b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617682d3c10 0x5617682d73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:06.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.144+0000 7fa92c053b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617682d7910 0x5617682d7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:06.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.144+0000 7fa92c053b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561768231c40 con 0x5617682d7910 2024-05-15T14:00:06.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.144+0000 7fa92c053b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5617682217e0 con 0x5617682d3c10 2024-05-15T14:00:06.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.144+0000 7fa92c053b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x561768225820 con 0x5617682d32f0 2024-05-15T14:00:06.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.144+0000 7fa91b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617682d7910 0x5617682d7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:06.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.144+0000 7fa91b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617682d7910 0x5617682d7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51876/0 (socket says 172.21.15.33:51876) 2024-05-15T14:00:06.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.144+0000 7fa91b7fe700 1 -- 172.21.15.33:0/35504058 learned_addr learned my addr 172.21.15.33:0/35504058 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:06.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.144+0000 7fa91affd700 1 --2- 172.21.15.33:0/35504058 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617682d32f0 0x5617682d36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:06.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.144+0000 7fa91a7fc700 1 --2- 172.21.15.33:0/35504058 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617682d3c10 0x5617682d73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:06.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.145+0000 7fa91b7fe700 1 -- 172.21.15.33:0/35504058 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617682d32f0 msgr2=0x5617682d36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:06.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.145+0000 7fa91b7fe700 1 --2- 172.21.15.33:0/35504058 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617682d32f0 0x5617682d36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.145+0000 7fa91b7fe700 1 -- 172.21.15.33:0/35504058 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617682d3c10 msgr2=0x5617682d73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:06.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.145+0000 7fa91b7fe700 1 --2- 172.21.15.33:0/35504058 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617682d3c10 0x5617682d73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.145+0000 7fa91b7fe700 1 -- 172.21.15.33:0/35504058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561768171880 con 0x5617682d7910 2024-05-15T14:00:06.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.145+0000 7fa91b7fe700 1 --2- 172.21.15.33:0/35504058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617682d7910 0x5617682d7d50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa91400cbe0 tx=0x7fa91400cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd62a1beb74a4daa server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:06.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.145+0000 7fa9017fa700 1 -- 172.21.15.33:0/35504058 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91400ed10 con 0x5617682d7910 2024-05-15T14:00:06.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.145+0000 7fa9017fa700 1 -- 172.21.15.33:0/35504058 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9140044d0 con 0x5617682d7910 2024-05-15T14:00:06.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.145+0000 7fa9017fa700 1 -- 172.21.15.33:0/35504058 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa914010650 con 0x5617682d7910 2024-05-15T14:00:06.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.146+0000 7fa92c053b80 1 -- 172.21.15.33:0/35504058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617682d7910 msgr2=0x5617682d7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:06.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.146+0000 7fa92c053b80 1 --2- 172.21.15.33:0/35504058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617682d7910 0x5617682d7d50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa91400cbe0 tx=0x7fa91400cef0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.146+0000 7fa92c053b80 1 -- 172.21.15.33:0/35504058 shutdown_connections 2024-05-15T14:00:06.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.146+0000 7fa92c053b80 1 --2- 172.21.15.33:0/35504058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617682d7910 0x5617682d7d50 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.146+0000 7fa92c053b80 1 --2- 172.21.15.33:0/35504058 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617682d32f0 0x5617682d36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.146+0000 7fa92c053b80 1 --2- 172.21.15.33:0/35504058 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617682d3c10 0x5617682d73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.146+0000 7fa92c053b80 1 -- 172.21.15.33:0/35504058 >> 172.21.15.33:0/35504058 conn(0x5617681fe200 msgr2=0x5617682cd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:06.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.146+0000 7fa92c053b80 1 -- 172.21.15.33:0/35504058 shutdown_connections 2024-05-15T14:00:06.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.146+0000 7fa92c053b80 1 -- 172.21.15.33:0/35504058 wait complete. 2024-05-15T14:00:06.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.146+0000 7fa92c053b80 1 -- start start 2024-05-15T14:00:06.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.146+0000 7fa92c053b80 1 Processor -- start 2024-05-15T14:00:06.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.147+0000 7fa92c053b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617682d32f0 0x5617682ded00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:06.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.147+0000 7fa92c053b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617682d3c10 0x5617682dc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:06.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.147+0000 7fa92c053b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617682d7910 0x5617682dc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:06.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.147+0000 7fa92c053b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561768127f50 con 0x5617682d3c10 2024-05-15T14:00:06.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.147+0000 7fa92c053b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5617681c3770 con 0x5617682d32f0 2024-05-15T14:00:06.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.147+0000 7fa92c053b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5617682dfb70 con 0x5617682d7910 2024-05-15T14:00:06.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.147+0000 7fa92c053b80 10 client.0 _ll_register_callbacks cb 0x5617681b8530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:00:06.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.147+0000 7fa91a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617682d3c10 0x5617682dc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:06.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.147+0000 7fa91a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617682d3c10 0x5617682dc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51890/0 (socket says 172.21.15.33:51890) 2024-05-15T14:00:06.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.147+0000 7fa91a7fc700 1 -- 172.21.15.33:0/2817972484 learned_addr learned my addr 172.21.15.33:0/2817972484 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:06.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.147+0000 7fa91affd700 1 --2- 172.21.15.33:0/2817972484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617682d32f0 0x5617682ded00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:06.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.147+0000 7fa91b7fe700 1 --2- 172.21.15.33:0/2817972484 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617682d7910 0x5617682dc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:06.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.147+0000 7fa91a7fc700 1 -- 172.21.15.33:0/2817972484 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617682d7910 msgr2=0x5617682dc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:06.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.147+0000 7fa91a7fc700 1 --2- 172.21.15.33:0/2817972484 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617682d7910 0x5617682dc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.147+0000 7fa91a7fc700 1 -- 172.21.15.33:0/2817972484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617682d32f0 msgr2=0x5617682ded00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:06.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.147+0000 7fa91a7fc700 1 --2- 172.21.15.33:0/2817972484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617682d32f0 0x5617682ded00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.147+0000 7fa91a7fc700 1 -- 172.21.15.33:0/2817972484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617681c0640 con 0x5617682d3c10 2024-05-15T14:00:06.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.147+0000 7fa91a7fc700 1 --2- 172.21.15.33:0/2817972484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617682d3c10 0x5617682dc0a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa9100a7a70 tx=0x7fa9100228e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7147e45688a200f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:06.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.147+0000 7fa8feff5700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:00:06.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.147+0000 7fa8feff5700 1 -- 172.21.15.33:0/2817972484 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9100cd0b0 con 0x5617682d3c10 2024-05-15T14:00:06.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.147+0000 7fa8feff5700 1 -- 172.21.15.33:0/2817972484 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fa9100b3140 con 0x5617682d3c10 2024-05-15T14:00:06.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.147+0000 7fa92c053b80 1 -- 172.21.15.33:0/2817972484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9140044d0 con 0x5617682d3c10 2024-05-15T14:00:06.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.148+0000 7fa8feff5700 1 -- 172.21.15.33:0/2817972484 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fa9100aa310 con 0x5617682d3c10 2024-05-15T14:00:06.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.148+0000 7fa92c053b80 10 client.15078 fetch_fsmap learned FSMap version 23 2024-05-15T14:00:06.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.148+0000 7fa92c053b80 1 -- 172.21.15.33:0/2817972484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5617682ea1b0 con 0x5617682d3c10 2024-05-15T14:00:06.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.148+0000 7fa8feff5700 1 -- 172.21.15.33:0/2817972484 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9140044d0 con 0x5617682d3c10 2024-05-15T14:00:06.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.148+0000 7fa8feff5700 1 -- 172.21.15.33:0/2817972484 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fa910070080 con 0x5617682d3c10 2024-05-15T14:00:06.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.148+0000 7fa92c053b80 10 client.15078 fetch_fsmap finished waiting for FSMap version 23 2024-05-15T14:00:06.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.148+0000 7fa92c053b80 10 client.15078 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-15T14:00:06.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.148+0000 7fa92c053b80 20 client.15078 populate_metadata read hostname 'smithi033' 2024-05-15T14:00:06.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.148+0000 7fa92c053b80 1 --2- 172.21.15.33:0/2817972484 >> [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] conn(0x5617682ec820 0x5617682edc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:06.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.148+0000 7fa92c053b80 4 client.15078 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:00:06.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.148+0000 7fa92c053b80 1 -- 172.21.15.33:0/2817972484 --> [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56176829f310 con 0x5617682ec820 2024-05-15T14:00:06.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.148+0000 7fa91affd700 1 --2- 172.21.15.33:0/2817972484 >> [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] conn(0x5617682ec820 0x5617682edc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:06.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.149+0000 7fa91affd700 1 --2- 172.21.15.33:0/2817972484 >> [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] conn(0x5617682ec820 0x5617682edc90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e50d0df0e3ebd61a server_cookie=27d318b588aab11d in_seq=0 out_seq=0 2024-05-15T14:00:06.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.149+0000 7fa8feff5700 10 client.15078 ms_handle_connect on v2:172.21.15.52:6838/2696044471 2024-05-15T14:00:06.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.150+0000 7fa8feff5700 1 -- 172.21.15.33:0/2817972484 <== mds.2 v2:172.21.15.52:6838/2696044471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56176829f310 con 0x5617682ec820 2024-05-15T14:00:06.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.150+0000 7fa8feff5700 10 client.15078 handle_command_reply: tid=0 2024-05-15T14:00:06.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.150+0000 7fa92c053b80 1 client.15078 shutdown 2024-05-15T14:00:06.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.150+0000 7fa92c053b80 1 -- 172.21.15.33:0/2817972484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617682d3c10 msgr2=0x5617682dc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:06.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.150+0000 7fa92c053b80 1 --2- 172.21.15.33:0/2817972484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617682d3c10 0x5617682dc0a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa9100a7a70 tx=0x7fa9100228e0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.150+0000 7fa92c053b80 1 -- 172.21.15.33:0/2817972484 shutdown_connections 2024-05-15T14:00:06.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.150+0000 7fa92c053b80 1 --2- 172.21.15.33:0/2817972484 >> [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] conn(0x5617682ec820 0x5617682edc90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.150+0000 7fa92c053b80 1 --2- 172.21.15.33:0/2817972484 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617682d7910 0x5617682dc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.150+0000 7fa92c053b80 1 --2- 172.21.15.33:0/2817972484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617682d32f0 0x5617682ded00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.150+0000 7fa92c053b80 1 --2- 172.21.15.33:0/2817972484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617682d3c10 0x5617682dc0a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.150+0000 7fa92c053b80 1 -- 172.21.15.33:0/2817972484 >> 172.21.15.33:0/2817972484 conn(0x5617681fe200 msgr2=0x5617682d8de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:06.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.150+0000 7fa8feff5700 0 client.15078 ms_handle_reset on v2:172.21.15.52:6838/2696044471 2024-05-15T14:00:06.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.150+0000 7fa92c053b80 1 -- 172.21.15.33:0/2817972484 shutdown_connections 2024-05-15T14:00:06.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.150+0000 7fa92c053b80 1 -- 172.21.15.33:0/2817972484 wait complete. 2024-05-15T14:00:06.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.150+0000 7fa92c053b80 20 client.15078 trim_cache size 0 max 16384 2024-05-15T14:00:06.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.171+0000 7fa92c053b80 1 Processor -- start 2024-05-15T14:00:06.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.171+0000 7fa92c053b80 1 -- start start 2024-05-15T14:00:06.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.172+0000 7fa92c053b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617682d32f0 0x5617682c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:06.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.172+0000 7fa92c053b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617682d3c10 0x5617682d5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:06.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.172+0000 7fa92c053b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617682d7910 0x5617682d5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:06.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.172+0000 7fa92c053b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5617682ebfd0 con 0x5617682d32f0 2024-05-15T14:00:06.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.172+0000 7fa92c053b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5617682ec6a0 con 0x5617682d7910 2024-05-15T14:00:06.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.172+0000 7fa92c053b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5617682ecd70 con 0x5617682d3c10 2024-05-15T14:00:06.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.172+0000 7fa91affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617682d32f0 0x5617682c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:06.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.172+0000 7fa91affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617682d32f0 0x5617682c7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51900/0 (socket says 172.21.15.33:51900) 2024-05-15T14:00:06.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.172+0000 7fa91affd700 1 -- 172.21.15.33:0/3098265174 learned_addr learned my addr 172.21.15.33:0/3098265174 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:06.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.172+0000 7fa91affd700 1 -- 172.21.15.33:0/3098265174 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617682d3c10 msgr2=0x5617682d5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:06.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.172+0000 7fa91affd700 1 --2- 172.21.15.33:0/3098265174 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617682d3c10 0x5617682d5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.172+0000 7fa91affd700 1 -- 172.21.15.33:0/3098265174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617682d7910 msgr2=0x5617682d5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:06.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.172+0000 7fa91affd700 1 --2- 172.21.15.33:0/3098265174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617682d7910 0x5617682d5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.172+0000 7fa91affd700 1 -- 172.21.15.33:0/3098265174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617682da610 con 0x5617682d32f0 2024-05-15T14:00:06.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.172+0000 7fa91affd700 1 --2- 172.21.15.33:0/3098265174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617682d32f0 0x5617682c7c40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0175e0 tx=0x7fa90c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be01835532a7874e server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:06.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.172+0000 7fa900ff9700 1 -- 172.21.15.33:0/3098265174 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c017e30 con 0x5617682d32f0 2024-05-15T14:00:06.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.172+0000 7fa900ff9700 1 -- 172.21.15.33:0/3098265174 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa90c02a680 con 0x5617682d32f0 2024-05-15T14:00:06.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.172+0000 7fa900ff9700 1 -- 172.21.15.33:0/3098265174 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c033ad0 con 0x5617682d32f0 2024-05-15T14:00:06.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.173+0000 7fa92c053b80 1 -- 172.21.15.33:0/3098265174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617682d32f0 msgr2=0x5617682c7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:06.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.173+0000 7fa92c053b80 1 --2- 172.21.15.33:0/3098265174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617682d32f0 0x5617682c7c40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0175e0 tx=0x7fa90c015000 comp rx=0 tx=0).stop 2024-05-15T14:00:06.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.173+0000 7fa92c053b80 1 -- 172.21.15.33:0/3098265174 shutdown_connections 2024-05-15T14:00:06.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.173+0000 7fa92c053b80 1 --2- 172.21.15.33:0/3098265174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617682d7910 0x5617682d5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.173+0000 7fa92c053b80 1 --2- 172.21.15.33:0/3098265174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617682d32f0 0x5617682c7c40 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.173+0000 7fa92c053b80 1 --2- 172.21.15.33:0/3098265174 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617682d3c10 0x5617682d5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.173+0000 7fa92c053b80 1 -- 172.21.15.33:0/3098265174 >> 172.21.15.33:0/3098265174 conn(0x5617681fe200 msgr2=0x5617682c7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:06.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.173+0000 7fa92c053b80 1 -- 172.21.15.33:0/3098265174 shutdown_connections 2024-05-15T14:00:06.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.173+0000 7fa92c053b80 1 -- 172.21.15.33:0/3098265174 wait complete. 2024-05-15T14:00:06.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.173+0000 7fa92c053b80 1 -- start start 2024-05-15T14:00:06.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.173+0000 7fa92c053b80 1 Processor -- start 2024-05-15T14:00:06.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.174+0000 7fa92c053b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617682d32f0 0x5617682f0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:06.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.174+0000 7fa92c053b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617682d3c10 0x5617682f07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:06.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.174+0000 7fa92c053b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617682d7910 0x5617682f5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:06.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.174+0000 7fa92c053b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5617682edb10 con 0x5617682d7910 2024-05-15T14:00:06.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.174+0000 7fa92c053b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5617682ed440 con 0x5617682d3c10 2024-05-15T14:00:06.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.174+0000 7fa92c053b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5617682ee1e0 con 0x5617682d32f0 2024-05-15T14:00:06.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.174+0000 7fa92c053b80 10 client.0 _ll_register_callbacks cb 0x5617681b8530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:00:06.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.174+0000 7fa91b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617682d7910 0x5617682f5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:06.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.174+0000 7fa91b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617682d7910 0x5617682f5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51906/0 (socket says 172.21.15.33:51906) 2024-05-15T14:00:06.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.174+0000 7fa91b7fe700 1 -- 172.21.15.33:0/1148281068 learned_addr learned my addr 172.21.15.33:0/1148281068 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:06.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.174+0000 7fa91affd700 1 --2- 172.21.15.33:0/1148281068 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617682d32f0 0x5617682f0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:06.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.174+0000 7fa91a7fc700 1 --2- 172.21.15.33:0/1148281068 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617682d3c10 0x5617682f07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:06.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.174+0000 7fa91b7fe700 1 -- 172.21.15.33:0/1148281068 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617682d32f0 msgr2=0x5617682f0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:06.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.174+0000 7fa91b7fe700 1 --2- 172.21.15.33:0/1148281068 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617682d32f0 0x5617682f0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.174+0000 7fa91b7fe700 1 -- 172.21.15.33:0/1148281068 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617682d3c10 msgr2=0x5617682f07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:06.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.174+0000 7fa91b7fe700 1 --2- 172.21.15.33:0/1148281068 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617682d3c10 0x5617682f07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.174+0000 7fa91b7fe700 1 -- 172.21.15.33:0/1148281068 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617682f5890 con 0x5617682d7910 2024-05-15T14:00:06.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.174+0000 7fa91b7fe700 1 --2- 172.21.15.33:0/1148281068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617682d7910 0x5617682f5250 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa91400cbb0 tx=0x7fa914005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b4236e2033b4a01 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:06.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.174+0000 7fa8feff5700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:00:06.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.175+0000 7fa8feff5700 1 -- 172.21.15.33:0/1148281068 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91402e060 con 0x5617682d7910 2024-05-15T14:00:06.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.175+0000 7fa8feff5700 1 -- 172.21.15.33:0/1148281068 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7fa91401e820 con 0x5617682d7910 2024-05-15T14:00:06.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.175+0000 7fa92c053b80 1 -- 172.21.15.33:0/1148281068 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa90c02a680 con 0x5617682d7910 2024-05-15T14:00:06.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.175+0000 7fa8feff5700 1 -- 172.21.15.33:0/1148281068 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fa91401ea40 con 0x5617682d7910 2024-05-15T14:00:06.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.175+0000 7fa92c053b80 10 client.15084 fetch_fsmap learned FSMap version 23 2024-05-15T14:00:06.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.175+0000 7fa92c053b80 1 -- 172.21.15.33:0/1148281068 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5617682f6b90 con 0x5617682d7910 2024-05-15T14:00:06.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.175+0000 7fa8feff5700 1 -- 172.21.15.33:0/1148281068 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa90c02a680 con 0x5617682d7910 2024-05-15T14:00:06.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.175+0000 7fa8feff5700 1 -- 172.21.15.33:0/1148281068 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fa914033070 con 0x5617682d7910 2024-05-15T14:00:06.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.175+0000 7fa92c053b80 10 client.15084 fetch_fsmap finished waiting for FSMap version 23 2024-05-15T14:00:06.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.175+0000 7fa92c053b80 10 client.15084 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-15T14:00:06.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.175+0000 7fa92c053b80 20 client.15084 populate_metadata read hostname 'smithi033' 2024-05-15T14:00:06.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.175+0000 7fa92c053b80 1 --2- 172.21.15.33:0/1148281068 >> [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] conn(0x56176835d320 0x56176835c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:06.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.175+0000 7fa92c053b80 4 client.15084 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-15T14:00:06.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.175+0000 7fa92c053b80 1 -- 172.21.15.33:0/1148281068 --> [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] -- command(tid 0: {"prefix": "status"}) -- 0x561768233b90 con 0x56176835d320 2024-05-15T14:00:06.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.175+0000 7fa91affd700 1 --2- 172.21.15.33:0/1148281068 >> [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] conn(0x56176835d320 0x56176835c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:06.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.176+0000 7fa8feff5700 10 client.15084 ms_handle_connect on v2:172.21.15.52:6838/2696044471 2024-05-15T14:00:06.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.176+0000 7fa91affd700 1 --2- 172.21.15.33:0/1148281068 >> [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] conn(0x56176835d320 0x56176835c030 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1ad24ac4dd0cc91c server_cookie=cf1788c967d00ba9 in_seq=0 out_seq=0 2024-05-15T14:00:06.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.176+0000 7fa8feff5700 1 -- 172.21.15.33:0/1148281068 <== mds.2 v2:172.21.15.52:6838/2696044471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x561768233b90 con 0x56176835d320 2024-05-15T14:00:06.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.176+0000 7fa8feff5700 10 client.15084 handle_command_reply: tid=0 2024-05-15T14:00:06.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.176+0000 7fa92c053b80 1 client.15084 shutdown 2024-05-15T14:00:06.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.176+0000 7fa92c053b80 1 -- 172.21.15.33:0/1148281068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617682d7910 msgr2=0x5617682f5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:06.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.176+0000 7fa92c053b80 1 --2- 172.21.15.33:0/1148281068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617682d7910 0x5617682f5250 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa91400cbb0 tx=0x7fa914005f60 comp rx=0 tx=0).stop 2024-05-15T14:00:06.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.176+0000 7fa92c053b80 1 -- 172.21.15.33:0/1148281068 shutdown_connections 2024-05-15T14:00:06.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.176+0000 7fa92c053b80 1 --2- 172.21.15.33:0/1148281068 >> [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] conn(0x56176835d320 0x56176835c030 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.176+0000 7fa92c053b80 1 --2- 172.21.15.33:0/1148281068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5617682d7910 0x5617682f5250 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.176+0000 7fa92c053b80 1 --2- 172.21.15.33:0/1148281068 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5617682d32f0 0x5617682f0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.176+0000 7fa8feff5700 0 client.15084 ms_handle_reset on v2:172.21.15.52:6838/2696044471 2024-05-15T14:00:06.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.176+0000 7fa92c053b80 1 --2- 172.21.15.33:0/1148281068 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617682d3c10 0x5617682f07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.176+0000 7fa92c053b80 1 -- 172.21.15.33:0/1148281068 >> 172.21.15.33:0/1148281068 conn(0x5617681fe200 msgr2=0x5617682d2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:06.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.176+0000 7fa92c053b80 1 -- 172.21.15.33:0/1148281068 shutdown_connections 2024-05-15T14:00:06.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.176+0000 7fa92c053b80 1 -- 172.21.15.33:0/1148281068 wait complete. 2024-05-15T14:00:06.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.176+0000 7fa92c053b80 20 client.15084 trim_cache size 0 max 16384 2024-05-15T14:00:06.183 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T14:00:06.183 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "bca78ca4-12c1-11ef-bc98-c7b262605968", 2024-05-15T14:00:06.184 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 2, 2024-05-15T14:00:06.184 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24461, 2024-05-15T14:00:06.184 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-05-15T14:00:06.184 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-05-15T14:00:06.184 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-05-15T14:00:06.184 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 153.69276092600001, 2024-05-15T14:00:06.184 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 23, 2024-05-15T14:00:06.184 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 82, 2024-05-15T14:00:06.184 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 80, 2024-05-15T14:00:06.184 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 182.384423355 2024-05-15T14:00:06.184 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T14:00:06.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.177+0000 7fa9217bd700 1 -- 172.21.15.33:0/464197426 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fa9040b2880 msgr2=0x7fa9040b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:06.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.177+0000 7fa9217bd700 1 --2- 172.21.15.33:0/464197426 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fa9040b2880 0x7fa9040b4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa91c19abc0 tx=0x7fa90c0094f0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.177+0000 7fa9217bd700 1 -- 172.21.15.33:0/464197426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa91c102810 msgr2=0x7fa91c19fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:06.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.177+0000 7fa9217bd700 1 --2- 172.21.15.33:0/464197426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa91c102810 0x7fa91c19fe00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa910009f50 tx=0x7fa9100037c0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.177+0000 7fa9217bd700 1 -- 172.21.15.33:0/464197426 shutdown_connections 2024-05-15T14:00:06.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.177+0000 7fa9217bd700 1 --2- 172.21.15.33:0/464197426 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7fa9040b2880 0x7fa9040b4d40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.177+0000 7fa9217bd700 1 --2- 172.21.15.33:0/464197426 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa91c106ef0 0x7fa91c199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.177+0000 7fa9217bd700 1 --2- 172.21.15.33:0/464197426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa91c101e60 0x7fa91c19f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.177+0000 7fa9217bd700 1 --2- 172.21.15.33:0/464197426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa91c102810 0x7fa91c19fe00 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:06.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.177+0000 7fa9217bd700 1 -- 172.21.15.33:0/464197426 >> 172.21.15.33:0/464197426 conn(0x7fa91c078ab0 msgr2=0x7fa91c100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:06.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.177+0000 7fa9217bd700 1 -- 172.21.15.33:0/464197426 shutdown_connections 2024-05-15T14:00:06.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:06.177+0000 7fa9217bd700 1 -- 172.21.15.33:0/464197426 wait complete. 2024-05-15T14:00:06.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:06 smithi033 ceph-mon[31650]: pgmap v372: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T14:00:06.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:06 smithi052 ceph-mon[36190]: pgmap v372: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T14:00:06.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:06 smithi081 ceph-mon[36434]: pgmap v372: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T14:00:06.865 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph tell mds.c status 2024-05-15T14:00:07.118 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:00:08.533 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:08 smithi033 ceph-mon[31650]: pgmap v373: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-15T14:00:08.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.748+0000 7f7e8c3a6700 1 -- 172.21.15.33:0/2539750498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e84109650 msgr2=0x7f7e84109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:08.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.748+0000 7f7e8c3a6700 1 --2- 172.21.15.33:0/2539750498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e84109650 0x7f7e84109a30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7e800099c0 tx=0x7f7e80009cd0 comp rx=0 tx=0).stop 2024-05-15T14:00:08.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.748+0000 7f7e8c3a6700 1 -- 172.21.15.33:0/2539750498 shutdown_connections 2024-05-15T14:00:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.748+0000 7f7e8c3a6700 1 --2- 172.21.15.33:0/2539750498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e8406c630 0x7f7e84110420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.748+0000 7f7e8c3a6700 1 --2- 172.21.15.33:0/2539750498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e84109650 0x7f7e84109a30 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.748+0000 7f7e8c3a6700 1 --2- 172.21.15.33:0/2539750498 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e8406bc70 0x7f7e8406c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.748+0000 7f7e8c3a6700 1 -- 172.21.15.33:0/2539750498 >> 172.21.15.33:0/2539750498 conn(0x7f7e840782b0 msgr2=0x7f7e840786c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.748+0000 7f7e8c3a6700 1 -- 172.21.15.33:0/2539750498 shutdown_connections 2024-05-15T14:00:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.748+0000 7f7e8c3a6700 1 -- 172.21.15.33:0/2539750498 wait complete. 2024-05-15T14:00:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.748+0000 7f7e8c3a6700 1 Processor -- start 2024-05-15T14:00:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.749+0000 7f7e8c3a6700 1 -- start start 2024-05-15T14:00:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.749+0000 7f7e8c3a6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e8406bc70 0x7f7e8419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.749+0000 7f7e8c3a6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e8406c630 0x7f7e8419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.749+0000 7f7e8c3a6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e84109650 0x7f7e841999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.749+0000 7f7e8c3a6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7e84113760 con 0x7f7e84109650 2024-05-15T14:00:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.749+0000 7f7e8c3a6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7e841135e0 con 0x7f7e8406c630 2024-05-15T14:00:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.749+0000 7f7e8c3a6700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f7e841138e0 con 0x7f7e8406bc70 2024-05-15T14:00:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.749+0000 7f7e8a943700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e84109650 0x7f7e841999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.749+0000 7f7e8a943700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e84109650 0x7f7e841999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51924/0 (socket says 172.21.15.33:51924) 2024-05-15T14:00:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.749+0000 7f7e8a943700 1 -- 172.21.15.33:0/3919319685 learned_addr learned my addr 172.21.15.33:0/3919319685 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.749+0000 7f7e8a142700 1 --2- 172.21.15.33:0/3919319685 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e8406bc70 0x7f7e8419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.749+0000 7f7e89941700 1 --2- 172.21.15.33:0/3919319685 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e8406c630 0x7f7e8419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.749+0000 7f7e8a943700 1 -- 172.21.15.33:0/3919319685 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e8406bc70 msgr2=0x7f7e8419f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.749+0000 7f7e8a943700 1 --2- 172.21.15.33:0/3919319685 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e8406bc70 0x7f7e8419f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.749+0000 7f7e8a943700 1 -- 172.21.15.33:0/3919319685 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e8406c630 msgr2=0x7f7e8419fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.749+0000 7f7e8a943700 1 --2- 172.21.15.33:0/3919319685 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e8406c630 0x7f7e8419fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.749+0000 7f7e8a943700 1 -- 172.21.15.33:0/3919319685 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8419a100 con 0x7f7e84109650 2024-05-15T14:00:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.749+0000 7f7e8a943700 1 --2- 172.21.15.33:0/3919319685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e84109650 0x7f7e841999a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c007ee0 tx=0x7f7e7c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.750+0000 7f7e7b7fe700 1 -- 172.21.15.33:0/3919319685 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c004bb0 con 0x7f7e84109650 2024-05-15T14:00:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.750+0000 7f7e7b7fe700 1 -- 172.21.15.33:0/3919319685 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e7c004d50 con 0x7f7e84109650 2024-05-15T14:00:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.750+0000 7f7e8c3a6700 1 -- 172.21.15.33:0/3919319685 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e8419a3f0 con 0x7f7e84109650 2024-05-15T14:00:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.750+0000 7f7e7b7fe700 1 -- 172.21.15.33:0/3919319685 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c005920 con 0x7f7e84109650 2024-05-15T14:00:08.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.750+0000 7f7e8c3a6700 1 -- 172.21.15.33:0/3919319685 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e841a66b0 con 0x7f7e84109650 2024-05-15T14:00:08.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.753+0000 7f7e7b7fe700 1 -- 172.21.15.33:0/3919319685 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7e7c01a020 con 0x7f7e84109650 2024-05-15T14:00:08.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.754+0000 7f7e96c3cb80 1 Processor -- start 2024-05-15T14:00:08.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.754+0000 7f7e96c3cb80 1 -- start start 2024-05-15T14:00:08.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.754+0000 7f7e7b7fe700 1 --2- 172.21.15.33:0/3919319685 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f7e700b2900 0x7f7e700b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:08.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.754+0000 7f7e89941700 1 --2- 172.21.15.33:0/3919319685 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f7e700b2900 0x7f7e700b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:08.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.754+0000 7f7e96c3cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2100cffa0 0x55e2100d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.754+0000 7f7e96c3cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2100d08c0 0x55e2100ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.754+0000 7f7e96c3cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2100cadc0 0x55e2100cb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.754+0000 7f7e7b7fe700 1 -- 172.21.15.33:0/3919319685 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f7e7c0e1800 con 0x7f7e84109650 2024-05-15T14:00:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.754+0000 7f7e96c3cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e210028d20 con 0x55e2100cadc0 2024-05-15T14:00:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.754+0000 7f7e96c3cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e2100188c0 con 0x55e2100cffa0 2024-05-15T14:00:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.754+0000 7f7e96c3cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55e21001c900 con 0x55e2100d08c0 2024-05-15T14:00:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.755+0000 7f7e89941700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2100d08c0 0x55e2100ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.755+0000 7f7e89941700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2100d08c0 0x55e2100ca7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:40022/0 (socket says 172.21.15.33:40022) 2024-05-15T14:00:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.755+0000 7f7e89941700 1 -- 172.21.15.33:0/1385816491 learned_addr learned my addr 172.21.15.33:0/1385816491 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.755+0000 7f7e8a142700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2100cffa0 0x55e2100d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.755+0000 7f7e8a943700 1 --2- 172.21.15.33:0/1385816491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2100cadc0 0x55e2100cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.755+0000 7f7e8a943700 1 -- 172.21.15.33:0/1385816491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2100d08c0 msgr2=0x55e2100ca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:08.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.755+0000 7f7e8a943700 1 --2- 172.21.15.33:0/1385816491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2100d08c0 0x55e2100ca7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:08.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.755+0000 7f7e8a943700 1 -- 172.21.15.33:0/1385816491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2100cffa0 msgr2=0x55e2100d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:08.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.755+0000 7f7e8a943700 1 --2- 172.21.15.33:0/1385816491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2100cffa0 0x55e2100d0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:08.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.755+0000 7f7e8a943700 1 -- 172.21.15.33:0/1385816491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e20ffc1f60 con 0x55e2100cadc0 2024-05-15T14:00:08.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.755+0000 7f7e89941700 1 --2- 172.21.15.33:0/1385816491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2100d08c0 0x55e2100ca7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:00:08.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.755+0000 7f7e8a142700 1 --2- 172.21.15.33:0/1385816491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2100cffa0 0x55e2100d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:00:08.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.755+0000 7f7e89941700 1 --2- 172.21.15.33:0/3919319685 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f7e700b2900 0x7f7e700b4dc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7e74013650 tx=0x7f7e74013a10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:08.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.755+0000 7f7e8a943700 1 --2- 172.21.15.33:0/1385816491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2100cadc0 0x55e2100cb220 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c0a6050 tx=0x7f7e7c0a87b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d5e46e8fbbeccf server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:08.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.755+0000 7f7e78d70700 1 -- 172.21.15.33:0/1385816491 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c0a88f0 con 0x55e2100cadc0 2024-05-15T14:00:08.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.755+0000 7f7e78d70700 1 -- 172.21.15.33:0/1385816491 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e7c0a94e0 con 0x55e2100cadc0 2024-05-15T14:00:08.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.755+0000 7f7e78d70700 1 -- 172.21.15.33:0/1385816491 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c0a9040 con 0x55e2100cadc0 2024-05-15T14:00:08.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.756+0000 7f7e96c3cb80 1 -- 172.21.15.33:0/1385816491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2100cadc0 msgr2=0x55e2100cb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:08.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.756+0000 7f7e96c3cb80 1 --2- 172.21.15.33:0/1385816491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2100cadc0 0x55e2100cb220 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c0a6050 tx=0x7f7e7c0a87b0 comp rx=0 tx=0).stop 2024-05-15T14:00:08.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.756+0000 7f7e96c3cb80 1 -- 172.21.15.33:0/1385816491 shutdown_connections 2024-05-15T14:00:08.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.756+0000 7f7e96c3cb80 1 --2- 172.21.15.33:0/1385816491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2100cadc0 0x55e2100cb220 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:08.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.756+0000 7f7e96c3cb80 1 --2- 172.21.15.33:0/1385816491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2100cffa0 0x55e2100d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:08.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.756+0000 7f7e96c3cb80 1 --2- 172.21.15.33:0/1385816491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2100d08c0 0x55e2100ca7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:08.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.756+0000 7f7e96c3cb80 1 -- 172.21.15.33:0/1385816491 >> 172.21.15.33:0/1385816491 conn(0x55e20fff7d60 msgr2=0x55e2100c4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:08.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.757+0000 7f7e96c3cb80 1 -- 172.21.15.33:0/1385816491 shutdown_connections 2024-05-15T14:00:08.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.757+0000 7f7e96c3cb80 1 -- 172.21.15.33:0/1385816491 wait complete. 2024-05-15T14:00:08.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.757+0000 7f7e96c3cb80 1 -- start start 2024-05-15T14:00:08.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.757+0000 7f7e96c3cb80 1 Processor -- start 2024-05-15T14:00:08.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.757+0000 7f7e96c3cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2100cadc0 0x55e2100d65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:08.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.757+0000 7f7e96c3cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2100cffa0 0x55e2100d3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:08.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.757+0000 7f7e96c3cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2100d08c0 0x55e2100d3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:08.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.757+0000 7f7e96c3cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e20ff1ef50 con 0x55e2100d08c0 2024-05-15T14:00:08.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.757+0000 7f7e96c3cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e20ff80d50 con 0x55e2100cffa0 2024-05-15T14:00:08.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.757+0000 7f7e96c3cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55e2100d72d0 con 0x55e2100cadc0 2024-05-15T14:00:08.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.757+0000 7f7e96c3cb80 10 client.0 _ll_register_callbacks cb 0x55e20ffb3900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:00:08.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.757+0000 7f7e8a943700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2100d08c0 0x55e2100d3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:08.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.757+0000 7f7e8a943700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2100d08c0 0x55e2100d3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51946/0 (socket says 172.21.15.33:51946) 2024-05-15T14:00:08.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.757+0000 7f7e8a943700 1 -- 172.21.15.33:0/2925680574 learned_addr learned my addr 172.21.15.33:0/2925680574 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:08.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.758+0000 7f7e8a142700 1 --2- 172.21.15.33:0/2925680574 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2100cadc0 0x55e2100d65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:08.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.758+0000 7f7e89941700 1 --2- 172.21.15.33:0/2925680574 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2100cffa0 0x55e2100d3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:08.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.758+0000 7f7e8a943700 1 -- 172.21.15.33:0/2925680574 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2100cadc0 msgr2=0x55e2100d65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:08.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.758+0000 7f7e8a943700 1 --2- 172.21.15.33:0/2925680574 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2100cadc0 0x55e2100d65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:08.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.758+0000 7f7e8a943700 1 -- 172.21.15.33:0/2925680574 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2100cffa0 msgr2=0x55e2100d3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:08.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.758+0000 7f7e8a943700 1 --2- 172.21.15.33:0/2925680574 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2100cffa0 0x55e2100d3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:08.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.758+0000 7f7e8a943700 1 -- 172.21.15.33:0/2925680574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e20ff68880 con 0x55e2100d08c0 2024-05-15T14:00:08.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.758+0000 7f7e8a943700 1 --2- 172.21.15.33:0/2925680574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2100d08c0 0x55e2100d3ed0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c0a6050 tx=0x7f7e7c0a9320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3b1beffe654454 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:08.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.758+0000 7f7e6dffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:00:08.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.758+0000 7f7e6dffb700 1 -- 172.21.15.33:0/2925680574 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c0c7020 con 0x55e2100d08c0 2024-05-15T14:00:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.758+0000 7f7e6dffb700 1 -- 172.21.15.33:0/2925680574 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f7e7c0b19c0 con 0x55e2100d08c0 2024-05-15T14:00:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.758+0000 7f7e96c3cb80 1 -- 172.21.15.33:0/2925680574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e7c0a94e0 con 0x55e2100d08c0 2024-05-15T14:00:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.758+0000 7f7e6dffb700 1 -- 172.21.15.33:0/2925680574 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f7e7c0a8de0 con 0x55e2100d08c0 2024-05-15T14:00:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.758+0000 7f7e96c3cb80 10 client.15096 fetch_fsmap learned FSMap version 23 2024-05-15T14:00:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.758+0000 7f7e96c3cb80 1 -- 172.21.15.33:0/2925680574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55e20ffd2710 con 0x55e2100d08c0 2024-05-15T14:00:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.758+0000 7f7e6dffb700 1 -- 172.21.15.33:0/2925680574 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e7c0a94e0 con 0x55e2100d08c0 2024-05-15T14:00:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.759+0000 7f7e6dffb700 1 -- 172.21.15.33:0/2925680574 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f7e7c0a4750 con 0x55e2100d08c0 2024-05-15T14:00:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.759+0000 7f7e96c3cb80 10 client.15096 fetch_fsmap finished waiting for FSMap version 23 2024-05-15T14:00:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.759+0000 7f7e96c3cb80 10 client.15096 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-15T14:00:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.759+0000 7f7e96c3cb80 20 client.15096 populate_metadata read hostname 'smithi033' 2024-05-15T14:00:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.759+0000 7f7e96c3cb80 1 --2- 172.21.15.33:0/2925680574 >> [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] conn(0x55e2100e2b60 0x55e2100e4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.759+0000 7f7e96c3cb80 4 client.15096 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:00:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.759+0000 7f7e96c3cb80 1 -- 172.21.15.33:0/2925680574 --> [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e21002ac70 con 0x55e2100e2b60 2024-05-15T14:00:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.759+0000 7f7e8a142700 1 --2- 172.21.15.33:0/2925680574 >> [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] conn(0x55e2100e2b60 0x55e2100e4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.760+0000 7f7e8a142700 1 --2- 172.21.15.33:0/2925680574 >> [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] conn(0x55e2100e2b60 0x55e2100e4d60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b16d065c13db2c server_cookie=89339d409265ce6c in_seq=0 out_seq=0 2024-05-15T14:00:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.760+0000 7f7e6dffb700 10 client.15096 ms_handle_connect on v2:172.21.15.81:6836/744100239 2024-05-15T14:00:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.760+0000 7f7e6dffb700 1 -- 172.21.15.33:0/2925680574 <== mds.1 v2:172.21.15.81:6836/744100239 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e21002ac70 con 0x55e2100e2b60 2024-05-15T14:00:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.760+0000 7f7e6dffb700 10 client.15096 handle_command_reply: tid=0 2024-05-15T14:00:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.761+0000 7f7e96c3cb80 1 client.15096 shutdown 2024-05-15T14:00:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.761+0000 7f7e96c3cb80 1 -- 172.21.15.33:0/2925680574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2100d08c0 msgr2=0x55e2100d3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.761+0000 7f7e96c3cb80 1 --2- 172.21.15.33:0/2925680574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2100d08c0 0x55e2100d3ed0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c0a6050 tx=0x7f7e7c0a9320 comp rx=0 tx=0).stop 2024-05-15T14:00:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.761+0000 7f7e96c3cb80 1 -- 172.21.15.33:0/2925680574 shutdown_connections 2024-05-15T14:00:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.761+0000 7f7e96c3cb80 1 --2- 172.21.15.33:0/2925680574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2100d08c0 0x55e2100d3ed0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.761+0000 7f7e96c3cb80 1 --2- 172.21.15.33:0/2925680574 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2100cadc0 0x55e2100d65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.761+0000 7f7e96c3cb80 1 --2- 172.21.15.33:0/2925680574 >> [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] conn(0x55e2100e2b60 0x55e2100e4d60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.761+0000 7f7e96c3cb80 1 --2- 172.21.15.33:0/2925680574 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2100cffa0 0x55e2100d3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.761+0000 7f7e96c3cb80 1 -- 172.21.15.33:0/2925680574 >> 172.21.15.33:0/2925680574 conn(0x55e20fff7d60 msgr2=0x55e2100c4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.761+0000 7f7e6dffb700 0 client.15096 ms_handle_reset on v2:172.21.15.81:6836/744100239 2024-05-15T14:00:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.761+0000 7f7e96c3cb80 1 -- 172.21.15.33:0/2925680574 shutdown_connections 2024-05-15T14:00:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.761+0000 7f7e96c3cb80 1 -- 172.21.15.33:0/2925680574 wait complete. 2024-05-15T14:00:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.761+0000 7f7e96c3cb80 20 client.15096 trim_cache size 0 max 16384 2024-05-15T14:00:08.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.782+0000 7f7e96c3cb80 1 Processor -- start 2024-05-15T14:00:08.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.782+0000 7f7e96c3cb80 1 -- start start 2024-05-15T14:00:08.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.782+0000 7f7e96c3cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2100cadc0 0x55e2100ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:08.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.782+0000 7f7e96c3cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2100cffa0 0x55e2100d4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:08.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.782+0000 7f7e96c3cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2100d08c0 0x55e2100d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:08.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.782+0000 7f7e96c3cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e2100e30a0 con 0x55e2100d08c0 2024-05-15T14:00:08.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.782+0000 7f7e96c3cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e2100e3770 con 0x55e2100cadc0 2024-05-15T14:00:08.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.782+0000 7f7e96c3cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55e2100e3e40 con 0x55e2100cffa0 2024-05-15T14:00:08.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.782+0000 7f7e8a943700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2100d08c0 0x55e2100d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:08.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.782+0000 7f7e8a943700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2100d08c0 0x55e2100d13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51962/0 (socket says 172.21.15.33:51962) 2024-05-15T14:00:08.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.782+0000 7f7e8a943700 1 -- 172.21.15.33:0/908256575 learned_addr learned my addr 172.21.15.33:0/908256575 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:08.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.782+0000 7f7e8a142700 1 --2- 172.21.15.33:0/908256575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2100cadc0 0x55e2100ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:08.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.782+0000 7f7e89941700 1 --2- 172.21.15.33:0/908256575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2100cffa0 0x55e2100d4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:08.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.783+0000 7f7e8a943700 1 -- 172.21.15.33:0/908256575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2100cffa0 msgr2=0x55e2100d4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:08.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.783+0000 7f7e8a943700 1 --2- 172.21.15.33:0/908256575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2100cffa0 0x55e2100d4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:08.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.783+0000 7f7e8a943700 1 -- 172.21.15.33:0/908256575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2100cadc0 msgr2=0x55e2100ca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:08.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.783+0000 7f7e8a943700 1 --2- 172.21.15.33:0/908256575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2100cadc0 0x55e2100ca7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:08.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.783+0000 7f7e8a943700 1 -- 172.21.15.33:0/908256575 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2100e3590 con 0x55e2100d08c0 2024-05-15T14:00:08.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.783+0000 7f7e89941700 1 --2- 172.21.15.33:0/908256575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2100cffa0 0x55e2100d4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:00:08.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.783+0000 7f7e8a943700 1 --2- 172.21.15.33:0/908256575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2100d08c0 0x55e2100d13e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c0a9440 tx=0x7f7e7c004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c3a49a5bc8b03a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:08.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.783+0000 7f7e6ffff700 1 -- 172.21.15.33:0/908256575 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c0c7040 con 0x55e2100d08c0 2024-05-15T14:00:08.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.783+0000 7f7e6ffff700 1 -- 172.21.15.33:0/908256575 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e7c0a94e0 con 0x55e2100d08c0 2024-05-15T14:00:08.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.783+0000 7f7e6ffff700 1 -- 172.21.15.33:0/908256575 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c0a8040 con 0x55e2100d08c0 2024-05-15T14:00:08.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.784+0000 7f7e96c3cb80 1 -- 172.21.15.33:0/908256575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2100d08c0 msgr2=0x55e2100d13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:08.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.784+0000 7f7e96c3cb80 1 --2- 172.21.15.33:0/908256575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2100d08c0 0x55e2100d13e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c0a9440 tx=0x7f7e7c004f00 comp rx=0 tx=0).stop 2024-05-15T14:00:08.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.784+0000 7f7e96c3cb80 1 -- 172.21.15.33:0/908256575 shutdown_connections 2024-05-15T14:00:08.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.784+0000 7f7e96c3cb80 1 --2- 172.21.15.33:0/908256575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2100d08c0 0x55e2100d13e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:08.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.784+0000 7f7e96c3cb80 1 --2- 172.21.15.33:0/908256575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2100cadc0 0x55e2100ca7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:08.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.784+0000 7f7e96c3cb80 1 --2- 172.21.15.33:0/908256575 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2100cffa0 0x55e2100d4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:08.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.784+0000 7f7e96c3cb80 1 -- 172.21.15.33:0/908256575 >> 172.21.15.33:0/908256575 conn(0x55e20fff7d60 msgr2=0x55e2100d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:08.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.784+0000 7f7e96c3cb80 1 -- 172.21.15.33:0/908256575 shutdown_connections 2024-05-15T14:00:08.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.784+0000 7f7e96c3cb80 1 -- 172.21.15.33:0/908256575 wait complete. 2024-05-15T14:00:08.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.784+0000 7f7e96c3cb80 1 -- start start 2024-05-15T14:00:08.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.784+0000 7f7e96c3cb80 1 Processor -- start 2024-05-15T14:00:08.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.784+0000 7f7e96c3cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2100cadc0 0x55e2100ee1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:08.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.784+0000 7f7e96c3cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2100cffa0 0x55e2100ee720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:08.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.784+0000 7f7e96c3cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2100d08c0 0x55e2100e9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:08.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.784+0000 7f7e96c3cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e2100e4be0 con 0x55e2100d08c0 2024-05-15T14:00:08.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.784+0000 7f7e96c3cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e2100e4510 con 0x55e2100cffa0 2024-05-15T14:00:08.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.784+0000 7f7e96c3cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55e2100e52b0 con 0x55e2100cadc0 2024-05-15T14:00:08.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.785+0000 7f7e96c3cb80 10 client.0 _ll_register_callbacks cb 0x55e20ffb3900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:00:08.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.785+0000 7f7e8a943700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2100d08c0 0x55e2100e9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:08.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.785+0000 7f7e8a943700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2100d08c0 0x55e2100e9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51974/0 (socket says 172.21.15.33:51974) 2024-05-15T14:00:08.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.785+0000 7f7e8a943700 1 -- 172.21.15.33:0/3553124507 learned_addr learned my addr 172.21.15.33:0/3553124507 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:08.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.785+0000 7f7e89941700 1 --2- 172.21.15.33:0/3553124507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2100cffa0 0x55e2100ee720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:08.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.785+0000 7f7e8a142700 1 --2- 172.21.15.33:0/3553124507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2100cadc0 0x55e2100ee1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:08.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.785+0000 7f7e8a943700 1 -- 172.21.15.33:0/3553124507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2100cadc0 msgr2=0x55e2100ee1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:08.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.785+0000 7f7e8a943700 1 --2- 172.21.15.33:0/3553124507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2100cadc0 0x55e2100ee1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:08.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.785+0000 7f7e8a943700 1 -- 172.21.15.33:0/3553124507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2100cffa0 msgr2=0x55e2100ee720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:08.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.785+0000 7f7e8a943700 1 --2- 172.21.15.33:0/3553124507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2100cffa0 0x55e2100ee720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:08.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.785+0000 7f7e8a943700 1 -- 172.21.15.33:0/3553124507 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2100e9840 con 0x55e2100d08c0 2024-05-15T14:00:08.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.785+0000 7f7e8a943700 1 --2- 172.21.15.33:0/3553124507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2100d08c0 0x55e2100e9190 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c0a6050 tx=0x7f7e7c0a9320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3696fcc50b2bbd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:08.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.785+0000 7f7e6dffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:00:08.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.785+0000 7f7e6dffb700 1 -- 172.21.15.33:0/3553124507 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c0c7040 con 0x55e2100d08c0 2024-05-15T14:00:08.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.785+0000 7f7e6dffb700 1 -- 172.21.15.33:0/3553124507 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f7e7c00db20 con 0x55e2100d08c0 2024-05-15T14:00:08.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.785+0000 7f7e96c3cb80 1 -- 172.21.15.33:0/3553124507 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e7c0a94e0 con 0x55e2100d08c0 2024-05-15T14:00:08.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.785+0000 7f7e6dffb700 1 -- 172.21.15.33:0/3553124507 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f7e7c0a7d60 con 0x55e2100d08c0 2024-05-15T14:00:08.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.786+0000 7f7e96c3cb80 10 client.15102 fetch_fsmap learned FSMap version 23 2024-05-15T14:00:08.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.786+0000 7f7e96c3cb80 1 -- 172.21.15.33:0/3553124507 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55e21014f700 con 0x55e2100d08c0 2024-05-15T14:00:08.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.786+0000 7f7e6dffb700 1 -- 172.21.15.33:0/3553124507 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e7c0a94e0 con 0x55e2100d08c0 2024-05-15T14:00:08.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.786+0000 7f7e6dffb700 1 -- 172.21.15.33:0/3553124507 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f7e7c0c3020 con 0x55e2100d08c0 2024-05-15T14:00:08.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.786+0000 7f7e96c3cb80 10 client.15102 fetch_fsmap finished waiting for FSMap version 23 2024-05-15T14:00:08.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.786+0000 7f7e96c3cb80 10 client.15102 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-15T14:00:08.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.786+0000 7f7e96c3cb80 20 client.15102 populate_metadata read hostname 'smithi033' 2024-05-15T14:00:08.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.786+0000 7f7e96c3cb80 1 --2- 172.21.15.33:0/3553124507 >> [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] conn(0x55e210153dd0 0x55e210152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:08.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.786+0000 7f7e96c3cb80 4 client.15102 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-15T14:00:08.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.786+0000 7f7e96c3cb80 1 -- 172.21.15.33:0/3553124507 --> [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] -- command(tid 0: {"prefix": "status"}) -- 0x55e210024450 con 0x55e210153dd0 2024-05-15T14:00:08.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.786+0000 7f7e8a142700 1 --2- 172.21.15.33:0/3553124507 >> [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] conn(0x55e210153dd0 0x55e210152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:08.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.787+0000 7f7e8a142700 1 --2- 172.21.15.33:0/3553124507 >> [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] conn(0x55e210153dd0 0x55e210152ae0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cd87caf8662138a5 server_cookie=8db195a3d4ea8a4b in_seq=0 out_seq=0 2024-05-15T14:00:08.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.787+0000 7f7e6dffb700 10 client.15102 ms_handle_connect on v2:172.21.15.81:6836/744100239 2024-05-15T14:00:08.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.787+0000 7f7e6dffb700 1 -- 172.21.15.33:0/3553124507 <== mds.1 v2:172.21.15.81:6836/744100239 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e210024450 con 0x55e210153dd0 2024-05-15T14:00:08.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.787+0000 7f7e6dffb700 10 client.15102 handle_command_reply: tid=0 2024-05-15T14:00:08.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.787+0000 7f7e96c3cb80 1 client.15102 shutdown 2024-05-15T14:00:08.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.787+0000 7f7e96c3cb80 1 -- 172.21.15.33:0/3553124507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2100d08c0 msgr2=0x55e2100e9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:08.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.787+0000 7f7e96c3cb80 1 --2- 172.21.15.33:0/3553124507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2100d08c0 0x55e2100e9190 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c0a6050 tx=0x7f7e7c0a9320 comp rx=0 tx=0).stop 2024-05-15T14:00:08.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.787+0000 7f7e96c3cb80 1 -- 172.21.15.33:0/3553124507 shutdown_connections 2024-05-15T14:00:08.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.787+0000 7f7e96c3cb80 1 --2- 172.21.15.33:0/3553124507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2100d08c0 0x55e2100e9190 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:08.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.787+0000 7f7e96c3cb80 1 --2- 172.21.15.33:0/3553124507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2100cadc0 0x55e2100ee1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:08.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.787+0000 7f7e96c3cb80 1 --2- 172.21.15.33:0/3553124507 >> [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] conn(0x55e210153dd0 0x55e210152ae0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:08.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.787+0000 7f7e96c3cb80 1 --2- 172.21.15.33:0/3553124507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2100cffa0 0x55e2100ee720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:08.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.787+0000 7f7e96c3cb80 1 -- 172.21.15.33:0/3553124507 >> 172.21.15.33:0/3553124507 conn(0x55e20fff7d60 msgr2=0x55e2100ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:08.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.787+0000 7f7e6dffb700 0 client.15102 ms_handle_reset on v2:172.21.15.81:6836/744100239 2024-05-15T14:00:08.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.787+0000 7f7e96c3cb80 1 -- 172.21.15.33:0/3553124507 shutdown_connections 2024-05-15T14:00:08.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.787+0000 7f7e96c3cb80 1 -- 172.21.15.33:0/3553124507 wait complete. 2024-05-15T14:00:08.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.787+0000 7f7e96c3cb80 20 client.15102 trim_cache size 0 max 16384 2024-05-15T14:00:08.795 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T14:00:08.795 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "bca78ca4-12c1-11ef-bc98-c7b262605968", 2024-05-15T14:00:08.795 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 1, 2024-05-15T14:00:08.795 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24463, 2024-05-15T14:00:08.795 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-05-15T14:00:08.795 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-05-15T14:00:08.795 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-05-15T14:00:08.795 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 157.31193174500001, 2024-05-15T14:00:08.795 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 23, 2024-05-15T14:00:08.795 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 82, 2024-05-15T14:00:08.795 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 80, 2024-05-15T14:00:08.795 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 200.53587023599999 2024-05-15T14:00:08.795 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T14:00:08.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.788+0000 7f7e8c3a6700 1 -- 172.21.15.33:0/3919319685 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f7e700b2900 msgr2=0x7f7e700b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:08.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.788+0000 7f7e8c3a6700 1 --2- 172.21.15.33:0/3919319685 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f7e700b2900 0x7f7e700b4dc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7e74013650 tx=0x7f7e74013a10 comp rx=0 tx=0).stop 2024-05-15T14:00:08.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.788+0000 7f7e8c3a6700 1 -- 172.21.15.33:0/3919319685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e84109650 msgr2=0x7f7e841999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.788+0000 7f7e8c3a6700 1 --2- 172.21.15.33:0/3919319685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e84109650 0x7f7e841999a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c007ee0 tx=0x7f7e7c00d3b0 comp rx=0 tx=0).stop 2024-05-15T14:00:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.788+0000 7f7e8c3a6700 1 -- 172.21.15.33:0/3919319685 shutdown_connections 2024-05-15T14:00:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.788+0000 7f7e8c3a6700 1 --2- 172.21.15.33:0/3919319685 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f7e700b2900 0x7f7e700b4dc0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.788+0000 7f7e8c3a6700 1 --2- 172.21.15.33:0/3919319685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e84109650 0x7f7e841999a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.788+0000 7f7e8c3a6700 1 --2- 172.21.15.33:0/3919319685 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e8406bc70 0x7f7e8419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.788+0000 7f7e8c3a6700 1 --2- 172.21.15.33:0/3919319685 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e8406c630 0x7f7e8419fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.788+0000 7f7e8c3a6700 1 -- 172.21.15.33:0/3919319685 >> 172.21.15.33:0/3919319685 conn(0x7f7e840782b0 msgr2=0x7f7e841013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.788+0000 7f7e8c3a6700 1 -- 172.21.15.33:0/3919319685 shutdown_connections 2024-05-15T14:00:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:08.788+0000 7f7e8c3a6700 1 -- 172.21.15.33:0/3919319685 wait complete. 2024-05-15T14:00:08.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:08 smithi052 ceph-mon[36190]: pgmap v373: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-15T14:00:08.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:08 smithi081 ceph-mon[36434]: pgmap v373: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-15T14:00:09.476 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph tell mds.i status 2024-05-15T14:00:09.728 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:00:10.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:10 smithi033 ceph-mon[31650]: pgmap v374: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T14:00:10.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:10 smithi052 ceph-mon[36190]: pgmap v374: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T14:00:10.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:10 smithi081 ceph-mon[36434]: pgmap v374: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-15T14:00:11.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.330+0000 7f66d8755700 1 -- 172.21.15.33:0/2657396682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66d01041a0 msgr2=0x7f66d01081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:11.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.330+0000 7f66d8755700 1 --2- 172.21.15.33:0/2657396682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66d01041a0 0x7f66d01081c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f66c00099c0 tx=0x7f66c0009cd0 comp rx=0 tx=0).stop 2024-05-15T14:00:11.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.330+0000 7f66d8755700 1 -- 172.21.15.33:0/2657396682 shutdown_connections 2024-05-15T14:00:11.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.330+0000 7f66d8755700 1 --2- 172.21.15.33:0/2657396682 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66d01088f0 0x7f66d0110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:11.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.330+0000 7f66d8755700 1 --2- 172.21.15.33:0/2657396682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66d01037f0 0x7f66d0103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:11.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.330+0000 7f66d8755700 1 --2- 172.21.15.33:0/2657396682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66d01041a0 0x7f66d01081c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:11.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.330+0000 7f66d8755700 1 -- 172.21.15.33:0/2657396682 >> 172.21.15.33:0/2657396682 conn(0x7f66d007a6a0 msgr2=0x7f66d007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:11.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.330+0000 7f66d8755700 1 -- 172.21.15.33:0/2657396682 shutdown_connections 2024-05-15T14:00:11.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.330+0000 7f66d8755700 1 -- 172.21.15.33:0/2657396682 wait complete. 2024-05-15T14:00:11.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.331+0000 7f66d8755700 1 Processor -- start 2024-05-15T14:00:11.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.331+0000 7f66d8755700 1 -- start start 2024-05-15T14:00:11.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.331+0000 7f66d8755700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66d01037f0 0x7f66d0075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:11.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.331+0000 7f66d8755700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66d01041a0 0x7f66d0075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:11.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.331+0000 7f66d8755700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66d01088f0 0x7f66d00762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:11.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.331+0000 7f66d8755700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f66d0113730 con 0x7f66d01041a0 2024-05-15T14:00:11.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.331+0000 7f66d8755700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f66d01135b0 con 0x7f66d01037f0 2024-05-15T14:00:11.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.331+0000 7f66d8755700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f66d01138b0 con 0x7f66d01088f0 2024-05-15T14:00:11.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.331+0000 7f66d5cf0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66d01041a0 0x7f66d0075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:11.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.331+0000 7f66d5cf0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66d01041a0 0x7f66d0075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51984/0 (socket says 172.21.15.33:51984) 2024-05-15T14:00:11.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.331+0000 7f66d5cf0700 1 -- 172.21.15.33:0/3964975948 learned_addr learned my addr 172.21.15.33:0/3964975948 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:11.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.331+0000 7f66d5cf0700 1 -- 172.21.15.33:0/3964975948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66d01088f0 msgr2=0x7f66d00762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:11.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.331+0000 7f66d6cf2700 1 --2- 172.21.15.33:0/3964975948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66d01088f0 0x7f66d00762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:11.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.331+0000 7f66d5cf0700 1 --2- 172.21.15.33:0/3964975948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66d01088f0 0x7f66d00762c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:11.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.331+0000 7f66d64f1700 1 --2- 172.21.15.33:0/3964975948 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66d01037f0 0x7f66d0075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:11.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.331+0000 7f66d5cf0700 1 -- 172.21.15.33:0/3964975948 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66d01037f0 msgr2=0x7f66d0075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:11.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.331+0000 7f66d5cf0700 1 --2- 172.21.15.33:0/3964975948 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66d01037f0 0x7f66d0075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:11.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.331+0000 7f66d5cf0700 1 -- 172.21.15.33:0/3964975948 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d01a8140 con 0x7f66d01041a0 2024-05-15T14:00:11.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.332+0000 7f66d5cf0700 1 --2- 172.21.15.33:0/3964975948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66d01041a0 0x7f66d0075d80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f66c0006010 tx=0x7f66c00056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:11.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.332+0000 7f66c77fe700 1 -- 172.21.15.33:0/3964975948 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c0026070 con 0x7f66d01041a0 2024-05-15T14:00:11.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.332+0000 7f66c77fe700 1 -- 172.21.15.33:0/3964975948 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66c000f460 con 0x7f66d01041a0 2024-05-15T14:00:11.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.332+0000 7f66d8755700 1 -- 172.21.15.33:0/3964975948 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66d01a83d0 con 0x7f66d01041a0 2024-05-15T14:00:11.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.332+0000 7f66c77fe700 1 -- 172.21.15.33:0/3964975948 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c0009e60 con 0x7f66d01041a0 2024-05-15T14:00:11.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.332+0000 7f66d8755700 1 -- 172.21.15.33:0/3964975948 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66d01a8820 con 0x7f66d01041a0 2024-05-15T14:00:11.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.334+0000 7f66c77fe700 1 -- 172.21.15.33:0/3964975948 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f66c0029070 con 0x7f66d01041a0 2024-05-15T14:00:11.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.335+0000 7f66c77fe700 1 --2- 172.21.15.33:0/3964975948 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f66bc0b2880 0x7f66bc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:11.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.335+0000 7f66d64f1700 1 --2- 172.21.15.33:0/3964975948 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f66bc0b2880 0x7f66bc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:11.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.335+0000 7f66c77fe700 1 -- 172.21.15.33:0/3964975948 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f66c00e2b70 con 0x7f66d01041a0 2024-05-15T14:00:11.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.336+0000 7f66d64f1700 1 --2- 172.21.15.33:0/3964975948 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f66bc0b2880 0x7f66bc0b4d40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f66cc006fd0 tx=0x7f66cc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:11.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.336+0000 7f66e2febb80 1 Processor -- start 2024-05-15T14:00:11.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.336+0000 7f66e2febb80 1 -- start start 2024-05-15T14:00:11.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.336+0000 7f66e2febb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55684c2e3fa0 0x55684c2e4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:11.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.336+0000 7f66e2febb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55684c2e48c0 0x55684c2de7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:11.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.336+0000 7f66e2febb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55684c2dedc0 0x55684c2df220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:11.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.336+0000 7f66e2febb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55684c23cd20 con 0x55684c2e48c0 2024-05-15T14:00:11.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.336+0000 7f66e2febb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55684c22c8c0 con 0x55684c2e3fa0 2024-05-15T14:00:11.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.336+0000 7f66e2febb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55684c230900 con 0x55684c2dedc0 2024-05-15T14:00:11.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.336+0000 7f66d5cf0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55684c2e48c0 0x55684c2de7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:11.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.336+0000 7f66d5cf0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55684c2e48c0 0x55684c2de7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51990/0 (socket says 172.21.15.33:51990) 2024-05-15T14:00:11.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.336+0000 7f66d5cf0700 1 -- 172.21.15.33:0/1150179568 learned_addr learned my addr 172.21.15.33:0/1150179568 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:11.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.336+0000 7f66d6cf2700 1 --2- 172.21.15.33:0/1150179568 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55684c2dedc0 0x55684c2df220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:11.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.336+0000 7f66d64f1700 1 --2- 172.21.15.33:0/1150179568 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55684c2e3fa0 0x55684c2e4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:11.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.336+0000 7f66d5cf0700 1 -- 172.21.15.33:0/1150179568 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55684c2dedc0 msgr2=0x55684c2df220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:11.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.337+0000 7f66d5cf0700 1 --2- 172.21.15.33:0/1150179568 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55684c2dedc0 0x55684c2df220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:11.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.337+0000 7f66d5cf0700 1 -- 172.21.15.33:0/1150179568 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55684c2e3fa0 msgr2=0x55684c2e4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:11.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.337+0000 7f66d5cf0700 1 --2- 172.21.15.33:0/1150179568 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55684c2e3fa0 0x55684c2e4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:11.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.337+0000 7f66d5cf0700 1 -- 172.21.15.33:0/1150179568 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55684c1d5f60 con 0x55684c2e48c0 2024-05-15T14:00:11.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.337+0000 7f66d5cf0700 1 --2- 172.21.15.33:0/1150179568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55684c2e48c0 0x55684c2de7a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f66c0003760 tx=0x7f66c00a99d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4db8dad60fa277b server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:11.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.337+0000 7f66c4ff9700 1 -- 172.21.15.33:0/1150179568 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c00aa1e0 con 0x55684c2e48c0 2024-05-15T14:00:11.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.337+0000 7f66c4ff9700 1 -- 172.21.15.33:0/1150179568 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66c00ae590 con 0x55684c2e48c0 2024-05-15T14:00:11.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.337+0000 7f66c4ff9700 1 -- 172.21.15.33:0/1150179568 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c00b2d80 con 0x55684c2e48c0 2024-05-15T14:00:11.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.338+0000 7f66e2febb80 1 -- 172.21.15.33:0/1150179568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55684c2e48c0 msgr2=0x55684c2de7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:11.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.338+0000 7f66e2febb80 1 --2- 172.21.15.33:0/1150179568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55684c2e48c0 0x55684c2de7a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f66c0003760 tx=0x7f66c00a99d0 comp rx=0 tx=0).stop 2024-05-15T14:00:11.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.338+0000 7f66e2febb80 1 -- 172.21.15.33:0/1150179568 shutdown_connections 2024-05-15T14:00:11.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.338+0000 7f66e2febb80 1 --2- 172.21.15.33:0/1150179568 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55684c2dedc0 0x55684c2df220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:11.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.338+0000 7f66e2febb80 1 --2- 172.21.15.33:0/1150179568 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55684c2e3fa0 0x55684c2e4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:11.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.338+0000 7f66e2febb80 1 --2- 172.21.15.33:0/1150179568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55684c2e48c0 0x55684c2de7a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:11.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.338+0000 7f66e2febb80 1 -- 172.21.15.33:0/1150179568 >> 172.21.15.33:0/1150179568 conn(0x55684c20bd60 msgr2=0x55684c2d8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:11.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.338+0000 7f66e2febb80 1 -- 172.21.15.33:0/1150179568 shutdown_connections 2024-05-15T14:00:11.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.338+0000 7f66e2febb80 1 -- 172.21.15.33:0/1150179568 wait complete. 2024-05-15T14:00:11.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.338+0000 7f66e2febb80 1 -- start start 2024-05-15T14:00:11.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.338+0000 7f66e2febb80 1 Processor -- start 2024-05-15T14:00:11.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.339+0000 7f66e2febb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55684c2dedc0 0x55684c2ea620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:11.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.339+0000 7f66e2febb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55684c2e3fa0 0x55684c2e79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:11.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.339+0000 7f66e2febb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55684c2e48c0 0x55684c2e7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:11.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.339+0000 7f66e2febb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55684c132f50 con 0x55684c2e3fa0 2024-05-15T14:00:11.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.339+0000 7f66e2febb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55684c194d50 con 0x55684c2e48c0 2024-05-15T14:00:11.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.339+0000 7f66e2febb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55684c2eb090 con 0x55684c2dedc0 2024-05-15T14:00:11.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.339+0000 7f66e2febb80 10 client.0 _ll_register_callbacks cb 0x55684c1c7900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:00:11.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.339+0000 7f66d5cf0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55684c2e3fa0 0x55684c2e79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:11.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.339+0000 7f66d5cf0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55684c2e3fa0 0x55684c2e79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52004/0 (socket says 172.21.15.33:52004) 2024-05-15T14:00:11.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.339+0000 7f66d5cf0700 1 -- 172.21.15.33:0/3279534379 learned_addr learned my addr 172.21.15.33:0/3279534379 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:11.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.339+0000 7f66d6cf2700 1 --2- 172.21.15.33:0/3279534379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55684c2e48c0 0x55684c2e7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:11.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.339+0000 7f66d64f1700 1 --2- 172.21.15.33:0/3279534379 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55684c2dedc0 0x55684c2ea620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:11.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.339+0000 7f66d5cf0700 1 -- 172.21.15.33:0/3279534379 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55684c2dedc0 msgr2=0x55684c2ea620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:11.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.339+0000 7f66d5cf0700 1 --2- 172.21.15.33:0/3279534379 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55684c2dedc0 0x55684c2ea620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:11.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.339+0000 7f66d5cf0700 1 -- 172.21.15.33:0/3279534379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55684c2e48c0 msgr2=0x55684c2e7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:11.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.339+0000 7f66d5cf0700 1 --2- 172.21.15.33:0/3279534379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55684c2e48c0 0x55684c2e7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:11.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.339+0000 7f66d5cf0700 1 -- 172.21.15.33:0/3279534379 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55684c17c880 con 0x55684c2e3fa0 2024-05-15T14:00:11.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.339+0000 7f66d64f1700 1 --2- 172.21.15.33:0/3279534379 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55684c2dedc0 0x55684c2ea620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:00:11.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.339+0000 7f66d5cf0700 1 --2- 172.21.15.33:0/3279534379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55684c2e3fa0 0x55684c2e79c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f66c00059d0 tx=0x7f66c0005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fb21901ac4ae7d server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:11.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.339+0000 7f66b9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:00:11.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.339+0000 7f66b9ffb700 1 -- 172.21.15.33:0/3279534379 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c00aa650 con 0x55684c2e3fa0 2024-05-15T14:00:11.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.339+0000 7f66b9ffb700 1 -- 172.21.15.33:0/3279534379 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f66c00ae170 con 0x55684c2e3fa0 2024-05-15T14:00:11.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.339+0000 7f66e2febb80 1 -- 172.21.15.33:0/3279534379 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66c00ae590 con 0x55684c2e3fa0 2024-05-15T14:00:11.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.339+0000 7f66b9ffb700 1 -- 172.21.15.33:0/3279534379 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f66c00ae370 con 0x55684c2e3fa0 2024-05-15T14:00:11.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.340+0000 7f66e2febb80 10 client.15114 fetch_fsmap learned FSMap version 23 2024-05-15T14:00:11.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.340+0000 7f66e2febb80 1 -- 172.21.15.33:0/3279534379 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55684c1e6710 con 0x55684c2e3fa0 2024-05-15T14:00:11.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.340+0000 7f66b9ffb700 1 -- 172.21.15.33:0/3279534379 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66c00ae590 con 0x55684c2e3fa0 2024-05-15T14:00:11.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.340+0000 7f66b9ffb700 1 -- 172.21.15.33:0/3279534379 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f66c00e0af0 con 0x55684c2e3fa0 2024-05-15T14:00:11.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.340+0000 7f66e2febb80 10 client.15114 fetch_fsmap finished waiting for FSMap version 23 2024-05-15T14:00:11.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.340+0000 7f66e2febb80 10 client.15114 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-15T14:00:11.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.340+0000 7f66e2febb80 20 client.15114 populate_metadata read hostname 'smithi033' 2024-05-15T14:00:11.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.340+0000 7f66e2febb80 1 --2- 172.21.15.33:0/3279534379 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55684c2f76f0 0x55684c2f8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:11.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.340+0000 7f66e2febb80 4 client.15114 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:00:11.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.340+0000 7f66e2febb80 1 -- 172.21.15.33:0/3279534379 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55684c23ec70 con 0x55684c2f76f0 2024-05-15T14:00:11.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.340+0000 7f66d64f1700 1 --2- 172.21.15.33:0/3279534379 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55684c2f76f0 0x55684c2f8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:11.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.341+0000 7f66d64f1700 1 --2- 172.21.15.33:0/3279534379 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55684c2f76f0 0x55684c2f8b60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1bce648ea745d8c server_cookie=9b424aea4d821216 in_seq=0 out_seq=0 2024-05-15T14:00:11.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.341+0000 7f66b9ffb700 10 client.15114 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:00:11.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.341+0000 7f66b9ffb700 1 -- 172.21.15.33:0/3279534379 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55684c23ec70 con 0x55684c2f76f0 2024-05-15T14:00:11.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.341+0000 7f66b9ffb700 10 client.15114 handle_command_reply: tid=0 2024-05-15T14:00:11.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.342+0000 7f66e2febb80 1 client.15114 shutdown 2024-05-15T14:00:11.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.342+0000 7f66e2febb80 1 -- 172.21.15.33:0/3279534379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55684c2e3fa0 msgr2=0x55684c2e79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:11.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.342+0000 7f66e2febb80 1 --2- 172.21.15.33:0/3279534379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55684c2e3fa0 0x55684c2e79c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f66c00059d0 tx=0x7f66c0005c00 comp rx=0 tx=0).stop 2024-05-15T14:00:11.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.342+0000 7f66e2febb80 1 -- 172.21.15.33:0/3279534379 shutdown_connections 2024-05-15T14:00:11.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.342+0000 7f66e2febb80 1 --2- 172.21.15.33:0/3279534379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55684c2e48c0 0x55684c2e7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:11.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.342+0000 7f66e2febb80 1 --2- 172.21.15.33:0/3279534379 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55684c2dedc0 0x55684c2ea620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:11.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.342+0000 7f66e2febb80 1 --2- 172.21.15.33:0/3279534379 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55684c2f76f0 0x55684c2f8b60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:11.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.342+0000 7f66e2febb80 1 --2- 172.21.15.33:0/3279534379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55684c2e3fa0 0x55684c2e79c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:11.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.342+0000 7f66b9ffb700 0 client.15114 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:00:11.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.342+0000 7f66e2febb80 1 -- 172.21.15.33:0/3279534379 >> 172.21.15.33:0/3279534379 conn(0x55684c20bd60 msgr2=0x55684c2d8630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:11.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.342+0000 7f66e2febb80 1 -- 172.21.15.33:0/3279534379 shutdown_connections 2024-05-15T14:00:11.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.342+0000 7f66e2febb80 1 -- 172.21.15.33:0/3279534379 wait complete. 2024-05-15T14:00:11.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.342+0000 7f66e2febb80 20 client.15114 trim_cache size 0 max 16384 2024-05-15T14:00:11.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.363+0000 7f66e2febb80 1 Processor -- start 2024-05-15T14:00:11.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.363+0000 7f66e2febb80 1 -- start start 2024-05-15T14:00:11.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.364+0000 7f66e2febb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55684c2dedc0 0x55684c2de7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:11.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.364+0000 7f66e2febb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55684c2e3fa0 0x55684c2e5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:11.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.364+0000 7f66e2febb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55684c2e48c0 0x55684c2d8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:11.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.364+0000 7f66e2febb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55684c2f6ea0 con 0x55684c2e3fa0 2024-05-15T14:00:11.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.364+0000 7f66e2febb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55684c2f7570 con 0x55684c2e48c0 2024-05-15T14:00:11.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.364+0000 7f66e2febb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55684c2f7c40 con 0x55684c2dedc0 2024-05-15T14:00:11.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.364+0000 7f66d5cf0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55684c2e3fa0 0x55684c2e5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:11.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.364+0000 7f66d5cf0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55684c2e3fa0 0x55684c2e5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52012/0 (socket says 172.21.15.33:52012) 2024-05-15T14:00:11.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.364+0000 7f66d5cf0700 1 -- 172.21.15.33:0/1513669104 learned_addr learned my addr 172.21.15.33:0/1513669104 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:11.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.364+0000 7f66d5cf0700 1 -- 172.21.15.33:0/1513669104 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55684c2dedc0 msgr2=0x55684c2de7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:11.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.364+0000 7f66d5cf0700 1 --2- 172.21.15.33:0/1513669104 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55684c2dedc0 0x55684c2de7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:11.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.364+0000 7f66d5cf0700 1 -- 172.21.15.33:0/1513669104 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55684c2e48c0 msgr2=0x55684c2d8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:11.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.364+0000 7f66d5cf0700 1 --2- 172.21.15.33:0/1513669104 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55684c2e48c0 0x55684c2d8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:11.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.364+0000 7f66d5cf0700 1 -- 172.21.15.33:0/1513669104 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55684c1e6710 con 0x55684c2e3fa0 2024-05-15T14:00:11.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.364+0000 7f66d5cf0700 1 --2- 172.21.15.33:0/1513669104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55684c2e3fa0 0x55684c2e5530 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f66c00a7450 tx=0x7f66c0005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f821b05376fb834b server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:11.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.364+0000 7f66bbfff700 1 -- 172.21.15.33:0/1513669104 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c00ae590 con 0x55684c2e3fa0 2024-05-15T14:00:11.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.364+0000 7f66bbfff700 1 -- 172.21.15.33:0/1513669104 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66c00aa650 con 0x55684c2e3fa0 2024-05-15T14:00:11.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.364+0000 7f66bbfff700 1 -- 172.21.15.33:0/1513669104 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c00b2b20 con 0x55684c2e3fa0 2024-05-15T14:00:11.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.365+0000 7f66e2febb80 1 -- 172.21.15.33:0/1513669104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55684c2e3fa0 msgr2=0x55684c2e5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:11.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.365+0000 7f66e2febb80 1 --2- 172.21.15.33:0/1513669104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55684c2e3fa0 0x55684c2e5530 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f66c00a7450 tx=0x7f66c0005c70 comp rx=0 tx=0).stop 2024-05-15T14:00:11.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.365+0000 7f66e2febb80 1 -- 172.21.15.33:0/1513669104 shutdown_connections 2024-05-15T14:00:11.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.365+0000 7f66e2febb80 1 --2- 172.21.15.33:0/1513669104 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55684c2e48c0 0x55684c2d8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:11.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.365+0000 7f66e2febb80 1 --2- 172.21.15.33:0/1513669104 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55684c2dedc0 0x55684c2de7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:11.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.365+0000 7f66e2febb80 1 --2- 172.21.15.33:0/1513669104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55684c2e3fa0 0x55684c2e5530 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:11.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.365+0000 7f66e2febb80 1 -- 172.21.15.33:0/1513669104 >> 172.21.15.33:0/1513669104 conn(0x55684c20bd60 msgr2=0x55684c2e2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:11.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.365+0000 7f66e2febb80 1 -- 172.21.15.33:0/1513669104 shutdown_connections 2024-05-15T14:00:11.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.365+0000 7f66e2febb80 1 -- 172.21.15.33:0/1513669104 wait complete. 2024-05-15T14:00:11.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.365+0000 7f66e2febb80 1 -- start start 2024-05-15T14:00:11.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.366+0000 7f66e2febb80 1 Processor -- start 2024-05-15T14:00:11.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.366+0000 7f66e2febb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55684c2dedc0 0x55684c301fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:11.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.366+0000 7f66e2febb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55684c2e3fa0 0x55684c302520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:11.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.366+0000 7f66e2febb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55684c2e48c0 0x55684c2fcf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:11.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.366+0000 7f66e2febb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55684c2f89e0 con 0x55684c2e3fa0 2024-05-15T14:00:11.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.366+0000 7f66e2febb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55684c2f8310 con 0x55684c2e48c0 2024-05-15T14:00:11.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.366+0000 7f66e2febb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55684c2f90b0 con 0x55684c2dedc0 2024-05-15T14:00:11.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.366+0000 7f66d5cf0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55684c2e3fa0 0x55684c302520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:11.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.366+0000 7f66d5cf0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55684c2e3fa0 0x55684c302520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52024/0 (socket says 172.21.15.33:52024) 2024-05-15T14:00:11.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.366+0000 7f66d5cf0700 1 -- 172.21.15.33:0/3249951536 learned_addr learned my addr 172.21.15.33:0/3249951536 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:11.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.366+0000 7f66e2febb80 10 client.0 _ll_register_callbacks cb 0x55684c1c7900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:00:11.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.366+0000 7f66d64f1700 1 --2- 172.21.15.33:0/3249951536 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55684c2dedc0 0x55684c301fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:11.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.366+0000 7f66d6cf2700 1 --2- 172.21.15.33:0/3249951536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55684c2e48c0 0x55684c2fcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:11.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.366+0000 7f66d5cf0700 1 -- 172.21.15.33:0/3249951536 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55684c2dedc0 msgr2=0x55684c301fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:11.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.366+0000 7f66d5cf0700 1 --2- 172.21.15.33:0/3249951536 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55684c2dedc0 0x55684c301fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:11.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.366+0000 7f66d5cf0700 1 -- 172.21.15.33:0/3249951536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55684c2e48c0 msgr2=0x55684c2fcf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:11.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.366+0000 7f66d5cf0700 1 --2- 172.21.15.33:0/3249951536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55684c2e48c0 0x55684c2fcf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:11.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.366+0000 7f66d5cf0700 1 -- 172.21.15.33:0/3249951536 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55684c2e45d0 con 0x55684c2e3fa0 2024-05-15T14:00:11.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.366+0000 7f66d6cf2700 1 --2- 172.21.15.33:0/3249951536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55684c2e48c0 0x55684c2fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:00:11.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.367+0000 7f66d5cf0700 1 --2- 172.21.15.33:0/3249951536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55684c2e3fa0 0x55684c302520 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f66c00a6260 tx=0x7f66c00a5d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeafbb69ebb62b04 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:11.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.367+0000 7f66b9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:00:11.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.367+0000 7f66b9ffb700 1 -- 172.21.15.33:0/3249951536 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c00ae590 con 0x55684c2e3fa0 2024-05-15T14:00:11.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.367+0000 7f66b9ffb700 1 -- 172.21.15.33:0/3249951536 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f66c00ae170 con 0x55684c2e3fa0 2024-05-15T14:00:11.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.367+0000 7f66e2febb80 1 -- 172.21.15.33:0/3249951536 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66c00aa650 con 0x55684c2e3fa0 2024-05-15T14:00:11.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.367+0000 7f66b9ffb700 1 -- 172.21.15.33:0/3249951536 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f66c00b2fb0 con 0x55684c2e3fa0 2024-05-15T14:00:11.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.367+0000 7f66e2febb80 10 client.15120 fetch_fsmap learned FSMap version 23 2024-05-15T14:00:11.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.367+0000 7f66e2febb80 1 -- 172.21.15.33:0/3249951536 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55684c2e24d0 con 0x55684c2e3fa0 2024-05-15T14:00:11.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.367+0000 7f66b9ffb700 1 -- 172.21.15.33:0/3249951536 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66c00aa650 con 0x55684c2e3fa0 2024-05-15T14:00:11.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.367+0000 7f66b9ffb700 1 -- 172.21.15.33:0/3249951536 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f66c00c4a40 con 0x55684c2e3fa0 2024-05-15T14:00:11.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.367+0000 7f66e2febb80 10 client.15120 fetch_fsmap finished waiting for FSMap version 23 2024-05-15T14:00:11.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.367+0000 7f66e2febb80 10 client.15120 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-15T14:00:11.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.367+0000 7f66e2febb80 20 client.15120 populate_metadata read hostname 'smithi033' 2024-05-15T14:00:11.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.367+0000 7f66e2febb80 1 --2- 172.21.15.33:0/3249951536 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55684c367570 0x55684c366280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:11.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.367+0000 7f66e2febb80 4 client.15120 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-15T14:00:11.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.367+0000 7f66e2febb80 1 -- 172.21.15.33:0/3249951536 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "status"}) -- 0x55684c238450 con 0x55684c367570 2024-05-15T14:00:11.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.368+0000 7f66d64f1700 1 --2- 172.21.15.33:0/3249951536 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55684c367570 0x55684c366280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:11.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.368+0000 7f66d64f1700 1 --2- 172.21.15.33:0/3249951536 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55684c367570 0x55684c366280 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dee60c6b26de0eb4 server_cookie=2afbee5f57ce757f in_seq=0 out_seq=0 2024-05-15T14:00:11.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.368+0000 7f66b9ffb700 10 client.15120 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:00:11.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.368+0000 7f66b9ffb700 1 -- 172.21.15.33:0/3249951536 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55684c238450 con 0x55684c367570 2024-05-15T14:00:11.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.368+0000 7f66b9ffb700 10 client.15120 handle_command_reply: tid=0 2024-05-15T14:00:11.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.368+0000 7f66e2febb80 1 client.15120 shutdown 2024-05-15T14:00:11.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.368+0000 7f66e2febb80 1 -- 172.21.15.33:0/3249951536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55684c2e3fa0 msgr2=0x55684c302520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:11.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.368+0000 7f66e2febb80 1 --2- 172.21.15.33:0/3249951536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55684c2e3fa0 0x55684c302520 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f66c00a6260 tx=0x7f66c00a5d30 comp rx=0 tx=0).stop 2024-05-15T14:00:11.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.368+0000 7f66e2febb80 1 -- 172.21.15.33:0/3249951536 shutdown_connections 2024-05-15T14:00:11.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.368+0000 7f66e2febb80 1 --2- 172.21.15.33:0/3249951536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55684c2e48c0 0x55684c2fcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:11.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.368+0000 7f66e2febb80 1 --2- 172.21.15.33:0/3249951536 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55684c2dedc0 0x55684c301fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:11.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.368+0000 7f66e2febb80 1 --2- 172.21.15.33:0/3249951536 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55684c367570 0x55684c366280 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:11.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.369+0000 7f66e2febb80 1 --2- 172.21.15.33:0/3249951536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55684c2e3fa0 0x55684c302520 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:11.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.369+0000 7f66e2febb80 1 -- 172.21.15.33:0/3249951536 >> 172.21.15.33:0/3249951536 conn(0x55684c20bd60 msgr2=0x55684c2e33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:11.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.369+0000 7f66b9ffb700 0 client.15120 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:00:11.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.369+0000 7f66e2febb80 1 -- 172.21.15.33:0/3249951536 shutdown_connections 2024-05-15T14:00:11.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.369+0000 7f66e2febb80 1 -- 172.21.15.33:0/3249951536 wait complete. 2024-05-15T14:00:11.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.369+0000 7f66e2febb80 20 client.15120 trim_cache size 0 max 16384 2024-05-15T14:00:11.377 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T14:00:11.377 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "bca78ca4-12c1-11ef-bc98-c7b262605968", 2024-05-15T14:00:11.377 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2024-05-15T14:00:11.378 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24478, 2024-05-15T14:00:11.378 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-05-15T14:00:11.378 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-05-15T14:00:11.378 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-05-15T14:00:11.378 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 171.98970597600001, 2024-05-15T14:00:11.378 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 23, 2024-05-15T14:00:11.378 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 82, 2024-05-15T14:00:11.378 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 80, 2024-05-15T14:00:11.378 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 197.704799111 2024-05-15T14:00:11.378 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T14:00:11.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.369+0000 7f66d8755700 1 -- 172.21.15.33:0/3964975948 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f66bc0b2880 msgr2=0x7f66bc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:11.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.369+0000 7f66d8755700 1 --2- 172.21.15.33:0/3964975948 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f66bc0b2880 0x7f66bc0b4d40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f66cc006fd0 tx=0x7f66cc008040 comp rx=0 tx=0).stop 2024-05-15T14:00:11.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.369+0000 7f66d8755700 1 -- 172.21.15.33:0/3964975948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66d01041a0 msgr2=0x7f66d0075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:11.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.369+0000 7f66d8755700 1 --2- 172.21.15.33:0/3964975948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66d01041a0 0x7f66d0075d80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f66c0006010 tx=0x7f66c00056a0 comp rx=0 tx=0).stop 2024-05-15T14:00:11.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.369+0000 7f66d8755700 1 -- 172.21.15.33:0/3964975948 shutdown_connections 2024-05-15T14:00:11.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.369+0000 7f66d8755700 1 --2- 172.21.15.33:0/3964975948 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f66bc0b2880 0x7f66bc0b4d40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:11.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.369+0000 7f66d8755700 1 --2- 172.21.15.33:0/3964975948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66d01088f0 0x7f66d00762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:11.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.369+0000 7f66d8755700 1 --2- 172.21.15.33:0/3964975948 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66d01037f0 0x7f66d0075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:11.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.369+0000 7f66d8755700 1 --2- 172.21.15.33:0/3964975948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66d01041a0 0x7f66d0075d80 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:11.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.369+0000 7f66d8755700 1 -- 172.21.15.33:0/3964975948 >> 172.21.15.33:0/3964975948 conn(0x7f66d007a6a0 msgr2=0x7f66d0109a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:11.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.369+0000 7f66d8755700 1 -- 172.21.15.33:0/3964975948 shutdown_connections 2024-05-15T14:00:11.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:11.370+0000 7f66d8755700 1 -- 172.21.15.33:0/3964975948 wait complete. 2024-05-15T14:00:12.213 INFO:teuthology.run_tasks:Running task sequential... 2024-05-15T14:00:12.223 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-15T14:00:12.223 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-15T14:00:12.223 INFO:teuthology.task.print:Enabling mgr modules 2024-05-15T14:00:12.224 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-05-15T14:00:12.224 INFO:teuthology.task.exec:Executing custom commands... 2024-05-15T14:00:12.224 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-05-15T14:00:12.225 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-05-15T14:00:12.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.336+0000 7f1899472640 1 -- 172.21.15.33:0/528072904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1894102be0 msgr2=0x7f1894103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:12.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.336+0000 7f1899472640 1 --2- 172.21.15.33:0/528072904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1894102be0 0x7f1894103060 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1888009900 tx=0x7f188802a410 comp rx=0 tx=0).stop 2024-05-15T14:00:12.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.336+0000 7f1899472640 1 -- 172.21.15.33:0/528072904 shutdown_connections 2024-05-15T14:00:12.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.336+0000 7f1899472640 1 --2- 172.21.15.33:0/528072904 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18941035a0 0x7f1894109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:12.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.336+0000 7f1899472640 1 --2- 172.21.15.33:0/528072904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1894102be0 0x7f1894103060 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:12.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.336+0000 7f1899472640 1 --2- 172.21.15.33:0/528072904 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18941019f0 0x7f1894101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:12.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.336+0000 7f1899472640 1 -- 172.21.15.33:0/528072904 >> 172.21.15.33:0/528072904 conn(0x7f18940fcf40 msgr2=0x7f18940ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:12.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.336+0000 7f1899472640 1 -- 172.21.15.33:0/528072904 shutdown_connections 2024-05-15T14:00:12.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.336+0000 7f1899472640 1 -- 172.21.15.33:0/528072904 wait complete. 2024-05-15T14:00:12.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.337+0000 7f1899472640 1 Processor -- start 2024-05-15T14:00:12.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.337+0000 7f1899472640 1 -- start start 2024-05-15T14:00:12.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.337+0000 7f1899472640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18941019f0 0x7f1894105980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:12.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.337+0000 7f1899472640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1894102be0 0x7f1894103fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:12.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.337+0000 7f1899472640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18941035a0 0x7f1894104500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:12.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.337+0000 7f1899472640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f189410d520 con 0x7f1894102be0 2024-05-15T14:00:12.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.337+0000 7f1899472640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f189410d3a0 con 0x7f18941019f0 2024-05-15T14:00:12.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.337+0000 7f1899472640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f189410d6a0 con 0x7f18941035a0 2024-05-15T14:00:12.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.337+0000 7f18927fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1894102be0 0x7f1894103fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:12.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.337+0000 7f18927fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1894102be0 0x7f1894103fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52052/0 (socket says 172.21.15.33:52052) 2024-05-15T14:00:12.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.337+0000 7f18927fc640 1 -- 172.21.15.33:0/554429191 learned_addr learned my addr 172.21.15.33:0/554429191 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:12.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.337+0000 7f1892ffd640 1 --2- 172.21.15.33:0/554429191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18941019f0 0x7f1894105980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:12.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.338+0000 7f18937fe640 1 --2- 172.21.15.33:0/554429191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18941035a0 0x7f1894104500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:12.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.338+0000 7f18927fc640 1 -- 172.21.15.33:0/554429191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18941035a0 msgr2=0x7f1894104500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:12.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.338+0000 7f18927fc640 1 --2- 172.21.15.33:0/554429191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18941035a0 0x7f1894104500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:12.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.338+0000 7f18927fc640 1 -- 172.21.15.33:0/554429191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18941019f0 msgr2=0x7f1894105980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:12.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.338+0000 7f18927fc640 1 --2- 172.21.15.33:0/554429191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18941019f0 0x7f1894105980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:12.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.338+0000 7f18927fc640 1 -- 172.21.15.33:0/554429191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1894104d90 con 0x7f1894102be0 2024-05-15T14:00:12.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.338+0000 7f18927fc640 1 --2- 172.21.15.33:0/554429191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1894102be0 0x7f1894103fc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1888005940 tx=0x7f18880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:12.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.338+0000 7f187bfff640 1 -- 172.21.15.33:0/554429191 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188803f030 con 0x7f1894102be0 2024-05-15T14:00:12.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.338+0000 7f187bfff640 1 -- 172.21.15.33:0/554429191 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1888003cb0 con 0x7f1894102be0 2024-05-15T14:00:12.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.338+0000 7f1899472640 1 -- 172.21.15.33:0/554429191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18941a2e30 con 0x7f1894102be0 2024-05-15T14:00:12.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.338+0000 7f187bfff640 1 -- 172.21.15.33:0/554429191 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1888035070 con 0x7f1894102be0 2024-05-15T14:00:12.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.338+0000 7f1899472640 1 -- 172.21.15.33:0/554429191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18941a3290 con 0x7f1894102be0 2024-05-15T14:00:12.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.339+0000 7f1899472640 1 -- 172.21.15.33:0/554429191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18941a3520 con 0x7f1894102be0 2024-05-15T14:00:12.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.341+0000 7f187bfff640 1 -- 172.21.15.33:0/554429191 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1888042030 con 0x7f1894102be0 2024-05-15T14:00:12.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.342+0000 7f187bfff640 1 --2- 172.21.15.33:0/554429191 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f187c0b27e0 0x7f187c0b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:12.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.342+0000 7f1892ffd640 1 --2- 172.21.15.33:0/554429191 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f187c0b27e0 0x7f187c0b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:12.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.342+0000 7f187bfff640 1 -- 172.21.15.33:0/554429191 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f1888102c80 con 0x7f1894102be0 2024-05-15T14:00:12.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.342+0000 7f1892ffd640 1 --2- 172.21.15.33:0/554429191 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f187c0b27e0 0x7f187c0b4ca0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1894102a40 tx=0x7f1884005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:12.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.346+0000 7f187bfff640 1 -- 172.21.15.33:0/554429191 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f18880ca960 con 0x7f1894102be0 2024-05-15T14:00:12.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:12 smithi033 ceph-mon[31650]: pgmap v375: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T14:00:12.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:12.589+0000 7f1899472640 1 -- 172.21.15.33:0/554429191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f18941a36f0 con 0x7f1894102be0 2024-05-15T14:00:12.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:12 smithi052 ceph-mon[36190]: pgmap v375: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T14:00:12.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:12 smithi081 ceph-mon[36434]: pgmap v375: 97 pgs: 97 active+clean; 640 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T14:00:13.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:13.421+0000 7f187bfff640 1 -- 172.21.15.33:0/554429191 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f1888003e60 con 0x7f1894102be0 2024-05-15T14:00:13.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:13.424+0000 7f1899472640 1 -- 172.21.15.33:0/554429191 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f187c0b27e0 msgr2=0x7f187c0b4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:13.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:13.424+0000 7f1899472640 1 --2- 172.21.15.33:0/554429191 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f187c0b27e0 0x7f187c0b4ca0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1894102a40 tx=0x7f1884005f90 comp rx=0 tx=0).stop 2024-05-15T14:00:13.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:13.424+0000 7f1899472640 1 -- 172.21.15.33:0/554429191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1894102be0 msgr2=0x7f1894103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:13.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:13.424+0000 7f1899472640 1 --2- 172.21.15.33:0/554429191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1894102be0 0x7f1894103fc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1888005940 tx=0x7f18880042f0 comp rx=0 tx=0).stop 2024-05-15T14:00:13.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:13.424+0000 7f1899472640 1 -- 172.21.15.33:0/554429191 shutdown_connections 2024-05-15T14:00:13.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:13.424+0000 7f1899472640 1 --2- 172.21.15.33:0/554429191 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f187c0b27e0 0x7f187c0b4ca0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:13.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:13.424+0000 7f1899472640 1 --2- 172.21.15.33:0/554429191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18941035a0 0x7f1894104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:13.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:13.424+0000 7f1899472640 1 --2- 172.21.15.33:0/554429191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1894102be0 0x7f1894103fc0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:13.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:13.424+0000 7f1899472640 1 --2- 172.21.15.33:0/554429191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f18941019f0 0x7f1894105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:13.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:13.424+0000 7f1899472640 1 -- 172.21.15.33:0/554429191 >> 172.21.15.33:0/554429191 conn(0x7f18940fcf40 msgr2=0x7f189406b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:13.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:13.424+0000 7f1899472640 1 -- 172.21.15.33:0/554429191 shutdown_connections 2024-05-15T14:00:13.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:13.424+0000 7f1899472640 1 -- 172.21.15.33:0/554429191 wait complete. 2024-05-15T14:00:13.446 INFO:teuthology.run_tasks:Running task kclient... 2024-05-15T14:00:13.470 INFO:tasks.kclient:Mounting kernel clients... 2024-05-15T14:00:13.470 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-15T14:00:13.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T14:00:13.470 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-05-15T14:00:13.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T14:00:13.488 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link delete ceph-brx 2024-05-15T14:00:13.557 INFO:teuthology.orchestra.run.smithi033.stderr:Cannot find device "ceph-brx" 2024-05-15T14:00:13.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-15T14:00:13.559 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-05-15T14:00:13.559 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-15T14:00:13.559 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-15T14:00:13.560 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs ls 2024-05-15T14:00:13.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:13 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/554429191' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-15T14:00:13.691 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:13 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: ignoring --setuser ceph since I am not root 2024-05-15T14:00:13.691 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:13 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: ignoring --setgroup ceph since I am not root 2024-05-15T14:00:13.691 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:13 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:13.616+0000 7f2a72d29200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-15T14:00:13.793 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:13 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/554429191' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-15T14:00:13.794 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 14:00:13 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: ignoring --setuser ceph since I am not root 2024-05-15T14:00:13.794 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 14:00:13 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: ignoring --setgroup ceph since I am not root 2024-05-15T14:00:13.794 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 14:00:13 smithi081 ceph-mgr[38151]: -- 172.21.15.81:0/4153105605 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556fd41345a0 con 0x556fd4108800 2024-05-15T14:00:13.794 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 14:00:13 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T14:00:13.606+0000 7f2b1a230200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-15T14:00:13.794 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 14:00:13 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T14:00:13.790+0000 7f2b1a230200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-15T14:00:13.803 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:13 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/554429191' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-15T14:00:13.803 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 14:00:13 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: ignoring --setuser ceph since I am not root 2024-05-15T14:00:13.804 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 14:00:13 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: ignoring --setgroup ceph since I am not root 2024-05-15T14:00:13.804 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 14:00:13 smithi052 ceph-mgr[37581]: -- 172.21.15.52:0/79244600 <== mon.2 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f9aaa845a0 con 0x55f9aaa58800 2024-05-15T14:00:13.804 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 14:00:13 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T14:00:13.609+0000 7f354ffc9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-15T14:00:13.804 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 14:00:13 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T14:00:13.802+0000 7f354ffc9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-15T14:00:13.852 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:00:14.090 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 14:00:13 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T14:00:13.909+0000 7f354ffc9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-15T14:00:14.095 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:13 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:13.820+0000 7f2a72d29200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-15T14:00:14.095 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:13 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:13.942+0000 7f2a72d29200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-15T14:00:14.106 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 14:00:13 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T14:00:13.890+0000 7f2b1a230200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-15T14:00:14.372 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:14 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:14.278+0000 7f2a72d29200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-15T14:00:14.372 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:14 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:14.370+0000 7f2a72d29200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-15T14:00:14.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:14 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/554429191' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-15T14:00:14.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:14 smithi052 ceph-mon[36190]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-15T14:00:14.590 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 14:00:14 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T14:00:14.233+0000 7f354ffc9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-15T14:00:14.591 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 14:00:14 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T14:00:14.322+0000 7f354ffc9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-15T14:00:14.609 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:14 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/554429191' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-15T14:00:14.609 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:14 smithi081 ceph-mon[36434]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-15T14:00:14.609 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 14:00:14 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T14:00:14.197+0000 7f2b1a230200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-15T14:00:14.610 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 14:00:14 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T14:00:14.281+0000 7f2b1a230200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-15T14:00:14.644 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:14 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/554429191' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-15T14:00:14.644 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:14 smithi033 ceph-mon[31650]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-15T14:00:15.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.450+0000 7f5e15788700 1 -- 172.21.15.33:0/3472619633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e10102df0 msgr2=0x7f5e1010f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:15.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.450+0000 7f5e15788700 1 --2- 172.21.15.33:0/3472619633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e10102df0 0x7f5e1010f2e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5e0800b0a0 tx=0x7f5e0800b3b0 comp rx=0 tx=0).stop 2024-05-15T14:00:15.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.450+0000 7f5e15788700 1 -- 172.21.15.33:0/3472619633 shutdown_connections 2024-05-15T14:00:15.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.450+0000 7f5e15788700 1 --2- 172.21.15.33:0/3472619633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e10102df0 0x7f5e1010f2e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:15.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.450+0000 7f5e15788700 1 --2- 172.21.15.33:0/3472619633 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e10108510 0x7f5e101088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:15.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.450+0000 7f5e15788700 1 --2- 172.21.15.33:0/3472619633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e10102430 0x7f5e101028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:15.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.450+0000 7f5e15788700 1 -- 172.21.15.33:0/3472619633 >> 172.21.15.33:0/3472619633 conn(0x7f5e100793b0 msgr2=0x7f5e100797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:15.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.450+0000 7f5e15788700 1 -- 172.21.15.33:0/3472619633 shutdown_connections 2024-05-15T14:00:15.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.450+0000 7f5e15788700 1 -- 172.21.15.33:0/3472619633 wait complete. 2024-05-15T14:00:15.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.451+0000 7f5e15788700 1 Processor -- start 2024-05-15T14:00:15.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.451+0000 7f5e15788700 1 -- start start 2024-05-15T14:00:15.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.451+0000 7f5e15788700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e10102430 0x7f5e1019a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:15.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.451+0000 7f5e15788700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e10102df0 0x7f5e1019aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:15.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.451+0000 7f5e15788700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e10108510 0x7f5e101a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:15.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.451+0000 7f5e15788700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5e101125c0 con 0x7f5e10102df0 2024-05-15T14:00:15.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.451+0000 7f5e15788700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5e10112440 con 0x7f5e10108510 2024-05-15T14:00:15.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.451+0000 7f5e15788700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f5e10112740 con 0x7f5e10102430 2024-05-15T14:00:15.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.451+0000 7f5e0effd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e10102430 0x7f5e1019a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:15.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.451+0000 7f5e0e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e10102df0 0x7f5e1019aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:15.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.451+0000 7f5e0effd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e10102430 0x7f5e1019a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:57102/0 (socket says 172.21.15.33:57102) 2024-05-15T14:00:15.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.451+0000 7f5e0effd700 1 -- 172.21.15.33:0/2833998112 learned_addr learned my addr 172.21.15.33:0/2833998112 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:15.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.451+0000 7f5e0e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e10102df0 0x7f5e1019aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43988/0 (socket says 172.21.15.33:43988) 2024-05-15T14:00:15.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.451+0000 7f5e0f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e10108510 0x7f5e101a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:15.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.452+0000 7f5e0effd700 1 -- 172.21.15.33:0/2833998112 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e10108510 msgr2=0x7f5e101a5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:15.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.452+0000 7f5e0effd700 1 --2- 172.21.15.33:0/2833998112 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e10108510 0x7f5e101a5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:15.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.452+0000 7f5e0effd700 1 -- 172.21.15.33:0/2833998112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e10102df0 msgr2=0x7f5e1019aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:15.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.452+0000 7f5e0effd700 1 --2- 172.21.15.33:0/2833998112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e10102df0 0x7f5e1019aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:15.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.452+0000 7f5e0effd700 1 -- 172.21.15.33:0/2833998112 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e101a5f60 con 0x7f5e10102430 2024-05-15T14:00:15.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.452+0000 7f5e0effd700 1 --2- 172.21.15.33:0/2833998112 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e10102430 0x7f5e1019a9a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5e0000e9a0 tx=0x7f5e0000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:15.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.452+0000 7f5df7fff700 1 -- 172.21.15.33:0/2833998112 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0000ca10 con 0x7f5e10102430 2024-05-15T14:00:15.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.452+0000 7f5df7fff700 1 -- 172.21.15.33:0/2833998112 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e0000cbb0 con 0x7f5e10102430 2024-05-15T14:00:15.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.452+0000 7f5df7fff700 1 -- 172.21.15.33:0/2833998112 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e00018680 con 0x7f5e10102430 2024-05-15T14:00:15.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.452+0000 7f5e15788700 1 -- 172.21.15.33:0/2833998112 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e101a6190 con 0x7f5e10102430 2024-05-15T14:00:15.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.452+0000 7f5e15788700 1 -- 172.21.15.33:0/2833998112 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e101a6650 con 0x7f5e10102430 2024-05-15T14:00:15.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.453+0000 7f5e15788700 1 -- 172.21.15.33:0/2833998112 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e1006a230 con 0x7f5e10102430 2024-05-15T14:00:15.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.454+0000 7f5df7fff700 1 -- 172.21.15.33:0/2833998112 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f5e00010430 con 0x7f5e10102430 2024-05-15T14:00:15.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.455+0000 7f5df7fff700 1 --2- 172.21.15.33:0/2833998112 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f5df80b28d0 0x7f5df80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:15.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.456+0000 7f5e0e7fc700 1 -- 172.21.15.33:0/2833998112 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f5df80b28d0 msgr2=0x7f5df80b4d90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1797827152 2024-05-15T14:00:15.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.456+0000 7f5df7fff700 1 -- 172.21.15.33:0/2833998112 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f5e000e26c0 con 0x7f5e10102430 2024-05-15T14:00:15.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.456+0000 7f5e0e7fc700 1 --2- 172.21.15.33:0/2833998112 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f5df80b28d0 0x7f5df80b4d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-15T14:00:15.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.458+0000 7f5df7fff700 1 -- 172.21.15.33:0/2833998112 <== mon.1 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5e000aa3a0 con 0x7f5e10102430 2024-05-15T14:00:15.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.657+0000 7f5e0e7fc700 1 -- 172.21.15.33:0/2833998112 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f5df80b28d0 msgr2=0x7f5df80b4d90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1797827152 2024-05-15T14:00:15.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.657+0000 7f5e0e7fc700 1 --2- 172.21.15.33:0/2833998112 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f5df80b28d0 0x7f5df80b4d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-15T14:00:15.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.722+0000 7f5e15788700 1 -- 172.21.15.33:0/2833998112 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5e1002e720 con 0x7f5e10102430 2024-05-15T14:00:15.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.722+0000 7f5df7fff700 1 -- 172.21.15.33:0/2833998112 <== mon.1 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) ==== 53+0+73 (secure 0 0 0) 0x7f5e000b2420 con 0x7f5e10102430 2024-05-15T14:00:15.724 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-15T14:00:15.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.726+0000 7f5e15788700 1 -- 172.21.15.33:0/2833998112 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f5df80b28d0 msgr2=0x7f5df80b4d90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-15T14:00:15.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.726+0000 7f5e15788700 1 --2- 172.21.15.33:0/2833998112 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f5df80b28d0 0x7f5df80b4d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:15.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.726+0000 7f5e15788700 1 -- 172.21.15.33:0/2833998112 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e10102430 msgr2=0x7f5e1019a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:15.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.726+0000 7f5e15788700 1 --2- 172.21.15.33:0/2833998112 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e10102430 0x7f5e1019a9a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5e0000e9a0 tx=0x7f5e0000ecb0 comp rx=0 tx=0).stop 2024-05-15T14:00:15.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.726+0000 7f5e15788700 1 -- 172.21.15.33:0/2833998112 shutdown_connections 2024-05-15T14:00:15.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.726+0000 7f5e15788700 1 --2- 172.21.15.33:0/2833998112 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f5df80b28d0 0x7f5df80b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:15.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.726+0000 7f5e15788700 1 --2- 172.21.15.33:0/2833998112 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e10108510 0x7f5e101a5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:15.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.726+0000 7f5e15788700 1 --2- 172.21.15.33:0/2833998112 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e10102430 0x7f5e1019a9a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:15.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.726+0000 7f5e15788700 1 --2- 172.21.15.33:0/2833998112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e10102df0 0x7f5e1019aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:15.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.726+0000 7f5e15788700 1 -- 172.21.15.33:0/2833998112 >> 172.21.15.33:0/2833998112 conn(0x7f5e100793b0 msgr2=0x7f5e10100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:15.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.727+0000 7f5e15788700 1 -- 172.21.15.33:0/2833998112 shutdown_connections 2024-05-15T14:00:15.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:15.727+0000 7f5e15788700 1 -- 172.21.15.33:0/2833998112 wait complete. 2024-05-15T14:00:15.775 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 14:00:15 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T14:00:15.451+0000 7f2b1a230200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-15T14:00:15.775 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 14:00:15 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T14:00:15.620+0000 7f2b1a230200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-15T14:00:15.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:15 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/2833998112' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-15T14:00:15.840 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 14:00:15 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T14:00:15.574+0000 7f354ffc9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-15T14:00:15.840 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 14:00:15 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T14:00:15.751+0000 7f354ffc9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-15T14:00:15.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:15 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/2833998112' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-15T14:00:15.941 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:15 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:15.663+0000 7f2a72d29200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-15T14:00:15.941 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:15 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:15.857+0000 7f2a72d29200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-15T14:00:16.076 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:15 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/2833998112' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-15T14:00:16.076 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 14:00:15 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T14:00:15.789+0000 7f2b1a230200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-15T14:00:16.232 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 14:00:15 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T14:00:15.929+0000 7f354ffc9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-15T14:00:16.348 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:16 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:16.038+0000 7f2a72d29200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-15T14:00:16.356 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 14:00:16 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T14:00:16.074+0000 7f2b1a230200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-15T14:00:16.356 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 14:00:16 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T14:00:16.166+0000 7f2b1a230200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-15T14:00:16.475 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-15T14:00:16.475 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-15T14:00:16.475 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2024-05-15T14:00:16.475 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-15T14:00:16.475 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-15T14:00:16.475 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-15T14:00:16.476 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-15T14:00:16.476 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-15T14:00:16.476 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-15T14:00:16.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T14:00:16.476 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2024-05-15T14:00:16.495 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-15T14:00:16.495 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-15T14:00:16.495 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2024-05-15T14:00:16.495 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-05-15T14:00:16.495 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2024-05-15T14:00:16.495 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-05-15T14:00:16.495 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-15T14:00:16.495 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2024-05-15T14:00:16.495 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2024-05-15T14:00:16.496 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-15T14:00:16.496 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2024-05-15T14:00:16.496 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2024-05-15T14:00:16.496 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-15T14:00:16.496 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2024-05-15T14:00:16.496 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2024-05-15T14:00:16.496 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-15T14:00:16.496 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 42061sec preferred_lft 42061sec 2024-05-15T14:00:16.496 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link noprefixroute 2024-05-15T14:00:16.496 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-05-15T14:00:16.496 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-15T14:00:16.496 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2024-05-15T14:00:16.496 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2024-05-15T14:00:16.497 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-15T14:00:16.497 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-15T14:00:16.497 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-05-15T14:00:16.497 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add name ceph-brx type bridge 2024-05-15T14:00:16.497 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr flush dev ceph-brx 2024-05-15T14:00:16.497 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set ceph-brx up 2024-05-15T14:00:16.497 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-15T14:00:16.497 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-05-15T14:00:16.539 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 14:00:16 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T14:00:16.231+0000 7f354ffc9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-15T14:00:16.540 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 14:00:16 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T14:00:16.326+0000 7f354ffc9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-15T14:00:16.540 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 14:00:16 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T14:00:16.538+0000 7f354ffc9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-15T14:00:16.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-15T14:00:16.599 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:16 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:16.347+0000 7f2a72d29200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-15T14:00:16.599 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:16 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:16.445+0000 7f2a72d29200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-15T14:00:16.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-15T14:00:16.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T14:00:16.701 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-15T14:00:16.728 INFO:teuthology.orchestra.run.smithi033.stdout:1 2024-05-15T14:00:16.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T14:00:16.730 DEBUG:teuthology.orchestra.run.smithi033:> route 2024-05-15T14:00:16.801 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2024-05-15T14:00:16.801 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-15T14:00:16.801 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-15T14:00:16.801 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-15T14:00:16.801 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-15T14:00:16.802 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-15T14:00:16.802 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-05-15T14:00:16.802 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-15T14:00:16.802 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-15T14:00:16.802 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-15T14:00:16.802 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-05-15T14:00:16.840 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 14:00:16 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T14:00:16.739+0000 7f354ffc9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-15T14:00:16.856 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 14:00:16 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T14:00:16.365+0000 7f2b1a230200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-15T14:00:16.856 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 14:00:16 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T14:00:16.558+0000 7f2b1a230200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-15T14:00:16.885 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:16 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:16.670+0000 7f2a72d29200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-15T14:00:16.885 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:16 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:16.883+0000 7f2a72d29200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-15T14:00:16.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-15T14:00:17.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-15T14:00:17.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T14:00:17.099 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-05-15T14:00:17.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T14:00:17.116 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list-id 2024-05-15T14:00:17.171 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-15T14:00:17.172 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-05-15T14:00:17.172 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-15T14:00:17.172 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-15T14:00:17.172 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-05-15T14:00:17.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-15T14:00:17.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-15T14:00:17.294 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-15T14:00:17.294 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-15T14:00:17.294 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-05-15T14:00:17.294 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-15T14:00:17.295 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-15T14:00:17.295 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-15T14:00:17.295 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-15T14:00:17.295 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-15T14:00:17.295 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-05-15T14:00:17.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-15T14:00:17.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-15T14:00:17.434 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-15T14:00:17.435 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-05-15T14:00:17.435 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2024-05-15T14:00:17.435 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set dev brx.0 master ceph-brx 2024-05-15T14:00:17.435 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-05-15T14:00:17.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-15T14:00:17.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-15T14:00:17.517 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-15T14:00:17.517 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-15T14:00:17.533 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-15T14:00:17.533 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-15T14:00:17.533 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-15T14:00:17.589 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-15T14:00:17.589 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-15T14:00:17.589 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/ee5a4962-7db7-4578-bad4-70ba3aff515c 2024-05-15T14:00:17.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T14:00:17.589 DEBUG:teuthology.orchestra.run.smithi033:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/ee5a4962-7db7-4578-bad4-70ba3aff515c /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-05-15T14:00:17.607 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 14:00:17 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T14:00:17.152+0000 7f2b1a230200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-15T14:00:17.607 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 14:00:17 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T14:00:17.369+0000 7f2b1a230200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-15T14:00:17.682 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-05-15T14:00:17.682 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-05-15T14:00:17.682 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2024-05-15T14:00:17.803 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-05-15T14:00:17.803 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-05-15T14:00:17.803 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2024-05-15T14:00:17.803 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.52:3300,172.21.15.81:3300" 2024-05-15T14:00:17.803 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:3300,172.21.15.52:3300,172.21.15.81:3300:/volumes/qa/sv_1/ee5a4962-7db7-4578-bad4-70ba3aff515c 2024-05-15T14:00:17.803 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=bca78ca4-12c1-11ef-bc98-c7b262605968" will pass to kernel 2024-05-15T14:00:17.804 INFO:teuthology.orchestra.run.smithi033.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-15T14:00:17.804 INFO:teuthology.orchestra.run.smithi033.stdout: You just mounted a file system that supports labels which does not 2024-05-15T14:00:17.804 INFO:teuthology.orchestra.run.smithi033.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-15T14:00:17.804 INFO:teuthology.orchestra.run.smithi033.stdout: applications will generate AVC messages and not be allowed access to 2024-05-15T14:00:17.804 INFO:teuthology.orchestra.run.smithi033.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-15T14:00:17.806 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-15T14:00:17.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T14:00:17.807 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-15T14:00:17.829 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:17 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:17.559+0000 7f2a72d29200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-15T14:00:17.829 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:17 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:17.802+0000 7f2a72d29200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-15T14:00:17.834 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-15T14:00:17.840 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 14:00:17 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T14:00:17.369+0000 7f354ffc9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-15T14:00:17.840 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 14:00:17 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T14:00:17.599+0000 7f354ffc9200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-15T14:00:17.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-15T14:00:17.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-15T14:00:17.933 INFO:teuthology.orchestra.run.smithi033.stdout:client24833 2024-05-15T14:00:17.934 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-15T14:00:17.934 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-15T14:00:17.951 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-15T14:00:18.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-15T14:00:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-15T14:00:18.051 INFO:teuthology.orchestra.run.smithi033.stdout:client24833 2024-05-15T14:00:18.051 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-15T14:00:18.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-15T14:00:18.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-15T14:00:18.106 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 14:00:17 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T14:00:17.788+0000 7f2b1a230200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-15T14:00:18.106 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 14:00:17 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T14:00:17.886+0000 7f2b1a230200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-15T14:00:18.110 INFO:teuthology.orchestra.run.smithi033.stdout:client24833 2024-05-15T14:00:18.110 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968.client24833/status') 2024-05-15T14:00:18.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-15T14:00:18.171 INFO:teuthology.orchestra.run.smithi033.stdout:instance: client.24833 (3)172.21.15.33:0/3173579709 2024-05-15T14:00:18.172 INFO:teuthology.orchestra.run.smithi033.stdout:blocklisted: false 2024-05-15T14:00:18.172 INFO:teuthology.orchestra.run.smithi033.stderr:0+1 records in 2024-05-15T14:00:18.172 INFO:teuthology.orchestra.run.smithi033.stderr:0+1 records out 2024-05-15T14:00:18.172 INFO:teuthology.orchestra.run.smithi033.stderr:71 bytes copied, 2.5514e-05 s, 2.8 MB/s 2024-05-15T14:00:18.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-15T14:00:18.181 INFO:teuthology.run_tasks:Running task exec... 2024-05-15T14:00:18.190 INFO:teuthology.task.exec:Executing custom commands... 2024-05-15T14:00:18.190 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-05-15T14:00:18.190 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-05-15T14:00:18.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.296+0000 7f2dac39d640 1 -- 172.21.15.33:0/2918489081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2da41035d0 msgr2=0x7f2da41059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:18.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.296+0000 7f2dac39d640 1 --2- 172.21.15.33:0/2918489081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2da41035d0 0x7f2da41059c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2da0009900 tx=0x7f2da002a410 comp rx=0 tx=0).stop 2024-05-15T14:00:18.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.297+0000 7f2dac39d640 1 -- 172.21.15.33:0/2918489081 shutdown_connections 2024-05-15T14:00:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.297+0000 7f2dac39d640 1 --2- 172.21.15.33:0/2918489081 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2da4108830 0x7f2da410ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.297+0000 7f2dac39d640 1 --2- 172.21.15.33:0/2918489081 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2da4105f00 0x7f2da41082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.297+0000 7f2dac39d640 1 --2- 172.21.15.33:0/2918489081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2da41035d0 0x7f2da41059c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.297+0000 7f2dac39d640 1 -- 172.21.15.33:0/2918489081 >> 172.21.15.33:0/2918489081 conn(0x7f2da40fcf20 msgr2=0x7f2da40ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.297+0000 7f2dac39d640 1 -- 172.21.15.33:0/2918489081 shutdown_connections 2024-05-15T14:00:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.297+0000 7f2dac39d640 1 -- 172.21.15.33:0/2918489081 wait complete. 2024-05-15T14:00:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.297+0000 7f2dac39d640 1 Processor -- start 2024-05-15T14:00:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.297+0000 7f2dac39d640 1 -- start start 2024-05-15T14:00:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.297+0000 7f2dac39d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2da41035d0 0x7f2da4198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.297+0000 7f2dac39d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2da4105f00 0x7f2da4199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.297+0000 7f2dac39d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2da4108830 0x7f2da41a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.297+0000 7f2dac39d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2da410e5f0 con 0x7f2da4105f00 2024-05-15T14:00:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.297+0000 7f2dac39d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2da410e470 con 0x7f2da4108830 2024-05-15T14:00:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.297+0000 7f2dac39d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2da410e770 con 0x7f2da41035d0 2024-05-15T14:00:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.298+0000 7f2da9938640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2da4105f00 0x7f2da4199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.298+0000 7f2daa139640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2da41035d0 0x7f2da4198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.298+0000 7f2da9938640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2da4105f00 0x7f2da4199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43996/0 (socket says 172.21.15.33:43996) 2024-05-15T14:00:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.298+0000 7f2da9938640 1 -- 172.21.15.33:0/225218347 learned_addr learned my addr 172.21.15.33:0/225218347 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.298+0000 7f2daa93a640 1 --2- 172.21.15.33:0/225218347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2da4108830 0x7f2da41a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.298+0000 7f2da9938640 1 -- 172.21.15.33:0/225218347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2da41035d0 msgr2=0x7f2da4198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.298+0000 7f2da9938640 1 --2- 172.21.15.33:0/225218347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2da41035d0 0x7f2da4198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.298+0000 7f2da9938640 1 -- 172.21.15.33:0/225218347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2da4108830 msgr2=0x7f2da41a02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.298+0000 7f2da9938640 1 --2- 172.21.15.33:0/225218347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2da4108830 0x7f2da41a02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.298+0000 7f2da9938640 1 -- 172.21.15.33:0/225218347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da41a0950 con 0x7f2da4105f00 2024-05-15T14:00:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.298+0000 7f2daa139640 1 --2- 172.21.15.33:0/225218347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2da41035d0 0x7f2da4198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:00:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.298+0000 7f2da9938640 1 --2- 172.21.15.33:0/225218347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2da4105f00 0x7f2da4199280 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2d9400e9c0 tx=0x7f2d9400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.298+0000 7f2d9b7fe640 1 -- 172.21.15.33:0/225218347 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9400cd80 con 0x7f2da4105f00 2024-05-15T14:00:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.298+0000 7f2dac39d640 1 -- 172.21.15.33:0/225218347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2da41a0c40 con 0x7f2da4105f00 2024-05-15T14:00:18.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.298+0000 7f2d9b7fe640 1 -- 172.21.15.33:0/225218347 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d940054d0 con 0x7f2da4105f00 2024-05-15T14:00:18.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.298+0000 7f2d9b7fe640 1 -- 172.21.15.33:0/225218347 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d94010430 con 0x7f2da4105f00 2024-05-15T14:00:18.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.298+0000 7f2dac39d640 1 -- 172.21.15.33:0/225218347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2da41a1190 con 0x7f2da4105f00 2024-05-15T14:00:18.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.299+0000 7f2dac39d640 1 -- 172.21.15.33:0/225218347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2da40fea10 con 0x7f2da4105f00 2024-05-15T14:00:18.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.301+0000 7f2d9b7fe640 1 -- 172.21.15.33:0/225218347 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f2d94004fb0 con 0x7f2da4105f00 2024-05-15T14:00:18.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.302+0000 7f2d9b7fe640 1 --2- 172.21.15.33:0/225218347 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f2d900b2830 0x7f2d900b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:18.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.302+0000 7f2daa139640 1 -- 172.21.15.33:0/225218347 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f2d900b2830 msgr2=0x7f2d900b4cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1797827152 2024-05-15T14:00:18.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.302+0000 7f2d9b7fe640 1 -- 172.21.15.33:0/225218347 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f2d940e2920 con 0x7f2da4105f00 2024-05-15T14:00:18.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.302+0000 7f2daa139640 1 --2- 172.21.15.33:0/225218347 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f2d900b2830 0x7f2d900b4cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-15T14:00:18.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.306+0000 7f2d9b7fe640 1 -- 172.21.15.33:0/225218347 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2d94054df0 con 0x7f2da4105f00 2024-05-15T14:00:18.340 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 14:00:18 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T14:00:18.041+0000 7f354ffc9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-15T14:00:18.340 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 14:00:18 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T14:00:18.141+0000 7f354ffc9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-15T14:00:18.440 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:18 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:18.272+0000 7f2a72d29200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-15T14:00:18.441 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:18 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:18.380+0000 7f2a72d29200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-15T14:00:18.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.503+0000 7f2daa139640 1 -- 172.21.15.33:0/225218347 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f2d900b2830 msgr2=0x7f2d900b4cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1797827152 2024-05-15T14:00:18.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.503+0000 7f2daa139640 1 --2- 172.21.15.33:0/225218347 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f2d900b2830 0x7f2d900b4cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-15T14:00:18.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.542+0000 7f2dac39d640 1 -- 172.21.15.33:0/225218347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f2da402dba0 con 0x7f2da4105f00 2024-05-15T14:00:18.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.594+0000 7f2d9b7fe640 1 -- 172.21.15.33:0/225218347 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v24) ==== 117+0+0 (secure 0 0 0) 0x7f2d940aa640 con 0x7f2da4105f00 2024-05-15T14:00:18.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.596+0000 7f2dac39d640 1 -- 172.21.15.33:0/225218347 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f2d900b2830 msgr2=0x7f2d900b4cf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-15T14:00:18.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.596+0000 7f2dac39d640 1 --2- 172.21.15.33:0/225218347 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f2d900b2830 0x7f2d900b4cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:18.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.596+0000 7f2dac39d640 1 -- 172.21.15.33:0/225218347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2da4105f00 msgr2=0x7f2da4199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:18.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.596+0000 7f2dac39d640 1 --2- 172.21.15.33:0/225218347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2da4105f00 0x7f2da4199280 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2d9400e9c0 tx=0x7f2d9400ee80 comp rx=0 tx=0).stop 2024-05-15T14:00:18.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.596+0000 7f2dac39d640 1 -- 172.21.15.33:0/225218347 shutdown_connections 2024-05-15T14:00:18.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.596+0000 7f2dac39d640 1 --2- 172.21.15.33:0/225218347 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f2d900b2830 0x7f2d900b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:18.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.596+0000 7f2dac39d640 1 --2- 172.21.15.33:0/225218347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2da4108830 0x7f2da41a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:18.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.596+0000 7f2dac39d640 1 --2- 172.21.15.33:0/225218347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2da4105f00 0x7f2da4199280 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:18.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.596+0000 7f2dac39d640 1 --2- 172.21.15.33:0/225218347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2da41035d0 0x7f2da4198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:18.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.596+0000 7f2dac39d640 1 -- 172.21.15.33:0/225218347 >> 172.21.15.33:0/225218347 conn(0x7f2da40fcf20 msgr2=0x7f2da4103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:18.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.597+0000 7f2dac39d640 1 -- 172.21.15.33:0/225218347 shutdown_connections 2024-05-15T14:00:18.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.597+0000 7f2dac39d640 1 -- 172.21.15.33:0/225218347 wait complete. 2024-05-15T14:00:18.599 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 14:00:18 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T14:00:18.250+0000 7f2b1a230200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-15T14:00:18.610 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-15T14:00:18.621 INFO:teuthology.run_tasks:Running task exec... 2024-05-15T14:00:18.629 INFO:teuthology.task.exec:Executing custom commands... 2024-05-15T14:00:18.629 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-05-15T14:00:18.629 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-05-15T14:00:18.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.736+0000 7f6f3b209640 1 -- 172.21.15.33:0/4154758084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f341019b0 msgr2=0x7f6f34101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:18.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.736+0000 7f6f3b209640 1 --2- 172.21.15.33:0/4154758084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f341019b0 0x7f6f34101dd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6f28009900 tx=0x7f6f2802a410 comp rx=0 tx=0).stop 2024-05-15T14:00:18.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.736+0000 7f6f3b209640 1 -- 172.21.15.33:0/4154758084 shutdown_connections 2024-05-15T14:00:18.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.736+0000 7f6f3b209640 1 --2- 172.21.15.33:0/4154758084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f34103560 0x7f6f34109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:18.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.736+0000 7f6f3b209640 1 --2- 172.21.15.33:0/4154758084 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f34102ba0 0x7f6f34103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:18.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.736+0000 7f6f3b209640 1 --2- 172.21.15.33:0/4154758084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f341019b0 0x7f6f34101dd0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:18.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.736+0000 7f6f3b209640 1 -- 172.21.15.33:0/4154758084 >> 172.21.15.33:0/4154758084 conn(0x7f6f340fcf20 msgr2=0x7f6f340ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:18.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.736+0000 7f6f3b209640 1 -- 172.21.15.33:0/4154758084 shutdown_connections 2024-05-15T14:00:18.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.736+0000 7f6f3b209640 1 -- 172.21.15.33:0/4154758084 wait complete. 2024-05-15T14:00:18.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.736+0000 7f6f3b209640 1 Processor -- start 2024-05-15T14:00:18.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.736+0000 7f6f3b209640 1 -- start start 2024-05-15T14:00:18.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.737+0000 7f6f3b209640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f341019b0 0x7f6f34105940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:18.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.737+0000 7f6f3b209640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f34102ba0 0x7f6f34103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:18.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.737+0000 7f6f3b209640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f34103560 0x7f6f341044c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:18.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.737+0000 7f6f3b209640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3410d520 con 0x7f6f34102ba0 2024-05-15T14:00:18.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.737+0000 7f6f3b209640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3410d3a0 con 0x7f6f34103560 2024-05-15T14:00:18.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.737+0000 7f6f3b209640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3410d6a0 con 0x7f6f341019b0 2024-05-15T14:00:18.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.737+0000 7f6f33fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f34102ba0 0x7f6f34103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:18.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.737+0000 7f6f38fa5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f341019b0 0x7f6f34105940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:18.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.737+0000 7f6f397a6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f34103560 0x7f6f341044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:18.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.737+0000 7f6f38fa5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f341019b0 0x7f6f34105940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:57140/0 (socket says 172.21.15.33:57140) 2024-05-15T14:00:18.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.737+0000 7f6f38fa5640 1 -- 172.21.15.33:0/937211174 learned_addr learned my addr 172.21.15.33:0/937211174 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:18.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.737+0000 7f6f33fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f34102ba0 0x7f6f34103f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44022/0 (socket says 172.21.15.33:44022) 2024-05-15T14:00:18.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.737+0000 7f6f38fa5640 1 -- 172.21.15.33:0/937211174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f34103560 msgr2=0x7f6f341044c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:18.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.737+0000 7f6f38fa5640 1 --2- 172.21.15.33:0/937211174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f34103560 0x7f6f341044c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:18.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.737+0000 7f6f38fa5640 1 -- 172.21.15.33:0/937211174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f34102ba0 msgr2=0x7f6f34103f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:18.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.737+0000 7f6f38fa5640 1 --2- 172.21.15.33:0/937211174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f34102ba0 0x7f6f34103f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:18.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.737+0000 7f6f38fa5640 1 -- 172.21.15.33:0/937211174 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f34104d50 con 0x7f6f341019b0 2024-05-15T14:00:18.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.737+0000 7f6f38fa5640 1 --2- 172.21.15.33:0/937211174 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f341019b0 0x7f6f34105940 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6f28031500 tx=0x7f6f28003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:18.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.737+0000 7f6f31ffb640 1 -- 172.21.15.33:0/937211174 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2803f070 con 0x7f6f341019b0 2024-05-15T14:00:18.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.737+0000 7f6f31ffb640 1 -- 172.21.15.33:0/937211174 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f28031c50 con 0x7f6f341019b0 2024-05-15T14:00:18.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.738+0000 7f6f3b209640 1 -- 172.21.15.33:0/937211174 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f341a2e90 con 0x7f6f341019b0 2024-05-15T14:00:18.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.738+0000 7f6f31ffb640 1 -- 172.21.15.33:0/937211174 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f28035030 con 0x7f6f341019b0 2024-05-15T14:00:18.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.738+0000 7f6f3b209640 1 -- 172.21.15.33:0/937211174 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f341a32a0 con 0x7f6f341019b0 2024-05-15T14:00:18.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.739+0000 7f6f3b209640 1 -- 172.21.15.33:0/937211174 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f341a3530 con 0x7f6f341019b0 2024-05-15T14:00:18.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.740+0000 7f6f31ffb640 1 -- 172.21.15.33:0/937211174 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f6f28042070 con 0x7f6f341019b0 2024-05-15T14:00:18.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.741+0000 7f6f31ffb640 1 --2- 172.21.15.33:0/937211174 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f6f240b2830 0x7f6f240b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:18.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.741+0000 7f6f33fff640 1 -- 172.21.15.33:0/937211174 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f6f240b2830 msgr2=0x7f6f240b4cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1797827152 2024-05-15T14:00:18.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.741+0000 7f6f31ffb640 1 -- 172.21.15.33:0/937211174 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f6f28102f30 con 0x7f6f341019b0 2024-05-15T14:00:18.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.741+0000 7f6f33fff640 1 --2- 172.21.15.33:0/937211174 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f6f240b2830 0x7f6f240b4cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-15T14:00:18.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.744+0000 7f6f31ffb640 1 -- 172.21.15.33:0/937211174 <== mon.1 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6f28132df0 con 0x7f6f341019b0 2024-05-15T14:00:18.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:18 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/225218347' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-15T14:00:18.840 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 14:00:18 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T14:00:18.528+0000 7f354ffc9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-15T14:00:18.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:18 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/225218347' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-15T14:00:18.857 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 14:00:18 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T14:00:18.611+0000 7f2b1a230200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-15T14:00:18.857 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 14:00:18 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T14:00:18.704+0000 7f2b1a230200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-15T14:00:18.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.935+0000 7f6f3b209640 1 -- 172.21.15.33:0/937211174 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f6f341a3700 con 0x7f6f341019b0 2024-05-15T14:00:18.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:18 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/225218347' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-15T14:00:18.941 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:18 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:18.784+0000 7f2a72d29200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-15T14:00:18.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.941+0000 7f6f31ffb640 1 -- 172.21.15.33:0/937211174 <== mon.1 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v29)=0 v29) ==== 129+0+0 (secure 0 0 0) 0x7f6f280ca5c0 con 0x7f6f341019b0 2024-05-15T14:00:18.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.942+0000 7f6f33fff640 1 -- 172.21.15.33:0/937211174 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f6f240b2830 msgr2=0x7f6f240b4cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1797827152 2024-05-15T14:00:18.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.942+0000 7f6f33fff640 1 --2- 172.21.15.33:0/937211174 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f6f240b2830 0x7f6f240b4cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-15T14:00:18.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.943+0000 7f6f3b209640 1 -- 172.21.15.33:0/937211174 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f6f240b2830 msgr2=0x7f6f240b4cf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-15T14:00:18.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.943+0000 7f6f3b209640 1 --2- 172.21.15.33:0/937211174 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f6f240b2830 0x7f6f240b4cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:18.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.943+0000 7f6f3b209640 1 -- 172.21.15.33:0/937211174 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f341019b0 msgr2=0x7f6f34105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:18.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.943+0000 7f6f3b209640 1 --2- 172.21.15.33:0/937211174 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f341019b0 0x7f6f34105940 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6f28031500 tx=0x7f6f28003c10 comp rx=0 tx=0).stop 2024-05-15T14:00:18.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.943+0000 7f6f3b209640 1 -- 172.21.15.33:0/937211174 shutdown_connections 2024-05-15T14:00:18.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.943+0000 7f6f3b209640 1 --2- 172.21.15.33:0/937211174 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f6f240b2830 0x7f6f240b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:18.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.943+0000 7f6f3b209640 1 --2- 172.21.15.33:0/937211174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f34103560 0x7f6f341044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:18.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.943+0000 7f6f3b209640 1 --2- 172.21.15.33:0/937211174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f34102ba0 0x7f6f34103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:18.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.943+0000 7f6f3b209640 1 --2- 172.21.15.33:0/937211174 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f341019b0 0x7f6f34105940 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:18.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.943+0000 7f6f3b209640 1 -- 172.21.15.33:0/937211174 >> 172.21.15.33:0/937211174 conn(0x7f6f340fcf20 msgr2=0x7f6f3406a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:18.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.943+0000 7f6f3b209640 1 -- 172.21.15.33:0/937211174 shutdown_connections 2024-05-15T14:00:18.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:18.943+0000 7f6f3b209640 1 -- 172.21.15.33:0/937211174 wait complete. 2024-05-15T14:00:18.955 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-05-15T14:00:19.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.060+0000 7f7346ebd640 1 -- 172.21.15.33:0/357526435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7340103560 msgr2=0x7f7340109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:19.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.060+0000 7f7346ebd640 1 --2- 172.21.15.33:0/357526435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7340103560 0x7f7340109df0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7338009900 tx=0x7f733802a410 comp rx=0 tx=0).stop 2024-05-15T14:00:19.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.060+0000 7f7346ebd640 1 -- 172.21.15.33:0/357526435 shutdown_connections 2024-05-15T14:00:19.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.060+0000 7f7346ebd640 1 --2- 172.21.15.33:0/357526435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7340103560 0x7f7340109df0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:19.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.060+0000 7f7346ebd640 1 --2- 172.21.15.33:0/357526435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7340102ba0 0x7f7340103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:19.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.060+0000 7f7346ebd640 1 --2- 172.21.15.33:0/357526435 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73401019b0 0x7f7340101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:19.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.060+0000 7f7346ebd640 1 -- 172.21.15.33:0/357526435 >> 172.21.15.33:0/357526435 conn(0x7f73400fcf20 msgr2=0x7f73400ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:19.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.060+0000 7f7346ebd640 1 -- 172.21.15.33:0/357526435 shutdown_connections 2024-05-15T14:00:19.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.060+0000 7f7346ebd640 1 -- 172.21.15.33:0/357526435 wait complete. 2024-05-15T14:00:19.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.061+0000 7f7346ebd640 1 Processor -- start 2024-05-15T14:00:19.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.061+0000 7f7346ebd640 1 -- start start 2024-05-15T14:00:19.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.061+0000 7f7346ebd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73401019b0 0x7f7340105940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:19.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.061+0000 7f7346ebd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7340102ba0 0x7f7340103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:19.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.061+0000 7f7346ebd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7340103560 0x7f73401044c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:19.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.061+0000 7f7346ebd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f734010d4c0 con 0x7f73401019b0 2024-05-15T14:00:19.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.061+0000 7f7346ebd640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f734010d340 con 0x7f7340103560 2024-05-15T14:00:19.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.061+0000 7f7346ebd640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f734010d640 con 0x7f7340102ba0 2024-05-15T14:00:19.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.061+0000 7f7344c59640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73401019b0 0x7f7340105940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:19.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.061+0000 7f7344c59640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73401019b0 0x7f7340105940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44036/0 (socket says 172.21.15.33:44036) 2024-05-15T14:00:19.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.061+0000 7f734545a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7340103560 0x7f73401044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:19.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.061+0000 7f7344c59640 1 -- 172.21.15.33:0/3291968094 learned_addr learned my addr 172.21.15.33:0/3291968094 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:19.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.061+0000 7f733ffff640 1 --2- 172.21.15.33:0/3291968094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7340102ba0 0x7f7340103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:19.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.062+0000 7f7344c59640 1 -- 172.21.15.33:0/3291968094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7340102ba0 msgr2=0x7f7340103f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:19.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.062+0000 7f7344c59640 1 --2- 172.21.15.33:0/3291968094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7340102ba0 0x7f7340103f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:19.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.062+0000 7f7344c59640 1 -- 172.21.15.33:0/3291968094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7340103560 msgr2=0x7f73401044c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:19.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.062+0000 7f7344c59640 1 --2- 172.21.15.33:0/3291968094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7340103560 0x7f73401044c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:19.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.062+0000 7f7344c59640 1 -- 172.21.15.33:0/3291968094 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7340104d50 con 0x7f73401019b0 2024-05-15T14:00:19.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.062+0000 7f734545a640 1 --2- 172.21.15.33:0/3291968094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7340103560 0x7f73401044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:00:19.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.062+0000 7f7344c59640 1 --2- 172.21.15.33:0/3291968094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73401019b0 0x7f7340105940 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f732800e9c0 tx=0x7f732800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:19.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.062+0000 7f733dffb640 1 -- 172.21.15.33:0/3291968094 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732800cde0 con 0x7f73401019b0 2024-05-15T14:00:19.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.062+0000 7f733dffb640 1 -- 172.21.15.33:0/3291968094 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73280108e0 con 0x7f73401019b0 2024-05-15T14:00:19.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.062+0000 7f733dffb640 1 -- 172.21.15.33:0/3291968094 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7328018960 con 0x7f73401019b0 2024-05-15T14:00:19.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.062+0000 7f7346ebd640 1 -- 172.21.15.33:0/3291968094 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73401a2dd0 con 0x7f73401019b0 2024-05-15T14:00:19.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.062+0000 7f7346ebd640 1 -- 172.21.15.33:0/3291968094 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73401a3230 con 0x7f73401019b0 2024-05-15T14:00:19.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.064+0000 7f7346ebd640 1 -- 172.21.15.33:0/3291968094 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73401a3520 con 0x7f73401019b0 2024-05-15T14:00:19.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.065+0000 7f733dffb640 1 -- 172.21.15.33:0/3291968094 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f7328010a90 con 0x7f73401019b0 2024-05-15T14:00:19.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.065+0000 7f733dffb640 1 --2- 172.21.15.33:0/3291968094 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f732c0b28b0 0x7f732c0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:19.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.066+0000 7f733ffff640 1 -- 172.21.15.33:0/3291968094 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f732c0b28b0 msgr2=0x7f732c0b4d70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1797827152 2024-05-15T14:00:19.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.066+0000 7f733dffb640 1 -- 172.21.15.33:0/3291968094 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f73280e25f0 con 0x7f73401019b0 2024-05-15T14:00:19.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.066+0000 7f733ffff640 1 --2- 172.21.15.33:0/3291968094 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f732c0b28b0 0x7f732c0b4d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-15T14:00:19.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.070+0000 7f733dffb640 1 -- 172.21.15.33:0/3291968094 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7328054df0 con 0x7f73401019b0 2024-05-15T14:00:19.259 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 14:00:18 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T14:00:18.905+0000 7f354ffc9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-15T14:00:19.259 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 14:00:19 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T14:00:19.002+0000 7f354ffc9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-15T14:00:19.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.265+0000 7f7346ebd640 1 -- 172.21.15.33:0/3291968094 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f73401a36f0 con 0x7f73401019b0 2024-05-15T14:00:19.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.267+0000 7f733ffff640 1 -- 172.21.15.33:0/3291968094 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f732c0b28b0 msgr2=0x7f732c0b4d70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1797827152 2024-05-15T14:00:19.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.267+0000 7f733ffff640 1 --2- 172.21.15.33:0/3291968094 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f732c0b28b0 0x7f732c0b4d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-15T14:00:19.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.269+0000 7f733dffb640 1 -- 172.21.15.33:0/3291968094 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) ==== 138+0+0 (secure 0 0 0) 0x7f73280aa310 con 0x7f73401019b0 2024-05-15T14:00:19.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.272+0000 7f7346ebd640 1 -- 172.21.15.33:0/3291968094 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f732c0b28b0 msgr2=0x7f732c0b4d70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-15T14:00:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.272+0000 7f7346ebd640 1 --2- 172.21.15.33:0/3291968094 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f732c0b28b0 0x7f732c0b4d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.272+0000 7f7346ebd640 1 -- 172.21.15.33:0/3291968094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73401019b0 msgr2=0x7f7340105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.272+0000 7f7346ebd640 1 --2- 172.21.15.33:0/3291968094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73401019b0 0x7f7340105940 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f732800e9c0 tx=0x7f732800ee80 comp rx=0 tx=0).stop 2024-05-15T14:00:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.272+0000 7f7346ebd640 1 -- 172.21.15.33:0/3291968094 shutdown_connections 2024-05-15T14:00:19.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.272+0000 7f7346ebd640 1 --2- 172.21.15.33:0/3291968094 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f732c0b28b0 0x7f732c0b4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:19.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.272+0000 7f7346ebd640 1 --2- 172.21.15.33:0/3291968094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7340103560 0x7f73401044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:19.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.272+0000 7f7346ebd640 1 --2- 172.21.15.33:0/3291968094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7340102ba0 0x7f7340103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:19.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.272+0000 7f7346ebd640 1 --2- 172.21.15.33:0/3291968094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73401019b0 0x7f7340105940 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:19.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.272+0000 7f7346ebd640 1 -- 172.21.15.33:0/3291968094 >> 172.21.15.33:0/3291968094 conn(0x7f73400fcf20 msgr2=0x7f73400fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:19.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.272+0000 7f7346ebd640 1 -- 172.21.15.33:0/3291968094 shutdown_connections 2024-05-15T14:00:19.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.272+0000 7f7346ebd640 1 -- 172.21.15.33:0/3291968094 wait complete. 2024-05-15T14:00:19.288 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:19 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:19.178+0000 7f2a72d29200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-15T14:00:19.289 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-05-15T14:00:19.356 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 14:00:18 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T14:00:18.951+0000 7f2b1a230200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-15T14:00:19.357 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 14:00:19 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T14:00:19.084+0000 7f2b1a230200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-15T14:00:19.357 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 14:00:19 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T14:00:19.178+0000 7f2b1a230200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-15T14:00:19.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.395+0000 7f79ddcf8640 1 -- 172.21.15.33:0/2676091116 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79d8102ba0 msgr2=0x7f79d8103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:19.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.395+0000 7f79ddcf8640 1 --2- 172.21.15.33:0/2676091116 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79d8102ba0 0x7f79d8103020 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f79c0009860 tx=0x7f79c002a340 comp rx=0 tx=0).stop 2024-05-15T14:00:19.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.395+0000 7f79ddcf8640 1 -- 172.21.15.33:0/2676091116 shutdown_connections 2024-05-15T14:00:19.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.395+0000 7f79ddcf8640 1 --2- 172.21.15.33:0/2676091116 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79d8103560 0x7f79d8109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:19.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.395+0000 7f79ddcf8640 1 --2- 172.21.15.33:0/2676091116 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79d8102ba0 0x7f79d8103020 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:19.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.395+0000 7f79ddcf8640 1 --2- 172.21.15.33:0/2676091116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79d81019b0 0x7f79d8101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:19.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.395+0000 7f79ddcf8640 1 -- 172.21.15.33:0/2676091116 >> 172.21.15.33:0/2676091116 conn(0x7f79d80fcf20 msgr2=0x7f79d80ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:19.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.395+0000 7f79ddcf8640 1 -- 172.21.15.33:0/2676091116 shutdown_connections 2024-05-15T14:00:19.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.395+0000 7f79ddcf8640 1 -- 172.21.15.33:0/2676091116 wait complete. 2024-05-15T14:00:19.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.396+0000 7f79ddcf8640 1 Processor -- start 2024-05-15T14:00:19.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.396+0000 7f79ddcf8640 1 -- start start 2024-05-15T14:00:19.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.396+0000 7f79ddcf8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79d81019b0 0x7f79d806ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:19.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.396+0000 7f79ddcf8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79d8102ba0 0x7f79d819da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:19.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.396+0000 7f79ddcf8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79d8103560 0x7f79d81a0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:19.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.396+0000 7f79ddcf8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f79d810d4e0 con 0x7f79d8102ba0 2024-05-15T14:00:19.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.396+0000 7f79ddcf8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f79d810d360 con 0x7f79d8103560 2024-05-15T14:00:19.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.396+0000 7f79ddcf8640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f79d810d660 con 0x7f79d81019b0 2024-05-15T14:00:19.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.396+0000 7f79d77fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79d81019b0 0x7f79d806ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:19.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.396+0000 7f79d77fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79d81019b0 0x7f79d806ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:57184/0 (socket says 172.21.15.33:57184) 2024-05-15T14:00:19.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.396+0000 7f79d77fe640 1 -- 172.21.15.33:0/1324541064 learned_addr learned my addr 172.21.15.33:0/1324541064 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:19.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.396+0000 7f79cffff640 1 --2- 172.21.15.33:0/1324541064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79d8102ba0 0x7f79d819da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:19.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.396+0000 7f79d7fff640 1 --2- 172.21.15.33:0/1324541064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79d8103560 0x7f79d81a0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:19.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.396+0000 7f79d77fe640 1 -- 172.21.15.33:0/1324541064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79d8103560 msgr2=0x7f79d81a0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:19.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.396+0000 7f79d77fe640 1 --2- 172.21.15.33:0/1324541064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79d8103560 0x7f79d81a0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:19.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.396+0000 7f79d77fe640 1 -- 172.21.15.33:0/1324541064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79d8102ba0 msgr2=0x7f79d819da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:19.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.396+0000 7f79d77fe640 1 --2- 172.21.15.33:0/1324541064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79d8102ba0 0x7f79d819da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:19.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.396+0000 7f79d77fe640 1 -- 172.21.15.33:0/1324541064 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d81a0990 con 0x7f79d81019b0 2024-05-15T14:00:19.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.397+0000 7f79d77fe640 1 --2- 172.21.15.33:0/1324541064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79d81019b0 0x7f79d806ade0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f79c800e9a0 tx=0x7f79c800ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:19.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.397+0000 7f79d57fa640 1 -- 172.21.15.33:0/1324541064 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c800cd80 con 0x7f79d81019b0 2024-05-15T14:00:19.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.397+0000 7f79d57fa640 1 -- 172.21.15.33:0/1324541064 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79c80108e0 con 0x7f79d81019b0 2024-05-15T14:00:19.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.397+0000 7f79ddcf8640 1 -- 172.21.15.33:0/1324541064 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79d81a0c20 con 0x7f79d81019b0 2024-05-15T14:00:19.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.397+0000 7f79d57fa640 1 -- 172.21.15.33:0/1324541064 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c8018960 con 0x7f79d81019b0 2024-05-15T14:00:19.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.397+0000 7f79ddcf8640 1 -- 172.21.15.33:0/1324541064 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79d81a1190 con 0x7f79d81019b0 2024-05-15T14:00:19.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.398+0000 7f79ddcf8640 1 -- 172.21.15.33:0/1324541064 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79d8108df0 con 0x7f79d81019b0 2024-05-15T14:00:19.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.399+0000 7f79d57fa640 1 -- 172.21.15.33:0/1324541064 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f79c8010a90 con 0x7f79d81019b0 2024-05-15T14:00:19.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.400+0000 7f79d57fa640 1 --2- 172.21.15.33:0/1324541064 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f79c40b28b0 0x7f79c40b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:19.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.400+0000 7f79cffff640 1 -- 172.21.15.33:0/1324541064 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f79c40b28b0 msgr2=0x7f79c40b4d70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1797827152 2024-05-15T14:00:19.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.400+0000 7f79d57fa640 1 -- 172.21.15.33:0/1324541064 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f79c80e2700 con 0x7f79d81019b0 2024-05-15T14:00:19.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.400+0000 7f79cffff640 1 --2- 172.21.15.33:0/1324541064 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f79c40b28b0 0x7f79c40b4d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-15T14:00:19.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.403+0000 7f79d57fa640 1 -- 172.21.15.33:0/1324541064 <== mon.1 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f79c80aa3e0 con 0x7f79d81019b0 2024-05-15T14:00:19.562 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:19 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:19.287+0000 7f2a72d29200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-15T14:00:19.562 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:19 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:19.560+0000 7f2a72d29200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-15T14:00:19.590 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 14:00:19 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T14:00:19.257+0000 7f354ffc9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-15T14:00:19.590 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 14:00:19 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T14:00:19.398+0000 7f354ffc9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-15T14:00:19.590 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 14:00:19 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T14:00:19.498+0000 7f354ffc9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-15T14:00:19.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.596+0000 7f79ddcf8640 1 -- 172.21.15.33:0/1324541064 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f79d80636c0 con 0x7f79d81019b0 2024-05-15T14:00:19.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.601+0000 7f79cffff640 1 -- 172.21.15.33:0/1324541064 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f79c40b28b0 msgr2=0x7f79c40b4d70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1797827152 2024-05-15T14:00:19.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.601+0000 7f79cffff640 1 --2- 172.21.15.33:0/1324541064 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f79c40b28b0 0x7f79c40b4d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-15T14:00:19.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.601+0000 7f79d57fa640 1 -- 172.21.15.33:0/1324541064 <== mon.1 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) ==== 133+0+0 (secure 0 0 0) 0x7f79c80a9ce0 con 0x7f79d81019b0 2024-05-15T14:00:19.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.604+0000 7f79ddcf8640 1 -- 172.21.15.33:0/1324541064 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f79c40b28b0 msgr2=0x7f79c40b4d70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-15T14:00:19.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.604+0000 7f79ddcf8640 1 --2- 172.21.15.33:0/1324541064 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f79c40b28b0 0x7f79c40b4d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:19.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.604+0000 7f79ddcf8640 1 -- 172.21.15.33:0/1324541064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79d81019b0 msgr2=0x7f79d806ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:19.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.604+0000 7f79ddcf8640 1 --2- 172.21.15.33:0/1324541064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79d81019b0 0x7f79d806ade0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f79c800e9a0 tx=0x7f79c800ee60 comp rx=0 tx=0).stop 2024-05-15T14:00:19.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.604+0000 7f79ddcf8640 1 -- 172.21.15.33:0/1324541064 shutdown_connections 2024-05-15T14:00:19.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.604+0000 7f79ddcf8640 1 --2- 172.21.15.33:0/1324541064 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f79c40b28b0 0x7f79c40b4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:19.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.604+0000 7f79ddcf8640 1 --2- 172.21.15.33:0/1324541064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f79d8103560 0x7f79d81a0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:19.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.604+0000 7f79ddcf8640 1 --2- 172.21.15.33:0/1324541064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79d8102ba0 0x7f79d819da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:19.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.604+0000 7f79ddcf8640 1 --2- 172.21.15.33:0/1324541064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79d81019b0 0x7f79d806ade0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:19.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.604+0000 7f79ddcf8640 1 -- 172.21.15.33:0/1324541064 >> 172.21.15.33:0/1324541064 conn(0x7f79d80fcf20 msgr2=0x7f79d8105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:19.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.604+0000 7f79ddcf8640 1 -- 172.21.15.33:0/1324541064 shutdown_connections 2024-05-15T14:00:19.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.604+0000 7f79ddcf8640 1 -- 172.21.15.33:0/1324541064 wait complete. 2024-05-15T14:00:19.617 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-05-15T14:00:19.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.725+0000 7f7696f12640 1 -- 172.21.15.33:0/899512157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76901019b0 msgr2=0x7f7690101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:19.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.725+0000 7f7696f12640 1 --2- 172.21.15.33:0/899512157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76901019b0 0x7f7690101dd0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7680009ca0 tx=0x7f768002a8c0 comp rx=0 tx=0).stop 2024-05-15T14:00:19.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.725+0000 7f7696f12640 1 -- 172.21.15.33:0/899512157 shutdown_connections 2024-05-15T14:00:19.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.725+0000 7f7696f12640 1 --2- 172.21.15.33:0/899512157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7690103560 0x7f7690109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:19.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.725+0000 7f7696f12640 1 --2- 172.21.15.33:0/899512157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7690102ba0 0x7f7690103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:19.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.725+0000 7f7696f12640 1 --2- 172.21.15.33:0/899512157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76901019b0 0x7f7690101dd0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:19.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.725+0000 7f7696f12640 1 -- 172.21.15.33:0/899512157 >> 172.21.15.33:0/899512157 conn(0x7f76900fcf20 msgr2=0x7f76900ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:19.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.725+0000 7f7696f12640 1 -- 172.21.15.33:0/899512157 shutdown_connections 2024-05-15T14:00:19.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.725+0000 7f7696f12640 1 -- 172.21.15.33:0/899512157 wait complete. 2024-05-15T14:00:19.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.726+0000 7f7696f12640 1 Processor -- start 2024-05-15T14:00:19.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.726+0000 7f7696f12640 1 -- start start 2024-05-15T14:00:19.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.726+0000 7f7696f12640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76901019b0 0x7f7690197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:19.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.726+0000 7f7696f12640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7690102ba0 0x7f7690198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:19.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.726+0000 7f7696f12640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7690103560 0x7f769019f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.726+0000 7f7696f12640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f769010d530 con 0x7f76901019b0 2024-05-15T14:00:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.726+0000 7f7696f12640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f769010d3b0 con 0x7f7690103560 2024-05-15T14:00:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.726+0000 7f7696f12640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f769010d6b0 con 0x7f7690102ba0 2024-05-15T14:00:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.726+0000 7f7694cae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76901019b0 0x7f7690197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.726+0000 7f7694cae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76901019b0 0x7f7690197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44082/0 (socket says 172.21.15.33:44082) 2024-05-15T14:00:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.726+0000 7f7694cae640 1 -- 172.21.15.33:0/2099331377 learned_addr learned my addr 172.21.15.33:0/2099331377 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.726+0000 7f76954af640 1 --2- 172.21.15.33:0/2099331377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7690103560 0x7f769019f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:19.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.726+0000 7f768ffff640 1 --2- 172.21.15.33:0/2099331377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7690102ba0 0x7f7690198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:19.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.727+0000 7f7694cae640 1 -- 172.21.15.33:0/2099331377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7690102ba0 msgr2=0x7f7690198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:19.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.727+0000 7f7694cae640 1 --2- 172.21.15.33:0/2099331377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7690102ba0 0x7f7690198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:19.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.727+0000 7f7694cae640 1 -- 172.21.15.33:0/2099331377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7690103560 msgr2=0x7f769019f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:19.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.727+0000 7f7694cae640 1 --2- 172.21.15.33:0/2099331377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7690103560 0x7f769019f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:19.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.727+0000 7f7694cae640 1 -- 172.21.15.33:0/2099331377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f769019f850 con 0x7f76901019b0 2024-05-15T14:00:19.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.727+0000 7f7694cae640 1 --2- 172.21.15.33:0/2099331377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76901019b0 0x7f7690197c40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7680005e30 tx=0x7f76800317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:19.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.727+0000 7f768dffb640 1 -- 172.21.15.33:0/2099331377 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768004a070 con 0x7f76901019b0 2024-05-15T14:00:19.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.727+0000 7f768dffb640 1 -- 172.21.15.33:0/2099331377 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7680003e00 con 0x7f76901019b0 2024-05-15T14:00:19.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.727+0000 7f7696f12640 1 -- 172.21.15.33:0/2099331377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f769006a270 con 0x7f76901019b0 2024-05-15T14:00:19.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.727+0000 7f768dffb640 1 -- 172.21.15.33:0/2099331377 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768003cc50 con 0x7f76901019b0 2024-05-15T14:00:19.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.727+0000 7f7696f12640 1 -- 172.21.15.33:0/2099331377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f769006a7c0 con 0x7f76901019b0 2024-05-15T14:00:19.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.728+0000 7f7696f12640 1 -- 172.21.15.33:0/2099331377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f769010dcf0 con 0x7f76901019b0 2024-05-15T14:00:19.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.730+0000 7f768dffb640 1 -- 172.21.15.33:0/2099331377 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f768004d070 con 0x7f76901019b0 2024-05-15T14:00:19.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.730+0000 7f768dffb640 1 --2- 172.21.15.33:0/2099331377 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f767c0b2830 0x7f767c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:19.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.730+0000 7f768ffff640 1 -- 172.21.15.33:0/2099331377 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f767c0b2830 msgr2=0x7f767c0b4cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1797827152 2024-05-15T14:00:19.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.731+0000 7f768dffb640 1 -- 172.21.15.33:0/2099331377 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8240+0+0 (secure 0 0 0) 0x7f7680105020 con 0x7f76901019b0 2024-05-15T14:00:19.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.731+0000 7f768ffff640 1 --2- 172.21.15.33:0/2099331377 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f767c0b2830 0x7f767c0b4cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-15T14:00:19.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.734+0000 7f768dffb640 1 -- 172.21.15.33:0/2099331377 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7680054080 con 0x7f76901019b0 2024-05-15T14:00:19.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:19 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/225218347' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-15T14:00:19.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:19 smithi033 ceph-mon[31650]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:00:19.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:19 smithi033 ceph-mon[31650]: from='client.? ' entity='client.admin' 2024-05-15T14:00:19.822 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:19 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/3291968094' entity='client.admin' 2024-05-15T14:00:19.822 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:19 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:19.710+0000 7f2a72d29200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-15T14:00:19.822 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:19 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:19.820+0000 7f2a72d29200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-15T14:00:19.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:19 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/225218347' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-15T14:00:19.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:19 smithi081 ceph-mon[36434]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:00:19.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:19 smithi081 ceph-mon[36434]: from='client.? ' entity='client.admin' 2024-05-15T14:00:19.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:19 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/3291968094' entity='client.admin' 2024-05-15T14:00:19.857 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 14:00:19 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T14:00:19.502+0000 7f2b1a230200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-15T14:00:19.857 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 14:00:19 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T14:00:19.594+0000 7f2b1a230200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-15T14:00:19.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.932+0000 7f768ffff640 1 -- 172.21.15.33:0/2099331377 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f767c0b2830 msgr2=0x7f767c0b4cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1797827152 2024-05-15T14:00:19.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.932+0000 7f768ffff640 1 --2- 172.21.15.33:0/2099331377 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f767c0b2830 0x7f767c0b4cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-15T14:00:19.937 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:19 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/225218347' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-15T14:00:19.937 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:19 smithi052 ceph-mon[36190]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:00:19.937 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:19 smithi052 ceph-mon[36190]: from='client.? ' entity='client.admin' 2024-05-15T14:00:19.937 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:19 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/3291968094' entity='client.admin' 2024-05-15T14:00:19.937 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 14:00:19 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T14:00:19.840+0000 7f354ffc9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-15T14:00:19.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:19.951+0000 7f7696f12640 1 -- 172.21.15.33:0/2099331377 --> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f7690108ce0 con 0x7f767c0b2830 2024-05-15T14:00:20.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:20.333+0000 7f768ffff640 1 -- 172.21.15.33:0/2099331377 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f767c0b2830 msgr2=0x7f767c0b4cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1797827152 2024-05-15T14:00:20.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:20.333+0000 7f768ffff640 1 --2- 172.21.15.33:0/2099331377 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f767c0b2830 0x7f767c0b4cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-15T14:00:20.340 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 14:00:19 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T14:00:19.935+0000 7f354ffc9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-15T14:00:20.440 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:20 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:20.164+0000 7f2a72d29200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-15T14:00:20.441 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:20 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:20.264+0000 7f2a72d29200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-15T14:00:20.605 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 14:00:20 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T14:00:20.197+0000 7f2b1a230200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-15T14:00:20.605 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 14:00:20 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z[38131]: 2024-05-15T14:00:20.412+0000 7f2b1a230200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-15T14:00:20.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:20.616+0000 7f768dffb640 1 -- 172.21.15.33:0/2099331377 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f76800c8980 con 0x7f76901019b0 2024-05-15T14:00:20.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:20 smithi052 ceph-mon[36190]: from='client.? ' entity='client.admin' 2024-05-15T14:00:20.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:20 smithi052 ceph-mon[36190]: Standby manager daemon z restarted 2024-05-15T14:00:20.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:20 smithi052 ceph-mon[36190]: Standby manager daemon z started 2024-05-15T14:00:20.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:20 smithi052 ceph-mon[36190]: from='mgr.? 172.21.15.81:0/949066591' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-15T14:00:20.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:20 smithi052 ceph-mon[36190]: from='mgr.? 172.21.15.81:0/949066591' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-15T14:00:20.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:20 smithi052 ceph-mon[36190]: from='mgr.? 172.21.15.81:0/949066591' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-15T14:00:20.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:20 smithi052 ceph-mon[36190]: from='mgr.? 172.21.15.81:0/949066591' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-15T14:00:20.841 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 14:00:20 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T14:00:20.567+0000 7f354ffc9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-15T14:00:20.841 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 14:00:20 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y[37577]: 2024-05-15T14:00:20.796+0000 7f354ffc9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-15T14:00:20.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:20 smithi081 ceph-mon[36434]: from='client.? ' entity='client.admin' 2024-05-15T14:00:20.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:20 smithi081 ceph-mon[36434]: Standby manager daemon z restarted 2024-05-15T14:00:20.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:20 smithi081 ceph-mon[36434]: Standby manager daemon z started 2024-05-15T14:00:20.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:20 smithi081 ceph-mon[36434]: from='mgr.? 172.21.15.81:0/949066591' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-15T14:00:20.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:20 smithi081 ceph-mon[36434]: from='mgr.? 172.21.15.81:0/949066591' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-15T14:00:20.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:20 smithi081 ceph-mon[36434]: from='mgr.? 172.21.15.81:0/949066591' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-15T14:00:20.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:20 smithi081 ceph-mon[36434]: from='mgr.? 172.21.15.81:0/949066591' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-15T14:00:20.919 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:20 smithi033 ceph-mon[31650]: from='client.? ' entity='client.admin' 2024-05-15T14:00:20.919 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:20 smithi033 ceph-mon[31650]: Standby manager daemon z restarted 2024-05-15T14:00:20.919 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:20 smithi033 ceph-mon[31650]: Standby manager daemon z started 2024-05-15T14:00:20.919 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:20 smithi033 ceph-mon[31650]: from='mgr.? 172.21.15.81:0/949066591' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-15T14:00:20.919 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:20 smithi033 ceph-mon[31650]: from='mgr.? 172.21.15.81:0/949066591' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-15T14:00:20.919 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:20 smithi033 ceph-mon[31650]: from='mgr.? 172.21.15.81:0/949066591' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-15T14:00:20.920 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:20 smithi033 ceph-mon[31650]: from='mgr.? 172.21.15.81:0/949066591' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-15T14:00:21.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:21.134+0000 7f768ffff640 1 -- 172.21.15.33:0/2099331377 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f767c0b2830 msgr2=0x7f767c0b4cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1797827152 2024-05-15T14:00:21.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:21.134+0000 7f768ffff640 1 --2- 172.21.15.33:0/2099331377 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f767c0b2830 0x7f767c0b4cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-15T14:00:21.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:21.169+0000 7f768dffb640 1 -- 172.21.15.33:0/2099331377 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mgrmap(e 21) ==== 149337+0+0 (secure 0 0 0) 0x7f76800f1190 con 0x7f76901019b0 2024-05-15T14:00:21.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:21.170+0000 7f768dffb640 1 -- 172.21.15.33:0/2099331377 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f767c0b2830 msgr2=0x7f767c0b4cf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-15T14:00:21.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:21.170+0000 7f768dffb640 1 --2- 172.21.15.33:0/2099331377 >> [v2:172.21.15.33:6800/1797827152,v1:172.21.15.33:6801/1797827152] conn(0x7f767c0b2830 0x7f767c0b4cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:21.190 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:20 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:20.917+0000 7f2a72d29200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-15T14:00:21.191 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:21 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:21.155+0000 7f2a72d29200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-15T14:00:21.614 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:21 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:21.408+0000 7f29d2d79700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-15T14:00:21.614 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:21 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:21.408+0000 7f29d2d79700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-15T14:00:21.614 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:21 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:21.408+0000 7f29d2d79700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-15T14:00:21.614 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:21 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:21.408+0000 7f29d2d79700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-15T14:00:21.614 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:21 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:21.408+0000 7f29d2d79700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-15T14:00:21.614 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:21 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:21.408+0000 7f29d1576700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-15T14:00:21.614 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:21 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:21.408+0000 7f29d1576700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-15T14:00:21.614 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:21 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:21.408+0000 7f29d1576700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-15T14:00:21.614 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:21 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:21.408+0000 7f29d1576700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-15T14:00:21.614 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:00:21 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:00:21.408+0000 7f29d1576700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-15T14:00:21.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-05-15T14:00:21.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: Standby manager daemon y restarted 2024-05-15T14:00:21.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: Standby manager daemon y started 2024-05-15T14:00:21.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.? 172.21.15.52:0/2156086231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-15T14:00:21.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.? 172.21.15.52:0/2156086231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-15T14:00:21.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.? 172.21.15.52:0/2156086231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-15T14:00:21.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.? 172.21.15.52:0/2156086231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-15T14:00:21.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: Active manager daemon x restarted 2024-05-15T14:00:21.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: Activating manager daemon x 2024-05-15T14:00:21.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: osdmap e83: 12 total, 12 up, 12 in 2024-05-15T14:00:21.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: mgrmap e21: x(active, starting, since 0.0135837s), standbys: y, z 2024-05-15T14:00:21.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-15T14:00:21.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-15T14:00:21.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-15T14:00:21.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-15T14:00:21.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-15T14:00:21.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-15T14:00:21.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-15T14:00:21.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-15T14:00:21.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-15T14:00:21.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-15T14:00:21.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-15T14:00:21.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-15T14:00:21.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-15T14:00:21.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-15T14:00:21.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-15T14:00:21.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-15T14:00:21.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-15T14:00:21.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-15T14:00:21.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T14:00:21.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T14:00:21.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T14:00:21.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T14:00:21.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T14:00:21.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T14:00:21.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-15T14:00:21.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-15T14:00:21.944 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-15T14:00:21.944 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-15T14:00:21.944 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-15T14:00:21.944 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-15T14:00:21.944 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-15T14:00:21.944 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-15T14:00:21.944 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-15T14:00:21.944 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: Manager daemon x is now available 2024-05-15T14:00:21.944 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:00:21.944 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-15T14:00:21.944 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:21 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-15T14:00:22.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-05-15T14:00:22.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: Standby manager daemon y restarted 2024-05-15T14:00:22.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: Standby manager daemon y started 2024-05-15T14:00:22.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.? 172.21.15.52:0/2156086231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-15T14:00:22.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.? 172.21.15.52:0/2156086231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-15T14:00:22.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.? 172.21.15.52:0/2156086231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-15T14:00:22.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.? 172.21.15.52:0/2156086231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-15T14:00:22.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: Active manager daemon x restarted 2024-05-15T14:00:22.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: Activating manager daemon x 2024-05-15T14:00:22.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: osdmap e83: 12 total, 12 up, 12 in 2024-05-15T14:00:22.091 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: mgrmap e21: x(active, starting, since 0.0135837s), standbys: y, z 2024-05-15T14:00:22.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-15T14:00:22.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-15T14:00:22.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-15T14:00:22.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-15T14:00:22.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-15T14:00:22.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-15T14:00:22.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-15T14:00:22.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-15T14:00:22.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-15T14:00:22.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-15T14:00:22.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-15T14:00:22.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-15T14:00:22.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-15T14:00:22.092 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-15T14:00:22.093 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-15T14:00:22.093 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-15T14:00:22.093 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-15T14:00:22.093 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-15T14:00:22.093 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T14:00:22.093 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T14:00:22.093 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T14:00:22.093 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T14:00:22.093 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T14:00:22.093 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T14:00:22.093 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-15T14:00:22.093 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-15T14:00:22.093 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-15T14:00:22.093 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-15T14:00:22.093 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-15T14:00:22.093 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-15T14:00:22.094 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-15T14:00:22.094 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-15T14:00:22.094 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-15T14:00:22.094 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: Manager daemon x is now available 2024-05-15T14:00:22.094 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:00:22.094 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-15T14:00:22.094 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:21 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-15T14:00:22.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-05-15T14:00:22.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: Standby manager daemon y restarted 2024-05-15T14:00:22.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: Standby manager daemon y started 2024-05-15T14:00:22.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.? 172.21.15.52:0/2156086231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-15T14:00:22.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.? 172.21.15.52:0/2156086231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-15T14:00:22.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.? 172.21.15.52:0/2156086231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-15T14:00:22.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.? 172.21.15.52:0/2156086231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-15T14:00:22.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: Active manager daemon x restarted 2024-05-15T14:00:22.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: Activating manager daemon x 2024-05-15T14:00:22.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: osdmap e83: 12 total, 12 up, 12 in 2024-05-15T14:00:22.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: mgrmap e21: x(active, starting, since 0.0135837s), standbys: y, z 2024-05-15T14:00:22.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-15T14:00:22.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-15T14:00:22.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-15T14:00:22.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-15T14:00:22.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-15T14:00:22.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-15T14:00:22.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-15T14:00:22.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-15T14:00:22.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-15T14:00:22.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-15T14:00:22.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-15T14:00:22.108 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-15T14:00:22.109 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-15T14:00:22.109 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-15T14:00:22.109 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-15T14:00:22.109 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-15T14:00:22.109 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-15T14:00:22.109 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-15T14:00:22.109 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-15T14:00:22.109 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-15T14:00:22.109 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-15T14:00:22.109 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-15T14:00:22.110 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-15T14:00:22.110 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-15T14:00:22.110 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-15T14:00:22.110 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-15T14:00:22.110 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-15T14:00:22.110 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-15T14:00:22.110 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-15T14:00:22.110 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-15T14:00:22.110 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-15T14:00:22.110 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-15T14:00:22.110 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-15T14:00:22.110 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: Manager daemon x is now available 2024-05-15T14:00:22.110 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:00:22.110 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-15T14:00:22.110 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:21 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-15T14:00:22.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.172+0000 7f768dffb640 1 -- 172.21.15.33:0/2099331377 <== mon.0 v2:172.21.15.33:3300/0 9 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7f768007e630 con 0x7f76901019b0 2024-05-15T14:00:22.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.173+0000 7f768dffb640 1 --2- 172.21.15.33:0/2099331377 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f767c036490 0x7f767c036870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:22.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.173+0000 7f768dffb640 1 -- 172.21.15.33:0/2099331377 --> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f7690108ce0 con 0x7f767c036490 2024-05-15T14:00:22.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.175+0000 7f768ffff640 1 --2- 172.21.15.33:0/2099331377 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f767c036490 0x7f767c036870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:22.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.176+0000 7f768ffff640 1 --2- 172.21.15.33:0/2099331377 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f767c036490 0x7f767c036870 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f76780097f0 tx=0x7f7678005f90 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:22.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.256+0000 7f768dffb640 1 -- 172.21.15.33:0/2099331377 <== mgr.15144 v2:172.21.15.33:6800/1796050977 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7690108ce0 con 0x7f767c036490 2024-05-15T14:00:22.258 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule set for path / 2024-05-15T14:00:22.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.258+0000 7f7696f12640 1 -- 172.21.15.33:0/2099331377 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f767c036490 msgr2=0x7f767c036870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:22.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.258+0000 7f7696f12640 1 --2- 172.21.15.33:0/2099331377 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f767c036490 0x7f767c036870 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f76780097f0 tx=0x7f7678005f90 comp rx=0 tx=0).stop 2024-05-15T14:00:22.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.258+0000 7f7696f12640 1 -- 172.21.15.33:0/2099331377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76901019b0 msgr2=0x7f7690197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:22.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.258+0000 7f7696f12640 1 --2- 172.21.15.33:0/2099331377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76901019b0 0x7f7690197c40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7680005e30 tx=0x7f76800317e0 comp rx=0 tx=0).stop 2024-05-15T14:00:22.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.258+0000 7f76954af640 1 -- 172.21.15.33:0/2099331377 reap_dead start 2024-05-15T14:00:22.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.259+0000 7f7696f12640 1 -- 172.21.15.33:0/2099331377 shutdown_connections 2024-05-15T14:00:22.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.259+0000 7f7696f12640 1 -- 172.21.15.33:0/2099331377 >> 172.21.15.33:0/2099331377 conn(0x7f76900fcf20 msgr2=0x7f7690105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:22.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.259+0000 7f7696f12640 1 -- 172.21.15.33:0/2099331377 shutdown_connections 2024-05-15T14:00:22.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.259+0000 7f7696f12640 1 -- 172.21.15.33:0/2099331377 wait complete. 2024-05-15T14:00:22.272 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-05-15T14:00:22.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.383+0000 7fc907d1b640 1 -- 172.21.15.33:0/1291099952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc900105f00 msgr2=0x7fc9001082f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:22.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.383+0000 7fc907d1b640 1 --2- 172.21.15.33:0/1291099952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc900105f00 0x7fc9001082f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0009900 tx=0x7fc8f002a410 comp rx=0 tx=0).stop 2024-05-15T14:00:22.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.383+0000 7fc907d1b640 1 -- 172.21.15.33:0/1291099952 shutdown_connections 2024-05-15T14:00:22.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.383+0000 7fc907d1b640 1 --2- 172.21.15.33:0/1291099952 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc900108830 0x7fc90010ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:22.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.383+0000 7fc907d1b640 1 --2- 172.21.15.33:0/1291099952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc900105f00 0x7fc9001082f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:22.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.383+0000 7fc907d1b640 1 --2- 172.21.15.33:0/1291099952 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9001035d0 0x7fc9001059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:22.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.383+0000 7fc907d1b640 1 -- 172.21.15.33:0/1291099952 >> 172.21.15.33:0/1291099952 conn(0x7fc9000fcf20 msgr2=0x7fc9000ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:22.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.383+0000 7fc907d1b640 1 -- 172.21.15.33:0/1291099952 shutdown_connections 2024-05-15T14:00:22.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.383+0000 7fc907d1b640 1 -- 172.21.15.33:0/1291099952 wait complete. 2024-05-15T14:00:22.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.383+0000 7fc907d1b640 1 Processor -- start 2024-05-15T14:00:22.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.384+0000 7fc907d1b640 1 -- start start 2024-05-15T14:00:22.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.384+0000 7fc907d1b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9001035d0 0x7fc900102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:22.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.384+0000 7fc907d1b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc900105f00 0x7fc900101580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:22.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.384+0000 7fc907d1b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc900108830 0x7fc900101ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:22.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.384+0000 7fc907d1b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc90010e600 con 0x7fc9001035d0 2024-05-15T14:00:22.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.384+0000 7fc907d1b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc90010e480 con 0x7fc900105f00 2024-05-15T14:00:22.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.384+0000 7fc907d1b640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc90010e780 con 0x7fc900108830 2024-05-15T14:00:22.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.384+0000 7fc905ab7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9001035d0 0x7fc900102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:22.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.384+0000 7fc905ab7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9001035d0 0x7fc900102ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44210/0 (socket says 172.21.15.33:44210) 2024-05-15T14:00:22.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.384+0000 7fc905ab7640 1 -- 172.21.15.33:0/3566752828 learned_addr learned my addr 172.21.15.33:0/3566752828 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:22.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.384+0000 7fc9062b8640 1 --2- 172.21.15.33:0/3566752828 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc900108830 0x7fc900101ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:22.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.384+0000 7fc9052b6640 1 --2- 172.21.15.33:0/3566752828 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc900105f00 0x7fc900101580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:22.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.384+0000 7fc905ab7640 1 -- 172.21.15.33:0/3566752828 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc900108830 msgr2=0x7fc900101ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:22.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.384+0000 7fc905ab7640 1 --2- 172.21.15.33:0/3566752828 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc900108830 0x7fc900101ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:22.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.384+0000 7fc905ab7640 1 -- 172.21.15.33:0/3566752828 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc900105f00 msgr2=0x7fc900101580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:22.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.384+0000 7fc905ab7640 1 --2- 172.21.15.33:0/3566752828 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc900105f00 0x7fc900101580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:22.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.384+0000 7fc905ab7640 1 -- 172.21.15.33:0/3566752828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc900102380 con 0x7fc9001035d0 2024-05-15T14:00:22.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.385+0000 7fc905ab7640 1 --2- 172.21.15.33:0/3566752828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9001035d0 0x7fc900102ef0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc00e9f0 tx=0x7fc8fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:22.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.385+0000 7fc8f6ffd640 1 -- 172.21.15.33:0/3566752828 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc00cd80 con 0x7fc9001035d0 2024-05-15T14:00:22.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.385+0000 7fc8f6ffd640 1 -- 172.21.15.33:0/3566752828 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8fc0108e0 con 0x7fc9001035d0 2024-05-15T14:00:22.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.385+0000 7fc907d1b640 1 -- 172.21.15.33:0/3566752828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9001a2ec0 con 0x7fc9001035d0 2024-05-15T14:00:22.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.385+0000 7fc8f6ffd640 1 -- 172.21.15.33:0/3566752828 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc018980 con 0x7fc9001035d0 2024-05-15T14:00:22.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.385+0000 7fc907d1b640 1 -- 172.21.15.33:0/3566752828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9001a3330 con 0x7fc9001035d0 2024-05-15T14:00:22.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.386+0000 7fc907d1b640 1 -- 172.21.15.33:0/3566752828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9000fea10 con 0x7fc9001035d0 2024-05-15T14:00:22.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.387+0000 7fc8f6ffd640 1 -- 172.21.15.33:0/3566752828 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7fc8fc010430 con 0x7fc9001035d0 2024-05-15T14:00:22.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.388+0000 7fc8f6ffd640 1 --2- 172.21.15.33:0/3566752828 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc8ec0b2660 0x7fc8ec0b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:22.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.388+0000 7fc8f6ffd640 1 -- 172.21.15.33:0/3566752828 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fc8fc0e21e0 con 0x7fc9001035d0 2024-05-15T14:00:22.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.388+0000 7fc9052b6640 1 --2- 172.21.15.33:0/3566752828 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc8ec0b2660 0x7fc8ec0b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:22.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.389+0000 7fc9052b6640 1 --2- 172.21.15.33:0/3566752828 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc8ec0b2660 0x7fc8ec0b4b20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0005ab0 tx=0x7fc8f0005a20 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:22.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.393+0000 7fc8f6ffd640 1 -- 172.21.15.33:0/3566752828 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc8fc0a9a90 con 0x7fc9001035d0 2024-05-15T14:00:22.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.609+0000 7fc907d1b640 1 -- 172.21.15.33:0/3566752828 --> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fc900062ff0 con 0x7fc8ec0b2660 2024-05-15T14:00:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.672+0000 7fc8f6ffd640 1 -- 172.21.15.33:0/3566752828 <== mgr.15144 v2:172.21.15.33:6800/1796050977 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fc900062ff0 con 0x7fc8ec0b2660 2024-05-15T14:00:22.674 INFO:teuthology.orchestra.run.smithi033.stdout:Retention added to path / 2024-05-15T14:00:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.675+0000 7fc907d1b640 1 -- 172.21.15.33:0/3566752828 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc8ec0b2660 msgr2=0x7fc8ec0b4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.675+0000 7fc907d1b640 1 --2- 172.21.15.33:0/3566752828 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc8ec0b2660 0x7fc8ec0b4b20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0005ab0 tx=0x7fc8f0005a20 comp rx=0 tx=0).stop 2024-05-15T14:00:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.675+0000 7fc907d1b640 1 -- 172.21.15.33:0/3566752828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9001035d0 msgr2=0x7fc900102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.675+0000 7fc907d1b640 1 --2- 172.21.15.33:0/3566752828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9001035d0 0x7fc900102ef0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc00e9f0 tx=0x7fc8fc00eeb0 comp rx=0 tx=0).stop 2024-05-15T14:00:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.675+0000 7fc907d1b640 1 -- 172.21.15.33:0/3566752828 shutdown_connections 2024-05-15T14:00:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.675+0000 7fc907d1b640 1 --2- 172.21.15.33:0/3566752828 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc8ec0b2660 0x7fc8ec0b4b20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.675+0000 7fc907d1b640 1 --2- 172.21.15.33:0/3566752828 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc900108830 0x7fc900101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.675+0000 7fc907d1b640 1 --2- 172.21.15.33:0/3566752828 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc900105f00 0x7fc900101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.675+0000 7fc907d1b640 1 --2- 172.21.15.33:0/3566752828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9001035d0 0x7fc900102ef0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.675+0000 7fc907d1b640 1 -- 172.21.15.33:0/3566752828 >> 172.21.15.33:0/3566752828 conn(0x7fc9000fcf20 msgr2=0x7fc9001040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.675+0000 7fc907d1b640 1 -- 172.21.15.33:0/3566752828 shutdown_connections 2024-05-15T14:00:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.675+0000 7fc907d1b640 1 -- 172.21.15.33:0/3566752828 wait complete. 2024-05-15T14:00:22.689 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-15T14:00:22.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.813+0000 7f14b9000640 1 -- 172.21.15.33:0/3101464798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14b4103570 msgr2=0x7f14b4109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:22.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.813+0000 7f14b9000640 1 --2- 172.21.15.33:0/3101464798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14b4103570 0x7f14b4109e00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009ca0 tx=0x7f14ac02a8c0 comp rx=0 tx=0).stop 2024-05-15T14:00:22.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.813+0000 7f14b9000640 1 -- 172.21.15.33:0/3101464798 shutdown_connections 2024-05-15T14:00:22.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.813+0000 7f14b9000640 1 --2- 172.21.15.33:0/3101464798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14b4103570 0x7f14b4109e00 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:22.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.813+0000 7f14b9000640 1 --2- 172.21.15.33:0/3101464798 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14b4102bb0 0x7f14b4103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:22.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.813+0000 7f14b9000640 1 --2- 172.21.15.33:0/3101464798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14b41019c0 0x7f14b4101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:22.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.813+0000 7f14b9000640 1 -- 172.21.15.33:0/3101464798 >> 172.21.15.33:0/3101464798 conn(0x7f14b40fcf30 msgr2=0x7f14b40ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:22.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.813+0000 7f14b9000640 1 -- 172.21.15.33:0/3101464798 shutdown_connections 2024-05-15T14:00:22.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.813+0000 7f14b9000640 1 -- 172.21.15.33:0/3101464798 wait complete. 2024-05-15T14:00:22.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.814+0000 7f14b9000640 1 Processor -- start 2024-05-15T14:00:22.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.814+0000 7f14b9000640 1 -- start start 2024-05-15T14:00:22.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.814+0000 7f14b9000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14b41019c0 0x7f14b406bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:22.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.814+0000 7f14b9000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14b4102bb0 0x7f14b419d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:22.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.814+0000 7f14b9000640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14b4103570 0x7f14b419ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:22.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.814+0000 7f14b9000640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f14b410d520 con 0x7f14b4102bb0 2024-05-15T14:00:22.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.814+0000 7f14b9000640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f14b410d3a0 con 0x7f14b4103570 2024-05-15T14:00:22.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.814+0000 7f14b9000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f14b410d6a0 con 0x7f14b41019c0 2024-05-15T14:00:22.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.815+0000 7f14abfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14b4102bb0 0x7f14b419d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:22.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.815+0000 7f14abfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14b4102bb0 0x7f14b419d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59980/0 (socket says 172.21.15.33:59980) 2024-05-15T14:00:22.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.815+0000 7f14abfff640 1 -- 172.21.15.33:0/1460517359 learned_addr learned my addr 172.21.15.33:0/1460517359 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:22.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.815+0000 7f14b259c640 1 --2- 172.21.15.33:0/1460517359 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14b41019c0 0x7f14b406bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:22.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.815+0000 7f14b2d9d640 1 --2- 172.21.15.33:0/1460517359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14b4103570 0x7f14b419ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:22.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.815+0000 7f14b259c640 1 -- 172.21.15.33:0/1460517359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14b4103570 msgr2=0x7f14b419ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:22.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.815+0000 7f14b259c640 1 --2- 172.21.15.33:0/1460517359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14b4103570 0x7f14b419ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:22.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.815+0000 7f14b259c640 1 -- 172.21.15.33:0/1460517359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14b4102bb0 msgr2=0x7f14b419d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:22.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.815+0000 7f14b259c640 1 --2- 172.21.15.33:0/1460517359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14b4102bb0 0x7f14b419d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:22.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.815+0000 7f14b259c640 1 -- 172.21.15.33:0/1460517359 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b41a08e0 con 0x7f14b41019c0 2024-05-15T14:00:22.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.815+0000 7f14abfff640 1 --2- 172.21.15.33:0/1460517359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14b4102bb0 0x7f14b419d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:00:22.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.815+0000 7f14b259c640 1 --2- 172.21.15.33:0/1460517359 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14b41019c0 0x7f14b406bc40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f14a400e9c0 tx=0x7f14a400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:22.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.815+0000 7f14ab7fe640 1 -- 172.21.15.33:0/1460517359 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a400ccc0 con 0x7f14b41019c0 2024-05-15T14:00:22.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.815+0000 7f14b9000640 1 -- 172.21.15.33:0/1460517359 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14b41a0b70 con 0x7f14b41019c0 2024-05-15T14:00:22.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.815+0000 7f14ab7fe640 1 -- 172.21.15.33:0/1460517359 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14a40035f0 con 0x7f14b41019c0 2024-05-15T14:00:22.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.815+0000 7f14ab7fe640 1 -- 172.21.15.33:0/1460517359 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a400ce60 con 0x7f14b41019c0 2024-05-15T14:00:22.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.815+0000 7f14b9000640 1 -- 172.21.15.33:0/1460517359 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14b41a10c0 con 0x7f14b41019c0 2024-05-15T14:00:22.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.817+0000 7f14b9000640 1 -- 172.21.15.33:0/1460517359 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14b4108ce0 con 0x7f14b41019c0 2024-05-15T14:00:22.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.818+0000 7f14ab7fe640 1 -- 172.21.15.33:0/1460517359 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7f14a4010690 con 0x7f14b41019c0 2024-05-15T14:00:22.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.819+0000 7f14ab7fe640 1 --2- 172.21.15.33:0/1460517359 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f14a00b25e0 0x7f14a00b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:22.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.819+0000 7f14ab7fe640 1 -- 172.21.15.33:0/1460517359 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f14a40e23b0 con 0x7f14b41019c0 2024-05-15T14:00:22.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.819+0000 7f14abfff640 1 --2- 172.21.15.33:0/1460517359 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f14a00b25e0 0x7f14a00b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:22.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.820+0000 7f14abfff640 1 --2- 172.21.15.33:0/1460517359 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f14a00b25e0 0x7f14a00b4aa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f149c0095f0 tx=0x7f149c005f90 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:22.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:22.823+0000 7f14ab7fe640 1 -- 172.21.15.33:0/1460517359 <== mon.1 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f14a40aa270 con 0x7f14b41019c0 2024-05-15T14:00:23.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.039+0000 7f14b9000640 1 -- 172.21.15.33:0/1460517359 --> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f14b4062e90 con 0x7f14a00b25e0 2024-05-15T14:00:23.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.047+0000 7f14ab7fe640 1 -- 172.21.15.33:0/1460517359 <== mgr.15144 v2:172.21.15.33:6800/1796050977 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f14b4062e90 con 0x7f14a00b25e0 2024-05-15T14:00:23.048 INFO:teuthology.orchestra.run.smithi033.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-15T00:00:00", "created": "2024-05-15T14:00:22", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-05-15T14:00:23.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.049+0000 7f14b9000640 1 -- 172.21.15.33:0/1460517359 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f14a00b25e0 msgr2=0x7f14a00b4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:23.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.049+0000 7f14b9000640 1 --2- 172.21.15.33:0/1460517359 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f14a00b25e0 0x7f14a00b4aa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f149c0095f0 tx=0x7f149c005f90 comp rx=0 tx=0).stop 2024-05-15T14:00:23.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.049+0000 7f14b9000640 1 -- 172.21.15.33:0/1460517359 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14b41019c0 msgr2=0x7f14b406bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:23.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.049+0000 7f14b9000640 1 --2- 172.21.15.33:0/1460517359 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14b41019c0 0x7f14b406bc40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f14a400e9c0 tx=0x7f14a400ee80 comp rx=0 tx=0).stop 2024-05-15T14:00:23.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.049+0000 7f14b9000640 1 -- 172.21.15.33:0/1460517359 shutdown_connections 2024-05-15T14:00:23.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.049+0000 7f14b9000640 1 --2- 172.21.15.33:0/1460517359 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f14a00b25e0 0x7f14a00b4aa0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:23.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.049+0000 7f14b9000640 1 --2- 172.21.15.33:0/1460517359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14b4103570 0x7f14b419ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:23.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.049+0000 7f14b9000640 1 --2- 172.21.15.33:0/1460517359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14b4102bb0 0x7f14b419d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:23.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.049+0000 7f14b9000640 1 --2- 172.21.15.33:0/1460517359 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14b41019c0 0x7f14b406bc40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:23.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.049+0000 7f14b9000640 1 -- 172.21.15.33:0/1460517359 >> 172.21.15.33:0/1460517359 conn(0x7f14b40fcf30 msgr2=0x7f14b4105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:23.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.049+0000 7f14b9000640 1 -- 172.21.15.33:0/1460517359 shutdown_connections 2024-05-15T14:00:23.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.049+0000 7f14b9000640 1 -- 172.21.15.33:0/1460517359 wait complete. 2024-05-15T14:00:23.062 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-15T14:00:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.176+0000 7faaf8e72640 1 -- 172.21.15.33:0/4057009302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaf41035f0 msgr2=0x7faaf41059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.176+0000 7faaf8e72640 1 --2- 172.21.15.33:0/4057009302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaf41035f0 0x7faaf41059e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7faae4009900 tx=0x7faae402a410 comp rx=0 tx=0).stop 2024-05-15T14:00:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.176+0000 7faaf8e72640 1 -- 172.21.15.33:0/4057009302 shutdown_connections 2024-05-15T14:00:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.176+0000 7faaf8e72640 1 --2- 172.21.15.33:0/4057009302 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faaf4108850 0x7faaf410ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.176+0000 7faaf8e72640 1 --2- 172.21.15.33:0/4057009302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faaf4105f20 0x7faaf4108310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.176+0000 7faaf8e72640 1 --2- 172.21.15.33:0/4057009302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaf41035f0 0x7faaf41059e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.176+0000 7faaf8e72640 1 -- 172.21.15.33:0/4057009302 >> 172.21.15.33:0/4057009302 conn(0x7faaf40fcf80 msgr2=0x7faaf40ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.176+0000 7faaf8e72640 1 -- 172.21.15.33:0/4057009302 shutdown_connections 2024-05-15T14:00:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.176+0000 7faaf8e72640 1 -- 172.21.15.33:0/4057009302 wait complete. 2024-05-15T14:00:23.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.177+0000 7faaf8e72640 1 Processor -- start 2024-05-15T14:00:23.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.177+0000 7faaf8e72640 1 -- start start 2024-05-15T14:00:23.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.177+0000 7faaf8e72640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faaf41035f0 0x7faaf4198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:23.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.177+0000 7faaf8e72640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faaf4105f20 0x7faaf4199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:23.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.178+0000 7faaf8e72640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaf4108850 0x7faaf41a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:23.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.178+0000 7faaf8e72640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faaf410e640 con 0x7faaf4108850 2024-05-15T14:00:23.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.178+0000 7faaf8e72640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faaf410e4c0 con 0x7faaf41035f0 2024-05-15T14:00:23.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.178+0000 7faaf8e72640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7faaf410e7c0 con 0x7faaf4105f20 2024-05-15T14:00:23.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.178+0000 7faaf2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaf4108850 0x7faaf41a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:23.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.178+0000 7faaf2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaf4108850 0x7faaf41a02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60004/0 (socket says 172.21.15.33:60004) 2024-05-15T14:00:23.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.178+0000 7faaf2d9d640 1 -- 172.21.15.33:0/3739014461 learned_addr learned my addr 172.21.15.33:0/3739014461 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:23.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.178+0000 7faaf259c640 1 --2- 172.21.15.33:0/3739014461 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faaf41035f0 0x7faaf4198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:23.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.178+0000 7faaf1d9b640 1 --2- 172.21.15.33:0/3739014461 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faaf4105f20 0x7faaf4199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:23.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.178+0000 7faaf2d9d640 1 -- 172.21.15.33:0/3739014461 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faaf4105f20 msgr2=0x7faaf4199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:23.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.178+0000 7faaf2d9d640 1 --2- 172.21.15.33:0/3739014461 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faaf4105f20 0x7faaf4199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:23.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.178+0000 7faaf2d9d640 1 -- 172.21.15.33:0/3739014461 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faaf41035f0 msgr2=0x7faaf4198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:23.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.178+0000 7faaf2d9d640 1 --2- 172.21.15.33:0/3739014461 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faaf41035f0 0x7faaf4198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:23.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.178+0000 7faaf2d9d640 1 -- 172.21.15.33:0/3739014461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf41a0960 con 0x7faaf4108850 2024-05-15T14:00:23.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.178+0000 7faaf2d9d640 1 --2- 172.21.15.33:0/3739014461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaf4108850 0x7faaf41a02f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7faaec00eb10 tx=0x7faaec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:23.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.179+0000 7faaeb7fe640 1 -- 172.21.15.33:0/3739014461 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec004b20 con 0x7faaf4108850 2024-05-15T14:00:23.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.179+0000 7faaeb7fe640 1 -- 172.21.15.33:0/3739014461 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaec004cc0 con 0x7faaf4108850 2024-05-15T14:00:23.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.179+0000 7faaf8e72640 1 -- 172.21.15.33:0/3739014461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaf41a0c50 con 0x7faaf4108850 2024-05-15T14:00:23.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.179+0000 7faaeb7fe640 1 -- 172.21.15.33:0/3739014461 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec010670 con 0x7faaf4108850 2024-05-15T14:00:23.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.179+0000 7faaf8e72640 1 -- 172.21.15.33:0/3739014461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaf41a11a0 con 0x7faaf4108850 2024-05-15T14:00:23.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.180+0000 7faaf8e72640 1 -- 172.21.15.33:0/3739014461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaf40fea70 con 0x7faaf4108850 2024-05-15T14:00:23.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.181+0000 7faaeb7fe640 1 -- 172.21.15.33:0/3739014461 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7faaec005020 con 0x7faaf4108850 2024-05-15T14:00:23.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.182+0000 7faaeb7fe640 1 --2- 172.21.15.33:0/3739014461 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7faae00b25e0 0x7faae00b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:23.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.182+0000 7faaeb7fe640 1 -- 172.21.15.33:0/3739014461 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7faaec0e2480 con 0x7faaf4108850 2024-05-15T14:00:23.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.182+0000 7faaf259c640 1 --2- 172.21.15.33:0/3739014461 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7faae00b25e0 0x7faae00b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:23.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.183+0000 7faaf259c640 1 --2- 172.21.15.33:0/3739014461 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7faae00b25e0 0x7faae00b4aa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7faae4000c00 tx=0x7faae40314a0 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:23.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.189+0000 7faaeb7fe640 1 -- 172.21.15.33:0/3739014461 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faaec0aa380 con 0x7faaf4108850 2024-05-15T14:00:23.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.204+0000 7faaeb7fe640 1 -- 172.21.15.33:0/3739014461 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 23) ==== 149772+0+0 (secure 0 0 0) 0x7faaec0a5f10 con 0x7faaf4108850 2024-05-15T14:00:23.368 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:23 smithi033 ceph-mon[31650]: mgrmap e22: x(active, since 1.01828s), standbys: y, z 2024-05-15T14:00:23.368 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:23 smithi033 ceph-mon[31650]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T14:00:23.369 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:23 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:23.369 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:23 smithi033 ceph-mon[31650]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T14:00:23.369 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:23 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:23.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.417+0000 7faaf8e72640 1 -- 172.21.15.33:0/3739014461 --> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7faaf4062e90 con 0x7faae00b25e0 2024-05-15T14:00:23.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.424+0000 7faaeb7fe640 1 -- 172.21.15.33:0/3739014461 <== mgr.15144 v2:172.21.15.33:6800/1796050977 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7faaf4062e90 con 0x7faae00b25e0 2024-05-15T14:00:23.426 INFO:teuthology.orchestra.run.smithi033.stdout:/ 1m 6m3h 2024-05-15T14:00:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.426+0000 7faaf8e72640 1 -- 172.21.15.33:0/3739014461 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7faae00b25e0 msgr2=0x7faae00b4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.426+0000 7faaf8e72640 1 --2- 172.21.15.33:0/3739014461 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7faae00b25e0 0x7faae00b4aa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7faae4000c00 tx=0x7faae40314a0 comp rx=0 tx=0).stop 2024-05-15T14:00:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.426+0000 7faaf8e72640 1 -- 172.21.15.33:0/3739014461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaf4108850 msgr2=0x7faaf41a02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.426+0000 7faaf8e72640 1 --2- 172.21.15.33:0/3739014461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaf4108850 0x7faaf41a02f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7faaec00eb10 tx=0x7faaec00c2d0 comp rx=0 tx=0).stop 2024-05-15T14:00:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.426+0000 7faaf8e72640 1 -- 172.21.15.33:0/3739014461 shutdown_connections 2024-05-15T14:00:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.426+0000 7faaf8e72640 1 --2- 172.21.15.33:0/3739014461 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7faae00b25e0 0x7faae00b4aa0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.426+0000 7faaf8e72640 1 --2- 172.21.15.33:0/3739014461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaf4108850 0x7faaf41a02f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.426+0000 7faaf8e72640 1 --2- 172.21.15.33:0/3739014461 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faaf4105f20 0x7faaf4199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.426+0000 7faaf8e72640 1 --2- 172.21.15.33:0/3739014461 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faaf41035f0 0x7faaf4198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.426+0000 7faaf8e72640 1 -- 172.21.15.33:0/3739014461 >> 172.21.15.33:0/3739014461 conn(0x7faaf40fcf80 msgr2=0x7faaf41040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.426+0000 7faaf8e72640 1 -- 172.21.15.33:0/3739014461 shutdown_connections 2024-05-15T14:00:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:23.426+0000 7faaf8e72640 1 -- 172.21.15.33:0/3739014461 wait complete. 2024-05-15T14:00:23.439 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-05-15T14:00:23.465 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-05-15T14:00:23.477 INFO:teuthology.run_tasks:Running task background_exec... 2024-05-15T14:00:23.486 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-05-15T14:00:23.486 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-05-15T14:00:23.508 INFO:teuthology.run_tasks:Running task quiescer... 2024-05-15T14:00:23.554 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-15T14:00:23.555 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-15T14:00:23.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:23 smithi052 ceph-mon[36190]: mgrmap e22: x(active, since 1.01828s), standbys: y, z 2024-05-15T14:00:23.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:23 smithi052 ceph-mon[36190]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T14:00:23.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:23 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:23.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:23 smithi052 ceph-mon[36190]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T14:00:23.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:23 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:23.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:23 smithi081 ceph-mon[36434]: mgrmap e22: x(active, since 1.01828s), standbys: y, z 2024-05-15T14:00:23.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:23 smithi081 ceph-mon[36434]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T14:00:23.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:23 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:23.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:23 smithi081 ceph-mon[36434]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T14:00:23.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:23 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:23.826 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:00:24.249 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:24 smithi033 ceph-mon[31650]: [15/May/2024:14:00:22] ENGINE Bus STARTING 2024-05-15T14:00:24.249 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:24 smithi033 ceph-mon[31650]: [15/May/2024:14:00:22] ENGINE Serving on http://172.21.15.33:8765 2024-05-15T14:00:24.249 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:24 smithi033 ceph-mon[31650]: [15/May/2024:14:00:22] ENGINE Serving on https://172.21.15.33:7150 2024-05-15T14:00:24.249 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:24 smithi033 ceph-mon[31650]: [15/May/2024:14:00:22] ENGINE Bus STARTED 2024-05-15T14:00:24.249 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:24 smithi033 ceph-mon[31650]: from='client.24940 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T14:00:24.249 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:24 smithi033 ceph-mon[31650]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T14:00:24.249 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:24 smithi033 ceph-mon[31650]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-05-15T14:00:24.250 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:24 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:24.250 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:24 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:24.250 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:24 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:24.250 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:24 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:24.250 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:24 smithi033 ceph-mon[31650]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T14:00:24.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:24 smithi052 ceph-mon[36190]: [15/May/2024:14:00:22] ENGINE Bus STARTING 2024-05-15T14:00:24.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:24 smithi052 ceph-mon[36190]: [15/May/2024:14:00:22] ENGINE Serving on http://172.21.15.33:8765 2024-05-15T14:00:24.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:24 smithi052 ceph-mon[36190]: [15/May/2024:14:00:22] ENGINE Serving on https://172.21.15.33:7150 2024-05-15T14:00:24.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:24 smithi052 ceph-mon[36190]: [15/May/2024:14:00:22] ENGINE Bus STARTED 2024-05-15T14:00:24.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:24 smithi052 ceph-mon[36190]: from='client.24940 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T14:00:24.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:24 smithi052 ceph-mon[36190]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T14:00:24.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:24 smithi052 ceph-mon[36190]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-05-15T14:00:24.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:24 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:24.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:24 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:24.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:24 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:24.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:24 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:24.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:24 smithi052 ceph-mon[36190]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T14:00:24.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:24 smithi081 ceph-mon[36434]: [15/May/2024:14:00:22] ENGINE Bus STARTING 2024-05-15T14:00:24.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:24 smithi081 ceph-mon[36434]: [15/May/2024:14:00:22] ENGINE Serving on http://172.21.15.33:8765 2024-05-15T14:00:24.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:24 smithi081 ceph-mon[36434]: [15/May/2024:14:00:22] ENGINE Serving on https://172.21.15.33:7150 2024-05-15T14:00:24.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:24 smithi081 ceph-mon[36434]: [15/May/2024:14:00:22] ENGINE Bus STARTED 2024-05-15T14:00:24.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:24 smithi081 ceph-mon[36434]: from='client.24940 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T14:00:24.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:24 smithi081 ceph-mon[36434]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T14:00:24.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:24 smithi081 ceph-mon[36434]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-05-15T14:00:24.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:24 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:24.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:24 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:24.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:24 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:24.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:24 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:24.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:24 smithi081 ceph-mon[36434]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T14:00:25.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.385+0000 7f3297bc5700 1 -- 172.21.15.33:0/2031407585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3290102e00 msgr2=0x7f329010f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:25.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.385+0000 7f3297bc5700 1 --2- 172.21.15.33:0/2031407585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3290102e00 0x7f329010f2f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f328800b0a0 tx=0x7f328800b3b0 comp rx=0 tx=0).stop 2024-05-15T14:00:25.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.385+0000 7f3297bc5700 1 -- 172.21.15.33:0/2031407585 shutdown_connections 2024-05-15T14:00:25.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.385+0000 7f3297bc5700 1 --2- 172.21.15.33:0/2031407585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3290102e00 0x7f329010f2f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:25.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.385+0000 7f3297bc5700 1 --2- 172.21.15.33:0/2031407585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3290108520 0x7f3290108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:25.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.385+0000 7f3297bc5700 1 --2- 172.21.15.33:0/2031407585 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3290102440 0x7f32901028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:25.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.385+0000 7f3297bc5700 1 -- 172.21.15.33:0/2031407585 >> 172.21.15.33:0/2031407585 conn(0x7f32900793f0 msgr2=0x7f3290079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:25.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.385+0000 7f3297bc5700 1 -- 172.21.15.33:0/2031407585 shutdown_connections 2024-05-15T14:00:25.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.386+0000 7f3297bc5700 1 -- 172.21.15.33:0/2031407585 wait complete. 2024-05-15T14:00:25.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.386+0000 7f3297bc5700 1 Processor -- start 2024-05-15T14:00:25.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.386+0000 7f3297bc5700 1 -- start start 2024-05-15T14:00:25.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.386+0000 7f3297bc5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3290102440 0x7f329019e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:25.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.386+0000 7f3297bc5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3290102e00 0x7f329019ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:25.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.386+0000 7f3297bc5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3290108520 0x7f3290198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:25.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.386+0000 7f3297bc5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f32901125d0 con 0x7f3290108520 2024-05-15T14:00:25.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.386+0000 7f3297bc5700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3290112450 con 0x7f3290102440 2024-05-15T14:00:25.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.386+0000 7f3297bc5700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3290112750 con 0x7f3290102e00 2024-05-15T14:00:25.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.387+0000 7f3296162700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3290108520 0x7f3290198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:25.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.387+0000 7f3296162700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3290108520 0x7f3290198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60044/0 (socket says 172.21.15.33:60044) 2024-05-15T14:00:25.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.387+0000 7f3296162700 1 -- 172.21.15.33:0/3090829224 learned_addr learned my addr 172.21.15.33:0/3090829224 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:25.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.387+0000 7f3295961700 1 --2- 172.21.15.33:0/3090829224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3290102440 0x7f329019e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:25.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.387+0000 7f3295160700 1 --2- 172.21.15.33:0/3090829224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3290102e00 0x7f329019ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:25.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.387+0000 7f3296162700 1 -- 172.21.15.33:0/3090829224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3290102e00 msgr2=0x7f329019ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:25.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.387+0000 7f3296162700 1 --2- 172.21.15.33:0/3090829224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3290102e00 0x7f329019ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:25.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.387+0000 7f3296162700 1 -- 172.21.15.33:0/3090829224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3290102440 msgr2=0x7f329019e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:25.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.387+0000 7f3296162700 1 --2- 172.21.15.33:0/3090829224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3290102440 0x7f329019e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:25.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.387+0000 7f3296162700 1 -- 172.21.15.33:0/3090829224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3290199070 con 0x7f3290108520 2024-05-15T14:00:25.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.387+0000 7f3296162700 1 --2- 172.21.15.33:0/3090829224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3290108520 0x7f3290198910 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f328800b830 tx=0x7f3288004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:25.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.387+0000 7f3286ffd700 1 -- 172.21.15.33:0/3090829224 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3288021070 con 0x7f3290108520 2024-05-15T14:00:25.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.387+0000 7f3297bc5700 1 -- 172.21.15.33:0/3090829224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3290199300 con 0x7f3290108520 2024-05-15T14:00:25.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.387+0000 7f3286ffd700 1 -- 172.21.15.33:0/3090829224 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3288007e30 con 0x7f3290108520 2024-05-15T14:00:25.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.387+0000 7f3286ffd700 1 -- 172.21.15.33:0/3090829224 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32880047d0 con 0x7f3290108520 2024-05-15T14:00:25.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.388+0000 7f3297bc5700 1 -- 172.21.15.33:0/3090829224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f329006b220 con 0x7f3290108520 2024-05-15T14:00:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.388+0000 7f3297bc5700 1 -- 172.21.15.33:0/3090829224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f329006a230 con 0x7f3290108520 2024-05-15T14:00:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.390+0000 7f3286ffd700 1 -- 172.21.15.33:0/3090829224 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 23) ==== 149772+0+0 (secure 0 0 0) 0x7f3288007980 con 0x7f3290108520 2024-05-15T14:00:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.391+0000 7f3286ffd700 1 --2- 172.21.15.33:0/3090829224 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f327c0b29f0 0x7f327c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:25.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.391+0000 7f3295961700 1 --2- 172.21.15.33:0/3090829224 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f327c0b29f0 0x7f327c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:25.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.391+0000 7f3286ffd700 1 -- 172.21.15.33:0/3090829224 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f32880e36f0 con 0x7f3290108520 2024-05-15T14:00:25.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.391+0000 7f3295961700 1 --2- 172.21.15.33:0/3090829224 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f327c0b29f0 0x7f327c0b4eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f328c005950 tx=0x7f328c00a300 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:25.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.394+0000 7f3286ffd700 1 -- 172.21.15.33:0/3090829224 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f32880ab110 con 0x7f3290108520 2024-05-15T14:00:25.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:25 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:25.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:25 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:25.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:25 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:25.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:25 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:25.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:25 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:25.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:25 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:25.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:25 smithi033 ceph-mon[31650]: Adjusting osd_memory_target on smithi081 to 176.2M 2024-05-15T14:00:25.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:25 smithi033 ceph-mon[31650]: Unable to set osd_memory_target on smithi081 to 184783155: error parsing value: Value '184783155' is below minimum 939524096 2024-05-15T14:00:25.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:25 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:25.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:25 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:25.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:25 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:25.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:25 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:25.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:25 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:25.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:25 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:25.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.553+0000 7f3286ffd700 1 -- 172.21.15.33:0/3090829224 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f32880a6ca0 con 0x7f3290108520 2024-05-15T14:00:25.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:25 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:25.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:25 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:25.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:25 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:25.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:25 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:25.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:25 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:25.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:25 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:25.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:25 smithi052 ceph-mon[36190]: Adjusting osd_memory_target on smithi081 to 176.2M 2024-05-15T14:00:25.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:25 smithi052 ceph-mon[36190]: Unable to set osd_memory_target on smithi081 to 184783155: error parsing value: Value '184783155' is below minimum 939524096 2024-05-15T14:00:25.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:25 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:25.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:25 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:25.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:25 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:25.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:25 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:25.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:25 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:25.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:25 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:25.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:25 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:25.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:25 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:25.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:25 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:25.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:25 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:25.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:25 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:25.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:25 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:25.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:25 smithi081 ceph-mon[36434]: Adjusting osd_memory_target on smithi081 to 176.2M 2024-05-15T14:00:25.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:25 smithi081 ceph-mon[36434]: Unable to set osd_memory_target on smithi081 to 184783155: error parsing value: Value '184783155' is below minimum 939524096 2024-05-15T14:00:25.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:25 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:25.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:25 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:25.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:25 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:25.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:25 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:25.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:25 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:25.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:25 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:25.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.646+0000 7f3297bc5700 1 -- 172.21.15.33:0/3090829224 --> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f329006b480 con 0x7f327c0b29f0 2024-05-15T14:00:25.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.662+0000 7f3286ffd700 1 -- 172.21.15.33:0/3090829224 <== mgr.15144 v2:172.21.15.33:6800/1796050977 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257730 (secure 0 0 0) 0x7f329006b480 con 0x7f327c0b29f0 2024-05-15T14:00:25.671 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T14:00:25.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.666+0000 7f3297bc5700 1 -- 172.21.15.33:0/3090829224 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f327c0b29f0 msgr2=0x7f327c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:25.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.666+0000 7f3297bc5700 1 --2- 172.21.15.33:0/3090829224 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f327c0b29f0 0x7f327c0b4eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f328c005950 tx=0x7f328c00a300 comp rx=0 tx=0).stop 2024-05-15T14:00:25.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.666+0000 7f3297bc5700 1 -- 172.21.15.33:0/3090829224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3290108520 msgr2=0x7f3290198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:25.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.666+0000 7f3297bc5700 1 --2- 172.21.15.33:0/3090829224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3290108520 0x7f3290198910 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f328800b830 tx=0x7f3288004ce0 comp rx=0 tx=0).stop 2024-05-15T14:00:25.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.666+0000 7f3297bc5700 1 -- 172.21.15.33:0/3090829224 shutdown_connections 2024-05-15T14:00:25.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.666+0000 7f3297bc5700 1 --2- 172.21.15.33:0/3090829224 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f327c0b29f0 0x7f327c0b4eb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:25.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.666+0000 7f3297bc5700 1 --2- 172.21.15.33:0/3090829224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3290108520 0x7f3290198910 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:25.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.666+0000 7f3297bc5700 1 --2- 172.21.15.33:0/3090829224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3290102440 0x7f329019e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:25.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.666+0000 7f3297bc5700 1 --2- 172.21.15.33:0/3090829224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3290102e00 0x7f329019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:25.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.666+0000 7f3297bc5700 1 -- 172.21.15.33:0/3090829224 >> 172.21.15.33:0/3090829224 conn(0x7f32900793f0 msgr2=0x7f3290100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:25.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.666+0000 7f3297bc5700 1 -- 172.21.15.33:0/3090829224 shutdown_connections 2024-05-15T14:00:25.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:25.666+0000 7f3297bc5700 1 -- 172.21.15.33:0/3090829224 wait complete. 2024-05-15T14:00:25.677 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-05-15T14:00:26.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:26 smithi033 ceph-mon[31650]: Adjusting osd_memory_target on smithi052 to 177.2M 2024-05-15T14:00:26.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:26 smithi033 ceph-mon[31650]: Unable to set osd_memory_target on smithi052 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-05-15T14:00:26.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:26 smithi033 ceph-mon[31650]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T14:00:26.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:26 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:26.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:26 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:26.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:26 smithi033 ceph-mon[31650]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-05-15T14:00:26.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:26 smithi052 ceph-mon[36190]: Adjusting osd_memory_target on smithi052 to 177.2M 2024-05-15T14:00:26.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:26 smithi052 ceph-mon[36190]: Unable to set osd_memory_target on smithi052 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-05-15T14:00:26.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:26 smithi052 ceph-mon[36190]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T14:00:26.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:26 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:26.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:26 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:26.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:26 smithi052 ceph-mon[36190]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-05-15T14:00:26.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:26 smithi081 ceph-mon[36434]: Adjusting osd_memory_target on smithi052 to 177.2M 2024-05-15T14:00:26.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:26 smithi081 ceph-mon[36434]: Unable to set osd_memory_target on smithi052 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-05-15T14:00:26.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:26 smithi081 ceph-mon[36434]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T14:00:26.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:26 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:26.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:26 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:26.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:26 smithi081 ceph-mon[36434]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-05-15T14:00:26.619 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-05-15T14:00:25.176712+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1188241,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1211,"num_read_kb":744,"num_write":418,"num_write_kb":3592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2951974,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":824,"ondisk_log_size":824,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":384300,"kb_used_data":14320,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124557524,"statfs":{"total":1151940427776,"available":1151546904576,"internally_reserved":0,"allocated":14663680,"data_stored":8838488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.988199"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.581443+0000","last_change":"2024-05-15T13:57:17.382728+0000","last_active":"2024-05-15T14:00:21.581443+0000","last_peered":"2024-05-15T14:00:21.581443+0000","last_clean":"2024-05-15T14:00:21.581443+0000","last_became_active":"2024-05-15T13:57:17.382315+0000","last_became_peered":"2024-05-15T13:57:17.382315+0000","last_unstale":"2024-05-15T14:00:21.581443+0000","last_undegraded":"2024-05-15T14:00:21.581443+0000","last_fullsized":"2024-05-15T14:00:21.581443+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:03:53.137916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.417116+0000","last_change":"2024-05-15T13:57:17.390880+0000","last_active":"2024-05-15T14:00:21.417116+0000","last_peered":"2024-05-15T14:00:21.417116+0000","last_clean":"2024-05-15T14:00:21.417116+0000","last_became_active":"2024-05-15T13:57:17.390560+0000","last_became_peered":"2024-05-15T13:57:17.390560+0000","last_unstale":"2024-05-15T14:00:21.417116+0000","last_undegraded":"2024-05-15T14:00:21.417116+0000","last_fullsized":"2024-05-15T14:00:21.417116+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:59:33.615670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.416802+0000","last_change":"2024-05-15T13:57:17.386495+0000","last_active":"2024-05-15T14:00:21.416802+0000","last_peered":"2024-05-15T14:00:21.416802+0000","last_clean":"2024-05-15T14:00:21.416802+0000","last_became_active":"2024-05-15T13:57:17.386183+0000","last_became_peered":"2024-05-15T13:57:17.386183+0000","last_unstale":"2024-05-15T14:00:21.416802+0000","last_undegraded":"2024-05-15T14:00:21.416802+0000","last_fullsized":"2024-05-15T14:00:21.416802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:36:13.202922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.164211+0000","last_change":"2024-05-15T13:57:17.385437+0000","last_active":"2024-05-15T14:00:21.164211+0000","last_peered":"2024-05-15T14:00:21.164211+0000","last_clean":"2024-05-15T14:00:21.164211+0000","last_became_active":"2024-05-15T13:57:17.384524+0000","last_became_peered":"2024-05-15T13:57:17.384524+0000","last_unstale":"2024-05-15T14:00:21.164211+0000","last_undegraded":"2024-05-15T14:00:21.164211+0000","last_fullsized":"2024-05-15T14:00:21.164211+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:45:17.207819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260575+0000","last_change":"2024-05-15T13:57:17.387859+0000","last_active":"2024-05-15T14:00:21.260575+0000","last_peered":"2024-05-15T14:00:21.260575+0000","last_clean":"2024-05-15T14:00:21.260575+0000","last_became_active":"2024-05-15T13:57:17.387491+0000","last_became_peered":"2024-05-15T13:57:17.387491+0000","last_unstale":"2024-05-15T14:00:21.260575+0000","last_undegraded":"2024-05-15T14:00:21.260575+0000","last_fullsized":"2024-05-15T14:00:21.260575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:43:08.212926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.269925+0000","last_change":"2024-05-15T13:57:17.382289+0000","last_active":"2024-05-15T14:00:21.269925+0000","last_peered":"2024-05-15T14:00:21.269925+0000","last_clean":"2024-05-15T14:00:21.269925+0000","last_became_active":"2024-05-15T13:57:17.380494+0000","last_became_peered":"2024-05-15T13:57:17.380494+0000","last_unstale":"2024-05-15T14:00:21.269925+0000","last_undegraded":"2024-05-15T14:00:21.269925+0000","last_fullsized":"2024-05-15T14:00:21.269925+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:11:37.097034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414501+0000","last_change":"2024-05-15T13:58:06.335776+0000","last_active":"2024-05-15T14:00:21.414501+0000","last_peered":"2024-05-15T14:00:21.414501+0000","last_clean":"2024-05-15T14:00:21.414501+0000","last_became_active":"2024-05-15T13:58:06.335239+0000","last_became_peered":"2024-05-15T13:58:06.335239+0000","last_unstale":"2024-05-15T14:00:21.414501+0000","last_undegraded":"2024-05-15T14:00:21.414501+0000","last_fullsized":"2024-05-15T14:00:21.414501+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:53:54.469082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260684+0000","last_change":"2024-05-15T13:57:17.380969+0000","last_active":"2024-05-15T14:00:21.260684+0000","last_peered":"2024-05-15T14:00:21.260684+0000","last_clean":"2024-05-15T14:00:21.260684+0000","last_became_active":"2024-05-15T13:57:17.380660+0000","last_became_peered":"2024-05-15T13:57:17.380660+0000","last_unstale":"2024-05-15T14:00:21.260684+0000","last_undegraded":"2024-05-15T14:00:21.260684+0000","last_fullsized":"2024-05-15T14:00:21.260684+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:01:41.790323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.331089+0000","last_change":"2024-05-15T13:57:17.394943+0000","last_active":"2024-05-15T14:00:21.331089+0000","last_peered":"2024-05-15T14:00:21.331089+0000","last_clean":"2024-05-15T14:00:21.331089+0000","last_became_active":"2024-05-15T13:57:17.394571+0000","last_became_peered":"2024-05-15T13:57:17.394571+0000","last_unstale":"2024-05-15T14:00:21.331089+0000","last_undegraded":"2024-05-15T14:00:21.331089+0000","last_fullsized":"2024-05-15T14:00:21.331089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:39:26.063096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.270341+0000","last_change":"2024-05-15T13:57:17.386012+0000","last_active":"2024-05-15T14:00:21.270341+0000","last_peered":"2024-05-15T14:00:21.270341+0000","last_clean":"2024-05-15T14:00:21.270341+0000","last_became_active":"2024-05-15T13:57:17.385636+0000","last_became_peered":"2024-05-15T13:57:17.385636+0000","last_unstale":"2024-05-15T14:00:21.270341+0000","last_undegraded":"2024-05-15T14:00:21.270341+0000","last_fullsized":"2024-05-15T14:00:21.270341+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:12:08.465959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414078+0000","last_change":"2024-05-15T13:57:17.389162+0000","last_active":"2024-05-15T14:00:21.414078+0000","last_peered":"2024-05-15T14:00:21.414078+0000","last_clean":"2024-05-15T14:00:21.414078+0000","last_became_active":"2024-05-15T13:57:17.388503+0000","last_became_peered":"2024-05-15T13:57:17.388503+0000","last_unstale":"2024-05-15T14:00:21.414078+0000","last_undegraded":"2024-05-15T14:00:21.414078+0000","last_fullsized":"2024-05-15T14:00:21.414078+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:24:54.354445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.164093+0000","last_change":"2024-05-15T13:57:17.381689+0000","last_active":"2024-05-15T14:00:21.164093+0000","last_peered":"2024-05-15T14:00:21.164093+0000","last_clean":"2024-05-15T14:00:21.164093+0000","last_became_active":"2024-05-15T13:57:17.381376+0000","last_became_peered":"2024-05-15T13:57:17.381376+0000","last_unstale":"2024-05-15T14:00:21.164093+0000","last_undegraded":"2024-05-15T14:00:21.164093+0000","last_fullsized":"2024-05-15T14:00:21.164093+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:03:52.652934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.413932+0000","last_change":"2024-05-15T13:57:17.382214+0000","last_active":"2024-05-15T14:00:21.413932+0000","last_peered":"2024-05-15T14:00:21.413932+0000","last_clean":"2024-05-15T14:00:21.413932+0000","last_became_active":"2024-05-15T13:57:17.381851+0000","last_became_peered":"2024-05-15T13:57:17.381851+0000","last_unstale":"2024-05-15T14:00:21.413932+0000","last_undegraded":"2024-05-15T14:00:21.413932+0000","last_fullsized":"2024-05-15T14:00:21.413932+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:18:39.854918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.416134+0000","last_change":"2024-05-15T13:57:17.390985+0000","last_active":"2024-05-15T14:00:21.416134+0000","last_peered":"2024-05-15T14:00:21.416134+0000","last_clean":"2024-05-15T14:00:21.416134+0000","last_became_active":"2024-05-15T13:57:17.390633+0000","last_became_peered":"2024-05-15T13:57:17.390633+0000","last_unstale":"2024-05-15T14:00:21.416134+0000","last_undegraded":"2024-05-15T14:00:21.416134+0000","last_fullsized":"2024-05-15T14:00:21.416134+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:32:47.177507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.416566+0000","last_change":"2024-05-15T13:57:17.383943+0000","last_active":"2024-05-15T14:00:21.416566+0000","last_peered":"2024-05-15T14:00:21.416566+0000","last_clean":"2024-05-15T14:00:21.416566+0000","last_became_active":"2024-05-15T13:57:17.383622+0000","last_became_peered":"2024-05-15T13:57:17.383622+0000","last_unstale":"2024-05-15T14:00:21.416566+0000","last_undegraded":"2024-05-15T14:00:21.416566+0000","last_fullsized":"2024-05-15T14:00:21.416566+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T00:30:23.249813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.330635+0000","last_change":"2024-05-15T13:57:17.390789+0000","last_active":"2024-05-15T14:00:21.330635+0000","last_peered":"2024-05-15T14:00:21.330635+0000","last_clean":"2024-05-15T14:00:21.330635+0000","last_became_active":"2024-05-15T13:57:17.390466+0000","last_became_peered":"2024-05-15T13:57:17.390466+0000","last_unstale":"2024-05-15T14:00:21.330635+0000","last_undegraded":"2024-05-15T14:00:21.330635+0000","last_fullsized":"2024-05-15T14:00:21.330635+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:07:24.746862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.581494+0000","last_change":"2024-05-15T13:57:17.384080+0000","last_active":"2024-05-15T14:00:21.581494+0000","last_peered":"2024-05-15T14:00:21.581494+0000","last_clean":"2024-05-15T14:00:21.581494+0000","last_became_active":"2024-05-15T13:57:17.383731+0000","last_became_peered":"2024-05-15T13:57:17.383731+0000","last_unstale":"2024-05-15T14:00:21.581494+0000","last_undegraded":"2024-05-15T14:00:21.581494+0000","last_fullsized":"2024-05-15T14:00:21.581494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:36:22.874337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.162965+0000","last_change":"2024-05-15T13:57:17.382209+0000","last_active":"2024-05-15T14:00:21.162965+0000","last_peered":"2024-05-15T14:00:21.162965+0000","last_clean":"2024-05-15T14:00:21.162965+0000","last_became_active":"2024-05-15T13:57:17.381737+0000","last_became_peered":"2024-05-15T13:57:17.381737+0000","last_unstale":"2024-05-15T14:00:21.162965+0000","last_undegraded":"2024-05-15T14:00:21.162965+0000","last_fullsized":"2024-05-15T14:00:21.162965+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:38:53.913745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.166353+0000","last_change":"2024-05-15T13:58:06.335718+0000","last_active":"2024-05-15T14:00:21.166353+0000","last_peered":"2024-05-15T14:00:21.166353+0000","last_clean":"2024-05-15T14:00:21.166353+0000","last_became_active":"2024-05-15T13:58:06.335383+0000","last_became_peered":"2024-05-15T13:58:06.335383+0000","last_unstale":"2024-05-15T14:00:21.166353+0000","last_undegraded":"2024-05-15T14:00:21.166353+0000","last_fullsized":"2024-05-15T14:00:21.166353+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T00:19:47.658973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.162928+0000","last_change":"2024-05-15T13:57:14.256057+0000","last_active":"2024-05-15T14:00:21.162928+0000","last_peered":"2024-05-15T14:00:21.162928+0000","last_clean":"2024-05-15T14:00:21.162928+0000","last_became_active":"2024-05-15T13:57:14.255413+0000","last_became_peered":"2024-05-15T13:57:14.255413+0000","last_unstale":"2024-05-15T14:00:21.162928+0000","last_undegraded":"2024-05-15T14:00:21.162928+0000","last_fullsized":"2024-05-15T14:00:21.162928+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:13:43.433634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.261168+0000","last_change":"2024-05-15T13:57:17.390548+0000","last_active":"2024-05-15T14:00:21.261168+0000","last_peered":"2024-05-15T14:00:21.261168+0000","last_clean":"2024-05-15T14:00:21.261168+0000","last_became_active":"2024-05-15T13:57:17.390144+0000","last_became_peered":"2024-05-15T13:57:17.390144+0000","last_unstale":"2024-05-15T14:00:21.261168+0000","last_undegraded":"2024-05-15T14:00:21.261168+0000","last_fullsized":"2024-05-15T14:00:21.261168+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:28:54.889994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.261392+0000","last_change":"2024-05-15T13:57:14.535704+0000","last_active":"2024-05-15T14:00:21.261392+0000","last_peered":"2024-05-15T14:00:21.261392+0000","last_clean":"2024-05-15T14:00:21.261392+0000","last_became_active":"2024-05-15T13:57:14.535356+0000","last_became_peered":"2024-05-15T13:57:14.535356+0000","last_unstale":"2024-05-15T14:00:21.261392+0000","last_undegraded":"2024-05-15T14:00:21.261392+0000","last_fullsized":"2024-05-15T14:00:21.261392+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:16:21.545507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.417396+0000","last_change":"2024-05-15T13:57:17.383482+0000","last_active":"2024-05-15T14:00:21.417396+0000","last_peered":"2024-05-15T14:00:21.417396+0000","last_clean":"2024-05-15T14:00:21.417396+0000","last_became_active":"2024-05-15T13:57:17.382973+0000","last_became_peered":"2024-05-15T13:57:17.382973+0000","last_unstale":"2024-05-15T14:00:21.417396+0000","last_undegraded":"2024-05-15T14:00:21.417396+0000","last_fullsized":"2024-05-15T14:00:21.417396+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:31:08.937166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.581765+0000","last_change":"2024-05-15T13:58:06.078918+0000","last_active":"2024-05-15T14:00:21.581765+0000","last_peered":"2024-05-15T14:00:21.581765+0000","last_clean":"2024-05-15T14:00:21.581765+0000","last_became_active":"2024-05-15T13:58:06.073173+0000","last_became_peered":"2024-05-15T13:58:06.073173+0000","last_unstale":"2024-05-15T14:00:21.581765+0000","last_undegraded":"2024-05-15T14:00:21.581765+0000","last_fullsized":"2024-05-15T14:00:21.581765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:21:10.582131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.330748+0000","last_change":"2024-05-15T13:57:17.388720+0000","last_active":"2024-05-15T14:00:21.330748+0000","last_peered":"2024-05-15T14:00:21.330748+0000","last_clean":"2024-05-15T14:00:21.330748+0000","last_became_active":"2024-05-15T13:57:17.388386+0000","last_became_peered":"2024-05-15T13:57:17.388386+0000","last_unstale":"2024-05-15T14:00:21.330748+0000","last_undegraded":"2024-05-15T14:00:21.330748+0000","last_fullsized":"2024-05-15T14:00:21.330748+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:02:40.543701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260605+0000","last_change":"2024-05-15T13:57:14.259064+0000","last_active":"2024-05-15T14:00:21.260605+0000","last_peered":"2024-05-15T14:00:21.260605+0000","last_clean":"2024-05-15T14:00:21.260605+0000","last_became_active":"2024-05-15T13:57:14.258683+0000","last_became_peered":"2024-05-15T13:57:14.258683+0000","last_unstale":"2024-05-15T14:00:21.260605+0000","last_undegraded":"2024-05-15T14:00:21.260605+0000","last_fullsized":"2024-05-15T14:00:21.260605+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:12:43.108066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.266572+0000","last_change":"2024-05-15T13:57:17.380929+0000","last_active":"2024-05-15T14:00:21.266572+0000","last_peered":"2024-05-15T14:00:21.266572+0000","last_clean":"2024-05-15T14:00:21.266572+0000","last_became_active":"2024-05-15T13:57:17.380599+0000","last_became_peered":"2024-05-15T13:57:17.380599+0000","last_unstale":"2024-05-15T14:00:21.266572+0000","last_undegraded":"2024-05-15T14:00:21.266572+0000","last_fullsized":"2024-05-15T14:00:21.266572+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:19:55.368088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'9","reported_seq":211,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.616417+0000","last_change":"2024-05-15T13:57:14.260410+0000","last_active":"2024-05-15T14:00:21.616417+0000","last_peered":"2024-05-15T14:00:21.616417+0000","last_clean":"2024-05-15T14:00:21.616417+0000","last_became_active":"2024-05-15T13:57:14.260076+0000","last_became_peered":"2024-05-15T13:57:14.260076+0000","last_unstale":"2024-05-15T14:00:21.616417+0000","last_undegraded":"2024-05-15T14:00:21.616417+0000","last_fullsized":"2024-05-15T14:00:21.616417+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T00:27:03.436237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1853,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":173,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260404+0000","last_change":"2024-05-15T13:57:17.394475+0000","last_active":"2024-05-15T14:00:21.260404+0000","last_peered":"2024-05-15T14:00:21.260404+0000","last_clean":"2024-05-15T14:00:21.260404+0000","last_became_active":"2024-05-15T13:57:17.393733+0000","last_became_peered":"2024-05-15T13:57:17.393733+0000","last_unstale":"2024-05-15T14:00:21.260404+0000","last_undegraded":"2024-05-15T14:00:21.260404+0000","last_fullsized":"2024-05-15T14:00:21.260404+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:55:06.198633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.269838+0000","last_change":"2024-05-15T13:57:14.256420+0000","last_active":"2024-05-15T14:00:21.269838+0000","last_peered":"2024-05-15T14:00:21.269838+0000","last_clean":"2024-05-15T14:00:21.269838+0000","last_became_active":"2024-05-15T13:57:14.255722+0000","last_became_peered":"2024-05-15T13:57:14.255722+0000","last_unstale":"2024-05-15T14:00:21.269838+0000","last_undegraded":"2024-05-15T14:00:21.269838+0000","last_fullsized":"2024-05-15T14:00:21.269838+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:21:30.499632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.266546+0000","last_change":"2024-05-15T13:58:06.074623+0000","last_active":"2024-05-15T14:00:21.266546+0000","last_peered":"2024-05-15T14:00:21.266546+0000","last_clean":"2024-05-15T14:00:21.266546+0000","last_became_active":"2024-05-15T13:58:06.074153+0000","last_became_peered":"2024-05-15T13:58:06.074153+0000","last_unstale":"2024-05-15T14:00:21.266546+0000","last_undegraded":"2024-05-15T14:00:21.266546+0000","last_fullsized":"2024-05-15T14:00:21.266546+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:08:22.922239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414371+0000","last_change":"2024-05-15T13:57:14.256192+0000","last_active":"2024-05-15T14:00:21.414371+0000","last_peered":"2024-05-15T14:00:21.414371+0000","last_clean":"2024-05-15T14:00:21.414371+0000","last_became_active":"2024-05-15T13:57:14.255839+0000","last_became_peered":"2024-05-15T13:57:14.255839+0000","last_unstale":"2024-05-15T14:00:21.414371+0000","last_undegraded":"2024-05-15T14:00:21.414371+0000","last_fullsized":"2024-05-15T14:00:21.414371+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:13:26.071403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260676+0000","last_change":"2024-05-15T13:57:17.386246+0000","last_active":"2024-05-15T14:00:21.260676+0000","last_peered":"2024-05-15T14:00:21.260676+0000","last_clean":"2024-05-15T14:00:21.260676+0000","last_became_active":"2024-05-15T13:57:17.385828+0000","last_became_peered":"2024-05-15T13:57:17.385828+0000","last_unstale":"2024-05-15T14:00:21.260676+0000","last_undegraded":"2024-05-15T14:00:21.260676+0000","last_fullsized":"2024-05-15T14:00:21.260676+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T00:24:38.656469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.266243+0000","last_change":"2024-05-15T13:57:14.259953+0000","last_active":"2024-05-15T14:00:21.266243+0000","last_peered":"2024-05-15T14:00:21.266243+0000","last_clean":"2024-05-15T14:00:21.266243+0000","last_became_active":"2024-05-15T13:57:14.259632+0000","last_became_peered":"2024-05-15T13:57:14.259632+0000","last_unstale":"2024-05-15T14:00:21.266243+0000","last_undegraded":"2024-05-15T14:00:21.266243+0000","last_fullsized":"2024-05-15T14:00:21.266243+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:30:28.361903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.162534+0000","last_change":"2024-05-15T13:57:17.382211+0000","last_active":"2024-05-15T14:00:21.162534+0000","last_peered":"2024-05-15T14:00:21.162534+0000","last_clean":"2024-05-15T14:00:21.162534+0000","last_became_active":"2024-05-15T13:57:17.381904+0000","last_became_peered":"2024-05-15T13:57:17.381904+0000","last_unstale":"2024-05-15T14:00:21.162534+0000","last_undegraded":"2024-05-15T14:00:21.162534+0000","last_fullsized":"2024-05-15T14:00:21.162534+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:53:49.250866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'5","reported_seq":201,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.270340+0000","last_change":"2024-05-15T13:57:14.254691+0000","last_active":"2024-05-15T14:00:21.270340+0000","last_peered":"2024-05-15T14:00:21.270340+0000","last_clean":"2024-05-15T14:00:21.270340+0000","last_became_active":"2024-05-15T13:57:14.254233+0000","last_became_peered":"2024-05-15T13:57:14.254233+0000","last_unstale":"2024-05-15T14:00:21.270340+0000","last_undegraded":"2024-05-15T14:00:21.270340+0000","last_fullsized":"2024-05-15T14:00:21.270340+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:41:31.719402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":167,"num_read_kb":167,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.331127+0000","last_change":"2024-05-15T13:57:17.382251+0000","last_active":"2024-05-15T14:00:21.331127+0000","last_peered":"2024-05-15T14:00:21.331127+0000","last_clean":"2024-05-15T14:00:21.331127+0000","last_became_active":"2024-05-15T13:57:17.381158+0000","last_became_peered":"2024-05-15T13:57:17.381158+0000","last_unstale":"2024-05-15T14:00:21.331127+0000","last_undegraded":"2024-05-15T14:00:21.331127+0000","last_fullsized":"2024-05-15T14:00:21.331127+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:24:55.971079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414941+0000","last_change":"2024-05-15T13:57:14.258350+0000","last_active":"2024-05-15T14:00:21.414941+0000","last_peered":"2024-05-15T14:00:21.414941+0000","last_clean":"2024-05-15T14:00:21.414941+0000","last_became_active":"2024-05-15T13:57:14.257982+0000","last_became_peered":"2024-05-15T13:57:14.257982+0000","last_unstale":"2024-05-15T14:00:21.414941+0000","last_undegraded":"2024-05-15T14:00:21.414941+0000","last_fullsized":"2024-05-15T14:00:21.414941+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:50:07.053401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.417513+0000","last_change":"2024-05-15T13:57:17.392114+0000","last_active":"2024-05-15T14:00:21.417513+0000","last_peered":"2024-05-15T14:00:21.417513+0000","last_clean":"2024-05-15T14:00:21.417513+0000","last_became_active":"2024-05-15T13:57:17.391732+0000","last_became_peered":"2024-05-15T13:57:17.391732+0000","last_unstale":"2024-05-15T14:00:21.417513+0000","last_undegraded":"2024-05-15T14:00:21.417513+0000","last_fullsized":"2024-05-15T14:00:21.417513+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:28:02.304910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.266603+0000","last_change":"2024-05-15T13:57:14.535381+0000","last_active":"2024-05-15T14:00:21.266603+0000","last_peered":"2024-05-15T14:00:21.266603+0000","last_clean":"2024-05-15T14:00:21.266603+0000","last_became_active":"2024-05-15T13:57:14.534931+0000","last_became_peered":"2024-05-15T13:57:14.534931+0000","last_unstale":"2024-05-15T14:00:21.266603+0000","last_undegraded":"2024-05-15T14:00:21.266603+0000","last_fullsized":"2024-05-15T14:00:21.266603+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:13:04.785139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.330881+0000","last_change":"2024-05-15T13:57:17.386103+0000","last_active":"2024-05-15T14:00:21.330881+0000","last_peered":"2024-05-15T14:00:21.330881+0000","last_clean":"2024-05-15T14:00:21.330881+0000","last_became_active":"2024-05-15T13:57:17.385738+0000","last_became_peered":"2024-05-15T13:57:17.385738+0000","last_unstale":"2024-05-15T14:00:21.330881+0000","last_undegraded":"2024-05-15T14:00:21.330881+0000","last_fullsized":"2024-05-15T14:00:21.330881+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:42:56.136164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'36","reported_seq":411,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.617897+0000","last_change":"2024-05-15T13:57:14.258903+0000","last_active":"2024-05-15T14:00:21.617897+0000","last_peered":"2024-05-15T14:00:21.617897+0000","last_clean":"2024-05-15T14:00:21.617897+0000","last_became_active":"2024-05-15T13:57:14.258453+0000","last_became_peered":"2024-05-15T13:57:14.258453+0000","last_unstale":"2024-05-15T14:00:21.617897+0000","last_undegraded":"2024-05-15T14:00:21.617897+0000","last_fullsized":"2024-05-15T14:00:21.617897+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:49:33.004534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67059,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":346,"num_read_kb":238,"num_write":37,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260683+0000","last_change":"2024-05-15T13:57:17.389717+0000","last_active":"2024-05-15T14:00:21.260683+0000","last_peered":"2024-05-15T14:00:21.260683+0000","last_clean":"2024-05-15T14:00:21.260683+0000","last_became_active":"2024-05-15T13:57:17.389188+0000","last_became_peered":"2024-05-15T13:57:17.389188+0000","last_unstale":"2024-05-15T14:00:21.260683+0000","last_undegraded":"2024-05-15T14:00:21.260683+0000","last_fullsized":"2024-05-15T14:00:21.260683+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:40:26.827507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.166366+0000","last_change":"2024-05-15T13:57:14.256007+0000","last_active":"2024-05-15T14:00:21.166366+0000","last_peered":"2024-05-15T14:00:21.166366+0000","last_clean":"2024-05-15T14:00:21.166366+0000","last_became_active":"2024-05-15T13:57:14.255656+0000","last_became_peered":"2024-05-15T13:57:14.255656+0000","last_unstale":"2024-05-15T14:00:21.166366+0000","last_undegraded":"2024-05-15T14:00:21.166366+0000","last_fullsized":"2024-05-15T14:00:21.166366+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:14:46.880981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.413924+0000","last_change":"2024-05-15T13:57:17.389971+0000","last_active":"2024-05-15T14:00:21.413924+0000","last_peered":"2024-05-15T14:00:21.413924+0000","last_clean":"2024-05-15T14:00:21.413924+0000","last_became_active":"2024-05-15T13:57:17.389633+0000","last_became_peered":"2024-05-15T13:57:17.389633+0000","last_unstale":"2024-05-15T14:00:21.413924+0000","last_undegraded":"2024-05-15T14:00:21.413924+0000","last_fullsized":"2024-05-15T14:00:21.413924+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:25:10.971086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.266315+0000","last_change":"2024-05-15T13:57:14.258086+0000","last_active":"2024-05-15T14:00:21.266315+0000","last_peered":"2024-05-15T14:00:21.266315+0000","last_clean":"2024-05-15T14:00:21.266315+0000","last_became_active":"2024-05-15T13:57:14.257736+0000","last_became_peered":"2024-05-15T13:57:14.257736+0000","last_unstale":"2024-05-15T14:00:21.266315+0000","last_undegraded":"2024-05-15T14:00:21.266315+0000","last_fullsized":"2024-05-15T14:00:21.266315+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T00:51:07.431725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414677+0000","last_change":"2024-05-15T13:58:06.335760+0000","last_active":"2024-05-15T14:00:21.414677+0000","last_peered":"2024-05-15T14:00:21.414677+0000","last_clean":"2024-05-15T14:00:21.414677+0000","last_became_active":"2024-05-15T13:58:06.335236+0000","last_became_peered":"2024-05-15T13:58:06.335236+0000","last_unstale":"2024-05-15T14:00:21.414677+0000","last_undegraded":"2024-05-15T14:00:21.414677+0000","last_fullsized":"2024-05-15T14:00:21.414677+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:04:47.007886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'6","reported_seq":207,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.616627+0000","last_change":"2024-05-15T13:57:14.261911+0000","last_active":"2024-05-15T14:00:21.616627+0000","last_peered":"2024-05-15T14:00:21.616627+0000","last_clean":"2024-05-15T14:00:21.616627+0000","last_became_active":"2024-05-15T13:57:14.261506+0000","last_became_peered":"2024-05-15T13:57:14.261506+0000","last_unstale":"2024-05-15T14:00:21.616627+0000","last_undegraded":"2024-05-15T14:00:21.616627+0000","last_fullsized":"2024-05-15T14:00:21.616627+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:18:21.021651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":173,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414154+0000","last_change":"2024-05-15T13:57:17.395074+0000","last_active":"2024-05-15T14:00:21.414154+0000","last_peered":"2024-05-15T14:00:21.414154+0000","last_clean":"2024-05-15T14:00:21.414154+0000","last_became_active":"2024-05-15T13:57:17.394712+0000","last_became_peered":"2024-05-15T13:57:17.394712+0000","last_unstale":"2024-05-15T14:00:21.414154+0000","last_undegraded":"2024-05-15T14:00:21.414154+0000","last_fullsized":"2024-05-15T14:00:21.414154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:58:25.385949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":69,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260349+0000","last_change":"2024-05-15T13:58:06.086231+0000","last_active":"2024-05-15T14:00:21.260349+0000","last_peered":"2024-05-15T14:00:21.260349+0000","last_clean":"2024-05-15T14:00:21.260349+0000","last_became_active":"2024-05-15T13:58:06.074640+0000","last_became_peered":"2024-05-15T13:58:06.074640+0000","last_unstale":"2024-05-15T14:00:21.260349+0000","last_undegraded":"2024-05-15T14:00:21.260349+0000","last_fullsized":"2024-05-15T14:00:21.260349+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:41:54.747228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.266202+0000","last_change":"2024-05-15T13:57:17.376895+0000","last_active":"2024-05-15T14:00:21.266202+0000","last_peered":"2024-05-15T14:00:21.266202+0000","last_clean":"2024-05-15T14:00:21.266202+0000","last_became_active":"2024-05-15T13:57:17.376499+0000","last_became_peered":"2024-05-15T13:57:17.376499+0000","last_unstale":"2024-05-15T14:00:21.266202+0000","last_undegraded":"2024-05-15T14:00:21.266202+0000","last_fullsized":"2024-05-15T14:00:21.266202+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:39:41.707151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.581900+0000","last_change":"2024-05-15T13:58:06.090051+0000","last_active":"2024-05-15T14:00:21.581900+0000","last_peered":"2024-05-15T14:00:21.581900+0000","last_clean":"2024-05-15T14:00:21.581900+0000","last_became_active":"2024-05-15T13:58:06.074177+0000","last_became_peered":"2024-05-15T13:58:06.074177+0000","last_unstale":"2024-05-15T14:00:21.581900+0000","last_undegraded":"2024-05-15T14:00:21.581900+0000","last_fullsized":"2024-05-15T14:00:21.581900+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:07:11.090910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.581230+0000","last_change":"2024-05-15T13:57:17.390640+0000","last_active":"2024-05-15T14:00:21.581230+0000","last_peered":"2024-05-15T14:00:21.581230+0000","last_clean":"2024-05-15T14:00:21.581230+0000","last_became_active":"2024-05-15T13:57:17.390293+0000","last_became_peered":"2024-05-15T13:57:17.390293+0000","last_unstale":"2024-05-15T14:00:21.581230+0000","last_undegraded":"2024-05-15T14:00:21.581230+0000","last_fullsized":"2024-05-15T14:00:21.581230+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:08:18.907391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.162761+0000","last_change":"2024-05-15T13:57:14.535321+0000","last_active":"2024-05-15T14:00:21.162761+0000","last_peered":"2024-05-15T14:00:21.162761+0000","last_clean":"2024-05-15T14:00:21.162761+0000","last_became_active":"2024-05-15T13:57:14.534794+0000","last_became_peered":"2024-05-15T13:57:14.534794+0000","last_unstale":"2024-05-15T14:00:21.162761+0000","last_undegraded":"2024-05-15T14:00:21.162761+0000","last_fullsized":"2024-05-15T14:00:21.162761+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:27:07.087671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.270131+0000","last_change":"2024-05-15T13:57:17.381249+0000","last_active":"2024-05-15T14:00:21.270131+0000","last_peered":"2024-05-15T14:00:21.270131+0000","last_clean":"2024-05-15T14:00:21.270131+0000","last_became_active":"2024-05-15T13:57:17.380474+0000","last_became_peered":"2024-05-15T13:57:17.380474+0000","last_unstale":"2024-05-15T14:00:21.270131+0000","last_undegraded":"2024-05-15T14:00:21.270131+0000","last_fullsized":"2024-05-15T14:00:21.270131+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:49:00.341085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414724+0000","last_change":"2024-05-15T13:57:14.257635+0000","last_active":"2024-05-15T14:00:21.414724+0000","last_peered":"2024-05-15T14:00:21.414724+0000","last_clean":"2024-05-15T14:00:21.414724+0000","last_became_active":"2024-05-15T13:57:14.257156+0000","last_became_peered":"2024-05-15T13:57:14.257156+0000","last_unstale":"2024-05-15T14:00:21.414724+0000","last_undegraded":"2024-05-15T14:00:21.414724+0000","last_fullsized":"2024-05-15T14:00:21.414724+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:11:48.742053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.269906+0000","last_change":"2024-05-15T13:57:17.391067+0000","last_active":"2024-05-15T14:00:21.269906+0000","last_peered":"2024-05-15T14:00:21.269906+0000","last_clean":"2024-05-15T14:00:21.269906+0000","last_became_active":"2024-05-15T13:57:17.390763+0000","last_became_peered":"2024-05-15T13:57:17.390763+0000","last_unstale":"2024-05-15T14:00:21.269906+0000","last_undegraded":"2024-05-15T14:00:21.269906+0000","last_fullsized":"2024-05-15T14:00:21.269906+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:28:46.506307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.266550+0000","last_change":"2024-05-15T13:57:17.391816+0000","last_active":"2024-05-15T14:00:21.266550+0000","last_peered":"2024-05-15T14:00:21.266550+0000","last_clean":"2024-05-15T14:00:21.266550+0000","last_became_active":"2024-05-15T13:57:17.389374+0000","last_became_peered":"2024-05-15T13:57:17.389374+0000","last_unstale":"2024-05-15T14:00:21.266550+0000","last_undegraded":"2024-05-15T14:00:21.266550+0000","last_fullsized":"2024-05-15T14:00:21.266550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:48:31.756133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":202,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.416332+0000","last_change":"2024-05-15T13:57:14.534946+0000","last_active":"2024-05-15T14:00:21.416332+0000","last_peered":"2024-05-15T14:00:21.416332+0000","last_clean":"2024-05-15T14:00:21.416332+0000","last_became_active":"2024-05-15T13:57:14.534535+0000","last_became_peered":"2024-05-15T13:57:14.534535+0000","last_unstale":"2024-05-15T14:00:21.416332+0000","last_undegraded":"2024-05-15T14:00:21.416332+0000","last_fullsized":"2024-05-15T14:00:21.416332+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:27:20.526800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":167,"num_read_kb":167,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260866+0000","last_change":"2024-05-15T13:57:17.391637+0000","last_active":"2024-05-15T14:00:21.260866+0000","last_peered":"2024-05-15T14:00:21.260866+0000","last_clean":"2024-05-15T14:00:21.260866+0000","last_became_active":"2024-05-15T13:57:17.391271+0000","last_became_peered":"2024-05-15T13:57:17.391271+0000","last_unstale":"2024-05-15T14:00:21.260866+0000","last_undegraded":"2024-05-15T14:00:21.260866+0000","last_fullsized":"2024-05-15T14:00:21.260866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:22:06.398387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.163907+0000","last_change":"2024-05-15T13:57:14.258510+0000","last_active":"2024-05-15T14:00:21.163907+0000","last_peered":"2024-05-15T14:00:21.163907+0000","last_clean":"2024-05-15T14:00:21.163907+0000","last_became_active":"2024-05-15T13:57:14.257544+0000","last_became_peered":"2024-05-15T13:57:14.257544+0000","last_unstale":"2024-05-15T14:00:21.163907+0000","last_undegraded":"2024-05-15T14:00:21.163907+0000","last_fullsized":"2024-05-15T14:00:21.163907+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:12:27.253931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":102,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414469+0000","last_change":"2024-05-15T13:55:53.940355+0000","last_active":"2024-05-15T14:00:21.414469+0000","last_peered":"2024-05-15T14:00:21.414469+0000","last_clean":"2024-05-15T14:00:21.414469+0000","last_became_active":"2024-05-15T13:55:53.934638+0000","last_became_peered":"2024-05-15T13:55:53.934638+0000","last_unstale":"2024-05-15T14:00:21.414469+0000","last_undegraded":"2024-05-15T14:00:21.414469+0000","last_fullsized":"2024-05-15T14:00:21.414469+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:52:33.556435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:52:33.556435+0000","last_clean_scrub_stamp":"2024-05-15T13:52:33.556435+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:02:55.332569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.413924+0000","last_change":"2024-05-15T13:57:17.382394+0000","last_active":"2024-05-15T14:00:21.413924+0000","last_peered":"2024-05-15T14:00:21.413924+0000","last_clean":"2024-05-15T14:00:21.413924+0000","last_became_active":"2024-05-15T13:57:17.382090+0000","last_became_peered":"2024-05-15T13:57:17.382090+0000","last_unstale":"2024-05-15T14:00:21.413924+0000","last_undegraded":"2024-05-15T14:00:21.413924+0000","last_fullsized":"2024-05-15T14:00:21.413924+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:01:34.653216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414627+0000","last_change":"2024-05-15T13:57:14.261549+0000","last_active":"2024-05-15T14:00:21.414627+0000","last_peered":"2024-05-15T14:00:21.414627+0000","last_clean":"2024-05-15T14:00:21.414627+0000","last_became_active":"2024-05-15T13:57:14.261165+0000","last_became_peered":"2024-05-15T13:57:14.261165+0000","last_unstale":"2024-05-15T14:00:21.414627+0000","last_undegraded":"2024-05-15T14:00:21.414627+0000","last_fullsized":"2024-05-15T14:00:21.414627+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:04:39.574440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260702+0000","last_change":"2024-05-15T13:57:14.535309+0000","last_active":"2024-05-15T14:00:21.260702+0000","last_peered":"2024-05-15T14:00:21.260702+0000","last_clean":"2024-05-15T14:00:21.260702+0000","last_became_active":"2024-05-15T13:57:14.534938+0000","last_became_peered":"2024-05-15T13:57:14.534938+0000","last_unstale":"2024-05-15T14:00:21.260702+0000","last_undegraded":"2024-05-15T14:00:21.260702+0000","last_fullsized":"2024-05-15T14:00:21.260702+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:34:53.373149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.413958+0000","last_change":"2024-05-15T13:57:17.388272+0000","last_active":"2024-05-15T14:00:21.413958+0000","last_peered":"2024-05-15T14:00:21.413958+0000","last_clean":"2024-05-15T14:00:21.413958+0000","last_became_active":"2024-05-15T13:57:17.387849+0000","last_became_peered":"2024-05-15T13:57:17.387849+0000","last_unstale":"2024-05-15T14:00:21.413958+0000","last_undegraded":"2024-05-15T14:00:21.413958+0000","last_fullsized":"2024-05-15T14:00:21.413958+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:47:47.323566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'47","reported_seq":99,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.374344+0000","last_change":"2024-05-15T13:57:14.256161+0000","last_active":"2024-05-15T14:00:21.374344+0000","last_peered":"2024-05-15T14:00:21.374344+0000","last_clean":"2024-05-15T14:00:21.374344+0000","last_became_active":"2024-05-15T13:57:14.255707+0000","last_became_peered":"2024-05-15T13:57:14.255707+0000","last_unstale":"2024-05-15T14:00:21.374344+0000","last_undegraded":"2024-05-15T14:00:21.374344+0000","last_fullsized":"2024-05-15T14:00:21.374344+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:24:57.200029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":85,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.261188+0000","last_change":"2024-05-15T13:57:17.389522+0000","last_active":"2024-05-15T14:00:21.261188+0000","last_peered":"2024-05-15T14:00:21.261188+0000","last_clean":"2024-05-15T14:00:21.261188+0000","last_became_active":"2024-05-15T13:57:17.389177+0000","last_became_peered":"2024-05-15T13:57:17.389177+0000","last_unstale":"2024-05-15T14:00:21.261188+0000","last_undegraded":"2024-05-15T14:00:21.261188+0000","last_fullsized":"2024-05-15T14:00:21.261188+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:23:35.716934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.164103+0000","last_change":"2024-05-15T13:57:14.254300+0000","last_active":"2024-05-15T14:00:21.164103+0000","last_peered":"2024-05-15T14:00:21.164103+0000","last_clean":"2024-05-15T14:00:21.164103+0000","last_became_active":"2024-05-15T13:57:14.253741+0000","last_became_peered":"2024-05-15T13:57:14.253741+0000","last_unstale":"2024-05-15T14:00:21.164103+0000","last_undegraded":"2024-05-15T14:00:21.164103+0000","last_fullsized":"2024-05-15T14:00:21.164103+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:25:34.018159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.416927+0000","last_change":"2024-05-15T13:57:17.380688+0000","last_active":"2024-05-15T14:00:21.416927+0000","last_peered":"2024-05-15T14:00:21.416927+0000","last_clean":"2024-05-15T14:00:21.416927+0000","last_became_active":"2024-05-15T13:57:17.380379+0000","last_became_peered":"2024-05-15T13:57:17.380379+0000","last_unstale":"2024-05-15T14:00:21.416927+0000","last_undegraded":"2024-05-15T14:00:21.416927+0000","last_fullsized":"2024-05-15T14:00:21.416927+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:32:09.190750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.330640+0000","last_change":"2024-05-15T13:57:14.534775+0000","last_active":"2024-05-15T14:00:21.330640+0000","last_peered":"2024-05-15T14:00:21.330640+0000","last_clean":"2024-05-15T14:00:21.330640+0000","last_became_active":"2024-05-15T13:57:14.534440+0000","last_became_peered":"2024-05-15T13:57:14.534440+0000","last_unstale":"2024-05-15T14:00:21.330640+0000","last_undegraded":"2024-05-15T14:00:21.330640+0000","last_fullsized":"2024-05-15T14:00:21.330640+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:21:49.047187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260352+0000","last_change":"2024-05-15T13:57:17.396252+0000","last_active":"2024-05-15T14:00:21.260352+0000","last_peered":"2024-05-15T14:00:21.260352+0000","last_clean":"2024-05-15T14:00:21.260352+0000","last_became_active":"2024-05-15T13:57:17.395853+0000","last_became_peered":"2024-05-15T13:57:17.395853+0000","last_unstale":"2024-05-15T14:00:21.260352+0000","last_undegraded":"2024-05-15T14:00:21.260352+0000","last_fullsized":"2024-05-15T14:00:21.260352+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:12:54.924053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260844+0000","last_change":"2024-05-15T13:57:14.258411+0000","last_active":"2024-05-15T14:00:21.260844+0000","last_peered":"2024-05-15T14:00:21.260844+0000","last_clean":"2024-05-15T14:00:21.260844+0000","last_became_active":"2024-05-15T13:57:14.257847+0000","last_became_peered":"2024-05-15T13:57:14.257847+0000","last_unstale":"2024-05-15T14:00:21.260844+0000","last_undegraded":"2024-05-15T14:00:21.260844+0000","last_fullsized":"2024-05-15T14:00:21.260844+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:49:32.368351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.581383+0000","last_change":"2024-05-15T13:57:17.388916+0000","last_active":"2024-05-15T14:00:21.581383+0000","last_peered":"2024-05-15T14:00:21.581383+0000","last_clean":"2024-05-15T14:00:21.581383+0000","last_became_active":"2024-05-15T13:57:17.388603+0000","last_became_peered":"2024-05-15T13:57:17.388603+0000","last_unstale":"2024-05-15T14:00:21.581383+0000","last_undegraded":"2024-05-15T14:00:21.581383+0000","last_fullsized":"2024-05-15T14:00:21.581383+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:38:46.650393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.166198+0000","last_change":"2024-05-15T13:57:14.256801+0000","last_active":"2024-05-15T14:00:21.166198+0000","last_peered":"2024-05-15T14:00:21.166198+0000","last_clean":"2024-05-15T14:00:21.166198+0000","last_became_active":"2024-05-15T13:57:14.256476+0000","last_became_peered":"2024-05-15T13:57:14.256476+0000","last_unstale":"2024-05-15T14:00:21.166198+0000","last_undegraded":"2024-05-15T14:00:21.166198+0000","last_fullsized":"2024-05-15T14:00:21.166198+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:22:34.495756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.261123+0000","last_change":"2024-05-15T13:57:17.380927+0000","last_active":"2024-05-15T14:00:21.261123+0000","last_peered":"2024-05-15T14:00:21.261123+0000","last_clean":"2024-05-15T14:00:21.261123+0000","last_became_active":"2024-05-15T13:57:17.380602+0000","last_became_peered":"2024-05-15T13:57:17.380602+0000","last_unstale":"2024-05-15T14:00:21.261123+0000","last_undegraded":"2024-05-15T14:00:21.261123+0000","last_fullsized":"2024-05-15T14:00:21.261123+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:37:13.920968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260541+0000","last_change":"2024-05-15T13:57:14.257954+0000","last_active":"2024-05-15T14:00:21.260541+0000","last_peered":"2024-05-15T14:00:21.260541+0000","last_clean":"2024-05-15T14:00:21.260541+0000","last_became_active":"2024-05-15T13:57:14.257601+0000","last_became_peered":"2024-05-15T13:57:14.257601+0000","last_unstale":"2024-05-15T14:00:21.260541+0000","last_undegraded":"2024-05-15T14:00:21.260541+0000","last_fullsized":"2024-05-15T14:00:21.260541+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:47:14.811264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414015+0000","last_change":"2024-05-15T13:57:17.385227+0000","last_active":"2024-05-15T14:00:21.414015+0000","last_peered":"2024-05-15T14:00:21.414015+0000","last_clean":"2024-05-15T14:00:21.414015+0000","last_became_active":"2024-05-15T13:57:17.384899+0000","last_became_peered":"2024-05-15T13:57:17.384899+0000","last_unstale":"2024-05-15T14:00:21.414015+0000","last_undegraded":"2024-05-15T14:00:21.414015+0000","last_fullsized":"2024-05-15T14:00:21.414015+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:04:29.162392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.166487+0000","last_change":"2024-05-15T13:57:14.535266+0000","last_active":"2024-05-15T14:00:21.166487+0000","last_peered":"2024-05-15T14:00:21.166487+0000","last_clean":"2024-05-15T14:00:21.166487+0000","last_became_active":"2024-05-15T13:57:14.534892+0000","last_became_peered":"2024-05-15T13:57:14.534892+0000","last_unstale":"2024-05-15T14:00:21.166487+0000","last_undegraded":"2024-05-15T14:00:21.166487+0000","last_fullsized":"2024-05-15T14:00:21.166487+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:49:50.934517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414053+0000","last_change":"2024-05-15T13:57:17.390330+0000","last_active":"2024-05-15T14:00:21.414053+0000","last_peered":"2024-05-15T14:00:21.414053+0000","last_clean":"2024-05-15T14:00:21.414053+0000","last_became_active":"2024-05-15T13:57:17.390000+0000","last_became_peered":"2024-05-15T13:57:17.390000+0000","last_unstale":"2024-05-15T14:00:21.414053+0000","last_undegraded":"2024-05-15T14:00:21.414053+0000","last_fullsized":"2024-05-15T14:00:21.414053+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:52:11.361549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'39","reported_seq":76,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.371727+0000","last_change":"2024-05-15T13:57:14.258174+0000","last_active":"2024-05-15T14:00:21.371727+0000","last_peered":"2024-05-15T14:00:21.371727+0000","last_clean":"2024-05-15T14:00:21.371727+0000","last_became_active":"2024-05-15T13:57:14.257763+0000","last_became_peered":"2024-05-15T13:57:14.257763+0000","last_unstale":"2024-05-15T14:00:21.371727+0000","last_undegraded":"2024-05-15T14:00:21.371727+0000","last_fullsized":"2024-05-15T14:00:21.371727+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:34:31.573845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414053+0000","last_change":"2024-05-15T13:57:17.391964+0000","last_active":"2024-05-15T14:00:21.414053+0000","last_peered":"2024-05-15T14:00:21.414053+0000","last_clean":"2024-05-15T14:00:21.414053+0000","last_became_active":"2024-05-15T13:57:17.391517+0000","last_became_peered":"2024-05-15T13:57:17.391517+0000","last_unstale":"2024-05-15T14:00:21.414053+0000","last_undegraded":"2024-05-15T14:00:21.414053+0000","last_fullsized":"2024-05-15T14:00:21.414053+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:54:36.883563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414380+0000","last_change":"2024-05-15T13:58:06.256119+0000","last_active":"2024-05-15T14:00:21.414380+0000","last_peered":"2024-05-15T14:00:21.414380+0000","last_clean":"2024-05-15T14:00:21.414380+0000","last_became_active":"2024-05-15T13:58:06.253495+0000","last_became_peered":"2024-05-15T13:58:06.253495+0000","last_unstale":"2024-05-15T14:00:21.414380+0000","last_undegraded":"2024-05-15T14:00:21.414380+0000","last_fullsized":"2024-05-15T14:00:21.414380+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:41:06.585181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414149+0000","last_change":"2024-05-15T13:57:17.389518+0000","last_active":"2024-05-15T14:00:21.414149+0000","last_peered":"2024-05-15T14:00:21.414149+0000","last_clean":"2024-05-15T14:00:21.414149+0000","last_became_active":"2024-05-15T13:57:17.388905+0000","last_became_peered":"2024-05-15T13:57:17.388905+0000","last_unstale":"2024-05-15T14:00:21.414149+0000","last_undegraded":"2024-05-15T14:00:21.414149+0000","last_fullsized":"2024-05-15T14:00:21.414149+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:00:51.109880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.581179+0000","last_change":"2024-05-15T13:57:17.394127+0000","last_active":"2024-05-15T14:00:21.581179+0000","last_peered":"2024-05-15T14:00:21.581179+0000","last_clean":"2024-05-15T14:00:21.581179+0000","last_became_active":"2024-05-15T13:57:17.393771+0000","last_became_peered":"2024-05-15T13:57:17.393771+0000","last_unstale":"2024-05-15T14:00:21.581179+0000","last_undegraded":"2024-05-15T14:00:21.581179+0000","last_fullsized":"2024-05-15T14:00:21.581179+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:41:33.491001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.166195+0000","last_change":"2024-05-15T13:57:17.392451+0000","last_active":"2024-05-15T14:00:21.166195+0000","last_peered":"2024-05-15T14:00:21.166195+0000","last_clean":"2024-05-15T14:00:21.166195+0000","last_became_active":"2024-05-15T13:57:17.392099+0000","last_became_peered":"2024-05-15T13:57:17.392099+0000","last_unstale":"2024-05-15T14:00:21.166195+0000","last_undegraded":"2024-05-15T14:00:21.166195+0000","last_fullsized":"2024-05-15T14:00:21.166195+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:00:14.774543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.166069+0000","last_change":"2024-05-15T13:57:17.385525+0000","last_active":"2024-05-15T14:00:21.166069+0000","last_peered":"2024-05-15T14:00:21.166069+0000","last_clean":"2024-05-15T14:00:21.166069+0000","last_became_active":"2024-05-15T13:57:17.384497+0000","last_became_peered":"2024-05-15T13:57:17.384497+0000","last_unstale":"2024-05-15T14:00:21.166069+0000","last_undegraded":"2024-05-15T14:00:21.166069+0000","last_fullsized":"2024-05-15T14:00:21.166069+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:50:39.259339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.416153+0000","last_change":"2024-05-15T13:57:17.379673+0000","last_active":"2024-05-15T14:00:21.416153+0000","last_peered":"2024-05-15T14:00:21.416153+0000","last_clean":"2024-05-15T14:00:21.416153+0000","last_became_active":"2024-05-15T13:57:17.379342+0000","last_became_peered":"2024-05-15T13:57:17.379342+0000","last_unstale":"2024-05-15T14:00:21.416153+0000","last_undegraded":"2024-05-15T14:00:21.416153+0000","last_fullsized":"2024-05-15T14:00:21.416153+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:26:43.982972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414364+0000","last_change":"2024-05-15T13:58:06.335360+0000","last_active":"2024-05-15T14:00:21.414364+0000","last_peered":"2024-05-15T14:00:21.414364+0000","last_clean":"2024-05-15T14:00:21.414364+0000","last_became_active":"2024-05-15T13:58:06.334899+0000","last_became_peered":"2024-05-15T13:58:06.334899+0000","last_unstale":"2024-05-15T14:00:21.414364+0000","last_undegraded":"2024-05-15T14:00:21.414364+0000","last_fullsized":"2024-05-15T14:00:21.414364+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:34:27.081929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.581196+0000","last_change":"2024-05-15T13:57:17.390939+0000","last_active":"2024-05-15T14:00:21.581196+0000","last_peered":"2024-05-15T14:00:21.581196+0000","last_clean":"2024-05-15T14:00:21.581196+0000","last_became_active":"2024-05-15T13:57:17.390281+0000","last_became_peered":"2024-05-15T13:57:17.390281+0000","last_unstale":"2024-05-15T14:00:21.581196+0000","last_undegraded":"2024-05-15T14:00:21.581196+0000","last_fullsized":"2024-05-15T14:00:21.581196+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:02:22.077307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.166336+0000","last_change":"2024-05-15T13:57:17.392009+0000","last_active":"2024-05-15T14:00:21.166336+0000","last_peered":"2024-05-15T14:00:21.166336+0000","last_clean":"2024-05-15T14:00:21.166336+0000","last_became_active":"2024-05-15T13:57:17.391059+0000","last_became_peered":"2024-05-15T13:57:17.391059+0000","last_unstale":"2024-05-15T14:00:21.166336+0000","last_undegraded":"2024-05-15T14:00:21.166336+0000","last_fullsized":"2024-05-15T14:00:21.166336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:51:46.787790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.261169+0000","last_change":"2024-05-15T13:57:17.383872+0000","last_active":"2024-05-15T14:00:21.261169+0000","last_peered":"2024-05-15T14:00:21.261169+0000","last_clean":"2024-05-15T14:00:21.261169+0000","last_became_active":"2024-05-15T13:57:17.383526+0000","last_became_peered":"2024-05-15T13:57:17.383526+0000","last_unstale":"2024-05-15T14:00:21.261169+0000","last_undegraded":"2024-05-15T14:00:21.261169+0000","last_fullsized":"2024-05-15T14:00:21.261169+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:43:39.343557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.266300+0000","last_change":"2024-05-15T13:57:17.377530+0000","last_active":"2024-05-15T14:00:21.266300+0000","last_peered":"2024-05-15T14:00:21.266300+0000","last_clean":"2024-05-15T14:00:21.266300+0000","last_became_active":"2024-05-15T13:57:17.377237+0000","last_became_peered":"2024-05-15T13:57:17.377237+0000","last_unstale":"2024-05-15T14:00:21.266300+0000","last_undegraded":"2024-05-15T14:00:21.266300+0000","last_fullsized":"2024-05-15T14:00:21.266300+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:29:02.173018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260755+0000","last_change":"2024-05-15T13:57:17.386208+0000","last_active":"2024-05-15T14:00:21.260755+0000","last_peered":"2024-05-15T14:00:21.260755+0000","last_clean":"2024-05-15T14:00:21.260755+0000","last_became_active":"2024-05-15T13:57:17.385853+0000","last_became_peered":"2024-05-15T13:57:17.385853+0000","last_unstale":"2024-05-15T14:00:21.260755+0000","last_undegraded":"2024-05-15T14:00:21.260755+0000","last_fullsized":"2024-05-15T14:00:21.260755+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:17:06.124653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.330803+0000","last_change":"2024-05-15T13:57:17.390074+0000","last_active":"2024-05-15T14:00:21.330803+0000","last_peered":"2024-05-15T14:00:21.330803+0000","last_clean":"2024-05-15T14:00:21.330803+0000","last_became_active":"2024-05-15T13:57:17.389765+0000","last_became_peered":"2024-05-15T13:57:17.389765+0000","last_unstale":"2024-05-15T14:00:21.330803+0000","last_undegraded":"2024-05-15T14:00:21.330803+0000","last_fullsized":"2024-05-15T14:00:21.330803+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:49:51.321180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414436+0000","last_change":"2024-05-15T13:57:17.389180+0000","last_active":"2024-05-15T14:00:21.414436+0000","last_peered":"2024-05-15T14:00:21.414436+0000","last_clean":"2024-05-15T14:00:21.414436+0000","last_became_active":"2024-05-15T13:57:17.388731+0000","last_became_peered":"2024-05-15T13:57:17.388731+0000","last_unstale":"2024-05-15T14:00:21.414436+0000","last_undegraded":"2024-05-15T14:00:21.414436+0000","last_fullsized":"2024-05-15T14:00:21.414436+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:19:00.655474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.581262+0000","last_change":"2024-05-15T13:57:17.394045+0000","last_active":"2024-05-15T14:00:21.581262+0000","last_peered":"2024-05-15T14:00:21.581262+0000","last_clean":"2024-05-15T14:00:21.581262+0000","last_became_active":"2024-05-15T13:57:17.393699+0000","last_became_peered":"2024-05-15T13:57:17.393699+0000","last_unstale":"2024-05-15T14:00:21.581262+0000","last_undegraded":"2024-05-15T14:00:21.581262+0000","last_fullsized":"2024-05-15T14:00:21.581262+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:05:37.510370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597639,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1130,"num_read_kb":675,"num_write":293,"num_write_kb":1440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":134,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1792917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645363,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":913408,"data_stored":441740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841592,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37044,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708108,"statfs":{"total":95995035648,"available":95957102592,"internally_reserved":0,"allocated":1748992,"data_stored":1232781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005117,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32468,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712684,"statfs":{"total":95995035648,"available":95961788416,"internally_reserved":0,"allocated":1318912,"data_stored":835170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201346,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36624,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708528,"statfs":{"total":95995035648,"available":95957532672,"internally_reserved":0,"allocated":1314816,"data_stored":835068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397575,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28504,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716648,"statfs":{"total":95995035648,"available":95965847552,"internally_reserved":0,"allocated":1519616,"data_stored":1035567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561102,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36212,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708940,"statfs":{"total":95995035648,"available":95957954560,"internally_reserved":0,"allocated":897024,"data_stored":432387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574949,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":507413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738474,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36232,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708920,"statfs":{"total":95995035648,"available":95957934080,"internally_reserved":0,"allocated":917504,"data_stored":433658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":509491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149787,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28440,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716712,"statfs":{"total":95995035648,"available":95965913088,"internally_reserved":0,"allocated":1458176,"data_stored":966470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953559,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":442815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757330,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36964,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708188,"statfs":{"total":95995035648,"available":95957184512,"internally_reserved":0,"allocated":1667072,"data_stored":1165928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":69528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":134307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":201160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-15T14:00:26.624 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-15T14:00:26.889 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:00:27.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:27 smithi052 ceph-mon[36190]: from='client.15243 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-15T14:00:27.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:27 smithi081 ceph-mon[36434]: from='client.15243 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-15T14:00:27.643 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:27 smithi033 ceph-mon[31650]: from='client.15243 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-15T14:00:28.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:28 smithi052 ceph-mon[36190]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 90 KiB/s wr, 66 op/s 2024-05-15T14:00:28.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:28 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:28.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:28 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:28.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:28 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:28.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:28 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:28.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:28 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:28.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:28 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:28.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:28 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:00:28.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:28 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:00:28.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:28 smithi081 ceph-mon[36434]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 90 KiB/s wr, 66 op/s 2024-05-15T14:00:28.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:28 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:28.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:28 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:28.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:28 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:28.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:28 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:28.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:28 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:28.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:28 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:28.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:28 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:00:28.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:28 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:00:28.613 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:28 smithi033 ceph-mon[31650]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 90 KiB/s wr, 66 op/s 2024-05-15T14:00:28.614 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:28 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:28.614 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:28 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:28.614 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:28 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:28.614 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:28 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:28.614 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:28 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:28.614 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:28 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:00:28.615 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:28 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:00:28.615 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:28 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:00:28.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.613+0000 7f40f44b2700 1 -- 172.21.15.33:0/1857710043 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40ec06b220 msgr2=0x7f40ec06b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:28.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.613+0000 7f40f44b2700 1 --2- 172.21.15.33:0/1857710043 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40ec06b220 0x7f40ec06b6a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f40dc009990 tx=0x7f40dc009ca0 comp rx=0 tx=0).stop 2024-05-15T14:00:28.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.613+0000 7f40f44b2700 1 -- 172.21.15.33:0/1857710043 shutdown_connections 2024-05-15T14:00:28.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.613+0000 7f40f44b2700 1 --2- 172.21.15.33:0/1857710043 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40ec06bbe0 0x7f40ec1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:28.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.613+0000 7f40f44b2700 1 --2- 172.21.15.33:0/1857710043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40ec1095b0 0x7f40ec109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:28.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.613+0000 7f40f44b2700 1 --2- 172.21.15.33:0/1857710043 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40ec06b220 0x7f40ec06b6a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:28.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.613+0000 7f40f44b2700 1 -- 172.21.15.33:0/1857710043 >> 172.21.15.33:0/1857710043 conn(0x7f40ec0793f0 msgr2=0x7f40ec079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:28.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.613+0000 7f40f44b2700 1 -- 172.21.15.33:0/1857710043 shutdown_connections 2024-05-15T14:00:28.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.613+0000 7f40f44b2700 1 -- 172.21.15.33:0/1857710043 wait complete. 2024-05-15T14:00:28.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.614+0000 7f40f44b2700 1 Processor -- start 2024-05-15T14:00:28.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.614+0000 7f40f44b2700 1 -- start start 2024-05-15T14:00:28.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.615+0000 7f40f44b2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40ec06b220 0x7f40ec19f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:28.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.615+0000 7f40f44b2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40ec06bbe0 0x7f40ec19fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:28.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.615+0000 7f40f44b2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40ec1095b0 0x7f40ec199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:28.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.615+0000 7f40f44b2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec1136f0 con 0x7f40ec1095b0 2024-05-15T14:00:28.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.615+0000 7f40f44b2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec113570 con 0x7f40ec06bbe0 2024-05-15T14:00:28.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.615+0000 7f40f44b2700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec113870 con 0x7f40ec06b220 2024-05-15T14:00:28.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.615+0000 7f40f2a4f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40ec1095b0 0x7f40ec199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:28.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.615+0000 7f40f2a4f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40ec1095b0 0x7f40ec199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60064/0 (socket says 172.21.15.33:60064) 2024-05-15T14:00:28.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.615+0000 7f40f2a4f700 1 -- 172.21.15.33:0/540917205 learned_addr learned my addr 172.21.15.33:0/540917205 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:28.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.615+0000 7f40f1a4d700 1 --2- 172.21.15.33:0/540917205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40ec06bbe0 0x7f40ec19fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:28.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.615+0000 7f40f224e700 1 --2- 172.21.15.33:0/540917205 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40ec06b220 0x7f40ec19f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:28.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.615+0000 7f40f1a4d700 1 -- 172.21.15.33:0/540917205 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40ec06b220 msgr2=0x7f40ec19f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:28.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.615+0000 7f40f1a4d700 1 --2- 172.21.15.33:0/540917205 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40ec06b220 0x7f40ec19f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:28.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.615+0000 7f40f1a4d700 1 -- 172.21.15.33:0/540917205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40ec1095b0 msgr2=0x7f40ec199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:28.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.615+0000 7f40f1a4d700 1 --2- 172.21.15.33:0/540917205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40ec1095b0 0x7f40ec199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:28.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.615+0000 7f40f1a4d700 1 -- 172.21.15.33:0/540917205 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40ec19a1b0 con 0x7f40ec06bbe0 2024-05-15T14:00:28.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.615+0000 7f40f1a4d700 1 --2- 172.21.15.33:0/540917205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40ec06bbe0 0x7f40ec19fd80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f40dc0059d0 tx=0x7f40dc0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:28.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.616+0000 7f40e37fe700 1 -- 172.21.15.33:0/540917205 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc027070 con 0x7f40ec06bbe0 2024-05-15T14:00:28.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.616+0000 7f40f44b2700 1 -- 172.21.15.33:0/540917205 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40ec19a440 con 0x7f40ec06bbe0 2024-05-15T14:00:28.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.616+0000 7f40e37fe700 1 -- 172.21.15.33:0/540917205 <== mon.2 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40dc022470 con 0x7f40ec06bbe0 2024-05-15T14:00:28.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.616+0000 7f40e37fe700 1 -- 172.21.15.33:0/540917205 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc017650 con 0x7f40ec06bbe0 2024-05-15T14:00:28.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.616+0000 7f40f44b2700 1 -- 172.21.15.33:0/540917205 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40ec1a6710 con 0x7f40ec06bbe0 2024-05-15T14:00:28.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.617+0000 7f40f44b2700 1 -- 172.21.15.33:0/540917205 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40ec06a230 con 0x7f40ec06bbe0 2024-05-15T14:00:28.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.618+0000 7f40e37fe700 1 -- 172.21.15.33:0/540917205 <== mon.2 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f40dc02a070 con 0x7f40ec06bbe0 2024-05-15T14:00:28.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.619+0000 7f40e37fe700 1 --2- 172.21.15.33:0/540917205 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f40d80b2a80 0x7f40d80b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:28.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.619+0000 7f40e37fe700 1 -- 172.21.15.33:0/540917205 <== mon.2 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f40dc0e3400 con 0x7f40ec06bbe0 2024-05-15T14:00:28.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.619+0000 7f40f224e700 1 --2- 172.21.15.33:0/540917205 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f40d80b2a80 0x7f40d80b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:28.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.620+0000 7f40f224e700 1 --2- 172.21.15.33:0/540917205 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f40d80b2a80 0x7f40d80b4f40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f40e8009680 tx=0x7f40e8009260 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:28.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.623+0000 7f40e37fe700 1 -- 172.21.15.33:0/540917205 <== mon.2 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f40dc013070 con 0x7f40ec06bbe0 2024-05-15T14:00:28.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.845+0000 7f40f44b2700 1 -- 172.21.15.33:0/540917205 --> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f40ec19af30 con 0x7f40d80b2a80 2024-05-15T14:00:28.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.852+0000 7f40e37fe700 1 -- 172.21.15.33:0/540917205 <== mgr.15144 v2:172.21.15.33:6800/1796050977 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257747 (secure 0 0 0) 0x7f40ec19af30 con 0x7f40d80b2a80 2024-05-15T14:00:28.858 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T14:00:28.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.856+0000 7f40f44b2700 1 -- 172.21.15.33:0/540917205 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f40d80b2a80 msgr2=0x7f40d80b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:28.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.856+0000 7f40f44b2700 1 --2- 172.21.15.33:0/540917205 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f40d80b2a80 0x7f40d80b4f40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f40e8009680 tx=0x7f40e8009260 comp rx=0 tx=0).stop 2024-05-15T14:00:28.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.856+0000 7f40f44b2700 1 -- 172.21.15.33:0/540917205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40ec06bbe0 msgr2=0x7f40ec19fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:28.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.856+0000 7f40f44b2700 1 --2- 172.21.15.33:0/540917205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40ec06bbe0 0x7f40ec19fd80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f40dc0059d0 tx=0x7f40dc0047b0 comp rx=0 tx=0).stop 2024-05-15T14:00:28.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.856+0000 7f40f44b2700 1 -- 172.21.15.33:0/540917205 shutdown_connections 2024-05-15T14:00:28.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.856+0000 7f40f44b2700 1 --2- 172.21.15.33:0/540917205 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f40d80b2a80 0x7f40d80b4f40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:28.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.856+0000 7f40f44b2700 1 --2- 172.21.15.33:0/540917205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40ec1095b0 0x7f40ec199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:28.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.856+0000 7f40f44b2700 1 --2- 172.21.15.33:0/540917205 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40ec06b220 0x7f40ec19f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:28.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.856+0000 7f40f44b2700 1 --2- 172.21.15.33:0/540917205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40ec06bbe0 0x7f40ec19fd80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:28.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.856+0000 7f40f44b2700 1 -- 172.21.15.33:0/540917205 >> 172.21.15.33:0/540917205 conn(0x7f40ec0793f0 msgr2=0x7f40ec101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:28.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.856+0000 7f40f44b2700 1 -- 172.21.15.33:0/540917205 shutdown_connections 2024-05-15T14:00:28.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:28.856+0000 7f40f44b2700 1 -- 172.21.15.33:0/540917205 wait complete. 2024-05-15T14:00:28.862 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-05-15T14:00:29.544 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:29 smithi033 ceph-mon[31650]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-05-15T14:00:29.544 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:29 smithi033 ceph-mon[31650]: Unable to set osd_memory_target on smithi033 to 185879142: error parsing value: Value '185879142' is below minimum 939524096 2024-05-15T14:00:29.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:29 smithi033 ceph-mon[31650]: Updating smithi033:/etc/ceph/ceph.conf 2024-05-15T14:00:29.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:29 smithi033 ceph-mon[31650]: Updating smithi052:/etc/ceph/ceph.conf 2024-05-15T14:00:29.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:29 smithi033 ceph-mon[31650]: Updating smithi081:/etc/ceph/ceph.conf 2024-05-15T14:00:29.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:29 smithi033 ceph-mon[31650]: Updating smithi033:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T14:00:29.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:29 smithi033 ceph-mon[31650]: Updating smithi081:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T14:00:29.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:29 smithi033 ceph-mon[31650]: Updating smithi052:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T14:00:29.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:29 smithi033 ceph-mon[31650]: Updating smithi081:/etc/ceph/ceph.client.admin.keyring 2024-05-15T14:00:29.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:29 smithi033 ceph-mon[31650]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-05-15T14:00:29.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:29 smithi033 ceph-mon[31650]: Updating smithi052:/etc/ceph/ceph.client.admin.keyring 2024-05-15T14:00:29.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:29 smithi033 ceph-mon[31650]: Updating smithi081:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-15T14:00:29.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:29 smithi033 ceph-mon[31650]: Updating smithi033:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-15T14:00:29.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:29 smithi033 ceph-mon[31650]: Updating smithi052:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-15T14:00:29.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:29 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:29.546 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:29 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:29.546 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:29 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:29.546 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:29 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:29.546 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:29 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:29.546 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:29 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:29.546 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:29 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:29.546 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:29 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:00:29.572 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-05-15T14:00:27.177019+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1188241,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1441,"num_read_kb":951,"num_write":519,"num_write_kb":4041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2951974,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":878,"ondisk_log_size":878,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":384628,"kb_used_data":14528,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124557196,"statfs":{"total":1151940427776,"available":1151546568704,"internally_reserved":0,"allocated":14876672,"data_stored":9044076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":230,"num_read_kb":207,"num_write":101,"num_write_kb":449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.988506"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.581443+0000","last_change":"2024-05-15T13:57:17.382728+0000","last_active":"2024-05-15T14:00:21.581443+0000","last_peered":"2024-05-15T14:00:21.581443+0000","last_clean":"2024-05-15T14:00:21.581443+0000","last_became_active":"2024-05-15T13:57:17.382315+0000","last_became_peered":"2024-05-15T13:57:17.382315+0000","last_unstale":"2024-05-15T14:00:21.581443+0000","last_undegraded":"2024-05-15T14:00:21.581443+0000","last_fullsized":"2024-05-15T14:00:21.581443+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:03:53.137916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.417116+0000","last_change":"2024-05-15T13:57:17.390880+0000","last_active":"2024-05-15T14:00:21.417116+0000","last_peered":"2024-05-15T14:00:21.417116+0000","last_clean":"2024-05-15T14:00:21.417116+0000","last_became_active":"2024-05-15T13:57:17.390560+0000","last_became_peered":"2024-05-15T13:57:17.390560+0000","last_unstale":"2024-05-15T14:00:21.417116+0000","last_undegraded":"2024-05-15T14:00:21.417116+0000","last_fullsized":"2024-05-15T14:00:21.417116+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:59:33.615670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.416802+0000","last_change":"2024-05-15T13:57:17.386495+0000","last_active":"2024-05-15T14:00:21.416802+0000","last_peered":"2024-05-15T14:00:21.416802+0000","last_clean":"2024-05-15T14:00:21.416802+0000","last_became_active":"2024-05-15T13:57:17.386183+0000","last_became_peered":"2024-05-15T13:57:17.386183+0000","last_unstale":"2024-05-15T14:00:21.416802+0000","last_undegraded":"2024-05-15T14:00:21.416802+0000","last_fullsized":"2024-05-15T14:00:21.416802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:36:13.202922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.164211+0000","last_change":"2024-05-15T13:57:17.385437+0000","last_active":"2024-05-15T14:00:21.164211+0000","last_peered":"2024-05-15T14:00:21.164211+0000","last_clean":"2024-05-15T14:00:21.164211+0000","last_became_active":"2024-05-15T13:57:17.384524+0000","last_became_peered":"2024-05-15T13:57:17.384524+0000","last_unstale":"2024-05-15T14:00:21.164211+0000","last_undegraded":"2024-05-15T14:00:21.164211+0000","last_fullsized":"2024-05-15T14:00:21.164211+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:45:17.207819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260575+0000","last_change":"2024-05-15T13:57:17.387859+0000","last_active":"2024-05-15T14:00:21.260575+0000","last_peered":"2024-05-15T14:00:21.260575+0000","last_clean":"2024-05-15T14:00:21.260575+0000","last_became_active":"2024-05-15T13:57:17.387491+0000","last_became_peered":"2024-05-15T13:57:17.387491+0000","last_unstale":"2024-05-15T14:00:21.260575+0000","last_undegraded":"2024-05-15T14:00:21.260575+0000","last_fullsized":"2024-05-15T14:00:21.260575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:43:08.212926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.269925+0000","last_change":"2024-05-15T13:57:17.382289+0000","last_active":"2024-05-15T14:00:21.269925+0000","last_peered":"2024-05-15T14:00:21.269925+0000","last_clean":"2024-05-15T14:00:21.269925+0000","last_became_active":"2024-05-15T13:57:17.380494+0000","last_became_peered":"2024-05-15T13:57:17.380494+0000","last_unstale":"2024-05-15T14:00:21.269925+0000","last_undegraded":"2024-05-15T14:00:21.269925+0000","last_fullsized":"2024-05-15T14:00:21.269925+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:11:37.097034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414501+0000","last_change":"2024-05-15T13:58:06.335776+0000","last_active":"2024-05-15T14:00:21.414501+0000","last_peered":"2024-05-15T14:00:21.414501+0000","last_clean":"2024-05-15T14:00:21.414501+0000","last_became_active":"2024-05-15T13:58:06.335239+0000","last_became_peered":"2024-05-15T13:58:06.335239+0000","last_unstale":"2024-05-15T14:00:21.414501+0000","last_undegraded":"2024-05-15T14:00:21.414501+0000","last_fullsized":"2024-05-15T14:00:21.414501+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:53:54.469082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260684+0000","last_change":"2024-05-15T13:57:17.380969+0000","last_active":"2024-05-15T14:00:21.260684+0000","last_peered":"2024-05-15T14:00:21.260684+0000","last_clean":"2024-05-15T14:00:21.260684+0000","last_became_active":"2024-05-15T13:57:17.380660+0000","last_became_peered":"2024-05-15T13:57:17.380660+0000","last_unstale":"2024-05-15T14:00:21.260684+0000","last_undegraded":"2024-05-15T14:00:21.260684+0000","last_fullsized":"2024-05-15T14:00:21.260684+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:01:41.790323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.331089+0000","last_change":"2024-05-15T13:57:17.394943+0000","last_active":"2024-05-15T14:00:21.331089+0000","last_peered":"2024-05-15T14:00:21.331089+0000","last_clean":"2024-05-15T14:00:21.331089+0000","last_became_active":"2024-05-15T13:57:17.394571+0000","last_became_peered":"2024-05-15T13:57:17.394571+0000","last_unstale":"2024-05-15T14:00:21.331089+0000","last_undegraded":"2024-05-15T14:00:21.331089+0000","last_fullsized":"2024-05-15T14:00:21.331089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:39:26.063096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.270341+0000","last_change":"2024-05-15T13:57:17.386012+0000","last_active":"2024-05-15T14:00:21.270341+0000","last_peered":"2024-05-15T14:00:21.270341+0000","last_clean":"2024-05-15T14:00:21.270341+0000","last_became_active":"2024-05-15T13:57:17.385636+0000","last_became_peered":"2024-05-15T13:57:17.385636+0000","last_unstale":"2024-05-15T14:00:21.270341+0000","last_undegraded":"2024-05-15T14:00:21.270341+0000","last_fullsized":"2024-05-15T14:00:21.270341+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:12:08.465959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414078+0000","last_change":"2024-05-15T13:57:17.389162+0000","last_active":"2024-05-15T14:00:21.414078+0000","last_peered":"2024-05-15T14:00:21.414078+0000","last_clean":"2024-05-15T14:00:21.414078+0000","last_became_active":"2024-05-15T13:57:17.388503+0000","last_became_peered":"2024-05-15T13:57:17.388503+0000","last_unstale":"2024-05-15T14:00:21.414078+0000","last_undegraded":"2024-05-15T14:00:21.414078+0000","last_fullsized":"2024-05-15T14:00:21.414078+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:24:54.354445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.164093+0000","last_change":"2024-05-15T13:57:17.381689+0000","last_active":"2024-05-15T14:00:21.164093+0000","last_peered":"2024-05-15T14:00:21.164093+0000","last_clean":"2024-05-15T14:00:21.164093+0000","last_became_active":"2024-05-15T13:57:17.381376+0000","last_became_peered":"2024-05-15T13:57:17.381376+0000","last_unstale":"2024-05-15T14:00:21.164093+0000","last_undegraded":"2024-05-15T14:00:21.164093+0000","last_fullsized":"2024-05-15T14:00:21.164093+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:03:52.652934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.413932+0000","last_change":"2024-05-15T13:57:17.382214+0000","last_active":"2024-05-15T14:00:21.413932+0000","last_peered":"2024-05-15T14:00:21.413932+0000","last_clean":"2024-05-15T14:00:21.413932+0000","last_became_active":"2024-05-15T13:57:17.381851+0000","last_became_peered":"2024-05-15T13:57:17.381851+0000","last_unstale":"2024-05-15T14:00:21.413932+0000","last_undegraded":"2024-05-15T14:00:21.413932+0000","last_fullsized":"2024-05-15T14:00:21.413932+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:18:39.854918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.416134+0000","last_change":"2024-05-15T13:57:17.390985+0000","last_active":"2024-05-15T14:00:21.416134+0000","last_peered":"2024-05-15T14:00:21.416134+0000","last_clean":"2024-05-15T14:00:21.416134+0000","last_became_active":"2024-05-15T13:57:17.390633+0000","last_became_peered":"2024-05-15T13:57:17.390633+0000","last_unstale":"2024-05-15T14:00:21.416134+0000","last_undegraded":"2024-05-15T14:00:21.416134+0000","last_fullsized":"2024-05-15T14:00:21.416134+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:32:47.177507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.416566+0000","last_change":"2024-05-15T13:57:17.383943+0000","last_active":"2024-05-15T14:00:21.416566+0000","last_peered":"2024-05-15T14:00:21.416566+0000","last_clean":"2024-05-15T14:00:21.416566+0000","last_became_active":"2024-05-15T13:57:17.383622+0000","last_became_peered":"2024-05-15T13:57:17.383622+0000","last_unstale":"2024-05-15T14:00:21.416566+0000","last_undegraded":"2024-05-15T14:00:21.416566+0000","last_fullsized":"2024-05-15T14:00:21.416566+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T00:30:23.249813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.330635+0000","last_change":"2024-05-15T13:57:17.390789+0000","last_active":"2024-05-15T14:00:21.330635+0000","last_peered":"2024-05-15T14:00:21.330635+0000","last_clean":"2024-05-15T14:00:21.330635+0000","last_became_active":"2024-05-15T13:57:17.390466+0000","last_became_peered":"2024-05-15T13:57:17.390466+0000","last_unstale":"2024-05-15T14:00:21.330635+0000","last_undegraded":"2024-05-15T14:00:21.330635+0000","last_fullsized":"2024-05-15T14:00:21.330635+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:07:24.746862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.581494+0000","last_change":"2024-05-15T13:57:17.384080+0000","last_active":"2024-05-15T14:00:21.581494+0000","last_peered":"2024-05-15T14:00:21.581494+0000","last_clean":"2024-05-15T14:00:21.581494+0000","last_became_active":"2024-05-15T13:57:17.383731+0000","last_became_peered":"2024-05-15T13:57:17.383731+0000","last_unstale":"2024-05-15T14:00:21.581494+0000","last_undegraded":"2024-05-15T14:00:21.581494+0000","last_fullsized":"2024-05-15T14:00:21.581494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:36:22.874337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.162965+0000","last_change":"2024-05-15T13:57:17.382209+0000","last_active":"2024-05-15T14:00:21.162965+0000","last_peered":"2024-05-15T14:00:21.162965+0000","last_clean":"2024-05-15T14:00:21.162965+0000","last_became_active":"2024-05-15T13:57:17.381737+0000","last_became_peered":"2024-05-15T13:57:17.381737+0000","last_unstale":"2024-05-15T14:00:21.162965+0000","last_undegraded":"2024-05-15T14:00:21.162965+0000","last_fullsized":"2024-05-15T14:00:21.162965+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:38:53.913745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.166353+0000","last_change":"2024-05-15T13:58:06.335718+0000","last_active":"2024-05-15T14:00:21.166353+0000","last_peered":"2024-05-15T14:00:21.166353+0000","last_clean":"2024-05-15T14:00:21.166353+0000","last_became_active":"2024-05-15T13:58:06.335383+0000","last_became_peered":"2024-05-15T13:58:06.335383+0000","last_unstale":"2024-05-15T14:00:21.166353+0000","last_undegraded":"2024-05-15T14:00:21.166353+0000","last_fullsized":"2024-05-15T14:00:21.166353+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T00:19:47.658973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.162928+0000","last_change":"2024-05-15T13:57:14.256057+0000","last_active":"2024-05-15T14:00:21.162928+0000","last_peered":"2024-05-15T14:00:21.162928+0000","last_clean":"2024-05-15T14:00:21.162928+0000","last_became_active":"2024-05-15T13:57:14.255413+0000","last_became_peered":"2024-05-15T13:57:14.255413+0000","last_unstale":"2024-05-15T14:00:21.162928+0000","last_undegraded":"2024-05-15T14:00:21.162928+0000","last_fullsized":"2024-05-15T14:00:21.162928+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:13:43.433634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.261168+0000","last_change":"2024-05-15T13:57:17.390548+0000","last_active":"2024-05-15T14:00:21.261168+0000","last_peered":"2024-05-15T14:00:21.261168+0000","last_clean":"2024-05-15T14:00:21.261168+0000","last_became_active":"2024-05-15T13:57:17.390144+0000","last_became_peered":"2024-05-15T13:57:17.390144+0000","last_unstale":"2024-05-15T14:00:21.261168+0000","last_undegraded":"2024-05-15T14:00:21.261168+0000","last_fullsized":"2024-05-15T14:00:21.261168+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:28:54.889994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.261392+0000","last_change":"2024-05-15T13:57:14.535704+0000","last_active":"2024-05-15T14:00:21.261392+0000","last_peered":"2024-05-15T14:00:21.261392+0000","last_clean":"2024-05-15T14:00:21.261392+0000","last_became_active":"2024-05-15T13:57:14.535356+0000","last_became_peered":"2024-05-15T13:57:14.535356+0000","last_unstale":"2024-05-15T14:00:21.261392+0000","last_undegraded":"2024-05-15T14:00:21.261392+0000","last_fullsized":"2024-05-15T14:00:21.261392+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:16:21.545507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.417396+0000","last_change":"2024-05-15T13:57:17.383482+0000","last_active":"2024-05-15T14:00:21.417396+0000","last_peered":"2024-05-15T14:00:21.417396+0000","last_clean":"2024-05-15T14:00:21.417396+0000","last_became_active":"2024-05-15T13:57:17.382973+0000","last_became_peered":"2024-05-15T13:57:17.382973+0000","last_unstale":"2024-05-15T14:00:21.417396+0000","last_undegraded":"2024-05-15T14:00:21.417396+0000","last_fullsized":"2024-05-15T14:00:21.417396+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:31:08.937166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.581765+0000","last_change":"2024-05-15T13:58:06.078918+0000","last_active":"2024-05-15T14:00:21.581765+0000","last_peered":"2024-05-15T14:00:21.581765+0000","last_clean":"2024-05-15T14:00:21.581765+0000","last_became_active":"2024-05-15T13:58:06.073173+0000","last_became_peered":"2024-05-15T13:58:06.073173+0000","last_unstale":"2024-05-15T14:00:21.581765+0000","last_undegraded":"2024-05-15T14:00:21.581765+0000","last_fullsized":"2024-05-15T14:00:21.581765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:21:10.582131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.330748+0000","last_change":"2024-05-15T13:57:17.388720+0000","last_active":"2024-05-15T14:00:21.330748+0000","last_peered":"2024-05-15T14:00:21.330748+0000","last_clean":"2024-05-15T14:00:21.330748+0000","last_became_active":"2024-05-15T13:57:17.388386+0000","last_became_peered":"2024-05-15T13:57:17.388386+0000","last_unstale":"2024-05-15T14:00:21.330748+0000","last_undegraded":"2024-05-15T14:00:21.330748+0000","last_fullsized":"2024-05-15T14:00:21.330748+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:02:40.543701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260605+0000","last_change":"2024-05-15T13:57:14.259064+0000","last_active":"2024-05-15T14:00:21.260605+0000","last_peered":"2024-05-15T14:00:21.260605+0000","last_clean":"2024-05-15T14:00:21.260605+0000","last_became_active":"2024-05-15T13:57:14.258683+0000","last_became_peered":"2024-05-15T13:57:14.258683+0000","last_unstale":"2024-05-15T14:00:21.260605+0000","last_undegraded":"2024-05-15T14:00:21.260605+0000","last_fullsized":"2024-05-15T14:00:21.260605+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:12:43.108066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.266572+0000","last_change":"2024-05-15T13:57:17.380929+0000","last_active":"2024-05-15T14:00:21.266572+0000","last_peered":"2024-05-15T14:00:21.266572+0000","last_clean":"2024-05-15T14:00:21.266572+0000","last_became_active":"2024-05-15T13:57:17.380599+0000","last_became_peered":"2024-05-15T13:57:17.380599+0000","last_unstale":"2024-05-15T14:00:21.266572+0000","last_undegraded":"2024-05-15T14:00:21.266572+0000","last_fullsized":"2024-05-15T14:00:21.266572+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:19:55.368088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'9","reported_seq":211,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.616417+0000","last_change":"2024-05-15T13:57:14.260410+0000","last_active":"2024-05-15T14:00:21.616417+0000","last_peered":"2024-05-15T14:00:21.616417+0000","last_clean":"2024-05-15T14:00:21.616417+0000","last_became_active":"2024-05-15T13:57:14.260076+0000","last_became_peered":"2024-05-15T13:57:14.260076+0000","last_unstale":"2024-05-15T14:00:21.616417+0000","last_undegraded":"2024-05-15T14:00:21.616417+0000","last_fullsized":"2024-05-15T14:00:21.616417+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T00:27:03.436237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1853,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":173,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260404+0000","last_change":"2024-05-15T13:57:17.394475+0000","last_active":"2024-05-15T14:00:21.260404+0000","last_peered":"2024-05-15T14:00:21.260404+0000","last_clean":"2024-05-15T14:00:21.260404+0000","last_became_active":"2024-05-15T13:57:17.393733+0000","last_became_peered":"2024-05-15T13:57:17.393733+0000","last_unstale":"2024-05-15T14:00:21.260404+0000","last_undegraded":"2024-05-15T14:00:21.260404+0000","last_fullsized":"2024-05-15T14:00:21.260404+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:55:06.198633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.269838+0000","last_change":"2024-05-15T13:57:14.256420+0000","last_active":"2024-05-15T14:00:21.269838+0000","last_peered":"2024-05-15T14:00:21.269838+0000","last_clean":"2024-05-15T14:00:21.269838+0000","last_became_active":"2024-05-15T13:57:14.255722+0000","last_became_peered":"2024-05-15T13:57:14.255722+0000","last_unstale":"2024-05-15T14:00:21.269838+0000","last_undegraded":"2024-05-15T14:00:21.269838+0000","last_fullsized":"2024-05-15T14:00:21.269838+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:21:30.499632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.266546+0000","last_change":"2024-05-15T13:58:06.074623+0000","last_active":"2024-05-15T14:00:21.266546+0000","last_peered":"2024-05-15T14:00:21.266546+0000","last_clean":"2024-05-15T14:00:21.266546+0000","last_became_active":"2024-05-15T13:58:06.074153+0000","last_became_peered":"2024-05-15T13:58:06.074153+0000","last_unstale":"2024-05-15T14:00:21.266546+0000","last_undegraded":"2024-05-15T14:00:21.266546+0000","last_fullsized":"2024-05-15T14:00:21.266546+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:08:22.922239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414371+0000","last_change":"2024-05-15T13:57:14.256192+0000","last_active":"2024-05-15T14:00:21.414371+0000","last_peered":"2024-05-15T14:00:21.414371+0000","last_clean":"2024-05-15T14:00:21.414371+0000","last_became_active":"2024-05-15T13:57:14.255839+0000","last_became_peered":"2024-05-15T13:57:14.255839+0000","last_unstale":"2024-05-15T14:00:21.414371+0000","last_undegraded":"2024-05-15T14:00:21.414371+0000","last_fullsized":"2024-05-15T14:00:21.414371+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:13:26.071403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260676+0000","last_change":"2024-05-15T13:57:17.386246+0000","last_active":"2024-05-15T14:00:21.260676+0000","last_peered":"2024-05-15T14:00:21.260676+0000","last_clean":"2024-05-15T14:00:21.260676+0000","last_became_active":"2024-05-15T13:57:17.385828+0000","last_became_peered":"2024-05-15T13:57:17.385828+0000","last_unstale":"2024-05-15T14:00:21.260676+0000","last_undegraded":"2024-05-15T14:00:21.260676+0000","last_fullsized":"2024-05-15T14:00:21.260676+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T00:24:38.656469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.266243+0000","last_change":"2024-05-15T13:57:14.259953+0000","last_active":"2024-05-15T14:00:21.266243+0000","last_peered":"2024-05-15T14:00:21.266243+0000","last_clean":"2024-05-15T14:00:21.266243+0000","last_became_active":"2024-05-15T13:57:14.259632+0000","last_became_peered":"2024-05-15T13:57:14.259632+0000","last_unstale":"2024-05-15T14:00:21.266243+0000","last_undegraded":"2024-05-15T14:00:21.266243+0000","last_fullsized":"2024-05-15T14:00:21.266243+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:30:28.361903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.162534+0000","last_change":"2024-05-15T13:57:17.382211+0000","last_active":"2024-05-15T14:00:21.162534+0000","last_peered":"2024-05-15T14:00:21.162534+0000","last_clean":"2024-05-15T14:00:21.162534+0000","last_became_active":"2024-05-15T13:57:17.381904+0000","last_became_peered":"2024-05-15T13:57:17.381904+0000","last_unstale":"2024-05-15T14:00:21.162534+0000","last_undegraded":"2024-05-15T14:00:21.162534+0000","last_fullsized":"2024-05-15T14:00:21.162534+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:53:49.250866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'5","reported_seq":201,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.270340+0000","last_change":"2024-05-15T13:57:14.254691+0000","last_active":"2024-05-15T14:00:21.270340+0000","last_peered":"2024-05-15T14:00:21.270340+0000","last_clean":"2024-05-15T14:00:21.270340+0000","last_became_active":"2024-05-15T13:57:14.254233+0000","last_became_peered":"2024-05-15T13:57:14.254233+0000","last_unstale":"2024-05-15T14:00:21.270340+0000","last_undegraded":"2024-05-15T14:00:21.270340+0000","last_fullsized":"2024-05-15T14:00:21.270340+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:41:31.719402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":167,"num_read_kb":167,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.331127+0000","last_change":"2024-05-15T13:57:17.382251+0000","last_active":"2024-05-15T14:00:21.331127+0000","last_peered":"2024-05-15T14:00:21.331127+0000","last_clean":"2024-05-15T14:00:21.331127+0000","last_became_active":"2024-05-15T13:57:17.381158+0000","last_became_peered":"2024-05-15T13:57:17.381158+0000","last_unstale":"2024-05-15T14:00:21.331127+0000","last_undegraded":"2024-05-15T14:00:21.331127+0000","last_fullsized":"2024-05-15T14:00:21.331127+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:24:55.971079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414941+0000","last_change":"2024-05-15T13:57:14.258350+0000","last_active":"2024-05-15T14:00:21.414941+0000","last_peered":"2024-05-15T14:00:21.414941+0000","last_clean":"2024-05-15T14:00:21.414941+0000","last_became_active":"2024-05-15T13:57:14.257982+0000","last_became_peered":"2024-05-15T13:57:14.257982+0000","last_unstale":"2024-05-15T14:00:21.414941+0000","last_undegraded":"2024-05-15T14:00:21.414941+0000","last_fullsized":"2024-05-15T14:00:21.414941+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:50:07.053401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.417513+0000","last_change":"2024-05-15T13:57:17.392114+0000","last_active":"2024-05-15T14:00:21.417513+0000","last_peered":"2024-05-15T14:00:21.417513+0000","last_clean":"2024-05-15T14:00:21.417513+0000","last_became_active":"2024-05-15T13:57:17.391732+0000","last_became_peered":"2024-05-15T13:57:17.391732+0000","last_unstale":"2024-05-15T14:00:21.417513+0000","last_undegraded":"2024-05-15T14:00:21.417513+0000","last_fullsized":"2024-05-15T14:00:21.417513+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:28:02.304910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.266603+0000","last_change":"2024-05-15T13:57:14.535381+0000","last_active":"2024-05-15T14:00:21.266603+0000","last_peered":"2024-05-15T14:00:21.266603+0000","last_clean":"2024-05-15T14:00:21.266603+0000","last_became_active":"2024-05-15T13:57:14.534931+0000","last_became_peered":"2024-05-15T13:57:14.534931+0000","last_unstale":"2024-05-15T14:00:21.266603+0000","last_undegraded":"2024-05-15T14:00:21.266603+0000","last_fullsized":"2024-05-15T14:00:21.266603+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:13:04.785139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.330881+0000","last_change":"2024-05-15T13:57:17.386103+0000","last_active":"2024-05-15T14:00:21.330881+0000","last_peered":"2024-05-15T14:00:21.330881+0000","last_clean":"2024-05-15T14:00:21.330881+0000","last_became_active":"2024-05-15T13:57:17.385738+0000","last_became_peered":"2024-05-15T13:57:17.385738+0000","last_unstale":"2024-05-15T14:00:21.330881+0000","last_undegraded":"2024-05-15T14:00:21.330881+0000","last_fullsized":"2024-05-15T14:00:21.330881+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:42:56.136164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'36","reported_seq":411,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.617897+0000","last_change":"2024-05-15T13:57:14.258903+0000","last_active":"2024-05-15T14:00:21.617897+0000","last_peered":"2024-05-15T14:00:21.617897+0000","last_clean":"2024-05-15T14:00:21.617897+0000","last_became_active":"2024-05-15T13:57:14.258453+0000","last_became_peered":"2024-05-15T13:57:14.258453+0000","last_unstale":"2024-05-15T14:00:21.617897+0000","last_undegraded":"2024-05-15T14:00:21.617897+0000","last_fullsized":"2024-05-15T14:00:21.617897+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:49:33.004534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67059,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":346,"num_read_kb":238,"num_write":37,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260683+0000","last_change":"2024-05-15T13:57:17.389717+0000","last_active":"2024-05-15T14:00:21.260683+0000","last_peered":"2024-05-15T14:00:21.260683+0000","last_clean":"2024-05-15T14:00:21.260683+0000","last_became_active":"2024-05-15T13:57:17.389188+0000","last_became_peered":"2024-05-15T13:57:17.389188+0000","last_unstale":"2024-05-15T14:00:21.260683+0000","last_undegraded":"2024-05-15T14:00:21.260683+0000","last_fullsized":"2024-05-15T14:00:21.260683+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:40:26.827507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.166366+0000","last_change":"2024-05-15T13:57:14.256007+0000","last_active":"2024-05-15T14:00:21.166366+0000","last_peered":"2024-05-15T14:00:21.166366+0000","last_clean":"2024-05-15T14:00:21.166366+0000","last_became_active":"2024-05-15T13:57:14.255656+0000","last_became_peered":"2024-05-15T13:57:14.255656+0000","last_unstale":"2024-05-15T14:00:21.166366+0000","last_undegraded":"2024-05-15T14:00:21.166366+0000","last_fullsized":"2024-05-15T14:00:21.166366+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:14:46.880981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.413924+0000","last_change":"2024-05-15T13:57:17.389971+0000","last_active":"2024-05-15T14:00:21.413924+0000","last_peered":"2024-05-15T14:00:21.413924+0000","last_clean":"2024-05-15T14:00:21.413924+0000","last_became_active":"2024-05-15T13:57:17.389633+0000","last_became_peered":"2024-05-15T13:57:17.389633+0000","last_unstale":"2024-05-15T14:00:21.413924+0000","last_undegraded":"2024-05-15T14:00:21.413924+0000","last_fullsized":"2024-05-15T14:00:21.413924+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:25:10.971086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.266315+0000","last_change":"2024-05-15T13:57:14.258086+0000","last_active":"2024-05-15T14:00:21.266315+0000","last_peered":"2024-05-15T14:00:21.266315+0000","last_clean":"2024-05-15T14:00:21.266315+0000","last_became_active":"2024-05-15T13:57:14.257736+0000","last_became_peered":"2024-05-15T13:57:14.257736+0000","last_unstale":"2024-05-15T14:00:21.266315+0000","last_undegraded":"2024-05-15T14:00:21.266315+0000","last_fullsized":"2024-05-15T14:00:21.266315+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T00:51:07.431725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414677+0000","last_change":"2024-05-15T13:58:06.335760+0000","last_active":"2024-05-15T14:00:21.414677+0000","last_peered":"2024-05-15T14:00:21.414677+0000","last_clean":"2024-05-15T14:00:21.414677+0000","last_became_active":"2024-05-15T13:58:06.335236+0000","last_became_peered":"2024-05-15T13:58:06.335236+0000","last_unstale":"2024-05-15T14:00:21.414677+0000","last_undegraded":"2024-05-15T14:00:21.414677+0000","last_fullsized":"2024-05-15T14:00:21.414677+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:04:47.007886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'6","reported_seq":207,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.616627+0000","last_change":"2024-05-15T13:57:14.261911+0000","last_active":"2024-05-15T14:00:21.616627+0000","last_peered":"2024-05-15T14:00:21.616627+0000","last_clean":"2024-05-15T14:00:21.616627+0000","last_became_active":"2024-05-15T13:57:14.261506+0000","last_became_peered":"2024-05-15T13:57:14.261506+0000","last_unstale":"2024-05-15T14:00:21.616627+0000","last_undegraded":"2024-05-15T14:00:21.616627+0000","last_fullsized":"2024-05-15T14:00:21.616627+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:18:21.021651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":173,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414154+0000","last_change":"2024-05-15T13:57:17.395074+0000","last_active":"2024-05-15T14:00:21.414154+0000","last_peered":"2024-05-15T14:00:21.414154+0000","last_clean":"2024-05-15T14:00:21.414154+0000","last_became_active":"2024-05-15T13:57:17.394712+0000","last_became_peered":"2024-05-15T13:57:17.394712+0000","last_unstale":"2024-05-15T14:00:21.414154+0000","last_undegraded":"2024-05-15T14:00:21.414154+0000","last_fullsized":"2024-05-15T14:00:21.414154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:58:25.385949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":69,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260349+0000","last_change":"2024-05-15T13:58:06.086231+0000","last_active":"2024-05-15T14:00:21.260349+0000","last_peered":"2024-05-15T14:00:21.260349+0000","last_clean":"2024-05-15T14:00:21.260349+0000","last_became_active":"2024-05-15T13:58:06.074640+0000","last_became_peered":"2024-05-15T13:58:06.074640+0000","last_unstale":"2024-05-15T14:00:21.260349+0000","last_undegraded":"2024-05-15T14:00:21.260349+0000","last_fullsized":"2024-05-15T14:00:21.260349+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:41:54.747228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.266202+0000","last_change":"2024-05-15T13:57:17.376895+0000","last_active":"2024-05-15T14:00:21.266202+0000","last_peered":"2024-05-15T14:00:21.266202+0000","last_clean":"2024-05-15T14:00:21.266202+0000","last_became_active":"2024-05-15T13:57:17.376499+0000","last_became_peered":"2024-05-15T13:57:17.376499+0000","last_unstale":"2024-05-15T14:00:21.266202+0000","last_undegraded":"2024-05-15T14:00:21.266202+0000","last_fullsized":"2024-05-15T14:00:21.266202+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:39:41.707151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.581900+0000","last_change":"2024-05-15T13:58:06.090051+0000","last_active":"2024-05-15T14:00:21.581900+0000","last_peered":"2024-05-15T14:00:21.581900+0000","last_clean":"2024-05-15T14:00:21.581900+0000","last_became_active":"2024-05-15T13:58:06.074177+0000","last_became_peered":"2024-05-15T13:58:06.074177+0000","last_unstale":"2024-05-15T14:00:21.581900+0000","last_undegraded":"2024-05-15T14:00:21.581900+0000","last_fullsized":"2024-05-15T14:00:21.581900+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:07:11.090910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.581230+0000","last_change":"2024-05-15T13:57:17.390640+0000","last_active":"2024-05-15T14:00:21.581230+0000","last_peered":"2024-05-15T14:00:21.581230+0000","last_clean":"2024-05-15T14:00:21.581230+0000","last_became_active":"2024-05-15T13:57:17.390293+0000","last_became_peered":"2024-05-15T13:57:17.390293+0000","last_unstale":"2024-05-15T14:00:21.581230+0000","last_undegraded":"2024-05-15T14:00:21.581230+0000","last_fullsized":"2024-05-15T14:00:21.581230+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:08:18.907391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.162761+0000","last_change":"2024-05-15T13:57:14.535321+0000","last_active":"2024-05-15T14:00:21.162761+0000","last_peered":"2024-05-15T14:00:21.162761+0000","last_clean":"2024-05-15T14:00:21.162761+0000","last_became_active":"2024-05-15T13:57:14.534794+0000","last_became_peered":"2024-05-15T13:57:14.534794+0000","last_unstale":"2024-05-15T14:00:21.162761+0000","last_undegraded":"2024-05-15T14:00:21.162761+0000","last_fullsized":"2024-05-15T14:00:21.162761+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:27:07.087671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.270131+0000","last_change":"2024-05-15T13:57:17.381249+0000","last_active":"2024-05-15T14:00:21.270131+0000","last_peered":"2024-05-15T14:00:21.270131+0000","last_clean":"2024-05-15T14:00:21.270131+0000","last_became_active":"2024-05-15T13:57:17.380474+0000","last_became_peered":"2024-05-15T13:57:17.380474+0000","last_unstale":"2024-05-15T14:00:21.270131+0000","last_undegraded":"2024-05-15T14:00:21.270131+0000","last_fullsized":"2024-05-15T14:00:21.270131+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:49:00.341085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414724+0000","last_change":"2024-05-15T13:57:14.257635+0000","last_active":"2024-05-15T14:00:21.414724+0000","last_peered":"2024-05-15T14:00:21.414724+0000","last_clean":"2024-05-15T14:00:21.414724+0000","last_became_active":"2024-05-15T13:57:14.257156+0000","last_became_peered":"2024-05-15T13:57:14.257156+0000","last_unstale":"2024-05-15T14:00:21.414724+0000","last_undegraded":"2024-05-15T14:00:21.414724+0000","last_fullsized":"2024-05-15T14:00:21.414724+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:11:48.742053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.269906+0000","last_change":"2024-05-15T13:57:17.391067+0000","last_active":"2024-05-15T14:00:21.269906+0000","last_peered":"2024-05-15T14:00:21.269906+0000","last_clean":"2024-05-15T14:00:21.269906+0000","last_became_active":"2024-05-15T13:57:17.390763+0000","last_became_peered":"2024-05-15T13:57:17.390763+0000","last_unstale":"2024-05-15T14:00:21.269906+0000","last_undegraded":"2024-05-15T14:00:21.269906+0000","last_fullsized":"2024-05-15T14:00:21.269906+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:28:46.506307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.266550+0000","last_change":"2024-05-15T13:57:17.391816+0000","last_active":"2024-05-15T14:00:21.266550+0000","last_peered":"2024-05-15T14:00:21.266550+0000","last_clean":"2024-05-15T14:00:21.266550+0000","last_became_active":"2024-05-15T13:57:17.389374+0000","last_became_peered":"2024-05-15T13:57:17.389374+0000","last_unstale":"2024-05-15T14:00:21.266550+0000","last_undegraded":"2024-05-15T14:00:21.266550+0000","last_fullsized":"2024-05-15T14:00:21.266550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:48:31.756133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":202,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.416332+0000","last_change":"2024-05-15T13:57:14.534946+0000","last_active":"2024-05-15T14:00:21.416332+0000","last_peered":"2024-05-15T14:00:21.416332+0000","last_clean":"2024-05-15T14:00:21.416332+0000","last_became_active":"2024-05-15T13:57:14.534535+0000","last_became_peered":"2024-05-15T13:57:14.534535+0000","last_unstale":"2024-05-15T14:00:21.416332+0000","last_undegraded":"2024-05-15T14:00:21.416332+0000","last_fullsized":"2024-05-15T14:00:21.416332+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:27:20.526800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":167,"num_read_kb":167,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260866+0000","last_change":"2024-05-15T13:57:17.391637+0000","last_active":"2024-05-15T14:00:21.260866+0000","last_peered":"2024-05-15T14:00:21.260866+0000","last_clean":"2024-05-15T14:00:21.260866+0000","last_became_active":"2024-05-15T13:57:17.391271+0000","last_became_peered":"2024-05-15T13:57:17.391271+0000","last_unstale":"2024-05-15T14:00:21.260866+0000","last_undegraded":"2024-05-15T14:00:21.260866+0000","last_fullsized":"2024-05-15T14:00:21.260866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:22:06.398387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.163907+0000","last_change":"2024-05-15T13:57:14.258510+0000","last_active":"2024-05-15T14:00:21.163907+0000","last_peered":"2024-05-15T14:00:21.163907+0000","last_clean":"2024-05-15T14:00:21.163907+0000","last_became_active":"2024-05-15T13:57:14.257544+0000","last_became_peered":"2024-05-15T13:57:14.257544+0000","last_unstale":"2024-05-15T14:00:21.163907+0000","last_undegraded":"2024-05-15T14:00:21.163907+0000","last_fullsized":"2024-05-15T14:00:21.163907+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:12:27.253931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":102,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414469+0000","last_change":"2024-05-15T13:55:53.940355+0000","last_active":"2024-05-15T14:00:21.414469+0000","last_peered":"2024-05-15T14:00:21.414469+0000","last_clean":"2024-05-15T14:00:21.414469+0000","last_became_active":"2024-05-15T13:55:53.934638+0000","last_became_peered":"2024-05-15T13:55:53.934638+0000","last_unstale":"2024-05-15T14:00:21.414469+0000","last_undegraded":"2024-05-15T14:00:21.414469+0000","last_fullsized":"2024-05-15T14:00:21.414469+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:52:33.556435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:52:33.556435+0000","last_clean_scrub_stamp":"2024-05-15T13:52:33.556435+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:02:55.332569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.413924+0000","last_change":"2024-05-15T13:57:17.382394+0000","last_active":"2024-05-15T14:00:21.413924+0000","last_peered":"2024-05-15T14:00:21.413924+0000","last_clean":"2024-05-15T14:00:21.413924+0000","last_became_active":"2024-05-15T13:57:17.382090+0000","last_became_peered":"2024-05-15T13:57:17.382090+0000","last_unstale":"2024-05-15T14:00:21.413924+0000","last_undegraded":"2024-05-15T14:00:21.413924+0000","last_fullsized":"2024-05-15T14:00:21.413924+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:01:34.653216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414627+0000","last_change":"2024-05-15T13:57:14.261549+0000","last_active":"2024-05-15T14:00:21.414627+0000","last_peered":"2024-05-15T14:00:21.414627+0000","last_clean":"2024-05-15T14:00:21.414627+0000","last_became_active":"2024-05-15T13:57:14.261165+0000","last_became_peered":"2024-05-15T13:57:14.261165+0000","last_unstale":"2024-05-15T14:00:21.414627+0000","last_undegraded":"2024-05-15T14:00:21.414627+0000","last_fullsized":"2024-05-15T14:00:21.414627+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:04:39.574440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260702+0000","last_change":"2024-05-15T13:57:14.535309+0000","last_active":"2024-05-15T14:00:21.260702+0000","last_peered":"2024-05-15T14:00:21.260702+0000","last_clean":"2024-05-15T14:00:21.260702+0000","last_became_active":"2024-05-15T13:57:14.534938+0000","last_became_peered":"2024-05-15T13:57:14.534938+0000","last_unstale":"2024-05-15T14:00:21.260702+0000","last_undegraded":"2024-05-15T14:00:21.260702+0000","last_fullsized":"2024-05-15T14:00:21.260702+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:34:53.373149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.413958+0000","last_change":"2024-05-15T13:57:17.388272+0000","last_active":"2024-05-15T14:00:21.413958+0000","last_peered":"2024-05-15T14:00:21.413958+0000","last_clean":"2024-05-15T14:00:21.413958+0000","last_became_active":"2024-05-15T13:57:17.387849+0000","last_became_peered":"2024-05-15T13:57:17.387849+0000","last_unstale":"2024-05-15T14:00:21.413958+0000","last_undegraded":"2024-05-15T14:00:21.413958+0000","last_fullsized":"2024-05-15T14:00:21.413958+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:47:47.323566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'101","reported_seq":222,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:23.422895+0000","last_change":"2024-05-15T13:57:14.256161+0000","last_active":"2024-05-15T14:00:23.422895+0000","last_peered":"2024-05-15T14:00:23.422895+0000","last_clean":"2024-05-15T14:00:23.422895+0000","last_became_active":"2024-05-15T13:57:14.255707+0000","last_became_peered":"2024-05-15T13:57:14.255707+0000","last_unstale":"2024-05-15T14:00:23.422895+0000","last_undegraded":"2024-05-15T14:00:23.422895+0000","last_fullsized":"2024-05-15T14:00:23.422895+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:24:57.200029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":186,"num_write_kb":1287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.261188+0000","last_change":"2024-05-15T13:57:17.389522+0000","last_active":"2024-05-15T14:00:21.261188+0000","last_peered":"2024-05-15T14:00:21.261188+0000","last_clean":"2024-05-15T14:00:21.261188+0000","last_became_active":"2024-05-15T13:57:17.389177+0000","last_became_peered":"2024-05-15T13:57:17.389177+0000","last_unstale":"2024-05-15T14:00:21.261188+0000","last_undegraded":"2024-05-15T14:00:21.261188+0000","last_fullsized":"2024-05-15T14:00:21.261188+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:23:35.716934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.164103+0000","last_change":"2024-05-15T13:57:14.254300+0000","last_active":"2024-05-15T14:00:21.164103+0000","last_peered":"2024-05-15T14:00:21.164103+0000","last_clean":"2024-05-15T14:00:21.164103+0000","last_became_active":"2024-05-15T13:57:14.253741+0000","last_became_peered":"2024-05-15T13:57:14.253741+0000","last_unstale":"2024-05-15T14:00:21.164103+0000","last_undegraded":"2024-05-15T14:00:21.164103+0000","last_fullsized":"2024-05-15T14:00:21.164103+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:25:34.018159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.416927+0000","last_change":"2024-05-15T13:57:17.380688+0000","last_active":"2024-05-15T14:00:21.416927+0000","last_peered":"2024-05-15T14:00:21.416927+0000","last_clean":"2024-05-15T14:00:21.416927+0000","last_became_active":"2024-05-15T13:57:17.380379+0000","last_became_peered":"2024-05-15T13:57:17.380379+0000","last_unstale":"2024-05-15T14:00:21.416927+0000","last_undegraded":"2024-05-15T14:00:21.416927+0000","last_fullsized":"2024-05-15T14:00:21.416927+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:32:09.190750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.330640+0000","last_change":"2024-05-15T13:57:14.534775+0000","last_active":"2024-05-15T14:00:21.330640+0000","last_peered":"2024-05-15T14:00:21.330640+0000","last_clean":"2024-05-15T14:00:21.330640+0000","last_became_active":"2024-05-15T13:57:14.534440+0000","last_became_peered":"2024-05-15T13:57:14.534440+0000","last_unstale":"2024-05-15T14:00:21.330640+0000","last_undegraded":"2024-05-15T14:00:21.330640+0000","last_fullsized":"2024-05-15T14:00:21.330640+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:21:49.047187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260352+0000","last_change":"2024-05-15T13:57:17.396252+0000","last_active":"2024-05-15T14:00:21.260352+0000","last_peered":"2024-05-15T14:00:21.260352+0000","last_clean":"2024-05-15T14:00:21.260352+0000","last_became_active":"2024-05-15T13:57:17.395853+0000","last_became_peered":"2024-05-15T13:57:17.395853+0000","last_unstale":"2024-05-15T14:00:21.260352+0000","last_undegraded":"2024-05-15T14:00:21.260352+0000","last_fullsized":"2024-05-15T14:00:21.260352+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:12:54.924053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260844+0000","last_change":"2024-05-15T13:57:14.258411+0000","last_active":"2024-05-15T14:00:21.260844+0000","last_peered":"2024-05-15T14:00:21.260844+0000","last_clean":"2024-05-15T14:00:21.260844+0000","last_became_active":"2024-05-15T13:57:14.257847+0000","last_became_peered":"2024-05-15T13:57:14.257847+0000","last_unstale":"2024-05-15T14:00:21.260844+0000","last_undegraded":"2024-05-15T14:00:21.260844+0000","last_fullsized":"2024-05-15T14:00:21.260844+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:49:32.368351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.581383+0000","last_change":"2024-05-15T13:57:17.388916+0000","last_active":"2024-05-15T14:00:21.581383+0000","last_peered":"2024-05-15T14:00:21.581383+0000","last_clean":"2024-05-15T14:00:21.581383+0000","last_became_active":"2024-05-15T13:57:17.388603+0000","last_became_peered":"2024-05-15T13:57:17.388603+0000","last_unstale":"2024-05-15T14:00:21.581383+0000","last_undegraded":"2024-05-15T14:00:21.581383+0000","last_fullsized":"2024-05-15T14:00:21.581383+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:38:46.650393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.166198+0000","last_change":"2024-05-15T13:57:14.256801+0000","last_active":"2024-05-15T14:00:21.166198+0000","last_peered":"2024-05-15T14:00:21.166198+0000","last_clean":"2024-05-15T14:00:21.166198+0000","last_became_active":"2024-05-15T13:57:14.256476+0000","last_became_peered":"2024-05-15T13:57:14.256476+0000","last_unstale":"2024-05-15T14:00:21.166198+0000","last_undegraded":"2024-05-15T14:00:21.166198+0000","last_fullsized":"2024-05-15T14:00:21.166198+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:22:34.495756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.261123+0000","last_change":"2024-05-15T13:57:17.380927+0000","last_active":"2024-05-15T14:00:21.261123+0000","last_peered":"2024-05-15T14:00:21.261123+0000","last_clean":"2024-05-15T14:00:21.261123+0000","last_became_active":"2024-05-15T13:57:17.380602+0000","last_became_peered":"2024-05-15T13:57:17.380602+0000","last_unstale":"2024-05-15T14:00:21.261123+0000","last_undegraded":"2024-05-15T14:00:21.261123+0000","last_fullsized":"2024-05-15T14:00:21.261123+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:37:13.920968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260541+0000","last_change":"2024-05-15T13:57:14.257954+0000","last_active":"2024-05-15T14:00:21.260541+0000","last_peered":"2024-05-15T14:00:21.260541+0000","last_clean":"2024-05-15T14:00:21.260541+0000","last_became_active":"2024-05-15T13:57:14.257601+0000","last_became_peered":"2024-05-15T13:57:14.257601+0000","last_unstale":"2024-05-15T14:00:21.260541+0000","last_undegraded":"2024-05-15T14:00:21.260541+0000","last_fullsized":"2024-05-15T14:00:21.260541+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:47:14.811264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414015+0000","last_change":"2024-05-15T13:57:17.385227+0000","last_active":"2024-05-15T14:00:21.414015+0000","last_peered":"2024-05-15T14:00:21.414015+0000","last_clean":"2024-05-15T14:00:21.414015+0000","last_became_active":"2024-05-15T13:57:17.384899+0000","last_became_peered":"2024-05-15T13:57:17.384899+0000","last_unstale":"2024-05-15T14:00:21.414015+0000","last_undegraded":"2024-05-15T14:00:21.414015+0000","last_fullsized":"2024-05-15T14:00:21.414015+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:04:29.162392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.166487+0000","last_change":"2024-05-15T13:57:14.535266+0000","last_active":"2024-05-15T14:00:21.166487+0000","last_peered":"2024-05-15T14:00:21.166487+0000","last_clean":"2024-05-15T14:00:21.166487+0000","last_became_active":"2024-05-15T13:57:14.534892+0000","last_became_peered":"2024-05-15T13:57:14.534892+0000","last_unstale":"2024-05-15T14:00:21.166487+0000","last_undegraded":"2024-05-15T14:00:21.166487+0000","last_fullsized":"2024-05-15T14:00:21.166487+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:49:50.934517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414053+0000","last_change":"2024-05-15T13:57:17.390330+0000","last_active":"2024-05-15T14:00:21.414053+0000","last_peered":"2024-05-15T14:00:21.414053+0000","last_clean":"2024-05-15T14:00:21.414053+0000","last_became_active":"2024-05-15T13:57:17.390000+0000","last_became_peered":"2024-05-15T13:57:17.390000+0000","last_unstale":"2024-05-15T14:00:21.414053+0000","last_undegraded":"2024-05-15T14:00:21.414053+0000","last_fullsized":"2024-05-15T14:00:21.414053+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:52:11.361549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'39","reported_seq":76,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.371727+0000","last_change":"2024-05-15T13:57:14.258174+0000","last_active":"2024-05-15T14:00:21.371727+0000","last_peered":"2024-05-15T14:00:21.371727+0000","last_clean":"2024-05-15T14:00:21.371727+0000","last_became_active":"2024-05-15T13:57:14.257763+0000","last_became_peered":"2024-05-15T13:57:14.257763+0000","last_unstale":"2024-05-15T14:00:21.371727+0000","last_undegraded":"2024-05-15T14:00:21.371727+0000","last_fullsized":"2024-05-15T14:00:21.371727+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:34:31.573845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414053+0000","last_change":"2024-05-15T13:57:17.391964+0000","last_active":"2024-05-15T14:00:21.414053+0000","last_peered":"2024-05-15T14:00:21.414053+0000","last_clean":"2024-05-15T14:00:21.414053+0000","last_became_active":"2024-05-15T13:57:17.391517+0000","last_became_peered":"2024-05-15T13:57:17.391517+0000","last_unstale":"2024-05-15T14:00:21.414053+0000","last_undegraded":"2024-05-15T14:00:21.414053+0000","last_fullsized":"2024-05-15T14:00:21.414053+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:54:36.883563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414380+0000","last_change":"2024-05-15T13:58:06.256119+0000","last_active":"2024-05-15T14:00:21.414380+0000","last_peered":"2024-05-15T14:00:21.414380+0000","last_clean":"2024-05-15T14:00:21.414380+0000","last_became_active":"2024-05-15T13:58:06.253495+0000","last_became_peered":"2024-05-15T13:58:06.253495+0000","last_unstale":"2024-05-15T14:00:21.414380+0000","last_undegraded":"2024-05-15T14:00:21.414380+0000","last_fullsized":"2024-05-15T14:00:21.414380+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:41:06.585181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414149+0000","last_change":"2024-05-15T13:57:17.389518+0000","last_active":"2024-05-15T14:00:21.414149+0000","last_peered":"2024-05-15T14:00:21.414149+0000","last_clean":"2024-05-15T14:00:21.414149+0000","last_became_active":"2024-05-15T13:57:17.388905+0000","last_became_peered":"2024-05-15T13:57:17.388905+0000","last_unstale":"2024-05-15T14:00:21.414149+0000","last_undegraded":"2024-05-15T14:00:21.414149+0000","last_fullsized":"2024-05-15T14:00:21.414149+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:00:51.109880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.581179+0000","last_change":"2024-05-15T13:57:17.394127+0000","last_active":"2024-05-15T14:00:21.581179+0000","last_peered":"2024-05-15T14:00:21.581179+0000","last_clean":"2024-05-15T14:00:21.581179+0000","last_became_active":"2024-05-15T13:57:17.393771+0000","last_became_peered":"2024-05-15T13:57:17.393771+0000","last_unstale":"2024-05-15T14:00:21.581179+0000","last_undegraded":"2024-05-15T14:00:21.581179+0000","last_fullsized":"2024-05-15T14:00:21.581179+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:41:33.491001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.166195+0000","last_change":"2024-05-15T13:57:17.392451+0000","last_active":"2024-05-15T14:00:21.166195+0000","last_peered":"2024-05-15T14:00:21.166195+0000","last_clean":"2024-05-15T14:00:21.166195+0000","last_became_active":"2024-05-15T13:57:17.392099+0000","last_became_peered":"2024-05-15T13:57:17.392099+0000","last_unstale":"2024-05-15T14:00:21.166195+0000","last_undegraded":"2024-05-15T14:00:21.166195+0000","last_fullsized":"2024-05-15T14:00:21.166195+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:00:14.774543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.166069+0000","last_change":"2024-05-15T13:57:17.385525+0000","last_active":"2024-05-15T14:00:21.166069+0000","last_peered":"2024-05-15T14:00:21.166069+0000","last_clean":"2024-05-15T14:00:21.166069+0000","last_became_active":"2024-05-15T13:57:17.384497+0000","last_became_peered":"2024-05-15T13:57:17.384497+0000","last_unstale":"2024-05-15T14:00:21.166069+0000","last_undegraded":"2024-05-15T14:00:21.166069+0000","last_fullsized":"2024-05-15T14:00:21.166069+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:50:39.259339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.416153+0000","last_change":"2024-05-15T13:57:17.379673+0000","last_active":"2024-05-15T14:00:21.416153+0000","last_peered":"2024-05-15T14:00:21.416153+0000","last_clean":"2024-05-15T14:00:21.416153+0000","last_became_active":"2024-05-15T13:57:17.379342+0000","last_became_peered":"2024-05-15T13:57:17.379342+0000","last_unstale":"2024-05-15T14:00:21.416153+0000","last_undegraded":"2024-05-15T14:00:21.416153+0000","last_fullsized":"2024-05-15T14:00:21.416153+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:26:43.982972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414364+0000","last_change":"2024-05-15T13:58:06.335360+0000","last_active":"2024-05-15T14:00:21.414364+0000","last_peered":"2024-05-15T14:00:21.414364+0000","last_clean":"2024-05-15T14:00:21.414364+0000","last_became_active":"2024-05-15T13:58:06.334899+0000","last_became_peered":"2024-05-15T13:58:06.334899+0000","last_unstale":"2024-05-15T14:00:21.414364+0000","last_undegraded":"2024-05-15T14:00:21.414364+0000","last_fullsized":"2024-05-15T14:00:21.414364+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:34:27.081929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.581196+0000","last_change":"2024-05-15T13:57:17.390939+0000","last_active":"2024-05-15T14:00:21.581196+0000","last_peered":"2024-05-15T14:00:21.581196+0000","last_clean":"2024-05-15T14:00:21.581196+0000","last_became_active":"2024-05-15T13:57:17.390281+0000","last_became_peered":"2024-05-15T13:57:17.390281+0000","last_unstale":"2024-05-15T14:00:21.581196+0000","last_undegraded":"2024-05-15T14:00:21.581196+0000","last_fullsized":"2024-05-15T14:00:21.581196+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:02:22.077307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.166336+0000","last_change":"2024-05-15T13:57:17.392009+0000","last_active":"2024-05-15T14:00:21.166336+0000","last_peered":"2024-05-15T14:00:21.166336+0000","last_clean":"2024-05-15T14:00:21.166336+0000","last_became_active":"2024-05-15T13:57:17.391059+0000","last_became_peered":"2024-05-15T13:57:17.391059+0000","last_unstale":"2024-05-15T14:00:21.166336+0000","last_undegraded":"2024-05-15T14:00:21.166336+0000","last_fullsized":"2024-05-15T14:00:21.166336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:51:46.787790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.261169+0000","last_change":"2024-05-15T13:57:17.383872+0000","last_active":"2024-05-15T14:00:21.261169+0000","last_peered":"2024-05-15T14:00:21.261169+0000","last_clean":"2024-05-15T14:00:21.261169+0000","last_became_active":"2024-05-15T13:57:17.383526+0000","last_became_peered":"2024-05-15T13:57:17.383526+0000","last_unstale":"2024-05-15T14:00:21.261169+0000","last_undegraded":"2024-05-15T14:00:21.261169+0000","last_fullsized":"2024-05-15T14:00:21.261169+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:43:39.343557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.266300+0000","last_change":"2024-05-15T13:57:17.377530+0000","last_active":"2024-05-15T14:00:21.266300+0000","last_peered":"2024-05-15T14:00:21.266300+0000","last_clean":"2024-05-15T14:00:21.266300+0000","last_became_active":"2024-05-15T13:57:17.377237+0000","last_became_peered":"2024-05-15T13:57:17.377237+0000","last_unstale":"2024-05-15T14:00:21.266300+0000","last_undegraded":"2024-05-15T14:00:21.266300+0000","last_fullsized":"2024-05-15T14:00:21.266300+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:29:02.173018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260755+0000","last_change":"2024-05-15T13:57:17.386208+0000","last_active":"2024-05-15T14:00:21.260755+0000","last_peered":"2024-05-15T14:00:21.260755+0000","last_clean":"2024-05-15T14:00:21.260755+0000","last_became_active":"2024-05-15T13:57:17.385853+0000","last_became_peered":"2024-05-15T13:57:17.385853+0000","last_unstale":"2024-05-15T14:00:21.260755+0000","last_undegraded":"2024-05-15T14:00:21.260755+0000","last_fullsized":"2024-05-15T14:00:21.260755+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:17:06.124653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.330803+0000","last_change":"2024-05-15T13:57:17.390074+0000","last_active":"2024-05-15T14:00:21.330803+0000","last_peered":"2024-05-15T14:00:21.330803+0000","last_clean":"2024-05-15T14:00:21.330803+0000","last_became_active":"2024-05-15T13:57:17.389765+0000","last_became_peered":"2024-05-15T13:57:17.389765+0000","last_unstale":"2024-05-15T14:00:21.330803+0000","last_undegraded":"2024-05-15T14:00:21.330803+0000","last_fullsized":"2024-05-15T14:00:21.330803+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:49:51.321180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414436+0000","last_change":"2024-05-15T13:57:17.389180+0000","last_active":"2024-05-15T14:00:21.414436+0000","last_peered":"2024-05-15T14:00:21.414436+0000","last_clean":"2024-05-15T14:00:21.414436+0000","last_became_active":"2024-05-15T13:57:17.388731+0000","last_became_peered":"2024-05-15T13:57:17.388731+0000","last_unstale":"2024-05-15T14:00:21.414436+0000","last_undegraded":"2024-05-15T14:00:21.414436+0000","last_fullsized":"2024-05-15T14:00:21.414436+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:19:00.655474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.581262+0000","last_change":"2024-05-15T13:57:17.394045+0000","last_active":"2024-05-15T14:00:21.581262+0000","last_peered":"2024-05-15T14:00:21.581262+0000","last_clean":"2024-05-15T14:00:21.581262+0000","last_became_active":"2024-05-15T13:57:17.393699+0000","last_became_peered":"2024-05-15T13:57:17.393699+0000","last_unstale":"2024-05-15T14:00:21.581262+0000","last_undegraded":"2024-05-15T14:00:21.581262+0000","last_fullsized":"2024-05-15T14:00:21.581262+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:05:37.510370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597639,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1360,"num_read_kb":882,"num_write":394,"num_write_kb":1889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":134,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2224128,"data_stored":1989549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":284,"ondisk_log_size":284,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645363,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":913408,"data_stored":441740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841592,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37044,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708108,"statfs":{"total":95995035648,"available":95957102592,"internally_reserved":0,"allocated":1748992,"data_stored":1232781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005118,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32464,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712688,"statfs":{"total":95995035648,"available":95961792512,"internally_reserved":0,"allocated":1318912,"data_stored":835170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201346,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36624,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708528,"statfs":{"total":95995035648,"available":95957532672,"internally_reserved":0,"allocated":1314816,"data_stored":835068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397575,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28504,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716648,"statfs":{"total":95995035648,"available":95965847552,"internally_reserved":0,"allocated":1519616,"data_stored":1035567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561102,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36212,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708940,"statfs":{"total":95995035648,"available":95957954560,"internally_reserved":0,"allocated":897024,"data_stored":432387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574949,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":507413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738475,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":933888,"data_stored":442614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":509491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149788,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28752,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716400,"statfs":{"total":95995035648,"available":95965593600,"internally_reserved":0,"allocated":1654784,"data_stored":1163102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953559,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":442815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757330,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36964,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708188,"statfs":{"total":95995035648,"available":95957184512,"internally_reserved":0,"allocated":1667072,"data_stored":1165928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":69528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":134307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":201160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-15T14:00:29.576 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-15T14:00:29.576 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-15T14:00:29.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:29 smithi052 ceph-mon[36190]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-05-15T14:00:29.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:29 smithi052 ceph-mon[36190]: Unable to set osd_memory_target on smithi033 to 185879142: error parsing value: Value '185879142' is below minimum 939524096 2024-05-15T14:00:29.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:29 smithi052 ceph-mon[36190]: Updating smithi033:/etc/ceph/ceph.conf 2024-05-15T14:00:29.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:29 smithi052 ceph-mon[36190]: Updating smithi052:/etc/ceph/ceph.conf 2024-05-15T14:00:29.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:29 smithi052 ceph-mon[36190]: Updating smithi081:/etc/ceph/ceph.conf 2024-05-15T14:00:29.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:29 smithi052 ceph-mon[36190]: Updating smithi033:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T14:00:29.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:29 smithi052 ceph-mon[36190]: Updating smithi081:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T14:00:29.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:29 smithi052 ceph-mon[36190]: Updating smithi052:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T14:00:29.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:29 smithi052 ceph-mon[36190]: Updating smithi081:/etc/ceph/ceph.client.admin.keyring 2024-05-15T14:00:29.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:29 smithi052 ceph-mon[36190]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-05-15T14:00:29.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:29 smithi052 ceph-mon[36190]: Updating smithi052:/etc/ceph/ceph.client.admin.keyring 2024-05-15T14:00:29.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:29 smithi052 ceph-mon[36190]: Updating smithi081:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-15T14:00:29.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:29 smithi052 ceph-mon[36190]: Updating smithi033:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-15T14:00:29.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:29 smithi052 ceph-mon[36190]: Updating smithi052:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-15T14:00:29.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:29 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:29.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:29 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:29.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:29 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:29.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:29 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:29.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:29 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:29.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:29 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:29.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:29 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:29.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:29 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:00:29.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:29 smithi081 ceph-mon[36434]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-05-15T14:00:29.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:29 smithi081 ceph-mon[36434]: Unable to set osd_memory_target on smithi033 to 185879142: error parsing value: Value '185879142' is below minimum 939524096 2024-05-15T14:00:29.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:29 smithi081 ceph-mon[36434]: Updating smithi033:/etc/ceph/ceph.conf 2024-05-15T14:00:29.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:29 smithi081 ceph-mon[36434]: Updating smithi052:/etc/ceph/ceph.conf 2024-05-15T14:00:29.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:29 smithi081 ceph-mon[36434]: Updating smithi081:/etc/ceph/ceph.conf 2024-05-15T14:00:29.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:29 smithi081 ceph-mon[36434]: Updating smithi033:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T14:00:29.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:29 smithi081 ceph-mon[36434]: Updating smithi081:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T14:00:29.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:29 smithi081 ceph-mon[36434]: Updating smithi052:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-15T14:00:29.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:29 smithi081 ceph-mon[36434]: Updating smithi081:/etc/ceph/ceph.client.admin.keyring 2024-05-15T14:00:29.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:29 smithi081 ceph-mon[36434]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-05-15T14:00:29.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:29 smithi081 ceph-mon[36434]: Updating smithi052:/etc/ceph/ceph.client.admin.keyring 2024-05-15T14:00:29.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:29 smithi081 ceph-mon[36434]: Updating smithi081:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-15T14:00:29.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:29 smithi081 ceph-mon[36434]: Updating smithi033:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-15T14:00:29.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:29 smithi081 ceph-mon[36434]: Updating smithi052:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-15T14:00:29.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:29 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:29.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:29 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:29.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:29 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:29.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:29 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:29.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:29 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:29.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:29 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:29.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:29 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:29.608 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:29 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:00:29.826 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:00:30.635 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:30 smithi033 ceph-mon[31650]: from='client.24932 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-15T14:00:30.636 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:30 smithi033 ceph-mon[31650]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 131 KiB/s wr, 69 op/s 2024-05-15T14:00:30.636 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:30 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:30.636 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:30 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:30.636 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:30 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:00:30.636 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:30 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:00:30.637 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:30 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:30.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:30 smithi052 ceph-mon[36190]: from='client.24932 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-15T14:00:30.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:30 smithi052 ceph-mon[36190]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 131 KiB/s wr, 69 op/s 2024-05-15T14:00:30.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:30 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:30.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:30 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:30.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:30 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:00:30.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:30 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:00:30.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:30 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:30.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:30 smithi081 ceph-mon[36434]: from='client.24932 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-15T14:00:30.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:30 smithi081 ceph-mon[36434]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 131 KiB/s wr, 69 op/s 2024-05-15T14:00:30.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:30 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:30.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:30 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:30.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:30 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:00:30.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:30 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:00:30.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:30 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:00:31.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.463+0000 7f554c661700 1 -- 172.21.15.33:0/1051590075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55441088f0 msgr2=0x7f5544110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:31.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.463+0000 7f554c661700 1 --2- 172.21.15.33:0/1051590075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55441088f0 0x7f5544110470 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f553c0099c0 tx=0x7f553c009cd0 comp rx=0 tx=0).stop 2024-05-15T14:00:31.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.463+0000 7f554c661700 1 -- 172.21.15.33:0/1051590075 shutdown_connections 2024-05-15T14:00:31.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.463+0000 7f554c661700 1 --2- 172.21.15.33:0/1051590075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55441088f0 0x7f5544110470 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:31.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.463+0000 7f554c661700 1 --2- 172.21.15.33:0/1051590075 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55441037f0 0x7f5544103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:31.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.463+0000 7f554c661700 1 --2- 172.21.15.33:0/1051590075 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55441041a0 0x7f55441081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:31.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.463+0000 7f554c661700 1 -- 172.21.15.33:0/1051590075 >> 172.21.15.33:0/1051590075 conn(0x7f554407a6a0 msgr2=0x7f554407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:31.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.463+0000 7f554c661700 1 -- 172.21.15.33:0/1051590075 shutdown_connections 2024-05-15T14:00:31.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.463+0000 7f554c661700 1 -- 172.21.15.33:0/1051590075 wait complete. 2024-05-15T14:00:31.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.464+0000 7f554c661700 1 Processor -- start 2024-05-15T14:00:31.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.464+0000 7f554c661700 1 -- start start 2024-05-15T14:00:31.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.464+0000 7f554c661700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55441037f0 0x7f5544075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:31.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.464+0000 7f554c661700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55441041a0 0x7f5544075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:31.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.464+0000 7f554c661700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55441088f0 0x7f55440762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:31.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.464+0000 7f554c661700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f55441136e0 con 0x7f55441088f0 2024-05-15T14:00:31.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.464+0000 7f554c661700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5544113560 con 0x7f55441041a0 2024-05-15T14:00:31.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.464+0000 7f554c661700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f5544113860 con 0x7f55441037f0 2024-05-15T14:00:31.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.464+0000 7f554abfe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55441088f0 0x7f55440762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:31.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.464+0000 7f554abfe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55441088f0 0x7f55440762c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60086/0 (socket says 172.21.15.33:60086) 2024-05-15T14:00:31.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.464+0000 7f554abfe700 1 -- 172.21.15.33:0/2513969333 learned_addr learned my addr 172.21.15.33:0/2513969333 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:31.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.464+0000 7f554a3fd700 1 --2- 172.21.15.33:0/2513969333 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55441037f0 0x7f5544075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:31.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.464+0000 7f5549bfc700 1 --2- 172.21.15.33:0/2513969333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55441041a0 0x7f5544075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:31.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.464+0000 7f554abfe700 1 -- 172.21.15.33:0/2513969333 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55441037f0 msgr2=0x7f5544075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:31.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.464+0000 7f554abfe700 1 --2- 172.21.15.33:0/2513969333 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55441037f0 0x7f5544075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:31.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.464+0000 7f554abfe700 1 -- 172.21.15.33:0/2513969333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55441041a0 msgr2=0x7f5544075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.464+0000 7f554abfe700 1 --2- 172.21.15.33:0/2513969333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55441041a0 0x7f5544075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.464+0000 7f554abfe700 1 -- 172.21.15.33:0/2513969333 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55441a80c0 con 0x7f55441088f0 2024-05-15T14:00:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.464+0000 7f5549bfc700 1 --2- 172.21.15.33:0/2513969333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55441041a0 0x7f5544075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:00:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.465+0000 7f554abfe700 1 --2- 172.21.15.33:0/2513969333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55441088f0 0x7f55440762c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f553c0059d0 tx=0x7f553c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.465+0000 7f55437fe700 1 -- 172.21.15.33:0/2513969333 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c026070 con 0x7f55441088f0 2024-05-15T14:00:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.465+0000 7f55437fe700 1 -- 172.21.15.33:0/2513969333 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f553c00f460 con 0x7f55441088f0 2024-05-15T14:00:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.465+0000 7f554c661700 1 -- 172.21.15.33:0/2513969333 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55441a8350 con 0x7f55441088f0 2024-05-15T14:00:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.465+0000 7f55437fe700 1 -- 172.21.15.33:0/2513969333 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c009e60 con 0x7f55441088f0 2024-05-15T14:00:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.465+0000 7f554c661700 1 -- 172.21.15.33:0/2513969333 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55441a87a0 con 0x7f55441088f0 2024-05-15T14:00:31.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.466+0000 7f554c661700 1 -- 172.21.15.33:0/2513969333 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f554406a230 con 0x7f55441088f0 2024-05-15T14:00:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.467+0000 7f55437fe700 1 -- 172.21.15.33:0/2513969333 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f553c029070 con 0x7f55441088f0 2024-05-15T14:00:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.468+0000 7f55437fe700 1 --2- 172.21.15.33:0/2513969333 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f552c0b2b00 0x7f552c0b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.468+0000 7f554a3fd700 1 --2- 172.21.15.33:0/2513969333 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f552c0b2b00 0x7f552c0b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.468+0000 7f55437fe700 1 -- 172.21.15.33:0/2513969333 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f553c0e3200 con 0x7f55441088f0 2024-05-15T14:00:31.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.468+0000 7f554a3fd700 1 --2- 172.21.15.33:0/2513969333 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f552c0b2b00 0x7f552c0b4fc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5534009550 tx=0x7f55340091b0 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:31.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.471+0000 7f55437fe700 1 -- 172.21.15.33:0/2513969333 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f553c030080 con 0x7f55441088f0 2024-05-15T14:00:31.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.679+0000 7f554c661700 1 -- 172.21.15.33:0/2513969333 --> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5544063980 con 0x7f552c0b2b00 2024-05-15T14:00:31.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.686+0000 7f55437fe700 1 -- 172.21.15.33:0/2513969333 <== mgr.15144 v2:172.21.15.33:6800/1796050977 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257758 (secure 0 0 0) 0x7f5544063980 con 0x7f552c0b2b00 2024-05-15T14:00:31.692 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T14:00:31.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.689+0000 7f554c661700 1 -- 172.21.15.33:0/2513969333 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f552c0b2b00 msgr2=0x7f552c0b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:31.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.689+0000 7f554c661700 1 --2- 172.21.15.33:0/2513969333 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f552c0b2b00 0x7f552c0b4fc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5534009550 tx=0x7f55340091b0 comp rx=0 tx=0).stop 2024-05-15T14:00:31.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.689+0000 7f554c661700 1 -- 172.21.15.33:0/2513969333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55441088f0 msgr2=0x7f55440762c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:31.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.689+0000 7f554c661700 1 --2- 172.21.15.33:0/2513969333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55441088f0 0x7f55440762c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f553c0059d0 tx=0x7f553c0045e0 comp rx=0 tx=0).stop 2024-05-15T14:00:31.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.689+0000 7f554c661700 1 -- 172.21.15.33:0/2513969333 shutdown_connections 2024-05-15T14:00:31.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.689+0000 7f554c661700 1 --2- 172.21.15.33:0/2513969333 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f552c0b2b00 0x7f552c0b4fc0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:31.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.689+0000 7f554c661700 1 --2- 172.21.15.33:0/2513969333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55441088f0 0x7f55440762c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:31.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.689+0000 7f554c661700 1 --2- 172.21.15.33:0/2513969333 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55441037f0 0x7f5544075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:31.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.689+0000 7f554c661700 1 --2- 172.21.15.33:0/2513969333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55441041a0 0x7f5544075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:31.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.689+0000 7f554c661700 1 -- 172.21.15.33:0/2513969333 >> 172.21.15.33:0/2513969333 conn(0x7f554407a6a0 msgr2=0x7f55441015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:31.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.689+0000 7f554c661700 1 -- 172.21.15.33:0/2513969333 shutdown_connections 2024-05-15T14:00:31.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:31.689+0000 7f554c661700 1 -- 172.21.15.33:0/2513969333 wait complete. 2024-05-15T14:00:31.695 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-05-15T14:00:32.264 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-05-15T14:00:31.178085+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1384873,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1557,"num_read_kb":1168,"num_write":558,"num_write_kb":4507,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2951974,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":911,"ondisk_log_size":911,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385264,"kb_used_data":14928,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124556560,"statfs":{"total":1151940427776,"available":1151545917440,"internally_reserved":0,"allocated":15286272,"data_stored":9448251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":346,"num_read_kb":424,"num_write":140,"num_write_kb":915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.989571"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.581443+0000","last_change":"2024-05-15T13:57:17.382728+0000","last_active":"2024-05-15T14:00:21.581443+0000","last_peered":"2024-05-15T14:00:21.581443+0000","last_clean":"2024-05-15T14:00:21.581443+0000","last_became_active":"2024-05-15T13:57:17.382315+0000","last_became_peered":"2024-05-15T13:57:17.382315+0000","last_unstale":"2024-05-15T14:00:21.581443+0000","last_undegraded":"2024-05-15T14:00:21.581443+0000","last_fullsized":"2024-05-15T14:00:21.581443+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:03:53.137916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.417116+0000","last_change":"2024-05-15T13:57:17.390880+0000","last_active":"2024-05-15T14:00:21.417116+0000","last_peered":"2024-05-15T14:00:21.417116+0000","last_clean":"2024-05-15T14:00:21.417116+0000","last_became_active":"2024-05-15T13:57:17.390560+0000","last_became_peered":"2024-05-15T13:57:17.390560+0000","last_unstale":"2024-05-15T14:00:21.417116+0000","last_undegraded":"2024-05-15T14:00:21.417116+0000","last_fullsized":"2024-05-15T14:00:21.417116+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:59:33.615670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.416802+0000","last_change":"2024-05-15T13:57:17.386495+0000","last_active":"2024-05-15T14:00:21.416802+0000","last_peered":"2024-05-15T14:00:21.416802+0000","last_clean":"2024-05-15T14:00:21.416802+0000","last_became_active":"2024-05-15T13:57:17.386183+0000","last_became_peered":"2024-05-15T13:57:17.386183+0000","last_unstale":"2024-05-15T14:00:21.416802+0000","last_undegraded":"2024-05-15T14:00:21.416802+0000","last_fullsized":"2024-05-15T14:00:21.416802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:36:13.202922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.164211+0000","last_change":"2024-05-15T13:57:17.385437+0000","last_active":"2024-05-15T14:00:21.164211+0000","last_peered":"2024-05-15T14:00:21.164211+0000","last_clean":"2024-05-15T14:00:21.164211+0000","last_became_active":"2024-05-15T13:57:17.384524+0000","last_became_peered":"2024-05-15T13:57:17.384524+0000","last_unstale":"2024-05-15T14:00:21.164211+0000","last_undegraded":"2024-05-15T14:00:21.164211+0000","last_fullsized":"2024-05-15T14:00:21.164211+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:45:17.207819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260575+0000","last_change":"2024-05-15T13:57:17.387859+0000","last_active":"2024-05-15T14:00:21.260575+0000","last_peered":"2024-05-15T14:00:21.260575+0000","last_clean":"2024-05-15T14:00:21.260575+0000","last_became_active":"2024-05-15T13:57:17.387491+0000","last_became_peered":"2024-05-15T13:57:17.387491+0000","last_unstale":"2024-05-15T14:00:21.260575+0000","last_undegraded":"2024-05-15T14:00:21.260575+0000","last_fullsized":"2024-05-15T14:00:21.260575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:43:08.212926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.269925+0000","last_change":"2024-05-15T13:57:17.382289+0000","last_active":"2024-05-15T14:00:21.269925+0000","last_peered":"2024-05-15T14:00:21.269925+0000","last_clean":"2024-05-15T14:00:21.269925+0000","last_became_active":"2024-05-15T13:57:17.380494+0000","last_became_peered":"2024-05-15T13:57:17.380494+0000","last_unstale":"2024-05-15T14:00:21.269925+0000","last_undegraded":"2024-05-15T14:00:21.269925+0000","last_fullsized":"2024-05-15T14:00:21.269925+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:11:37.097034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414501+0000","last_change":"2024-05-15T13:58:06.335776+0000","last_active":"2024-05-15T14:00:21.414501+0000","last_peered":"2024-05-15T14:00:21.414501+0000","last_clean":"2024-05-15T14:00:21.414501+0000","last_became_active":"2024-05-15T13:58:06.335239+0000","last_became_peered":"2024-05-15T13:58:06.335239+0000","last_unstale":"2024-05-15T14:00:21.414501+0000","last_undegraded":"2024-05-15T14:00:21.414501+0000","last_fullsized":"2024-05-15T14:00:21.414501+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:53:54.469082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260684+0000","last_change":"2024-05-15T13:57:17.380969+0000","last_active":"2024-05-15T14:00:21.260684+0000","last_peered":"2024-05-15T14:00:21.260684+0000","last_clean":"2024-05-15T14:00:21.260684+0000","last_became_active":"2024-05-15T13:57:17.380660+0000","last_became_peered":"2024-05-15T13:57:17.380660+0000","last_unstale":"2024-05-15T14:00:21.260684+0000","last_undegraded":"2024-05-15T14:00:21.260684+0000","last_fullsized":"2024-05-15T14:00:21.260684+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:01:41.790323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.331089+0000","last_change":"2024-05-15T13:57:17.394943+0000","last_active":"2024-05-15T14:00:21.331089+0000","last_peered":"2024-05-15T14:00:21.331089+0000","last_clean":"2024-05-15T14:00:21.331089+0000","last_became_active":"2024-05-15T13:57:17.394571+0000","last_became_peered":"2024-05-15T13:57:17.394571+0000","last_unstale":"2024-05-15T14:00:21.331089+0000","last_undegraded":"2024-05-15T14:00:21.331089+0000","last_fullsized":"2024-05-15T14:00:21.331089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:39:26.063096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.270341+0000","last_change":"2024-05-15T13:57:17.386012+0000","last_active":"2024-05-15T14:00:21.270341+0000","last_peered":"2024-05-15T14:00:21.270341+0000","last_clean":"2024-05-15T14:00:21.270341+0000","last_became_active":"2024-05-15T13:57:17.385636+0000","last_became_peered":"2024-05-15T13:57:17.385636+0000","last_unstale":"2024-05-15T14:00:21.270341+0000","last_undegraded":"2024-05-15T14:00:21.270341+0000","last_fullsized":"2024-05-15T14:00:21.270341+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:12:08.465959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414078+0000","last_change":"2024-05-15T13:57:17.389162+0000","last_active":"2024-05-15T14:00:21.414078+0000","last_peered":"2024-05-15T14:00:21.414078+0000","last_clean":"2024-05-15T14:00:21.414078+0000","last_became_active":"2024-05-15T13:57:17.388503+0000","last_became_peered":"2024-05-15T13:57:17.388503+0000","last_unstale":"2024-05-15T14:00:21.414078+0000","last_undegraded":"2024-05-15T14:00:21.414078+0000","last_fullsized":"2024-05-15T14:00:21.414078+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:24:54.354445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.164093+0000","last_change":"2024-05-15T13:57:17.381689+0000","last_active":"2024-05-15T14:00:21.164093+0000","last_peered":"2024-05-15T14:00:21.164093+0000","last_clean":"2024-05-15T14:00:21.164093+0000","last_became_active":"2024-05-15T13:57:17.381376+0000","last_became_peered":"2024-05-15T13:57:17.381376+0000","last_unstale":"2024-05-15T14:00:21.164093+0000","last_undegraded":"2024-05-15T14:00:21.164093+0000","last_fullsized":"2024-05-15T14:00:21.164093+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:03:52.652934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.413932+0000","last_change":"2024-05-15T13:57:17.382214+0000","last_active":"2024-05-15T14:00:21.413932+0000","last_peered":"2024-05-15T14:00:21.413932+0000","last_clean":"2024-05-15T14:00:21.413932+0000","last_became_active":"2024-05-15T13:57:17.381851+0000","last_became_peered":"2024-05-15T13:57:17.381851+0000","last_unstale":"2024-05-15T14:00:21.413932+0000","last_undegraded":"2024-05-15T14:00:21.413932+0000","last_fullsized":"2024-05-15T14:00:21.413932+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:18:39.854918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.416134+0000","last_change":"2024-05-15T13:57:17.390985+0000","last_active":"2024-05-15T14:00:21.416134+0000","last_peered":"2024-05-15T14:00:21.416134+0000","last_clean":"2024-05-15T14:00:21.416134+0000","last_became_active":"2024-05-15T13:57:17.390633+0000","last_became_peered":"2024-05-15T13:57:17.390633+0000","last_unstale":"2024-05-15T14:00:21.416134+0000","last_undegraded":"2024-05-15T14:00:21.416134+0000","last_fullsized":"2024-05-15T14:00:21.416134+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:32:47.177507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.416566+0000","last_change":"2024-05-15T13:57:17.383943+0000","last_active":"2024-05-15T14:00:21.416566+0000","last_peered":"2024-05-15T14:00:21.416566+0000","last_clean":"2024-05-15T14:00:21.416566+0000","last_became_active":"2024-05-15T13:57:17.383622+0000","last_became_peered":"2024-05-15T13:57:17.383622+0000","last_unstale":"2024-05-15T14:00:21.416566+0000","last_undegraded":"2024-05-15T14:00:21.416566+0000","last_fullsized":"2024-05-15T14:00:21.416566+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T00:30:23.249813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.330635+0000","last_change":"2024-05-15T13:57:17.390789+0000","last_active":"2024-05-15T14:00:21.330635+0000","last_peered":"2024-05-15T14:00:21.330635+0000","last_clean":"2024-05-15T14:00:21.330635+0000","last_became_active":"2024-05-15T13:57:17.390466+0000","last_became_peered":"2024-05-15T13:57:17.390466+0000","last_unstale":"2024-05-15T14:00:21.330635+0000","last_undegraded":"2024-05-15T14:00:21.330635+0000","last_fullsized":"2024-05-15T14:00:21.330635+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:07:24.746862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.581494+0000","last_change":"2024-05-15T13:57:17.384080+0000","last_active":"2024-05-15T14:00:21.581494+0000","last_peered":"2024-05-15T14:00:21.581494+0000","last_clean":"2024-05-15T14:00:21.581494+0000","last_became_active":"2024-05-15T13:57:17.383731+0000","last_became_peered":"2024-05-15T13:57:17.383731+0000","last_unstale":"2024-05-15T14:00:21.581494+0000","last_undegraded":"2024-05-15T14:00:21.581494+0000","last_fullsized":"2024-05-15T14:00:21.581494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:36:22.874337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.162965+0000","last_change":"2024-05-15T13:57:17.382209+0000","last_active":"2024-05-15T14:00:21.162965+0000","last_peered":"2024-05-15T14:00:21.162965+0000","last_clean":"2024-05-15T14:00:21.162965+0000","last_became_active":"2024-05-15T13:57:17.381737+0000","last_became_peered":"2024-05-15T13:57:17.381737+0000","last_unstale":"2024-05-15T14:00:21.162965+0000","last_undegraded":"2024-05-15T14:00:21.162965+0000","last_fullsized":"2024-05-15T14:00:21.162965+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:38:53.913745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.166353+0000","last_change":"2024-05-15T13:58:06.335718+0000","last_active":"2024-05-15T14:00:21.166353+0000","last_peered":"2024-05-15T14:00:21.166353+0000","last_clean":"2024-05-15T14:00:21.166353+0000","last_became_active":"2024-05-15T13:58:06.335383+0000","last_became_peered":"2024-05-15T13:58:06.335383+0000","last_unstale":"2024-05-15T14:00:21.166353+0000","last_undegraded":"2024-05-15T14:00:21.166353+0000","last_fullsized":"2024-05-15T14:00:21.166353+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T00:19:47.658973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.162928+0000","last_change":"2024-05-15T13:57:14.256057+0000","last_active":"2024-05-15T14:00:21.162928+0000","last_peered":"2024-05-15T14:00:21.162928+0000","last_clean":"2024-05-15T14:00:21.162928+0000","last_became_active":"2024-05-15T13:57:14.255413+0000","last_became_peered":"2024-05-15T13:57:14.255413+0000","last_unstale":"2024-05-15T14:00:21.162928+0000","last_undegraded":"2024-05-15T14:00:21.162928+0000","last_fullsized":"2024-05-15T14:00:21.162928+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:13:43.433634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.261168+0000","last_change":"2024-05-15T13:57:17.390548+0000","last_active":"2024-05-15T14:00:21.261168+0000","last_peered":"2024-05-15T14:00:21.261168+0000","last_clean":"2024-05-15T14:00:21.261168+0000","last_became_active":"2024-05-15T13:57:17.390144+0000","last_became_peered":"2024-05-15T13:57:17.390144+0000","last_unstale":"2024-05-15T14:00:21.261168+0000","last_undegraded":"2024-05-15T14:00:21.261168+0000","last_fullsized":"2024-05-15T14:00:21.261168+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:28:54.889994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.261392+0000","last_change":"2024-05-15T13:57:14.535704+0000","last_active":"2024-05-15T14:00:21.261392+0000","last_peered":"2024-05-15T14:00:21.261392+0000","last_clean":"2024-05-15T14:00:21.261392+0000","last_became_active":"2024-05-15T13:57:14.535356+0000","last_became_peered":"2024-05-15T13:57:14.535356+0000","last_unstale":"2024-05-15T14:00:21.261392+0000","last_undegraded":"2024-05-15T14:00:21.261392+0000","last_fullsized":"2024-05-15T14:00:21.261392+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:16:21.545507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.417396+0000","last_change":"2024-05-15T13:57:17.383482+0000","last_active":"2024-05-15T14:00:21.417396+0000","last_peered":"2024-05-15T14:00:21.417396+0000","last_clean":"2024-05-15T14:00:21.417396+0000","last_became_active":"2024-05-15T13:57:17.382973+0000","last_became_peered":"2024-05-15T13:57:17.382973+0000","last_unstale":"2024-05-15T14:00:21.417396+0000","last_undegraded":"2024-05-15T14:00:21.417396+0000","last_fullsized":"2024-05-15T14:00:21.417396+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:31:08.937166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.581765+0000","last_change":"2024-05-15T13:58:06.078918+0000","last_active":"2024-05-15T14:00:21.581765+0000","last_peered":"2024-05-15T14:00:21.581765+0000","last_clean":"2024-05-15T14:00:21.581765+0000","last_became_active":"2024-05-15T13:58:06.073173+0000","last_became_peered":"2024-05-15T13:58:06.073173+0000","last_unstale":"2024-05-15T14:00:21.581765+0000","last_undegraded":"2024-05-15T14:00:21.581765+0000","last_fullsized":"2024-05-15T14:00:21.581765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:21:10.582131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.330748+0000","last_change":"2024-05-15T13:57:17.388720+0000","last_active":"2024-05-15T14:00:21.330748+0000","last_peered":"2024-05-15T14:00:21.330748+0000","last_clean":"2024-05-15T14:00:21.330748+0000","last_became_active":"2024-05-15T13:57:17.388386+0000","last_became_peered":"2024-05-15T13:57:17.388386+0000","last_unstale":"2024-05-15T14:00:21.330748+0000","last_undegraded":"2024-05-15T14:00:21.330748+0000","last_fullsized":"2024-05-15T14:00:21.330748+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:02:40.543701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260605+0000","last_change":"2024-05-15T13:57:14.259064+0000","last_active":"2024-05-15T14:00:21.260605+0000","last_peered":"2024-05-15T14:00:21.260605+0000","last_clean":"2024-05-15T14:00:21.260605+0000","last_became_active":"2024-05-15T13:57:14.258683+0000","last_became_peered":"2024-05-15T13:57:14.258683+0000","last_unstale":"2024-05-15T14:00:21.260605+0000","last_undegraded":"2024-05-15T14:00:21.260605+0000","last_fullsized":"2024-05-15T14:00:21.260605+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:12:43.108066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.266572+0000","last_change":"2024-05-15T13:57:17.380929+0000","last_active":"2024-05-15T14:00:21.266572+0000","last_peered":"2024-05-15T14:00:21.266572+0000","last_clean":"2024-05-15T14:00:21.266572+0000","last_became_active":"2024-05-15T13:57:17.380599+0000","last_became_peered":"2024-05-15T13:57:17.380599+0000","last_unstale":"2024-05-15T14:00:21.266572+0000","last_undegraded":"2024-05-15T14:00:21.266572+0000","last_fullsized":"2024-05-15T14:00:21.266572+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:19:55.368088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'9","reported_seq":216,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:25.622573+0000","last_change":"2024-05-15T13:57:14.260410+0000","last_active":"2024-05-15T14:00:25.622573+0000","last_peered":"2024-05-15T14:00:25.622573+0000","last_clean":"2024-05-15T14:00:25.622573+0000","last_became_active":"2024-05-15T13:57:14.260076+0000","last_became_peered":"2024-05-15T13:57:14.260076+0000","last_unstale":"2024-05-15T14:00:25.622573+0000","last_undegraded":"2024-05-15T14:00:25.622573+0000","last_fullsized":"2024-05-15T14:00:25.622573+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T00:27:03.436237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1853,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":178,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260404+0000","last_change":"2024-05-15T13:57:17.394475+0000","last_active":"2024-05-15T14:00:21.260404+0000","last_peered":"2024-05-15T14:00:21.260404+0000","last_clean":"2024-05-15T14:00:21.260404+0000","last_became_active":"2024-05-15T13:57:17.393733+0000","last_became_peered":"2024-05-15T13:57:17.393733+0000","last_unstale":"2024-05-15T14:00:21.260404+0000","last_undegraded":"2024-05-15T14:00:21.260404+0000","last_fullsized":"2024-05-15T14:00:21.260404+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:55:06.198633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.269838+0000","last_change":"2024-05-15T13:57:14.256420+0000","last_active":"2024-05-15T14:00:21.269838+0000","last_peered":"2024-05-15T14:00:21.269838+0000","last_clean":"2024-05-15T14:00:21.269838+0000","last_became_active":"2024-05-15T13:57:14.255722+0000","last_became_peered":"2024-05-15T13:57:14.255722+0000","last_unstale":"2024-05-15T14:00:21.269838+0000","last_undegraded":"2024-05-15T14:00:21.269838+0000","last_fullsized":"2024-05-15T14:00:21.269838+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:21:30.499632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.266546+0000","last_change":"2024-05-15T13:58:06.074623+0000","last_active":"2024-05-15T14:00:21.266546+0000","last_peered":"2024-05-15T14:00:21.266546+0000","last_clean":"2024-05-15T14:00:21.266546+0000","last_became_active":"2024-05-15T13:58:06.074153+0000","last_became_peered":"2024-05-15T13:58:06.074153+0000","last_unstale":"2024-05-15T14:00:21.266546+0000","last_undegraded":"2024-05-15T14:00:21.266546+0000","last_fullsized":"2024-05-15T14:00:21.266546+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:08:22.922239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414371+0000","last_change":"2024-05-15T13:57:14.256192+0000","last_active":"2024-05-15T14:00:21.414371+0000","last_peered":"2024-05-15T14:00:21.414371+0000","last_clean":"2024-05-15T14:00:21.414371+0000","last_became_active":"2024-05-15T13:57:14.255839+0000","last_became_peered":"2024-05-15T13:57:14.255839+0000","last_unstale":"2024-05-15T14:00:21.414371+0000","last_undegraded":"2024-05-15T14:00:21.414371+0000","last_fullsized":"2024-05-15T14:00:21.414371+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:13:26.071403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260676+0000","last_change":"2024-05-15T13:57:17.386246+0000","last_active":"2024-05-15T14:00:21.260676+0000","last_peered":"2024-05-15T14:00:21.260676+0000","last_clean":"2024-05-15T14:00:21.260676+0000","last_became_active":"2024-05-15T13:57:17.385828+0000","last_became_peered":"2024-05-15T13:57:17.385828+0000","last_unstale":"2024-05-15T14:00:21.260676+0000","last_undegraded":"2024-05-15T14:00:21.260676+0000","last_fullsized":"2024-05-15T14:00:21.260676+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T00:24:38.656469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.266243+0000","last_change":"2024-05-15T13:57:14.259953+0000","last_active":"2024-05-15T14:00:21.266243+0000","last_peered":"2024-05-15T14:00:21.266243+0000","last_clean":"2024-05-15T14:00:21.266243+0000","last_became_active":"2024-05-15T13:57:14.259632+0000","last_became_peered":"2024-05-15T13:57:14.259632+0000","last_unstale":"2024-05-15T14:00:21.266243+0000","last_undegraded":"2024-05-15T14:00:21.266243+0000","last_fullsized":"2024-05-15T14:00:21.266243+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:30:28.361903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.162534+0000","last_change":"2024-05-15T13:57:17.382211+0000","last_active":"2024-05-15T14:00:21.162534+0000","last_peered":"2024-05-15T14:00:21.162534+0000","last_clean":"2024-05-15T14:00:21.162534+0000","last_became_active":"2024-05-15T13:57:17.381904+0000","last_became_peered":"2024-05-15T13:57:17.381904+0000","last_unstale":"2024-05-15T14:00:21.162534+0000","last_undegraded":"2024-05-15T14:00:21.162534+0000","last_fullsized":"2024-05-15T14:00:21.162534+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:53:49.250866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'5","reported_seq":206,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:25.621835+0000","last_change":"2024-05-15T13:57:14.254691+0000","last_active":"2024-05-15T14:00:25.621835+0000","last_peered":"2024-05-15T14:00:25.621835+0000","last_clean":"2024-05-15T14:00:25.621835+0000","last_became_active":"2024-05-15T13:57:14.254233+0000","last_became_peered":"2024-05-15T13:57:14.254233+0000","last_unstale":"2024-05-15T14:00:25.621835+0000","last_undegraded":"2024-05-15T14:00:25.621835+0000","last_fullsized":"2024-05-15T14:00:25.621835+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:41:31.719402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":172,"num_read_kb":172,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.331127+0000","last_change":"2024-05-15T13:57:17.382251+0000","last_active":"2024-05-15T14:00:21.331127+0000","last_peered":"2024-05-15T14:00:21.331127+0000","last_clean":"2024-05-15T14:00:21.331127+0000","last_became_active":"2024-05-15T13:57:17.381158+0000","last_became_peered":"2024-05-15T13:57:17.381158+0000","last_unstale":"2024-05-15T14:00:21.331127+0000","last_undegraded":"2024-05-15T14:00:21.331127+0000","last_fullsized":"2024-05-15T14:00:21.331127+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:24:55.971079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414941+0000","last_change":"2024-05-15T13:57:14.258350+0000","last_active":"2024-05-15T14:00:21.414941+0000","last_peered":"2024-05-15T14:00:21.414941+0000","last_clean":"2024-05-15T14:00:21.414941+0000","last_became_active":"2024-05-15T13:57:14.257982+0000","last_became_peered":"2024-05-15T13:57:14.257982+0000","last_unstale":"2024-05-15T14:00:21.414941+0000","last_undegraded":"2024-05-15T14:00:21.414941+0000","last_fullsized":"2024-05-15T14:00:21.414941+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:50:07.053401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.417513+0000","last_change":"2024-05-15T13:57:17.392114+0000","last_active":"2024-05-15T14:00:21.417513+0000","last_peered":"2024-05-15T14:00:21.417513+0000","last_clean":"2024-05-15T14:00:21.417513+0000","last_became_active":"2024-05-15T13:57:17.391732+0000","last_became_peered":"2024-05-15T13:57:17.391732+0000","last_unstale":"2024-05-15T14:00:21.417513+0000","last_undegraded":"2024-05-15T14:00:21.417513+0000","last_fullsized":"2024-05-15T14:00:21.417513+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:28:02.304910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.266603+0000","last_change":"2024-05-15T13:57:14.535381+0000","last_active":"2024-05-15T14:00:21.266603+0000","last_peered":"2024-05-15T14:00:21.266603+0000","last_clean":"2024-05-15T14:00:21.266603+0000","last_became_active":"2024-05-15T13:57:14.534931+0000","last_became_peered":"2024-05-15T13:57:14.534931+0000","last_unstale":"2024-05-15T14:00:21.266603+0000","last_undegraded":"2024-05-15T14:00:21.266603+0000","last_fullsized":"2024-05-15T14:00:21.266603+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:13:04.785139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.330881+0000","last_change":"2024-05-15T13:57:17.386103+0000","last_active":"2024-05-15T14:00:21.330881+0000","last_peered":"2024-05-15T14:00:21.330881+0000","last_clean":"2024-05-15T14:00:21.330881+0000","last_became_active":"2024-05-15T13:57:17.385738+0000","last_became_peered":"2024-05-15T13:57:17.385738+0000","last_unstale":"2024-05-15T14:00:21.330881+0000","last_undegraded":"2024-05-15T14:00:21.330881+0000","last_fullsized":"2024-05-15T14:00:21.330881+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:42:56.136164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'36","reported_seq":421,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:26.623186+0000","last_change":"2024-05-15T13:57:14.258903+0000","last_active":"2024-05-15T14:00:26.623186+0000","last_peered":"2024-05-15T14:00:26.623186+0000","last_clean":"2024-05-15T14:00:26.623186+0000","last_became_active":"2024-05-15T13:57:14.258453+0000","last_became_peered":"2024-05-15T13:57:14.258453+0000","last_unstale":"2024-05-15T14:00:26.623186+0000","last_undegraded":"2024-05-15T14:00:26.623186+0000","last_fullsized":"2024-05-15T14:00:26.623186+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:49:33.004534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67059,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":356,"num_read_kb":243,"num_write":37,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260683+0000","last_change":"2024-05-15T13:57:17.389717+0000","last_active":"2024-05-15T14:00:21.260683+0000","last_peered":"2024-05-15T14:00:21.260683+0000","last_clean":"2024-05-15T14:00:21.260683+0000","last_became_active":"2024-05-15T13:57:17.389188+0000","last_became_peered":"2024-05-15T13:57:17.389188+0000","last_unstale":"2024-05-15T14:00:21.260683+0000","last_undegraded":"2024-05-15T14:00:21.260683+0000","last_fullsized":"2024-05-15T14:00:21.260683+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:40:26.827507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.166366+0000","last_change":"2024-05-15T13:57:14.256007+0000","last_active":"2024-05-15T14:00:21.166366+0000","last_peered":"2024-05-15T14:00:21.166366+0000","last_clean":"2024-05-15T14:00:21.166366+0000","last_became_active":"2024-05-15T13:57:14.255656+0000","last_became_peered":"2024-05-15T13:57:14.255656+0000","last_unstale":"2024-05-15T14:00:21.166366+0000","last_undegraded":"2024-05-15T14:00:21.166366+0000","last_fullsized":"2024-05-15T14:00:21.166366+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:14:46.880981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.413924+0000","last_change":"2024-05-15T13:57:17.389971+0000","last_active":"2024-05-15T14:00:21.413924+0000","last_peered":"2024-05-15T14:00:21.413924+0000","last_clean":"2024-05-15T14:00:21.413924+0000","last_became_active":"2024-05-15T13:57:17.389633+0000","last_became_peered":"2024-05-15T13:57:17.389633+0000","last_unstale":"2024-05-15T14:00:21.413924+0000","last_undegraded":"2024-05-15T14:00:21.413924+0000","last_fullsized":"2024-05-15T14:00:21.413924+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:25:10.971086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.266315+0000","last_change":"2024-05-15T13:57:14.258086+0000","last_active":"2024-05-15T14:00:21.266315+0000","last_peered":"2024-05-15T14:00:21.266315+0000","last_clean":"2024-05-15T14:00:21.266315+0000","last_became_active":"2024-05-15T13:57:14.257736+0000","last_became_peered":"2024-05-15T13:57:14.257736+0000","last_unstale":"2024-05-15T14:00:21.266315+0000","last_undegraded":"2024-05-15T14:00:21.266315+0000","last_fullsized":"2024-05-15T14:00:21.266315+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T00:51:07.431725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414677+0000","last_change":"2024-05-15T13:58:06.335760+0000","last_active":"2024-05-15T14:00:21.414677+0000","last_peered":"2024-05-15T14:00:21.414677+0000","last_clean":"2024-05-15T14:00:21.414677+0000","last_became_active":"2024-05-15T13:58:06.335236+0000","last_became_peered":"2024-05-15T13:58:06.335236+0000","last_unstale":"2024-05-15T14:00:21.414677+0000","last_undegraded":"2024-05-15T14:00:21.414677+0000","last_fullsized":"2024-05-15T14:00:21.414677+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:04:47.007886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'6","reported_seq":212,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:25.622090+0000","last_change":"2024-05-15T13:57:14.261911+0000","last_active":"2024-05-15T14:00:25.622090+0000","last_peered":"2024-05-15T14:00:25.622090+0000","last_clean":"2024-05-15T14:00:25.622090+0000","last_became_active":"2024-05-15T13:57:14.261506+0000","last_became_peered":"2024-05-15T13:57:14.261506+0000","last_unstale":"2024-05-15T14:00:25.622090+0000","last_undegraded":"2024-05-15T14:00:25.622090+0000","last_fullsized":"2024-05-15T14:00:25.622090+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:18:21.021651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":178,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414154+0000","last_change":"2024-05-15T13:57:17.395074+0000","last_active":"2024-05-15T14:00:21.414154+0000","last_peered":"2024-05-15T14:00:21.414154+0000","last_clean":"2024-05-15T14:00:21.414154+0000","last_became_active":"2024-05-15T13:57:17.394712+0000","last_became_peered":"2024-05-15T13:57:17.394712+0000","last_unstale":"2024-05-15T14:00:21.414154+0000","last_undegraded":"2024-05-15T14:00:21.414154+0000","last_fullsized":"2024-05-15T14:00:21.414154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:58:25.385949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":69,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260349+0000","last_change":"2024-05-15T13:58:06.086231+0000","last_active":"2024-05-15T14:00:21.260349+0000","last_peered":"2024-05-15T14:00:21.260349+0000","last_clean":"2024-05-15T14:00:21.260349+0000","last_became_active":"2024-05-15T13:58:06.074640+0000","last_became_peered":"2024-05-15T13:58:06.074640+0000","last_unstale":"2024-05-15T14:00:21.260349+0000","last_undegraded":"2024-05-15T14:00:21.260349+0000","last_fullsized":"2024-05-15T14:00:21.260349+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:41:54.747228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.266202+0000","last_change":"2024-05-15T13:57:17.376895+0000","last_active":"2024-05-15T14:00:21.266202+0000","last_peered":"2024-05-15T14:00:21.266202+0000","last_clean":"2024-05-15T14:00:21.266202+0000","last_became_active":"2024-05-15T13:57:17.376499+0000","last_became_peered":"2024-05-15T13:57:17.376499+0000","last_unstale":"2024-05-15T14:00:21.266202+0000","last_undegraded":"2024-05-15T14:00:21.266202+0000","last_fullsized":"2024-05-15T14:00:21.266202+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:39:41.707151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.581900+0000","last_change":"2024-05-15T13:58:06.090051+0000","last_active":"2024-05-15T14:00:21.581900+0000","last_peered":"2024-05-15T14:00:21.581900+0000","last_clean":"2024-05-15T14:00:21.581900+0000","last_became_active":"2024-05-15T13:58:06.074177+0000","last_became_peered":"2024-05-15T13:58:06.074177+0000","last_unstale":"2024-05-15T14:00:21.581900+0000","last_undegraded":"2024-05-15T14:00:21.581900+0000","last_fullsized":"2024-05-15T14:00:21.581900+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:07:11.090910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.581230+0000","last_change":"2024-05-15T13:57:17.390640+0000","last_active":"2024-05-15T14:00:21.581230+0000","last_peered":"2024-05-15T14:00:21.581230+0000","last_clean":"2024-05-15T14:00:21.581230+0000","last_became_active":"2024-05-15T13:57:17.390293+0000","last_became_peered":"2024-05-15T13:57:17.390293+0000","last_unstale":"2024-05-15T14:00:21.581230+0000","last_undegraded":"2024-05-15T14:00:21.581230+0000","last_fullsized":"2024-05-15T14:00:21.581230+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:08:18.907391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.162761+0000","last_change":"2024-05-15T13:57:14.535321+0000","last_active":"2024-05-15T14:00:21.162761+0000","last_peered":"2024-05-15T14:00:21.162761+0000","last_clean":"2024-05-15T14:00:21.162761+0000","last_became_active":"2024-05-15T13:57:14.534794+0000","last_became_peered":"2024-05-15T13:57:14.534794+0000","last_unstale":"2024-05-15T14:00:21.162761+0000","last_undegraded":"2024-05-15T14:00:21.162761+0000","last_fullsized":"2024-05-15T14:00:21.162761+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:27:07.087671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.270131+0000","last_change":"2024-05-15T13:57:17.381249+0000","last_active":"2024-05-15T14:00:21.270131+0000","last_peered":"2024-05-15T14:00:21.270131+0000","last_clean":"2024-05-15T14:00:21.270131+0000","last_became_active":"2024-05-15T13:57:17.380474+0000","last_became_peered":"2024-05-15T13:57:17.380474+0000","last_unstale":"2024-05-15T14:00:21.270131+0000","last_undegraded":"2024-05-15T14:00:21.270131+0000","last_fullsized":"2024-05-15T14:00:21.270131+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:49:00.341085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414724+0000","last_change":"2024-05-15T13:57:14.257635+0000","last_active":"2024-05-15T14:00:21.414724+0000","last_peered":"2024-05-15T14:00:21.414724+0000","last_clean":"2024-05-15T14:00:21.414724+0000","last_became_active":"2024-05-15T13:57:14.257156+0000","last_became_peered":"2024-05-15T13:57:14.257156+0000","last_unstale":"2024-05-15T14:00:21.414724+0000","last_undegraded":"2024-05-15T14:00:21.414724+0000","last_fullsized":"2024-05-15T14:00:21.414724+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:11:48.742053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.269906+0000","last_change":"2024-05-15T13:57:17.391067+0000","last_active":"2024-05-15T14:00:21.269906+0000","last_peered":"2024-05-15T14:00:21.269906+0000","last_clean":"2024-05-15T14:00:21.269906+0000","last_became_active":"2024-05-15T13:57:17.390763+0000","last_became_peered":"2024-05-15T13:57:17.390763+0000","last_unstale":"2024-05-15T14:00:21.269906+0000","last_undegraded":"2024-05-15T14:00:21.269906+0000","last_fullsized":"2024-05-15T14:00:21.269906+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:28:46.506307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.266550+0000","last_change":"2024-05-15T13:57:17.391816+0000","last_active":"2024-05-15T14:00:21.266550+0000","last_peered":"2024-05-15T14:00:21.266550+0000","last_clean":"2024-05-15T14:00:21.266550+0000","last_became_active":"2024-05-15T13:57:17.389374+0000","last_became_peered":"2024-05-15T13:57:17.389374+0000","last_unstale":"2024-05-15T14:00:21.266550+0000","last_undegraded":"2024-05-15T14:00:21.266550+0000","last_fullsized":"2024-05-15T14:00:21.266550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:48:31.756133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":207,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:25.621460+0000","last_change":"2024-05-15T13:57:14.534946+0000","last_active":"2024-05-15T14:00:25.621460+0000","last_peered":"2024-05-15T14:00:25.621460+0000","last_clean":"2024-05-15T14:00:25.621460+0000","last_became_active":"2024-05-15T13:57:14.534535+0000","last_became_peered":"2024-05-15T13:57:14.534535+0000","last_unstale":"2024-05-15T14:00:25.621460+0000","last_undegraded":"2024-05-15T14:00:25.621460+0000","last_fullsized":"2024-05-15T14:00:25.621460+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:27:20.526800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":172,"num_read_kb":172,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260866+0000","last_change":"2024-05-15T13:57:17.391637+0000","last_active":"2024-05-15T14:00:21.260866+0000","last_peered":"2024-05-15T14:00:21.260866+0000","last_clean":"2024-05-15T14:00:21.260866+0000","last_became_active":"2024-05-15T13:57:17.391271+0000","last_became_peered":"2024-05-15T13:57:17.391271+0000","last_unstale":"2024-05-15T14:00:21.260866+0000","last_undegraded":"2024-05-15T14:00:21.260866+0000","last_fullsized":"2024-05-15T14:00:21.260866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:22:06.398387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.163907+0000","last_change":"2024-05-15T13:57:14.258510+0000","last_active":"2024-05-15T14:00:21.163907+0000","last_peered":"2024-05-15T14:00:21.163907+0000","last_clean":"2024-05-15T14:00:21.163907+0000","last_became_active":"2024-05-15T13:57:14.257544+0000","last_became_peered":"2024-05-15T13:57:14.257544+0000","last_unstale":"2024-05-15T14:00:21.163907+0000","last_undegraded":"2024-05-15T14:00:21.163907+0000","last_fullsized":"2024-05-15T14:00:21.163907+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:12:27.253931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"83'99","reported_seq":130,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:23.267165+0000","last_change":"2024-05-15T13:55:53.940355+0000","last_active":"2024-05-15T14:00:23.267165+0000","last_peered":"2024-05-15T14:00:23.267165+0000","last_clean":"2024-05-15T14:00:23.267165+0000","last_became_active":"2024-05-15T13:55:53.934638+0000","last_became_peered":"2024-05-15T13:55:53.934638+0000","last_unstale":"2024-05-15T14:00:23.267165+0000","last_undegraded":"2024-05-15T14:00:23.267165+0000","last_fullsized":"2024-05-15T14:00:23.267165+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:52:33.556435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:52:33.556435+0000","last_clean_scrub_stamp":"2024-05-15T13:52:33.556435+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:02:55.332569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.413924+0000","last_change":"2024-05-15T13:57:17.382394+0000","last_active":"2024-05-15T14:00:21.413924+0000","last_peered":"2024-05-15T14:00:21.413924+0000","last_clean":"2024-05-15T14:00:21.413924+0000","last_became_active":"2024-05-15T13:57:17.382090+0000","last_became_peered":"2024-05-15T13:57:17.382090+0000","last_unstale":"2024-05-15T14:00:21.413924+0000","last_undegraded":"2024-05-15T14:00:21.413924+0000","last_fullsized":"2024-05-15T14:00:21.413924+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:01:34.653216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414627+0000","last_change":"2024-05-15T13:57:14.261549+0000","last_active":"2024-05-15T14:00:21.414627+0000","last_peered":"2024-05-15T14:00:21.414627+0000","last_clean":"2024-05-15T14:00:21.414627+0000","last_became_active":"2024-05-15T13:57:14.261165+0000","last_became_peered":"2024-05-15T13:57:14.261165+0000","last_unstale":"2024-05-15T14:00:21.414627+0000","last_undegraded":"2024-05-15T14:00:21.414627+0000","last_fullsized":"2024-05-15T14:00:21.414627+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:04:39.574440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260702+0000","last_change":"2024-05-15T13:57:14.535309+0000","last_active":"2024-05-15T14:00:21.260702+0000","last_peered":"2024-05-15T14:00:21.260702+0000","last_clean":"2024-05-15T14:00:21.260702+0000","last_became_active":"2024-05-15T13:57:14.534938+0000","last_became_peered":"2024-05-15T13:57:14.534938+0000","last_unstale":"2024-05-15T14:00:21.260702+0000","last_undegraded":"2024-05-15T14:00:21.260702+0000","last_fullsized":"2024-05-15T14:00:21.260702+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:34:53.373149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.413958+0000","last_change":"2024-05-15T13:57:17.388272+0000","last_active":"2024-05-15T14:00:21.413958+0000","last_peered":"2024-05-15T14:00:21.413958+0000","last_clean":"2024-05-15T14:00:21.413958+0000","last_became_active":"2024-05-15T13:57:17.387849+0000","last_became_peered":"2024-05-15T13:57:17.387849+0000","last_unstale":"2024-05-15T14:00:21.413958+0000","last_undegraded":"2024-05-15T14:00:21.413958+0000","last_fullsized":"2024-05-15T14:00:21.413958+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:47:47.323566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'101","reported_seq":222,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:23.422895+0000","last_change":"2024-05-15T13:57:14.256161+0000","last_active":"2024-05-15T14:00:23.422895+0000","last_peered":"2024-05-15T14:00:23.422895+0000","last_clean":"2024-05-15T14:00:23.422895+0000","last_became_active":"2024-05-15T13:57:14.255707+0000","last_became_peered":"2024-05-15T13:57:14.255707+0000","last_unstale":"2024-05-15T14:00:23.422895+0000","last_undegraded":"2024-05-15T14:00:23.422895+0000","last_fullsized":"2024-05-15T14:00:23.422895+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:24:57.200029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":186,"num_write_kb":1287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.261188+0000","last_change":"2024-05-15T13:57:17.389522+0000","last_active":"2024-05-15T14:00:21.261188+0000","last_peered":"2024-05-15T14:00:21.261188+0000","last_clean":"2024-05-15T14:00:21.261188+0000","last_became_active":"2024-05-15T13:57:17.389177+0000","last_became_peered":"2024-05-15T13:57:17.389177+0000","last_unstale":"2024-05-15T14:00:21.261188+0000","last_undegraded":"2024-05-15T14:00:21.261188+0000","last_fullsized":"2024-05-15T14:00:21.261188+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:23:35.716934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.164103+0000","last_change":"2024-05-15T13:57:14.254300+0000","last_active":"2024-05-15T14:00:21.164103+0000","last_peered":"2024-05-15T14:00:21.164103+0000","last_clean":"2024-05-15T14:00:21.164103+0000","last_became_active":"2024-05-15T13:57:14.253741+0000","last_became_peered":"2024-05-15T13:57:14.253741+0000","last_unstale":"2024-05-15T14:00:21.164103+0000","last_undegraded":"2024-05-15T14:00:21.164103+0000","last_fullsized":"2024-05-15T14:00:21.164103+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:25:34.018159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.416927+0000","last_change":"2024-05-15T13:57:17.380688+0000","last_active":"2024-05-15T14:00:21.416927+0000","last_peered":"2024-05-15T14:00:21.416927+0000","last_clean":"2024-05-15T14:00:21.416927+0000","last_became_active":"2024-05-15T13:57:17.380379+0000","last_became_peered":"2024-05-15T13:57:17.380379+0000","last_unstale":"2024-05-15T14:00:21.416927+0000","last_undegraded":"2024-05-15T14:00:21.416927+0000","last_fullsized":"2024-05-15T14:00:21.416927+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:32:09.190750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.330640+0000","last_change":"2024-05-15T13:57:14.534775+0000","last_active":"2024-05-15T14:00:21.330640+0000","last_peered":"2024-05-15T14:00:21.330640+0000","last_clean":"2024-05-15T14:00:21.330640+0000","last_became_active":"2024-05-15T13:57:14.534440+0000","last_became_peered":"2024-05-15T13:57:14.534440+0000","last_unstale":"2024-05-15T14:00:21.330640+0000","last_undegraded":"2024-05-15T14:00:21.330640+0000","last_fullsized":"2024-05-15T14:00:21.330640+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:21:49.047187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260352+0000","last_change":"2024-05-15T13:57:17.396252+0000","last_active":"2024-05-15T14:00:21.260352+0000","last_peered":"2024-05-15T14:00:21.260352+0000","last_clean":"2024-05-15T14:00:21.260352+0000","last_became_active":"2024-05-15T13:57:17.395853+0000","last_became_peered":"2024-05-15T13:57:17.395853+0000","last_unstale":"2024-05-15T14:00:21.260352+0000","last_undegraded":"2024-05-15T14:00:21.260352+0000","last_fullsized":"2024-05-15T14:00:21.260352+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:12:54.924053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260844+0000","last_change":"2024-05-15T13:57:14.258411+0000","last_active":"2024-05-15T14:00:21.260844+0000","last_peered":"2024-05-15T14:00:21.260844+0000","last_clean":"2024-05-15T14:00:21.260844+0000","last_became_active":"2024-05-15T13:57:14.257847+0000","last_became_peered":"2024-05-15T13:57:14.257847+0000","last_unstale":"2024-05-15T14:00:21.260844+0000","last_undegraded":"2024-05-15T14:00:21.260844+0000","last_fullsized":"2024-05-15T14:00:21.260844+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:49:32.368351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.581383+0000","last_change":"2024-05-15T13:57:17.388916+0000","last_active":"2024-05-15T14:00:21.581383+0000","last_peered":"2024-05-15T14:00:21.581383+0000","last_clean":"2024-05-15T14:00:21.581383+0000","last_became_active":"2024-05-15T13:57:17.388603+0000","last_became_peered":"2024-05-15T13:57:17.388603+0000","last_unstale":"2024-05-15T14:00:21.581383+0000","last_undegraded":"2024-05-15T14:00:21.581383+0000","last_fullsized":"2024-05-15T14:00:21.581383+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:38:46.650393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.166198+0000","last_change":"2024-05-15T13:57:14.256801+0000","last_active":"2024-05-15T14:00:21.166198+0000","last_peered":"2024-05-15T14:00:21.166198+0000","last_clean":"2024-05-15T14:00:21.166198+0000","last_became_active":"2024-05-15T13:57:14.256476+0000","last_became_peered":"2024-05-15T13:57:14.256476+0000","last_unstale":"2024-05-15T14:00:21.166198+0000","last_undegraded":"2024-05-15T14:00:21.166198+0000","last_fullsized":"2024-05-15T14:00:21.166198+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:22:34.495756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.261123+0000","last_change":"2024-05-15T13:57:17.380927+0000","last_active":"2024-05-15T14:00:21.261123+0000","last_peered":"2024-05-15T14:00:21.261123+0000","last_clean":"2024-05-15T14:00:21.261123+0000","last_became_active":"2024-05-15T13:57:17.380602+0000","last_became_peered":"2024-05-15T13:57:17.380602+0000","last_unstale":"2024-05-15T14:00:21.261123+0000","last_undegraded":"2024-05-15T14:00:21.261123+0000","last_fullsized":"2024-05-15T14:00:21.261123+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T14:37:13.920968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260541+0000","last_change":"2024-05-15T13:57:14.257954+0000","last_active":"2024-05-15T14:00:21.260541+0000","last_peered":"2024-05-15T14:00:21.260541+0000","last_clean":"2024-05-15T14:00:21.260541+0000","last_became_active":"2024-05-15T13:57:14.257601+0000","last_became_peered":"2024-05-15T13:57:14.257601+0000","last_unstale":"2024-05-15T14:00:21.260541+0000","last_undegraded":"2024-05-15T14:00:21.260541+0000","last_fullsized":"2024-05-15T14:00:21.260541+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:47:14.811264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414015+0000","last_change":"2024-05-15T13:57:17.385227+0000","last_active":"2024-05-15T14:00:21.414015+0000","last_peered":"2024-05-15T14:00:21.414015+0000","last_clean":"2024-05-15T14:00:21.414015+0000","last_became_active":"2024-05-15T13:57:17.384899+0000","last_became_peered":"2024-05-15T13:57:17.384899+0000","last_unstale":"2024-05-15T14:00:21.414015+0000","last_undegraded":"2024-05-15T14:00:21.414015+0000","last_fullsized":"2024-05-15T14:00:21.414015+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:04:29.162392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.166487+0000","last_change":"2024-05-15T13:57:14.535266+0000","last_active":"2024-05-15T14:00:21.166487+0000","last_peered":"2024-05-15T14:00:21.166487+0000","last_clean":"2024-05-15T14:00:21.166487+0000","last_became_active":"2024-05-15T13:57:14.534892+0000","last_became_peered":"2024-05-15T13:57:14.534892+0000","last_unstale":"2024-05-15T14:00:21.166487+0000","last_undegraded":"2024-05-15T14:00:21.166487+0000","last_fullsized":"2024-05-15T14:00:21.166487+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:49:50.934517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414053+0000","last_change":"2024-05-15T13:57:17.390330+0000","last_active":"2024-05-15T14:00:21.414053+0000","last_peered":"2024-05-15T14:00:21.414053+0000","last_clean":"2024-05-15T14:00:21.414053+0000","last_became_active":"2024-05-15T13:57:17.390000+0000","last_became_peered":"2024-05-15T13:57:17.390000+0000","last_unstale":"2024-05-15T14:00:21.414053+0000","last_undegraded":"2024-05-15T14:00:21.414053+0000","last_fullsized":"2024-05-15T14:00:21.414053+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T17:52:11.361549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'65","reported_seq":125,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:23.044357+0000","last_change":"2024-05-15T13:57:14.258174+0000","last_active":"2024-05-15T14:00:23.044357+0000","last_peered":"2024-05-15T14:00:23.044357+0000","last_clean":"2024-05-15T14:00:23.044357+0000","last_became_active":"2024-05-15T13:57:14.257763+0000","last_became_peered":"2024-05-15T13:57:14.257763+0000","last_unstale":"2024-05-15T14:00:23.044357+0000","last_undegraded":"2024-05-15T14:00:23.044357+0000","last_fullsized":"2024-05-15T14:00:23.044357+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:34:31.573845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414053+0000","last_change":"2024-05-15T13:57:17.391964+0000","last_active":"2024-05-15T14:00:21.414053+0000","last_peered":"2024-05-15T14:00:21.414053+0000","last_clean":"2024-05-15T14:00:21.414053+0000","last_became_active":"2024-05-15T13:57:17.391517+0000","last_became_peered":"2024-05-15T13:57:17.391517+0000","last_unstale":"2024-05-15T14:00:21.414053+0000","last_undegraded":"2024-05-15T14:00:21.414053+0000","last_fullsized":"2024-05-15T14:00:21.414053+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:54:36.883563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414380+0000","last_change":"2024-05-15T13:58:06.256119+0000","last_active":"2024-05-15T14:00:21.414380+0000","last_peered":"2024-05-15T14:00:21.414380+0000","last_clean":"2024-05-15T14:00:21.414380+0000","last_became_active":"2024-05-15T13:58:06.253495+0000","last_became_peered":"2024-05-15T13:58:06.253495+0000","last_unstale":"2024-05-15T14:00:21.414380+0000","last_undegraded":"2024-05-15T14:00:21.414380+0000","last_fullsized":"2024-05-15T14:00:21.414380+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:13.237112+0000","last_clean_scrub_stamp":"2024-05-15T13:57:13.237112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T20:41:06.585181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414149+0000","last_change":"2024-05-15T13:57:17.389518+0000","last_active":"2024-05-15T14:00:21.414149+0000","last_peered":"2024-05-15T14:00:21.414149+0000","last_clean":"2024-05-15T14:00:21.414149+0000","last_became_active":"2024-05-15T13:57:17.388905+0000","last_became_peered":"2024-05-15T13:57:17.388905+0000","last_unstale":"2024-05-15T14:00:21.414149+0000","last_undegraded":"2024-05-15T14:00:21.414149+0000","last_fullsized":"2024-05-15T14:00:21.414149+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:00:51.109880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.581179+0000","last_change":"2024-05-15T13:57:17.394127+0000","last_active":"2024-05-15T14:00:21.581179+0000","last_peered":"2024-05-15T14:00:21.581179+0000","last_clean":"2024-05-15T14:00:21.581179+0000","last_became_active":"2024-05-15T13:57:17.393771+0000","last_became_peered":"2024-05-15T13:57:17.393771+0000","last_unstale":"2024-05-15T14:00:21.581179+0000","last_undegraded":"2024-05-15T14:00:21.581179+0000","last_fullsized":"2024-05-15T14:00:21.581179+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T22:41:33.491001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.166195+0000","last_change":"2024-05-15T13:57:17.392451+0000","last_active":"2024-05-15T14:00:21.166195+0000","last_peered":"2024-05-15T14:00:21.166195+0000","last_clean":"2024-05-15T14:00:21.166195+0000","last_became_active":"2024-05-15T13:57:17.392099+0000","last_became_peered":"2024-05-15T13:57:17.392099+0000","last_unstale":"2024-05-15T14:00:21.166195+0000","last_undegraded":"2024-05-15T14:00:21.166195+0000","last_fullsized":"2024-05-15T14:00:21.166195+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:00:14.774543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.166069+0000","last_change":"2024-05-15T13:57:17.385525+0000","last_active":"2024-05-15T14:00:21.166069+0000","last_peered":"2024-05-15T14:00:21.166069+0000","last_clean":"2024-05-15T14:00:21.166069+0000","last_became_active":"2024-05-15T13:57:17.384497+0000","last_became_peered":"2024-05-15T13:57:17.384497+0000","last_unstale":"2024-05-15T14:00:21.166069+0000","last_undegraded":"2024-05-15T14:00:21.166069+0000","last_fullsized":"2024-05-15T14:00:21.166069+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:50:39.259339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.416153+0000","last_change":"2024-05-15T13:57:17.379673+0000","last_active":"2024-05-15T14:00:21.416153+0000","last_peered":"2024-05-15T14:00:21.416153+0000","last_clean":"2024-05-15T14:00:21.416153+0000","last_became_active":"2024-05-15T13:57:17.379342+0000","last_became_peered":"2024-05-15T13:57:17.379342+0000","last_unstale":"2024-05-15T14:00:21.416153+0000","last_undegraded":"2024-05-15T14:00:21.416153+0000","last_fullsized":"2024-05-15T14:00:21.416153+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:26:43.982972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414364+0000","last_change":"2024-05-15T13:58:06.335360+0000","last_active":"2024-05-15T14:00:21.414364+0000","last_peered":"2024-05-15T14:00:21.414364+0000","last_clean":"2024-05-15T14:00:21.414364+0000","last_became_active":"2024-05-15T13:58:06.334899+0000","last_became_peered":"2024-05-15T13:58:06.334899+0000","last_unstale":"2024-05-15T14:00:21.414364+0000","last_undegraded":"2024-05-15T14:00:21.414364+0000","last_fullsized":"2024-05-15T14:00:21.414364+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T15:34:27.081929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.581196+0000","last_change":"2024-05-15T13:57:17.390939+0000","last_active":"2024-05-15T14:00:21.581196+0000","last_peered":"2024-05-15T14:00:21.581196+0000","last_clean":"2024-05-15T14:00:21.581196+0000","last_became_active":"2024-05-15T13:57:17.390281+0000","last_became_peered":"2024-05-15T13:57:17.390281+0000","last_unstale":"2024-05-15T14:00:21.581196+0000","last_undegraded":"2024-05-15T14:00:21.581196+0000","last_fullsized":"2024-05-15T14:00:21.581196+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T18:02:22.077307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.166336+0000","last_change":"2024-05-15T13:57:17.392009+0000","last_active":"2024-05-15T14:00:21.166336+0000","last_peered":"2024-05-15T14:00:21.166336+0000","last_clean":"2024-05-15T14:00:21.166336+0000","last_became_active":"2024-05-15T13:57:17.391059+0000","last_became_peered":"2024-05-15T13:57:17.391059+0000","last_unstale":"2024-05-15T14:00:21.166336+0000","last_undegraded":"2024-05-15T14:00:21.166336+0000","last_fullsized":"2024-05-15T14:00:21.166336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T19:51:46.787790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.261169+0000","last_change":"2024-05-15T13:57:17.383872+0000","last_active":"2024-05-15T14:00:21.261169+0000","last_peered":"2024-05-15T14:00:21.261169+0000","last_clean":"2024-05-15T14:00:21.261169+0000","last_became_active":"2024-05-15T13:57:17.383526+0000","last_became_peered":"2024-05-15T13:57:17.383526+0000","last_unstale":"2024-05-15T14:00:21.261169+0000","last_undegraded":"2024-05-15T14:00:21.261169+0000","last_fullsized":"2024-05-15T14:00:21.261169+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:43:39.343557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.266300+0000","last_change":"2024-05-15T13:57:17.377530+0000","last_active":"2024-05-15T14:00:21.266300+0000","last_peered":"2024-05-15T14:00:21.266300+0000","last_clean":"2024-05-15T14:00:21.266300+0000","last_became_active":"2024-05-15T13:57:17.377237+0000","last_became_peered":"2024-05-15T13:57:17.377237+0000","last_unstale":"2024-05-15T14:00:21.266300+0000","last_undegraded":"2024-05-15T14:00:21.266300+0000","last_fullsized":"2024-05-15T14:00:21.266300+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T16:29:02.173018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.260755+0000","last_change":"2024-05-15T13:57:17.386208+0000","last_active":"2024-05-15T14:00:21.260755+0000","last_peered":"2024-05-15T14:00:21.260755+0000","last_clean":"2024-05-15T14:00:21.260755+0000","last_became_active":"2024-05-15T13:57:17.385853+0000","last_became_peered":"2024-05-15T13:57:17.385853+0000","last_unstale":"2024-05-15T14:00:21.260755+0000","last_undegraded":"2024-05-15T14:00:21.260755+0000","last_fullsized":"2024-05-15T14:00:21.260755+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:17:06.124653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.330803+0000","last_change":"2024-05-15T13:57:17.390074+0000","last_active":"2024-05-15T14:00:21.330803+0000","last_peered":"2024-05-15T14:00:21.330803+0000","last_clean":"2024-05-15T14:00:21.330803+0000","last_became_active":"2024-05-15T13:57:17.389765+0000","last_became_peered":"2024-05-15T13:57:17.389765+0000","last_unstale":"2024-05-15T14:00:21.330803+0000","last_undegraded":"2024-05-15T14:00:21.330803+0000","last_fullsized":"2024-05-15T14:00:21.330803+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T23:49:51.321180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.414436+0000","last_change":"2024-05-15T13:57:17.389180+0000","last_active":"2024-05-15T14:00:21.414436+0000","last_peered":"2024-05-15T14:00:21.414436+0000","last_clean":"2024-05-15T14:00:21.414436+0000","last_became_active":"2024-05-15T13:57:17.388731+0000","last_became_peered":"2024-05-15T13:57:17.388731+0000","last_unstale":"2024-05-15T14:00:21.414436+0000","last_undegraded":"2024-05-15T14:00:21.414436+0000","last_fullsized":"2024-05-15T14:00:21.414436+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-17T01:19:00.655474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-15T14:00:21.581262+0000","last_change":"2024-05-15T13:57:17.394045+0000","last_active":"2024-05-15T14:00:21.581262+0000","last_peered":"2024-05-15T14:00:21.581262+0000","last_clean":"2024-05-15T14:00:21.581262+0000","last_became_active":"2024-05-15T13:57:17.393699+0000","last_became_peered":"2024-05-15T13:57:17.393699+0000","last_unstale":"2024-05-15T14:00:21.581262+0000","last_undegraded":"2024-05-15T14:00:21.581262+0000","last_fullsized":"2024-05-15T14:00:21.581262+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-15T13:57:16.358413+0000","last_clean_scrub_stamp":"2024-05-15T13:57:16.358413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-16T21:05:37.510370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":794271,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1413,"num_read_kb":920,"num_write":421,"num_write_kb":2355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":134,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2617344,"data_stored":2382813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":310,"ondisk_log_size":310,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645364,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":913408,"data_stored":441740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841593,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37356,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707796,"statfs":{"total":95995035648,"available":95956783104,"internally_reserved":0,"allocated":1945600,"data_stored":1429413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005118,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32464,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712688,"statfs":{"total":95995035648,"available":95961792512,"internally_reserved":0,"allocated":1318912,"data_stored":835170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201347,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36620,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708532,"statfs":{"total":95995035648,"available":95957536768,"internally_reserved":0,"allocated":1314816,"data_stored":835068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397576,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28500,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716652,"statfs":{"total":95995035648,"available":95965851648,"internally_reserved":0,"allocated":1519616,"data_stored":1035567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561103,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36232,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708920,"statfs":{"total":95995035648,"available":95957934080,"internally_reserved":0,"allocated":913408,"data_stored":441343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":508195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738475,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":933888,"data_stored":442614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":510664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149788,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28752,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716400,"statfs":{"total":95995035648,"available":95965593600,"internally_reserved":0,"allocated":1654784,"data_stored":1163102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953560,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":442815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757331,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37276,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707876,"statfs":{"total":95995035648,"available":95956865024,"internally_reserved":0,"allocated":1863680,"data_stored":1362560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":69528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":330939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":397792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-15T14:00:32.268 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T14:00:32.522 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:00:32.644 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:32 smithi033 ceph-mon[31650]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 102 KiB/s wr, 54 op/s 2024-05-15T14:00:32.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:32 smithi052 ceph-mon[36190]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 102 KiB/s wr, 54 op/s 2024-05-15T14:00:32.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:32 smithi081 ceph-mon[36434]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 102 KiB/s wr, 54 op/s 2024-05-15T14:00:33.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:33 smithi033 ceph-mon[31650]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-15T14:00:33.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:33 smithi052 ceph-mon[36190]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-15T14:00:33.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:33 smithi081 ceph-mon[36434]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-15T14:00:34.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.040+0000 7f2f1d10d700 1 -- 172.21.15.33:0/1073442181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f1806bd10 msgr2=0x7f2f1806c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:34.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.040+0000 7f2f1d10d700 1 --2- 172.21.15.33:0/1073442181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f1806bd10 0x7f2f1806c190 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000b0d0 tx=0x7f2f0000b3e0 comp rx=0 tx=0).stop 2024-05-15T14:00:34.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.040+0000 7f2f1d10d700 1 -- 172.21.15.33:0/1073442181 shutdown_connections 2024-05-15T14:00:34.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.040+0000 7f2f1d10d700 1 --2- 172.21.15.33:0/1073442181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f1806c6d0 0x7f2f18110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:34.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.040+0000 7f2f1d10d700 1 --2- 172.21.15.33:0/1073442181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f18109630 0x7f2f18109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:34.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.040+0000 7f2f1d10d700 1 --2- 172.21.15.33:0/1073442181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f1806bd10 0x7f2f1806c190 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:34.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.040+0000 7f2f1d10d700 1 -- 172.21.15.33:0/1073442181 >> 172.21.15.33:0/1073442181 conn(0x7f2f180794e0 msgr2=0x7f2f180798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:34.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.040+0000 7f2f1d10d700 1 -- 172.21.15.33:0/1073442181 shutdown_connections 2024-05-15T14:00:34.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.040+0000 7f2f1d10d700 1 -- 172.21.15.33:0/1073442181 wait complete. 2024-05-15T14:00:34.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.041+0000 7f2f1d10d700 1 Processor -- start 2024-05-15T14:00:34.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.041+0000 7f2f1d10d700 1 -- start start 2024-05-15T14:00:34.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.042+0000 7f2f1d10d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f1806bd10 0x7f2f1819f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:34.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.042+0000 7f2f1d10d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f1806c6d0 0x7f2f1819fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:34.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.042+0000 7f2f1d10d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f18109630 0x7f2f18199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:34.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.042+0000 7f2f1d10d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2f18113790 con 0x7f2f1806c6d0 2024-05-15T14:00:34.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.042+0000 7f2f1d10d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2f18113610 con 0x7f2f18109630 2024-05-15T14:00:34.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.042+0000 7f2f1d10d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2f18113910 con 0x7f2f1806bd10 2024-05-15T14:00:34.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.042+0000 7f2f1659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f1806c6d0 0x7f2f1819fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:34.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.042+0000 7f2f1659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f1806c6d0 0x7f2f1819fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43900/0 (socket says 172.21.15.33:43900) 2024-05-15T14:00:34.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.042+0000 7f2f1659c700 1 -- 172.21.15.33:0/3121946583 learned_addr learned my addr 172.21.15.33:0/3121946583 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:34.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.042+0000 7f2f1759e700 1 --2- 172.21.15.33:0/3121946583 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f18109630 0x7f2f18199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:34.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.042+0000 7f2f16d9d700 1 --2- 172.21.15.33:0/3121946583 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f1806bd10 0x7f2f1819f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:34.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.042+0000 7f2f1659c700 1 -- 172.21.15.33:0/3121946583 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f1806bd10 msgr2=0x7f2f1819f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:34.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.042+0000 7f2f1659c700 1 --2- 172.21.15.33:0/3121946583 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f1806bd10 0x7f2f1819f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:34.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.042+0000 7f2f1659c700 1 -- 172.21.15.33:0/3121946583 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f18109630 msgr2=0x7f2f18199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:34.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.042+0000 7f2f1659c700 1 --2- 172.21.15.33:0/3121946583 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f18109630 0x7f2f18199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:34.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.042+0000 7f2f1659c700 1 -- 172.21.15.33:0/3121946583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f1819a250 con 0x7f2f1806c6d0 2024-05-15T14:00:34.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.042+0000 7f2f1659c700 1 --2- 172.21.15.33:0/3121946583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f1806c6d0 0x7f2f1819fe70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2f00007a20 tx=0x7f2f00007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:34.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.042+0000 7f2f0ffff700 1 -- 172.21.15.33:0/3121946583 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0001f070 con 0x7f2f1806c6d0 2024-05-15T14:00:34.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.043+0000 7f2f1d10d700 1 -- 172.21.15.33:0/3121946583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f1819a4e0 con 0x7f2f1806c6d0 2024-05-15T14:00:34.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.043+0000 7f2f1d10d700 1 -- 172.21.15.33:0/3121946583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f181a6800 con 0x7f2f1806c6d0 2024-05-15T14:00:34.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.043+0000 7f2f0ffff700 1 -- 172.21.15.33:0/3121946583 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f00003c00 con 0x7f2f1806c6d0 2024-05-15T14:00:34.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.043+0000 7f2f0ffff700 1 -- 172.21.15.33:0/3121946583 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0001b440 con 0x7f2f1806c6d0 2024-05-15T14:00:34.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.044+0000 7f2f1d10d700 1 -- 172.21.15.33:0/3121946583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f1810bac0 con 0x7f2f1806c6d0 2024-05-15T14:00:34.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.045+0000 7f2f0ffff700 1 -- 172.21.15.33:0/3121946583 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f2f00022070 con 0x7f2f1806c6d0 2024-05-15T14:00:34.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.046+0000 7f2f0ffff700 1 --2- 172.21.15.33:0/3121946583 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f2f040b2a80 0x7f2f040b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:34.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.046+0000 7f2f16d9d700 1 --2- 172.21.15.33:0/3121946583 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f2f040b2a80 0x7f2f040b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:34.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.046+0000 7f2f0ffff700 1 -- 172.21.15.33:0/3121946583 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f2f000e34d0 con 0x7f2f1806c6d0 2024-05-15T14:00:34.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.046+0000 7f2f16d9d700 1 --2- 172.21.15.33:0/3121946583 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f2f040b2a80 0x7f2f040b4f40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2f10006fd0 tx=0x7f2f10008040 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:34.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.049+0000 7f2f0ffff700 1 -- 172.21.15.33:0/3121946583 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2f00029030 con 0x7f2f1806c6d0 2024-05-15T14:00:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.320+0000 7f2f1d10d700 1 -- 172.21.15.33:0/3121946583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f1806a230 con 0x7f2f1806c6d0 2024-05-15T14:00:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.321+0000 7f2f0ffff700 1 -- 172.21.15.33:0/3121946583 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11858 (secure 0 0 0) 0x7f2f00009cb0 con 0x7f2f1806c6d0 2024-05-15T14:00:34.324 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T14:00:34.324 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2030117633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2030117633},{"type":"v1","addr":"172.21.15.33:6839","nonce":2030117633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1918979061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1918979061},{"type":"v1","addr":"172.21.15.33:6841","nonce":1918979061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T13:57:19.056417+0000","modified":"2024-05-15T14:00:18.543619+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24463,"mds_2":24461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/1629697841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1629697841},{"type":"v1","addr":"172.21.15.52:6837","nonce":1629697841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/3773839758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3773839758},{"type":"v1","addr":"172.21.15.81:6835","nonce":3773839758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.52:6839/2696044471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2696044471},{"type":"v1","addr":"172.21.15.52:6839","nonce":2696044471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.81:6839/1554395940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":1554395940},{"type":"v1","addr":"172.21.15.81:6839","nonce":1554395940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24463,24478,24461]},"id":1}]} 2024-05-15T14:00:34.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.323+0000 7f2f1d10d700 1 -- 172.21.15.33:0/3121946583 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f2f040b2a80 msgr2=0x7f2f040b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:34.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.323+0000 7f2f1d10d700 1 --2- 172.21.15.33:0/3121946583 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f2f040b2a80 0x7f2f040b4f40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2f10006fd0 tx=0x7f2f10008040 comp rx=0 tx=0).stop 2024-05-15T14:00:34.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.323+0000 7f2f1d10d700 1 -- 172.21.15.33:0/3121946583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f1806c6d0 msgr2=0x7f2f1819fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:34.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.323+0000 7f2f1d10d700 1 --2- 172.21.15.33:0/3121946583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f1806c6d0 0x7f2f1819fe70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2f00007a20 tx=0x7f2f00007a50 comp rx=0 tx=0).stop 2024-05-15T14:00:34.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.323+0000 7f2f1d10d700 1 -- 172.21.15.33:0/3121946583 shutdown_connections 2024-05-15T14:00:34.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.323+0000 7f2f1d10d700 1 --2- 172.21.15.33:0/3121946583 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f2f040b2a80 0x7f2f040b4f40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:34.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.323+0000 7f2f1d10d700 1 --2- 172.21.15.33:0/3121946583 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f18109630 0x7f2f18199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:34.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.323+0000 7f2f1d10d700 1 --2- 172.21.15.33:0/3121946583 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f1806bd10 0x7f2f1819f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:34.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.323+0000 7f2f1d10d700 1 --2- 172.21.15.33:0/3121946583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f1806c6d0 0x7f2f1819fe70 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:34.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.323+0000 7f2f1d10d700 1 -- 172.21.15.33:0/3121946583 >> 172.21.15.33:0/3121946583 conn(0x7f2f180794e0 msgr2=0x7f2f181011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:34.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.324+0000 7f2f1d10d700 1 -- 172.21.15.33:0/3121946583 shutdown_connections 2024-05-15T14:00:34.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:34.324+0000 7f2f1d10d700 1 -- 172.21.15.33:0/3121946583 wait complete. 2024-05-15T14:00:34.327 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 24 2024-05-15T14:00:34.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:34 smithi033 ceph-mon[31650]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 83 KiB/s wr, 46 op/s 2024-05-15T14:00:34.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:34 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/3121946583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:00:34.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:34 smithi052 ceph-mon[36190]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 83 KiB/s wr, 46 op/s 2024-05-15T14:00:34.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:34 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/3121946583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:00:34.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:34 smithi081 ceph-mon[36434]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 83 KiB/s wr, 46 op/s 2024-05-15T14:00:34.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:34 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/3121946583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:00:34.998 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T14:00:35.253 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:00:36.593 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:36 smithi033 ceph-mon[31650]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 42 op/s 2024-05-15T14:00:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.671+0000 7f879a8f8640 1 -- 172.21.15.33:0/1440643442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87941045e0 msgr2=0x7f879410cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.671+0000 7f879a8f8640 1 --2- 172.21.15.33:0/1440643442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87941045e0 0x7f879410cac0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8788009900 tx=0x7f878802a480 comp rx=0 tx=0).stop 2024-05-15T14:00:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.671+0000 7f879a8f8640 1 -- 172.21.15.33:0/1440643442 shutdown_connections 2024-05-15T14:00:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.671+0000 7f879a8f8640 1 --2- 172.21.15.33:0/1440643442 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f879410d090 0x7f879410f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.671+0000 7f879a8f8640 1 --2- 172.21.15.33:0/1440643442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87941045e0 0x7f879410cac0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.671+0000 7f879a8f8640 1 --2- 172.21.15.33:0/1440643442 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8794103cc0 0x7f87941040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.671+0000 7f879a8f8640 1 -- 172.21.15.33:0/1440643442 >> 172.21.15.33:0/1440643442 conn(0x7f8794078a70 msgr2=0x7f8794078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.671+0000 7f879a8f8640 1 -- 172.21.15.33:0/1440643442 shutdown_connections 2024-05-15T14:00:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.671+0000 7f879a8f8640 1 -- 172.21.15.33:0/1440643442 wait complete. 2024-05-15T14:00:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.672+0000 7f879a8f8640 1 Processor -- start 2024-05-15T14:00:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.672+0000 7f879a8f8640 1 -- start start 2024-05-15T14:00:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.672+0000 7f879a8f8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8794103cc0 0x7f8794199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.672+0000 7f879a8f8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87941045e0 0x7f8794199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.672+0000 7f879a8f8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f879410d090 0x7f879419db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.672+0000 7f879a8f8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8794112cd0 con 0x7f87941045e0 2024-05-15T14:00:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.672+0000 7f879a8f8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8794112b50 con 0x7f8794103cc0 2024-05-15T14:00:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.672+0000 7f879a8f8640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f8794112e50 con 0x7f879410d090 2024-05-15T14:00:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.672+0000 7f87937fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87941045e0 0x7f8794199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.672+0000 7f87937fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87941045e0 0x7f8794199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43910/0 (socket says 172.21.15.33:43910) 2024-05-15T14:00:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.672+0000 7f87937fe640 1 -- 172.21.15.33:0/1298193473 learned_addr learned my addr 172.21.15.33:0/1298193473 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.673+0000 7f8798e95640 1 --2- 172.21.15.33:0/1298193473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f879410d090 0x7f879419db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.673+0000 7f8793fff640 1 --2- 172.21.15.33:0/1298193473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8794103cc0 0x7f8794199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.673+0000 7f87937fe640 1 -- 172.21.15.33:0/1298193473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f879410d090 msgr2=0x7f879419db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.673+0000 7f87937fe640 1 --2- 172.21.15.33:0/1298193473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f879410d090 0x7f879419db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.673+0000 7f87937fe640 1 -- 172.21.15.33:0/1298193473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8794103cc0 msgr2=0x7f8794199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.673+0000 7f87937fe640 1 --2- 172.21.15.33:0/1298193473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8794103cc0 0x7f8794199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.673+0000 7f87937fe640 1 -- 172.21.15.33:0/1298193473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f879419e240 con 0x7f87941045e0 2024-05-15T14:00:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.673+0000 7f8793fff640 1 --2- 172.21.15.33:0/1298193473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8794103cc0 0x7f8794199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:00:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.673+0000 7f87937fe640 1 --2- 172.21.15.33:0/1298193473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87941045e0 0x7f8794199790 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8788005940 tx=0x7f8788004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.673+0000 7f87917fa640 1 -- 172.21.15.33:0/1298193473 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878803f030 con 0x7f87941045e0 2024-05-15T14:00:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.673+0000 7f879a8f8640 1 -- 172.21.15.33:0/1298193473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f879419e4d0 con 0x7f87941045e0 2024-05-15T14:00:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.673+0000 7f87917fa640 1 -- 172.21.15.33:0/1298193473 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8788003aa0 con 0x7f87941045e0 2024-05-15T14:00:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.673+0000 7f87917fa640 1 -- 172.21.15.33:0/1298193473 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8788035070 con 0x7f87941045e0 2024-05-15T14:00:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.673+0000 7f879a8f8640 1 -- 172.21.15.33:0/1298193473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87941a5dc0 con 0x7f87941045e0 2024-05-15T14:00:36.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.676+0000 7f87917fa640 1 -- 172.21.15.33:0/1298193473 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f8788042030 con 0x7f87941045e0 2024-05-15T14:00:36.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.676+0000 7f879d60e740 1 Processor -- start 2024-05-15T14:00:36.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.676+0000 7f879d60e740 1 -- start start 2024-05-15T14:00:36.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.676+0000 7f879d60e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558af5a6c7f0 0x558af5a6cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:36.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.676+0000 7f879d60e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558af5a6d110 0x558af5a7b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.676+0000 7f879d60e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558af5a7bee0 0x558af5a7e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.676+0000 7f879d60e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558af59dfc30 con 0x558af5a7bee0 2024-05-15T14:00:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.676+0000 7f879d60e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558af59cf7d0 con 0x558af5a6d110 2024-05-15T14:00:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.676+0000 7f879d60e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x558af59d3810 con 0x558af5a6c7f0 2024-05-15T14:00:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.676+0000 7f8798e95640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558af5a6c7f0 0x558af5a6cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.676+0000 7f87937fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558af5a7bee0 0x558af5a7e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.676+0000 7f8798e95640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558af5a6c7f0 0x558af5a6cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:60720/0 (socket says 172.21.15.33:60720) 2024-05-15T14:00:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.676+0000 7f8798e95640 1 -- 172.21.15.33:0/2865799586 learned_addr learned my addr 172.21.15.33:0/2865799586 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.676+0000 7f8793fff640 1 --2- 172.21.15.33:0/2865799586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558af5a6d110 0x558af5a7b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.677+0000 7f8798e95640 1 -- 172.21.15.33:0/2865799586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558af5a6d110 msgr2=0x558af5a7b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.677+0000 7f8798e95640 1 --2- 172.21.15.33:0/2865799586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558af5a6d110 0x558af5a7b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.677+0000 7f8798e95640 1 -- 172.21.15.33:0/2865799586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558af5a7bee0 msgr2=0x558af5a7e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.677+0000 7f8798e95640 1 --2- 172.21.15.33:0/2865799586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558af5a7bee0 0x558af5a7e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.677+0000 7f8798e95640 1 -- 172.21.15.33:0/2865799586 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558af56e4210 con 0x558af5a6c7f0 2024-05-15T14:00:36.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.677+0000 7f87917fa640 1 --2- 172.21.15.33:0/1298193473 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f877c0b29e0 0x7f877c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:36.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.677+0000 7f8793fff640 1 --2- 172.21.15.33:0/1298193473 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f877c0b29e0 0x7f877c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:36.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.677+0000 7f8798e95640 1 --2- 172.21.15.33:0/2865799586 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558af5a6c7f0 0x558af5a6cbd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f878c009f30 tx=0x7f878c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eda0dffd83233da9 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:36.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.677+0000 7f87917fa640 1 -- 172.21.15.33:0/1298193473 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f8788102c90 con 0x7f87941045e0 2024-05-15T14:00:36.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.677+0000 7f877a7fc640 1 -- 172.21.15.33:0/2865799586 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c004d20 con 0x558af5a6c7f0 2024-05-15T14:00:36.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.677+0000 7f877a7fc640 1 -- 172.21.15.33:0/2865799586 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f878c004050 con 0x558af5a6c7f0 2024-05-15T14:00:36.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.677+0000 7f877a7fc640 1 -- 172.21.15.33:0/2865799586 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c005220 con 0x558af5a6c7f0 2024-05-15T14:00:36.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.678+0000 7f879d60e740 1 -- 172.21.15.33:0/2865799586 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558af5a6c7f0 msgr2=0x558af5a6cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:36.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.678+0000 7f8793fff640 1 --2- 172.21.15.33:0/1298193473 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f877c0b29e0 0x7f877c0b4ea0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8784005d00 tx=0x7f8784005c50 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:36.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.678+0000 7f879d60e740 1 --2- 172.21.15.33:0/2865799586 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558af5a6c7f0 0x558af5a6cbd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f878c009f30 tx=0x7f878c0118d0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.678+0000 7f879d60e740 1 -- 172.21.15.33:0/2865799586 shutdown_connections 2024-05-15T14:00:36.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.678+0000 7f879d60e740 1 --2- 172.21.15.33:0/2865799586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558af5a7bee0 0x558af5a7e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.678+0000 7f879d60e740 1 --2- 172.21.15.33:0/2865799586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558af5a6d110 0x558af5a7b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.678+0000 7f879d60e740 1 --2- 172.21.15.33:0/2865799586 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558af5a6c7f0 0x558af5a6cbd0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.678+0000 7f879d60e740 1 -- 172.21.15.33:0/2865799586 >> 172.21.15.33:0/2865799586 conn(0x558af5a68e00 msgr2=0x558af5a691e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:36.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.678+0000 7f879d60e740 1 -- 172.21.15.33:0/2865799586 shutdown_connections 2024-05-15T14:00:36.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.678+0000 7f879d60e740 1 -- 172.21.15.33:0/2865799586 wait complete. 2024-05-15T14:00:36.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.678+0000 7f879d60e740 1 -- start start 2024-05-15T14:00:36.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.679+0000 7f879d60e740 1 Processor -- start 2024-05-15T14:00:36.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.679+0000 7f879d60e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558af5a6c7f0 0x558af5a7b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:36.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.679+0000 7f879d60e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558af5a6d110 0x558af5a78ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:36.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.679+0000 7f879d60e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558af5a7bee0 0x558af5a79000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:36.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.679+0000 7f879d60e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558af56b5790 con 0x558af5a6c7f0 2024-05-15T14:00:36.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.679+0000 7f879d60e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558af598b890 con 0x558af5a6d110 2024-05-15T14:00:36.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.679+0000 7f879d60e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x558af5a7ebb0 con 0x558af5a7bee0 2024-05-15T14:00:36.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.679+0000 7f879d60e740 10 client.0 _ll_register_callbacks cb 0x558af598ed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:00:36.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.679+0000 7f8793fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558af5a6c7f0 0x558af5a7b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:36.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.679+0000 7f8793fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558af5a6c7f0 0x558af5a7b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43934/0 (socket says 172.21.15.33:43934) 2024-05-15T14:00:36.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.679+0000 7f8793fff640 1 -- 172.21.15.33:0/2881259298 learned_addr learned my addr 172.21.15.33:0/2881259298 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:36.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.679+0000 7f87937fe640 1 --2- 172.21.15.33:0/2881259298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558af5a6d110 0x558af5a78ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:36.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.679+0000 7f8798e95640 1 --2- 172.21.15.33:0/2881259298 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558af5a7bee0 0x558af5a79000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:36.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.679+0000 7f8793fff640 1 -- 172.21.15.33:0/2881259298 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558af5a7bee0 msgr2=0x558af5a79000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:36.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.679+0000 7f8793fff640 1 --2- 172.21.15.33:0/2881259298 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558af5a7bee0 0x558af5a79000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.679+0000 7f8793fff640 1 -- 172.21.15.33:0/2881259298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558af5a6d110 msgr2=0x558af5a78ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:36.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.679+0000 7f8793fff640 1 --2- 172.21.15.33:0/2881259298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558af5a6d110 0x558af5a78ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.679+0000 7f8793fff640 1 -- 172.21.15.33:0/2881259298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558af57b0620 con 0x558af5a6c7f0 2024-05-15T14:00:36.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.680+0000 7f8793fff640 1 --2- 172.21.15.33:0/2881259298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558af5a6c7f0 0x558af5a7b7d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f87840126d0 tx=0x7f87840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46bd3d6e49245af4 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:36.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.680+0000 7f87737fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:00:36.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.680+0000 7f87737fe640 1 -- 172.21.15.33:0/2881259298 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878401f8f0 con 0x558af5a6c7f0 2024-05-15T14:00:36.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.680+0000 7f87737fe640 1 -- 172.21.15.33:0/2881259298 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f878401fa90 con 0x558af5a6c7f0 2024-05-15T14:00:36.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.680+0000 7f879d60e740 1 -- 172.21.15.33:0/2881259298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f878c004050 con 0x558af5a6c7f0 2024-05-15T14:00:36.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.680+0000 7f87737fe640 1 -- 172.21.15.33:0/2881259298 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f878401fcb0 con 0x558af5a6c7f0 2024-05-15T14:00:36.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.680+0000 7f879d60e740 10 client.15273 fetch_fsmap learned FSMap version 24 2024-05-15T14:00:36.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.680+0000 7f879d60e740 1 -- 172.21.15.33:0/2881259298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558af5a8a650 con 0x558af5a6c7f0 2024-05-15T14:00:36.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.680+0000 7f87737fe640 1 -- 172.21.15.33:0/2881259298 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f878c004050 con 0x558af5a6c7f0 2024-05-15T14:00:36.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.680+0000 7f87737fe640 1 -- 172.21.15.33:0/2881259298 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f878401d3b0 con 0x558af5a6c7f0 2024-05-15T14:00:36.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.680+0000 7f879d60e740 10 client.15273 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:00:36.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.680+0000 7f879d60e740 10 client.15273 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:00:36.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.680+0000 7f879d60e740 20 client.15273 populate_metadata read hostname 'smithi033' 2024-05-15T14:00:36.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.680+0000 7f879d60e740 1 --2- 172.21.15.33:0/2881259298 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558af5a8bb10 0x558af5a8df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:36.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.680+0000 7f879d60e740 4 client.15273 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:00:36.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.680+0000 7f879d60e740 1 -- 172.21.15.33:0/2881259298 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558af59e1b80 con 0x558af5a8bb10 2024-05-15T14:00:36.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.681+0000 7f87937fe640 1 --2- 172.21.15.33:0/2881259298 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558af5a8bb10 0x558af5a8df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:36.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.681+0000 7f87937fe640 1 --2- 172.21.15.33:0/2881259298 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558af5a8bb10 0x558af5a8df00 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf82db5b7216a04b server_cookie=6924c5cfb5dc1093 in_seq=0 out_seq=0 2024-05-15T14:00:36.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.681+0000 7f87737fe640 10 client.15273 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:00:36.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.682+0000 7f87737fe640 1 -- 172.21.15.33:0/2881259298 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558af59e1b80 con 0x558af5a8bb10 2024-05-15T14:00:36.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.682+0000 7f87737fe640 10 client.15273 handle_command_reply: tid=0 2024-05-15T14:00:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.682+0000 7f879d60e740 1 client.15273 shutdown 2024-05-15T14:00:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.682+0000 7f879d60e740 1 -- 172.21.15.33:0/2881259298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558af5a6c7f0 msgr2=0x558af5a7b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.682+0000 7f879d60e740 1 --2- 172.21.15.33:0/2881259298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558af5a6c7f0 0x558af5a7b7d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f87840126d0 tx=0x7f87840046c0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.682+0000 7f879d60e740 1 -- 172.21.15.33:0/2881259298 shutdown_connections 2024-05-15T14:00:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.682+0000 7f879d60e740 1 --2- 172.21.15.33:0/2881259298 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558af5a8bb10 0x558af5a8df00 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.682+0000 7f879d60e740 1 --2- 172.21.15.33:0/2881259298 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558af5a7bee0 0x558af5a79000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.682+0000 7f879d60e740 1 --2- 172.21.15.33:0/2881259298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558af5a6d110 0x558af5a78ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.682+0000 7f87737fe640 0 client.15273 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:00:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.682+0000 7f879d60e740 1 --2- 172.21.15.33:0/2881259298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558af5a6c7f0 0x558af5a7b7d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.682+0000 7f879d60e740 1 -- 172.21.15.33:0/2881259298 >> 172.21.15.33:0/2881259298 conn(0x558af5a68e00 msgr2=0x558af5a6ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.682+0000 7f879d60e740 1 -- 172.21.15.33:0/2881259298 shutdown_connections 2024-05-15T14:00:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.682+0000 7f879d60e740 1 -- 172.21.15.33:0/2881259298 wait complete. 2024-05-15T14:00:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.682+0000 7f879d60e740 20 client.15273 trim_cache size 0 max 16384 2024-05-15T14:00:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.698+0000 7f879d60e740 1 Processor -- start 2024-05-15T14:00:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.698+0000 7f879d60e740 1 -- start start 2024-05-15T14:00:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.698+0000 7f879d60e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558af5a6c7f0 0x558af5a741d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.698+0000 7f879d60e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558af5a6d110 0x558af5a7b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.698+0000 7f879d60e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558af5a7bee0 0x558af5a67f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.698+0000 7f879d60e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f87941aed90 con 0x558af5a6c7f0 2024-05-15T14:00:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.698+0000 7f879d60e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f87941ac660 con 0x558af5a6d110 2024-05-15T14:00:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.698+0000 7f879d60e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f87941aa4b0 con 0x558af5a7bee0 2024-05-15T14:00:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.698+0000 7f8793fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558af5a6c7f0 0x558af5a741d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.698+0000 7f8793fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558af5a6c7f0 0x558af5a741d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43940/0 (socket says 172.21.15.33:43940) 2024-05-15T14:00:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.698+0000 7f8793fff640 1 -- 172.21.15.33:0/1463541496 learned_addr learned my addr 172.21.15.33:0/1463541496 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.698+0000 7f8798e95640 1 --2- 172.21.15.33:0/1463541496 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558af5a7bee0 0x558af5a67f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.698+0000 7f8793fff640 1 -- 172.21.15.33:0/1463541496 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558af5a7bee0 msgr2=0x558af5a67f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.698+0000 7f8793fff640 1 --2- 172.21.15.33:0/1463541496 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558af5a7bee0 0x558af5a67f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.698+0000 7f8793fff640 1 -- 172.21.15.33:0/1463541496 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558af5a6d110 msgr2=0x558af5a7b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.698+0000 7f87937fe640 1 --2- 172.21.15.33:0/1463541496 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558af5a6d110 0x558af5a7b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.698+0000 7f8793fff640 1 --2- 172.21.15.33:0/1463541496 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558af5a6d110 0x558af5a7b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.698+0000 7f8793fff640 1 -- 172.21.15.33:0/1463541496 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558af5a6d650 con 0x558af5a6c7f0 2024-05-15T14:00:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.699+0000 7f8793fff640 1 --2- 172.21.15.33:0/1463541496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558af5a6c7f0 0x558af5a741d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8784000c00 tx=0x7f878400bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa468eb3536d9497 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.699+0000 7f8779ffb640 1 -- 172.21.15.33:0/1463541496 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8784034070 con 0x558af5a6c7f0 2024-05-15T14:00:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.699+0000 7f8779ffb640 1 -- 172.21.15.33:0/1463541496 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f878403e070 con 0x558af5a6c7f0 2024-05-15T14:00:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.699+0000 7f8779ffb640 1 -- 172.21.15.33:0/1463541496 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87840396a0 con 0x558af5a6c7f0 2024-05-15T14:00:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.699+0000 7f879d60e740 1 -- 172.21.15.33:0/1463541496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558af5a6c7f0 msgr2=0x558af5a741d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.699+0000 7f879d60e740 1 --2- 172.21.15.33:0/1463541496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558af5a6c7f0 0x558af5a741d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8784000c00 tx=0x7f878400bf70 comp rx=0 tx=0).stop 2024-05-15T14:00:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.700+0000 7f879d60e740 1 -- 172.21.15.33:0/1463541496 shutdown_connections 2024-05-15T14:00:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.700+0000 7f879d60e740 1 --2- 172.21.15.33:0/1463541496 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558af5a7bee0 0x558af5a67f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.700+0000 7f879d60e740 1 --2- 172.21.15.33:0/1463541496 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558af5a6d110 0x558af5a7b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.700+0000 7f879d60e740 1 --2- 172.21.15.33:0/1463541496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558af5a6c7f0 0x558af5a741d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.700+0000 7f879d60e740 1 -- 172.21.15.33:0/1463541496 >> 172.21.15.33:0/1463541496 conn(0x558af5a68e00 msgr2=0x558af5a67870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.700+0000 7f879d60e740 1 -- 172.21.15.33:0/1463541496 shutdown_connections 2024-05-15T14:00:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.700+0000 7f879d60e740 1 -- 172.21.15.33:0/1463541496 wait complete. 2024-05-15T14:00:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.700+0000 7f879d60e740 1 -- start start 2024-05-15T14:00:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.700+0000 7f879d60e740 1 Processor -- start 2024-05-15T14:00:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.700+0000 7f879d60e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558af5a6c7f0 0x558af5a75c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:36.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.700+0000 7f879d60e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558af5a6d110 0x558af5a761a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.700+0000 7f879d60e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558af5a7bee0 0x558af5af5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.700+0000 7f879d60e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f87941aab20 con 0x558af5a6c7f0 2024-05-15T14:00:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.700+0000 7f879d60e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f87941abf90 con 0x558af5a7bee0 2024-05-15T14:00:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.700+0000 7f879d60e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f87941ab1f0 con 0x558af5a6d110 2024-05-15T14:00:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.701+0000 7f879d60e740 10 client.0 _ll_register_callbacks cb 0x7f878400be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:00:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.701+0000 7f8793fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558af5a6c7f0 0x558af5a75c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.701+0000 7f8793fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558af5a6c7f0 0x558af5a75c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43944/0 (socket says 172.21.15.33:43944) 2024-05-15T14:00:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.701+0000 7f8793fff640 1 -- 172.21.15.33:0/2260025589 learned_addr learned my addr 172.21.15.33:0/2260025589 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.701+0000 7f8798e95640 1 --2- 172.21.15.33:0/2260025589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558af5a7bee0 0x558af5af5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.701+0000 7f87937fe640 1 --2- 172.21.15.33:0/2260025589 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558af5a6d110 0x558af5a761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.701+0000 7f8793fff640 1 -- 172.21.15.33:0/2260025589 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558af5a6d110 msgr2=0x558af5a761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.701+0000 7f8793fff640 1 --2- 172.21.15.33:0/2260025589 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558af5a6d110 0x558af5a761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.701+0000 7f8793fff640 1 -- 172.21.15.33:0/2260025589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558af5a7bee0 msgr2=0x558af5af5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.701+0000 7f8793fff640 1 --2- 172.21.15.33:0/2260025589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558af5a7bee0 0x558af5af5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.701+0000 7f8793fff640 1 -- 172.21.15.33:0/2260025589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558af5a68c30 con 0x558af5a6c7f0 2024-05-15T14:00:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.701+0000 7f87937fe640 1 --2- 172.21.15.33:0/2260025589 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558af5a6d110 0x558af5a761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:00:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.701+0000 7f8798e95640 1 --2- 172.21.15.33:0/2260025589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558af5a7bee0 0x558af5af5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:00:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.701+0000 7f8793fff640 1 --2- 172.21.15.33:0/2260025589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558af5a6c7f0 0x558af5a75c60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f87840126d0 tx=0x7f8784012f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de39e2825ba7b5a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.701+0000 7f87737fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:00:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.701+0000 7f87737fe640 1 -- 172.21.15.33:0/2260025589 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8784034040 con 0x558af5a6c7f0 2024-05-15T14:00:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.701+0000 7f879d60e740 1 -- 172.21.15.33:0/2260025589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f878403e070 con 0x558af5a6c7f0 2024-05-15T14:00:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.701+0000 7f87737fe640 1 -- 172.21.15.33:0/2260025589 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f878401f900 con 0x558af5a6c7f0 2024-05-15T14:00:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.701+0000 7f87737fe640 1 -- 172.21.15.33:0/2260025589 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8784051050 con 0x558af5a6c7f0 2024-05-15T14:00:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.702+0000 7f879d60e740 10 client.15279 fetch_fsmap learned FSMap version 24 2024-05-15T14:00:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.702+0000 7f879d60e740 1 -- 172.21.15.33:0/2260025589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558af5af67d0 con 0x558af5a6c7f0 2024-05-15T14:00:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.702+0000 7f87737fe640 1 -- 172.21.15.33:0/2260025589 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f878403e070 con 0x558af5a6c7f0 2024-05-15T14:00:36.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.702+0000 7f87737fe640 1 -- 172.21.15.33:0/2260025589 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f8784039440 con 0x558af5a6c7f0 2024-05-15T14:00:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.702+0000 7f879d60e740 10 client.15279 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:00:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.702+0000 7f879d60e740 10 client.15279 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:00:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.702+0000 7f879d60e740 20 client.15279 populate_metadata read hostname 'smithi033' 2024-05-15T14:00:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.702+0000 7f879d60e740 1 --2- 172.21.15.33:0/2260025589 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558af5af6e00 0x558af5af9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.702+0000 7f879d60e740 4 client.15279 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:00:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.702+0000 7f879d60e740 1 -- 172.21.15.33:0/2260025589 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558af5a79390 con 0x558af5af6e00 2024-05-15T14:00:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.702+0000 7f87937fe640 1 --2- 172.21.15.33:0/2260025589 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558af5af6e00 0x558af5af9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.703+0000 7f87737fe640 10 client.15279 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:00:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.703+0000 7f87937fe640 1 --2- 172.21.15.33:0/2260025589 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558af5af6e00 0x558af5af9220 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=301834a1c9eed176 server_cookie=8da53be9ff6e6c7a in_seq=0 out_seq=0 2024-05-15T14:00:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.715+0000 7f87737fe640 1 -- 172.21.15.33:0/2260025589 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558af5a79390 con 0x558af5af6e00 2024-05-15T14:00:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.715+0000 7f87737fe640 10 client.15279 handle_command_reply: tid=0 2024-05-15T14:00:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.715+0000 7f879d60e740 1 client.15279 shutdown 2024-05-15T14:00:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.715+0000 7f879d60e740 1 -- 172.21.15.33:0/2260025589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558af5a6c7f0 msgr2=0x558af5a75c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.715+0000 7f879d60e740 1 --2- 172.21.15.33:0/2260025589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558af5a6c7f0 0x558af5a75c60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f87840126d0 tx=0x7f8784012f30 comp rx=0 tx=0).stop 2024-05-15T14:00:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.715+0000 7f879d60e740 1 -- 172.21.15.33:0/2260025589 shutdown_connections 2024-05-15T14:00:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.715+0000 7f879d60e740 1 --2- 172.21.15.33:0/2260025589 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558af5af6e00 0x558af5af9220 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.715+0000 7f879d60e740 1 --2- 172.21.15.33:0/2260025589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558af5a7bee0 0x558af5af5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.715+0000 7f87737fe640 0 client.15279 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:00:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.715+0000 7f879d60e740 1 --2- 172.21.15.33:0/2260025589 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558af5a6d110 0x558af5a761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.715+0000 7f879d60e740 1 --2- 172.21.15.33:0/2260025589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558af5a6c7f0 0x558af5a75c60 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.715+0000 7f879d60e740 1 -- 172.21.15.33:0/2260025589 >> 172.21.15.33:0/2260025589 conn(0x558af5a68e00 msgr2=0x558af5a68100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.715+0000 7f879d60e740 1 -- 172.21.15.33:0/2260025589 shutdown_connections 2024-05-15T14:00:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.715+0000 7f879d60e740 1 -- 172.21.15.33:0/2260025589 wait complete. 2024-05-15T14:00:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.716+0000 7f879d60e740 20 client.15279 trim_cache size 0 max 16384 2024-05-15T14:00:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:00:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:00:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:00:36.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.716+0000 7f879a8f8640 1 -- 172.21.15.33:0/1298193473 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f877c0b29e0 msgr2=0x7f877c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:36.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.716+0000 7f879a8f8640 1 --2- 172.21.15.33:0/1298193473 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f877c0b29e0 0x7f877c0b4ea0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8784005d00 tx=0x7f8784005c50 comp rx=0 tx=0).stop 2024-05-15T14:00:36.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.716+0000 7f879a8f8640 1 -- 172.21.15.33:0/1298193473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87941045e0 msgr2=0x7f8794199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:36.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.716+0000 7f879a8f8640 1 --2- 172.21.15.33:0/1298193473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87941045e0 0x7f8794199790 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8788005940 tx=0x7f8788004270 comp rx=0 tx=0).stop 2024-05-15T14:00:36.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.716+0000 7f879a8f8640 1 -- 172.21.15.33:0/1298193473 shutdown_connections 2024-05-15T14:00:36.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.716+0000 7f879a8f8640 1 --2- 172.21.15.33:0/1298193473 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f877c0b29e0 0x7f877c0b4ea0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.716+0000 7f879a8f8640 1 --2- 172.21.15.33:0/1298193473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f879410d090 0x7f879419db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.716+0000 7f879a8f8640 1 --2- 172.21.15.33:0/1298193473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87941045e0 0x7f8794199790 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.716+0000 7f879a8f8640 1 --2- 172.21.15.33:0/1298193473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8794103cc0 0x7f8794199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.716+0000 7f879a8f8640 1 -- 172.21.15.33:0/1298193473 >> 172.21.15.33:0/1298193473 conn(0x7f8794078a70 msgr2=0x7f87940feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:36.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.716+0000 7f879a8f8640 1 -- 172.21.15.33:0/1298193473 shutdown_connections 2024-05-15T14:00:36.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:36.716+0000 7f879a8f8640 1 -- 172.21.15.33:0/1298193473 wait complete. 2024-05-15T14:00:36.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:36 smithi052 ceph-mon[36190]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 42 op/s 2024-05-15T14:00:36.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:36 smithi081 ceph-mon[36434]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 42 op/s 2024-05-15T14:00:36.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.862+0000 7f0ac157f700 1 -- 172.21.15.33:0/469027023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0abc1084f0 msgr2=0x7f0abc1088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:36.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.862+0000 7f0ac157f700 1 --2- 172.21.15.33:0/469027023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0abc1084f0 0x7f0abc1088d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0aac009990 tx=0x7f0aac009ca0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.863+0000 7f0ac157f700 1 -- 172.21.15.33:0/469027023 shutdown_connections 2024-05-15T14:00:36.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.863+0000 7f0ac157f700 1 --2- 172.21.15.33:0/469027023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0abc102e20 0x7f0abc10f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.863+0000 7f0ac157f700 1 --2- 172.21.15.33:0/469027023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0abc1084f0 0x7f0abc1088d0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.863+0000 7f0ac157f700 1 --2- 172.21.15.33:0/469027023 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0abc102460 0x7f0abc1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.863+0000 7f0ac157f700 1 -- 172.21.15.33:0/469027023 >> 172.21.15.33:0/469027023 conn(0x7f0abc0794e0 msgr2=0x7f0abc0798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:36.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.863+0000 7f0ac157f700 1 -- 172.21.15.33:0/469027023 shutdown_connections 2024-05-15T14:00:36.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.863+0000 7f0ac157f700 1 -- 172.21.15.33:0/469027023 wait complete. 2024-05-15T14:00:36.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.863+0000 7f0ac157f700 1 Processor -- start 2024-05-15T14:00:36.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.864+0000 7f0ac157f700 1 -- start start 2024-05-15T14:00:36.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.864+0000 7f0ac157f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0abc102460 0x7f0abc19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.864+0000 7f0ac157f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0abc102e20 0x7f0abc19af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.864+0000 7f0ac157f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0abc1084f0 0x7f0abc1a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.864+0000 7f0ac157f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc112620 con 0x7f0abc1084f0 2024-05-15T14:00:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.864+0000 7f0ac157f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc1124a0 con 0x7f0abc102e20 2024-05-15T14:00:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.864+0000 7f0ac157f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc1127a0 con 0x7f0abc102460 2024-05-15T14:00:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.864+0000 7f0abb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0abc1084f0 0x7f0abc1a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.864+0000 7f0abaffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0abc102460 0x7f0abc19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.864+0000 7f0abb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0abc1084f0 0x7f0abc1a5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43962/0 (socket says 172.21.15.33:43962) 2024-05-15T14:00:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.864+0000 7f0abb7fe700 1 -- 172.21.15.33:0/2740272974 learned_addr learned my addr 172.21.15.33:0/2740272974 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.864+0000 7f0aba7fc700 1 --2- 172.21.15.33:0/2740272974 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0abc102e20 0x7f0abc19af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.864+0000 7f0abb7fe700 1 -- 172.21.15.33:0/2740272974 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0abc102460 msgr2=0x7f0abc19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.864+0000 7f0abb7fe700 1 --2- 172.21.15.33:0/2740272974 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0abc102460 0x7f0abc19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.864+0000 7f0abb7fe700 1 -- 172.21.15.33:0/2740272974 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0abc102e20 msgr2=0x7f0abc19af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:36.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.864+0000 7f0abb7fe700 1 --2- 172.21.15.33:0/2740272974 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0abc102e20 0x7f0abc19af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:36.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.865+0000 7f0abb7fe700 1 -- 172.21.15.33:0/2740272974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0abc198830 con 0x7f0abc1084f0 2024-05-15T14:00:36.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.865+0000 7f0abb7fe700 1 --2- 172.21.15.33:0/2740272974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0abc1084f0 0x7f0abc1a5ac0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4007d70 tx=0x7f0ab400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:36.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.865+0000 7f0aa3fff700 1 -- 172.21.15.33:0/2740272974 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4016020 con 0x7f0abc1084f0 2024-05-15T14:00:36.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.865+0000 7f0aa3fff700 1 -- 172.21.15.33:0/2740272974 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ab400ac90 con 0x7f0abc1084f0 2024-05-15T14:00:36.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.865+0000 7f0ac157f700 1 -- 172.21.15.33:0/2740272974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0abc198b20 con 0x7f0abc1084f0 2024-05-15T14:00:36.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.865+0000 7f0aa3fff700 1 -- 172.21.15.33:0/2740272974 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab40115e0 con 0x7f0abc1084f0 2024-05-15T14:00:36.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.865+0000 7f0ac157f700 1 -- 172.21.15.33:0/2740272974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0abc199010 con 0x7f0abc1084f0 2024-05-15T14:00:36.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.866+0000 7f0ac157f700 1 -- 172.21.15.33:0/2740272974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0abc06a230 con 0x7f0abc1084f0 2024-05-15T14:00:36.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.868+0000 7f0aa3fff700 1 -- 172.21.15.33:0/2740272974 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f0ab4019020 con 0x7f0abc1084f0 2024-05-15T14:00:36.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.869+0000 7f0aa3fff700 1 --2- 172.21.15.33:0/2740272974 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0aa40b2a80 0x7f0aa40b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:36.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.869+0000 7f0abaffd700 1 --2- 172.21.15.33:0/2740272974 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0aa40b2a80 0x7f0aa40b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:36.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.869+0000 7f0aa3fff700 1 -- 172.21.15.33:0/2740272974 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f0ab40e2b00 con 0x7f0abc1084f0 2024-05-15T14:00:36.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.870+0000 7f0abaffd700 1 --2- 172.21.15.33:0/2740272974 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0aa40b2a80 0x7f0aa40b4f40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0aac00b2e0 tx=0x7f0aac005960 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:36.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:36.873+0000 7f0aa3fff700 1 -- 172.21.15.33:0/2740272974 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0ab4020020 con 0x7f0abc1084f0 2024-05-15T14:00:37.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:37.132+0000 7f0ac157f700 1 -- 172.21.15.33:0/2740272974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0abc19a500 con 0x7f0abc1084f0 2024-05-15T14:00:37.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:37.133+0000 7f0aa3fff700 1 -- 172.21.15.33:0/2740272974 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11858 (secure 0 0 0) 0x7f0ab40aa430 con 0x7f0abc1084f0 2024-05-15T14:00:37.136 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T14:00:37.136 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2030117633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2030117633},{"type":"v1","addr":"172.21.15.33:6839","nonce":2030117633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1918979061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1918979061},{"type":"v1","addr":"172.21.15.33:6841","nonce":1918979061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T13:57:19.056417+0000","modified":"2024-05-15T14:00:18.543619+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24463,"mds_2":24461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/1629697841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1629697841},{"type":"v1","addr":"172.21.15.52:6837","nonce":1629697841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/3773839758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3773839758},{"type":"v1","addr":"172.21.15.81:6835","nonce":3773839758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.52:6839/2696044471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2696044471},{"type":"v1","addr":"172.21.15.52:6839","nonce":2696044471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.81:6839/1554395940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":1554395940},{"type":"v1","addr":"172.21.15.81:6839","nonce":1554395940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24463,24478,24461]},"id":1}]} 2024-05-15T14:00:37.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:37.135+0000 7f0ac157f700 1 -- 172.21.15.33:0/2740272974 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0aa40b2a80 msgr2=0x7f0aa40b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:37.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:37.135+0000 7f0ac157f700 1 --2- 172.21.15.33:0/2740272974 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0aa40b2a80 0x7f0aa40b4f40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0aac00b2e0 tx=0x7f0aac005960 comp rx=0 tx=0).stop 2024-05-15T14:00:37.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:37.135+0000 7f0ac157f700 1 -- 172.21.15.33:0/2740272974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0abc1084f0 msgr2=0x7f0abc1a5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:37.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:37.135+0000 7f0ac157f700 1 --2- 172.21.15.33:0/2740272974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0abc1084f0 0x7f0abc1a5ac0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4007d70 tx=0x7f0ab400a340 comp rx=0 tx=0).stop 2024-05-15T14:00:37.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:37.135+0000 7f0ac157f700 1 -- 172.21.15.33:0/2740272974 shutdown_connections 2024-05-15T14:00:37.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:37.135+0000 7f0ac157f700 1 --2- 172.21.15.33:0/2740272974 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0aa40b2a80 0x7f0aa40b4f40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:37.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:37.135+0000 7f0ac157f700 1 --2- 172.21.15.33:0/2740272974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0abc1084f0 0x7f0abc1a5ac0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:37.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:37.135+0000 7f0ac157f700 1 --2- 172.21.15.33:0/2740272974 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0abc102460 0x7f0abc19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:37.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:37.135+0000 7f0ac157f700 1 --2- 172.21.15.33:0/2740272974 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0abc102e20 0x7f0abc19af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:37.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:37.136+0000 7f0ac157f700 1 -- 172.21.15.33:0/2740272974 >> 172.21.15.33:0/2740272974 conn(0x7f0abc0794e0 msgr2=0x7f0abc1050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:37.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:37.136+0000 7f0ac157f700 1 -- 172.21.15.33:0/2740272974 shutdown_connections 2024-05-15T14:00:37.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:37.136+0000 7f0ac157f700 1 -- 172.21.15.33:0/2740272974 wait complete. 2024-05-15T14:00:37.139 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 24 2024-05-15T14:00:37.688 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:37 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/2740272974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:00:37.717 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-15T14:00:37.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:37 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/2740272974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:00:37.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:37 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/2740272974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:00:37.969 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:00:38.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:38 smithi033 ceph-mon[31650]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-05-15T14:00:38.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:38 smithi052 ceph-mon[36190]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-05-15T14:00:38.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:38 smithi081 ceph-mon[36434]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-05-15T14:00:39.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.629+0000 7f9c9de0f700 1 -- 172.21.15.33:0/2255558222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c98102930 msgr2=0x7f9c98110430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:39.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.629+0000 7f9c9de0f700 1 --2- 172.21.15.33:0/2255558222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c98102930 0x7f9c98110430 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000b0a0 tx=0x7f9c9000b3b0 comp rx=0 tx=0).stop 2024-05-15T14:00:39.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.629+0000 7f9c9de0f700 1 -- 172.21.15.33:0/2255558222 shutdown_connections 2024-05-15T14:00:39.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.629+0000 7f9c9de0f700 1 --2- 172.21.15.33:0/2255558222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c98102930 0x7f9c98110430 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:39.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.629+0000 7f9c9de0f700 1 --2- 172.21.15.33:0/2255558222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c98109720 0x7f9c98109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:39.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.629+0000 7f9c9de0f700 1 --2- 172.21.15.33:0/2255558222 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c98101f70 0x7f9c981023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:39.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.629+0000 7f9c9de0f700 1 -- 172.21.15.33:0/2255558222 >> 172.21.15.33:0/2255558222 conn(0x7f9c980776b0 msgr2=0x7f9c98077ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:39.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.629+0000 7f9c9de0f700 1 -- 172.21.15.33:0/2255558222 shutdown_connections 2024-05-15T14:00:39.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.629+0000 7f9c9de0f700 1 -- 172.21.15.33:0/2255558222 wait complete. 2024-05-15T14:00:39.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.630+0000 7f9c9de0f700 1 Processor -- start 2024-05-15T14:00:39.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.630+0000 7f9c9de0f700 1 -- start start 2024-05-15T14:00:39.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.630+0000 7f9c9de0f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c98101f70 0x7f9c9819f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:39.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.630+0000 7f9c9de0f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c98102930 0x7f9c9819fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:39.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.630+0000 7f9c9de0f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c98109720 0x7f9c98199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:39.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.630+0000 7f9c9de0f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9c98113750 con 0x7f9c98101f70 2024-05-15T14:00:39.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.630+0000 7f9c9de0f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9c981135d0 con 0x7f9c98109720 2024-05-15T14:00:39.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.630+0000 7f9c9de0f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9c981138d0 con 0x7f9c98102930 2024-05-15T14:00:39.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.630+0000 7f9c977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c98101f70 0x7f9c9819f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:39.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.630+0000 7f9c977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c98101f70 0x7f9c9819f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43966/0 (socket says 172.21.15.33:43966) 2024-05-15T14:00:39.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.630+0000 7f9c977fe700 1 -- 172.21.15.33:0/1663717924 learned_addr learned my addr 172.21.15.33:0/1663717924 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:39.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.630+0000 7f9c96ffd700 1 --2- 172.21.15.33:0/1663717924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c98102930 0x7f9c9819fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:39.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.630+0000 7f9c97fff700 1 --2- 172.21.15.33:0/1663717924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c98109720 0x7f9c98199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:39.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.631+0000 7f9c977fe700 1 -- 172.21.15.33:0/1663717924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c98102930 msgr2=0x7f9c9819fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:39.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.631+0000 7f9c977fe700 1 --2- 172.21.15.33:0/1663717924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c98102930 0x7f9c9819fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:39.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.631+0000 7f9c977fe700 1 -- 172.21.15.33:0/1663717924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c98109720 msgr2=0x7f9c98199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:39.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.631+0000 7f9c977fe700 1 --2- 172.21.15.33:0/1663717924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c98109720 0x7f9c98199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:39.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.631+0000 7f9c977fe700 1 -- 172.21.15.33:0/1663717924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c9819a0d0 con 0x7f9c98101f70 2024-05-15T14:00:39.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.631+0000 7f9c97fff700 1 --2- 172.21.15.33:0/1663717924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c98109720 0x7f9c98199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:00:39.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.631+0000 7f9c96ffd700 1 --2- 172.21.15.33:0/1663717924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c98102930 0x7f9c9819fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:00:39.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.631+0000 7f9c977fe700 1 --2- 172.21.15.33:0/1663717924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c98101f70 0x7f9c9819f7b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800d980 tx=0x7f9c8800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:39.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.631+0000 7f9c94ff9700 1 -- 172.21.15.33:0/1663717924 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c88017070 con 0x7f9c98101f70 2024-05-15T14:00:39.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.631+0000 7f9c9de0f700 1 -- 172.21.15.33:0/1663717924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c9819a3c0 con 0x7f9c98101f70 2024-05-15T14:00:39.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.631+0000 7f9c94ff9700 1 -- 172.21.15.33:0/1663717924 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c88004830 con 0x7f9c98101f70 2024-05-15T14:00:39.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.631+0000 7f9c94ff9700 1 -- 172.21.15.33:0/1663717924 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c88005440 con 0x7f9c98101f70 2024-05-15T14:00:39.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.631+0000 7f9c9de0f700 1 -- 172.21.15.33:0/1663717924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c981a6680 con 0x7f9c98101f70 2024-05-15T14:00:39.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.632+0000 7f9c9de0f700 1 -- 172.21.15.33:0/1663717924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c9806b710 con 0x7f9c98101f70 2024-05-15T14:00:39.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.633+0000 7f9c94ff9700 1 -- 172.21.15.33:0/1663717924 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f9c880055e0 con 0x7f9c98101f70 2024-05-15T14:00:39.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.634+0000 7f9c94ff9700 1 --2- 172.21.15.33:0/1663717924 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f9c800b2a80 0x7f9c800b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:39.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.634+0000 7f9c96ffd700 1 --2- 172.21.15.33:0/1663717924 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f9c800b2a80 0x7f9c800b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:39.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.634+0000 7f9c94ff9700 1 -- 172.21.15.33:0/1663717924 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f9c880e26a0 con 0x7f9c98101f70 2024-05-15T14:00:39.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.635+0000 7f9c96ffd700 1 --2- 172.21.15.33:0/1663717924 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f9c800b2a80 0x7f9c800b4f40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c005fd0 tx=0x7f9c8c005f60 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:39.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.639+0000 7f9c94ff9700 1 -- 172.21.15.33:0/1663717924 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9c88020020 con 0x7f9c98101f70 2024-05-15T14:00:39.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.855+0000 7f9c9de0f700 1 -- 172.21.15.33:0/1663717924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c9802e560 con 0x7f9c98101f70 2024-05-15T14:00:39.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.857+0000 7f9c94ff9700 1 -- 172.21.15.33:0/1663717924 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22971 (secure 0 0 0) 0x7f9c880a9f20 con 0x7f9c98101f70 2024-05-15T14:00:39.859 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T14:00:39.859 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":83,"fsid":"bca78ca4-12c1-11ef-bc98-c7b262605968","created":"2024-05-15T13:49:10.327613+0000","modified":"2024-05-15T14:00:21.157321+0000","last_up_change":"2024-05-15T13:56:16.097711+0000","last_in_change":"2024-05-15T13:55:56.722143+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-15T13:52:32.970844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-15T13:57:12.324802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-15T13:57:15.898406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b434111-810f-450a-b63d-c97080e9f8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6803","nonce":887409130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6805","nonce":887409130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6809","nonce":887409130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6807","nonce":887409130}]},"public_addr":"172.21.15.33:6803/887409130","cluster_addr":"172.21.15.33:6805/887409130","heartbeat_back_addr":"172.21.15.33:6809/887409130","heartbeat_front_addr":"172.21.15.33:6807/887409130","state":["exists","up"]},{"osd":1,"uuid":"0b578d4e-a841-4b27-bc95-d6c3bae580fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6801","nonce":2794180003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6803","nonce":2794180003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6807","nonce":2794180003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6805","nonce":2794180003}]},"public_addr":"172.21.15.52:6801/2794180003","cluster_addr":"172.21.15.52:6803/2794180003","heartbeat_back_addr":"172.21.15.52:6807/2794180003","heartbeat_front_addr":"172.21.15.52:6805/2794180003","state":["exists","up"]},{"osd":2,"uuid":"0e9c9390-ed33-464b-8f39-bca75941e05e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6801","nonce":315610978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6803","nonce":315610978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6807","nonce":315610978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6805","nonce":315610978}]},"public_addr":"172.21.15.81:6801/315610978","cluster_addr":"172.21.15.81:6803/315610978","heartbeat_back_addr":"172.21.15.81:6807/315610978","heartbeat_front_addr":"172.21.15.81:6805/315610978","state":["exists","up"]},{"osd":3,"uuid":"ecdb24c1-fb15-42a5-9bae-436bbae9512a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6811","nonce":818533501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6813","nonce":818533501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6817","nonce":818533501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6815","nonce":818533501}]},"public_addr":"172.21.15.33:6811/818533501","cluster_addr":"172.21.15.33:6813/818533501","heartbeat_back_addr":"172.21.15.33:6817/818533501","heartbeat_front_addr":"172.21.15.33:6815/818533501","state":["exists","up"]},{"osd":4,"uuid":"706752a0-b17f-4563-a7a9-d5078bf0f3b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6809","nonce":3010133776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6811","nonce":3010133776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6815","nonce":3010133776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6813","nonce":3010133776}]},"public_addr":"172.21.15.52:6809/3010133776","cluster_addr":"172.21.15.52:6811/3010133776","heartbeat_back_addr":"172.21.15.52:6815/3010133776","heartbeat_front_addr":"172.21.15.52:6813/3010133776","state":["exists","up"]},{"osd":5,"uuid":"3d45ab97-f364-4903-974d-8acc961363d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6809","nonce":2132616297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6811","nonce":2132616297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6815","nonce":2132616297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6813","nonce":2132616297}]},"public_addr":"172.21.15.81:6809/2132616297","cluster_addr":"172.21.15.81:6811/2132616297","heartbeat_back_addr":"172.21.15.81:6815/2132616297","heartbeat_front_addr":"172.21.15.81:6813/2132616297","state":["exists","up"]},{"osd":6,"uuid":"3be524c5-dcf9-4742-b896-ea9a44a6b391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6819","nonce":1260691801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6821","nonce":1260691801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6825","nonce":1260691801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6823","nonce":1260691801}]},"public_addr":"172.21.15.33:6819/1260691801","cluster_addr":"172.21.15.33:6821/1260691801","heartbeat_back_addr":"172.21.15.33:6825/1260691801","heartbeat_front_addr":"172.21.15.33:6823/1260691801","state":["exists","up"]},{"osd":7,"uuid":"bda2f920-4f2c-4eed-8127-83c1973676e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6817","nonce":1654595813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6819","nonce":1654595813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6823","nonce":1654595813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6821","nonce":1654595813}]},"public_addr":"172.21.15.52:6817/1654595813","cluster_addr":"172.21.15.52:6819/1654595813","heartbeat_back_addr":"172.21.15.52:6823/1654595813","heartbeat_front_addr":"172.21.15.52:6821/1654595813","state":["exists","up"]},{"osd":8,"uuid":"550585b8-0df4-49ee-95c8-3d084744dbd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6817","nonce":3009661748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6819","nonce":3009661748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6823","nonce":3009661748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6821","nonce":3009661748}]},"public_addr":"172.21.15.81:6817/3009661748","cluster_addr":"172.21.15.81:6819/3009661748","heartbeat_back_addr":"172.21.15.81:6823/3009661748","heartbeat_front_addr":"172.21.15.81:6821/3009661748","state":["exists","up"]},{"osd":9,"uuid":"1d781846-26f3-4b23-b22c-c994a8509a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6827","nonce":2843220858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6829","nonce":2843220858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6833","nonce":2843220858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6831","nonce":2843220858}]},"public_addr":"172.21.15.33:6827/2843220858","cluster_addr":"172.21.15.33:6829/2843220858","heartbeat_back_addr":"172.21.15.33:6833/2843220858","heartbeat_front_addr":"172.21.15.33:6831/2843220858","state":["exists","up"]},{"osd":10,"uuid":"af58c940-fdc4-418b-a638-7dc44438b34c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6825","nonce":2422349831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6827","nonce":2422349831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6831","nonce":2422349831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6829","nonce":2422349831}]},"public_addr":"172.21.15.52:6825/2422349831","cluster_addr":"172.21.15.52:6827/2422349831","heartbeat_back_addr":"172.21.15.52:6831/2422349831","heartbeat_front_addr":"172.21.15.52:6829/2422349831","state":["exists","up"]},{"osd":11,"uuid":"ab84aaea-2086-4842-8bf4-628842fce46a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6825","nonce":326313090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6827","nonce":326313090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6831","nonce":326313090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6829","nonce":326313090}]},"public_addr":"172.21.15.81:6825/326313090","cluster_addr":"172.21.15.81:6827/326313090","heartbeat_back_addr":"172.21.15.81:6831/326313090","heartbeat_front_addr":"172.21.15.81:6829/326313090","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:51:41.493136+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:52:05.784773+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:52:29.769429+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:52:54.891620+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:53:19.412922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:53:43.921841+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:54:09.486732+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:54:34.064370+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:54:58.655304+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:55:24.548693+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:55:49.941191+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:56:14.919558+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1564238110":"2024-05-16T14:00:21.157290+0000","172.21.15.33:6800/1797827152":"2024-05-16T14:00:21.157290+0000","172.21.15.33:0/436121571":"2024-05-16T13:50:04.881656+0000","172.21.15.33:0/2647407310":"2024-05-16T14:00:21.157290+0000","172.21.15.33:0/2326878092":"2024-05-16T14:00:21.157290+0000","172.21.15.33:6801/2680375465":"2024-05-16T13:50:04.881656+0000","172.21.15.33:0/2259371407":"2024-05-16T14:00:21.157290+0000","172.21.15.33:6800/2680375465":"2024-05-16T13:50:04.881656+0000","172.21.15.33:0/1022787179":"2024-05-16T13:49:37.564643+0000","172.21.15.33:0/3720711369":"2024-05-16T13:50:04.881656+0000","172.21.15.33:0/1282466838":"2024-05-16T13:49:37.564643+0000","172.21.15.33:0/3022248969":"2024-05-16T13:49:37.564643+0000","172.21.15.33:6801/3303495364":"2024-05-16T13:49:37.564643+0000","172.21.15.33:6801/1797827152":"2024-05-16T14:00:21.157290+0000","172.21.15.33:6800/3303495364":"2024-05-16T13:49:37.564643+0000","172.21.15.33:0/4282812692":"2024-05-16T13:50:04.881656+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-15T14:00:39.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.859+0000 7f9c9de0f700 1 -- 172.21.15.33:0/1663717924 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f9c800b2a80 msgr2=0x7f9c800b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:39.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.859+0000 7f9c9de0f700 1 --2- 172.21.15.33:0/1663717924 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f9c800b2a80 0x7f9c800b4f40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c005fd0 tx=0x7f9c8c005f60 comp rx=0 tx=0).stop 2024-05-15T14:00:39.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.859+0000 7f9c9de0f700 1 -- 172.21.15.33:0/1663717924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c98101f70 msgr2=0x7f9c9819f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:39.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.859+0000 7f9c9de0f700 1 --2- 172.21.15.33:0/1663717924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c98101f70 0x7f9c9819f7b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800d980 tx=0x7f9c8800dd40 comp rx=0 tx=0).stop 2024-05-15T14:00:39.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.859+0000 7f9c9de0f700 1 -- 172.21.15.33:0/1663717924 shutdown_connections 2024-05-15T14:00:39.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.859+0000 7f9c9de0f700 1 --2- 172.21.15.33:0/1663717924 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f9c800b2a80 0x7f9c800b4f40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:39.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.859+0000 7f9c9de0f700 1 --2- 172.21.15.33:0/1663717924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9c98109720 0x7f9c98199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:39.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.859+0000 7f9c9de0f700 1 --2- 172.21.15.33:0/1663717924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c98101f70 0x7f9c9819f7b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:39.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.859+0000 7f9c9de0f700 1 --2- 172.21.15.33:0/1663717924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c98102930 0x7f9c9819fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:39.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.859+0000 7f9c9de0f700 1 -- 172.21.15.33:0/1663717924 >> 172.21.15.33:0/1663717924 conn(0x7f9c980776b0 msgr2=0x7f9c98100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:39.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.859+0000 7f9c9de0f700 1 -- 172.21.15.33:0/1663717924 shutdown_connections 2024-05-15T14:00:39.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:39.859+0000 7f9c9de0f700 1 -- 172.21.15.33:0/1663717924 wait complete. 2024-05-15T14:00:40.445 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-05-15T14:00:40.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:40 smithi033 ceph-mon[31650]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 41 KiB/s wr, 22 op/s 2024-05-15T14:00:40.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:40 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/1663717924' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T14:00:40.699 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:00:40.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:40 smithi052 ceph-mon[36190]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 41 KiB/s wr, 22 op/s 2024-05-15T14:00:40.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:40 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/1663717924' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T14:00:40.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:40 smithi081 ceph-mon[36434]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 41 KiB/s wr, 22 op/s 2024-05-15T14:00:40.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:40 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/1663717924' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T14:00:42.643 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:42 smithi033 ceph-mon[31650]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-15T14:00:42.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:42 smithi052 ceph-mon[36190]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-15T14:00:42.913 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:42 smithi081 ceph-mon[36434]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-15T14:00:42.914 INFO:teuthology.orchestra.run.smithi033.stdout:{"run_dir":"/var/run/ceph"} 2024-05-15T14:00:42.914 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-05-15T14:00:42.917 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 413 2024-05-15T14:00:42.917 DEBUG:tasks.quiescer:Yielding 2024-05-15T14:00:42.918 INFO:teuthology.run_tasks:Running task workunit... 2024-05-15T14:00:42.932 INFO:tasks.workunit:Pulling workunits from ref 9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-15T14:00:42.932 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-15T14:00:42.932 INFO:tasks.workunit:timeout=3h 2024-05-15T14:00:42.932 INFO:tasks.workunit:cleanup=True 2024-05-15T14:00:42.933 DEBUG:teuthology.orchestra.run.smithi033:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-15T14:00:42.933 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T14:00:42.951 INFO:teuthology.orchestra.run.smithi033.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-15T14:00:42.951 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-15T14:00:42.951 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-05-15T14:00:42.951 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-15T14:00:42.951 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-05-15T14:00:42.951 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-05-15 13:57:50.045583496 +0000 2024-05-15T14:00:42.952 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-05-15 13:57:50.045583496 +0000 2024-05-15T14:00:42.952 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-05-15 14:00:17.830118757 +0000 2024-05-15T14:00:42.952 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-05-15 13:57:50.045583496 +0000 2024-05-15T14:00:42.952 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-15T14:00:42.952 DEBUG:teuthology.orchestra.run.smithi033:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-15T14:00:43.028 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 9b63866860f0b71216b2c0a73aedd2e0a8a5c98b 2024-05-15T14:00:43.086 INFO:tasks.workunit.client.0.smithi033.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-15T14:00:43.192 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:00:44.529 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:44 smithi033 ceph-mon[31650]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-05-15T14:00:44.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.790+0000 7f0a313aa700 1 -- 172.21.15.33:0/2558879880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a2c1095b0 msgr2=0x7f0a2c109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:44.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.790+0000 7f0a313aa700 1 --2- 172.21.15.33:0/2558879880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a2c1095b0 0x7f0a2c109990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c00b0a0 tx=0x7f0a1c00b3b0 comp rx=0 tx=0).stop 2024-05-15T14:00:44.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.790+0000 7f0a313aa700 1 -- 172.21.15.33:0/2558879880 shutdown_connections 2024-05-15T14:00:44.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.790+0000 7f0a313aa700 1 --2- 172.21.15.33:0/2558879880 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a2c06bbe0 0x7f0a2c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:44.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.790+0000 7f0a313aa700 1 --2- 172.21.15.33:0/2558879880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a2c1095b0 0x7f0a2c109990 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:44.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.790+0000 7f0a313aa700 1 --2- 172.21.15.33:0/2558879880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a2c06b220 0x7f0a2c06b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:44.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.790+0000 7f0a313aa700 1 -- 172.21.15.33:0/2558879880 >> 172.21.15.33:0/2558879880 conn(0x7f0a2c0793f0 msgr2=0x7f0a2c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:44.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.790+0000 7f0a313aa700 1 -- 172.21.15.33:0/2558879880 shutdown_connections 2024-05-15T14:00:44.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.790+0000 7f0a313aa700 1 -- 172.21.15.33:0/2558879880 wait complete. 2024-05-15T14:00:44.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.791+0000 7f0a313aa700 1 Processor -- start 2024-05-15T14:00:44.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.791+0000 7f0a313aa700 1 -- start start 2024-05-15T14:00:44.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.791+0000 7f0a313aa700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a2c06b220 0x7f0a2c102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:44.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.792+0000 7f0a313aa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a2c06bbe0 0x7f0a2c102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:44.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.792+0000 7f0a313aa700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a2c1095b0 0x7f0a2c102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:44.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.792+0000 7f0a313aa700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2c1136e0 con 0x7f0a2c1095b0 2024-05-15T14:00:44.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.792+0000 7f0a313aa700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2c113560 con 0x7f0a2c06bbe0 2024-05-15T14:00:44.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.792+0000 7f0a313aa700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2c113860 con 0x7f0a2c06b220 2024-05-15T14:00:44.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.792+0000 7f0a2b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a2c1095b0 0x7f0a2c102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:44.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.792+0000 7f0a2b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a2c1095b0 0x7f0a2c102f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58598/0 (socket says 172.21.15.33:58598) 2024-05-15T14:00:44.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.792+0000 7f0a2b7fe700 1 -- 172.21.15.33:0/2353862854 learned_addr learned my addr 172.21.15.33:0/2353862854 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:44.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.792+0000 7f0a2affd700 1 --2- 172.21.15.33:0/2353862854 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a2c06b220 0x7f0a2c102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:44.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.792+0000 7f0a2a7fc700 1 --2- 172.21.15.33:0/2353862854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a2c06bbe0 0x7f0a2c102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:44.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.792+0000 7f0a2b7fe700 1 -- 172.21.15.33:0/2353862854 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a2c06b220 msgr2=0x7f0a2c102510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:44.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.792+0000 7f0a2b7fe700 1 --2- 172.21.15.33:0/2353862854 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a2c06b220 0x7f0a2c102510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:44.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.792+0000 7f0a2b7fe700 1 -- 172.21.15.33:0/2353862854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a2c06bbe0 msgr2=0x7f0a2c102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:44.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.792+0000 7f0a2b7fe700 1 --2- 172.21.15.33:0/2353862854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a2c06bbe0 0x7f0a2c102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:44.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.792+0000 7f0a2b7fe700 1 -- 172.21.15.33:0/2353862854 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a2c1a81c0 con 0x7f0a2c1095b0 2024-05-15T14:00:44.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.792+0000 7f0a2b7fe700 1 --2- 172.21.15.33:0/2353862854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a2c1095b0 0x7f0a2c102f90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0a2400d980 tx=0x7f0a2400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:44.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.792+0000 7f0a13fff700 1 -- 172.21.15.33:0/2353862854 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a24017070 con 0x7f0a2c1095b0 2024-05-15T14:00:44.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.792+0000 7f0a13fff700 1 -- 172.21.15.33:0/2353862854 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a24005020 con 0x7f0a2c1095b0 2024-05-15T14:00:44.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.792+0000 7f0a313aa700 1 -- 172.21.15.33:0/2353862854 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a2c1a84b0 con 0x7f0a2c1095b0 2024-05-15T14:00:44.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.793+0000 7f0a13fff700 1 -- 172.21.15.33:0/2353862854 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a240099b0 con 0x7f0a2c1095b0 2024-05-15T14:00:44.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.793+0000 7f0a313aa700 1 -- 172.21.15.33:0/2353862854 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a2c1a89f0 con 0x7f0a2c1095b0 2024-05-15T14:00:44.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.794+0000 7f0a313aa700 1 -- 172.21.15.33:0/2353862854 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a2c10ba90 con 0x7f0a2c1095b0 2024-05-15T14:00:44.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.795+0000 7f0a13fff700 1 -- 172.21.15.33:0/2353862854 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f0a24009b50 con 0x7f0a2c1095b0 2024-05-15T14:00:44.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.796+0000 7f0a13fff700 1 --2- 172.21.15.33:0/2353862854 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0a140b2b00 0x7f0a140b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:44.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.796+0000 7f0a2affd700 1 --2- 172.21.15.33:0/2353862854 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0a140b2b00 0x7f0a140b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:44.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.796+0000 7f0a13fff700 1 -- 172.21.15.33:0/2353862854 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f0a240e29d0 con 0x7f0a2c1095b0 2024-05-15T14:00:44.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.796+0000 7f0a2affd700 1 --2- 172.21.15.33:0/2353862854 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0a140b2b00 0x7f0a140b4fc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c009e60 tx=0x7f0a1c00b7b0 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:44.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:44.799+0000 7f0a13fff700 1 -- 172.21.15.33:0/2353862854 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0a24020020 con 0x7f0a2c1095b0 2024-05-15T14:00:44.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:44 smithi052 ceph-mon[36190]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-05-15T14:00:44.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:44 smithi081 ceph-mon[36434]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-05-15T14:00:45.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:45.066+0000 7f0a313aa700 1 -- 172.21.15.33:0/2353862854 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a2c103a40 con 0x7f0a2c1095b0 2024-05-15T14:00:45.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:45.067+0000 7f0a13fff700 1 -- 172.21.15.33:0/2353862854 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11858 (secure 0 0 0) 0x7f0a240aa2d0 con 0x7f0a2c1095b0 2024-05-15T14:00:45.069 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T14:00:45.069 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2030117633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2030117633},{"type":"v1","addr":"172.21.15.33:6839","nonce":2030117633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1918979061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1918979061},{"type":"v1","addr":"172.21.15.33:6841","nonce":1918979061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T13:57:19.056417+0000","modified":"2024-05-15T14:00:18.543619+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24463,"mds_2":24461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/1629697841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1629697841},{"type":"v1","addr":"172.21.15.52:6837","nonce":1629697841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/3773839758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3773839758},{"type":"v1","addr":"172.21.15.81:6835","nonce":3773839758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.52:6839/2696044471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2696044471},{"type":"v1","addr":"172.21.15.52:6839","nonce":2696044471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.81:6839/1554395940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":1554395940},{"type":"v1","addr":"172.21.15.81:6839","nonce":1554395940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24463,24478,24461]},"id":1}]} 2024-05-15T14:00:45.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:45.069+0000 7f0a313aa700 1 -- 172.21.15.33:0/2353862854 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0a140b2b00 msgr2=0x7f0a140b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:45.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:45.069+0000 7f0a313aa700 1 --2- 172.21.15.33:0/2353862854 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0a140b2b00 0x7f0a140b4fc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c009e60 tx=0x7f0a1c00b7b0 comp rx=0 tx=0).stop 2024-05-15T14:00:45.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:45.069+0000 7f0a313aa700 1 -- 172.21.15.33:0/2353862854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a2c1095b0 msgr2=0x7f0a2c102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:45.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:45.069+0000 7f0a313aa700 1 --2- 172.21.15.33:0/2353862854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a2c1095b0 0x7f0a2c102f90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0a2400d980 tx=0x7f0a2400dc90 comp rx=0 tx=0).stop 2024-05-15T14:00:45.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:45.069+0000 7f0a313aa700 1 -- 172.21.15.33:0/2353862854 shutdown_connections 2024-05-15T14:00:45.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:45.069+0000 7f0a313aa700 1 --2- 172.21.15.33:0/2353862854 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0a140b2b00 0x7f0a140b4fc0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:45.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:45.070+0000 7f0a313aa700 1 --2- 172.21.15.33:0/2353862854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a2c1095b0 0x7f0a2c102f90 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:45.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:45.070+0000 7f0a313aa700 1 --2- 172.21.15.33:0/2353862854 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0a2c06b220 0x7f0a2c102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:45.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:45.070+0000 7f0a313aa700 1 --2- 172.21.15.33:0/2353862854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a2c06bbe0 0x7f0a2c102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:45.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:45.070+0000 7f0a313aa700 1 -- 172.21.15.33:0/2353862854 >> 172.21.15.33:0/2353862854 conn(0x7f0a2c0793f0 msgr2=0x7f0a2c101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:45.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:45.070+0000 7f0a313aa700 1 -- 172.21.15.33:0/2353862854 shutdown_connections 2024-05-15T14:00:45.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:45.070+0000 7f0a313aa700 1 -- 172.21.15.33:0/2353862854 wait complete. 2024-05-15T14:00:45.075 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 24 2024-05-15T14:00:45.641 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:45 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/2353862854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:00:45.671 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-15T13:57:19.056417+0000', 'modified': '2024-05-15T14:00:18.543619+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24478, 'mds_1': 24463, 'mds_2': 24461}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24424': {'gid': 24424, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.81:6833/1872861431', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6832', 'nonce': 1872861431}, {'type': 'v1', 'addr': '172.21.15.81:6833', 'nonce': 1872861431}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.52:6837/1629697841', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6836', 'nonce': 1629697841}, {'type': 'v1', 'addr': '172.21.15.52:6837', 'nonce': 1629697841}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.81:6835/3773839758', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6834', 'nonce': 3773839758}, {'type': 'v1', 'addr': '172.21.15.81:6835', 'nonce': 3773839758}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24461': {'gid': 24461, 'name': 'b', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.52:6839/2696044471', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6838', 'nonce': 2696044471}, {'type': 'v1', 'addr': '172.21.15.52:6839', 'nonce': 2696044471}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'c', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.81:6837/744100239', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6836', 'nonce': 744100239}, {'type': 'v1', 'addr': '172.21.15.81:6837', 'nonce': 744100239}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.81:6839/1554395940', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6838', 'nonce': 1554395940}, {'type': 'v1', 'addr': '172.21.15.81:6839', 'nonce': 1554395940}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24478, 'qdb_cluster': [24463, 24478, 24461]} 2024-05-15T14:00:45.671 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-15T14:00:45.671 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph tell mds.b status 2024-05-15T14:00:45.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:45 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/2353862854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:00:45.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:45 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/2353862854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:00:45.964 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:00:46.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:46 smithi033 ceph-mon[31650]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-15T14:00:46.741 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:46 smithi052 ceph-mon[36190]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-15T14:00:46.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:46 smithi081 ceph-mon[36434]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-15T14:00:47.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.548+0000 7f72c7e38700 1 -- 172.21.15.33:0/703301386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72c0101f00 msgr2=0x7f72c0102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:47.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.548+0000 7f72c7e38700 1 --2- 172.21.15.33:0/703301386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72c0101f00 0x7f72c0102380 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f72b0009990 tx=0x7f72b0009ca0 comp rx=0 tx=0).stop 2024-05-15T14:00:47.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.548+0000 7f72c7e38700 1 -- 172.21.15.33:0/703301386 shutdown_connections 2024-05-15T14:00:47.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.548+0000 7f72c7e38700 1 --2- 172.21.15.33:0/703301386 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72c01028c0 0x7f72c0110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:47.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.548+0000 7f72c7e38700 1 --2- 172.21.15.33:0/703301386 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f72c0108620 0x7f72c0108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:47.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.548+0000 7f72c7e38700 1 --2- 172.21.15.33:0/703301386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72c0101f00 0x7f72c0102380 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:47.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.548+0000 7f72c7e38700 1 -- 172.21.15.33:0/703301386 >> 172.21.15.33:0/703301386 conn(0x7f72c0077780 msgr2=0x7f72c0077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:47.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.548+0000 7f72c7e38700 1 -- 172.21.15.33:0/703301386 shutdown_connections 2024-05-15T14:00:47.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.548+0000 7f72c7e38700 1 -- 172.21.15.33:0/703301386 wait complete. 2024-05-15T14:00:47.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.548+0000 7f72c7e38700 1 Processor -- start 2024-05-15T14:00:47.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.549+0000 7f72c7e38700 1 -- start start 2024-05-15T14:00:47.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.549+0000 7f72c7e38700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72c0101f00 0x7f72c01030e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:47.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.549+0000 7f72c7e38700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f72c01028c0 0x7f72c0103620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:47.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.549+0000 7f72c7e38700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72c0108620 0x7f72c0103b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:47.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.549+0000 7f72c7e38700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f72c01137b0 con 0x7f72c0108620 2024-05-15T14:00:47.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.549+0000 7f72c7e38700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f72c0113630 con 0x7f72c0101f00 2024-05-15T14:00:47.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.549+0000 7f72c7e38700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f72c0113930 con 0x7f72c01028c0 2024-05-15T14:00:47.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.549+0000 7f72c53d3700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f72c01028c0 0x7f72c0103620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:47.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.549+0000 7f72c63d5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72c0108620 0x7f72c0103b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:47.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.549+0000 7f72c53d3700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f72c01028c0 0x7f72c0103620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:39442/0 (socket says 172.21.15.33:39442) 2024-05-15T14:00:47.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.549+0000 7f72c53d3700 1 -- 172.21.15.33:0/2771899403 learned_addr learned my addr 172.21.15.33:0/2771899403 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:47.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.549+0000 7f72c5bd4700 1 --2- 172.21.15.33:0/2771899403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72c0101f00 0x7f72c01030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:47.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.549+0000 7f72c63d5700 1 -- 172.21.15.33:0/2771899403 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f72c01028c0 msgr2=0x7f72c0103620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:47.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.549+0000 7f72c63d5700 1 --2- 172.21.15.33:0/2771899403 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f72c01028c0 0x7f72c0103620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:47.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.549+0000 7f72c63d5700 1 -- 172.21.15.33:0/2771899403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72c0101f00 msgr2=0x7f72c01030e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:47.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.549+0000 7f72c63d5700 1 --2- 172.21.15.33:0/2771899403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72c0101f00 0x7f72c01030e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:47.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.550+0000 7f72c63d5700 1 -- 172.21.15.33:0/2771899403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72c01a8230 con 0x7f72c0108620 2024-05-15T14:00:47.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.550+0000 7f72c63d5700 1 --2- 172.21.15.33:0/2771899403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72c0108620 0x7f72c0103b60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f72b800be20 tx=0x7f72b8009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:47.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.550+0000 7f72b6ffd700 1 -- 172.21.15.33:0/2771899403 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b8019070 con 0x7f72c0108620 2024-05-15T14:00:47.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.550+0000 7f72b6ffd700 1 -- 172.21.15.33:0/2771899403 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72b800cb90 con 0x7f72c0108620 2024-05-15T14:00:47.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.550+0000 7f72b6ffd700 1 -- 172.21.15.33:0/2771899403 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b80125a0 con 0x7f72c0108620 2024-05-15T14:00:47.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.550+0000 7f72c7e38700 1 -- 172.21.15.33:0/2771899403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72c01a8520 con 0x7f72c0108620 2024-05-15T14:00:47.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.550+0000 7f72c7e38700 1 -- 172.21.15.33:0/2771899403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72c01a8a60 con 0x7f72c0108620 2024-05-15T14:00:47.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.553+0000 7f72b6ffd700 1 -- 172.21.15.33:0/2771899403 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f72b801c070 con 0x7f72c0108620 2024-05-15T14:00:47.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.554+0000 7f72b6ffd700 1 --2- 172.21.15.33:0/2771899403 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f72ac0b2b00 0x7f72ac0b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:47.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.554+0000 7f72c5bd4700 1 --2- 172.21.15.33:0/2771899403 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f72ac0b2b00 0x7f72ac0b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:47.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.554+0000 7f72b6ffd700 1 -- 172.21.15.33:0/2771899403 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f72b80e1bc0 con 0x7f72c0108620 2024-05-15T14:00:47.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.554+0000 7f72c5bd4700 1 --2- 172.21.15.33:0/2771899403 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f72ac0b2b00 0x7f72ac0b4fc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f72c0078ce0 tx=0x7f72bc00b410 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:47.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.554+0000 7f72d26ceb80 1 Processor -- start 2024-05-15T14:00:47.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.555+0000 7f72d26ceb80 1 -- start start 2024-05-15T14:00:47.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.555+0000 7f72d26ceb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbbc8a42f0 0x55fbbc8a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:47.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.555+0000 7f72d26ceb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbbc8a4c10 0x55fbbc8a83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:47.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.555+0000 7f72d26ceb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fbbc8a8910 0x55fbbc8a8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:47.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.555+0000 7f72d26ceb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fbbc802c40 con 0x55fbbc8a42f0 2024-05-15T14:00:47.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.555+0000 7f72d26ceb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fbbc7f27e0 con 0x55fbbc8a8910 2024-05-15T14:00:47.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.555+0000 7f72d26ceb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55fbbc7f6820 con 0x55fbbc8a4c10 2024-05-15T14:00:47.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.555+0000 7f72c5bd4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbbc8a42f0 0x55fbbc8a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:47.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.555+0000 7f72c5bd4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbbc8a42f0 0x55fbbc8a46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58628/0 (socket says 172.21.15.33:58628) 2024-05-15T14:00:47.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.555+0000 7f72c5bd4700 1 -- 172.21.15.33:0/3117089189 learned_addr learned my addr 172.21.15.33:0/3117089189 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:47.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.555+0000 7f72c53d3700 1 --2- 172.21.15.33:0/3117089189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbbc8a4c10 0x55fbbc8a83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:47.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.555+0000 7f72c63d5700 1 --2- 172.21.15.33:0/3117089189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fbbc8a8910 0x55fbbc8a8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:47.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.555+0000 7f72c5bd4700 1 -- 172.21.15.33:0/3117089189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbbc8a4c10 msgr2=0x55fbbc8a83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:47.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.555+0000 7f72c5bd4700 1 --2- 172.21.15.33:0/3117089189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbbc8a4c10 0x55fbbc8a83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:47.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.555+0000 7f72c5bd4700 1 -- 172.21.15.33:0/3117089189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fbbc8a8910 msgr2=0x55fbbc8a8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:47.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.555+0000 7f72c5bd4700 1 --2- 172.21.15.33:0/3117089189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fbbc8a8910 0x55fbbc8a8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:47.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.555+0000 7f72c5bd4700 1 -- 172.21.15.33:0/3117089189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbbc742880 con 0x55fbbc8a42f0 2024-05-15T14:00:47.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.555+0000 7f72c5bd4700 1 --2- 172.21.15.33:0/3117089189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbbc8a42f0 0x55fbbc8a46d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f72bc010fa0 tx=0x7f72bc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=723a7cc472c25bb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:47.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.556+0000 7f72abfff700 1 -- 172.21.15.33:0/3117089189 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc011db0 con 0x55fbbc8a42f0 2024-05-15T14:00:47.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.556+0000 7f72abfff700 1 -- 172.21.15.33:0/3117089189 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72bc00be40 con 0x55fbbc8a42f0 2024-05-15T14:00:47.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.556+0000 7f72abfff700 1 -- 172.21.15.33:0/3117089189 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc01c710 con 0x55fbbc8a42f0 2024-05-15T14:00:47.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.556+0000 7f72d26ceb80 1 -- 172.21.15.33:0/3117089189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbbc8a42f0 msgr2=0x55fbbc8a46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:47.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.556+0000 7f72d26ceb80 1 --2- 172.21.15.33:0/3117089189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbbc8a42f0 0x55fbbc8a46d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f72bc010fa0 tx=0x7f72bc013000 comp rx=0 tx=0).stop 2024-05-15T14:00:47.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.557+0000 7f72d26ceb80 1 -- 172.21.15.33:0/3117089189 shutdown_connections 2024-05-15T14:00:47.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.557+0000 7f72d26ceb80 1 --2- 172.21.15.33:0/3117089189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fbbc8a8910 0x55fbbc8a8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:47.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.557+0000 7f72d26ceb80 1 --2- 172.21.15.33:0/3117089189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbbc8a42f0 0x55fbbc8a46d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:47.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.557+0000 7f72d26ceb80 1 --2- 172.21.15.33:0/3117089189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbbc8a4c10 0x55fbbc8a83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:47.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.557+0000 7f72d26ceb80 1 -- 172.21.15.33:0/3117089189 >> 172.21.15.33:0/3117089189 conn(0x55fbbc7cf200 msgr2=0x55fbbc89e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:47.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.557+0000 7f72d26ceb80 1 -- 172.21.15.33:0/3117089189 shutdown_connections 2024-05-15T14:00:47.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.557+0000 7f72d26ceb80 1 -- 172.21.15.33:0/3117089189 wait complete. 2024-05-15T14:00:47.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.557+0000 7f72d26ceb80 1 -- start start 2024-05-15T14:00:47.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.557+0000 7f72d26ceb80 1 Processor -- start 2024-05-15T14:00:47.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.558+0000 7f72d26ceb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbbc8a42f0 0x55fbbc8afd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:47.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.558+0000 7f72d26ceb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fbbc8a4c10 0x55fbbc8ad120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:47.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.558+0000 7f72d26ceb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbbc8a8910 0x55fbbc8ad660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:47.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.558+0000 7f72d26ceb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fbbc6f8f50 con 0x55fbbc8a8910 2024-05-15T14:00:47.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.558+0000 7f72d26ceb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fbbc794770 con 0x55fbbc8a4c10 2024-05-15T14:00:47.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.558+0000 7f72d26ceb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55fbbc8b0b70 con 0x55fbbc8a42f0 2024-05-15T14:00:47.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.558+0000 7f72d26ceb80 10 client.0 _ll_register_callbacks cb 0x55fbbc789530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:00:47.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.558+0000 7f72c63d5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbbc8a8910 0x55fbbc8ad660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:47.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.558+0000 7f72c63d5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbbc8a8910 0x55fbbc8ad660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58640/0 (socket says 172.21.15.33:58640) 2024-05-15T14:00:47.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.558+0000 7f72c63d5700 1 -- 172.21.15.33:0/2977540443 learned_addr learned my addr 172.21.15.33:0/2977540443 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:47.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.558+0000 7f72c53d3700 1 --2- 172.21.15.33:0/2977540443 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fbbc8a4c10 0x55fbbc8ad120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:47.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.558+0000 7f72c5bd4700 1 --2- 172.21.15.33:0/2977540443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbbc8a42f0 0x55fbbc8afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:47.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.558+0000 7f72c63d5700 1 -- 172.21.15.33:0/2977540443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbbc8a42f0 msgr2=0x55fbbc8afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:47.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.558+0000 7f72c63d5700 1 --2- 172.21.15.33:0/2977540443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbbc8a42f0 0x55fbbc8afd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:47.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.558+0000 7f72c63d5700 1 -- 172.21.15.33:0/2977540443 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fbbc8a4c10 msgr2=0x55fbbc8ad120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:47.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.558+0000 7f72c63d5700 1 --2- 172.21.15.33:0/2977540443 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fbbc8a4c10 0x55fbbc8ad120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:47.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.558+0000 7f72c63d5700 1 -- 172.21.15.33:0/2977540443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbbc791640 con 0x55fbbc8a8910 2024-05-15T14:00:47.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.558+0000 7f72c5bd4700 1 --2- 172.21.15.33:0/2977540443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbbc8a42f0 0x55fbbc8afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:00:47.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.558+0000 7f72c63d5700 1 --2- 172.21.15.33:0/2977540443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbbc8a8910 0x55fbbc8ad660 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f72b80a60c0 tx=0x7f72b80df810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7379bc282c7a9a41 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:47.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.558+0000 7f72a97fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:00:47.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.558+0000 7f72a97fa700 1 -- 172.21.15.33:0/2977540443 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b80ad200 con 0x55fbbc8a8910 2024-05-15T14:00:47.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.558+0000 7f72a97fa700 1 -- 172.21.15.33:0/2977540443 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f72b80b1a30 con 0x55fbbc8a8910 2024-05-15T14:00:47.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.558+0000 7f72d26ceb80 1 -- 172.21.15.33:0/2977540443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fbbc870310 con 0x55fbbc8a8910 2024-05-15T14:00:47.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.559+0000 7f72a97fa700 1 -- 172.21.15.33:0/2977540443 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f72b80b1cb0 con 0x55fbbc8a8910 2024-05-15T14:00:47.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.559+0000 7f72d26ceb80 10 client.15309 fetch_fsmap learned FSMap version 24 2024-05-15T14:00:47.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.559+0000 7f72d26ceb80 1 -- 172.21.15.33:0/2977540443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fbbc8bb290 con 0x55fbbc8a8910 2024-05-15T14:00:47.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.559+0000 7f72a97fa700 1 -- 172.21.15.33:0/2977540443 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55fbbc870310 con 0x55fbbc8a8910 2024-05-15T14:00:47.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.559+0000 7f72a97fa700 1 -- 172.21.15.33:0/2977540443 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f72b80a7d20 con 0x55fbbc8a8910 2024-05-15T14:00:47.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.559+0000 7f72d26ceb80 10 client.15309 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:00:47.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.559+0000 7f72d26ceb80 10 client.15309 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-15T14:00:47.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.559+0000 7f72d26ceb80 20 client.15309 populate_metadata read hostname 'smithi033' 2024-05-15T14:00:47.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.559+0000 7f72d26ceb80 1 --2- 172.21.15.33:0/2977540443 >> [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] conn(0x55fbbc8bd830 0x55fbbc8beca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:47.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.559+0000 7f72d26ceb80 4 client.15309 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:00:47.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.559+0000 7f72d26ceb80 1 -- 172.21.15.33:0/2977540443 --> [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbbc804b90 con 0x55fbbc8bd830 2024-05-15T14:00:47.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.559+0000 7f72c5bd4700 1 --2- 172.21.15.33:0/2977540443 >> [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] conn(0x55fbbc8bd830 0x55fbbc8beca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:47.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.560+0000 7f72c5bd4700 1 --2- 172.21.15.33:0/2977540443 >> [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] conn(0x55fbbc8bd830 0x55fbbc8beca0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=47fa40215d28bcf7 server_cookie=534aa919c81e8b8d in_seq=0 out_seq=0 2024-05-15T14:00:47.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.560+0000 7f72a97fa700 10 client.15309 ms_handle_connect on v2:172.21.15.52:6838/2696044471 2024-05-15T14:00:47.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.560+0000 7f72a97fa700 1 -- 172.21.15.33:0/2977540443 <== mds.2 v2:172.21.15.52:6838/2696044471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fbbc804b90 con 0x55fbbc8bd830 2024-05-15T14:00:47.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.560+0000 7f72a97fa700 10 client.15309 handle_command_reply: tid=0 2024-05-15T14:00:47.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.560+0000 7f72d26ceb80 1 client.15309 shutdown 2024-05-15T14:00:47.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.561+0000 7f72d26ceb80 1 -- 172.21.15.33:0/2977540443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbbc8a8910 msgr2=0x55fbbc8ad660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:47.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.561+0000 7f72d26ceb80 1 --2- 172.21.15.33:0/2977540443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbbc8a8910 0x55fbbc8ad660 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f72b80a60c0 tx=0x7f72b80df810 comp rx=0 tx=0).stop 2024-05-15T14:00:47.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.561+0000 7f72d26ceb80 1 -- 172.21.15.33:0/2977540443 shutdown_connections 2024-05-15T14:00:47.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.561+0000 7f72d26ceb80 1 --2- 172.21.15.33:0/2977540443 >> [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] conn(0x55fbbc8bd830 0x55fbbc8beca0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.561+0000 7f72d26ceb80 1 --2- 172.21.15.33:0/2977540443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbbc8a8910 0x55fbbc8ad660 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.561+0000 7f72d26ceb80 1 --2- 172.21.15.33:0/2977540443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbbc8a42f0 0x55fbbc8afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.561+0000 7f72d26ceb80 1 --2- 172.21.15.33:0/2977540443 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fbbc8a4c10 0x55fbbc8ad120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.561+0000 7f72a97fa700 0 client.15309 ms_handle_reset on v2:172.21.15.52:6838/2696044471 2024-05-15T14:00:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.561+0000 7f72d26ceb80 1 -- 172.21.15.33:0/2977540443 >> 172.21.15.33:0/2977540443 conn(0x55fbbc7cf200 msgr2=0x55fbbc8a9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.561+0000 7f72d26ceb80 1 -- 172.21.15.33:0/2977540443 shutdown_connections 2024-05-15T14:00:47.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.561+0000 7f72d26ceb80 1 -- 172.21.15.33:0/2977540443 wait complete. 2024-05-15T14:00:47.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.561+0000 7f72d26ceb80 20 client.15309 trim_cache size 0 max 16384 2024-05-15T14:00:47.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.582+0000 7f72d26ceb80 1 Processor -- start 2024-05-15T14:00:47.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.582+0000 7f72d26ceb80 1 -- start start 2024-05-15T14:00:47.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.582+0000 7f72d26ceb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbbc8a42f0 0x55fbbc8afd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:47.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.582+0000 7f72d26ceb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fbbc8a4c10 0x55fbbc8ad690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:47.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.582+0000 7f72d26ceb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbbc8a8910 0x55fbbc8af5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:47.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.582+0000 7f72d26ceb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fbbc8bcfe0 con 0x55fbbc8a8910 2024-05-15T14:00:47.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.582+0000 7f72d26ceb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fbbc8bd6b0 con 0x55fbbc8a4c10 2024-05-15T14:00:47.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.582+0000 7f72d26ceb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55fbbc8bdd80 con 0x55fbbc8a42f0 2024-05-15T14:00:47.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.583+0000 7f72c63d5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbbc8a8910 0x55fbbc8af5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:47.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.583+0000 7f72c63d5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbbc8a8910 0x55fbbc8af5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58646/0 (socket says 172.21.15.33:58646) 2024-05-15T14:00:47.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.583+0000 7f72c63d5700 1 -- 172.21.15.33:0/3157012335 learned_addr learned my addr 172.21.15.33:0/3157012335 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:47.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.583+0000 7f72c5bd4700 1 --2- 172.21.15.33:0/3157012335 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbbc8a42f0 0x55fbbc8afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:47.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.583+0000 7f72c53d3700 1 --2- 172.21.15.33:0/3157012335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fbbc8a4c10 0x55fbbc8ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:47.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.583+0000 7f72c63d5700 1 -- 172.21.15.33:0/3157012335 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbbc8a42f0 msgr2=0x55fbbc8afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:47.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.583+0000 7f72c63d5700 1 --2- 172.21.15.33:0/3157012335 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbbc8a42f0 0x55fbbc8afd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:47.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.583+0000 7f72c63d5700 1 -- 172.21.15.33:0/3157012335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fbbc8a4c10 msgr2=0x55fbbc8ad690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:47.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.583+0000 7f72c63d5700 1 --2- 172.21.15.33:0/3157012335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fbbc8a4c10 0x55fbbc8ad690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:47.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.583+0000 7f72c63d5700 1 -- 172.21.15.33:0/3157012335 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbbc8ab690 con 0x55fbbc8a8910 2024-05-15T14:00:47.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.583+0000 7f72c53d3700 1 --2- 172.21.15.33:0/3157012335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fbbc8a4c10 0x55fbbc8ad690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:00:47.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.583+0000 7f72c63d5700 1 --2- 172.21.15.33:0/3157012335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbbc8a8910 0x55fbbc8af5c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f72b80e1eb0 tx=0x7f72b80ae6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81cf1be201250606 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:47.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.583+0000 7f72ab7fe700 1 -- 172.21.15.33:0/3157012335 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b80ad200 con 0x55fbbc8a8910 2024-05-15T14:00:47.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.583+0000 7f72ab7fe700 1 -- 172.21.15.33:0/3157012335 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72b80ab070 con 0x55fbbc8a8910 2024-05-15T14:00:47.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.583+0000 7f72ab7fe700 1 -- 172.21.15.33:0/3157012335 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b80a8c20 con 0x55fbbc8a8910 2024-05-15T14:00:47.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.584+0000 7f72d26ceb80 1 -- 172.21.15.33:0/3157012335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbbc8a8910 msgr2=0x55fbbc8af5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:47.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.584+0000 7f72d26ceb80 1 --2- 172.21.15.33:0/3157012335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbbc8a8910 0x55fbbc8af5c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f72b80e1eb0 tx=0x7f72b80ae6a0 comp rx=0 tx=0).stop 2024-05-15T14:00:47.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.584+0000 7f72d26ceb80 1 -- 172.21.15.33:0/3157012335 shutdown_connections 2024-05-15T14:00:47.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.584+0000 7f72d26ceb80 1 --2- 172.21.15.33:0/3157012335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbbc8a8910 0x55fbbc8af5c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:47.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.584+0000 7f72d26ceb80 1 --2- 172.21.15.33:0/3157012335 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbbc8a42f0 0x55fbbc8afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:47.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.584+0000 7f72d26ceb80 1 --2- 172.21.15.33:0/3157012335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fbbc8a4c10 0x55fbbc8ad690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:47.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.584+0000 7f72d26ceb80 1 -- 172.21.15.33:0/3157012335 >> 172.21.15.33:0/3157012335 conn(0x55fbbc7cf200 msgr2=0x55fbbc8a76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:47.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.584+0000 7f72d26ceb80 1 -- 172.21.15.33:0/3157012335 shutdown_connections 2024-05-15T14:00:47.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.584+0000 7f72d26ceb80 1 -- 172.21.15.33:0/3157012335 wait complete. 2024-05-15T14:00:47.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.584+0000 7f72d26ceb80 1 -- start start 2024-05-15T14:00:47.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.584+0000 7f72d26ceb80 1 Processor -- start 2024-05-15T14:00:47.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.585+0000 7f72d26ceb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fbbc8a42f0 0x55fbbc8c8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:47.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.585+0000 7f72d26ceb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbbc8a4c10 0x55fbbc8c30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:47.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.585+0000 7f72d26ceb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbbc8a8910 0x55fbbc8c3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:47.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.585+0000 7f72d26ceb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fbbc8beb20 con 0x55fbbc8a8910 2024-05-15T14:00:47.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.585+0000 7f72d26ceb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fbbc8be450 con 0x55fbbc8a42f0 2024-05-15T14:00:47.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.585+0000 7f72d26ceb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55fbbc8bf1f0 con 0x55fbbc8a4c10 2024-05-15T14:00:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.585+0000 7f72d26ceb80 10 client.0 _ll_register_callbacks cb 0x55fbbc789530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:00:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.585+0000 7f72c63d5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbbc8a8910 0x55fbbc8c3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.585+0000 7f72c63d5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbbc8a8910 0x55fbbc8c3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58662/0 (socket says 172.21.15.33:58662) 2024-05-15T14:00:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.585+0000 7f72c63d5700 1 -- 172.21.15.33:0/536249496 learned_addr learned my addr 172.21.15.33:0/536249496 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.585+0000 7f72c53d3700 1 --2- 172.21.15.33:0/536249496 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbbc8a4c10 0x55fbbc8c30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.585+0000 7f72c5bd4700 1 --2- 172.21.15.33:0/536249496 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fbbc8a42f0 0x55fbbc8c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.585+0000 7f72c63d5700 1 -- 172.21.15.33:0/536249496 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbbc8a4c10 msgr2=0x55fbbc8c30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.585+0000 7f72c63d5700 1 --2- 172.21.15.33:0/536249496 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbbc8a4c10 0x55fbbc8c30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.585+0000 7f72c63d5700 1 -- 172.21.15.33:0/536249496 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fbbc8a42f0 msgr2=0x55fbbc8c8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.585+0000 7f72c63d5700 1 --2- 172.21.15.33:0/536249496 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fbbc8a42f0 0x55fbbc8c8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.585+0000 7f72c63d5700 1 -- 172.21.15.33:0/536249496 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbbc928ca0 con 0x55fbbc8a8910 2024-05-15T14:00:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.585+0000 7f72c5bd4700 1 --2- 172.21.15.33:0/536249496 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fbbc8a42f0 0x55fbbc8c8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:00:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.586+0000 7f72c63d5700 1 --2- 172.21.15.33:0/536249496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbbc8a8910 0x55fbbc8c3750 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f72b80a8840 tx=0x7f72b80ae5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255c7bfa202d6de3 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.586+0000 7f72a97fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:00:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.586+0000 7f72a97fa700 1 -- 172.21.15.33:0/536249496 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b80a8c20 con 0x55fbbc8a8910 2024-05-15T14:00:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.586+0000 7f72a97fa700 1 -- 172.21.15.33:0/536249496 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f72b80a49d0 con 0x55fbbc8a8910 2024-05-15T14:00:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.586+0000 7f72d26ceb80 1 -- 172.21.15.33:0/536249496 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72b80ab070 con 0x55fbbc8a8910 2024-05-15T14:00:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.586+0000 7f72a97fa700 1 -- 172.21.15.33:0/536249496 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f72b80ad200 con 0x55fbbc8a8910 2024-05-15T14:00:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.586+0000 7f72d26ceb80 10 client.15315 fetch_fsmap learned FSMap version 24 2024-05-15T14:00:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.586+0000 7f72d26ceb80 1 -- 172.21.15.33:0/536249496 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fbbc929d90 con 0x55fbbc8a8910 2024-05-15T14:00:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.586+0000 7f72a97fa700 1 -- 172.21.15.33:0/536249496 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72b80ab070 con 0x55fbbc8a8910 2024-05-15T14:00:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.586+0000 7f72a97fa700 1 -- 172.21.15.33:0/536249496 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f72b80c30d0 con 0x55fbbc8a8910 2024-05-15T14:00:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.586+0000 7f72d26ceb80 10 client.15315 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:00:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.586+0000 7f72d26ceb80 10 client.15315 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-15T14:00:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.586+0000 7f72d26ceb80 20 client.15315 populate_metadata read hostname 'smithi033' 2024-05-15T14:00:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.586+0000 7f72d26ceb80 1 --2- 172.21.15.33:0/536249496 >> [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] conn(0x55fbbc92e4f0 0x55fbbc92d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.586+0000 7f72d26ceb80 4 client.15315 mds_command: new command op to 24461 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-15T14:00:47.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.586+0000 7f72d26ceb80 1 -- 172.21.15.33:0/536249496 --> [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] -- command(tid 0: {"prefix": "status"}) -- 0x55fbbc7fe370 con 0x55fbbc92e4f0 2024-05-15T14:00:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.587+0000 7f72c5bd4700 1 --2- 172.21.15.33:0/536249496 >> [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] conn(0x55fbbc92e4f0 0x55fbbc92d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.587+0000 7f72c5bd4700 1 --2- 172.21.15.33:0/536249496 >> [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] conn(0x55fbbc92e4f0 0x55fbbc92d200 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=159fdabf0d25500b server_cookie=e7bece6a20cba87d in_seq=0 out_seq=0 2024-05-15T14:00:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.587+0000 7f72a97fa700 10 client.15315 ms_handle_connect on v2:172.21.15.52:6838/2696044471 2024-05-15T14:00:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.587+0000 7f72a97fa700 1 -- 172.21.15.33:0/536249496 <== mds.2 v2:172.21.15.52:6838/2696044471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55fbbc7fe370 con 0x55fbbc92e4f0 2024-05-15T14:00:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.587+0000 7f72a97fa700 10 client.15315 handle_command_reply: tid=0 2024-05-15T14:00:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.587+0000 7f72d26ceb80 1 client.15315 shutdown 2024-05-15T14:00:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.588+0000 7f72d26ceb80 1 -- 172.21.15.33:0/536249496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbbc8a8910 msgr2=0x55fbbc8c3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.588+0000 7f72d26ceb80 1 --2- 172.21.15.33:0/536249496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbbc8a8910 0x55fbbc8c3750 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f72b80a8840 tx=0x7f72b80ae5e0 comp rx=0 tx=0).stop 2024-05-15T14:00:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.588+0000 7f72d26ceb80 1 -- 172.21.15.33:0/536249496 shutdown_connections 2024-05-15T14:00:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.588+0000 7f72d26ceb80 1 --2- 172.21.15.33:0/536249496 >> [v2:172.21.15.52:6838/2696044471,v1:172.21.15.52:6839/2696044471] conn(0x55fbbc92e4f0 0x55fbbc92d200 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.588+0000 7f72d26ceb80 1 --2- 172.21.15.33:0/536249496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbbc8a8910 0x55fbbc8c3750 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:47.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.588+0000 7f72d26ceb80 1 --2- 172.21.15.33:0/536249496 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fbbc8a42f0 0x55fbbc8c8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:47.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.588+0000 7f72d26ceb80 1 --2- 172.21.15.33:0/536249496 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fbbc8a4c10 0x55fbbc8c30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:47.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.588+0000 7f72a97fa700 0 client.15315 ms_handle_reset on v2:172.21.15.52:6838/2696044471 2024-05-15T14:00:47.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.588+0000 7f72d26ceb80 1 -- 172.21.15.33:0/536249496 >> 172.21.15.33:0/536249496 conn(0x55fbbc7cf200 msgr2=0x55fbbc8a7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:47.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.588+0000 7f72d26ceb80 1 -- 172.21.15.33:0/536249496 shutdown_connections 2024-05-15T14:00:47.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.588+0000 7f72d26ceb80 1 -- 172.21.15.33:0/536249496 wait complete. 2024-05-15T14:00:47.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.588+0000 7f72d26ceb80 20 client.15315 trim_cache size 0 max 16384 2024-05-15T14:00:47.597 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T14:00:47.598 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "bca78ca4-12c1-11ef-bc98-c7b262605968", 2024-05-15T14:00:47.598 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 2, 2024-05-15T14:00:47.598 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24461, 2024-05-15T14:00:47.598 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-05-15T14:00:47.598 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-05-15T14:00:47.598 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-05-15T14:00:47.598 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 195.10513561100001, 2024-05-15T14:00:47.598 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 24, 2024-05-15T14:00:47.598 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 83, 2024-05-15T14:00:47.598 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 82, 2024-05-15T14:00:47.598 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 223.79679769800001 2024-05-15T14:00:47.598 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T14:00:47.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.588+0000 7f72c7e38700 1 -- 172.21.15.33:0/2771899403 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f72ac0b2b00 msgr2=0x7f72ac0b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:47.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.588+0000 7f72c7e38700 1 --2- 172.21.15.33:0/2771899403 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f72ac0b2b00 0x7f72ac0b4fc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f72c0078ce0 tx=0x7f72bc00b410 comp rx=0 tx=0).stop 2024-05-15T14:00:47.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.588+0000 7f72c7e38700 1 -- 172.21.15.33:0/2771899403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72c0108620 msgr2=0x7f72c0103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:47.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.588+0000 7f72c7e38700 1 --2- 172.21.15.33:0/2771899403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72c0108620 0x7f72c0103b60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f72b800be20 tx=0x7f72b8009ed0 comp rx=0 tx=0).stop 2024-05-15T14:00:47.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.588+0000 7f72c7e38700 1 -- 172.21.15.33:0/2771899403 shutdown_connections 2024-05-15T14:00:47.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.588+0000 7f72c7e38700 1 --2- 172.21.15.33:0/2771899403 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f72ac0b2b00 0x7f72ac0b4fc0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:47.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.588+0000 7f72c7e38700 1 --2- 172.21.15.33:0/2771899403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72c0108620 0x7f72c0103b60 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:47.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.588+0000 7f72c7e38700 1 --2- 172.21.15.33:0/2771899403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f72c0101f00 0x7f72c01030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:47.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.588+0000 7f72c7e38700 1 --2- 172.21.15.33:0/2771899403 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f72c01028c0 0x7f72c0103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:47.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.588+0000 7f72c7e38700 1 -- 172.21.15.33:0/2771899403 >> 172.21.15.33:0/2771899403 conn(0x7f72c0077780 msgr2=0x7f72c0100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:47.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.589+0000 7f72c7e38700 1 -- 172.21.15.33:0/2771899403 shutdown_connections 2024-05-15T14:00:47.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:47.589+0000 7f72c7e38700 1 -- 172.21.15.33:0/2771899403 wait complete. 2024-05-15T14:00:48.228 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph tell mds.c status 2024-05-15T14:00:48.480 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:00:48.826 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:48 smithi033 ceph-mon[31650]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-05-15T14:00:48.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:48 smithi052 ceph-mon[36190]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-05-15T14:00:48.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:48 smithi081 ceph-mon[36434]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-05-15T14:00:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.813+0000 7fdaf9683640 1 -- 172.21.15.33:0/2884054636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf4103cc0 msgr2=0x7fdaf41040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.813+0000 7fdaf9683640 1 --2- 172.21.15.33:0/2884054636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf4103cc0 0x7fdaf41040a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdae4009900 tx=0x7fdae402a4a0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.813+0000 7fdaf9683640 1 -- 172.21.15.33:0/2884054636 shutdown_connections 2024-05-15T14:00:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.813+0000 7fdaf9683640 1 --2- 172.21.15.33:0/2884054636 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdaf410d090 0x7fdaf410f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.813+0000 7fdaf9683640 1 --2- 172.21.15.33:0/2884054636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdaf41045e0 0x7fdaf410cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.813+0000 7fdaf9683640 1 --2- 172.21.15.33:0/2884054636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf4103cc0 0x7fdaf41040a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.813+0000 7fdaf9683640 1 -- 172.21.15.33:0/2884054636 >> 172.21.15.33:0/2884054636 conn(0x7fdaf4078a70 msgr2=0x7fdaf4078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.813+0000 7fdaf9683640 1 -- 172.21.15.33:0/2884054636 shutdown_connections 2024-05-15T14:00:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.813+0000 7fdaf9683640 1 -- 172.21.15.33:0/2884054636 wait complete. 2024-05-15T14:00:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.814+0000 7fdaf9683640 1 Processor -- start 2024-05-15T14:00:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.814+0000 7fdaf9683640 1 -- start start 2024-05-15T14:00:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.814+0000 7fdaf9683640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdaf4103cc0 0x7fdaf41992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.814+0000 7fdaf9683640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdaf41045e0 0x7fdaf4199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:49.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.814+0000 7fdaf9683640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf410d090 0x7fdaf419db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:49.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.814+0000 7fdaf9683640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf4112d30 con 0x7fdaf410d090 2024-05-15T14:00:49.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.814+0000 7fdaf9683640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf4112bb0 con 0x7fdaf4103cc0 2024-05-15T14:00:49.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.814+0000 7fdaf9683640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf4112eb0 con 0x7fdaf41045e0 2024-05-15T14:00:49.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.814+0000 7fdaf37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf410d090 0x7fdaf419db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:49.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.814+0000 7fdaf37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf410d090 0x7fdaf419db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58680/0 (socket says 172.21.15.33:58680) 2024-05-15T14:00:49.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.814+0000 7fdaf37fe640 1 -- 172.21.15.33:0/1995500532 learned_addr learned my addr 172.21.15.33:0/1995500532 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:49.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.814+0000 7fdaf2ffd640 1 --2- 172.21.15.33:0/1995500532 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdaf4103cc0 0x7fdaf41992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:49.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.814+0000 7fdaf27fc640 1 --2- 172.21.15.33:0/1995500532 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdaf41045e0 0x7fdaf4199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:49.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.815+0000 7fdaf37fe640 1 -- 172.21.15.33:0/1995500532 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdaf41045e0 msgr2=0x7fdaf4199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:49.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.815+0000 7fdaf37fe640 1 --2- 172.21.15.33:0/1995500532 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdaf41045e0 0x7fdaf4199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.815+0000 7fdaf37fe640 1 -- 172.21.15.33:0/1995500532 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdaf4103cc0 msgr2=0x7fdaf41992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:49.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.815+0000 7fdaf37fe640 1 --2- 172.21.15.33:0/1995500532 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdaf4103cc0 0x7fdaf41992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.815+0000 7fdaf37fe640 1 -- 172.21.15.33:0/1995500532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf419e2c0 con 0x7fdaf410d090 2024-05-15T14:00:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.815+0000 7fdaf37fe640 1 --2- 172.21.15.33:0/1995500532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf410d090 0x7fdaf419db90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00b9e0 tx=0x7fdaec00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.815+0000 7fdadbfff640 1 -- 172.21.15.33:0/1995500532 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec019070 con 0x7fdaf410d090 2024-05-15T14:00:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.815+0000 7fdadbfff640 1 -- 172.21.15.33:0/1995500532 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaec014440 con 0x7fdaf410d090 2024-05-15T14:00:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.815+0000 7fdaf9683640 1 -- 172.21.15.33:0/1995500532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf419e5b0 con 0x7fdaf410d090 2024-05-15T14:00:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.815+0000 7fdadbfff640 1 -- 172.21.15.33:0/1995500532 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec0125a0 con 0x7fdaf410d090 2024-05-15T14:00:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.815+0000 7fdaf9683640 1 -- 172.21.15.33:0/1995500532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf41a5e40 con 0x7fdaf410d090 2024-05-15T14:00:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.818+0000 7fdadbfff640 1 -- 172.21.15.33:0/1995500532 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7fdaec01c070 con 0x7fdaf410d090 2024-05-15T14:00:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.819+0000 7fdafc3a3740 1 Processor -- start 2024-05-15T14:00:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.819+0000 7fdafc3a3740 1 -- start start 2024-05-15T14:00:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.819+0000 7fdadbfff640 1 --2- 172.21.15.33:0/1995500532 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fdadc0b29e0 0x7fdadc0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.819+0000 7fdafc3a3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1a24257d0 0x55d1a2425bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.819+0000 7fdafc3a3740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d1a24260f0 0x55d1a24349a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.819+0000 7fdafc3a3740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1a2434ee0 0x55d1a24372d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.819+0000 7fdaf27fc640 1 --2- 172.21.15.33:0/1995500532 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fdadc0b29e0 0x7fdadc0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.819+0000 7fdafc3a3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d1a2398c30 con 0x55d1a24257d0 2024-05-15T14:00:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.819+0000 7fdafc3a3740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d1a23887d0 con 0x55d1a24260f0 2024-05-15T14:00:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.819+0000 7fdafc3a3740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55d1a238c810 con 0x55d1a2434ee0 2024-05-15T14:00:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.819+0000 7fdadbfff640 1 -- 172.21.15.33:0/1995500532 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fdaec0e1b90 con 0x7fdaf410d090 2024-05-15T14:00:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.819+0000 7fdaf2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1a24257d0 0x55d1a2425bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.819+0000 7fdaf2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1a24257d0 0x55d1a2425bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58692/0 (socket says 172.21.15.33:58692) 2024-05-15T14:00:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.819+0000 7fdaf2ffd640 1 -- 172.21.15.33:0/2653375679 learned_addr learned my addr 172.21.15.33:0/2653375679 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.819+0000 7fdaf37fe640 1 --2- 172.21.15.33:0/2653375679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1a2434ee0 0x55d1a24372d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.820+0000 7fdaf27fc640 1 --2- 172.21.15.33:0/2653375679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d1a24260f0 0x55d1a24349a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.820+0000 7fdaf2ffd640 1 -- 172.21.15.33:0/2653375679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1a2434ee0 msgr2=0x55d1a24372d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.820+0000 7fdaf2ffd640 1 --2- 172.21.15.33:0/2653375679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1a2434ee0 0x55d1a24372d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.820+0000 7fdaf2ffd640 1 -- 172.21.15.33:0/2653375679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d1a24260f0 msgr2=0x55d1a24349a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.820+0000 7fdaf2ffd640 1 --2- 172.21.15.33:0/2653375679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d1a24260f0 0x55d1a24349a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.820+0000 7fdaf2ffd640 1 -- 172.21.15.33:0/2653375679 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1a209d210 con 0x55d1a24257d0 2024-05-15T14:00:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.820+0000 7fdaf27fc640 1 --2- 172.21.15.33:0/2653375679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d1a24260f0 0x55d1a24349a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:00:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.820+0000 7fdaf37fe640 1 --2- 172.21.15.33:0/2653375679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1a2434ee0 0x55d1a24372d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:00:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.820+0000 7fdaf27fc640 1 --2- 172.21.15.33:0/1995500532 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fdadc0b29e0 0x7fdadc0b4ea0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdaf410efa0 tx=0x7fdae80103d0 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.820+0000 7fdaf2ffd640 1 --2- 172.21.15.33:0/2653375679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1a24257d0 0x55d1a2425bb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdae4033000 tx=0x7fdae4032e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7e34da91892040 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.820+0000 7fdad97fa640 1 -- 172.21.15.33:0/2653375679 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae4035040 con 0x55d1a24257d0 2024-05-15T14:00:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.820+0000 7fdad97fa640 1 -- 172.21.15.33:0/2653375679 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdae40316a0 con 0x55d1a24257d0 2024-05-15T14:00:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.820+0000 7fdad97fa640 1 -- 172.21.15.33:0/2653375679 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae4004750 con 0x55d1a24257d0 2024-05-15T14:00:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.821+0000 7fdafc3a3740 1 -- 172.21.15.33:0/2653375679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1a24257d0 msgr2=0x55d1a2425bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.821+0000 7fdafc3a3740 1 --2- 172.21.15.33:0/2653375679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1a24257d0 0x55d1a2425bb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdae4033000 tx=0x7fdae4032e60 comp rx=0 tx=0).stop 2024-05-15T14:00:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.821+0000 7fdafc3a3740 1 -- 172.21.15.33:0/2653375679 shutdown_connections 2024-05-15T14:00:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.821+0000 7fdafc3a3740 1 --2- 172.21.15.33:0/2653375679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1a2434ee0 0x55d1a24372d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.821+0000 7fdafc3a3740 1 --2- 172.21.15.33:0/2653375679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d1a24260f0 0x55d1a24349a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.821+0000 7fdafc3a3740 1 --2- 172.21.15.33:0/2653375679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1a24257d0 0x55d1a2425bb0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.821+0000 7fdafc3a3740 1 -- 172.21.15.33:0/2653375679 >> 172.21.15.33:0/2653375679 conn(0x55d1a2421e00 msgr2=0x55d1a24221e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.821+0000 7fdafc3a3740 1 -- 172.21.15.33:0/2653375679 shutdown_connections 2024-05-15T14:00:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.821+0000 7fdafc3a3740 1 -- 172.21.15.33:0/2653375679 wait complete. 2024-05-15T14:00:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.821+0000 7fdafc3a3740 1 -- start start 2024-05-15T14:00:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.821+0000 7fdafc3a3740 1 Processor -- start 2024-05-15T14:00:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.821+0000 7fdafc3a3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1a24257d0 0x55d1a24348a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.821+0000 7fdafc3a3740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d1a24260f0 0x55d1a2431b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.821+0000 7fdafc3a3740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1a2434ee0 0x55d1a24321a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.821+0000 7fdafc3a3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d1a206e790 con 0x55d1a24257d0 2024-05-15T14:00:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.821+0000 7fdafc3a3740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d1a2344890 con 0x55d1a24260f0 2024-05-15T14:00:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.821+0000 7fdafc3a3740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55d1a2437b50 con 0x55d1a2434ee0 2024-05-15T14:00:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.822+0000 7fdafc3a3740 10 client.0 _ll_register_callbacks cb 0x55d1a2347d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:00:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.822+0000 7fdaf2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1a24257d0 0x55d1a24348a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.822+0000 7fdaf2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1a24257d0 0x55d1a24348a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58696/0 (socket says 172.21.15.33:58696) 2024-05-15T14:00:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.822+0000 7fdaf2ffd640 1 -- 172.21.15.33:0/3691883023 learned_addr learned my addr 172.21.15.33:0/3691883023 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.822+0000 7fdaf37fe640 1 --2- 172.21.15.33:0/3691883023 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1a2434ee0 0x55d1a24321a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.822+0000 7fdaf27fc640 1 --2- 172.21.15.33:0/3691883023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d1a24260f0 0x55d1a2431b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.822+0000 7fdaf2ffd640 1 -- 172.21.15.33:0/3691883023 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1a2434ee0 msgr2=0x55d1a24321a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.822+0000 7fdaf2ffd640 1 --2- 172.21.15.33:0/3691883023 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1a2434ee0 0x55d1a24321a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.822+0000 7fdaf2ffd640 1 -- 172.21.15.33:0/3691883023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d1a24260f0 msgr2=0x55d1a2431b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.822+0000 7fdaf2ffd640 1 --2- 172.21.15.33:0/3691883023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d1a24260f0 0x55d1a2431b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.822+0000 7fdaf2ffd640 1 -- 172.21.15.33:0/3691883023 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1a2169620 con 0x55d1a24257d0 2024-05-15T14:00:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.822+0000 7fdaf27fc640 1 --2- 172.21.15.33:0/3691883023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d1a24260f0 0x55d1a2431b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:00:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.822+0000 7fdaf2ffd640 1 --2- 172.21.15.33:0/3691883023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1a24257d0 0x55d1a24348a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fdae4009900 tx=0x7fdae402a4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce113249ddb7cb41 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.822+0000 7fdad6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:00:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.822+0000 7fdad6ff5640 1 -- 172.21.15.33:0/3691883023 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae404c070 con 0x55d1a24257d0 2024-05-15T14:00:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.822+0000 7fdad6ff5640 1 -- 172.21.15.33:0/3691883023 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fdae4004020 con 0x55d1a24257d0 2024-05-15T14:00:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.822+0000 7fdafc3a3740 1 -- 172.21.15.33:0/3691883023 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdae40316a0 con 0x55d1a24257d0 2024-05-15T14:00:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.822+0000 7fdad6ff5640 1 -- 172.21.15.33:0/3691883023 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdae4057050 con 0x55d1a24257d0 2024-05-15T14:00:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.822+0000 7fdafc3a3740 10 client.15327 fetch_fsmap learned FSMap version 24 2024-05-15T14:00:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.823+0000 7fdafc3a3740 1 -- 172.21.15.33:0/3691883023 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d1a24435a0 con 0x55d1a24257d0 2024-05-15T14:00:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.823+0000 7fdad6ff5640 1 -- 172.21.15.33:0/3691883023 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdae40316a0 con 0x55d1a24257d0 2024-05-15T14:00:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.823+0000 7fdad6ff5640 1 -- 172.21.15.33:0/3691883023 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fdae40035f0 con 0x55d1a24257d0 2024-05-15T14:00:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.823+0000 7fdafc3a3740 10 client.15327 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:00:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.823+0000 7fdafc3a3740 10 client.15327 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:00:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.823+0000 7fdafc3a3740 20 client.15327 populate_metadata read hostname 'smithi033' 2024-05-15T14:00:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.823+0000 7fdafc3a3740 1 --2- 172.21.15.33:0/3691883023 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55d1a24448f0 0x55d1a2446ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.823+0000 7fdafc3a3740 4 client.15327 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:00:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.823+0000 7fdafc3a3740 1 -- 172.21.15.33:0/3691883023 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1a239ab80 con 0x55d1a24448f0 2024-05-15T14:00:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.855+0000 7fdaf27fc640 1 --2- 172.21.15.33:0/3691883023 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55d1a24448f0 0x55d1a2446ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.856+0000 7fdaf27fc640 1 --2- 172.21.15.33:0/3691883023 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55d1a24448f0 0x55d1a2446ce0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65333253738ef6b0 server_cookie=27210cc539172b61 in_seq=0 out_seq=0 2024-05-15T14:00:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.856+0000 7fdad6ff5640 10 client.15327 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:00:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.858+0000 7fdad6ff5640 1 -- 172.21.15.33:0/3691883023 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d1a239ab80 con 0x55d1a24448f0 2024-05-15T14:00:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.858+0000 7fdad6ff5640 10 client.15327 handle_command_reply: tid=0 2024-05-15T14:00:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.858+0000 7fdafc3a3740 1 client.15327 shutdown 2024-05-15T14:00:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.858+0000 7fdafc3a3740 1 -- 172.21.15.33:0/3691883023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1a24257d0 msgr2=0x55d1a24348a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.858+0000 7fdafc3a3740 1 --2- 172.21.15.33:0/3691883023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1a24257d0 0x55d1a24348a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fdae4009900 tx=0x7fdae402a4a0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.858+0000 7fdafc3a3740 1 -- 172.21.15.33:0/3691883023 shutdown_connections 2024-05-15T14:00:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.858+0000 7fdafc3a3740 1 --2- 172.21.15.33:0/3691883023 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55d1a24448f0 0x55d1a2446ce0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.858+0000 7fdafc3a3740 1 --2- 172.21.15.33:0/3691883023 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1a2434ee0 0x55d1a24321a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.858+0000 7fdafc3a3740 1 --2- 172.21.15.33:0/3691883023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d1a24260f0 0x55d1a2431b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.858+0000 7fdad6ff5640 0 client.15327 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:00:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.858+0000 7fdafc3a3740 1 --2- 172.21.15.33:0/3691883023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1a24257d0 0x55d1a24348a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.858+0000 7fdafc3a3740 1 -- 172.21.15.33:0/3691883023 >> 172.21.15.33:0/3691883023 conn(0x55d1a2421e00 msgr2=0x55d1a2427c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.858+0000 7fdafc3a3740 1 -- 172.21.15.33:0/3691883023 shutdown_connections 2024-05-15T14:00:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.858+0000 7fdafc3a3740 1 -- 172.21.15.33:0/3691883023 wait complete. 2024-05-15T14:00:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.858+0000 7fdafc3a3740 20 client.15327 trim_cache size 0 max 16384 2024-05-15T14:00:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.872+0000 7fdafc3a3740 1 Processor -- start 2024-05-15T14:00:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.872+0000 7fdafc3a3740 1 -- start start 2024-05-15T14:00:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.872+0000 7fdafc3a3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1a24257d0 0x55d1a2440a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.872+0000 7fdafc3a3740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d1a24260f0 0x55d1a24343d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.872+0000 7fdafc3a3740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1a2434ee0 0x55d1a244c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.872+0000 7fdafc3a3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf41aedb0 con 0x55d1a24257d0 2024-05-15T14:00:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.872+0000 7fdafc3a3740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf41ac680 con 0x55d1a24260f0 2024-05-15T14:00:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.872+0000 7fdafc3a3740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf41aa4d0 con 0x55d1a2434ee0 2024-05-15T14:00:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.872+0000 7fdaf2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1a24257d0 0x55d1a2440a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.873+0000 7fdaf2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1a24257d0 0x55d1a2440a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58712/0 (socket says 172.21.15.33:58712) 2024-05-15T14:00:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.873+0000 7fdaf2ffd640 1 -- 172.21.15.33:0/70919368 learned_addr learned my addr 172.21.15.33:0/70919368 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.873+0000 7fdaf27fc640 1 --2- 172.21.15.33:0/70919368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d1a24260f0 0x55d1a24343d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.873+0000 7fdaf37fe640 1 --2- 172.21.15.33:0/70919368 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1a2434ee0 0x55d1a244c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.873+0000 7fdaf2ffd640 1 -- 172.21.15.33:0/70919368 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1a2434ee0 msgr2=0x55d1a244c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.873+0000 7fdaf2ffd640 1 --2- 172.21.15.33:0/70919368 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1a2434ee0 0x55d1a244c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.873+0000 7fdaf2ffd640 1 -- 172.21.15.33:0/70919368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d1a24260f0 msgr2=0x55d1a24343d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.873+0000 7fdaf2ffd640 1 --2- 172.21.15.33:0/70919368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d1a24260f0 0x55d1a24343d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.873+0000 7fdaf2ffd640 1 -- 172.21.15.33:0/70919368 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1a241d600 con 0x55d1a24257d0 2024-05-15T14:00:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.873+0000 7fdaf27fc640 1 --2- 172.21.15.33:0/70919368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d1a24260f0 0x55d1a24343d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:00:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.873+0000 7fdaf2ffd640 1 --2- 172.21.15.33:0/70919368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1a24257d0 0x55d1a2440a60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdae40039c0 tx=0x7fdae4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42d67775115dba5 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.873+0000 7fdad8ff9640 1 -- 172.21.15.33:0/70919368 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae404c070 con 0x55d1a24257d0 2024-05-15T14:00:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.873+0000 7fdad8ff9640 1 -- 172.21.15.33:0/70919368 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdae4003c30 con 0x55d1a24257d0 2024-05-15T14:00:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.873+0000 7fdad8ff9640 1 -- 172.21.15.33:0/70919368 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae4035040 con 0x55d1a24257d0 2024-05-15T14:00:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.874+0000 7fdafc3a3740 1 -- 172.21.15.33:0/70919368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1a24257d0 msgr2=0x55d1a2440a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.874+0000 7fdafc3a3740 1 --2- 172.21.15.33:0/70919368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1a24257d0 0x55d1a2440a60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdae40039c0 tx=0x7fdae4005fd0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.874+0000 7fdafc3a3740 1 -- 172.21.15.33:0/70919368 shutdown_connections 2024-05-15T14:00:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.874+0000 7fdafc3a3740 1 --2- 172.21.15.33:0/70919368 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1a2434ee0 0x55d1a244c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.874+0000 7fdafc3a3740 1 --2- 172.21.15.33:0/70919368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d1a24260f0 0x55d1a24343d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.874+0000 7fdafc3a3740 1 --2- 172.21.15.33:0/70919368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1a24257d0 0x55d1a2440a60 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.874+0000 7fdafc3a3740 1 -- 172.21.15.33:0/70919368 >> 172.21.15.33:0/70919368 conn(0x55d1a2421e00 msgr2=0x55d1a242d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.874+0000 7fdafc3a3740 1 -- 172.21.15.33:0/70919368 shutdown_connections 2024-05-15T14:00:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.874+0000 7fdafc3a3740 1 -- 172.21.15.33:0/70919368 wait complete. 2024-05-15T14:00:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.874+0000 7fdafc3a3740 1 -- start start 2024-05-15T14:00:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.874+0000 7fdafc3a3740 1 Processor -- start 2024-05-15T14:00:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.874+0000 7fdafc3a3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1a24257d0 0x55d1a243d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.874+0000 7fdafc3a3740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1a24260f0 0x55d1a2438420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.874+0000 7fdafc3a3740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d1a2434ee0 0x55d1a2438960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.874+0000 7fdafc3a3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf41aab40 con 0x55d1a24257d0 2024-05-15T14:00:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.874+0000 7fdafc3a3740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf41abfb0 con 0x55d1a2434ee0 2024-05-15T14:00:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.874+0000 7fdafc3a3740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf41ab210 con 0x55d1a24260f0 2024-05-15T14:00:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.874+0000 7fdafc3a3740 10 client.0 _ll_register_callbacks cb 0x7fdae4032b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:00:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.874+0000 7fdaf2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1a24257d0 0x55d1a243d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.874+0000 7fdaf2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1a24257d0 0x55d1a243d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58714/0 (socket says 172.21.15.33:58714) 2024-05-15T14:00:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.874+0000 7fdaf2ffd640 1 -- 172.21.15.33:0/2090514907 learned_addr learned my addr 172.21.15.33:0/2090514907 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.875+0000 7fdaf37fe640 1 --2- 172.21.15.33:0/2090514907 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d1a2434ee0 0x55d1a2438960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.875+0000 7fdaf27fc640 1 --2- 172.21.15.33:0/2090514907 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1a24260f0 0x55d1a2438420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.875+0000 7fdaf2ffd640 1 -- 172.21.15.33:0/2090514907 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1a24260f0 msgr2=0x55d1a2438420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.875+0000 7fdaf2ffd640 1 --2- 172.21.15.33:0/2090514907 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1a24260f0 0x55d1a2438420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.875+0000 7fdaf2ffd640 1 -- 172.21.15.33:0/2090514907 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d1a2434ee0 msgr2=0x55d1a2438960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.875+0000 7fdaf2ffd640 1 --2- 172.21.15.33:0/2090514907 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d1a2434ee0 0x55d1a2438960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.875+0000 7fdaf2ffd640 1 -- 172.21.15.33:0/2090514907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1a2438ed0 con 0x55d1a24257d0 2024-05-15T14:00:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.875+0000 7fdaf27fc640 1 --2- 172.21.15.33:0/2090514907 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1a24260f0 0x55d1a2438420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:00:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.875+0000 7fdaf2ffd640 1 --2- 172.21.15.33:0/2090514907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1a24257d0 0x55d1a243d0b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdae40053b0 tx=0x7fdae4005410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2952d2c753f98bc server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.875+0000 7fdad6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:00:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.875+0000 7fdad6ff5640 1 -- 172.21.15.33:0/2090514907 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae4060020 con 0x55d1a24257d0 2024-05-15T14:00:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.875+0000 7fdad6ff5640 1 -- 172.21.15.33:0/2090514907 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fdae4031a60 con 0x55d1a24257d0 2024-05-15T14:00:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.875+0000 7fdafc3a3740 1 -- 172.21.15.33:0/2090514907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdae4003c30 con 0x55d1a24257d0 2024-05-15T14:00:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.875+0000 7fdad6ff5640 1 -- 172.21.15.33:0/2090514907 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdae4048020 con 0x55d1a24257d0 2024-05-15T14:00:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.875+0000 7fdafc3a3740 10 client.15333 fetch_fsmap learned FSMap version 24 2024-05-15T14:00:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.875+0000 7fdafc3a3740 1 -- 172.21.15.33:0/2090514907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d1a24aec40 con 0x55d1a24257d0 2024-05-15T14:00:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.875+0000 7fdad6ff5640 1 -- 172.21.15.33:0/2090514907 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdae4003c30 con 0x55d1a24257d0 2024-05-15T14:00:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.876+0000 7fdad6ff5640 1 -- 172.21.15.33:0/2090514907 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fdae4004900 con 0x55d1a24257d0 2024-05-15T14:00:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.876+0000 7fdafc3a3740 10 client.15333 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:00:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.876+0000 7fdafc3a3740 10 client.15333 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:00:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.876+0000 7fdafc3a3740 20 client.15333 populate_metadata read hostname 'smithi033' 2024-05-15T14:00:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.876+0000 7fdafc3a3740 1 --2- 172.21.15.33:0/2090514907 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55d1a24af300 0x55d1a24b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.876+0000 7fdafc3a3740 4 client.15333 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:00:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.876+0000 7fdafc3a3740 1 -- 172.21.15.33:0/2090514907 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d1a2394360 con 0x55d1a24af300 2024-05-15T14:00:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.876+0000 7fdaf27fc640 1 --2- 172.21.15.33:0/2090514907 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55d1a24af300 0x55d1a24b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.876+0000 7fdaf27fc640 1 --2- 172.21.15.33:0/2090514907 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55d1a24af300 0x55d1a24b1720 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b629107bab50ffe3 server_cookie=9eb3cb7ce11c3777 in_seq=0 out_seq=0 2024-05-15T14:00:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.876+0000 7fdad6ff5640 10 client.15333 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:00:49.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.885+0000 7fdad6ff5640 1 -- 172.21.15.33:0/2090514907 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d1a2394360 con 0x55d1a24af300 2024-05-15T14:00:49.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.885+0000 7fdad6ff5640 10 client.15333 handle_command_reply: tid=0 2024-05-15T14:00:49.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.885+0000 7fdafc3a3740 1 client.15333 shutdown 2024-05-15T14:00:49.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.885+0000 7fdafc3a3740 1 -- 172.21.15.33:0/2090514907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1a24257d0 msgr2=0x55d1a243d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:49.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.885+0000 7fdafc3a3740 1 --2- 172.21.15.33:0/2090514907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1a24257d0 0x55d1a243d0b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdae40053b0 tx=0x7fdae4005410 comp rx=0 tx=0).stop 2024-05-15T14:00:49.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.885+0000 7fdafc3a3740 1 -- 172.21.15.33:0/2090514907 shutdown_connections 2024-05-15T14:00:49.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.885+0000 7fdafc3a3740 1 --2- 172.21.15.33:0/2090514907 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55d1a24af300 0x55d1a24b1720 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.885+0000 7fdafc3a3740 1 --2- 172.21.15.33:0/2090514907 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d1a2434ee0 0x55d1a2438960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.885+0000 7fdafc3a3740 1 --2- 172.21.15.33:0/2090514907 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1a24260f0 0x55d1a2438420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.885+0000 7fdad6ff5640 0 client.15333 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:00:49.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.885+0000 7fdafc3a3740 1 --2- 172.21.15.33:0/2090514907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d1a24257d0 0x55d1a243d0b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.885+0000 7fdafc3a3740 1 -- 172.21.15.33:0/2090514907 >> 172.21.15.33:0/2090514907 conn(0x55d1a2421e00 msgr2=0x55d1a2420870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:49.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.885+0000 7fdafc3a3740 1 -- 172.21.15.33:0/2090514907 shutdown_connections 2024-05-15T14:00:49.891 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.885+0000 7fdafc3a3740 1 -- 172.21.15.33:0/2090514907 wait complete. 2024-05-15T14:00:49.891 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.886+0000 7fdafc3a3740 20 client.15333 trim_cache size 0 max 16384 2024-05-15T14:00:49.891 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:00:49.891 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:00:49.891 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:00:49.892 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.886+0000 7fdaf9683640 1 -- 172.21.15.33:0/1995500532 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fdadc0b29e0 msgr2=0x7fdadc0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:49.892 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.886+0000 7fdaf9683640 1 --2- 172.21.15.33:0/1995500532 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fdadc0b29e0 0x7fdadc0b4ea0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdaf410efa0 tx=0x7fdae80103d0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.892 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.886+0000 7fdaf9683640 1 -- 172.21.15.33:0/1995500532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf410d090 msgr2=0x7fdaf419db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:49.892 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.886+0000 7fdaf9683640 1 --2- 172.21.15.33:0/1995500532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf410d090 0x7fdaf419db90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00b9e0 tx=0x7fdaec00bea0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.892 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.886+0000 7fdaf9683640 1 -- 172.21.15.33:0/1995500532 shutdown_connections 2024-05-15T14:00:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.886+0000 7fdaf9683640 1 --2- 172.21.15.33:0/1995500532 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fdadc0b29e0 0x7fdadc0b4ea0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.886+0000 7fdaf9683640 1 --2- 172.21.15.33:0/1995500532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf410d090 0x7fdaf419db90 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.886+0000 7fdaf9683640 1 --2- 172.21.15.33:0/1995500532 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdaf41045e0 0x7fdaf4199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.886+0000 7fdaf9683640 1 --2- 172.21.15.33:0/1995500532 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdaf4103cc0 0x7fdaf41992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.886+0000 7fdaf9683640 1 -- 172.21.15.33:0/1995500532 >> 172.21.15.33:0/1995500532 conn(0x7fdaf4078a70 msgr2=0x7fdaf40feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.886+0000 7fdaf9683640 1 -- 172.21.15.33:0/1995500532 shutdown_connections 2024-05-15T14:00:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:00:49.886+0000 7fdaf9683640 1 -- 172.21.15.33:0/1995500532 wait complete. 2024-05-15T14:00:50.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.030+0000 7fbd8fe7e700 1 -- 172.21.15.33:0/2283625930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd881088f0 msgr2=0x7fbd88110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:50.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.030+0000 7fbd8fe7e700 1 --2- 172.21.15.33:0/2283625930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd881088f0 0x7fbd88110470 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbd8000b0d0 tx=0x7fbd8000b3e0 comp rx=0 tx=0).stop 2024-05-15T14:00:50.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.030+0000 7fbd8fe7e700 1 -- 172.21.15.33:0/2283625930 shutdown_connections 2024-05-15T14:00:50.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.030+0000 7fbd8fe7e700 1 --2- 172.21.15.33:0/2283625930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd881088f0 0x7fbd88110470 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:50.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.030+0000 7fbd8fe7e700 1 --2- 172.21.15.33:0/2283625930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd881037f0 0x7fbd88103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:50.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.030+0000 7fbd8fe7e700 1 --2- 172.21.15.33:0/2283625930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd881041a0 0x7fbd881081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:50.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.030+0000 7fbd8fe7e700 1 -- 172.21.15.33:0/2283625930 >> 172.21.15.33:0/2283625930 conn(0x7fbd8807a6a0 msgr2=0x7fbd8807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:50.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.030+0000 7fbd8fe7e700 1 -- 172.21.15.33:0/2283625930 shutdown_connections 2024-05-15T14:00:50.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.031+0000 7fbd8fe7e700 1 -- 172.21.15.33:0/2283625930 wait complete. 2024-05-15T14:00:50.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.031+0000 7fbd8fe7e700 1 Processor -- start 2024-05-15T14:00:50.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.031+0000 7fbd8fe7e700 1 -- start start 2024-05-15T14:00:50.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.031+0000 7fbd8fe7e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd881037f0 0x7fbd88075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:50.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.031+0000 7fbd8fe7e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd881041a0 0x7fbd88075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:50.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.031+0000 7fbd8fe7e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd881088f0 0x7fbd880762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:50.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.031+0000 7fbd8fe7e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbd881136e0 con 0x7fbd881041a0 2024-05-15T14:00:50.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.031+0000 7fbd8fe7e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbd88113560 con 0x7fbd881037f0 2024-05-15T14:00:50.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.031+0000 7fbd8fe7e700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbd88113860 con 0x7fbd881088f0 2024-05-15T14:00:50.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.031+0000 7fbd8d419700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd881041a0 0x7fbd88075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:50.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.032+0000 7fbd8d419700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd881041a0 0x7fbd88075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58730/0 (socket says 172.21.15.33:58730) 2024-05-15T14:00:50.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.032+0000 7fbd8d419700 1 -- 172.21.15.33:0/2747759166 learned_addr learned my addr 172.21.15.33:0/2747759166 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:50.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.032+0000 7fbd8e41b700 1 --2- 172.21.15.33:0/2747759166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd881088f0 0x7fbd880762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:50.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.032+0000 7fbd8dc1a700 1 --2- 172.21.15.33:0/2747759166 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd881037f0 0x7fbd88075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:50.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.032+0000 7fbd8d419700 1 -- 172.21.15.33:0/2747759166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd881088f0 msgr2=0x7fbd880762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:50.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.032+0000 7fbd8d419700 1 --2- 172.21.15.33:0/2747759166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd881088f0 0x7fbd880762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:50.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.032+0000 7fbd8d419700 1 -- 172.21.15.33:0/2747759166 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd881037f0 msgr2=0x7fbd88075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:50.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.032+0000 7fbd8d419700 1 --2- 172.21.15.33:0/2747759166 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd881037f0 0x7fbd88075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:50.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.032+0000 7fbd8d419700 1 -- 172.21.15.33:0/2747759166 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd881a8050 con 0x7fbd881041a0 2024-05-15T14:00:50.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.032+0000 7fbd8d419700 1 --2- 172.21.15.33:0/2747759166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd881041a0 0x7fbd88075d80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbd7800c920 tx=0x7fbd7800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:50.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.032+0000 7fbd7effd700 1 -- 172.21.15.33:0/2747759166 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd78004a30 con 0x7fbd881041a0 2024-05-15T14:00:50.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.032+0000 7fbd7effd700 1 -- 172.21.15.33:0/2747759166 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd780054d0 con 0x7fbd881041a0 2024-05-15T14:00:50.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.032+0000 7fbd8fe7e700 1 -- 172.21.15.33:0/2747759166 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd881a8340 con 0x7fbd881041a0 2024-05-15T14:00:50.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.032+0000 7fbd7effd700 1 -- 172.21.15.33:0/2747759166 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd78007c70 con 0x7fbd881041a0 2024-05-15T14:00:50.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.032+0000 7fbd8fe7e700 1 -- 172.21.15.33:0/2747759166 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd881a8880 con 0x7fbd881041a0 2024-05-15T14:00:50.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.035+0000 7fbd7effd700 1 -- 172.21.15.33:0/2747759166 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7fbd78004c60 con 0x7fbd881041a0 2024-05-15T14:00:50.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.036+0000 7fbd7effd700 1 --2- 172.21.15.33:0/2747759166 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbd740b2a80 0x7fbd740b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:50.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.036+0000 7fbd9a714b80 1 Processor -- start 2024-05-15T14:00:50.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.036+0000 7fbd8dc1a700 1 --2- 172.21.15.33:0/2747759166 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbd740b2a80 0x7fbd740b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:50.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.036+0000 7fbd7effd700 1 -- 172.21.15.33:0/2747759166 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fbd780e2650 con 0x7fbd881041a0 2024-05-15T14:00:50.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.036+0000 7fbd9a714b80 1 -- start start 2024-05-15T14:00:50.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.036+0000 7fbd9a714b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff1fee82f0 0x55ff1fee86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:50.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.036+0000 7fbd9a714b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff1fee8c10 0x55ff1feec3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:50.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.036+0000 7fbd9a714b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff1feec910 0x55ff1feecd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:50.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.036+0000 7fbd8dc1a700 1 --2- 172.21.15.33:0/2747759166 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbd740b2a80 0x7fbd740b4f40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbd84009c60 tx=0x7fbd84009260 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:50.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.037+0000 7fbd9a714b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ff1fe46c40 con 0x55ff1feec910 2024-05-15T14:00:50.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.037+0000 7fbd9a714b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ff1fe367e0 con 0x55ff1fee8c10 2024-05-15T14:00:50.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.037+0000 7fbd9a714b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55ff1fe3a820 con 0x55ff1fee82f0 2024-05-15T14:00:50.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.037+0000 7fbd8e41b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff1feec910 0x55ff1feecd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:50.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.037+0000 7fbd8e41b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff1feec910 0x55ff1feecd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58732/0 (socket says 172.21.15.33:58732) 2024-05-15T14:00:50.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.037+0000 7fbd8e41b700 1 -- 172.21.15.33:0/1634687837 learned_addr learned my addr 172.21.15.33:0/1634687837 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:50.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.037+0000 7fbd8d419700 1 --2- 172.21.15.33:0/1634687837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff1fee8c10 0x55ff1feec3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:50.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.037+0000 7fbd8dc1a700 1 --2- 172.21.15.33:0/1634687837 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff1fee82f0 0x55ff1fee86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:50.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.037+0000 7fbd8e41b700 1 -- 172.21.15.33:0/1634687837 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff1fee82f0 msgr2=0x55ff1fee86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:50.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.037+0000 7fbd8e41b700 1 --2- 172.21.15.33:0/1634687837 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff1fee82f0 0x55ff1fee86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:50.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.037+0000 7fbd8e41b700 1 -- 172.21.15.33:0/1634687837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff1fee8c10 msgr2=0x55ff1feec3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:50.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.037+0000 7fbd8e41b700 1 --2- 172.21.15.33:0/1634687837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff1fee8c10 0x55ff1feec3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:50.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.037+0000 7fbd8e41b700 1 -- 172.21.15.33:0/1634687837 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff1fd86880 con 0x55ff1feec910 2024-05-15T14:00:50.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.037+0000 7fbd8e41b700 1 --2- 172.21.15.33:0/1634687837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff1feec910 0x55ff1feecd50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbd80005ac0 tx=0x7fbd80005af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de277b106b6678a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:50.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.037+0000 7fbd73fff700 1 -- 172.21.15.33:0/1634687837 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8000baa0 con 0x55ff1feec910 2024-05-15T14:00:50.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.037+0000 7fbd73fff700 1 -- 172.21.15.33:0/1634687837 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd80020070 con 0x55ff1feec910 2024-05-15T14:00:50.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.037+0000 7fbd73fff700 1 -- 172.21.15.33:0/1634687837 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd800055f0 con 0x55ff1feec910 2024-05-15T14:00:50.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.038+0000 7fbd9a714b80 1 -- 172.21.15.33:0/1634687837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff1feec910 msgr2=0x55ff1feecd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:50.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.038+0000 7fbd9a714b80 1 --2- 172.21.15.33:0/1634687837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff1feec910 0x55ff1feecd50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbd80005ac0 tx=0x7fbd80005af0 comp rx=0 tx=0).stop 2024-05-15T14:00:50.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.038+0000 7fbd9a714b80 1 -- 172.21.15.33:0/1634687837 shutdown_connections 2024-05-15T14:00:50.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.038+0000 7fbd9a714b80 1 --2- 172.21.15.33:0/1634687837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff1feec910 0x55ff1feecd50 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:50.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.038+0000 7fbd9a714b80 1 --2- 172.21.15.33:0/1634687837 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff1fee82f0 0x55ff1fee86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:50.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.038+0000 7fbd9a714b80 1 --2- 172.21.15.33:0/1634687837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff1fee8c10 0x55ff1feec3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:50.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.038+0000 7fbd9a714b80 1 -- 172.21.15.33:0/1634687837 >> 172.21.15.33:0/1634687837 conn(0x55ff1fe13200 msgr2=0x55ff1fee2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:50.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.038+0000 7fbd9a714b80 1 -- 172.21.15.33:0/1634687837 shutdown_connections 2024-05-15T14:00:50.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.038+0000 7fbd9a714b80 1 -- 172.21.15.33:0/1634687837 wait complete. 2024-05-15T14:00:50.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.039+0000 7fbd9a714b80 1 -- start start 2024-05-15T14:00:50.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.039+0000 7fbd9a714b80 1 Processor -- start 2024-05-15T14:00:50.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.039+0000 7fbd9a714b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff1fee82f0 0x55ff1fef3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:50.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.039+0000 7fbd9a714b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff1fee8c10 0x55ff1fef1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:50.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.039+0000 7fbd9a714b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff1feec910 0x55ff1fef1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:50.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.039+0000 7fbd9a714b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ff1fd3cf50 con 0x55ff1fee82f0 2024-05-15T14:00:50.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.039+0000 7fbd9a714b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ff1fdd8770 con 0x55ff1fee8c10 2024-05-15T14:00:50.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.039+0000 7fbd9a714b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55ff1fef4b10 con 0x55ff1feec910 2024-05-15T14:00:50.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.039+0000 7fbd9a714b80 10 client.0 _ll_register_callbacks cb 0x55ff1fdcd530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:00:50.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.039+0000 7fbd8dc1a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff1fee82f0 0x55ff1fef3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:50.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.039+0000 7fbd8dc1a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff1fee82f0 0x55ff1fef3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58746/0 (socket says 172.21.15.33:58746) 2024-05-15T14:00:50.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.039+0000 7fbd8dc1a700 1 -- 172.21.15.33:0/2595521186 learned_addr learned my addr 172.21.15.33:0/2595521186 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:50.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.040+0000 7fbd8d419700 1 --2- 172.21.15.33:0/2595521186 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff1fee8c10 0x55ff1fef1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:50.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.040+0000 7fbd8e41b700 1 --2- 172.21.15.33:0/2595521186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff1feec910 0x55ff1fef1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:50.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.040+0000 7fbd8dc1a700 1 -- 172.21.15.33:0/2595521186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff1feec910 msgr2=0x55ff1fef1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:50.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.040+0000 7fbd8dc1a700 1 --2- 172.21.15.33:0/2595521186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff1feec910 0x55ff1fef1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:50.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.040+0000 7fbd8dc1a700 1 -- 172.21.15.33:0/2595521186 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff1fee8c10 msgr2=0x55ff1fef1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:50.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.040+0000 7fbd8dc1a700 1 --2- 172.21.15.33:0/2595521186 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff1fee8c10 0x55ff1fef1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:50.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.040+0000 7fbd8dc1a700 1 -- 172.21.15.33:0/2595521186 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff1fdd5640 con 0x55ff1fee82f0 2024-05-15T14:00:50.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.040+0000 7fbd8e41b700 1 --2- 172.21.15.33:0/2595521186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff1feec910 0x55ff1fef1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:00:50.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.040+0000 7fbd8d419700 1 --2- 172.21.15.33:0/2595521186 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff1fee8c10 0x55ff1fef1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:00:50.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.040+0000 7fbd8dc1a700 1 --2- 172.21.15.33:0/2595521186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff1fee82f0 0x55ff1fef3d90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbd84003720 tx=0x7fbd840047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07325054d3cf437 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:50.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.040+0000 7fbd717fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:00:50.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.040+0000 7fbd717fa700 1 -- 172.21.15.33:0/2595521186 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd84004a30 con 0x55ff1fee82f0 2024-05-15T14:00:50.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.040+0000 7fbd717fa700 1 -- 172.21.15.33:0/2595521186 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fbd84004bd0 con 0x55ff1fee82f0 2024-05-15T14:00:50.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.040+0000 7fbd9a714b80 1 -- 172.21.15.33:0/2595521186 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd80020070 con 0x55ff1fee82f0 2024-05-15T14:00:50.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.040+0000 7fbd717fa700 1 -- 172.21.15.33:0/2595521186 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fbd84004dd0 con 0x55ff1fee82f0 2024-05-15T14:00:50.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.040+0000 7fbd9a714b80 10 client.15345 fetch_fsmap learned FSMap version 24 2024-05-15T14:00:50.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.040+0000 7fbd9a714b80 1 -- 172.21.15.33:0/2595521186 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ff1feff230 con 0x55ff1fee82f0 2024-05-15T14:00:50.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.041+0000 7fbd717fa700 1 -- 172.21.15.33:0/2595521186 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd80020070 con 0x55ff1fee82f0 2024-05-15T14:00:50.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.041+0000 7fbd717fa700 1 -- 172.21.15.33:0/2595521186 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fbd8401f9b0 con 0x55ff1fee82f0 2024-05-15T14:00:50.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.041+0000 7fbd9a714b80 10 client.15345 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:00:50.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.041+0000 7fbd9a714b80 10 client.15345 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-15T14:00:50.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.041+0000 7fbd9a714b80 20 client.15345 populate_metadata read hostname 'smithi033' 2024-05-15T14:00:50.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.041+0000 7fbd9a714b80 1 --2- 172.21.15.33:0/2595521186 >> [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] conn(0x55ff1ff01850 0x55ff1ff02cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:50.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.041+0000 7fbd9a714b80 4 client.15345 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:00:50.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.041+0000 7fbd9a714b80 1 -- 172.21.15.33:0/2595521186 --> [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff1feb4310 con 0x55ff1ff01850 2024-05-15T14:00:50.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.041+0000 7fbd8d419700 1 --2- 172.21.15.33:0/2595521186 >> [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] conn(0x55ff1ff01850 0x55ff1ff02cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:50.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.041+0000 7fbd8d419700 1 --2- 172.21.15.33:0/2595521186 >> [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] conn(0x55ff1ff01850 0x55ff1ff02cc0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7fb38cf24b08a0d5 server_cookie=61c5bf29f3fd2b07 in_seq=0 out_seq=0 2024-05-15T14:00:50.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.041+0000 7fbd717fa700 10 client.15345 ms_handle_connect on v2:172.21.15.81:6836/744100239 2024-05-15T14:00:50.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.042+0000 7fbd717fa700 1 -- 172.21.15.33:0/2595521186 <== mds.1 v2:172.21.15.81:6836/744100239 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ff1feb4310 con 0x55ff1ff01850 2024-05-15T14:00:50.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.042+0000 7fbd717fa700 10 client.15345 handle_command_reply: tid=0 2024-05-15T14:00:50.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.042+0000 7fbd9a714b80 1 client.15345 shutdown 2024-05-15T14:00:50.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.043+0000 7fbd9a714b80 1 -- 172.21.15.33:0/2595521186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff1fee82f0 msgr2=0x55ff1fef3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:50.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.043+0000 7fbd9a714b80 1 --2- 172.21.15.33:0/2595521186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff1fee82f0 0x55ff1fef3d90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbd84003720 tx=0x7fbd840047c0 comp rx=0 tx=0).stop 2024-05-15T14:00:50.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.043+0000 7fbd9a714b80 1 -- 172.21.15.33:0/2595521186 shutdown_connections 2024-05-15T14:00:50.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.043+0000 7fbd9a714b80 1 --2- 172.21.15.33:0/2595521186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff1feec910 0x55ff1fef1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:50.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.043+0000 7fbd9a714b80 1 --2- 172.21.15.33:0/2595521186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff1fee82f0 0x55ff1fef3d90 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:50.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.043+0000 7fbd9a714b80 1 --2- 172.21.15.33:0/2595521186 >> [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] conn(0x55ff1ff01850 0x55ff1ff02cc0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:50.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.043+0000 7fbd9a714b80 1 --2- 172.21.15.33:0/2595521186 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff1fee8c10 0x55ff1fef1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:50.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.043+0000 7fbd717fa700 0 client.15345 ms_handle_reset on v2:172.21.15.81:6836/744100239 2024-05-15T14:00:50.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.043+0000 7fbd9a714b80 1 -- 172.21.15.33:0/2595521186 >> 172.21.15.33:0/2595521186 conn(0x55ff1fe13200 msgr2=0x55ff1feedde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:50.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.043+0000 7fbd9a714b80 1 -- 172.21.15.33:0/2595521186 shutdown_connections 2024-05-15T14:00:50.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.043+0000 7fbd9a714b80 1 -- 172.21.15.33:0/2595521186 wait complete. 2024-05-15T14:00:50.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.043+0000 7fbd9a714b80 20 client.15345 trim_cache size 0 max 16384 2024-05-15T14:00:50.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.064+0000 7fbd9a714b80 1 Processor -- start 2024-05-15T14:00:50.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.064+0000 7fbd9a714b80 1 -- start start 2024-05-15T14:00:50.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.064+0000 7fbd9a714b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff1fee82f0 0x55ff1fedcc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:50.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.064+0000 7fbd9a714b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff1fee8c10 0x55ff1feea340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:50.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.064+0000 7fbd9a714b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff1feec910 0x55ff1feea880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:50.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.064+0000 7fbd9a714b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ff1ff01000 con 0x55ff1feec910 2024-05-15T14:00:50.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.064+0000 7fbd9a714b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ff1ff016d0 con 0x55ff1fee82f0 2024-05-15T14:00:50.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.064+0000 7fbd9a714b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55ff1ff01da0 con 0x55ff1fee8c10 2024-05-15T14:00:50.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.064+0000 7fbd8e41b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff1feec910 0x55ff1feea880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:50.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.064+0000 7fbd8e41b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff1feec910 0x55ff1feea880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58752/0 (socket says 172.21.15.33:58752) 2024-05-15T14:00:50.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.064+0000 7fbd8e41b700 1 -- 172.21.15.33:0/1074502302 learned_addr learned my addr 172.21.15.33:0/1074502302 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:50.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.065+0000 7fbd8d419700 1 --2- 172.21.15.33:0/1074502302 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff1fee8c10 0x55ff1feea340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:50.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.065+0000 7fbd8dc1a700 1 --2- 172.21.15.33:0/1074502302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff1fee82f0 0x55ff1fedcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:50.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.065+0000 7fbd8e41b700 1 -- 172.21.15.33:0/1074502302 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff1fee8c10 msgr2=0x55ff1feea340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:50.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.065+0000 7fbd8e41b700 1 --2- 172.21.15.33:0/1074502302 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff1fee8c10 0x55ff1feea340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:50.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.065+0000 7fbd8e41b700 1 -- 172.21.15.33:0/1074502302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff1fee82f0 msgr2=0x55ff1fedcc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:50.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.065+0000 7fbd8e41b700 1 --2- 172.21.15.33:0/1074502302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff1fee82f0 0x55ff1fedcc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:50.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.065+0000 7fbd8e41b700 1 -- 172.21.15.33:0/1074502302 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff1fef19f0 con 0x55ff1feec910 2024-05-15T14:00:50.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.065+0000 7fbd8e41b700 1 --2- 172.21.15.33:0/1074502302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff1feec910 0x55ff1feea880 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbd8000b420 tx=0x7fbd80006190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ace004829b18b9b server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:50.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.065+0000 7fbd737fe700 1 -- 172.21.15.33:0/1074502302 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8000bca0 con 0x55ff1feec910 2024-05-15T14:00:50.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.065+0000 7fbd737fe700 1 -- 172.21.15.33:0/1074502302 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd80029070 con 0x55ff1feec910 2024-05-15T14:00:50.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.065+0000 7fbd737fe700 1 -- 172.21.15.33:0/1074502302 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd80005830 con 0x55ff1feec910 2024-05-15T14:00:50.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.066+0000 7fbd9a714b80 1 -- 172.21.15.33:0/1074502302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff1feec910 msgr2=0x55ff1feea880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:50.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.066+0000 7fbd9a714b80 1 --2- 172.21.15.33:0/1074502302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff1feec910 0x55ff1feea880 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbd8000b420 tx=0x7fbd80006190 comp rx=0 tx=0).stop 2024-05-15T14:00:50.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.066+0000 7fbd9a714b80 1 -- 172.21.15.33:0/1074502302 shutdown_connections 2024-05-15T14:00:50.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.066+0000 7fbd9a714b80 1 --2- 172.21.15.33:0/1074502302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff1feec910 0x55ff1feea880 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:50.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.066+0000 7fbd9a714b80 1 --2- 172.21.15.33:0/1074502302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff1fee82f0 0x55ff1fedcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:50.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.066+0000 7fbd9a714b80 1 --2- 172.21.15.33:0/1074502302 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff1fee8c10 0x55ff1feea340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:50.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.066+0000 7fbd9a714b80 1 -- 172.21.15.33:0/1074502302 >> 172.21.15.33:0/1074502302 conn(0x55ff1fe13200 msgr2=0x55ff1fedc530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:50.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.066+0000 7fbd9a714b80 1 -- 172.21.15.33:0/1074502302 shutdown_connections 2024-05-15T14:00:50.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.066+0000 7fbd9a714b80 1 -- 172.21.15.33:0/1074502302 wait complete. 2024-05-15T14:00:50.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.066+0000 7fbd9a714b80 1 -- start start 2024-05-15T14:00:50.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.066+0000 7fbd9a714b80 1 Processor -- start 2024-05-15T14:00:50.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.067+0000 7fbd9a714b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff1fee82f0 0x55ff1ff052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:50.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.067+0000 7fbd9a714b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff1fee8c10 0x55ff1ff05800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:50.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.067+0000 7fbd9a714b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff1feec910 0x55ff1ff0a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:50.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.067+0000 7fbd9a714b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ff1ff02b40 con 0x55ff1feec910 2024-05-15T14:00:50.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.067+0000 7fbd9a714b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ff1ff02470 con 0x55ff1fee8c10 2024-05-15T14:00:50.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.067+0000 7fbd9a714b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55ff1ff03210 con 0x55ff1fee82f0 2024-05-15T14:00:50.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.067+0000 7fbd9a714b80 10 client.0 _ll_register_callbacks cb 0x55ff1fdcd530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:00:50.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.067+0000 7fbd8dc1a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff1fee82f0 0x55ff1ff052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:50.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.067+0000 7fbd8e41b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff1feec910 0x55ff1ff0a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:50.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.067+0000 7fbd8dc1a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff1fee82f0 0x55ff1ff052c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:39570/0 (socket says 172.21.15.33:39570) 2024-05-15T14:00:50.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.067+0000 7fbd8dc1a700 1 -- 172.21.15.33:0/3319927420 learned_addr learned my addr 172.21.15.33:0/3319927420 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:50.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.067+0000 7fbd8d419700 1 --2- 172.21.15.33:0/3319927420 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff1fee8c10 0x55ff1ff05800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:50.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.067+0000 7fbd8dc1a700 1 -- 172.21.15.33:0/3319927420 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff1fee8c10 msgr2=0x55ff1ff05800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:50.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.067+0000 7fbd8dc1a700 1 --2- 172.21.15.33:0/3319927420 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff1fee8c10 0x55ff1ff05800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:50.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.067+0000 7fbd8dc1a700 1 -- 172.21.15.33:0/3319927420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff1feec910 msgr2=0x55ff1ff0a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:50.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.067+0000 7fbd8dc1a700 1 --2- 172.21.15.33:0/3319927420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff1feec910 0x55ff1ff0a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:50.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.067+0000 7fbd8dc1a700 1 -- 172.21.15.33:0/3319927420 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff1ff0a770 con 0x55ff1fee82f0 2024-05-15T14:00:50.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.067+0000 7fbd8e41b700 1 --2- 172.21.15.33:0/3319927420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff1feec910 0x55ff1ff0a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:00:50.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.068+0000 7fbd8dc1a700 1 --2- 172.21.15.33:0/3319927420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff1fee82f0 0x55ff1ff052c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbd84009340 tx=0x7fbd84014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=189c8241b3e5c7fa server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:50.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.068+0000 7fbd717fa700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-05-15T14:00:50.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.068+0000 7fbd717fa700 1 -- 172.21.15.33:0/3319927420 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd84036040 con 0x55ff1fee82f0 2024-05-15T14:00:50.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.068+0000 7fbd9a714b80 1 -- 172.21.15.33:0/3319927420 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd80029070 con 0x55ff1fee82f0 2024-05-15T14:00:50.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.068+0000 7fbd717fa700 1 -- 172.21.15.33:0/3319927420 <== mon.1 v2:172.21.15.81:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fbd84030a90 con 0x55ff1fee82f0 2024-05-15T14:00:50.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.068+0000 7fbd717fa700 1 -- 172.21.15.33:0/3319927420 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fbd84030c90 con 0x55ff1fee82f0 2024-05-15T14:00:50.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.068+0000 7fbd9a714b80 10 client.25054 fetch_fsmap learned FSMap version 24 2024-05-15T14:00:50.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.068+0000 7fbd9a714b80 1 -- 172.21.15.33:0/3319927420 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ff1ff0baa0 con 0x55ff1fee82f0 2024-05-15T14:00:50.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.068+0000 7fbd717fa700 1 -- 172.21.15.33:0/3319927420 <== mon.1 v2:172.21.15.81:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd80029070 con 0x55ff1fee82f0 2024-05-15T14:00:50.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.068+0000 7fbd717fa700 1 -- 172.21.15.33:0/3319927420 <== mon.1 v2:172.21.15.81:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fbd84004f70 con 0x55ff1fee82f0 2024-05-15T14:00:50.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.068+0000 7fbd9a714b80 10 client.25054 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:00:50.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.069+0000 7fbd9a714b80 10 client.25054 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-15T14:00:50.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.069+0000 7fbd9a714b80 20 client.25054 populate_metadata read hostname 'smithi033' 2024-05-15T14:00:50.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.069+0000 7fbd9a714b80 1 --2- 172.21.15.33:0/3319927420 >> [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] conn(0x55ff1ff72650 0x55ff1ff71360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:50.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.069+0000 7fbd9a714b80 4 client.25054 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-15T14:00:50.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.069+0000 7fbd9a714b80 1 -- 172.21.15.33:0/3319927420 --> [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] -- command(tid 0: {"prefix": "status"}) -- 0x55ff1fe48b90 con 0x55ff1ff72650 2024-05-15T14:00:50.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.069+0000 7fbd8d419700 1 --2- 172.21.15.33:0/3319927420 >> [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] conn(0x55ff1ff72650 0x55ff1ff71360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:50.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.069+0000 7fbd8d419700 1 --2- 172.21.15.33:0/3319927420 >> [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] conn(0x55ff1ff72650 0x55ff1ff71360 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a154708257fdbe40 server_cookie=bd968fc159186218 in_seq=0 out_seq=0 2024-05-15T14:00:50.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.069+0000 7fbd717fa700 10 client.25054 ms_handle_connect on v2:172.21.15.81:6836/744100239 2024-05-15T14:00:50.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.070+0000 7fbd717fa700 1 -- 172.21.15.33:0/3319927420 <== mds.1 v2:172.21.15.81:6836/744100239 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ff1fe48b90 con 0x55ff1ff72650 2024-05-15T14:00:50.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.070+0000 7fbd717fa700 10 client.25054 handle_command_reply: tid=0 2024-05-15T14:00:50.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.070+0000 7fbd9a714b80 1 client.25054 shutdown 2024-05-15T14:00:50.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.070+0000 7fbd9a714b80 1 -- 172.21.15.33:0/3319927420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff1fee82f0 msgr2=0x55ff1ff052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:50.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.070+0000 7fbd9a714b80 1 --2- 172.21.15.33:0/3319927420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff1fee82f0 0x55ff1ff052c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbd84009340 tx=0x7fbd84014000 comp rx=0 tx=0).stop 2024-05-15T14:00:50.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.070+0000 7fbd9a714b80 1 -- 172.21.15.33:0/3319927420 shutdown_connections 2024-05-15T14:00:50.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.070+0000 7fbd9a714b80 1 --2- 172.21.15.33:0/3319927420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff1feec910 0x55ff1ff0a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:50.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.070+0000 7fbd9a714b80 1 --2- 172.21.15.33:0/3319927420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff1fee82f0 0x55ff1ff052c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:50.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.070+0000 7fbd9a714b80 1 --2- 172.21.15.33:0/3319927420 >> [v2:172.21.15.81:6836/744100239,v1:172.21.15.81:6837/744100239] conn(0x55ff1ff72650 0x55ff1ff71360 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:50.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.070+0000 7fbd9a714b80 1 --2- 172.21.15.33:0/3319927420 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ff1fee8c10 0x55ff1ff05800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:50.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.070+0000 7fbd717fa700 0 client.25054 ms_handle_reset on v2:172.21.15.81:6836/744100239 2024-05-15T14:00:50.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.070+0000 7fbd9a714b80 1 -- 172.21.15.33:0/3319927420 >> 172.21.15.33:0/3319927420 conn(0x55ff1fe13200 msgr2=0x55ff1fef1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:50.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.070+0000 7fbd9a714b80 1 -- 172.21.15.33:0/3319927420 shutdown_connections 2024-05-15T14:00:50.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.070+0000 7fbd9a714b80 1 -- 172.21.15.33:0/3319927420 wait complete. 2024-05-15T14:00:50.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.070+0000 7fbd9a714b80 20 client.25054 trim_cache size 0 max 16384 2024-05-15T14:00:50.078 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T14:00:50.078 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "bca78ca4-12c1-11ef-bc98-c7b262605968", 2024-05-15T14:00:50.078 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 1, 2024-05-15T14:00:50.078 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24463, 2024-05-15T14:00:50.078 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-05-15T14:00:50.078 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-05-15T14:00:50.078 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-05-15T14:00:50.078 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 198.59562007700001, 2024-05-15T14:00:50.078 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 24, 2024-05-15T14:00:50.079 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 83, 2024-05-15T14:00:50.079 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 82, 2024-05-15T14:00:50.079 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 241.819558234 2024-05-15T14:00:50.079 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T14:00:50.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.071+0000 7fbd8fe7e700 1 -- 172.21.15.33:0/2747759166 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbd740b2a80 msgr2=0x7fbd740b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:50.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.071+0000 7fbd8fe7e700 1 --2- 172.21.15.33:0/2747759166 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbd740b2a80 0x7fbd740b4f40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbd84009c60 tx=0x7fbd84009260 comp rx=0 tx=0).stop 2024-05-15T14:00:50.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.071+0000 7fbd8fe7e700 1 -- 172.21.15.33:0/2747759166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd881041a0 msgr2=0x7fbd88075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:50.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.071+0000 7fbd8fe7e700 1 --2- 172.21.15.33:0/2747759166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd881041a0 0x7fbd88075d80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbd7800c920 tx=0x7fbd7800cc30 comp rx=0 tx=0).stop 2024-05-15T14:00:50.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.071+0000 7fbd8fe7e700 1 -- 172.21.15.33:0/2747759166 shutdown_connections 2024-05-15T14:00:50.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.071+0000 7fbd8fe7e700 1 --2- 172.21.15.33:0/2747759166 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbd740b2a80 0x7fbd740b4f40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:50.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.071+0000 7fbd8fe7e700 1 --2- 172.21.15.33:0/2747759166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd881088f0 0x7fbd880762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:50.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.071+0000 7fbd8fe7e700 1 --2- 172.21.15.33:0/2747759166 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd881037f0 0x7fbd88075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:50.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.071+0000 7fbd8fe7e700 1 --2- 172.21.15.33:0/2747759166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd881041a0 0x7fbd88075d80 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:50.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.071+0000 7fbd8fe7e700 1 -- 172.21.15.33:0/2747759166 >> 172.21.15.33:0/2747759166 conn(0x7fbd8807a6a0 msgr2=0x7fbd881016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:50.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.071+0000 7fbd8fe7e700 1 -- 172.21.15.33:0/2747759166 shutdown_connections 2024-05-15T14:00:50.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:50.071+0000 7fbd8fe7e700 1 -- 172.21.15.33:0/2747759166 wait complete. 2024-05-15T14:00:50.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:50 smithi033 ceph-mon[31650]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-05-15T14:00:50.749 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph tell mds.i status 2024-05-15T14:00:50.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:50 smithi052 ceph-mon[36190]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-05-15T14:00:50.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:50 smithi081 ceph-mon[36434]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-05-15T14:00:50.999 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:00:52.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.579+0000 7f179e125700 1 -- 172.21.15.33:0/1305813923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1798102e30 msgr2=0x7f179810f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:52.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.579+0000 7f179e125700 1 --2- 172.21.15.33:0/1305813923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1798102e30 0x7f179810f2f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f179000b0a0 tx=0x7f179000b3b0 comp rx=0 tx=0).stop 2024-05-15T14:00:52.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.579+0000 7f179e125700 1 -- 172.21.15.33:0/1305813923 shutdown_connections 2024-05-15T14:00:52.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.579+0000 7f179e125700 1 --2- 172.21.15.33:0/1305813923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1798102e30 0x7f179810f2f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:52.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.579+0000 7f179e125700 1 --2- 172.21.15.33:0/1305813923 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17981084b0 0x7f1798108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:52.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.579+0000 7f179e125700 1 --2- 172.21.15.33:0/1305813923 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1798102470 0x7f17981028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:52.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.579+0000 7f179e125700 1 -- 172.21.15.33:0/1305813923 >> 172.21.15.33:0/1305813923 conn(0x7f17980793f0 msgr2=0x7f1798079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:52.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.579+0000 7f179e125700 1 -- 172.21.15.33:0/1305813923 shutdown_connections 2024-05-15T14:00:52.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.579+0000 7f179e125700 1 -- 172.21.15.33:0/1305813923 wait complete. 2024-05-15T14:00:52.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.579+0000 7f179e125700 1 Processor -- start 2024-05-15T14:00:52.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.580+0000 7f179e125700 1 -- start start 2024-05-15T14:00:52.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.580+0000 7f179e125700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1798102470 0x7f179819f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:52.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.580+0000 7f179e125700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1798102e30 0x7f179819fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:52.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.580+0000 7f179e125700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17981084b0 0x7f1798199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:52.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.580+0000 7f179e125700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f17981125f0 con 0x7f1798102470 2024-05-15T14:00:52.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.580+0000 7f179e125700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1798112470 con 0x7f1798102e30 2024-05-15T14:00:52.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.580+0000 7f179e125700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1798112770 con 0x7f17981084b0 2024-05-15T14:00:52.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.580+0000 7f17977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1798102470 0x7f179819f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:52.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.580+0000 7f1797fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17981084b0 0x7f1798199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:52.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.580+0000 7f17977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1798102470 0x7f179819f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58796/0 (socket says 172.21.15.33:58796) 2024-05-15T14:00:52.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.580+0000 7f17977fe700 1 -- 172.21.15.33:0/2656756610 learned_addr learned my addr 172.21.15.33:0/2656756610 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:52.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.580+0000 7f1796ffd700 1 --2- 172.21.15.33:0/2656756610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1798102e30 0x7f179819fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:52.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.580+0000 7f17977fe700 1 -- 172.21.15.33:0/2656756610 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17981084b0 msgr2=0x7f1798199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:52.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.580+0000 7f17977fe700 1 --2- 172.21.15.33:0/2656756610 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17981084b0 0x7f1798199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:52.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.580+0000 7f17977fe700 1 -- 172.21.15.33:0/2656756610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1798102e30 msgr2=0x7f179819fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:52.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.580+0000 7f17977fe700 1 --2- 172.21.15.33:0/2656756610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1798102e30 0x7f179819fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:52.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.580+0000 7f17977fe700 1 -- 172.21.15.33:0/2656756610 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f179819a180 con 0x7f1798102470 2024-05-15T14:00:52.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.580+0000 7f1797fff700 1 --2- 172.21.15.33:0/2656756610 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17981084b0 0x7f1798199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:00:52.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.581+0000 7f17977fe700 1 --2- 172.21.15.33:0/2656756610 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1798102470 0x7f179819f810 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f178800d980 tx=0x7f178800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:52.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.581+0000 7f1794ff9700 1 -- 172.21.15.33:0/2656756610 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788016070 con 0x7f1798102470 2024-05-15T14:00:52.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.581+0000 7f179e125700 1 -- 172.21.15.33:0/2656756610 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f179819a470 con 0x7f1798102470 2024-05-15T14:00:52.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.581+0000 7f1794ff9700 1 -- 172.21.15.33:0/2656756610 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1788004830 con 0x7f1798102470 2024-05-15T14:00:52.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.581+0000 7f1794ff9700 1 -- 172.21.15.33:0/2656756610 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17880054c0 con 0x7f1798102470 2024-05-15T14:00:52.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.581+0000 7f179e125700 1 -- 172.21.15.33:0/2656756610 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17981a66e0 con 0x7f1798102470 2024-05-15T14:00:52.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.583+0000 7f1794ff9700 1 -- 172.21.15.33:0/2656756610 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f1788004220 con 0x7f1798102470 2024-05-15T14:00:52.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.584+0000 7f1794ff9700 1 --2- 172.21.15.33:0/2656756610 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f17800b2a80 0x7f17800b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:52.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.584+0000 7f1796ffd700 1 --2- 172.21.15.33:0/2656756610 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f17800b2a80 0x7f17800b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:52.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.584+0000 7f1794ff9700 1 -- 172.21.15.33:0/2656756610 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f17880e2580 con 0x7f1798102470 2024-05-15T14:00:52.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.585+0000 7f1796ffd700 1 --2- 172.21.15.33:0/2656756610 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f17800b2a80 0x7f17800b4f40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f179806cd50 tx=0x7f178c009260 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:52.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.585+0000 7f17a89bbb80 1 Processor -- start 2024-05-15T14:00:52.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.585+0000 7f17a89bbb80 1 -- start start 2024-05-15T14:00:52.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.585+0000 7f17a89bbb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce75659fa0 0x55ce7565a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:52.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.585+0000 7f17a89bbb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce7565a8c0 0x55ce756547a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:52.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.585+0000 7f17a89bbb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce75654dc0 0x55ce75655220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:52.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.585+0000 7f17a89bbb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ce755b2d20 con 0x55ce75659fa0 2024-05-15T14:00:52.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.585+0000 7f17a89bbb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ce755a28c0 con 0x55ce75654dc0 2024-05-15T14:00:52.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.585+0000 7f17a89bbb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55ce755a6900 con 0x55ce7565a8c0 2024-05-15T14:00:52.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.585+0000 7f17977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce75659fa0 0x55ce7565a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:52.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.585+0000 7f17977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce75659fa0 0x55ce7565a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58800/0 (socket says 172.21.15.33:58800) 2024-05-15T14:00:52.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.585+0000 7f17977fe700 1 -- 172.21.15.33:0/2450692755 learned_addr learned my addr 172.21.15.33:0/2450692755 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:52.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.585+0000 7f1796ffd700 1 --2- 172.21.15.33:0/2450692755 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce7565a8c0 0x55ce756547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:52.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.585+0000 7f17977fe700 1 -- 172.21.15.33:0/2450692755 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce7565a8c0 msgr2=0x55ce756547a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:52.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.585+0000 7f1797fff700 1 --2- 172.21.15.33:0/2450692755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce75654dc0 0x55ce75655220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:52.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.585+0000 7f17977fe700 1 --2- 172.21.15.33:0/2450692755 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce7565a8c0 0x55ce756547a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:52.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.585+0000 7f17977fe700 1 -- 172.21.15.33:0/2450692755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce75654dc0 msgr2=0x55ce75655220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:52.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.585+0000 7f17977fe700 1 --2- 172.21.15.33:0/2450692755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce75654dc0 0x55ce75655220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:52.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.585+0000 7f17977fe700 1 -- 172.21.15.33:0/2450692755 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce7554bf60 con 0x55ce75659fa0 2024-05-15T14:00:52.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.586+0000 7f17977fe700 1 --2- 172.21.15.33:0/2450692755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce75659fa0 0x55ce7565a380 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f178800b540 tx=0x7f1788004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ca98a1b8adcc7e server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:52.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.586+0000 7f177dffb700 1 -- 172.21.15.33:0/2450692755 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17880a9700 con 0x55ce75659fa0 2024-05-15T14:00:52.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.586+0000 7f177dffb700 1 -- 172.21.15.33:0/2450692755 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17880adab0 con 0x55ce75659fa0 2024-05-15T14:00:52.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.586+0000 7f177dffb700 1 -- 172.21.15.33:0/2450692755 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17880a9960 con 0x55ce75659fa0 2024-05-15T14:00:52.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.587+0000 7f17a89bbb80 1 -- 172.21.15.33:0/2450692755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce75659fa0 msgr2=0x55ce7565a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:52.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.587+0000 7f17a89bbb80 1 --2- 172.21.15.33:0/2450692755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce75659fa0 0x55ce7565a380 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f178800b540 tx=0x7f1788004660 comp rx=0 tx=0).stop 2024-05-15T14:00:52.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.587+0000 7f17a89bbb80 1 -- 172.21.15.33:0/2450692755 shutdown_connections 2024-05-15T14:00:52.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.587+0000 7f17a89bbb80 1 --2- 172.21.15.33:0/2450692755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce75654dc0 0x55ce75655220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:52.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.587+0000 7f17a89bbb80 1 --2- 172.21.15.33:0/2450692755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce75659fa0 0x55ce7565a380 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:52.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.587+0000 7f17a89bbb80 1 --2- 172.21.15.33:0/2450692755 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce7565a8c0 0x55ce756547a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:52.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.587+0000 7f17a89bbb80 1 -- 172.21.15.33:0/2450692755 >> 172.21.15.33:0/2450692755 conn(0x55ce75581d60 msgr2=0x55ce7564ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:52.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.587+0000 7f17a89bbb80 1 -- 172.21.15.33:0/2450692755 shutdown_connections 2024-05-15T14:00:52.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.587+0000 7f17a89bbb80 1 -- 172.21.15.33:0/2450692755 wait complete. 2024-05-15T14:00:52.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.587+0000 7f17a89bbb80 1 -- start start 2024-05-15T14:00:52.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.587+0000 7f17a89bbb80 1 Processor -- start 2024-05-15T14:00:52.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.588+0000 7f17a89bbb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce75654dc0 0x55ce756605e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:52.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.588+0000 7f17a89bbb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce75659fa0 0x55ce7565d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:52.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.588+0000 7f17a89bbb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce7565a8c0 0x55ce7565dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:52.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.588+0000 7f17a89bbb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ce754a8f50 con 0x55ce7565a8c0 2024-05-15T14:00:52.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.588+0000 7f17a89bbb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ce7550ad50 con 0x55ce75654dc0 2024-05-15T14:00:52.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.588+0000 7f17a89bbb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55ce756612d0 con 0x55ce75659fa0 2024-05-15T14:00:52.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.588+0000 7f17a89bbb80 10 client.0 _ll_register_callbacks cb 0x55ce7553d900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:00:52.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.588+0000 7f1797fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce7565a8c0 0x55ce7565dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:52.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.588+0000 7f1796ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce75659fa0 0x55ce7565d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:52.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.588+0000 7f1797fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce7565a8c0 0x55ce7565dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58808/0 (socket says 172.21.15.33:58808) 2024-05-15T14:00:52.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.588+0000 7f1796ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce75659fa0 0x55ce7565d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:39610/0 (socket says 172.21.15.33:39610) 2024-05-15T14:00:52.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.588+0000 7f1797fff700 1 -- 172.21.15.33:0/925887763 learned_addr learned my addr 172.21.15.33:0/925887763 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:52.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.588+0000 7f17977fe700 1 --2- 172.21.15.33:0/925887763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce75654dc0 0x55ce756605e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:52.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.588+0000 7f1797fff700 1 -- 172.21.15.33:0/925887763 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce75659fa0 msgr2=0x55ce7565d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:52.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.588+0000 7f1797fff700 1 --2- 172.21.15.33:0/925887763 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce75659fa0 0x55ce7565d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:52.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.588+0000 7f1797fff700 1 -- 172.21.15.33:0/925887763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce75654dc0 msgr2=0x55ce756605e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:52.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.588+0000 7f1797fff700 1 --2- 172.21.15.33:0/925887763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce75654dc0 0x55ce756605e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:52.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.588+0000 7f1797fff700 1 -- 172.21.15.33:0/925887763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce754f2880 con 0x55ce7565a8c0 2024-05-15T14:00:52.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.588+0000 7f1797fff700 1 --2- 172.21.15.33:0/925887763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce7565a8c0 0x55ce7565dec0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1790004c60 tx=0x7f1790004c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aacb91c20a712476 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:52.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.588+0000 7f1776ffd700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:00:52.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.589+0000 7f1776ffd700 1 -- 172.21.15.33:0/925887763 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1790021070 con 0x55ce7565a8c0 2024-05-15T14:00:52.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.589+0000 7f1776ffd700 1 -- 172.21.15.33:0/925887763 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f17900076e0 con 0x55ce7565a8c0 2024-05-15T14:00:52.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.589+0000 7f17a89bbb80 1 -- 172.21.15.33:0/925887763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17880adab0 con 0x55ce7565a8c0 2024-05-15T14:00:52.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.589+0000 7f1776ffd700 1 -- 172.21.15.33:0/925887763 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f17900078e0 con 0x55ce7565a8c0 2024-05-15T14:00:52.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.589+0000 7f17a89bbb80 10 client.15363 fetch_fsmap learned FSMap version 24 2024-05-15T14:00:52.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.589+0000 7f17a89bbb80 1 -- 172.21.15.33:0/925887763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ce7555c710 con 0x55ce7565a8c0 2024-05-15T14:00:52.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.589+0000 7f1776ffd700 1 -- 172.21.15.33:0/925887763 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17880adab0 con 0x55ce7565a8c0 2024-05-15T14:00:52.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.589+0000 7f1776ffd700 1 -- 172.21.15.33:0/925887763 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f1790011070 con 0x55ce7565a8c0 2024-05-15T14:00:52.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.589+0000 7f17a89bbb80 10 client.15363 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:00:52.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.589+0000 7f17a89bbb80 10 client.15363 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-15T14:00:52.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.589+0000 7f17a89bbb80 20 client.15363 populate_metadata read hostname 'smithi033' 2024-05-15T14:00:52.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.589+0000 7f17a89bbb80 1 --2- 172.21.15.33:0/925887763 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55ce7566df00 0x55ce7566f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:52.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.589+0000 7f17a89bbb80 4 client.15363 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:00:52.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.589+0000 7f17a89bbb80 1 -- 172.21.15.33:0/925887763 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce755b4c70 con 0x55ce7566df00 2024-05-15T14:00:52.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.590+0000 7f17977fe700 1 --2- 172.21.15.33:0/925887763 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55ce7566df00 0x55ce7566f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:52.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.590+0000 7f17977fe700 1 --2- 172.21.15.33:0/925887763 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55ce7566df00 0x55ce7566f370 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ff6ab4996f5dad6 server_cookie=b3e7e9e42dbc0da2 in_seq=0 out_seq=0 2024-05-15T14:00:52.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.590+0000 7f1776ffd700 10 client.15363 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:00:52.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.590+0000 7f1776ffd700 1 -- 172.21.15.33:0/925887763 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ce755b4c70 con 0x55ce7566df00 2024-05-15T14:00:52.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.590+0000 7f1776ffd700 10 client.15363 handle_command_reply: tid=0 2024-05-15T14:00:52.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.590+0000 7f17a89bbb80 1 client.15363 shutdown 2024-05-15T14:00:52.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.591+0000 7f17a89bbb80 1 -- 172.21.15.33:0/925887763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce7565a8c0 msgr2=0x55ce7565dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:52.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.591+0000 7f17a89bbb80 1 --2- 172.21.15.33:0/925887763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce7565a8c0 0x55ce7565dec0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1790004c60 tx=0x7f1790004c90 comp rx=0 tx=0).stop 2024-05-15T14:00:52.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.591+0000 7f17a89bbb80 1 -- 172.21.15.33:0/925887763 shutdown_connections 2024-05-15T14:00:52.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.591+0000 7f17a89bbb80 1 --2- 172.21.15.33:0/925887763 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55ce7566df00 0x55ce7566f370 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:52.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.591+0000 7f17a89bbb80 1 --2- 172.21.15.33:0/925887763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce7565a8c0 0x55ce7565dec0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:52.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.591+0000 7f17a89bbb80 1 --2- 172.21.15.33:0/925887763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce75654dc0 0x55ce756605e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:52.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.591+0000 7f17a89bbb80 1 --2- 172.21.15.33:0/925887763 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce75659fa0 0x55ce7565d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:52.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.591+0000 7f1776ffd700 0 client.15363 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:00:52.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.591+0000 7f17a89bbb80 1 -- 172.21.15.33:0/925887763 >> 172.21.15.33:0/925887763 conn(0x55ce75581d60 msgr2=0x55ce7564e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:52.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.591+0000 7f17a89bbb80 1 -- 172.21.15.33:0/925887763 shutdown_connections 2024-05-15T14:00:52.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.591+0000 7f17a89bbb80 1 -- 172.21.15.33:0/925887763 wait complete. 2024-05-15T14:00:52.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.591+0000 7f17a89bbb80 20 client.15363 trim_cache size 0 max 16384 2024-05-15T14:00:52.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.612+0000 7f17a89bbb80 1 Processor -- start 2024-05-15T14:00:52.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.612+0000 7f17a89bbb80 1 -- start start 2024-05-15T14:00:52.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.612+0000 7f17a89bbb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce75654dc0 0x55ce7565fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:52.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.612+0000 7f17a89bbb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce75659fa0 0x55ce7564edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:52.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.612+0000 7f17a89bbb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce7565a8c0 0x55ce756787d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:52.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.612+0000 7f17a89bbb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ce7566d6b0 con 0x55ce75659fa0 2024-05-15T14:00:52.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.612+0000 7f17a89bbb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ce7566dd80 con 0x55ce75654dc0 2024-05-15T14:00:52.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.612+0000 7f17a89bbb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55ce7566e450 con 0x55ce7565a8c0 2024-05-15T14:00:52.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.612+0000 7f1796ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce75659fa0 0x55ce7564edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:52.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.612+0000 7f1796ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce75659fa0 0x55ce7564edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58822/0 (socket says 172.21.15.33:58822) 2024-05-15T14:00:52.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.612+0000 7f1796ffd700 1 -- 172.21.15.33:0/2504814243 learned_addr learned my addr 172.21.15.33:0/2504814243 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:52.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.612+0000 7f1797fff700 1 --2- 172.21.15.33:0/2504814243 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce7565a8c0 0x55ce756787d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:52.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.612+0000 7f17977fe700 1 --2- 172.21.15.33:0/2504814243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce75654dc0 0x55ce7565fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:52.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.612+0000 7f1796ffd700 1 -- 172.21.15.33:0/2504814243 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce7565a8c0 msgr2=0x55ce756787d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:52.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.612+0000 7f1796ffd700 1 --2- 172.21.15.33:0/2504814243 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce7565a8c0 0x55ce756787d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:52.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.612+0000 7f1796ffd700 1 -- 172.21.15.33:0/2504814243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce75654dc0 msgr2=0x55ce7565fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:52.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.612+0000 7f1796ffd700 1 --2- 172.21.15.33:0/2504814243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce75654dc0 0x55ce7565fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:52.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.612+0000 7f1796ffd700 1 -- 172.21.15.33:0/2504814243 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce75650af0 con 0x55ce75659fa0 2024-05-15T14:00:52.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.613+0000 7f1796ffd700 1 --2- 172.21.15.33:0/2504814243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce75659fa0 0x55ce7564edb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f178c009ab0 tx=0x7f178c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=220e9d0b5296ac69 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:52.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.613+0000 7f177d7fa700 1 -- 172.21.15.33:0/2504814243 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c004110 con 0x55ce75659fa0 2024-05-15T14:00:52.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.613+0000 7f177d7fa700 1 -- 172.21.15.33:0/2504814243 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f178c012ce0 con 0x55ce75659fa0 2024-05-15T14:00:52.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.613+0000 7f177d7fa700 1 -- 172.21.15.33:0/2504814243 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c003ae0 con 0x55ce75659fa0 2024-05-15T14:00:52.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.613+0000 7f17a89bbb80 1 -- 172.21.15.33:0/2504814243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce75659fa0 msgr2=0x55ce7564edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:52.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.613+0000 7f17a89bbb80 1 --2- 172.21.15.33:0/2504814243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce75659fa0 0x55ce7564edb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f178c009ab0 tx=0x7f178c014000 comp rx=0 tx=0).stop 2024-05-15T14:00:52.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.613+0000 7f17a89bbb80 1 -- 172.21.15.33:0/2504814243 shutdown_connections 2024-05-15T14:00:52.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.613+0000 7f17a89bbb80 1 --2- 172.21.15.33:0/2504814243 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce7565a8c0 0x55ce756787d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:52.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.613+0000 7f17a89bbb80 1 --2- 172.21.15.33:0/2504814243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce75654dc0 0x55ce7565fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:52.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.613+0000 7f17a89bbb80 1 --2- 172.21.15.33:0/2504814243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce75659fa0 0x55ce7564edb0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:52.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.613+0000 7f17a89bbb80 1 -- 172.21.15.33:0/2504814243 >> 172.21.15.33:0/2504814243 conn(0x55ce75581d60 msgr2=0x55ce7565c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:52.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.614+0000 7f17a89bbb80 1 -- 172.21.15.33:0/2504814243 shutdown_connections 2024-05-15T14:00:52.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.614+0000 7f17a89bbb80 1 -- 172.21.15.33:0/2504814243 wait complete. 2024-05-15T14:00:52.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.614+0000 7f17a89bbb80 1 -- start start 2024-05-15T14:00:52.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.614+0000 7f17a89bbb80 1 Processor -- start 2024-05-15T14:00:52.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.614+0000 7f17a89bbb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce75654dc0 0x55ce75678ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:52.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.614+0000 7f17a89bbb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce75659fa0 0x55ce756737a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:52.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.614+0000 7f17a89bbb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce7565a8c0 0x55ce75673ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:52.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.614+0000 7f17a89bbb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ce7566f1f0 con 0x55ce75659fa0 2024-05-15T14:00:52.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.614+0000 7f17a89bbb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ce7566eb20 con 0x55ce75654dc0 2024-05-15T14:00:52.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.614+0000 7f17a89bbb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55ce7566f8c0 con 0x55ce7565a8c0 2024-05-15T14:00:52.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.614+0000 7f17a89bbb80 10 client.0 _ll_register_callbacks cb 0x55ce7553d900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:00:52.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.614+0000 7f1796ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce75659fa0 0x55ce756737a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:52.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.615+0000 7f1796ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce75659fa0 0x55ce756737a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58828/0 (socket says 172.21.15.33:58828) 2024-05-15T14:00:52.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.615+0000 7f1796ffd700 1 -- 172.21.15.33:0/1737453234 learned_addr learned my addr 172.21.15.33:0/1737453234 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:00:52.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.615+0000 7f1797fff700 1 --2- 172.21.15.33:0/1737453234 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce7565a8c0 0x55ce75673ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:52.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.615+0000 7f17977fe700 1 --2- 172.21.15.33:0/1737453234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce75654dc0 0x55ce75678ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:52.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.615+0000 7f1796ffd700 1 -- 172.21.15.33:0/1737453234 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce7565a8c0 msgr2=0x55ce75673ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:52.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.615+0000 7f1796ffd700 1 --2- 172.21.15.33:0/1737453234 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce7565a8c0 0x55ce75673ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:52.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.615+0000 7f1796ffd700 1 -- 172.21.15.33:0/1737453234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce75654dc0 msgr2=0x55ce75678ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:00:52.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.615+0000 7f1796ffd700 1 --2- 172.21.15.33:0/1737453234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce75654dc0 0x55ce75678ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:52.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.615+0000 7f1796ffd700 1 -- 172.21.15.33:0/1737453234 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce756742a0 con 0x55ce75659fa0 2024-05-15T14:00:52.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.615+0000 7f1796ffd700 1 --2- 172.21.15.33:0/1737453234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce75659fa0 0x55ce756737a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f178c0097d0 tx=0x7f178c004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e3ee9d98feeda6 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:00:52.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.615+0000 7f1776ffd700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:00:52.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.615+0000 7f1776ffd700 1 -- 172.21.15.33:0/1737453234 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c005450 con 0x55ce75659fa0 2024-05-15T14:00:52.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.615+0000 7f1776ffd700 1 -- 172.21.15.33:0/1737453234 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f178c0055f0 con 0x55ce75659fa0 2024-05-15T14:00:52.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.615+0000 7f17a89bbb80 1 -- 172.21.15.33:0/1737453234 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f178c012ce0 con 0x55ce75659fa0 2024-05-15T14:00:52.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.615+0000 7f1776ffd700 1 -- 172.21.15.33:0/1737453234 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f178c004750 con 0x55ce75659fa0 2024-05-15T14:00:52.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.616+0000 7f17a89bbb80 10 client.15369 fetch_fsmap learned FSMap version 24 2024-05-15T14:00:52.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.616+0000 7f17a89bbb80 1 -- 172.21.15.33:0/1737453234 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ce756da140 con 0x55ce75659fa0 2024-05-15T14:00:52.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.616+0000 7f1776ffd700 1 -- 172.21.15.33:0/1737453234 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f178c012ce0 con 0x55ce75659fa0 2024-05-15T14:00:52.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.616+0000 7f1776ffd700 1 -- 172.21.15.33:0/1737453234 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f178c003d30 con 0x55ce75659fa0 2024-05-15T14:00:52.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.616+0000 7f17a89bbb80 10 client.15369 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:00:52.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.616+0000 7f17a89bbb80 10 client.15369 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-15T14:00:52.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.616+0000 7f17a89bbb80 20 client.15369 populate_metadata read hostname 'smithi033' 2024-05-15T14:00:52.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.616+0000 7f17a89bbb80 1 --2- 172.21.15.33:0/1737453234 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55ce756de990 0x55ce756dd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:00:52.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.616+0000 7f17a89bbb80 4 client.15369 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-15T14:00:52.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.616+0000 7f17a89bbb80 1 -- 172.21.15.33:0/1737453234 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "status"}) -- 0x55ce755ae450 con 0x55ce756de990 2024-05-15T14:00:52.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.616+0000 7f17977fe700 1 --2- 172.21.15.33:0/1737453234 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55ce756de990 0x55ce756dd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:00:52.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.617+0000 7f1776ffd700 10 client.15369 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:00:52.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.617+0000 7f17977fe700 1 --2- 172.21.15.33:0/1737453234 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55ce756de990 0x55ce756dd6a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2e07fc296483a65 server_cookie=d853d979708a3d0 in_seq=0 out_seq=0 2024-05-15T14:00:52.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.617+0000 7f1776ffd700 1 -- 172.21.15.33:0/1737453234 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ce755ae450 con 0x55ce756de990 2024-05-15T14:00:52.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.617+0000 7f1776ffd700 10 client.15369 handle_command_reply: tid=0 2024-05-15T14:00:52.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.617+0000 7f17a89bbb80 1 client.15369 shutdown 2024-05-15T14:00:52.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.617+0000 7f17a89bbb80 1 -- 172.21.15.33:0/1737453234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce75659fa0 msgr2=0x55ce756737a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:52.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.617+0000 7f17a89bbb80 1 --2- 172.21.15.33:0/1737453234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce75659fa0 0x55ce756737a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f178c0097d0 tx=0x7f178c004340 comp rx=0 tx=0).stop 2024-05-15T14:00:52.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.617+0000 7f17a89bbb80 1 -- 172.21.15.33:0/1737453234 shutdown_connections 2024-05-15T14:00:52.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.617+0000 7f17a89bbb80 1 --2- 172.21.15.33:0/1737453234 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ce7565a8c0 0x55ce75673ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:52.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.617+0000 7f17a89bbb80 1 --2- 172.21.15.33:0/1737453234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ce75654dc0 0x55ce75678ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:52.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.617+0000 7f17a89bbb80 1 --2- 172.21.15.33:0/1737453234 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55ce756de990 0x55ce756dd6a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:52.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.617+0000 7f17a89bbb80 1 --2- 172.21.15.33:0/1737453234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ce75659fa0 0x55ce756737a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:52.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.617+0000 7f17a89bbb80 1 -- 172.21.15.33:0/1737453234 >> 172.21.15.33:0/1737453234 conn(0x55ce75581d60 msgr2=0x55ce7565aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:52.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.617+0000 7f1776ffd700 0 client.15369 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:00:52.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.617+0000 7f17a89bbb80 1 -- 172.21.15.33:0/1737453234 shutdown_connections 2024-05-15T14:00:52.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.617+0000 7f17a89bbb80 1 -- 172.21.15.33:0/1737453234 wait complete. 2024-05-15T14:00:52.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.617+0000 7f17a89bbb80 20 client.15369 trim_cache size 0 max 16384 2024-05-15T14:00:52.625 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T14:00:52.625 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "bca78ca4-12c1-11ef-bc98-c7b262605968", 2024-05-15T14:00:52.625 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2024-05-15T14:00:52.625 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24478, 2024-05-15T14:00:52.625 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-05-15T14:00:52.625 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-05-15T14:00:52.626 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-05-15T14:00:52.626 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 213.23810348200001, 2024-05-15T14:00:52.626 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 24, 2024-05-15T14:00:52.626 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 83, 2024-05-15T14:00:52.626 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 82, 2024-05-15T14:00:52.626 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 238.95319506300001 2024-05-15T14:00:52.626 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T14:00:52.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.618+0000 7f179e125700 1 -- 172.21.15.33:0/2656756610 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f17800b2a80 msgr2=0x7f17800b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:52.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.618+0000 7f179e125700 1 --2- 172.21.15.33:0/2656756610 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f17800b2a80 0x7f17800b4f40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f179806cd50 tx=0x7f178c009260 comp rx=0 tx=0).stop 2024-05-15T14:00:52.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.618+0000 7f179e125700 1 -- 172.21.15.33:0/2656756610 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1798102470 msgr2=0x7f179819f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:00:52.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.618+0000 7f179e125700 1 --2- 172.21.15.33:0/2656756610 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1798102470 0x7f179819f810 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f178800d980 tx=0x7f178800dc90 comp rx=0 tx=0).stop 2024-05-15T14:00:52.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.618+0000 7f179e125700 1 -- 172.21.15.33:0/2656756610 shutdown_connections 2024-05-15T14:00:52.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.618+0000 7f179e125700 1 --2- 172.21.15.33:0/2656756610 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f17800b2a80 0x7f17800b4f40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:52.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.618+0000 7f179e125700 1 --2- 172.21.15.33:0/2656756610 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17981084b0 0x7f1798199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:52.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.618+0000 7f179e125700 1 --2- 172.21.15.33:0/2656756610 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1798102470 0x7f179819f810 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:52.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.618+0000 7f179e125700 1 --2- 172.21.15.33:0/2656756610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1798102e30 0x7f179819fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:00:52.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.618+0000 7f179e125700 1 -- 172.21.15.33:0/2656756610 >> 172.21.15.33:0/2656756610 conn(0x7f17980793f0 msgr2=0x7f1798106300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:00:52.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.618+0000 7f179e125700 1 -- 172.21.15.33:0/2656756610 shutdown_connections 2024-05-15T14:00:52.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:00:52.618+0000 7f179e125700 1 -- 172.21.15.33:0/2656756610 wait complete. 2024-05-15T14:00:52.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:52 smithi052 ceph-mon[36190]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 853 B/s wr, 5 op/s 2024-05-15T14:00:52.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:52 smithi081 ceph-mon[36434]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 853 B/s wr, 5 op/s 2024-05-15T14:00:52.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:52 smithi033 ceph-mon[31650]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 853 B/s wr, 5 op/s 2024-05-15T14:00:53.358 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-05-15T14:00:53.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:53 smithi033 ceph-mon[31650]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-05-15T14:00:53.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:53 smithi052 ceph-mon[36190]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-05-15T14:00:53.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:53 smithi081 ceph-mon[36434]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-05-15T14:00:56.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:56 smithi052 ceph-mon[36190]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-15T14:00:56.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:56 smithi081 ceph-mon[36434]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-15T14:00:56.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:56 smithi033 ceph-mon[31650]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-05-15T14:00:58.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:00:58 smithi052 ceph-mon[36190]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-05-15T14:00:58.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:00:58 smithi081 ceph-mon[36434]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-05-15T14:00:58.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:00:58 smithi033 ceph-mon[31650]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2024-05-15T14:01:00.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:00 smithi052 ceph-mon[36190]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-15T14:01:00.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:00 smithi081 ceph-mon[36434]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-15T14:01:00.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:00 smithi033 ceph-mon[31650]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-15T14:01:00.941 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:01:00 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:01:00.685+0000 7f29b7943700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-15T14:01:00.941 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:01:00 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:01:00.685+0000 7f29b7943700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-15T14:01:00.941 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:01:00 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:01:00.685+0000 7f29b7943700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-15T14:01:00.941 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:01:00 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:01:00.685+0000 7f29b7943700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-15T14:01:00.941 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:01:00 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:01:00.685+0000 7f29b7943700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-15T14:01:02.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:02 smithi052 ceph-mon[36190]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-15T14:01:02.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:02 smithi052 ceph-mon[36190]: mgrmap e25: x(active, since 40s), standbys: y, z 2024-05-15T14:01:02.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:02 smithi081 ceph-mon[36434]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-15T14:01:02.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:02 smithi081 ceph-mon[36434]: mgrmap e25: x(active, since 40s), standbys: y, z 2024-05-15T14:01:02.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:02 smithi033 ceph-mon[31650]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-15T14:01:02.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:02 smithi033 ceph-mon[31650]: mgrmap e25: x(active, since 40s), standbys: y, z 2024-05-15T14:01:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.990+0000 7faf71264640 1 -- 172.21.15.33:0/11106271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf6c107940 msgr2=0x7faf6c107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.990+0000 7faf71264640 1 --2- 172.21.15.33:0/11106271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf6c107940 0x7faf6c107d20 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7faf5c009cd0 tx=0x7faf5c02a8e0 comp rx=0 tx=0).stop 2024-05-15T14:01:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.990+0000 7faf71264640 1 -- 172.21.15.33:0/11106271 shutdown_connections 2024-05-15T14:01:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.990+0000 7faf71264640 1 --2- 172.21.15.33:0/11106271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf6c1022f0 0x7faf6c10e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.990+0000 7faf71264640 1 --2- 172.21.15.33:0/11106271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf6c101930 0x7faf6c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.990+0000 7faf71264640 1 --2- 172.21.15.33:0/11106271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf6c107940 0x7faf6c107d20 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.990+0000 7faf71264640 1 -- 172.21.15.33:0/11106271 >> 172.21.15.33:0/11106271 conn(0x7faf6c078a70 msgr2=0x7faf6c078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.990+0000 7faf71264640 1 -- 172.21.15.33:0/11106271 shutdown_connections 2024-05-15T14:01:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.990+0000 7faf71264640 1 -- 172.21.15.33:0/11106271 wait complete. 2024-05-15T14:01:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.990+0000 7faf71264640 1 Processor -- start 2024-05-15T14:01:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.991+0000 7faf71264640 1 -- start start 2024-05-15T14:01:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.991+0000 7faf71264640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf6c101930 0x7faf6c19a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.991+0000 7faf71264640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf6c1022f0 0x7faf6c19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.991+0000 7faf71264640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf6c107940 0x7faf6c1a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.991+0000 7faf71264640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faf6c111c20 con 0x7faf6c1022f0 2024-05-15T14:01:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.991+0000 7faf71264640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faf6c111aa0 con 0x7faf6c107940 2024-05-15T14:01:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.991+0000 7faf71264640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7faf6c111da0 con 0x7faf6c101930 2024-05-15T14:01:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.991+0000 7faf6a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf6c1022f0 0x7faf6c19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.991+0000 7faf6a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf6c1022f0 0x7faf6c19a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57596/0 (socket says 172.21.15.33:57596) 2024-05-15T14:01:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.991+0000 7faf6a7fc640 1 -- 172.21.15.33:0/1978449056 learned_addr learned my addr 172.21.15.33:0/1978449056 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.991+0000 7faf6affd640 1 --2- 172.21.15.33:0/1978449056 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf6c101930 0x7faf6c19a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.991+0000 7faf6b7fe640 1 --2- 172.21.15.33:0/1978449056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf6c107940 0x7faf6c1a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.991+0000 7faf6a7fc640 1 -- 172.21.15.33:0/1978449056 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf6c101930 msgr2=0x7faf6c19a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.991+0000 7faf6a7fc640 1 --2- 172.21.15.33:0/1978449056 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf6c101930 0x7faf6c19a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.991+0000 7faf6a7fc640 1 -- 172.21.15.33:0/1978449056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf6c107940 msgr2=0x7faf6c1a50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.991+0000 7faf6a7fc640 1 --2- 172.21.15.33:0/1978449056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf6c107940 0x7faf6c1a50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.991+0000 7faf6a7fc640 1 -- 172.21.15.33:0/1978449056 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf6c1a5620 con 0x7faf6c1022f0 2024-05-15T14:01:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.992+0000 7faf6a7fc640 1 --2- 172.21.15.33:0/1978449056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf6c1022f0 0x7faf6c19a5a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7faf6000d950 tx=0x7faf6000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.992+0000 7faf53fff640 1 -- 172.21.15.33:0/1978449056 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf60017030 con 0x7faf6c1022f0 2024-05-15T14:01:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.992+0000 7faf71264640 1 -- 172.21.15.33:0/1978449056 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf6c1a5850 con 0x7faf6c1022f0 2024-05-15T14:01:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.992+0000 7faf53fff640 1 -- 172.21.15.33:0/1978449056 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf60004020 con 0x7faf6c1022f0 2024-05-15T14:01:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.992+0000 7faf53fff640 1 -- 172.21.15.33:0/1978449056 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf600047a0 con 0x7faf6c1022f0 2024-05-15T14:01:02.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.992+0000 7faf71264640 1 -- 172.21.15.33:0/1978449056 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf6c1a5d70 con 0x7faf6c1022f0 2024-05-15T14:01:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.994+0000 7faf73f7c740 1 Processor -- start 2024-05-15T14:01:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.995+0000 7faf73f7c740 1 -- start start 2024-05-15T14:01:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.995+0000 7faf53fff640 1 -- 172.21.15.33:0/1978449056 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7faf60009a30 con 0x7faf6c1022f0 2024-05-15T14:01:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.995+0000 7faf73f7c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a47cdae8f0 0x55a47cdaecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.995+0000 7faf73f7c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a47cdaf210 0x55a47cdb9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.995+0000 7faf73f7c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a47cdb9ad0 0x55a47cdbbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.995+0000 7faf73f7c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a47cd21c30 con 0x55a47cdaf210 2024-05-15T14:01:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.995+0000 7faf73f7c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a47cd117d0 con 0x55a47cdae8f0 2024-05-15T14:01:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.995+0000 7faf73f7c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55a47cd15810 con 0x55a47cdb9ad0 2024-05-15T14:01:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.995+0000 7faf6affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a47cdaf210 0x55a47cdb9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.995+0000 7faf6affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a47cdaf210 0x55a47cdb9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57608/0 (socket says 172.21.15.33:57608) 2024-05-15T14:01:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.995+0000 7faf6affd640 1 -- 172.21.15.33:0/2758570273 learned_addr learned my addr 172.21.15.33:0/2758570273 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.995+0000 7faf6b7fe640 1 --2- 172.21.15.33:0/2758570273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a47cdae8f0 0x55a47cdaecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.995+0000 7faf6a7fc640 1 --2- 172.21.15.33:0/2758570273 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a47cdb9ad0 0x55a47cdbbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.995+0000 7faf6affd640 1 -- 172.21.15.33:0/2758570273 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a47cdb9ad0 msgr2=0x55a47cdbbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.995+0000 7faf6affd640 1 --2- 172.21.15.33:0/2758570273 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a47cdb9ad0 0x55a47cdbbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.995+0000 7faf6affd640 1 -- 172.21.15.33:0/2758570273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a47cdae8f0 msgr2=0x55a47cdaecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.995+0000 7faf6affd640 1 --2- 172.21.15.33:0/2758570273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a47cdae8f0 0x55a47cdaecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.995+0000 7faf6affd640 1 -- 172.21.15.33:0/2758570273 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a47ca26210 con 0x55a47cdaf210 2024-05-15T14:01:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.995+0000 7faf53fff640 1 --2- 172.21.15.33:0/1978449056 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7faf540b2a70 0x7faf540b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.996+0000 7faf6affd640 1 --2- 172.21.15.33:0/2758570273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a47cdaf210 0x55a47cdb9590 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7faf5c02ae00 tx=0x7faf5c0314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=174cbdae013f37d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.996+0000 7faf6b7fe640 1 --2- 172.21.15.33:0/1978449056 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7faf540b2a70 0x7faf540b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.996+0000 7faf53fff640 1 -- 172.21.15.33:0/1978449056 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7faf600e2340 con 0x7faf6c1022f0 2024-05-15T14:01:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.996+0000 7faf517fa640 1 -- 172.21.15.33:0/2758570273 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c048070 con 0x55a47cdaf210 2024-05-15T14:01:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.996+0000 7faf517fa640 1 -- 172.21.15.33:0/2758570273 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf5c036070 con 0x55a47cdaf210 2024-05-15T14:01:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.996+0000 7faf517fa640 1 -- 172.21.15.33:0/2758570273 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c004360 con 0x55a47cdaf210 2024-05-15T14:01:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.997+0000 7faf73f7c740 1 -- 172.21.15.33:0/2758570273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a47cdaf210 msgr2=0x55a47cdb9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.997+0000 7faf6b7fe640 1 --2- 172.21.15.33:0/1978449056 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7faf540b2a70 0x7faf540b4f30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faf640045d0 tx=0x7faf64005f90 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.997+0000 7faf73f7c740 1 --2- 172.21.15.33:0/2758570273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a47cdaf210 0x55a47cdb9590 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7faf5c02ae00 tx=0x7faf5c0314e0 comp rx=0 tx=0).stop 2024-05-15T14:01:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.997+0000 7faf73f7c740 1 -- 172.21.15.33:0/2758570273 shutdown_connections 2024-05-15T14:01:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.997+0000 7faf73f7c740 1 --2- 172.21.15.33:0/2758570273 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a47cdb9ad0 0x55a47cdbbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.997+0000 7faf73f7c740 1 --2- 172.21.15.33:0/2758570273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a47cdaf210 0x55a47cdb9590 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.997+0000 7faf73f7c740 1 --2- 172.21.15.33:0/2758570273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a47cdae8f0 0x55a47cdaecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.997+0000 7faf73f7c740 1 -- 172.21.15.33:0/2758570273 >> 172.21.15.33:0/2758570273 conn(0x55a47cdaae00 msgr2=0x55a47cdab1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.997+0000 7faf73f7c740 1 -- 172.21.15.33:0/2758570273 shutdown_connections 2024-05-15T14:01:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.997+0000 7faf73f7c740 1 -- 172.21.15.33:0/2758570273 wait complete. 2024-05-15T14:01:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.997+0000 7faf73f7c740 1 -- start start 2024-05-15T14:01:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.997+0000 7faf73f7c740 1 Processor -- start 2024-05-15T14:01:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.997+0000 7faf73f7c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a47cdae8f0 0x55a47cdb9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.997+0000 7faf73f7c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a47cdaf210 0x55a47cdb6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.997+0000 7faf73f7c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a47cdb9ad0 0x55a47cdb6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.997+0000 7faf73f7c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a47c9f7790 con 0x55a47cdb9ad0 2024-05-15T14:01:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.997+0000 7faf73f7c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a47cccd890 con 0x55a47cdaf210 2024-05-15T14:01:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.997+0000 7faf73f7c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55a47cdbc740 con 0x55a47cdae8f0 2024-05-15T14:01:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.998+0000 7faf73f7c740 10 client.0 _ll_register_callbacks cb 0x55a47ccd0d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:01:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.998+0000 7faf6b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a47cdb9ad0 0x55a47cdb6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.998+0000 7faf6b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a47cdb9ad0 0x55a47cdb6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57612/0 (socket says 172.21.15.33:57612) 2024-05-15T14:01:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.998+0000 7faf6b7fe640 1 -- 172.21.15.33:0/3042114868 learned_addr learned my addr 172.21.15.33:0/3042114868 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.998+0000 7faf6a7fc640 1 --2- 172.21.15.33:0/3042114868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a47cdaf210 0x55a47cdb6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.998+0000 7faf6affd640 1 --2- 172.21.15.33:0/3042114868 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a47cdae8f0 0x55a47cdb9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.998+0000 7faf6b7fe640 1 -- 172.21.15.33:0/3042114868 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a47cdae8f0 msgr2=0x55a47cdb9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.998+0000 7faf6b7fe640 1 --2- 172.21.15.33:0/3042114868 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a47cdae8f0 0x55a47cdb9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.998+0000 7faf6b7fe640 1 -- 172.21.15.33:0/3042114868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a47cdaf210 msgr2=0x55a47cdb6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.998+0000 7faf6b7fe640 1 --2- 172.21.15.33:0/3042114868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a47cdaf210 0x55a47cdb6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.998+0000 7faf6b7fe640 1 -- 172.21.15.33:0/3042114868 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a47caf2620 con 0x55a47cdb9ad0 2024-05-15T14:01:03.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.998+0000 7faf6b7fe640 1 --2- 172.21.15.33:0/3042114868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a47cdb9ad0 0x55a47cdb6bc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7faf640116a0 tx=0x7faf640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5637ceff7b0ff364 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.998+0000 7faf4eff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:01:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.998+0000 7faf4eff5640 1 -- 172.21.15.33:0/3042114868 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf64029060 con 0x55a47cdb9ad0 2024-05-15T14:01:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.998+0000 7faf4eff5640 1 -- 172.21.15.33:0/3042114868 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7faf6401c7d0 con 0x55a47cdb9ad0 2024-05-15T14:01:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.998+0000 7faf73f7c740 1 -- 172.21.15.33:0/3042114868 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf5c036070 con 0x55a47cdb9ad0 2024-05-15T14:01:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.999+0000 7faf4eff5640 1 -- 172.21.15.33:0/3042114868 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7faf6401c9d0 con 0x55a47cdb9ad0 2024-05-15T14:01:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.999+0000 7faf73f7c740 10 client.15387 fetch_fsmap learned FSMap version 24 2024-05-15T14:01:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.999+0000 7faf73f7c740 1 -- 172.21.15.33:0/3042114868 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a47cdc8170 con 0x55a47cdb9ad0 2024-05-15T14:01:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.999+0000 7faf4eff5640 1 -- 172.21.15.33:0/3042114868 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf5c036070 con 0x55a47cdb9ad0 2024-05-15T14:01:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.999+0000 7faf4eff5640 1 -- 172.21.15.33:0/3042114868 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7faf64018040 con 0x55a47cdb9ad0 2024-05-15T14:01:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.999+0000 7faf73f7c740 10 client.15387 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:01:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.999+0000 7faf73f7c740 10 client.15387 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:01:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.999+0000 7faf73f7c740 20 client.15387 populate_metadata read hostname 'smithi033' 2024-05-15T14:01:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.999+0000 7faf73f7c740 1 --2- 172.21.15.33:0/3042114868 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55a47cdc9630 0x55a47cdcba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.999+0000 7faf73f7c740 4 client.15387 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:01:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.999+0000 7faf73f7c740 1 -- 172.21.15.33:0/3042114868 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a47cd23b80 con 0x55a47cdc9630 2024-05-15T14:01:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:02.999+0000 7faf6affd640 1 --2- 172.21.15.33:0/3042114868 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55a47cdc9630 0x55a47cdcba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.000+0000 7faf6affd640 1 --2- 172.21.15.33:0/3042114868 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55a47cdc9630 0x55a47cdcba20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9aabd23922409b52 server_cookie=3bd5738785d27c57 in_seq=0 out_seq=0 2024-05-15T14:01:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.000+0000 7faf4eff5640 10 client.15387 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:01:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.001+0000 7faf4eff5640 1 -- 172.21.15.33:0/3042114868 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a47cd23b80 con 0x55a47cdc9630 2024-05-15T14:01:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.001+0000 7faf4eff5640 10 client.15387 handle_command_reply: tid=0 2024-05-15T14:01:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.001+0000 7faf73f7c740 1 client.15387 shutdown 2024-05-15T14:01:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.001+0000 7faf73f7c740 1 -- 172.21.15.33:0/3042114868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a47cdb9ad0 msgr2=0x55a47cdb6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.001+0000 7faf73f7c740 1 --2- 172.21.15.33:0/3042114868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a47cdb9ad0 0x55a47cdb6bc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7faf640116a0 tx=0x7faf640046c0 comp rx=0 tx=0).stop 2024-05-15T14:01:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.001+0000 7faf73f7c740 1 -- 172.21.15.33:0/3042114868 shutdown_connections 2024-05-15T14:01:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.001+0000 7faf73f7c740 1 --2- 172.21.15.33:0/3042114868 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55a47cdc9630 0x55a47cdcba20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.001+0000 7faf73f7c740 1 --2- 172.21.15.33:0/3042114868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a47cdb9ad0 0x55a47cdb6bc0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.001+0000 7faf73f7c740 1 --2- 172.21.15.33:0/3042114868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a47cdaf210 0x55a47cdb6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.001+0000 7faf73f7c740 1 --2- 172.21.15.33:0/3042114868 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a47cdae8f0 0x55a47cdb9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.001+0000 7faf4eff5640 0 client.15387 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:01:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.001+0000 7faf73f7c740 1 -- 172.21.15.33:0/3042114868 >> 172.21.15.33:0/3042114868 conn(0x55a47cdaae00 msgr2=0x55a47cdaa100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.001+0000 7faf73f7c740 1 -- 172.21.15.33:0/3042114868 shutdown_connections 2024-05-15T14:01:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.001+0000 7faf73f7c740 1 -- 172.21.15.33:0/3042114868 wait complete. 2024-05-15T14:01:03.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.001+0000 7faf73f7c740 20 client.15387 trim_cache size 0 max 16384 2024-05-15T14:01:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.016+0000 7faf73f7c740 1 Processor -- start 2024-05-15T14:01:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.016+0000 7faf73f7c740 1 -- start start 2024-05-15T14:01:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.016+0000 7faf73f7c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a47cdae8f0 0x55a47cdc5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.016+0000 7faf73f7c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a47cdaf210 0x55a47cdd1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.016+0000 7faf73f7c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a47cdb9ad0 0x55a47cdd1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.016+0000 7faf73f7c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faf6c1b0e70 con 0x55a47cdb9ad0 2024-05-15T14:01:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.016+0000 7faf73f7c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faf6c1ae740 con 0x55a47cdae8f0 2024-05-15T14:01:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.016+0000 7faf73f7c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7faf6c1ac590 con 0x55a47cdaf210 2024-05-15T14:01:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.017+0000 7faf6b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a47cdb9ad0 0x55a47cdd1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.017+0000 7faf6b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a47cdb9ad0 0x55a47cdd1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57614/0 (socket says 172.21.15.33:57614) 2024-05-15T14:01:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.017+0000 7faf6b7fe640 1 -- 172.21.15.33:0/4224142619 learned_addr learned my addr 172.21.15.33:0/4224142619 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.017+0000 7faf6a7fc640 1 --2- 172.21.15.33:0/4224142619 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a47cdaf210 0x55a47cdd1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.017+0000 7faf6b7fe640 1 -- 172.21.15.33:0/4224142619 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a47cdaf210 msgr2=0x55a47cdd1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.017+0000 7faf6b7fe640 1 --2- 172.21.15.33:0/4224142619 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a47cdaf210 0x55a47cdd1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.017+0000 7faf6b7fe640 1 -- 172.21.15.33:0/4224142619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a47cdae8f0 msgr2=0x55a47cdc5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.017+0000 7faf6affd640 1 --2- 172.21.15.33:0/4224142619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a47cdae8f0 0x55a47cdc5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.017+0000 7faf6b7fe640 1 --2- 172.21.15.33:0/4224142619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a47cdae8f0 0x55a47cdc5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.017+0000 7faf6b7fe640 1 -- 172.21.15.33:0/4224142619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a47cdaac30 con 0x55a47cdb9ad0 2024-05-15T14:01:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.017+0000 7faf6a7fc640 1 --2- 172.21.15.33:0/4224142619 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a47cdaf210 0x55a47cdd1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:01:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.017+0000 7faf6affd640 1 --2- 172.21.15.33:0/4224142619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a47cdae8f0 0x55a47cdc5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:01:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.017+0000 7faf6b7fe640 1 --2- 172.21.15.33:0/4224142619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a47cdb9ad0 0x55a47cdd1870 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7faf6400c520 tx=0x7faf6401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2985b6a46d1da6f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.017+0000 7faf50ff9640 1 -- 172.21.15.33:0/4224142619 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6401c270 con 0x55a47cdb9ad0 2024-05-15T14:01:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.017+0000 7faf50ff9640 1 -- 172.21.15.33:0/4224142619 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf640054f0 con 0x55a47cdb9ad0 2024-05-15T14:01:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.017+0000 7faf50ff9640 1 -- 172.21.15.33:0/4224142619 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf64004b20 con 0x55a47cdb9ad0 2024-05-15T14:01:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.018+0000 7faf73f7c740 1 -- 172.21.15.33:0/4224142619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a47cdb9ad0 msgr2=0x55a47cdd1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.018+0000 7faf73f7c740 1 --2- 172.21.15.33:0/4224142619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a47cdb9ad0 0x55a47cdd1870 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7faf6400c520 tx=0x7faf6401c960 comp rx=0 tx=0).stop 2024-05-15T14:01:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.018+0000 7faf73f7c740 1 -- 172.21.15.33:0/4224142619 shutdown_connections 2024-05-15T14:01:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.018+0000 7faf73f7c740 1 --2- 172.21.15.33:0/4224142619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a47cdb9ad0 0x55a47cdd1870 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.018+0000 7faf73f7c740 1 --2- 172.21.15.33:0/4224142619 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a47cdaf210 0x55a47cdd1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.018+0000 7faf73f7c740 1 --2- 172.21.15.33:0/4224142619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a47cdae8f0 0x55a47cdc5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.018+0000 7faf73f7c740 1 -- 172.21.15.33:0/4224142619 >> 172.21.15.33:0/4224142619 conn(0x55a47cdaae00 msgr2=0x55a47cdaa100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.018+0000 7faf73f7c740 1 -- 172.21.15.33:0/4224142619 shutdown_connections 2024-05-15T14:01:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.018+0000 7faf73f7c740 1 -- 172.21.15.33:0/4224142619 wait complete. 2024-05-15T14:01:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.018+0000 7faf73f7c740 1 -- start start 2024-05-15T14:01:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.018+0000 7faf73f7c740 1 Processor -- start 2024-05-15T14:01:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.018+0000 7faf73f7c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a47cdae8f0 0x55a47cdbf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.018+0000 7faf73f7c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a47cdaf210 0x55a47cdbfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.018+0000 7faf73f7c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a47cdb9ad0 0x55a47cdc0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.018+0000 7faf73f7c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faf6c1acc00 con 0x55a47cdae8f0 2024-05-15T14:01:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.018+0000 7faf73f7c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faf6c1ae070 con 0x55a47cdaf210 2024-05-15T14:01:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.019+0000 7faf73f7c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7faf6c1ad2d0 con 0x55a47cdb9ad0 2024-05-15T14:01:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.019+0000 7faf73f7c740 10 client.0 _ll_register_callbacks cb 0x7faf64003f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:01:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.019+0000 7faf6affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a47cdae8f0 0x55a47cdbf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.019+0000 7faf6affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a47cdae8f0 0x55a47cdbf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57628/0 (socket says 172.21.15.33:57628) 2024-05-15T14:01:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.019+0000 7faf6affd640 1 -- 172.21.15.33:0/725912249 learned_addr learned my addr 172.21.15.33:0/725912249 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.019+0000 7faf6a7fc640 1 --2- 172.21.15.33:0/725912249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a47cdaf210 0x55a47cdbfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.019+0000 7faf6b7fe640 1 --2- 172.21.15.33:0/725912249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a47cdb9ad0 0x55a47cdc0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.019+0000 7faf6affd640 1 -- 172.21.15.33:0/725912249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a47cdb9ad0 msgr2=0x55a47cdc0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.019+0000 7faf6affd640 1 --2- 172.21.15.33:0/725912249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a47cdb9ad0 0x55a47cdc0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:03.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.019+0000 7faf6affd640 1 -- 172.21.15.33:0/725912249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a47cdaf210 msgr2=0x55a47cdbfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:03.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.019+0000 7faf6affd640 1 --2- 172.21.15.33:0/725912249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a47cdaf210 0x55a47cdbfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:03.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.019+0000 7faf6affd640 1 -- 172.21.15.33:0/725912249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a47ce33190 con 0x55a47cdae8f0 2024-05-15T14:01:03.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.019+0000 7faf6affd640 1 --2- 172.21.15.33:0/725912249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a47cdae8f0 0x55a47cdbf7d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7faf5c009a60 tx=0x7faf5c009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a290100cb5ddbd0d server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:03.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.019+0000 7faf4eff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:01:03.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.019+0000 7faf4eff5640 1 -- 172.21.15.33:0/725912249 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c059070 con 0x55a47cdae8f0 2024-05-15T14:01:03.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.019+0000 7faf4eff5640 1 -- 172.21.15.33:0/725912249 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7faf5c031a90 con 0x55a47cdae8f0 2024-05-15T14:01:03.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.019+0000 7faf73f7c740 1 -- 172.21.15.33:0/725912249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf640054f0 con 0x55a47cdae8f0 2024-05-15T14:01:03.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.019+0000 7faf4eff5640 1 -- 172.21.15.33:0/725912249 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7faf5c062050 con 0x55a47cdae8f0 2024-05-15T14:01:03.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.019+0000 7faf73f7c740 10 client.15393 fetch_fsmap learned FSMap version 24 2024-05-15T14:01:03.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.020+0000 7faf73f7c740 1 -- 172.21.15.33:0/725912249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a47ce34450 con 0x55a47cdae8f0 2024-05-15T14:01:03.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.020+0000 7faf4eff5640 1 -- 172.21.15.33:0/725912249 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf640054f0 con 0x55a47cdae8f0 2024-05-15T14:01:03.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.020+0000 7faf4eff5640 1 -- 172.21.15.33:0/725912249 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7faf5c05c020 con 0x55a47cdae8f0 2024-05-15T14:01:03.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.020+0000 7faf73f7c740 10 client.15393 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:01:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.020+0000 7faf73f7c740 10 client.15393 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:01:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.020+0000 7faf73f7c740 20 client.15393 populate_metadata read hostname 'smithi033' 2024-05-15T14:01:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.020+0000 7faf73f7c740 1 --2- 172.21.15.33:0/725912249 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55a47ce34850 0x55a47ce36c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.020+0000 7faf73f7c740 4 client.15393 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:01:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.020+0000 7faf73f7c740 1 -- 172.21.15.33:0/725912249 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a47cd1d360 con 0x55a47ce34850 2024-05-15T14:01:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.020+0000 7faf6a7fc640 1 --2- 172.21.15.33:0/725912249 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55a47ce34850 0x55a47ce36c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.020+0000 7faf4eff5640 10 client.15393 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:01:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.020+0000 7faf6a7fc640 1 --2- 172.21.15.33:0/725912249 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55a47ce34850 0x55a47ce36c40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3d318cb20b86509 server_cookie=5558174fbf660b44 in_seq=0 out_seq=0 2024-05-15T14:01:03.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.679+0000 7faf4eff5640 1 -- 172.21.15.33:0/725912249 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a47cd1d360 con 0x55a47ce34850 2024-05-15T14:01:03.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.679+0000 7faf4eff5640 10 client.15393 handle_command_reply: tid=0 2024-05-15T14:01:03.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.679+0000 7faf73f7c740 1 client.15393 shutdown 2024-05-15T14:01:03.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.679+0000 7faf73f7c740 1 -- 172.21.15.33:0/725912249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a47cdae8f0 msgr2=0x55a47cdbf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:03.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.679+0000 7faf73f7c740 1 --2- 172.21.15.33:0/725912249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a47cdae8f0 0x55a47cdbf7d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7faf5c009a60 tx=0x7faf5c009560 comp rx=0 tx=0).stop 2024-05-15T14:01:03.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.679+0000 7faf73f7c740 1 -- 172.21.15.33:0/725912249 shutdown_connections 2024-05-15T14:01:03.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.679+0000 7faf73f7c740 1 --2- 172.21.15.33:0/725912249 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55a47ce34850 0x55a47ce36c40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:03.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.679+0000 7faf73f7c740 1 --2- 172.21.15.33:0/725912249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a47cdb9ad0 0x55a47cdc0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:03.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.679+0000 7faf73f7c740 1 --2- 172.21.15.33:0/725912249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a47cdaf210 0x55a47cdbfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:03.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.679+0000 7faf73f7c740 1 --2- 172.21.15.33:0/725912249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a47cdae8f0 0x55a47cdbf7d0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:03.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.679+0000 7faf4eff5640 0 client.15393 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:01:03.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.679+0000 7faf73f7c740 1 -- 172.21.15.33:0/725912249 >> 172.21.15.33:0/725912249 conn(0x55a47cdaae00 msgr2=0x55a47cdc4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:03.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.679+0000 7faf73f7c740 1 -- 172.21.15.33:0/725912249 shutdown_connections 2024-05-15T14:01:03.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.679+0000 7faf73f7c740 1 -- 172.21.15.33:0/725912249 wait complete. 2024-05-15T14:01:03.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.679+0000 7faf73f7c740 20 client.15393 trim_cache size 0 max 16384 2024-05-15T14:01:03.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:01:03.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:01:03.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:01:03.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.680+0000 7faf71264640 1 -- 172.21.15.33:0/1978449056 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7faf540b2a70 msgr2=0x7faf540b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:03.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.680+0000 7faf71264640 1 --2- 172.21.15.33:0/1978449056 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7faf540b2a70 0x7faf540b4f30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faf640045d0 tx=0x7faf64005f90 comp rx=0 tx=0).stop 2024-05-15T14:01:03.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.680+0000 7faf71264640 1 -- 172.21.15.33:0/1978449056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf6c1022f0 msgr2=0x7faf6c19a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:03.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.680+0000 7faf71264640 1 --2- 172.21.15.33:0/1978449056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf6c1022f0 0x7faf6c19a5a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7faf6000d950 tx=0x7faf6000de10 comp rx=0 tx=0).stop 2024-05-15T14:01:03.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.680+0000 7faf71264640 1 -- 172.21.15.33:0/1978449056 shutdown_connections 2024-05-15T14:01:03.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.680+0000 7faf71264640 1 --2- 172.21.15.33:0/1978449056 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7faf540b2a70 0x7faf540b4f30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:03.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.680+0000 7faf71264640 1 --2- 172.21.15.33:0/1978449056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faf6c107940 0x7faf6c1a50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:03.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.680+0000 7faf71264640 1 --2- 172.21.15.33:0/1978449056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf6c1022f0 0x7faf6c19a5a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:03.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.680+0000 7faf71264640 1 --2- 172.21.15.33:0/1978449056 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faf6c101930 0x7faf6c19a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.680+0000 7faf71264640 1 -- 172.21.15.33:0/1978449056 >> 172.21.15.33:0/1978449056 conn(0x7faf6c078a70 msgr2=0x7faf6c104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.680+0000 7faf71264640 1 -- 172.21.15.33:0/1978449056 shutdown_connections 2024-05-15T14:01:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:03.680+0000 7faf71264640 1 -- 172.21.15.33:0/1978449056 wait complete. 2024-05-15T14:01:04.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:04 smithi052 ceph-mon[36190]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-05-15T14:01:04.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:04 smithi081 ceph-mon[36434]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-05-15T14:01:04.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:04 smithi033 ceph-mon[31650]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-05-15T14:01:06.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:06 smithi052 ceph-mon[36190]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-15T14:01:06.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:06 smithi081 ceph-mon[36434]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-15T14:01:06.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:06 smithi033 ceph-mon[31650]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-15T14:01:08.359 INFO:tasks.quiescer.fs.[cephfs]:Total duration (48.0) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-15T14:01:08.359 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.0 2024-05-15T14:01:08.359 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T14:01:08.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:08 smithi052 ceph-mon[36190]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-05-15T14:01:08.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:08 smithi081 ceph-mon[36434]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-05-15T14:01:08.618 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:01:08.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:08 smithi033 ceph-mon[31650]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-05-15T14:01:10.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.185+0000 7fbd20941700 1 -- 172.21.15.33:0/1676679940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd1c108910 msgr2=0x7fbd1c110490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:10.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.185+0000 7fbd20941700 1 --2- 172.21.15.33:0/1676679940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd1c108910 0x7fbd1c110490 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbd140099c0 tx=0x7fbd14009cd0 comp rx=0 tx=0).stop 2024-05-15T14:01:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.185+0000 7fbd20941700 1 -- 172.21.15.33:0/1676679940 shutdown_connections 2024-05-15T14:01:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.185+0000 7fbd20941700 1 --2- 172.21.15.33:0/1676679940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd1c108910 0x7fbd1c110490 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.185+0000 7fbd20941700 1 --2- 172.21.15.33:0/1676679940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd1c103820 0x7fbd1c103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.185+0000 7fbd20941700 1 --2- 172.21.15.33:0/1676679940 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd1c1041d0 0x7fbd1c1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.185+0000 7fbd20941700 1 -- 172.21.15.33:0/1676679940 >> 172.21.15.33:0/1676679940 conn(0x7fbd1c07a6b0 msgr2=0x7fbd1c07aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.185+0000 7fbd20941700 1 -- 172.21.15.33:0/1676679940 shutdown_connections 2024-05-15T14:01:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.186+0000 7fbd20941700 1 -- 172.21.15.33:0/1676679940 wait complete. 2024-05-15T14:01:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.186+0000 7fbd20941700 1 Processor -- start 2024-05-15T14:01:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.186+0000 7fbd20941700 1 -- start start 2024-05-15T14:01:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.187+0000 7fbd20941700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd1c103820 0x7fbd1c19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.187+0000 7fbd20941700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd1c1041d0 0x7fbd1c19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.187+0000 7fbd20941700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd1c108910 0x7fbd1c1999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.187+0000 7fbd20941700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c113700 con 0x7fbd1c103820 2024-05-15T14:01:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.187+0000 7fbd20941700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c113580 con 0x7fbd1c108910 2024-05-15T14:01:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.187+0000 7fbd20941700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c113880 con 0x7fbd1c1041d0 2024-05-15T14:01:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.187+0000 7fbd1a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd1c103820 0x7fbd1c19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.187+0000 7fbd1a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd1c103820 0x7fbd1c19f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57658/0 (socket says 172.21.15.33:57658) 2024-05-15T14:01:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.187+0000 7fbd1a59c700 1 -- 172.21.15.33:0/991274629 learned_addr learned my addr 172.21.15.33:0/991274629 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.187+0000 7fbd1ad9d700 1 --2- 172.21.15.33:0/991274629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd1c108910 0x7fbd1c1999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.187+0000 7fbd19d9b700 1 --2- 172.21.15.33:0/991274629 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd1c1041d0 0x7fbd1c19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.187+0000 7fbd1a59c700 1 -- 172.21.15.33:0/991274629 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd1c1041d0 msgr2=0x7fbd1c19fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.187+0000 7fbd1a59c700 1 --2- 172.21.15.33:0/991274629 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd1c1041d0 0x7fbd1c19fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.187+0000 7fbd1a59c700 1 -- 172.21.15.33:0/991274629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd1c108910 msgr2=0x7fbd1c1999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.187+0000 7fbd1a59c700 1 --2- 172.21.15.33:0/991274629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd1c108910 0x7fbd1c1999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.187+0000 7fbd1a59c700 1 -- 172.21.15.33:0/991274629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd1c19a120 con 0x7fbd1c103820 2024-05-15T14:01:10.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.187+0000 7fbd1a59c700 1 --2- 172.21.15.33:0/991274629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd1c103820 0x7fbd1c19f800 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c00d980 tx=0x7fbd0c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:10.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.188+0000 7fbd0b7fe700 1 -- 172.21.15.33:0/991274629 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c016070 con 0x7fbd1c103820 2024-05-15T14:01:10.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.188+0000 7fbd20941700 1 -- 172.21.15.33:0/991274629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd1c19a410 con 0x7fbd1c103820 2024-05-15T14:01:10.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.188+0000 7fbd0b7fe700 1 -- 172.21.15.33:0/991274629 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd0c004830 con 0x7fbd1c103820 2024-05-15T14:01:10.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.188+0000 7fbd0b7fe700 1 -- 172.21.15.33:0/991274629 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c005540 con 0x7fbd1c103820 2024-05-15T14:01:10.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.188+0000 7fbd20941700 1 -- 172.21.15.33:0/991274629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd1c1a66d0 con 0x7fbd1c103820 2024-05-15T14:01:10.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.189+0000 7fbd20941700 1 -- 172.21.15.33:0/991274629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd1c06a230 con 0x7fbd1c103820 2024-05-15T14:01:10.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.190+0000 7fbd0b7fe700 1 -- 172.21.15.33:0/991274629 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7fbd0c004220 con 0x7fbd1c103820 2024-05-15T14:01:10.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.191+0000 7fbd0b7fe700 1 --2- 172.21.15.33:0/991274629 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbd040b2b90 0x7fbd040b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:10.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.191+0000 7fbd0b7fe700 1 -- 172.21.15.33:0/991274629 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fbd0c0e2960 con 0x7fbd1c103820 2024-05-15T14:01:10.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.191+0000 7fbd19d9b700 1 --2- 172.21.15.33:0/991274629 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbd040b2b90 0x7fbd040b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:10.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.192+0000 7fbd19d9b700 1 --2- 172.21.15.33:0/991274629 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbd040b2b90 0x7fbd040b5050 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c108190 tx=0x7fbd10009180 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:10.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.197+0000 7fbd0b7fe700 1 -- 172.21.15.33:0/991274629 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbd0c01f030 con 0x7fbd1c103820 2024-05-15T14:01:10.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:10 smithi033 ceph-mon[31650]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-05-15T14:01:10.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.469+0000 7fbd20941700 1 -- 172.21.15.33:0/991274629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd1c19aef0 con 0x7fbd1c103820 2024-05-15T14:01:10.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.470+0000 7fbd0b7fe700 1 -- 172.21.15.33:0/991274629 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11858 (secure 0 0 0) 0x7fbd0c0aa100 con 0x7fbd1c103820 2024-05-15T14:01:10.471 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T14:01:10.471 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2030117633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2030117633},{"type":"v1","addr":"172.21.15.33:6839","nonce":2030117633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1918979061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1918979061},{"type":"v1","addr":"172.21.15.33:6841","nonce":1918979061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T13:57:19.056417+0000","modified":"2024-05-15T14:00:18.543619+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24463,"mds_2":24461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/1629697841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1629697841},{"type":"v1","addr":"172.21.15.52:6837","nonce":1629697841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/3773839758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3773839758},{"type":"v1","addr":"172.21.15.81:6835","nonce":3773839758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.52:6839/2696044471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2696044471},{"type":"v1","addr":"172.21.15.52:6839","nonce":2696044471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.81:6839/1554395940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":1554395940},{"type":"v1","addr":"172.21.15.81:6839","nonce":1554395940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24463,24478,24461]},"id":1}]} 2024-05-15T14:01:10.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.472+0000 7fbd20941700 1 -- 172.21.15.33:0/991274629 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbd040b2b90 msgr2=0x7fbd040b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:10.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.472+0000 7fbd20941700 1 --2- 172.21.15.33:0/991274629 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbd040b2b90 0x7fbd040b5050 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c108190 tx=0x7fbd10009180 comp rx=0 tx=0).stop 2024-05-15T14:01:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.472+0000 7fbd20941700 1 -- 172.21.15.33:0/991274629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd1c103820 msgr2=0x7fbd1c19f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.472+0000 7fbd20941700 1 --2- 172.21.15.33:0/991274629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd1c103820 0x7fbd1c19f800 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c00d980 tx=0x7fbd0c00dc90 comp rx=0 tx=0).stop 2024-05-15T14:01:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.472+0000 7fbd20941700 1 -- 172.21.15.33:0/991274629 shutdown_connections 2024-05-15T14:01:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.472+0000 7fbd20941700 1 --2- 172.21.15.33:0/991274629 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbd040b2b90 0x7fbd040b5050 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.472+0000 7fbd20941700 1 --2- 172.21.15.33:0/991274629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd1c108910 0x7fbd1c1999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.472+0000 7fbd20941700 1 --2- 172.21.15.33:0/991274629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd1c103820 0x7fbd1c19f800 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.472+0000 7fbd20941700 1 --2- 172.21.15.33:0/991274629 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd1c1041d0 0x7fbd1c19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.472+0000 7fbd20941700 1 -- 172.21.15.33:0/991274629 >> 172.21.15.33:0/991274629 conn(0x7fbd1c07a6b0 msgr2=0x7fbd1c101610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.472+0000 7fbd20941700 1 -- 172.21.15.33:0/991274629 shutdown_connections 2024-05-15T14:01:10.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:10.472+0000 7fbd20941700 1 -- 172.21.15.33:0/991274629 wait complete. 2024-05-15T14:01:10.475 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 24 2024-05-15T14:01:10.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:10 smithi052 ceph-mon[36190]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-05-15T14:01:10.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:10 smithi081 ceph-mon[36434]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-05-15T14:01:11.081 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24478 quiesce db / --timeout 90 --expiration 144.0 --await-for 45.0' 2024-05-15T14:01:11.082 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24478 quiesce db / --timeout 90 --expiration 144.0 --await-for 45.0 2024-05-15T14:01:11.337 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:01:11.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:11 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/991274629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:01:11.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:11 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/991274629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:01:11.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:11 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/991274629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:01:12.610 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:12 smithi033 ceph-mon[31650]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-05-15T14:01:12.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.811+0000 7f43ce2a6700 1 -- 172.21.15.33:0/618250912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c8108840 msgr2=0x7f43c8110450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:12.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.811+0000 7f43ce2a6700 1 --2- 172.21.15.33:0/618250912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c8108840 0x7f43c8110450 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f43c000b0d0 tx=0x7f43c000b3e0 comp rx=0 tx=0).stop 2024-05-15T14:01:12.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.811+0000 7f43ce2a6700 1 -- 172.21.15.33:0/618250912 shutdown_connections 2024-05-15T14:01:12.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.811+0000 7f43ce2a6700 1 --2- 172.21.15.33:0/618250912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c8108840 0x7f43c8110450 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:12.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.811+0000 7f43ce2a6700 1 --2- 172.21.15.33:0/618250912 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43c8103850 0x7f43c8103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:12.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.811+0000 7f43ce2a6700 1 --2- 172.21.15.33:0/618250912 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43c8104200 0x7f43c8108160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:12.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.811+0000 7f43ce2a6700 1 -- 172.21.15.33:0/618250912 >> 172.21.15.33:0/618250912 conn(0x7f43c807a610 msgr2=0x7f43c807aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:12.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.811+0000 7f43ce2a6700 1 -- 172.21.15.33:0/618250912 shutdown_connections 2024-05-15T14:01:12.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.811+0000 7f43ce2a6700 1 -- 172.21.15.33:0/618250912 wait complete. 2024-05-15T14:01:12.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.811+0000 7f43ce2a6700 1 Processor -- start 2024-05-15T14:01:12.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.811+0000 7f43ce2a6700 1 -- start start 2024-05-15T14:01:12.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.812+0000 7f43ce2a6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43c8103850 0x7f43c819f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:12.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.812+0000 7f43ce2a6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c8104200 0x7f43c819fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:12.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.812+0000 7f43ce2a6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43c8108840 0x7f43c8199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:12.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.812+0000 7f43ce2a6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f43c8113760 con 0x7f43c8104200 2024-05-15T14:01:12.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.812+0000 7f43ce2a6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f43c81135e0 con 0x7f43c8108840 2024-05-15T14:01:12.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.812+0000 7f43ce2a6700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f43c81138e0 con 0x7f43c8103850 2024-05-15T14:01:12.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.812+0000 7f43bffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c8104200 0x7f43c819fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:12.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.812+0000 7f43bffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c8104200 0x7f43c819fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53238/0 (socket says 172.21.15.33:53238) 2024-05-15T14:01:12.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.812+0000 7f43bffff700 1 -- 172.21.15.33:0/2250396418 learned_addr learned my addr 172.21.15.33:0/2250396418 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:12.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.812+0000 7f43c7fff700 1 --2- 172.21.15.33:0/2250396418 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43c8103850 0x7f43c819f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:12.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.812+0000 7f43cc843700 1 --2- 172.21.15.33:0/2250396418 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43c8108840 0x7f43c8199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:12.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.812+0000 7f43bffff700 1 -- 172.21.15.33:0/2250396418 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43c8103850 msgr2=0x7f43c819f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:12.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.812+0000 7f43bffff700 1 --2- 172.21.15.33:0/2250396418 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43c8103850 0x7f43c819f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:12.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.812+0000 7f43bffff700 1 -- 172.21.15.33:0/2250396418 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43c8108840 msgr2=0x7f43c8199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:12.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.812+0000 7f43bffff700 1 --2- 172.21.15.33:0/2250396418 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43c8108840 0x7f43c8199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:12.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.812+0000 7f43bffff700 1 -- 172.21.15.33:0/2250396418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c8199fe0 con 0x7f43c8104200 2024-05-15T14:01:12.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.812+0000 7f43bffff700 1 --2- 172.21.15.33:0/2250396418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c8104200 0x7f43c819fbc0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f43b000c9e0 tx=0x7f43b000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:12.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.813+0000 7f43c5ffb700 1 -- 172.21.15.33:0/2250396418 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b0007aa0 con 0x7f43c8104200 2024-05-15T14:01:12.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.813+0000 7f43c5ffb700 1 -- 172.21.15.33:0/2250396418 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43b0007c40 con 0x7f43c8104200 2024-05-15T14:01:12.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.813+0000 7f43c5ffb700 1 -- 172.21.15.33:0/2250396418 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b000fe60 con 0x7f43c8104200 2024-05-15T14:01:12.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.813+0000 7f43ce2a6700 1 -- 172.21.15.33:0/2250396418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c819a2d0 con 0x7f43c8104200 2024-05-15T14:01:12.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.813+0000 7f43ce2a6700 1 -- 172.21.15.33:0/2250396418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c81a64e0 con 0x7f43c8104200 2024-05-15T14:01:12.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.815+0000 7f43c5ffb700 1 -- 172.21.15.33:0/2250396418 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f43b000f450 con 0x7f43c8104200 2024-05-15T14:01:12.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.816+0000 7f43c5ffb700 1 --2- 172.21.15.33:0/2250396418 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f43b40b2c40 0x7f43b40b5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:12.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.816+0000 7f43c7fff700 1 --2- 172.21.15.33:0/2250396418 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f43b40b2c40 0x7f43b40b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:12.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.816+0000 7f43c5ffb700 1 -- 172.21.15.33:0/2250396418 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f43b00e2740 con 0x7f43c8104200 2024-05-15T14:01:12.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.816+0000 7f43c7fff700 1 --2- 172.21.15.33:0/2250396418 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f43b40b2c40 0x7f43b40b5100 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f43b800ac50 tx=0x7f43b800a380 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:12.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.817+0000 7f43d8b3cb80 1 Processor -- start 2024-05-15T14:01:12.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.817+0000 7f43d8b3cb80 1 -- start start 2024-05-15T14:01:12.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.817+0000 7f43d8b3cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecc5c59320 0x55ecc5c59700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:12.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.817+0000 7f43d8b3cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc5c59c40 0x55ecc5c5d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:12.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.817+0000 7f43d8b3cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecc5c5dae0 0x55ecc5c5df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:12.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.817+0000 7f43d8b3cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ecc5bb2f00 con 0x55ecc5c5dae0 2024-05-15T14:01:12.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.817+0000 7f43d8b3cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ecc5ba2aa0 con 0x55ecc5c59320 2024-05-15T14:01:12.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.817+0000 7f43d8b3cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55ecc5ba6ae0 con 0x55ecc5c59c40 2024-05-15T14:01:12.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.817+0000 7f43cc843700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecc5c5dae0 0x55ecc5c5df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:12.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.817+0000 7f43cc843700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecc5c5dae0 0x55ecc5c5df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53242/0 (socket says 172.21.15.33:53242) 2024-05-15T14:01:12.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.817+0000 7f43cc843700 1 -- 172.21.15.33:0/64369147 learned_addr learned my addr 172.21.15.33:0/64369147 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:12.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.817+0000 7f43c7fff700 1 --2- 172.21.15.33:0/64369147 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecc5c59320 0x55ecc5c59700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:12.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.817+0000 7f43bffff700 1 --2- 172.21.15.33:0/64369147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc5c59c40 0x55ecc5c5d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:12.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.817+0000 7f43cc843700 1 -- 172.21.15.33:0/64369147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc5c59c40 msgr2=0x55ecc5c5d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:12.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.817+0000 7f43cc843700 1 --2- 172.21.15.33:0/64369147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc5c59c40 0x55ecc5c5d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:12.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.817+0000 7f43cc843700 1 -- 172.21.15.33:0/64369147 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecc5c59320 msgr2=0x55ecc5c59700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:12.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.817+0000 7f43cc843700 1 --2- 172.21.15.33:0/64369147 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecc5c59320 0x55ecc5c59700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:12.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.817+0000 7f43cc843700 1 -- 172.21.15.33:0/64369147 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecc5b4c140 con 0x55ecc5c5dae0 2024-05-15T14:01:12.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.818+0000 7f43cc843700 1 --2- 172.21.15.33:0/64369147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecc5c5dae0 0x55ecc5c5df40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f43c00091b0 tx=0x7f43c00091e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85d158b6cf26db8 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:12.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.818+0000 7f43be7fc700 1 -- 172.21.15.33:0/64369147 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c0003c00 con 0x55ecc5c5dae0 2024-05-15T14:01:12.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.818+0000 7f43be7fc700 1 -- 172.21.15.33:0/64369147 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43c0009440 con 0x55ecc5c5dae0 2024-05-15T14:01:12.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.818+0000 7f43be7fc700 1 -- 172.21.15.33:0/64369147 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c00053e0 con 0x55ecc5c5dae0 2024-05-15T14:01:12.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.818+0000 7f43d8b3cb80 1 -- 172.21.15.33:0/64369147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecc5c5dae0 msgr2=0x55ecc5c5df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:12.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.818+0000 7f43d8b3cb80 1 --2- 172.21.15.33:0/64369147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecc5c5dae0 0x55ecc5c5df40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f43c00091b0 tx=0x7f43c00091e0 comp rx=0 tx=0).stop 2024-05-15T14:01:12.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.819+0000 7f43d8b3cb80 1 -- 172.21.15.33:0/64369147 shutdown_connections 2024-05-15T14:01:12.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.819+0000 7f43d8b3cb80 1 --2- 172.21.15.33:0/64369147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecc5c5dae0 0x55ecc5c5df40 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:12.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.819+0000 7f43d8b3cb80 1 --2- 172.21.15.33:0/64369147 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecc5c59320 0x55ecc5c59700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:12.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.819+0000 7f43d8b3cb80 1 --2- 172.21.15.33:0/64369147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc5c59c40 0x55ecc5c5d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:12.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.819+0000 7f43d8b3cb80 1 -- 172.21.15.33:0/64369147 >> 172.21.15.33:0/64369147 conn(0x55ecc5b81f40 msgr2=0x55ecc5c4f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:12.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.819+0000 7f43d8b3cb80 1 -- 172.21.15.33:0/64369147 shutdown_connections 2024-05-15T14:01:12.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.819+0000 7f43d8b3cb80 1 -- 172.21.15.33:0/64369147 wait complete. 2024-05-15T14:01:12.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.819+0000 7f43d8b3cb80 1 -- start start 2024-05-15T14:01:12.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.819+0000 7f43d8b3cb80 1 Processor -- start 2024-05-15T14:01:12.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.819+0000 7f43d8b3cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecc5c59320 0x55ecc5c65020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:12.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.819+0000 7f43d8b3cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc5c59c40 0x55ecc5c623c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:12.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.820+0000 7f43d8b3cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecc5c5dae0 0x55ecc5c62900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:12.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.820+0000 7f43d8b3cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ecc5aa9130 con 0x55ecc5c59320 2024-05-15T14:01:12.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.820+0000 7f43d8b3cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ecc5b0af30 con 0x55ecc5c5dae0 2024-05-15T14:01:12.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.820+0000 7f43d8b3cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55ecc5c65cb0 con 0x55ecc5c59c40 2024-05-15T14:01:12.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.820+0000 7f43d8b3cb80 10 client.0 _ll_register_callbacks cb 0x55ecc5b3dae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:01:12.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.820+0000 7f43c7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecc5c59320 0x55ecc5c65020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:12.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.820+0000 7f43c7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecc5c59320 0x55ecc5c65020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53258/0 (socket says 172.21.15.33:53258) 2024-05-15T14:01:12.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.820+0000 7f43c7fff700 1 -- 172.21.15.33:0/3274316701 learned_addr learned my addr 172.21.15.33:0/3274316701 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:12.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.820+0000 7f43cc843700 1 --2- 172.21.15.33:0/3274316701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecc5c5dae0 0x55ecc5c62900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:12.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.820+0000 7f43bffff700 1 --2- 172.21.15.33:0/3274316701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc5c59c40 0x55ecc5c623c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:12.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.820+0000 7f43c7fff700 1 -- 172.21.15.33:0/3274316701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc5c59c40 msgr2=0x55ecc5c623c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:12.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.820+0000 7f43c7fff700 1 --2- 172.21.15.33:0/3274316701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc5c59c40 0x55ecc5c623c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:12.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.820+0000 7f43c7fff700 1 -- 172.21.15.33:0/3274316701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecc5c5dae0 msgr2=0x55ecc5c62900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:12.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.820+0000 7f43c7fff700 1 --2- 172.21.15.33:0/3274316701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecc5c5dae0 0x55ecc5c62900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:12.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.820+0000 7f43c7fff700 1 -- 172.21.15.33:0/3274316701 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecc5af2a60 con 0x55ecc5c59320 2024-05-15T14:01:12.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.820+0000 7f43c7fff700 1 --2- 172.21.15.33:0/3274316701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecc5c59320 0x55ecc5c65020 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f43b8003930 tx=0x7f43b80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c21e1a4d84503fb server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:12.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.820+0000 7f43a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:01:12.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.820+0000 7f43a77fe700 1 -- 172.21.15.33:0/3274316701 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b8016070 con 0x55ecc5c59320 2024-05-15T14:01:12.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.820+0000 7f43a77fe700 1 -- 172.21.15.33:0/3274316701 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f43b80108c0 con 0x55ecc5c59320 2024-05-15T14:01:12.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.820+0000 7f43d8b3cb80 1 -- 172.21.15.33:0/3274316701 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43c0009440 con 0x55ecc5c59320 2024-05-15T14:01:12.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.821+0000 7f43a77fe700 1 -- 172.21.15.33:0/3274316701 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f43b8010b00 con 0x55ecc5c59320 2024-05-15T14:01:12.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.821+0000 7f43d8b3cb80 10 client.15411 fetch_fsmap learned FSMap version 24 2024-05-15T14:01:12.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.821+0000 7f43d8b3cb80 1 -- 172.21.15.33:0/3274316701 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ecc5b5c8f0 con 0x55ecc5c59320 2024-05-15T14:01:12.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.821+0000 7f43a77fe700 1 -- 172.21.15.33:0/3274316701 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43c0009440 con 0x55ecc5c59320 2024-05-15T14:01:12.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.821+0000 7f43a77fe700 1 -- 172.21.15.33:0/3274316701 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f43b8027470 con 0x55ecc5c59320 2024-05-15T14:01:12.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.821+0000 7f43d8b3cb80 10 client.15411 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:01:12.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.821+0000 7f43d8b3cb80 10 client.15411 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:01:12.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.821+0000 7f43d8b3cb80 20 client.15411 populate_metadata read hostname 'smithi033' 2024-05-15T14:01:12.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.821+0000 7f43d8b3cb80 1 --2- 172.21.15.33:0/3274316701 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55ecc5c72850 0x55ecc5c74c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:12.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.821+0000 7f43d8b3cb80 4 client.15411 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:01:12.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.821+0000 7f43d8b3cb80 1 -- 172.21.15.33:0/3274316701 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecc5bb4e50 con 0x55ecc5c72850 2024-05-15T14:01:12.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.821+0000 7f43bffff700 1 --2- 172.21.15.33:0/3274316701 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55ecc5c72850 0x55ecc5c74c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:12.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.822+0000 7f43bffff700 1 --2- 172.21.15.33:0/3274316701 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55ecc5c72850 0x55ecc5c74c40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad163d6ce9df473e server_cookie=bb916815a04e25f3 in_seq=0 out_seq=0 2024-05-15T14:01:12.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.822+0000 7f43a77fe700 10 client.15411 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:01:12.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.822+0000 7f43a77fe700 1 -- 172.21.15.33:0/3274316701 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ecc5bb4e50 con 0x55ecc5c72850 2024-05-15T14:01:12.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.822+0000 7f43a77fe700 10 client.15411 handle_command_reply: tid=0 2024-05-15T14:01:12.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.822+0000 7f43d8b3cb80 1 client.15411 shutdown 2024-05-15T14:01:12.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.823+0000 7f43d8b3cb80 1 -- 172.21.15.33:0/3274316701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecc5c59320 msgr2=0x55ecc5c65020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:12.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.823+0000 7f43d8b3cb80 1 --2- 172.21.15.33:0/3274316701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecc5c59320 0x55ecc5c65020 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f43b8003930 tx=0x7f43b80047c0 comp rx=0 tx=0).stop 2024-05-15T14:01:12.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.823+0000 7f43d8b3cb80 1 -- 172.21.15.33:0/3274316701 shutdown_connections 2024-05-15T14:01:12.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.823+0000 7f43d8b3cb80 1 --2- 172.21.15.33:0/3274316701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecc5c5dae0 0x55ecc5c62900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:12.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.823+0000 7f43d8b3cb80 1 --2- 172.21.15.33:0/3274316701 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55ecc5c72850 0x55ecc5c74c40 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:12.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.823+0000 7f43d8b3cb80 1 --2- 172.21.15.33:0/3274316701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecc5c59320 0x55ecc5c65020 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:12.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.823+0000 7f43d8b3cb80 1 --2- 172.21.15.33:0/3274316701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc5c59c40 0x55ecc5c623c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:12.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.823+0000 7f43a77fe700 0 client.15411 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:01:12.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.823+0000 7f43d8b3cb80 1 -- 172.21.15.33:0/3274316701 >> 172.21.15.33:0/3274316701 conn(0x55ecc5b81f40 msgr2=0x55ecc5c5ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:12.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.823+0000 7f43d8b3cb80 1 -- 172.21.15.33:0/3274316701 shutdown_connections 2024-05-15T14:01:12.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.823+0000 7f43d8b3cb80 1 -- 172.21.15.33:0/3274316701 wait complete. 2024-05-15T14:01:12.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.823+0000 7f43d8b3cb80 20 client.15411 trim_cache size 0 max 16384 2024-05-15T14:01:12.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:12 smithi052 ceph-mon[36190]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-05-15T14:01:12.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.843+0000 7f43d8b3cb80 1 Processor -- start 2024-05-15T14:01:12.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.843+0000 7f43d8b3cb80 1 -- start start 2024-05-15T14:01:12.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.843+0000 7f43d8b3cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc5c59320 0x55ecc5c6ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:12.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.843+0000 7f43d8b3cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecc5c59c40 0x55ecc5c499e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:12.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.843+0000 7f43d8b3cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecc5c5dae0 0x55ecc5c5eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:12.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.843+0000 7f43d8b3cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f43c81ae180 con 0x55ecc5c59c40 2024-05-15T14:01:12.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.843+0000 7f43d8b3cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f43c81aba10 con 0x55ecc5c5dae0 2024-05-15T14:01:12.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.843+0000 7f43d8b3cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f43c81a9860 con 0x55ecc5c59320 2024-05-15T14:01:12.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.843+0000 7f43bffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecc5c59c40 0x55ecc5c499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:12.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.843+0000 7f43bffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecc5c59c40 0x55ecc5c499e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53262/0 (socket says 172.21.15.33:53262) 2024-05-15T14:01:12.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.843+0000 7f43bffff700 1 -- 172.21.15.33:0/3732322544 learned_addr learned my addr 172.21.15.33:0/3732322544 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:12.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.843+0000 7f43cc843700 1 --2- 172.21.15.33:0/3732322544 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecc5c5dae0 0x55ecc5c5eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:12.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.843+0000 7f43c7fff700 1 --2- 172.21.15.33:0/3732322544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc5c59320 0x55ecc5c6ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:12.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.844+0000 7f43bffff700 1 -- 172.21.15.33:0/3732322544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc5c59320 msgr2=0x55ecc5c6ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:12.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.844+0000 7f43bffff700 1 --2- 172.21.15.33:0/3732322544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc5c59320 0x55ecc5c6ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:12.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.844+0000 7f43bffff700 1 -- 172.21.15.33:0/3732322544 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecc5c5dae0 msgr2=0x55ecc5c5eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:12.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.844+0000 7f43bffff700 1 --2- 172.21.15.33:0/3732322544 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecc5c5dae0 0x55ecc5c5eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:12.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.844+0000 7f43bffff700 1 -- 172.21.15.33:0/3732322544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecc5c60930 con 0x55ecc5c59c40 2024-05-15T14:01:12.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.844+0000 7f43bffff700 1 --2- 172.21.15.33:0/3732322544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecc5c59c40 0x55ecc5c499e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f43b00a9480 tx=0x7f43b000fb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25cf97b2a8a2db2e server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:12.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.844+0000 7f43bdffb700 1 -- 172.21.15.33:0/3732322544 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b00c7020 con 0x55ecc5c59c40 2024-05-15T14:01:12.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.844+0000 7f43bdffb700 1 -- 172.21.15.33:0/3732322544 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43b0007df0 con 0x55ecc5c59c40 2024-05-15T14:01:12.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.844+0000 7f43bdffb700 1 -- 172.21.15.33:0/3732322544 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b00c2030 con 0x55ecc5c59c40 2024-05-15T14:01:12.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.845+0000 7f43d8b3cb80 1 -- 172.21.15.33:0/3732322544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecc5c59c40 msgr2=0x55ecc5c499e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:12.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.845+0000 7f43d8b3cb80 1 --2- 172.21.15.33:0/3732322544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecc5c59c40 0x55ecc5c499e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f43b00a9480 tx=0x7f43b000fb50 comp rx=0 tx=0).stop 2024-05-15T14:01:12.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.845+0000 7f43d8b3cb80 1 -- 172.21.15.33:0/3732322544 shutdown_connections 2024-05-15T14:01:12.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.845+0000 7f43d8b3cb80 1 --2- 172.21.15.33:0/3732322544 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecc5c5dae0 0x55ecc5c5eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:12.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.845+0000 7f43d8b3cb80 1 --2- 172.21.15.33:0/3732322544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc5c59320 0x55ecc5c6ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:12.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.845+0000 7f43d8b3cb80 1 --2- 172.21.15.33:0/3732322544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecc5c59c40 0x55ecc5c499e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:12.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.845+0000 7f43d8b3cb80 1 -- 172.21.15.33:0/3732322544 >> 172.21.15.33:0/3732322544 conn(0x55ecc5b81f40 msgr2=0x55ecc5c53d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:12.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.845+0000 7f43d8b3cb80 1 -- 172.21.15.33:0/3732322544 shutdown_connections 2024-05-15T14:01:12.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.845+0000 7f43d8b3cb80 1 -- 172.21.15.33:0/3732322544 wait complete. 2024-05-15T14:01:12.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.845+0000 7f43d8b3cb80 1 -- start start 2024-05-15T14:01:12.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.845+0000 7f43d8b3cb80 1 Processor -- start 2024-05-15T14:01:12.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.846+0000 7f43d8b3cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecc5c59320 0x55ecc5c6af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:12.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.846+0000 7f43d8b3cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc5c59c40 0x55ecc5c6b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:12.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.846+0000 7f43d8b3cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecc5c5dae0 0x55ecc5c5a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:12.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.846+0000 7f43d8b3cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f43c81a9ed0 con 0x55ecc5c59320 2024-05-15T14:01:12.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.846+0000 7f43d8b3cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f43c81ab340 con 0x55ecc5c5dae0 2024-05-15T14:01:12.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.846+0000 7f43d8b3cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f43c81aa5a0 con 0x55ecc5c59c40 2024-05-15T14:01:12.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.846+0000 7f43d8b3cb80 10 client.0 _ll_register_callbacks cb 0x55ecc5b3dae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:01:12.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.846+0000 7f43c7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecc5c59320 0x55ecc5c6af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:12.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.846+0000 7f43c7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecc5c59320 0x55ecc5c6af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53272/0 (socket says 172.21.15.33:53272) 2024-05-15T14:01:12.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.846+0000 7f43c7fff700 1 -- 172.21.15.33:0/643681800 learned_addr learned my addr 172.21.15.33:0/643681800 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:12.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.846+0000 7f43cc843700 1 --2- 172.21.15.33:0/643681800 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecc5c5dae0 0x55ecc5c5a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:12.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.846+0000 7f43bffff700 1 --2- 172.21.15.33:0/643681800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc5c59c40 0x55ecc5c6b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:12.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.846+0000 7f43c7fff700 1 -- 172.21.15.33:0/643681800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc5c59c40 msgr2=0x55ecc5c6b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:12.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.846+0000 7f43c7fff700 1 --2- 172.21.15.33:0/643681800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc5c59c40 0x55ecc5c6b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:12.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.846+0000 7f43c7fff700 1 -- 172.21.15.33:0/643681800 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecc5c5dae0 msgr2=0x55ecc5c5a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:12.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.846+0000 7f43c7fff700 1 --2- 172.21.15.33:0/643681800 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecc5c5dae0 0x55ecc5c5a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:12.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.846+0000 7f43c7fff700 1 -- 172.21.15.33:0/643681800 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecc5c5ab40 con 0x55ecc5c59320 2024-05-15T14:01:12.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.846+0000 7f43c7fff700 1 --2- 172.21.15.33:0/643681800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecc5c59320 0x55ecc5c6af40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f43b800fa30 tx=0x7f43b8025460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab43b31726394683 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:12.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.846+0000 7f43a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:01:12.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.846+0000 7f43a77fe700 1 -- 172.21.15.33:0/643681800 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b8025580 con 0x55ecc5c59320 2024-05-15T14:01:12.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.846+0000 7f43a77fe700 1 -- 172.21.15.33:0/643681800 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f43b8025720 con 0x55ecc5c59320 2024-05-15T14:01:12.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.846+0000 7f43d8b3cb80 1 -- 172.21.15.33:0/643681800 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43b0007df0 con 0x55ecc5c59320 2024-05-15T14:01:12.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.847+0000 7f43a77fe700 1 -- 172.21.15.33:0/643681800 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f43b8025920 con 0x55ecc5c59320 2024-05-15T14:01:12.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.847+0000 7f43d8b3cb80 10 client.15417 fetch_fsmap learned FSMap version 24 2024-05-15T14:01:12.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.847+0000 7f43d8b3cb80 1 -- 172.21.15.33:0/643681800 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ecc5cde990 con 0x55ecc5c59320 2024-05-15T14:01:12.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.847+0000 7f43a77fe700 1 -- 172.21.15.33:0/643681800 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43b0007df0 con 0x55ecc5c59320 2024-05-15T14:01:12.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.847+0000 7f43a77fe700 1 -- 172.21.15.33:0/643681800 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f43b8039bf0 con 0x55ecc5c59320 2024-05-15T14:01:12.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.847+0000 7f43d8b3cb80 10 client.15417 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:01:12.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.847+0000 7f43d8b3cb80 10 client.15417 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:01:12.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.847+0000 7f43d8b3cb80 20 client.15417 populate_metadata read hostname 'smithi033' 2024-05-15T14:01:12.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.847+0000 7f43d8b3cb80 1 --2- 172.21.15.33:0/643681800 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55ecc5cdf060 0x55ecc5ce1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:12.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.847+0000 7f43d8b3cb80 4 client.15417 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.0, "await_for": 45.0}] 2024-05-15T14:01:12.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.847+0000 7f43d8b3cb80 1 -- 172.21.15.33:0/643681800 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.0, "await_for": 45.0}) -- 0x55ecc5bae630 con 0x55ecc5cdf060 2024-05-15T14:01:12.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.848+0000 7f43bffff700 1 --2- 172.21.15.33:0/643681800 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55ecc5cdf060 0x55ecc5ce1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:12.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.848+0000 7f43bffff700 1 --2- 172.21.15.33:0/643681800 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55ecc5cdf060 0x55ecc5ce1450 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4c6a4c147be1f97 server_cookie=2882eb49b43f4c99 in_seq=0 out_seq=0 2024-05-15T14:01:12.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.848+0000 7f43a77fe700 10 client.15417 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:01:12.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:12 smithi081 ceph-mon[36434]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-05-15T14:01:12.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.864+0000 7f43a77fe700 1 -- 172.21.15.33:0/643681800 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55ecc5bae630 con 0x55ecc5cdf060 2024-05-15T14:01:12.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.864+0000 7f43a77fe700 10 client.15417 handle_command_reply: tid=0 2024-05-15T14:01:12.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.864+0000 7f43d8b3cb80 1 client.15417 shutdown 2024-05-15T14:01:12.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.865+0000 7f43d8b3cb80 1 -- 172.21.15.33:0/643681800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecc5c59320 msgr2=0x55ecc5c6af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:12.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.865+0000 7f43d8b3cb80 1 --2- 172.21.15.33:0/643681800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecc5c59320 0x55ecc5c6af40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f43b800fa30 tx=0x7f43b8025460 comp rx=0 tx=0).stop 2024-05-15T14:01:12.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.865+0000 7f43d8b3cb80 1 -- 172.21.15.33:0/643681800 shutdown_connections 2024-05-15T14:01:12.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.865+0000 7f43d8b3cb80 1 --2- 172.21.15.33:0/643681800 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ecc5c5dae0 0x55ecc5c5a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:12.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.865+0000 7f43d8b3cb80 1 --2- 172.21.15.33:0/643681800 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55ecc5cdf060 0x55ecc5ce1450 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:12.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.865+0000 7f43a77fe700 0 client.15417 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:01:12.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.865+0000 7f43d8b3cb80 1 --2- 172.21.15.33:0/643681800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecc5c59320 0x55ecc5c6af40 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:12.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.865+0000 7f43d8b3cb80 1 --2- 172.21.15.33:0/643681800 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ecc5c59c40 0x55ecc5c6b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:12.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.865+0000 7f43d8b3cb80 1 -- 172.21.15.33:0/643681800 >> 172.21.15.33:0/643681800 conn(0x55ecc5b81f40 msgr2=0x55ecc5c5c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:12.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.865+0000 7f43d8b3cb80 1 -- 172.21.15.33:0/643681800 shutdown_connections 2024-05-15T14:01:12.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.865+0000 7f43d8b3cb80 1 -- 172.21.15.33:0/643681800 wait complete. 2024-05-15T14:01:12.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.865+0000 7f43d8b3cb80 20 client.15417 trim_cache size 0 max 16384 2024-05-15T14:01:12.868 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T14:01:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 24, 2024-05-15T14:01:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24478, 2024-05-15T14:01:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 2, 2024-05-15T14:01:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-05-15T14:01:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: "2fa87dae": { 2024-05-15T14:01:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 2, 2024-05-15T14:01:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-05-15T14:01:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:01:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-05-15T14:01:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:01:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:01:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 90.0, 2024-05-15T14:01:12.869 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 144.0, 2024-05-15T14:01:12.869 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-05-15T14:01:12.869 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-05-15T14:01:12.869 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-05-15T14:01:12.869 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:01:12.869 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-05-15T14:01:12.869 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:01:12.869 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:01:12.869 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:01:12.869 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:01:12.869 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:01:12.869 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:01:12.869 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T14:01:12.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.866+0000 7f43ce2a6700 1 -- 172.21.15.33:0/2250396418 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f43b40b2c40 msgr2=0x7f43b40b5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:12.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.866+0000 7f43ce2a6700 1 --2- 172.21.15.33:0/2250396418 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f43b40b2c40 0x7f43b40b5100 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f43b800ac50 tx=0x7f43b800a380 comp rx=0 tx=0).stop 2024-05-15T14:01:12.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.866+0000 7f43ce2a6700 1 -- 172.21.15.33:0/2250396418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c8104200 msgr2=0x7f43c819fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:12.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.866+0000 7f43ce2a6700 1 --2- 172.21.15.33:0/2250396418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c8104200 0x7f43c819fbc0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f43b000c9e0 tx=0x7f43b000ccf0 comp rx=0 tx=0).stop 2024-05-15T14:01:12.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.866+0000 7f43ce2a6700 1 -- 172.21.15.33:0/2250396418 shutdown_connections 2024-05-15T14:01:12.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.866+0000 7f43ce2a6700 1 --2- 172.21.15.33:0/2250396418 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f43b40b2c40 0x7f43b40b5100 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:12.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.866+0000 7f43ce2a6700 1 --2- 172.21.15.33:0/2250396418 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43c8108840 0x7f43c8199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:12.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.866+0000 7f43ce2a6700 1 --2- 172.21.15.33:0/2250396418 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43c8103850 0x7f43c819f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:12.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.866+0000 7f43ce2a6700 1 --2- 172.21.15.33:0/2250396418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c8104200 0x7f43c819fbc0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:12.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.866+0000 7f43ce2a6700 1 -- 172.21.15.33:0/2250396418 >> 172.21.15.33:0/2250396418 conn(0x7f43c807a610 msgr2=0x7f43c81014c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:12.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.866+0000 7f43ce2a6700 1 -- 172.21.15.33:0/2250396418 shutdown_connections 2024-05-15T14:01:12.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:12.866+0000 7f43ce2a6700 1 -- 172.21.15.33:0/2250396418 wait complete. 2024-05-15T14:01:13.469 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2fa87dae', quiesce took 5.1 seconds. Will release after: 18.9 2024-05-15T14:01:14.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:14 smithi033 ceph-mon[31650]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-05-15T14:01:14.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:14 smithi052 ceph-mon[36190]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-05-15T14:01:14.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:14 smithi081 ceph-mon[36434]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-05-15T14:01:16.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:16 smithi033 ceph-mon[31650]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-05-15T14:01:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.786+0000 7f1b25223640 1 -- 172.21.15.33:0/2252795698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b20102a40 msgr2=0x7f1b20102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.786+0000 7f1b25223640 1 --2- 172.21.15.33:0/2252795698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b20102a40 0x7f1b20102ec0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1b14009930 tx=0x7f1b1402a420 comp rx=0 tx=0).stop 2024-05-15T14:01:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.786+0000 7f1b25223640 1 -- 172.21.15.33:0/2252795698 shutdown_connections 2024-05-15T14:01:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.786+0000 7f1b25223640 1 --2- 172.21.15.33:0/2252795698 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1b20103400 0x7f1b2010f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.786+0000 7f1b25223640 1 --2- 172.21.15.33:0/2252795698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b20102a40 0x7f1b20102ec0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.786+0000 7f1b25223640 1 --2- 172.21.15.33:0/2252795698 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b20108a50 0x7f1b20108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.786+0000 7f1b25223640 1 -- 172.21.15.33:0/2252795698 >> 172.21.15.33:0/2252795698 conn(0x7f1b200fe510 msgr2=0x7f1b20100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.786+0000 7f1b25223640 1 -- 172.21.15.33:0/2252795698 shutdown_connections 2024-05-15T14:01:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.786+0000 7f1b25223640 1 -- 172.21.15.33:0/2252795698 wait complete. 2024-05-15T14:01:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.787+0000 7f1b25223640 1 Processor -- start 2024-05-15T14:01:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.787+0000 7f1b25223640 1 -- start start 2024-05-15T14:01:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.787+0000 7f1b25223640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b20102a40 0x7f1b20199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.787+0000 7f1b25223640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1b20103400 0x7f1b20199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.787+0000 7f1b25223640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b20108a50 0x7f1b2019dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.787+0000 7f1b25223640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1b20112d10 con 0x7f1b20108a50 2024-05-15T14:01:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.787+0000 7f1b25223640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1b20112b90 con 0x7f1b20103400 2024-05-15T14:01:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.787+0000 7f1b25223640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1b20112e90 con 0x7f1b20102a40 2024-05-15T14:01:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.787+0000 7f1b1f59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b20108a50 0x7f1b2019dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.787+0000 7f1b1f59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b20108a50 0x7f1b2019dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53284/0 (socket says 172.21.15.33:53284) 2024-05-15T14:01:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.787+0000 7f1b1f59e640 1 -- 172.21.15.33:0/4209194241 learned_addr learned my addr 172.21.15.33:0/4209194241 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.787+0000 7f1b1ed9d640 1 --2- 172.21.15.33:0/4209194241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b20102a40 0x7f1b20199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.787+0000 7f1b1e59c640 1 --2- 172.21.15.33:0/4209194241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1b20103400 0x7f1b20199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.788+0000 7f1b1f59e640 1 -- 172.21.15.33:0/4209194241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b20102a40 msgr2=0x7f1b20199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.788+0000 7f1b1f59e640 1 --2- 172.21.15.33:0/4209194241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b20102a40 0x7f1b20199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.788+0000 7f1b1f59e640 1 -- 172.21.15.33:0/4209194241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1b20103400 msgr2=0x7f1b20199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.788+0000 7f1b1f59e640 1 --2- 172.21.15.33:0/4209194241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1b20103400 0x7f1b20199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.788+0000 7f1b1f59e640 1 -- 172.21.15.33:0/4209194241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b2019e200 con 0x7f1b20108a50 2024-05-15T14:01:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.788+0000 7f1b1e59c640 1 --2- 172.21.15.33:0/4209194241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1b20103400 0x7f1b20199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:01:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.788+0000 7f1b1f59e640 1 --2- 172.21.15.33:0/4209194241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b20108a50 0x7f1b2019dad0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f1b1800cce0 tx=0x7f1b1800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.788+0000 7f1b07fff640 1 -- 172.21.15.33:0/4209194241 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1800f450 con 0x7f1b20108a50 2024-05-15T14:01:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.788+0000 7f1b25223640 1 -- 172.21.15.33:0/4209194241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b2019e4f0 con 0x7f1b20108a50 2024-05-15T14:01:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.788+0000 7f1b07fff640 1 -- 172.21.15.33:0/4209194241 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b1800faa0 con 0x7f1b20108a50 2024-05-15T14:01:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.788+0000 7f1b07fff640 1 -- 172.21.15.33:0/4209194241 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b18018b70 con 0x7f1b20108a50 2024-05-15T14:01:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.788+0000 7f1b25223640 1 -- 172.21.15.33:0/4209194241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b201a5d80 con 0x7f1b20108a50 2024-05-15T14:01:16.799 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.791+0000 7f1b07fff640 1 -- 172.21.15.33:0/4209194241 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f1b18018d10 con 0x7f1b20108a50 2024-05-15T14:01:16.799 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.791+0000 7f1b2801d740 1 Processor -- start 2024-05-15T14:01:16.799 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.791+0000 7f1b2801d740 1 -- start start 2024-05-15T14:01:16.799 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.791+0000 7f1b2801d740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4826078f0 0x55d482607cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.791+0000 7f1b2801d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d482608210 0x55d482612590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.791+0000 7f1b2801d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d482612ad0 0x55d482614ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.791+0000 7f1b2801d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d48257ac30 con 0x55d482612ad0 2024-05-15T14:01:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.791+0000 7f1b2801d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d48256a7d0 con 0x55d482608210 2024-05-15T14:01:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.791+0000 7f1b2801d740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55d48256e810 con 0x55d4826078f0 2024-05-15T14:01:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.792+0000 7f1b1e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d482612ad0 0x55d482614ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.792+0000 7f1b1e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d482612ad0 0x55d482614ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53300/0 (socket says 172.21.15.33:53300) 2024-05-15T14:01:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.792+0000 7f1b1e59c640 1 -- 172.21.15.33:0/3773466083 learned_addr learned my addr 172.21.15.33:0/3773466083 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.792+0000 7f1b1ed9d640 1 --2- 172.21.15.33:0/3773466083 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d482608210 0x55d482612590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.792+0000 7f1b1e59c640 1 --2- 172.21.15.33:0/3773466083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4826078f0 0x55d482607cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.792+0000 7f1b1e59c640 1 -- 172.21.15.33:0/3773466083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4826078f0 msgr2=0x55d482607cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.792+0000 7f1b1e59c640 1 --2- 172.21.15.33:0/3773466083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4826078f0 0x55d482607cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.792+0000 7f1b1e59c640 1 -- 172.21.15.33:0/3773466083 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d482608210 msgr2=0x55d482612590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.792+0000 7f1b1e59c640 1 --2- 172.21.15.33:0/3773466083 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d482608210 0x55d482612590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:16.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.792+0000 7f1b1e59c640 1 -- 172.21.15.33:0/3773466083 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d48227f210 con 0x55d482612ad0 2024-05-15T14:01:16.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.792+0000 7f1b07fff640 1 --2- 172.21.15.33:0/4209194241 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f1b080b2a70 0x7f1b080b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:16.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.792+0000 7f1b1e59c640 1 --2- 172.21.15.33:0/3773466083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d482612ad0 0x55d482614ec0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1b140057c0 tx=0x7f1b140056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64f3772bc48a870 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:16.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.792+0000 7f1b1ed9d640 1 --2- 172.21.15.33:0/4209194241 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f1b080b2a70 0x7f1b080b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:16.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.792+0000 7f1b07fff640 1 -- 172.21.15.33:0/4209194241 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f1b180e2ce0 con 0x7f1b20108a50 2024-05-15T14:01:16.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.792+0000 7f1b057fa640 1 -- 172.21.15.33:0/3773466083 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b14035040 con 0x55d482612ad0 2024-05-15T14:01:16.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.792+0000 7f1b057fa640 1 -- 172.21.15.33:0/3773466083 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b14031620 con 0x55d482612ad0 2024-05-15T14:01:16.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.792+0000 7f1b057fa640 1 -- 172.21.15.33:0/3773466083 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b14003c30 con 0x55d482612ad0 2024-05-15T14:01:16.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.793+0000 7f1b2801d740 1 -- 172.21.15.33:0/3773466083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d482612ad0 msgr2=0x55d482614ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.793+0000 7f1b1ed9d640 1 --2- 172.21.15.33:0/4209194241 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f1b080b2a70 0x7f1b080b4f30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x55d482608e90 tx=0x7f1b1000b390 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.793+0000 7f1b2801d740 1 --2- 172.21.15.33:0/3773466083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d482612ad0 0x55d482614ec0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1b140057c0 tx=0x7f1b140056d0 comp rx=0 tx=0).stop 2024-05-15T14:01:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.793+0000 7f1b2801d740 1 -- 172.21.15.33:0/3773466083 shutdown_connections 2024-05-15T14:01:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.793+0000 7f1b2801d740 1 --2- 172.21.15.33:0/3773466083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d482612ad0 0x55d482614ec0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.793+0000 7f1b2801d740 1 --2- 172.21.15.33:0/3773466083 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d482608210 0x55d482612590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.793+0000 7f1b2801d740 1 --2- 172.21.15.33:0/3773466083 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4826078f0 0x55d482607cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.793+0000 7f1b2801d740 1 -- 172.21.15.33:0/3773466083 >> 172.21.15.33:0/3773466083 conn(0x55d482603e00 msgr2=0x55d4826041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.793+0000 7f1b2801d740 1 -- 172.21.15.33:0/3773466083 shutdown_connections 2024-05-15T14:01:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.793+0000 7f1b2801d740 1 -- 172.21.15.33:0/3773466083 wait complete. 2024-05-15T14:01:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.794+0000 7f1b2801d740 1 -- start start 2024-05-15T14:01:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.794+0000 7f1b2801d740 1 Processor -- start 2024-05-15T14:01:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.794+0000 7f1b2801d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4826078f0 0x55d482612420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.794+0000 7f1b2801d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d482608210 0x55d48260f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.794+0000 7f1b2801d740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d482612ad0 0x55d48260fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.794+0000 7f1b2801d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d482250790 con 0x55d4826078f0 2024-05-15T14:01:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.794+0000 7f1b2801d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d482526890 con 0x55d482608210 2024-05-15T14:01:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.794+0000 7f1b2801d740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55d482615740 con 0x55d482612ad0 2024-05-15T14:01:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.794+0000 7f1b2801d740 10 client.0 _ll_register_callbacks cb 0x55d482529d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:01:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.794+0000 7f1b1ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4826078f0 0x55d482612420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.794+0000 7f1b1ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4826078f0 0x55d482612420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53310/0 (socket says 172.21.15.33:53310) 2024-05-15T14:01:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.794+0000 7f1b1ed9d640 1 -- 172.21.15.33:0/2465327855 learned_addr learned my addr 172.21.15.33:0/2465327855 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.794+0000 7f1b1e59c640 1 --2- 172.21.15.33:0/2465327855 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d482608210 0x55d48260f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.794+0000 7f1b1f59e640 1 --2- 172.21.15.33:0/2465327855 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d482612ad0 0x55d48260fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.794+0000 7f1b1ed9d640 1 -- 172.21.15.33:0/2465327855 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d482612ad0 msgr2=0x55d48260fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:16.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.794+0000 7f1b1ed9d640 1 --2- 172.21.15.33:0/2465327855 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d482612ad0 0x55d48260fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.794+0000 7f1b1ed9d640 1 -- 172.21.15.33:0/2465327855 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d482608210 msgr2=0x55d48260f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.794+0000 7f1b1ed9d640 1 --2- 172.21.15.33:0/2465327855 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d482608210 0x55d48260f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.794+0000 7f1b1ed9d640 1 -- 172.21.15.33:0/2465327855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d48234b620 con 0x55d4826078f0 2024-05-15T14:01:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.794+0000 7f1b1f59e640 1 --2- 172.21.15.33:0/2465327855 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d482612ad0 0x55d48260fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:01:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.795+0000 7f1b1ed9d640 1 --2- 172.21.15.33:0/2465327855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4826078f0 0x55d482612420 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1b10003cb0 tx=0x7f1b10004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a909d0c2e63511 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.795+0000 7f1b02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:01:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.795+0000 7f1b02ff5640 1 -- 172.21.15.33:0/2465327855 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1002c060 con 0x55d4826078f0 2024-05-15T14:01:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.795+0000 7f1b02ff5640 1 -- 172.21.15.33:0/2465327855 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f1b10017030 con 0x55d4826078f0 2024-05-15T14:01:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.795+0000 7f1b2801d740 1 -- 172.21.15.33:0/2465327855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b14031620 con 0x55d4826078f0 2024-05-15T14:01:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.795+0000 7f1b02ff5640 1 -- 172.21.15.33:0/2465327855 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1b10030450 con 0x55d4826078f0 2024-05-15T14:01:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.795+0000 7f1b2801d740 10 client.15429 fetch_fsmap learned FSMap version 24 2024-05-15T14:01:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.795+0000 7f1b2801d740 1 -- 172.21.15.33:0/2465327855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d482621200 con 0x55d4826078f0 2024-05-15T14:01:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.795+0000 7f1b02ff5640 1 -- 172.21.15.33:0/2465327855 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b14031620 con 0x55d4826078f0 2024-05-15T14:01:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.795+0000 7f1b02ff5640 1 -- 172.21.15.33:0/2465327855 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f1b10027530 con 0x55d4826078f0 2024-05-15T14:01:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.795+0000 7f1b2801d740 10 client.15429 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:01:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.795+0000 7f1b2801d740 10 client.15429 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:01:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.795+0000 7f1b2801d740 20 client.15429 populate_metadata read hostname 'smithi033' 2024-05-15T14:01:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.795+0000 7f1b2801d740 1 --2- 172.21.15.33:0/2465327855 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55d482622710 0x55d482624b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.796+0000 7f1b2801d740 4 client.15429 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:01:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.796+0000 7f1b2801d740 1 -- 172.21.15.33:0/2465327855 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d48257cb80 con 0x55d482622710 2024-05-15T14:01:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.796+0000 7f1b1e59c640 1 --2- 172.21.15.33:0/2465327855 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55d482622710 0x55d482624b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.796+0000 7f1b1e59c640 1 --2- 172.21.15.33:0/2465327855 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55d482622710 0x55d482624b00 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fc82092f8870285 server_cookie=cded5278f7d125b2 in_seq=0 out_seq=0 2024-05-15T14:01:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.796+0000 7f1b02ff5640 10 client.15429 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:01:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.797+0000 7f1b02ff5640 1 -- 172.21.15.33:0/2465327855 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d48257cb80 con 0x55d482622710 2024-05-15T14:01:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.797+0000 7f1b02ff5640 10 client.15429 handle_command_reply: tid=0 2024-05-15T14:01:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.797+0000 7f1b2801d740 1 client.15429 shutdown 2024-05-15T14:01:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.797+0000 7f1b2801d740 1 -- 172.21.15.33:0/2465327855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4826078f0 msgr2=0x55d482612420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.797+0000 7f1b2801d740 1 --2- 172.21.15.33:0/2465327855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4826078f0 0x55d482612420 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1b10003cb0 tx=0x7f1b10004730 comp rx=0 tx=0).stop 2024-05-15T14:01:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.797+0000 7f1b2801d740 1 -- 172.21.15.33:0/2465327855 shutdown_connections 2024-05-15T14:01:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.797+0000 7f1b2801d740 1 --2- 172.21.15.33:0/2465327855 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55d482622710 0x55d482624b00 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.797+0000 7f1b2801d740 1 --2- 172.21.15.33:0/2465327855 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d482612ad0 0x55d48260fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.797+0000 7f1b2801d740 1 --2- 172.21.15.33:0/2465327855 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d482608210 0x55d48260f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.797+0000 7f1b02ff5640 0 client.15429 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:01:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.797+0000 7f1b2801d740 1 --2- 172.21.15.33:0/2465327855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4826078f0 0x55d482612420 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.797+0000 7f1b2801d740 1 -- 172.21.15.33:0/2465327855 >> 172.21.15.33:0/2465327855 conn(0x55d482603e00 msgr2=0x55d482603100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.797+0000 7f1b2801d740 1 -- 172.21.15.33:0/2465327855 shutdown_connections 2024-05-15T14:01:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.797+0000 7f1b2801d740 1 -- 172.21.15.33:0/2465327855 wait complete. 2024-05-15T14:01:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.797+0000 7f1b2801d740 20 client.15429 trim_cache size 0 max 16384 2024-05-15T14:01:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.813+0000 7f1b2801d740 1 Processor -- start 2024-05-15T14:01:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.813+0000 7f1b2801d740 1 -- start start 2024-05-15T14:01:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.813+0000 7f1b2801d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4826078f0 0x55d48262a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.813+0000 7f1b2801d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d482608210 0x55d48262a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.813+0000 7f1b2801d740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d482612ad0 0x55d48260c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.813+0000 7f1b2801d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1b201b0e40 con 0x55d482608210 2024-05-15T14:01:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.813+0000 7f1b2801d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1b201ae710 con 0x55d4826078f0 2024-05-15T14:01:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.813+0000 7f1b2801d740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1b201ac560 con 0x55d482612ad0 2024-05-15T14:01:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.813+0000 7f1b1e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d482608210 0x55d48262a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.813+0000 7f1b1e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d482608210 0x55d48262a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53312/0 (socket says 172.21.15.33:53312) 2024-05-15T14:01:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.813+0000 7f1b1e59c640 1 -- 172.21.15.33:0/3516744856 learned_addr learned my addr 172.21.15.33:0/3516744856 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.813+0000 7f1b1f59e640 1 --2- 172.21.15.33:0/3516744856 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d482612ad0 0x55d48260c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.813+0000 7f1b1ed9d640 1 --2- 172.21.15.33:0/3516744856 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4826078f0 0x55d48262a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.813+0000 7f1b1e59c640 1 -- 172.21.15.33:0/3516744856 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d482612ad0 msgr2=0x55d48260c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.813+0000 7f1b1e59c640 1 --2- 172.21.15.33:0/3516744856 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d482612ad0 0x55d48260c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.813+0000 7f1b1e59c640 1 -- 172.21.15.33:0/3516744856 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4826078f0 msgr2=0x55d48262a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.813+0000 7f1b1e59c640 1 --2- 172.21.15.33:0/3516744856 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4826078f0 0x55d48262a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.813+0000 7f1b1e59c640 1 -- 172.21.15.33:0/3516744856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d48260dc60 con 0x55d482608210 2024-05-15T14:01:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.813+0000 7f1b1ed9d640 1 --2- 172.21.15.33:0/3516744856 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4826078f0 0x55d48262a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:01:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.814+0000 7f1b1e59c640 1 --2- 172.21.15.33:0/3516744856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d482608210 0x55d48262a950 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1b14033000 tx=0x7f1b14003870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=591c18d522d5d038 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.814+0000 7f1b04ff9640 1 -- 172.21.15.33:0/3516744856 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b14035040 con 0x55d482608210 2024-05-15T14:01:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.814+0000 7f1b04ff9640 1 -- 172.21.15.33:0/3516744856 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b140054d0 con 0x55d482608210 2024-05-15T14:01:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.814+0000 7f1b04ff9640 1 -- 172.21.15.33:0/3516744856 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b14004840 con 0x55d482608210 2024-05-15T14:01:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.814+0000 7f1b2801d740 1 -- 172.21.15.33:0/3516744856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d482608210 msgr2=0x55d48262a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.814+0000 7f1b2801d740 1 --2- 172.21.15.33:0/3516744856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d482608210 0x55d48262a950 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1b14033000 tx=0x7f1b14003870 comp rx=0 tx=0).stop 2024-05-15T14:01:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.814+0000 7f1b2801d740 1 -- 172.21.15.33:0/3516744856 shutdown_connections 2024-05-15T14:01:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.814+0000 7f1b2801d740 1 --2- 172.21.15.33:0/3516744856 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d482612ad0 0x55d48260c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.814+0000 7f1b2801d740 1 --2- 172.21.15.33:0/3516744856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d482608210 0x55d48262a950 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.814+0000 7f1b2801d740 1 --2- 172.21.15.33:0/3516744856 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d4826078f0 0x55d48262a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.814+0000 7f1b2801d740 1 -- 172.21.15.33:0/3516744856 >> 172.21.15.33:0/3516744856 conn(0x55d482603e00 msgr2=0x55d48261d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.814+0000 7f1b2801d740 1 -- 172.21.15.33:0/3516744856 shutdown_connections 2024-05-15T14:01:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.814+0000 7f1b2801d740 1 -- 172.21.15.33:0/3516744856 wait complete. 2024-05-15T14:01:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.815+0000 7f1b2801d740 1 -- start start 2024-05-15T14:01:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.815+0000 7f1b2801d740 1 Processor -- start 2024-05-15T14:01:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.815+0000 7f1b2801d740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4826078f0 0x55d48261af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.815+0000 7f1b2801d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d482608210 0x55d4826182b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.815+0000 7f1b2801d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d482612ad0 0x55d4826187f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.815+0000 7f1b2801d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1b201acbd0 con 0x55d482612ad0 2024-05-15T14:01:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.815+0000 7f1b2801d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1b201ae040 con 0x55d482608210 2024-05-15T14:01:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.815+0000 7f1b2801d740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1b201ad2a0 con 0x55d4826078f0 2024-05-15T14:01:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.815+0000 7f1b2801d740 10 client.0 _ll_register_callbacks cb 0x7f1b10003b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:01:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.815+0000 7f1b1f59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d482612ad0 0x55d4826187f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.815+0000 7f1b1f59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d482612ad0 0x55d4826187f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53316/0 (socket says 172.21.15.33:53316) 2024-05-15T14:01:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.815+0000 7f1b1f59e640 1 -- 172.21.15.33:0/2089897684 learned_addr learned my addr 172.21.15.33:0/2089897684 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.815+0000 7f1b1ed9d640 1 --2- 172.21.15.33:0/2089897684 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4826078f0 0x55d48261af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.815+0000 7f1b1e59c640 1 --2- 172.21.15.33:0/2089897684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d482608210 0x55d4826182b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.815+0000 7f1b1f59e640 1 -- 172.21.15.33:0/2089897684 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4826078f0 msgr2=0x55d48261af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.815+0000 7f1b1f59e640 1 --2- 172.21.15.33:0/2089897684 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4826078f0 0x55d48261af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.815+0000 7f1b1f59e640 1 -- 172.21.15.33:0/2089897684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d482608210 msgr2=0x55d4826182b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.815+0000 7f1b1f59e640 1 --2- 172.21.15.33:0/2089897684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d482608210 0x55d4826182b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.815+0000 7f1b1f59e640 1 -- 172.21.15.33:0/2089897684 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d482619020 con 0x55d482612ad0 2024-05-15T14:01:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.815+0000 7f1b1e59c640 1 --2- 172.21.15.33:0/2089897684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d482608210 0x55d4826182b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:01:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.816+0000 7f1b1f59e640 1 --2- 172.21.15.33:0/2089897684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d482612ad0 0x55d4826187f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1b180a6ff0 tx=0x7f1b180aa3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ea04cd8a1ce2dc server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.816+0000 7f1b02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:01:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.816+0000 7f1b02ff5640 1 -- 172.21.15.33:0/2089897684 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b180a9d80 con 0x55d482612ad0 2024-05-15T14:01:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.816+0000 7f1b02ff5640 1 -- 172.21.15.33:0/2089897684 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f1b180a6370 con 0x55d482612ad0 2024-05-15T14:01:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.816+0000 7f1b2801d740 1 -- 172.21.15.33:0/2089897684 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b140054d0 con 0x55d482612ad0 2024-05-15T14:01:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.816+0000 7f1b02ff5640 1 -- 172.21.15.33:0/2089897684 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1b180ae130 con 0x55d482612ad0 2024-05-15T14:01:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.816+0000 7f1b2801d740 10 client.15435 fetch_fsmap learned FSMap version 24 2024-05-15T14:01:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.816+0000 7f1b2801d740 1 -- 172.21.15.33:0/2089897684 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d48268d040 con 0x55d482612ad0 2024-05-15T14:01:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.816+0000 7f1b02ff5640 1 -- 172.21.15.33:0/2089897684 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b140054d0 con 0x55d482612ad0 2024-05-15T14:01:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.816+0000 7f1b02ff5640 1 -- 172.21.15.33:0/2089897684 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f1b180a90e0 con 0x55d482612ad0 2024-05-15T14:01:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.816+0000 7f1b2801d740 10 client.15435 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:01:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.816+0000 7f1b2801d740 10 client.15435 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:01:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.816+0000 7f1b2801d740 20 client.15435 populate_metadata read hostname 'smithi033' 2024-05-15T14:01:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.816+0000 7f1b2801d740 1 --2- 172.21.15.33:0/2089897684 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55d48268d830 0x55d48268fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.816+0000 7f1b2801d740 4 client.15435 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:01:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.816+0000 7f1b2801d740 1 -- 172.21.15.33:0/2089897684 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d482576360 con 0x55d48268d830 2024-05-15T14:01:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.817+0000 7f1b1ed9d640 1 --2- 172.21.15.33:0/2089897684 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55d48268d830 0x55d48268fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.817+0000 7f1b1ed9d640 1 --2- 172.21.15.33:0/2089897684 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55d48268d830 0x55d48268fc20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=303c5673d1af0e4f server_cookie=5543120af7a0363c in_seq=0 out_seq=0 2024-05-15T14:01:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.817+0000 7f1b02ff5640 10 client.15435 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:01:16.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.824+0000 7f1b02ff5640 1 -- 172.21.15.33:0/2089897684 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d482576360 con 0x55d48268d830 2024-05-15T14:01:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.825+0000 7f1b02ff5640 10 client.15435 handle_command_reply: tid=0 2024-05-15T14:01:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.825+0000 7f1b2801d740 1 client.15435 shutdown 2024-05-15T14:01:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.825+0000 7f1b2801d740 1 -- 172.21.15.33:0/2089897684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d482612ad0 msgr2=0x55d4826187f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.825+0000 7f1b2801d740 1 --2- 172.21.15.33:0/2089897684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d482612ad0 0x55d4826187f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1b180a6ff0 tx=0x7f1b180aa3a0 comp rx=0 tx=0).stop 2024-05-15T14:01:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.825+0000 7f1b2801d740 1 -- 172.21.15.33:0/2089897684 shutdown_connections 2024-05-15T14:01:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.825+0000 7f1b2801d740 1 --2- 172.21.15.33:0/2089897684 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55d48268d830 0x55d48268fc20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.825+0000 7f1b2801d740 1 --2- 172.21.15.33:0/2089897684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d482612ad0 0x55d4826187f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.825+0000 7f1b2801d740 1 --2- 172.21.15.33:0/2089897684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d482608210 0x55d4826182b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.825+0000 7f1b02ff5640 0 client.15435 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:01:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.825+0000 7f1b2801d740 1 --2- 172.21.15.33:0/2089897684 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d4826078f0 0x55d48261af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.825+0000 7f1b2801d740 1 -- 172.21.15.33:0/2089897684 >> 172.21.15.33:0/2089897684 conn(0x55d482603e00 msgr2=0x55d48261de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.825+0000 7f1b2801d740 1 -- 172.21.15.33:0/2089897684 shutdown_connections 2024-05-15T14:01:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.825+0000 7f1b2801d740 1 -- 172.21.15.33:0/2089897684 wait complete. 2024-05-15T14:01:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.825+0000 7f1b2801d740 20 client.15435 trim_cache size 0 max 16384 2024-05-15T14:01:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:01:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:01:16.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:01:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.825+0000 7f1b25223640 1 -- 172.21.15.33:0/4209194241 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f1b080b2a70 msgr2=0x7f1b080b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.825+0000 7f1b25223640 1 --2- 172.21.15.33:0/4209194241 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f1b080b2a70 0x7f1b080b4f30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x55d482608e90 tx=0x7f1b1000b390 comp rx=0 tx=0).stop 2024-05-15T14:01:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.825+0000 7f1b25223640 1 -- 172.21.15.33:0/4209194241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b20108a50 msgr2=0x7f1b2019dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.825+0000 7f1b25223640 1 --2- 172.21.15.33:0/4209194241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b20108a50 0x7f1b2019dad0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f1b1800cce0 tx=0x7f1b1800a620 comp rx=0 tx=0).stop 2024-05-15T14:01:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.826+0000 7f1b25223640 1 -- 172.21.15.33:0/4209194241 shutdown_connections 2024-05-15T14:01:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.826+0000 7f1b25223640 1 --2- 172.21.15.33:0/4209194241 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f1b080b2a70 0x7f1b080b4f30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.826+0000 7f1b25223640 1 --2- 172.21.15.33:0/4209194241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b20108a50 0x7f1b2019dad0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.826+0000 7f1b25223640 1 --2- 172.21.15.33:0/4209194241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1b20103400 0x7f1b20199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:16.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.826+0000 7f1b25223640 1 --2- 172.21.15.33:0/4209194241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1b20102a40 0x7f1b20199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.826+0000 7f1b25223640 1 -- 172.21.15.33:0/4209194241 >> 172.21.15.33:0/4209194241 conn(0x7f1b200fe510 msgr2=0x7f1b200fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.826+0000 7f1b25223640 1 -- 172.21.15.33:0/4209194241 shutdown_connections 2024-05-15T14:01:16.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:16.826+0000 7f1b25223640 1 -- 172.21.15.33:0/4209194241 wait complete. 2024-05-15T14:01:16.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:16 smithi052 ceph-mon[36190]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-05-15T14:01:16.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:16 smithi081 ceph-mon[36434]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-05-15T14:01:18.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:18 smithi033 ceph-mon[31650]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.5 KiB/s wr, 10 op/s 2024-05-15T14:01:18.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:18 smithi052 ceph-mon[36190]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.5 KiB/s wr, 10 op/s 2024-05-15T14:01:18.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:18 smithi081 ceph-mon[36434]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.5 KiB/s wr, 10 op/s 2024-05-15T14:01:20.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:20 smithi033 ceph-mon[31650]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-05-15T14:01:20.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:20 smithi052 ceph-mon[36190]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-05-15T14:01:20.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:20 smithi081 ceph-mon[36434]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-05-15T14:01:22.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:22 smithi033 ceph-mon[31650]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-05-15T14:01:22.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:22 smithi052 ceph-mon[36190]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-05-15T14:01:22.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:22 smithi081 ceph-mon[36434]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-05-15T14:01:24.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:24 smithi052 ceph-mon[36190]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-05-15T14:01:24.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:24 smithi081 ceph-mon[36434]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-05-15T14:01:24.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:24 smithi033 ceph-mon[31650]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-05-15T14:01:26.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:26 smithi052 ceph-mon[36190]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-05-15T14:01:26.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:26 smithi081 ceph-mon[36434]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-05-15T14:01:26.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:26 smithi033 ceph-mon[31650]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-05-15T14:01:28.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:28 smithi052 ceph-mon[36190]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-15T14:01:28.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:28 smithi081 ceph-mon[36434]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-15T14:01:28.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:28 smithi033 ceph-mon[31650]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-15T14:01:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.916+0000 7fea596c8640 1 -- 172.21.15.33:0/1962057283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea54102310 msgr2=0x7fea5410e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.916+0000 7fea596c8640 1 --2- 172.21.15.33:0/1962057283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea54102310 0x7fea5410e820 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fea4c009cd0 tx=0x7fea4c02a8e0 comp rx=0 tx=0).stop 2024-05-15T14:01:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.916+0000 7fea596c8640 1 -- 172.21.15.33:0/1962057283 shutdown_connections 2024-05-15T14:01:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.916+0000 7fea596c8640 1 --2- 172.21.15.33:0/1962057283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea54102310 0x7fea5410e820 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.916+0000 7fea596c8640 1 --2- 172.21.15.33:0/1962057283 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea54101950 0x7fea54101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.916+0000 7fea596c8640 1 --2- 172.21.15.33:0/1962057283 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea54107960 0x7fea54107d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.916+0000 7fea596c8640 1 -- 172.21.15.33:0/1962057283 >> 172.21.15.33:0/1962057283 conn(0x7fea54078ad0 msgr2=0x7fea54078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.916+0000 7fea596c8640 1 -- 172.21.15.33:0/1962057283 shutdown_connections 2024-05-15T14:01:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.916+0000 7fea596c8640 1 -- 172.21.15.33:0/1962057283 wait complete. 2024-05-15T14:01:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.917+0000 7fea596c8640 1 Processor -- start 2024-05-15T14:01:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.917+0000 7fea596c8640 1 -- start start 2024-05-15T14:01:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.917+0000 7fea596c8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea54101950 0x7fea54103fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.917+0000 7fea596c8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea54102310 0x7fea54104500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.917+0000 7fea596c8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea54107960 0x7fea54104a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.917+0000 7fea596c8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fea54111c20 con 0x7fea54101950 2024-05-15T14:01:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.917+0000 7fea596c8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fea54111aa0 con 0x7fea54107960 2024-05-15T14:01:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.917+0000 7fea596c8640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fea54111da0 con 0x7fea54102310 2024-05-15T14:01:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.917+0000 7fea52ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea54101950 0x7fea54103fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.917+0000 7fea52ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea54101950 0x7fea54103fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34006/0 (socket says 172.21.15.33:34006) 2024-05-15T14:01:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.917+0000 7fea52ffd640 1 -- 172.21.15.33:0/4241528990 learned_addr learned my addr 172.21.15.33:0/4241528990 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.917+0000 7fea527fc640 1 --2- 172.21.15.33:0/4241528990 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea54102310 0x7fea54104500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.917+0000 7fea537fe640 1 --2- 172.21.15.33:0/4241528990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea54107960 0x7fea54104a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.918+0000 7fea52ffd640 1 -- 172.21.15.33:0/4241528990 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea54102310 msgr2=0x7fea54104500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.918+0000 7fea52ffd640 1 --2- 172.21.15.33:0/4241528990 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea54102310 0x7fea54104500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.918+0000 7fea52ffd640 1 -- 172.21.15.33:0/4241528990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea54107960 msgr2=0x7fea54104a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.918+0000 7fea52ffd640 1 --2- 172.21.15.33:0/4241528990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea54107960 0x7fea54104a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.918+0000 7fea52ffd640 1 -- 172.21.15.33:0/4241528990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea541a7750 con 0x7fea54101950 2024-05-15T14:01:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.918+0000 7fea527fc640 1 --2- 172.21.15.33:0/4241528990 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea54102310 0x7fea54104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:01:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.918+0000 7fea52ffd640 1 --2- 172.21.15.33:0/4241528990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea54101950 0x7fea54103fc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fea4400e9f0 tx=0x7fea4400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.918+0000 7fea3bfff640 1 -- 172.21.15.33:0/4241528990 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4400cd80 con 0x7fea54101950 2024-05-15T14:01:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.918+0000 7fea596c8640 1 -- 172.21.15.33:0/4241528990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea541a7a40 con 0x7fea54101950 2024-05-15T14:01:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.918+0000 7fea3bfff640 1 -- 172.21.15.33:0/4241528990 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea440108e0 con 0x7fea54101950 2024-05-15T14:01:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.918+0000 7fea3bfff640 1 -- 172.21.15.33:0/4241528990 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea44018980 con 0x7fea54101950 2024-05-15T14:01:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.918+0000 7fea596c8640 1 -- 172.21.15.33:0/4241528990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea541a7fe0 con 0x7fea54101950 2024-05-15T14:01:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.921+0000 7fea3bfff640 1 -- 172.21.15.33:0/4241528990 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7fea44010430 con 0x7fea54101950 2024-05-15T14:01:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.921+0000 7fea5c3e9740 1 Processor -- start 2024-05-15T14:01:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.921+0000 7fea5c3e9740 1 -- start start 2024-05-15T14:01:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.921+0000 7fea5c3e9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfae7e8f0 0x55fdfae7ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.921+0000 7fea5c3e9740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdfae7f210 0x55fdfae89590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.921+0000 7fea5c3e9740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdfae89ad0 0x55fdfae8bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.921+0000 7fea5c3e9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fdfadf1c30 con 0x55fdfae7e8f0 2024-05-15T14:01:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.921+0000 7fea5c3e9740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fdfade17d0 con 0x55fdfae89ad0 2024-05-15T14:01:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.921+0000 7fea5c3e9740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55fdfade5810 con 0x55fdfae7f210 2024-05-15T14:01:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.921+0000 7fea537fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfae7e8f0 0x55fdfae7ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.921+0000 7fea537fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfae7e8f0 0x55fdfae7ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34010/0 (socket says 172.21.15.33:34010) 2024-05-15T14:01:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.921+0000 7fea537fe640 1 -- 172.21.15.33:0/4261179774 learned_addr learned my addr 172.21.15.33:0/4261179774 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.921+0000 7fea52ffd640 1 --2- 172.21.15.33:0/4261179774 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdfae7f210 0x55fdfae89590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.921+0000 7fea527fc640 1 --2- 172.21.15.33:0/4261179774 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdfae89ad0 0x55fdfae8bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.922+0000 7fea537fe640 1 -- 172.21.15.33:0/4261179774 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdfae7f210 msgr2=0x55fdfae89590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.922+0000 7fea537fe640 1 --2- 172.21.15.33:0/4261179774 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdfae7f210 0x55fdfae89590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.922+0000 7fea537fe640 1 -- 172.21.15.33:0/4261179774 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdfae89ad0 msgr2=0x55fdfae8bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.922+0000 7fea537fe640 1 --2- 172.21.15.33:0/4261179774 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdfae89ad0 0x55fdfae8bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.922+0000 7fea537fe640 1 -- 172.21.15.33:0/4261179774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdfaaf6210 con 0x55fdfae7e8f0 2024-05-15T14:01:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.922+0000 7fea3bfff640 1 --2- 172.21.15.33:0/4241528990 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fea3c0b2a70 0x7fea3c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.922+0000 7fea537fe640 1 --2- 172.21.15.33:0/4261179774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfae7e8f0 0x55fdfae7ecd0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fea4c009cd0 tx=0x7fea4c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff78a3a294bafeb server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.922+0000 7fea52ffd640 1 --2- 172.21.15.33:0/4241528990 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fea3c0b2a70 0x7fea3c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.922+0000 7fea397fa640 1 -- 172.21.15.33:0/4261179774 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c049070 con 0x55fdfae7e8f0 2024-05-15T14:01:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.922+0000 7fea397fa640 1 -- 172.21.15.33:0/4261179774 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea4c036040 con 0x55fdfae7e8f0 2024-05-15T14:01:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.922+0000 7fea397fa640 1 -- 172.21.15.33:0/4261179774 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c004040 con 0x55fdfae7e8f0 2024-05-15T14:01:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.922+0000 7fea3bfff640 1 -- 172.21.15.33:0/4241528990 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fea440e2b50 con 0x7fea54101950 2024-05-15T14:01:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.923+0000 7fea5c3e9740 1 -- 172.21.15.33:0/4261179774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfae7e8f0 msgr2=0x55fdfae7ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.923+0000 7fea5c3e9740 1 --2- 172.21.15.33:0/4261179774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfae7e8f0 0x55fdfae7ecd0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fea4c009cd0 tx=0x7fea4c009a20 comp rx=0 tx=0).stop 2024-05-15T14:01:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.923+0000 7fea52ffd640 1 --2- 172.21.15.33:0/4241528990 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fea3c0b2a70 0x7fea3c0b4f30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fea440e0190 tx=0x7fea440e6660 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.923+0000 7fea5c3e9740 1 -- 172.21.15.33:0/4261179774 shutdown_connections 2024-05-15T14:01:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.923+0000 7fea5c3e9740 1 --2- 172.21.15.33:0/4261179774 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdfae89ad0 0x55fdfae8bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.923+0000 7fea5c3e9740 1 --2- 172.21.15.33:0/4261179774 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdfae7f210 0x55fdfae89590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.923+0000 7fea5c3e9740 1 --2- 172.21.15.33:0/4261179774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfae7e8f0 0x55fdfae7ecd0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.923+0000 7fea5c3e9740 1 -- 172.21.15.33:0/4261179774 >> 172.21.15.33:0/4261179774 conn(0x55fdfae7ae00 msgr2=0x55fdfae7b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.923+0000 7fea5c3e9740 1 -- 172.21.15.33:0/4261179774 shutdown_connections 2024-05-15T14:01:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.923+0000 7fea5c3e9740 1 -- 172.21.15.33:0/4261179774 wait complete. 2024-05-15T14:01:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.923+0000 7fea5c3e9740 1 -- start start 2024-05-15T14:01:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.923+0000 7fea5c3e9740 1 Processor -- start 2024-05-15T14:01:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.924+0000 7fea5c3e9740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdfae7e8f0 0x55fdfae894a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.924+0000 7fea5c3e9740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdfae7f210 0x55fdfae86790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.924+0000 7fea5c3e9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfae89ad0 0x55fdfae86da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.924+0000 7fea5c3e9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fdfaac7790 con 0x55fdfae89ad0 2024-05-15T14:01:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.924+0000 7fea5c3e9740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fdfad9d890 con 0x55fdfae7e8f0 2024-05-15T14:01:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.924+0000 7fea5c3e9740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55fdfae8c740 con 0x55fdfae7f210 2024-05-15T14:01:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.924+0000 7fea537fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfae89ad0 0x55fdfae86da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.924+0000 7fea5c3e9740 10 client.0 _ll_register_callbacks cb 0x55fdfada0d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:01:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.924+0000 7fea537fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfae89ad0 0x55fdfae86da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34022/0 (socket says 172.21.15.33:34022) 2024-05-15T14:01:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.924+0000 7fea537fe640 1 -- 172.21.15.33:0/1567983129 learned_addr learned my addr 172.21.15.33:0/1567983129 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.924+0000 7fea527fc640 1 --2- 172.21.15.33:0/1567983129 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdfae7f210 0x55fdfae86790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.924+0000 7fea537fe640 1 --2- 172.21.15.33:0/1567983129 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdfae7e8f0 0x55fdfae894a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.924+0000 7fea537fe640 1 -- 172.21.15.33:0/1567983129 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdfae7f210 msgr2=0x55fdfae86790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.924+0000 7fea537fe640 1 --2- 172.21.15.33:0/1567983129 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdfae7f210 0x55fdfae86790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.924+0000 7fea537fe640 1 -- 172.21.15.33:0/1567983129 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdfae7e8f0 msgr2=0x55fdfae894a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.924+0000 7fea537fe640 1 --2- 172.21.15.33:0/1567983129 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdfae7e8f0 0x55fdfae894a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.924+0000 7fea537fe640 1 -- 172.21.15.33:0/1567983129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdfabc2620 con 0x55fdfae89ad0 2024-05-15T14:01:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.924+0000 7fea537fe640 1 --2- 172.21.15.33:0/1567983129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfae89ad0 0x55fdfae86da0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fea4c009620 tx=0x7fea4c003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a28d90996e25ba server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.924+0000 7fea36ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:01:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.924+0000 7fea36ff5640 1 -- 172.21.15.33:0/1567983129 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c049040 con 0x55fdfae89ad0 2024-05-15T14:01:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.924+0000 7fea36ff5640 1 -- 172.21.15.33:0/1567983129 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fea4c004220 con 0x55fdfae89ad0 2024-05-15T14:01:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.924+0000 7fea5c3e9740 1 -- 172.21.15.33:0/1567983129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea4c036040 con 0x55fdfae89ad0 2024-05-15T14:01:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.925+0000 7fea36ff5640 1 -- 172.21.15.33:0/1567983129 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fea4c031c90 con 0x55fdfae89ad0 2024-05-15T14:01:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.925+0000 7fea5c3e9740 10 client.15447 fetch_fsmap learned FSMap version 24 2024-05-15T14:01:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.925+0000 7fea5c3e9740 1 -- 172.21.15.33:0/1567983129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fdfae981a0 con 0x55fdfae89ad0 2024-05-15T14:01:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.925+0000 7fea36ff5640 1 -- 172.21.15.33:0/1567983129 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea4c036040 con 0x55fdfae89ad0 2024-05-15T14:01:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.925+0000 7fea36ff5640 1 -- 172.21.15.33:0/1567983129 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fea4c04f070 con 0x55fdfae89ad0 2024-05-15T14:01:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.925+0000 7fea5c3e9740 10 client.15447 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:01:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.925+0000 7fea5c3e9740 10 client.15447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:01:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.925+0000 7fea5c3e9740 20 client.15447 populate_metadata read hostname 'smithi033' 2024-05-15T14:01:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.925+0000 7fea5c3e9740 1 --2- 172.21.15.33:0/1567983129 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55fdfae994f0 0x55fdfae9b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.925+0000 7fea5c3e9740 4 client.15447 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:01:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.925+0000 7fea5c3e9740 1 -- 172.21.15.33:0/1567983129 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdfadf3b80 con 0x55fdfae994f0 2024-05-15T14:01:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.925+0000 7fea527fc640 1 --2- 172.21.15.33:0/1567983129 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55fdfae994f0 0x55fdfae9b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.926+0000 7fea527fc640 1 --2- 172.21.15.33:0/1567983129 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55fdfae994f0 0x55fdfae9b8e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=183344693ad7315 server_cookie=ddb2af8f0580603b in_seq=0 out_seq=0 2024-05-15T14:01:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.926+0000 7fea36ff5640 10 client.15447 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:01:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.926+0000 7fea36ff5640 1 -- 172.21.15.33:0/1567983129 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fdfadf3b80 con 0x55fdfae994f0 2024-05-15T14:01:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.926+0000 7fea36ff5640 10 client.15447 handle_command_reply: tid=0 2024-05-15T14:01:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.926+0000 7fea5c3e9740 1 client.15447 shutdown 2024-05-15T14:01:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.926+0000 7fea5c3e9740 1 -- 172.21.15.33:0/1567983129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfae89ad0 msgr2=0x55fdfae86da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.926+0000 7fea5c3e9740 1 --2- 172.21.15.33:0/1567983129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfae89ad0 0x55fdfae86da0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fea4c009620 tx=0x7fea4c003c30 comp rx=0 tx=0).stop 2024-05-15T14:01:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.926+0000 7fea5c3e9740 1 -- 172.21.15.33:0/1567983129 shutdown_connections 2024-05-15T14:01:29.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.926+0000 7fea5c3e9740 1 --2- 172.21.15.33:0/1567983129 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55fdfae994f0 0x55fdfae9b8e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.926+0000 7fea5c3e9740 1 --2- 172.21.15.33:0/1567983129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfae89ad0 0x55fdfae86da0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.926+0000 7fea5c3e9740 1 --2- 172.21.15.33:0/1567983129 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdfae7f210 0x55fdfae86790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.926+0000 7fea36ff5640 0 client.15447 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:01:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.926+0000 7fea5c3e9740 1 --2- 172.21.15.33:0/1567983129 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdfae7e8f0 0x55fdfae894a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.926+0000 7fea5c3e9740 1 -- 172.21.15.33:0/1567983129 >> 172.21.15.33:0/1567983129 conn(0x55fdfae7ae00 msgr2=0x55fdfae7a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.927+0000 7fea5c3e9740 1 -- 172.21.15.33:0/1567983129 shutdown_connections 2024-05-15T14:01:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.927+0000 7fea5c3e9740 1 -- 172.21.15.33:0/1567983129 wait complete. 2024-05-15T14:01:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.927+0000 7fea5c3e9740 20 client.15447 trim_cache size 0 max 16384 2024-05-15T14:01:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.942+0000 7fea5c3e9740 1 Processor -- start 2024-05-15T14:01:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.942+0000 7fea5c3e9740 1 -- start start 2024-05-15T14:01:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.942+0000 7fea5c3e9740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdfae7e8f0 0x55fdfae88fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.942+0000 7fea5c3e9740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdfae7f210 0x55fdfae9e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.942+0000 7fea5c3e9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfae89ad0 0x55fdfae9ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:29.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.942+0000 7fea5c3e9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fea541b3150 con 0x55fdfae89ad0 2024-05-15T14:01:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.942+0000 7fea5c3e9740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fea541b0a20 con 0x55fdfae7f210 2024-05-15T14:01:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.942+0000 7fea5c3e9740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fea541ae870 con 0x55fdfae7e8f0 2024-05-15T14:01:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.942+0000 7fea537fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfae89ad0 0x55fdfae9ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.942+0000 7fea527fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdfae7f210 0x55fdfae9e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.942+0000 7fea537fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfae89ad0 0x55fdfae9ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34038/0 (socket says 172.21.15.33:34038) 2024-05-15T14:01:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.942+0000 7fea527fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdfae7f210 0x55fdfae9e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:47976/0 (socket says 172.21.15.33:47976) 2024-05-15T14:01:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.942+0000 7fea537fe640 1 -- 172.21.15.33:0/1778907006 learned_addr learned my addr 172.21.15.33:0/1778907006 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.942+0000 7fea537fe640 1 --2- 172.21.15.33:0/1778907006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdfae7e8f0 0x55fdfae88fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.943+0000 7fea537fe640 1 -- 172.21.15.33:0/1778907006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdfae7e8f0 msgr2=0x55fdfae88fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:29.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.943+0000 7fea537fe640 1 --2- 172.21.15.33:0/1778907006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdfae7e8f0 0x55fdfae88fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.943+0000 7fea537fe640 1 -- 172.21.15.33:0/1778907006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdfae7f210 msgr2=0x55fdfae9e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.943+0000 7fea537fe640 1 --2- 172.21.15.33:0/1778907006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdfae7f210 0x55fdfae9e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.943+0000 7fea537fe640 1 -- 172.21.15.33:0/1778907006 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdfae9f380 con 0x55fdfae89ad0 2024-05-15T14:01:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.943+0000 7fea527fc640 1 --2- 172.21.15.33:0/1778907006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdfae7f210 0x55fdfae9e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:01:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.943+0000 7fea537fe640 1 --2- 172.21.15.33:0/1778907006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfae89ad0 0x55fdfae9ee40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x55fdfae874e0 tx=0x7fea4c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2938c1b7f5c4cff server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.943+0000 7fea38ff9640 1 -- 172.21.15.33:0/1778907006 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c03f040 con 0x55fdfae89ad0 2024-05-15T14:01:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.943+0000 7fea38ff9640 1 -- 172.21.15.33:0/1778907006 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea4c036070 con 0x55fdfae89ad0 2024-05-15T14:01:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.943+0000 7fea38ff9640 1 -- 172.21.15.33:0/1778907006 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c004040 con 0x55fdfae89ad0 2024-05-15T14:01:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.944+0000 7fea5c3e9740 1 -- 172.21.15.33:0/1778907006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfae89ad0 msgr2=0x55fdfae9ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.944+0000 7fea5c3e9740 1 --2- 172.21.15.33:0/1778907006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfae89ad0 0x55fdfae9ee40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x55fdfae874e0 tx=0x7fea4c009a20 comp rx=0 tx=0).stop 2024-05-15T14:01:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.944+0000 7fea5c3e9740 1 -- 172.21.15.33:0/1778907006 shutdown_connections 2024-05-15T14:01:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.944+0000 7fea5c3e9740 1 --2- 172.21.15.33:0/1778907006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfae89ad0 0x55fdfae9ee40 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.944+0000 7fea5c3e9740 1 --2- 172.21.15.33:0/1778907006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdfae7f210 0x55fdfae9e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.944+0000 7fea5c3e9740 1 --2- 172.21.15.33:0/1778907006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdfae7e8f0 0x55fdfae88fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.944+0000 7fea5c3e9740 1 -- 172.21.15.33:0/1778907006 >> 172.21.15.33:0/1778907006 conn(0x55fdfae7ae00 msgr2=0x55fdfae81ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.944+0000 7fea5c3e9740 1 -- 172.21.15.33:0/1778907006 shutdown_connections 2024-05-15T14:01:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.944+0000 7fea5c3e9740 1 -- 172.21.15.33:0/1778907006 wait complete. 2024-05-15T14:01:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.944+0000 7fea5c3e9740 1 -- start start 2024-05-15T14:01:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.944+0000 7fea5c3e9740 1 Processor -- start 2024-05-15T14:01:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.944+0000 7fea5c3e9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfae7e8f0 0x55fdfae9ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.944+0000 7fea5c3e9740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdfae7f210 0x55fdfae9beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.944+0000 7fea5c3e9740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdfae89ad0 0x55fdfae9c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.944+0000 7fea5c3e9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fea541aeee0 con 0x55fdfae7e8f0 2024-05-15T14:01:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.944+0000 7fea5c3e9740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fea541b0350 con 0x55fdfae89ad0 2024-05-15T14:01:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.944+0000 7fea5c3e9740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fea541af5b0 con 0x55fdfae7f210 2024-05-15T14:01:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.944+0000 7fea5c3e9740 10 client.0 _ll_register_callbacks cb 0x7fea4c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:01:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.945+0000 7fea537fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfae7e8f0 0x55fdfae9ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.945+0000 7fea537fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfae7e8f0 0x55fdfae9ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34044/0 (socket says 172.21.15.33:34044) 2024-05-15T14:01:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.945+0000 7fea537fe640 1 -- 172.21.15.33:0/2436912644 learned_addr learned my addr 172.21.15.33:0/2436912644 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:29.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.945+0000 7fea527fc640 1 --2- 172.21.15.33:0/2436912644 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdfae7f210 0x55fdfae9beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:29.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.945+0000 7fea537fe640 1 --2- 172.21.15.33:0/2436912644 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdfae89ad0 0x55fdfae9c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:29.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.945+0000 7fea537fe640 1 -- 172.21.15.33:0/2436912644 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdfae7f210 msgr2=0x55fdfae9beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:29.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.945+0000 7fea537fe640 1 --2- 172.21.15.33:0/2436912644 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdfae7f210 0x55fdfae9beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:29.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.945+0000 7fea537fe640 1 -- 172.21.15.33:0/2436912644 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdfae89ad0 msgr2=0x55fdfae9c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:29.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.945+0000 7fea537fe640 1 --2- 172.21.15.33:0/2436912644 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdfae89ad0 0x55fdfae9c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:29.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.945+0000 7fea537fe640 1 -- 172.21.15.33:0/2436912644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdfae9ca30 con 0x55fdfae7e8f0 2024-05-15T14:01:29.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.945+0000 7fea537fe640 1 --2- 172.21.15.33:0/2436912644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfae7e8f0 0x55fdfae9ead0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fea4c009bf0 tx=0x7fea4c030b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7efbfc9ba71ca398 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:29.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.945+0000 7fea36ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:01:29.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.945+0000 7fea36ff5640 1 -- 172.21.15.33:0/2436912644 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c03f070 con 0x55fdfae7e8f0 2024-05-15T14:01:29.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.945+0000 7fea36ff5640 1 -- 172.21.15.33:0/2436912644 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fea4c003dd0 con 0x55fdfae7e8f0 2024-05-15T14:01:29.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.945+0000 7fea5c3e9740 1 -- 172.21.15.33:0/2436912644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea4c036070 con 0x55fdfae7e8f0 2024-05-15T14:01:29.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.945+0000 7fea36ff5640 1 -- 172.21.15.33:0/2436912644 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fea4c049040 con 0x55fdfae7e8f0 2024-05-15T14:01:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.945+0000 7fea5c3e9740 10 client.15453 fetch_fsmap learned FSMap version 24 2024-05-15T14:01:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.945+0000 7fea5c3e9740 1 -- 172.21.15.33:0/2436912644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fdfaf03890 con 0x55fdfae7e8f0 2024-05-15T14:01:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.946+0000 7fea36ff5640 1 -- 172.21.15.33:0/2436912644 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea4c036070 con 0x55fdfae7e8f0 2024-05-15T14:01:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.946+0000 7fea36ff5640 1 -- 172.21.15.33:0/2436912644 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fea4c04f020 con 0x55fdfae7e8f0 2024-05-15T14:01:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.946+0000 7fea5c3e9740 10 client.15453 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:01:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.946+0000 7fea5c3e9740 10 client.15453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:01:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.946+0000 7fea5c3e9740 20 client.15453 populate_metadata read hostname 'smithi033' 2024-05-15T14:01:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.946+0000 7fea5c3e9740 1 --2- 172.21.15.33:0/2436912644 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55fdfaf03d00 0x55fdfaf060f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.946+0000 7fea5c3e9740 4 client.15453 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:01:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.946+0000 7fea5c3e9740 1 -- 172.21.15.33:0/2436912644 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fdfaded360 con 0x55fdfaf03d00 2024-05-15T14:01:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.946+0000 7fea527fc640 1 --2- 172.21.15.33:0/2436912644 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55fdfaf03d00 0x55fdfaf060f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.946+0000 7fea36ff5640 10 client.15453 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:01:29.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.946+0000 7fea527fc640 1 --2- 172.21.15.33:0/2436912644 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55fdfaf03d00 0x55fdfaf060f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=229fb084b97d7a9d server_cookie=64be68f80ec2ef45 in_seq=0 out_seq=0 2024-05-15T14:01:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.949+0000 7fea36ff5640 1 -- 172.21.15.33:0/2436912644 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fdfaded360 con 0x55fdfaf03d00 2024-05-15T14:01:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.949+0000 7fea36ff5640 10 client.15453 handle_command_reply: tid=0 2024-05-15T14:01:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.949+0000 7fea5c3e9740 1 client.15453 shutdown 2024-05-15T14:01:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.949+0000 7fea5c3e9740 1 -- 172.21.15.33:0/2436912644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfae7e8f0 msgr2=0x55fdfae9ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.949+0000 7fea5c3e9740 1 --2- 172.21.15.33:0/2436912644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfae7e8f0 0x55fdfae9ead0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fea4c009bf0 tx=0x7fea4c030b30 comp rx=0 tx=0).stop 2024-05-15T14:01:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.949+0000 7fea5c3e9740 1 -- 172.21.15.33:0/2436912644 shutdown_connections 2024-05-15T14:01:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.949+0000 7fea5c3e9740 1 --2- 172.21.15.33:0/2436912644 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55fdfaf03d00 0x55fdfaf060f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.949+0000 7fea5c3e9740 1 --2- 172.21.15.33:0/2436912644 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fdfae89ad0 0x55fdfae9c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.949+0000 7fea5c3e9740 1 --2- 172.21.15.33:0/2436912644 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdfae7f210 0x55fdfae9beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.949+0000 7fea36ff5640 0 client.15453 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:01:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.949+0000 7fea5c3e9740 1 --2- 172.21.15.33:0/2436912644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fdfae7e8f0 0x55fdfae9ead0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.949+0000 7fea5c3e9740 1 -- 172.21.15.33:0/2436912644 >> 172.21.15.33:0/2436912644 conn(0x55fdfae7ae00 msgr2=0x55fdfae813f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.950+0000 7fea5c3e9740 1 -- 172.21.15.33:0/2436912644 shutdown_connections 2024-05-15T14:01:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.950+0000 7fea5c3e9740 1 -- 172.21.15.33:0/2436912644 wait complete. 2024-05-15T14:01:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.950+0000 7fea5c3e9740 20 client.15453 trim_cache size 0 max 16384 2024-05-15T14:01:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:01:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:01:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:01:29.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.950+0000 7fea596c8640 1 -- 172.21.15.33:0/4241528990 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fea3c0b2a70 msgr2=0x7fea3c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:29.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.950+0000 7fea596c8640 1 --2- 172.21.15.33:0/4241528990 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fea3c0b2a70 0x7fea3c0b4f30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fea440e0190 tx=0x7fea440e6660 comp rx=0 tx=0).stop 2024-05-15T14:01:29.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.950+0000 7fea596c8640 1 -- 172.21.15.33:0/4241528990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea54101950 msgr2=0x7fea54103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:29.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.950+0000 7fea596c8640 1 --2- 172.21.15.33:0/4241528990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea54101950 0x7fea54103fc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fea4400e9f0 tx=0x7fea4400eeb0 comp rx=0 tx=0).stop 2024-05-15T14:01:29.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.950+0000 7fea596c8640 1 -- 172.21.15.33:0/4241528990 shutdown_connections 2024-05-15T14:01:29.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.950+0000 7fea596c8640 1 --2- 172.21.15.33:0/4241528990 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fea3c0b2a70 0x7fea3c0b4f30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:29.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.950+0000 7fea596c8640 1 --2- 172.21.15.33:0/4241528990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea54107960 0x7fea54104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:29.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.950+0000 7fea596c8640 1 --2- 172.21.15.33:0/4241528990 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea54102310 0x7fea54104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:29.959 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.950+0000 7fea596c8640 1 --2- 172.21.15.33:0/4241528990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea54101950 0x7fea54103fc0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:29.959 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.950+0000 7fea596c8640 1 -- 172.21.15.33:0/4241528990 >> 172.21.15.33:0/4241528990 conn(0x7fea54078ad0 msgr2=0x7fea5406b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:29.959 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.950+0000 7fea596c8640 1 -- 172.21.15.33:0/4241528990 shutdown_connections 2024-05-15T14:01:29.959 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:29.950+0000 7fea596c8640 1 -- 172.21.15.33:0/4241528990 wait complete. 2024-05-15T14:01:30.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:30 smithi052 ceph-mon[36190]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-15T14:01:30.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:30 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:01:30.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:30 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:01:30.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:30 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:01:30.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:30 smithi081 ceph-mon[36434]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-15T14:01:30.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:30 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:01:30.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:30 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:01:30.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:30 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:01:30.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:30 smithi033 ceph-mon[31650]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-15T14:01:30.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:30 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:01:30.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:30 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:01:30.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:30 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:01:31.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:31 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:01:31.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:31 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:01:31.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:31 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:01:31.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:31 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:01:31.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:31 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:01:31.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:31 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:01:31.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:31 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:01:31.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:31 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:01:31.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:31 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:01:32.370 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T14:01:32.634 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:01:32.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:32 smithi033 ceph-mon[31650]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-15T14:01:32.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:32 smithi052 ceph-mon[36190]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-15T14:01:32.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:32 smithi081 ceph-mon[36434]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-15T14:01:34.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.227+0000 7fa38caad700 1 -- 172.21.15.33:0/2433347952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa388108520 msgr2=0x7fa388108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:34.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.227+0000 7fa38caad700 1 --2- 172.21.15.33:0/2433347952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa388108520 0x7fa388108900 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa378009990 tx=0x7fa378009ca0 comp rx=0 tx=0).stop 2024-05-15T14:01:34.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.227+0000 7fa38caad700 1 -- 172.21.15.33:0/2433347952 shutdown_connections 2024-05-15T14:01:34.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.227+0000 7fa38caad700 1 --2- 172.21.15.33:0/2433347952 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa388102e00 0x7fa38810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:34.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.227+0000 7fa38caad700 1 --2- 172.21.15.33:0/2433347952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa388108520 0x7fa388108900 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:34.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.227+0000 7fa38caad700 1 --2- 172.21.15.33:0/2433347952 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa388102440 0x7fa3881028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:34.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.227+0000 7fa38caad700 1 -- 172.21.15.33:0/2433347952 >> 172.21.15.33:0/2433347952 conn(0x7fa3880793f0 msgr2=0x7fa388079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:34.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.228+0000 7fa38caad700 1 -- 172.21.15.33:0/2433347952 shutdown_connections 2024-05-15T14:01:34.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.228+0000 7fa38caad700 1 -- 172.21.15.33:0/2433347952 wait complete. 2024-05-15T14:01:34.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.228+0000 7fa38caad700 1 Processor -- start 2024-05-15T14:01:34.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.229+0000 7fa38caad700 1 -- start start 2024-05-15T14:01:34.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.229+0000 7fa38caad700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa388102440 0x7fa38819e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:34.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.229+0000 7fa38caad700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa388102e00 0x7fa38819ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:34.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.229+0000 7fa38caad700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa388108520 0x7fa3881988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:34.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.229+0000 7fa38caad700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa3881125d0 con 0x7fa388108520 2024-05-15T14:01:34.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.229+0000 7fa38caad700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa388112450 con 0x7fa388102e00 2024-05-15T14:01:34.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.229+0000 7fa38caad700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa388112750 con 0x7fa388102440 2024-05-15T14:01:34.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.229+0000 7fa386d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa388108520 0x7fa3881988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:34.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.229+0000 7fa386d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa388108520 0x7fa3881988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42426/0 (socket says 172.21.15.33:42426) 2024-05-15T14:01:34.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.229+0000 7fa386d9d700 1 -- 172.21.15.33:0/2087194566 learned_addr learned my addr 172.21.15.33:0/2087194566 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:34.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.229+0000 7fa38659c700 1 --2- 172.21.15.33:0/2087194566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa388102440 0x7fa38819e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:34.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.229+0000 7fa385d9b700 1 --2- 172.21.15.33:0/2087194566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa388102e00 0x7fa38819ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:34.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.229+0000 7fa386d9d700 1 -- 172.21.15.33:0/2087194566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa388102440 msgr2=0x7fa38819e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:34.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.229+0000 7fa386d9d700 1 --2- 172.21.15.33:0/2087194566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa388102440 0x7fa38819e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:34.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.229+0000 7fa386d9d700 1 -- 172.21.15.33:0/2087194566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa388102e00 msgr2=0x7fa38819ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:34.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.229+0000 7fa386d9d700 1 --2- 172.21.15.33:0/2087194566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa388102e00 0x7fa38819ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:34.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.229+0000 7fa386d9d700 1 -- 172.21.15.33:0/2087194566 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa388199050 con 0x7fa388108520 2024-05-15T14:01:34.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.229+0000 7fa385d9b700 1 --2- 172.21.15.33:0/2087194566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa388102e00 0x7fa38819ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:01:34.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.230+0000 7fa386d9d700 1 --2- 172.21.15.33:0/2087194566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa388108520 0x7fa3881988f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa380007f10 tx=0x7fa38000f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:34.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.230+0000 7fa3777fe700 1 -- 172.21.15.33:0/2087194566 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa380018070 con 0x7fa388108520 2024-05-15T14:01:34.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.230+0000 7fa3777fe700 1 -- 172.21.15.33:0/2087194566 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa380004020 con 0x7fa388108520 2024-05-15T14:01:34.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.230+0000 7fa38caad700 1 -- 172.21.15.33:0/2087194566 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa388199340 con 0x7fa388108520 2024-05-15T14:01:34.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.230+0000 7fa3777fe700 1 -- 172.21.15.33:0/2087194566 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3800126a0 con 0x7fa388108520 2024-05-15T14:01:34.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.230+0000 7fa38caad700 1 -- 172.21.15.33:0/2087194566 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa38806b220 con 0x7fa388108520 2024-05-15T14:01:34.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.231+0000 7fa38caad700 1 -- 172.21.15.33:0/2087194566 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa38806a230 con 0x7fa388108520 2024-05-15T14:01:34.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.232+0000 7fa3777fe700 1 -- 172.21.15.33:0/2087194566 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7fa3800044f0 con 0x7fa388108520 2024-05-15T14:01:34.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.233+0000 7fa3777fe700 1 --2- 172.21.15.33:0/2087194566 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fa3700b2b10 0x7fa3700b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:34.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.234+0000 7fa38659c700 1 --2- 172.21.15.33:0/2087194566 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fa3700b2b10 0x7fa3700b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:34.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.234+0000 7fa3777fe700 1 -- 172.21.15.33:0/2087194566 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fa3800e4f20 con 0x7fa388108520 2024-05-15T14:01:34.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.234+0000 7fa38659c700 1 --2- 172.21.15.33:0/2087194566 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fa3700b2b10 0x7fa3700b4fd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa388199a30 tx=0x7fa378005960 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:34.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.237+0000 7fa3777fe700 1 -- 172.21.15.33:0/2087194566 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa3800ac6c0 con 0x7fa388108520 2024-05-15T14:01:34.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.501+0000 7fa38caad700 1 -- 172.21.15.33:0/2087194566 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa38806bc50 con 0x7fa388108520 2024-05-15T14:01:34.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.502+0000 7fa3777fe700 1 -- 172.21.15.33:0/2087194566 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11858 (secure 0 0 0) 0x7fa3800abfc0 con 0x7fa388108520 2024-05-15T14:01:34.505 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T14:01:34.505 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2030117633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2030117633},{"type":"v1","addr":"172.21.15.33:6839","nonce":2030117633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1918979061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1918979061},{"type":"v1","addr":"172.21.15.33:6841","nonce":1918979061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T13:57:19.056417+0000","modified":"2024-05-15T14:00:18.543619+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24463,"mds_2":24461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/1629697841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1629697841},{"type":"v1","addr":"172.21.15.52:6837","nonce":1629697841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/3773839758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3773839758},{"type":"v1","addr":"172.21.15.81:6835","nonce":3773839758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.52:6839/2696044471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2696044471},{"type":"v1","addr":"172.21.15.52:6839","nonce":2696044471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.81:6839/1554395940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":1554395940},{"type":"v1","addr":"172.21.15.81:6839","nonce":1554395940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24463,24478,24461]},"id":1}]} 2024-05-15T14:01:34.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.504+0000 7fa38caad700 1 -- 172.21.15.33:0/2087194566 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fa3700b2b10 msgr2=0x7fa3700b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:34.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.504+0000 7fa38caad700 1 --2- 172.21.15.33:0/2087194566 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fa3700b2b10 0x7fa3700b4fd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa388199a30 tx=0x7fa378005960 comp rx=0 tx=0).stop 2024-05-15T14:01:34.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.504+0000 7fa38caad700 1 -- 172.21.15.33:0/2087194566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa388108520 msgr2=0x7fa3881988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:34.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.504+0000 7fa38caad700 1 --2- 172.21.15.33:0/2087194566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa388108520 0x7fa3881988f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa380007f10 tx=0x7fa38000f5c0 comp rx=0 tx=0).stop 2024-05-15T14:01:34.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.504+0000 7fa38caad700 1 -- 172.21.15.33:0/2087194566 shutdown_connections 2024-05-15T14:01:34.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.504+0000 7fa38caad700 1 --2- 172.21.15.33:0/2087194566 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fa3700b2b10 0x7fa3700b4fd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:34.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.504+0000 7fa38caad700 1 --2- 172.21.15.33:0/2087194566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa388108520 0x7fa3881988f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:34.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.504+0000 7fa38caad700 1 --2- 172.21.15.33:0/2087194566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa388102440 0x7fa38819e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:34.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.504+0000 7fa38caad700 1 --2- 172.21.15.33:0/2087194566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa388102e00 0x7fa38819ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:34.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.504+0000 7fa38caad700 1 -- 172.21.15.33:0/2087194566 >> 172.21.15.33:0/2087194566 conn(0x7fa3880793f0 msgr2=0x7fa388106410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:34.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.505+0000 7fa38caad700 1 -- 172.21.15.33:0/2087194566 shutdown_connections 2024-05-15T14:01:34.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:34.505+0000 7fa38caad700 1 -- 172.21.15.33:0/2087194566 wait complete. 2024-05-15T14:01:34.508 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 24 2024-05-15T14:01:34.820 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:34 smithi033 ceph-mon[31650]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 8 op/s 2024-05-15T14:01:34.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:34 smithi052 ceph-mon[36190]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 8 op/s 2024-05-15T14:01:34.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:34 smithi081 ceph-mon[36434]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 8 op/s 2024-05-15T14:01:35.279 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24478 quiesce db --set-id 2fa87dae --release --await' 2024-05-15T14:01:35.279 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24478 quiesce db --set-id 2fa87dae --release --await 2024-05-15T14:01:35.539 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:01:35.662 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:35 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/2087194566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:01:35.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:35 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/2087194566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:01:35.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:35 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/2087194566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:01:36.796 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:36 smithi033 ceph-mon[31650]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-15T14:01:36.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:36 smithi052 ceph-mon[36190]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-15T14:01:36.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:36 smithi081 ceph-mon[36434]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-15T14:01:37.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.144+0000 7f47e9579700 1 -- 172.21.15.33:0/3884393391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e41041d0 msgr2=0x7f47e41081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:37.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.144+0000 7f47e9579700 1 --2- 172.21.15.33:0/3884393391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e41041d0 0x7f47e41081e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f47d80099c0 tx=0x7f47d8009cd0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.144+0000 7f47e9579700 1 -- 172.21.15.33:0/3884393391 shutdown_connections 2024-05-15T14:01:37.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.144+0000 7f47e9579700 1 --2- 172.21.15.33:0/3884393391 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f47e4108910 0x7f47e4110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.144+0000 7f47e9579700 1 --2- 172.21.15.33:0/3884393391 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47e4103820 0x7f47e4103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.144+0000 7f47e9579700 1 --2- 172.21.15.33:0/3884393391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e41041d0 0x7f47e41081e0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.144+0000 7f47e9579700 1 -- 172.21.15.33:0/3884393391 >> 172.21.15.33:0/3884393391 conn(0x7f47e407a6b0 msgr2=0x7f47e407aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:37.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.144+0000 7f47e9579700 1 -- 172.21.15.33:0/3884393391 shutdown_connections 2024-05-15T14:01:37.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.144+0000 7f47e9579700 1 -- 172.21.15.33:0/3884393391 wait complete. 2024-05-15T14:01:37.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.145+0000 7f47e9579700 1 Processor -- start 2024-05-15T14:01:37.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.145+0000 7f47e9579700 1 -- start start 2024-05-15T14:01:37.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.145+0000 7f47e9579700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f47e4103820 0x7f47e419bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:37.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.145+0000 7f47e9579700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e41041d0 0x7f47e419c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:37.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.145+0000 7f47e9579700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47e4108910 0x7f47e41a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:37.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.145+0000 7f47e9579700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f47e41136f0 con 0x7f47e41041d0 2024-05-15T14:01:37.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.145+0000 7f47e9579700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f47e4113570 con 0x7f47e4103820 2024-05-15T14:01:37.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.145+0000 7f47e9579700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f47e4113870 con 0x7f47e4108910 2024-05-15T14:01:37.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.145+0000 7f47e27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e41041d0 0x7f47e419c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:37.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.145+0000 7f47e27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e41041d0 0x7f47e419c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42446/0 (socket says 172.21.15.33:42446) 2024-05-15T14:01:37.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.145+0000 7f47e27fc700 1 -- 172.21.15.33:0/3589623759 learned_addr learned my addr 172.21.15.33:0/3589623759 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:37.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.145+0000 7f47e37fe700 1 --2- 172.21.15.33:0/3589623759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47e4108910 0x7f47e41a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:37.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.145+0000 7f47e2ffd700 1 --2- 172.21.15.33:0/3589623759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f47e4103820 0x7f47e419bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:37.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.145+0000 7f47e27fc700 1 -- 172.21.15.33:0/3589623759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47e4108910 msgr2=0x7f47e41a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:37.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.145+0000 7f47e27fc700 1 --2- 172.21.15.33:0/3589623759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47e4108910 0x7f47e41a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.146+0000 7f47e27fc700 1 -- 172.21.15.33:0/3589623759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f47e4103820 msgr2=0x7f47e419bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:37.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.146+0000 7f47e27fc700 1 --2- 172.21.15.33:0/3589623759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f47e4103820 0x7f47e419bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.146+0000 7f47e27fc700 1 -- 172.21.15.33:0/3589623759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e41a0d40 con 0x7f47e41041d0 2024-05-15T14:01:37.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.146+0000 7f47e2ffd700 1 --2- 172.21.15.33:0/3589623759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f47e4103820 0x7f47e419bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:01:37.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.146+0000 7f47e27fc700 1 --2- 172.21.15.33:0/3589623759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e41041d0 0x7f47e419c2c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f47d800b580 tx=0x7f47d80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:37.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.146+0000 7f47cbfff700 1 -- 172.21.15.33:0/3589623759 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d8027070 con 0x7f47e41041d0 2024-05-15T14:01:37.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.146+0000 7f47cbfff700 1 -- 172.21.15.33:0/3589623759 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47d8003f20 con 0x7f47e41041d0 2024-05-15T14:01:37.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.146+0000 7f47e9579700 1 -- 172.21.15.33:0/3589623759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47e41a0fd0 con 0x7f47e41041d0 2024-05-15T14:01:37.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.146+0000 7f47cbfff700 1 -- 172.21.15.33:0/3589623759 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d8022d70 con 0x7f47e41041d0 2024-05-15T14:01:37.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.146+0000 7f47e9579700 1 -- 172.21.15.33:0/3589623759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47e41a88b0 con 0x7f47e41041d0 2024-05-15T14:01:37.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.148+0000 7f47cbfff700 1 -- 172.21.15.33:0/3589623759 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f47d8022500 con 0x7f47e41041d0 2024-05-15T14:01:37.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.149+0000 7f47cbfff700 1 --2- 172.21.15.33:0/3589623759 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f47cc0b2b10 0x7f47cc0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:37.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.149+0000 7f47cbfff700 1 -- 172.21.15.33:0/3589623759 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f47d80e3540 con 0x7f47e41041d0 2024-05-15T14:01:37.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.149+0000 7f47e2ffd700 1 --2- 172.21.15.33:0/3589623759 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f47cc0b2b10 0x7f47cc0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:37.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.150+0000 7f47f3e0fb80 1 Processor -- start 2024-05-15T14:01:37.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.150+0000 7f47f3e0fb80 1 -- start start 2024-05-15T14:01:37.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.150+0000 7f47e2ffd700 1 --2- 172.21.15.33:0/3589623759 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f47cc0b2b10 0x7f47cc0b4fd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f47d40094f0 tx=0x7f47d4009040 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:37.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.150+0000 7f47f3e0fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e8279fda0 0x562e827a0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:37.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.150+0000 7f47f3e0fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562e827a06c0 0x562e8279a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:37.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.150+0000 7f47f3e0fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e8279aad0 0x562e8279af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:37.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.150+0000 7f47f3e0fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562e826f8d90 con 0x562e8279aad0 2024-05-15T14:01:37.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.150+0000 7f47f3e0fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562e826e8930 con 0x562e827a06c0 2024-05-15T14:01:37.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.150+0000 7f47f3e0fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x562e826ec970 con 0x562e8279fda0 2024-05-15T14:01:37.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.150+0000 7f47e37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e8279aad0 0x562e8279af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:37.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.150+0000 7f47e37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e8279aad0 0x562e8279af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42450/0 (socket says 172.21.15.33:42450) 2024-05-15T14:01:37.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.150+0000 7f47e37fe700 1 -- 172.21.15.33:0/3198282959 learned_addr learned my addr 172.21.15.33:0/3198282959 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:37.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.150+0000 7f47e2ffd700 1 --2- 172.21.15.33:0/3198282959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e8279fda0 0x562e827a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:37.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.150+0000 7f47e27fc700 1 --2- 172.21.15.33:0/3198282959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562e827a06c0 0x562e8279a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:37.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.150+0000 7f47e37fe700 1 -- 172.21.15.33:0/3198282959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e8279fda0 msgr2=0x562e827a0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:37.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.150+0000 7f47e37fe700 1 --2- 172.21.15.33:0/3198282959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e8279fda0 0x562e827a0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.150+0000 7f47e37fe700 1 -- 172.21.15.33:0/3198282959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562e827a06c0 msgr2=0x562e8279a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:37.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.150+0000 7f47e37fe700 1 --2- 172.21.15.33:0/3198282959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562e827a06c0 0x562e8279a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.150+0000 7f47e37fe700 1 -- 172.21.15.33:0/3198282959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e826389d0 con 0x562e8279aad0 2024-05-15T14:01:37.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.151+0000 7f47e37fe700 1 --2- 172.21.15.33:0/3198282959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e8279aad0 0x562e8279af50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f47dc00acb0 tx=0x7f47dc00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa5be3a7e4129981 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:37.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.151+0000 7f47c97fa700 1 -- 172.21.15.33:0/3198282959 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc004830 con 0x562e8279aad0 2024-05-15T14:01:37.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.151+0000 7f47c97fa700 1 -- 172.21.15.33:0/3198282959 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47dc005020 con 0x562e8279aad0 2024-05-15T14:01:37.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.151+0000 7f47c97fa700 1 -- 172.21.15.33:0/3198282959 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc0186a0 con 0x562e8279aad0 2024-05-15T14:01:37.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.152+0000 7f47f3e0fb80 1 -- 172.21.15.33:0/3198282959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e8279aad0 msgr2=0x562e8279af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:37.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.152+0000 7f47f3e0fb80 1 --2- 172.21.15.33:0/3198282959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e8279aad0 0x562e8279af50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f47dc00acb0 tx=0x7f47dc00f3c0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.152+0000 7f47f3e0fb80 1 -- 172.21.15.33:0/3198282959 shutdown_connections 2024-05-15T14:01:37.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.152+0000 7f47f3e0fb80 1 --2- 172.21.15.33:0/3198282959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e8279aad0 0x562e8279af50 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.152+0000 7f47f3e0fb80 1 --2- 172.21.15.33:0/3198282959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e8279fda0 0x562e827a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.152+0000 7f47f3e0fb80 1 --2- 172.21.15.33:0/3198282959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562e827a06c0 0x562e8279a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.152+0000 7f47f3e0fb80 1 -- 172.21.15.33:0/3198282959 >> 172.21.15.33:0/3198282959 conn(0x562e826c5350 msgr2=0x562e82794ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:37.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.152+0000 7f47f3e0fb80 1 -- 172.21.15.33:0/3198282959 shutdown_connections 2024-05-15T14:01:37.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.152+0000 7f47f3e0fb80 1 -- 172.21.15.33:0/3198282959 wait complete. 2024-05-15T14:01:37.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.152+0000 7f47f3e0fb80 1 -- start start 2024-05-15T14:01:37.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.152+0000 7f47f3e0fb80 1 Processor -- start 2024-05-15T14:01:37.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.152+0000 7f47f3e0fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e8279aad0 0x562e827a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:37.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.152+0000 7f47f3e0fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562e8279fda0 0x562e827a3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:37.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.152+0000 7f47f3e0fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e827a06c0 0x562e827a3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:37.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.152+0000 7f47f3e0fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562e825ef0a0 con 0x562e827a06c0 2024-05-15T14:01:37.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.152+0000 7f47f3e0fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562e8268a8c0 con 0x562e8279fda0 2024-05-15T14:01:37.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.152+0000 7f47f3e0fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x562e827a7100 con 0x562e8279aad0 2024-05-15T14:01:37.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.153+0000 7f47f3e0fb80 10 client.0 _ll_register_callbacks cb 0x562e8267f680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:01:37.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.153+0000 7f47e37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e827a06c0 0x562e827a3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:37.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.153+0000 7f47e37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e827a06c0 0x562e827a3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42460/0 (socket says 172.21.15.33:42460) 2024-05-15T14:01:37.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.153+0000 7f47e37fe700 1 -- 172.21.15.33:0/2946346309 learned_addr learned my addr 172.21.15.33:0/2946346309 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:37.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.153+0000 7f47e27fc700 1 --2- 172.21.15.33:0/2946346309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562e8279fda0 0x562e827a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:37.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.153+0000 7f47e2ffd700 1 --2- 172.21.15.33:0/2946346309 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e8279aad0 0x562e827a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:37.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.153+0000 7f47e37fe700 1 -- 172.21.15.33:0/2946346309 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e8279aad0 msgr2=0x562e827a63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:37.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.153+0000 7f47e37fe700 1 --2- 172.21.15.33:0/2946346309 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e8279aad0 0x562e827a63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.153+0000 7f47e37fe700 1 -- 172.21.15.33:0/2946346309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562e8279fda0 msgr2=0x562e827a3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:37.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.153+0000 7f47e37fe700 1 --2- 172.21.15.33:0/2946346309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562e8279fda0 0x562e827a3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.153+0000 7f47e37fe700 1 -- 172.21.15.33:0/2946346309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e82687790 con 0x562e827a06c0 2024-05-15T14:01:37.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.153+0000 7f47e27fc700 1 --2- 172.21.15.33:0/2946346309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562e8279fda0 0x562e827a3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:01:37.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.153+0000 7f47e2ffd700 1 --2- 172.21.15.33:0/2946346309 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e8279aad0 0x562e827a63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:01:37.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.153+0000 7f47e37fe700 1 --2- 172.21.15.33:0/2946346309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e827a06c0 0x562e827a3c90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f47dc00f7c0 tx=0x7f47dc0057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e8e20938fc187c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:37.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.153+0000 7f47c6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:01:37.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.153+0000 7f47c6ff5700 1 -- 172.21.15.33:0/2946346309 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc026020 con 0x562e827a06c0 2024-05-15T14:01:37.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.153+0000 7f47f3e0fb80 1 -- 172.21.15.33:0/2946346309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47dc005020 con 0x562e827a06c0 2024-05-15T14:01:37.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.153+0000 7f47c6ff5700 1 -- 172.21.15.33:0/2946346309 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f47dc004750 con 0x562e827a06c0 2024-05-15T14:01:37.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.153+0000 7f47c6ff5700 1 -- 172.21.15.33:0/2946346309 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f47dc031050 con 0x562e827a06c0 2024-05-15T14:01:37.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.154+0000 7f47f3e0fb80 10 client.15471 fetch_fsmap learned FSMap version 24 2024-05-15T14:01:37.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.154+0000 7f47f3e0fb80 1 -- 172.21.15.33:0/2946346309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562e827b17f0 con 0x562e827a06c0 2024-05-15T14:01:37.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.154+0000 7f47c6ff5700 1 -- 172.21.15.33:0/2946346309 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47dc005020 con 0x562e827a06c0 2024-05-15T14:01:37.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.154+0000 7f47c6ff5700 1 -- 172.21.15.33:0/2946346309 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f47dc003c60 con 0x562e827a06c0 2024-05-15T14:01:37.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.154+0000 7f47f3e0fb80 10 client.15471 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:01:37.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.154+0000 7f47f3e0fb80 10 client.15471 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:01:37.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.154+0000 7f47f3e0fb80 20 client.15471 populate_metadata read hostname 'smithi033' 2024-05-15T14:01:37.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.154+0000 7f47f3e0fb80 1 --2- 172.21.15.33:0/2946346309 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x562e827b3d70 0x562e827b6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:37.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.154+0000 7f47f3e0fb80 4 client.15471 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:01:37.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.154+0000 7f47f3e0fb80 1 -- 172.21.15.33:0/2946346309 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e82766460 con 0x562e827b3d70 2024-05-15T14:01:37.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.154+0000 7f47e2ffd700 1 --2- 172.21.15.33:0/2946346309 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x562e827b3d70 0x562e827b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:37.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.155+0000 7f47e2ffd700 1 --2- 172.21.15.33:0/2946346309 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x562e827b3d70 0x562e827b6160 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2def89405a7a6090 server_cookie=db202950cc73d5ac in_seq=0 out_seq=0 2024-05-15T14:01:37.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.155+0000 7f47c6ff5700 10 client.15471 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:01:37.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.156+0000 7f47c6ff5700 1 -- 172.21.15.33:0/2946346309 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562e82766460 con 0x562e827b3d70 2024-05-15T14:01:37.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.156+0000 7f47c6ff5700 10 client.15471 handle_command_reply: tid=0 2024-05-15T14:01:37.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.156+0000 7f47f3e0fb80 1 client.15471 shutdown 2024-05-15T14:01:37.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.156+0000 7f47f3e0fb80 1 -- 172.21.15.33:0/2946346309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e827a06c0 msgr2=0x562e827a3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:37.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.156+0000 7f47f3e0fb80 1 --2- 172.21.15.33:0/2946346309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e827a06c0 0x562e827a3c90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f47dc00f7c0 tx=0x7f47dc0057e0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.156+0000 7f47f3e0fb80 1 -- 172.21.15.33:0/2946346309 shutdown_connections 2024-05-15T14:01:37.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.156+0000 7f47f3e0fb80 1 --2- 172.21.15.33:0/2946346309 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x562e827b3d70 0x562e827b6160 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.156+0000 7f47f3e0fb80 1 --2- 172.21.15.33:0/2946346309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e827a06c0 0x562e827a3c90 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.156+0000 7f47f3e0fb80 1 --2- 172.21.15.33:0/2946346309 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e8279aad0 0x562e827a63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.156+0000 7f47f3e0fb80 1 --2- 172.21.15.33:0/2946346309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562e8279fda0 0x562e827a3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.156+0000 7f47f3e0fb80 1 -- 172.21.15.33:0/2946346309 >> 172.21.15.33:0/2946346309 conn(0x562e826c5350 msgr2=0x562e8278f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:37.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.156+0000 7f47c6ff5700 0 client.15471 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:01:37.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.156+0000 7f47f3e0fb80 1 -- 172.21.15.33:0/2946346309 shutdown_connections 2024-05-15T14:01:37.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.156+0000 7f47f3e0fb80 1 -- 172.21.15.33:0/2946346309 wait complete. 2024-05-15T14:01:37.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.156+0000 7f47f3e0fb80 20 client.15471 trim_cache size 0 max 16384 2024-05-15T14:01:37.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.176+0000 7f47f3e0fb80 1 Processor -- start 2024-05-15T14:01:37.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.176+0000 7f47f3e0fb80 1 -- start start 2024-05-15T14:01:37.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.176+0000 7f47f3e0fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e8279aad0 0x562e8279a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:37.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.176+0000 7f47f3e0fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562e8279fda0 0x562e827b0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:37.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.176+0000 7f47f3e0fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e827a06c0 0x562e8278f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:37.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.176+0000 7f47f3e0fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f47e41b26c0 con 0x562e827a06c0 2024-05-15T14:01:37.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.176+0000 7f47f3e0fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f47e41aff50 con 0x562e8279fda0 2024-05-15T14:01:37.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.176+0000 7f47f3e0fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f47e41adda0 con 0x562e8279aad0 2024-05-15T14:01:37.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.176+0000 7f47e37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e827a06c0 0x562e8278f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:37.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.176+0000 7f47e37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e827a06c0 0x562e8278f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42474/0 (socket says 172.21.15.33:42474) 2024-05-15T14:01:37.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.176+0000 7f47e37fe700 1 -- 172.21.15.33:0/2287605569 learned_addr learned my addr 172.21.15.33:0/2287605569 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:37.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.177+0000 7f47e2ffd700 1 --2- 172.21.15.33:0/2287605569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e8279aad0 0x562e8279a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:37.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.177+0000 7f47e37fe700 1 -- 172.21.15.33:0/2287605569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e8279aad0 msgr2=0x562e8279a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:37.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.177+0000 7f47e37fe700 1 --2- 172.21.15.33:0/2287605569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e8279aad0 0x562e8279a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.177+0000 7f47e37fe700 1 -- 172.21.15.33:0/2287605569 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562e8279fda0 msgr2=0x562e827b0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:37.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.177+0000 7f47e37fe700 1 --2- 172.21.15.33:0/2287605569 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562e8279fda0 0x562e827b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.177+0000 7f47e37fe700 1 -- 172.21.15.33:0/2287605569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e827a3520 con 0x562e827a06c0 2024-05-15T14:01:37.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.177+0000 7f47e2ffd700 1 --2- 172.21.15.33:0/2287605569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e8279aad0 0x562e8279a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:01:37.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.177+0000 7f47e37fe700 1 --2- 172.21.15.33:0/2287605569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e827a06c0 0x562e8278f8e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f47dc000c00 tx=0x7f47dc0056c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a09030ea331900 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:37.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.177+0000 7f47c8ff9700 1 -- 172.21.15.33:0/2287605569 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc03a070 con 0x562e827a06c0 2024-05-15T14:01:37.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.177+0000 7f47c8ff9700 1 -- 172.21.15.33:0/2287605569 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47dc00a2d0 con 0x562e827a06c0 2024-05-15T14:01:37.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.177+0000 7f47c8ff9700 1 -- 172.21.15.33:0/2287605569 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc026070 con 0x562e827a06c0 2024-05-15T14:01:37.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.178+0000 7f47f3e0fb80 1 -- 172.21.15.33:0/2287605569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e827a06c0 msgr2=0x562e8278f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:37.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.178+0000 7f47f3e0fb80 1 --2- 172.21.15.33:0/2287605569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e827a06c0 0x562e8278f8e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f47dc000c00 tx=0x7f47dc0056c0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.178+0000 7f47f3e0fb80 1 -- 172.21.15.33:0/2287605569 shutdown_connections 2024-05-15T14:01:37.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.178+0000 7f47f3e0fb80 1 --2- 172.21.15.33:0/2287605569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e827a06c0 0x562e8278f8e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.178+0000 7f47f3e0fb80 1 --2- 172.21.15.33:0/2287605569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e8279aad0 0x562e8279a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.178+0000 7f47f3e0fb80 1 --2- 172.21.15.33:0/2287605569 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562e8279fda0 0x562e827b0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.178+0000 7f47f3e0fb80 1 -- 172.21.15.33:0/2287605569 >> 172.21.15.33:0/2287605569 conn(0x562e826c5350 msgr2=0x562e8279e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:37.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.178+0000 7f47f3e0fb80 1 -- 172.21.15.33:0/2287605569 shutdown_connections 2024-05-15T14:01:37.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.178+0000 7f47f3e0fb80 1 -- 172.21.15.33:0/2287605569 wait complete. 2024-05-15T14:01:37.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.178+0000 7f47f3e0fb80 1 -- start start 2024-05-15T14:01:37.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.178+0000 7f47f3e0fb80 1 Processor -- start 2024-05-15T14:01:37.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.179+0000 7f47f3e0fb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e8279aad0 0x562e827ac730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:37.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.179+0000 7f47f3e0fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e8279fda0 0x562e827a9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:37.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.179+0000 7f47f3e0fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562e827a06c0 0x562e827a98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:37.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.179+0000 7f47f3e0fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f47e41ae410 con 0x562e8279fda0 2024-05-15T14:01:37.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.179+0000 7f47f3e0fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f47e41af880 con 0x562e827a06c0 2024-05-15T14:01:37.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.179+0000 7f47f3e0fb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f47e41aeae0 con 0x562e8279aad0 2024-05-15T14:01:37.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.179+0000 7f47f3e0fb80 10 client.0 _ll_register_callbacks cb 0x562e8267f680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:01:37.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.179+0000 7f47e27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e8279fda0 0x562e827a9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:37.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.179+0000 7f47e27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e8279fda0 0x562e827a9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42486/0 (socket says 172.21.15.33:42486) 2024-05-15T14:01:37.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.179+0000 7f47e27fc700 1 -- 172.21.15.33:0/3058841026 learned_addr learned my addr 172.21.15.33:0/3058841026 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:37.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.179+0000 7f47e2ffd700 1 --2- 172.21.15.33:0/3058841026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e8279aad0 0x562e827ac730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:37.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.179+0000 7f47e37fe700 1 --2- 172.21.15.33:0/3058841026 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562e827a06c0 0x562e827a98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:37.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.179+0000 7f47e27fc700 1 -- 172.21.15.33:0/3058841026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e8279aad0 msgr2=0x562e827ac730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:37.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.179+0000 7f47e27fc700 1 --2- 172.21.15.33:0/3058841026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e8279aad0 0x562e827ac730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.179+0000 7f47e27fc700 1 -- 172.21.15.33:0/3058841026 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562e827a06c0 msgr2=0x562e827a98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:37.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.179+0000 7f47e27fc700 1 --2- 172.21.15.33:0/3058841026 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562e827a06c0 0x562e827a98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.179+0000 7f47e27fc700 1 -- 172.21.15.33:0/3058841026 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e827a9ee0 con 0x562e8279fda0 2024-05-15T14:01:37.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.180+0000 7f47e27fc700 1 --2- 172.21.15.33:0/3058841026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e8279fda0 0x562e827a9360 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f47d80aa370 tx=0x7f47d80aa0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f18c3e1a97e86d server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:37.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.180+0000 7f47c6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:01:37.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.180+0000 7f47c6ff5700 1 -- 172.21.15.33:0/3058841026 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d80a9de0 con 0x562e8279fda0 2024-05-15T14:01:37.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.180+0000 7f47c6ff5700 1 -- 172.21.15.33:0/3058841026 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f47d80e3110 con 0x562e8279fda0 2024-05-15T14:01:37.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.180+0000 7f47f3e0fb80 1 -- 172.21.15.33:0/3058841026 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562e826face0 con 0x562e8279fda0 2024-05-15T14:01:37.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.180+0000 7f47c6ff5700 1 -- 172.21.15.33:0/3058841026 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f47d80aa180 con 0x562e8279fda0 2024-05-15T14:01:37.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.180+0000 7f47f3e0fb80 10 client.15477 fetch_fsmap learned FSMap version 24 2024-05-15T14:01:37.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.180+0000 7f47f3e0fb80 1 -- 172.21.15.33:0/3058841026 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562e82827a00 con 0x562e8279fda0 2024-05-15T14:01:37.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.180+0000 7f47c6ff5700 1 -- 172.21.15.33:0/3058841026 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562e826face0 con 0x562e8279fda0 2024-05-15T14:01:37.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.180+0000 7f47c6ff5700 1 -- 172.21.15.33:0/3058841026 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f47d80afbb0 con 0x562e8279fda0 2024-05-15T14:01:37.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.180+0000 7f47f3e0fb80 10 client.15477 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:01:37.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.180+0000 7f47f3e0fb80 10 client.15477 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:01:37.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.180+0000 7f47f3e0fb80 20 client.15477 populate_metadata read hostname 'smithi033' 2024-05-15T14:01:37.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.180+0000 7f47f3e0fb80 1 --2- 172.21.15.33:0/3058841026 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x562e82828110 0x562e8282a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:37.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.180+0000 7f47f3e0fb80 4 client.15477 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2fa87dae", "release": true, "await": true}] 2024-05-15T14:01:37.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.181+0000 7f47f3e0fb80 1 -- 172.21.15.33:0/3058841026 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2fa87dae", "release": true, "await": true}) -- 0x562e826f44c0 con 0x562e82828110 2024-05-15T14:01:37.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.181+0000 7f47e2ffd700 1 --2- 172.21.15.33:0/3058841026 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x562e82828110 0x562e8282a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:37.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.181+0000 7f47e2ffd700 1 --2- 172.21.15.33:0/3058841026 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x562e82828110 0x562e8282a530 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=499cddee0cea42e7 server_cookie=8c1560c4eeb5232e in_seq=0 out_seq=0 2024-05-15T14:01:37.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.181+0000 7f47c6ff5700 10 client.15477 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:01:37.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.183+0000 7f47c6ff5700 1 -- 172.21.15.33:0/3058841026 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x562e826f44c0 con 0x562e82828110 2024-05-15T14:01:37.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.183+0000 7f47c6ff5700 10 client.15477 handle_command_reply: tid=0 2024-05-15T14:01:37.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.183+0000 7f47f3e0fb80 1 client.15477 shutdown 2024-05-15T14:01:37.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.183+0000 7f47f3e0fb80 1 -- 172.21.15.33:0/3058841026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e8279fda0 msgr2=0x562e827a9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:37.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.183+0000 7f47f3e0fb80 1 --2- 172.21.15.33:0/3058841026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e8279fda0 0x562e827a9360 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f47d80aa370 tx=0x7f47d80aa0f0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.183+0000 7f47f3e0fb80 1 -- 172.21.15.33:0/3058841026 shutdown_connections 2024-05-15T14:01:37.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.183+0000 7f47f3e0fb80 1 --2- 172.21.15.33:0/3058841026 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562e827a06c0 0x562e827a98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.183+0000 7f47f3e0fb80 1 --2- 172.21.15.33:0/3058841026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e8279aad0 0x562e827ac730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.183+0000 7f47f3e0fb80 1 --2- 172.21.15.33:0/3058841026 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x562e82828110 0x562e8282a530 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.183+0000 7f47f3e0fb80 1 --2- 172.21.15.33:0/3058841026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562e8279fda0 0x562e827a9360 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.183+0000 7f47c6ff5700 0 client.15477 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:01:37.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.183+0000 7f47f3e0fb80 1 -- 172.21.15.33:0/3058841026 >> 172.21.15.33:0/3058841026 conn(0x562e826c5350 msgr2=0x562e8279f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:37.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.183+0000 7f47f3e0fb80 1 -- 172.21.15.33:0/3058841026 shutdown_connections 2024-05-15T14:01:37.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.183+0000 7f47f3e0fb80 1 -- 172.21.15.33:0/3058841026 wait complete. 2024-05-15T14:01:37.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.183+0000 7f47f3e0fb80 20 client.15477 trim_cache size 0 max 16384 2024-05-15T14:01:37.189 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T14:01:37.189 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 24, 2024-05-15T14:01:37.189 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24478, 2024-05-15T14:01:37.189 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 4, 2024-05-15T14:01:37.190 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-05-15T14:01:37.190 INFO:teuthology.orchestra.run.smithi033.stdout: "2fa87dae": { 2024-05-15T14:01:37.190 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 4, 2024-05-15T14:01:37.190 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-05-15T14:01:37.190 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:01:37.190 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-05-15T14:01:37.190 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:01:37.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:01:37.190 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 90.0, 2024-05-15T14:01:37.190 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 144.0, 2024-05-15T14:01:37.190 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-05-15T14:01:37.190 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-05-15T14:01:37.190 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-05-15T14:01:37.190 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:01:37.191 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-05-15T14:01:37.191 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:01:37.191 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:01:37.191 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:01:37.191 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:01:37.191 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:01:37.191 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:01:37.191 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T14:01:37.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.184+0000 7f47e9579700 1 -- 172.21.15.33:0/3589623759 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f47cc0b2b10 msgr2=0x7f47cc0b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:37.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.184+0000 7f47e9579700 1 --2- 172.21.15.33:0/3589623759 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f47cc0b2b10 0x7f47cc0b4fd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f47d40094f0 tx=0x7f47d4009040 comp rx=0 tx=0).stop 2024-05-15T14:01:37.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.184+0000 7f47e9579700 1 -- 172.21.15.33:0/3589623759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e41041d0 msgr2=0x7f47e419c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:37.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.184+0000 7f47e9579700 1 --2- 172.21.15.33:0/3589623759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e41041d0 0x7f47e419c2c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f47d800b580 tx=0x7f47d80047b0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.184+0000 7f47e9579700 1 -- 172.21.15.33:0/3589623759 shutdown_connections 2024-05-15T14:01:37.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.184+0000 7f47e9579700 1 --2- 172.21.15.33:0/3589623759 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f47cc0b2b10 0x7f47cc0b4fd0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.184+0000 7f47e9579700 1 --2- 172.21.15.33:0/3589623759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47e4108910 0x7f47e41a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.184+0000 7f47e9579700 1 --2- 172.21.15.33:0/3589623759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f47e4103820 0x7f47e419bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.184+0000 7f47e9579700 1 --2- 172.21.15.33:0/3589623759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e41041d0 0x7f47e419c2c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:37.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.184+0000 7f47e9579700 1 -- 172.21.15.33:0/3589623759 >> 172.21.15.33:0/3589623759 conn(0x7f47e407a6b0 msgr2=0x7f47e4109a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:37.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.184+0000 7f47e9579700 1 -- 172.21.15.33:0/3589623759 shutdown_connections 2024-05-15T14:01:37.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:37.184+0000 7f47e9579700 1 -- 172.21.15.33:0/3589623759 wait complete. 2024-05-15T14:01:37.943 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2fa87dae', total seconds elapsed: 29.6 2024-05-15T14:01:37.943 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.0 2024-05-15T14:01:37.943 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T14:01:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:01:38.589 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:38 smithi033 ceph-mon[31650]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 8 op/s 2024-05-15T14:01:38.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:38 smithi052 ceph-mon[36190]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 8 op/s 2024-05-15T14:01:38.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:38 smithi081 ceph-mon[36434]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 8 op/s 2024-05-15T14:01:40.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.076+0000 7f98f7ae1700 1 -- 172.21.15.33:0/1094126788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98f0108520 msgr2=0x7f98f0108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:40.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.076+0000 7f98f7ae1700 1 --2- 172.21.15.33:0/1094126788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98f0108520 0x7f98f0108900 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f98ec00b0a0 tx=0x7f98ec00b3b0 comp rx=0 tx=0).stop 2024-05-15T14:01:40.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.076+0000 7f98f7ae1700 1 -- 172.21.15.33:0/1094126788 shutdown_connections 2024-05-15T14:01:40.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.076+0000 7f98f7ae1700 1 --2- 172.21.15.33:0/1094126788 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f98f0102e00 0x7f98f010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:40.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.076+0000 7f98f7ae1700 1 --2- 172.21.15.33:0/1094126788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98f0108520 0x7f98f0108900 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:40.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.076+0000 7f98f7ae1700 1 --2- 172.21.15.33:0/1094126788 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98f0102440 0x7f98f01028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:40.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.076+0000 7f98f7ae1700 1 -- 172.21.15.33:0/1094126788 >> 172.21.15.33:0/1094126788 conn(0x7f98f00793f0 msgr2=0x7f98f0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:40.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.076+0000 7f98f7ae1700 1 -- 172.21.15.33:0/1094126788 shutdown_connections 2024-05-15T14:01:40.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.076+0000 7f98f7ae1700 1 -- 172.21.15.33:0/1094126788 wait complete. 2024-05-15T14:01:40.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.076+0000 7f98f7ae1700 1 Processor -- start 2024-05-15T14:01:40.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.076+0000 7f98f7ae1700 1 -- start start 2024-05-15T14:01:40.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.077+0000 7f98f7ae1700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98f0102440 0x7f98f0104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:40.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.077+0000 7f98f7ae1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98f0102e00 0x7f98f01045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:40.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.077+0000 7f98f7ae1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f98f0108520 0x7f98f0104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:40.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.077+0000 7f98f7ae1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f98f01125d0 con 0x7f98f0102e00 2024-05-15T14:01:40.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.077+0000 7f98f7ae1700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f98f0112450 con 0x7f98f0108520 2024-05-15T14:01:40.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.077+0000 7f98f7ae1700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f98f0112750 con 0x7f98f0102440 2024-05-15T14:01:40.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.077+0000 7f98f507c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98f0102e00 0x7f98f01045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:40.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.077+0000 7f98f507c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98f0102e00 0x7f98f01045c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42496/0 (socket says 172.21.15.33:42496) 2024-05-15T14:01:40.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.077+0000 7f98f507c700 1 -- 172.21.15.33:0/1268051101 learned_addr learned my addr 172.21.15.33:0/1268051101 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:40.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.077+0000 7f98f587d700 1 --2- 172.21.15.33:0/1268051101 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98f0102440 0x7f98f0104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:40.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.077+0000 7f98f607e700 1 --2- 172.21.15.33:0/1268051101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f98f0108520 0x7f98f0104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:40.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.077+0000 7f98f507c700 1 -- 172.21.15.33:0/1268051101 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98f0102440 msgr2=0x7f98f0104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:40.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.077+0000 7f98f507c700 1 --2- 172.21.15.33:0/1268051101 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98f0102440 0x7f98f0104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:40.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.077+0000 7f98f507c700 1 -- 172.21.15.33:0/1268051101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f98f0108520 msgr2=0x7f98f0104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:40.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.077+0000 7f98f507c700 1 --2- 172.21.15.33:0/1268051101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f98f0108520 0x7f98f0104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:40.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.077+0000 7f98f507c700 1 -- 172.21.15.33:0/1268051101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98f01a80e0 con 0x7f98f0102e00 2024-05-15T14:01:40.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.077+0000 7f98f507c700 1 --2- 172.21.15.33:0/1268051101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98f0102e00 0x7f98f01045c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f98e000d950 tx=0x7f98e000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:40.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.078+0000 7f98e6ffd700 1 -- 172.21.15.33:0/1268051101 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e0016070 con 0x7f98f0102e00 2024-05-15T14:01:40.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.078+0000 7f98e6ffd700 1 -- 172.21.15.33:0/1268051101 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98e000fd40 con 0x7f98f0102e00 2024-05-15T14:01:40.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.078+0000 7f98f7ae1700 1 -- 172.21.15.33:0/1268051101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98f01a83d0 con 0x7f98f0102e00 2024-05-15T14:01:40.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.078+0000 7f98e6ffd700 1 -- 172.21.15.33:0/1268051101 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e000e3b0 con 0x7f98f0102e00 2024-05-15T14:01:40.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.078+0000 7f98f7ae1700 1 -- 172.21.15.33:0/1268051101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98f01a8960 con 0x7f98f0102e00 2024-05-15T14:01:40.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.079+0000 7f98f7ae1700 1 -- 172.21.15.33:0/1268051101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98f006a230 con 0x7f98f0102e00 2024-05-15T14:01:40.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.081+0000 7f98e6ffd700 1 -- 172.21.15.33:0/1268051101 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f98e0019070 con 0x7f98f0102e00 2024-05-15T14:01:40.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.081+0000 7f98e6ffd700 1 --2- 172.21.15.33:0/1268051101 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f98dc0b2b90 0x7f98dc0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:40.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.081+0000 7f98f587d700 1 --2- 172.21.15.33:0/1268051101 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f98dc0b2b90 0x7f98dc0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:40.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.081+0000 7f98e6ffd700 1 -- 172.21.15.33:0/1268051101 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f98e00e2290 con 0x7f98f0102e00 2024-05-15T14:01:40.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.082+0000 7f98f587d700 1 --2- 172.21.15.33:0/1268051101 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f98dc0b2b90 0x7f98dc0b5050 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f98ec009e60 tx=0x7f98ec00b7b0 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:40.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.089+0000 7f98e6ffd700 1 -- 172.21.15.33:0/1268051101 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f98e00a9a30 con 0x7f98f0102e00 2024-05-15T14:01:40.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.374+0000 7f98f7ae1700 1 -- 172.21.15.33:0/1268051101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98f01055b0 con 0x7f98f0102e00 2024-05-15T14:01:40.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.375+0000 7f98e6ffd700 1 -- 172.21.15.33:0/1268051101 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11858 (secure 0 0 0) 0x7f98f01055b0 con 0x7f98f0102e00 2024-05-15T14:01:40.378 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T14:01:40.378 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2030117633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2030117633},{"type":"v1","addr":"172.21.15.33:6839","nonce":2030117633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1918979061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1918979061},{"type":"v1","addr":"172.21.15.33:6841","nonce":1918979061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T13:57:19.056417+0000","modified":"2024-05-15T14:00:18.543619+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24463,"mds_2":24461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/1629697841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1629697841},{"type":"v1","addr":"172.21.15.52:6837","nonce":1629697841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/3773839758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3773839758},{"type":"v1","addr":"172.21.15.81:6835","nonce":3773839758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.52:6839/2696044471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2696044471},{"type":"v1","addr":"172.21.15.52:6839","nonce":2696044471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.81:6839/1554395940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":1554395940},{"type":"v1","addr":"172.21.15.81:6839","nonce":1554395940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24463,24478,24461]},"id":1}]} 2024-05-15T14:01:40.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.378+0000 7f98f7ae1700 1 -- 172.21.15.33:0/1268051101 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f98dc0b2b90 msgr2=0x7f98dc0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:40.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.378+0000 7f98f7ae1700 1 --2- 172.21.15.33:0/1268051101 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f98dc0b2b90 0x7f98dc0b5050 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f98ec009e60 tx=0x7f98ec00b7b0 comp rx=0 tx=0).stop 2024-05-15T14:01:40.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.378+0000 7f98f7ae1700 1 -- 172.21.15.33:0/1268051101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98f0102e00 msgr2=0x7f98f01045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:40.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.378+0000 7f98f7ae1700 1 --2- 172.21.15.33:0/1268051101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98f0102e00 0x7f98f01045c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f98e000d950 tx=0x7f98e000dc60 comp rx=0 tx=0).stop 2024-05-15T14:01:40.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.378+0000 7f98f7ae1700 1 -- 172.21.15.33:0/1268051101 shutdown_connections 2024-05-15T14:01:40.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.378+0000 7f98f7ae1700 1 --2- 172.21.15.33:0/1268051101 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f98dc0b2b90 0x7f98dc0b5050 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:40.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.378+0000 7f98f7ae1700 1 --2- 172.21.15.33:0/1268051101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f98f0108520 0x7f98f0104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:40.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.378+0000 7f98f7ae1700 1 --2- 172.21.15.33:0/1268051101 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98f0102440 0x7f98f0104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:40.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.378+0000 7f98f7ae1700 1 --2- 172.21.15.33:0/1268051101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98f0102e00 0x7f98f01045c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:40.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.378+0000 7f98f7ae1700 1 -- 172.21.15.33:0/1268051101 >> 172.21.15.33:0/1268051101 conn(0x7f98f00793f0 msgr2=0x7f98f006b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:40.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.378+0000 7f98f7ae1700 1 -- 172.21.15.33:0/1268051101 shutdown_connections 2024-05-15T14:01:40.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:40.378+0000 7f98f7ae1700 1 -- 172.21.15.33:0/1268051101 wait complete. 2024-05-15T14:01:40.382 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 24 2024-05-15T14:01:40.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:40 smithi033 ceph-mon[31650]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-15T14:01:40.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:40 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/1268051101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:01:40.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:40 smithi052 ceph-mon[36190]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-15T14:01:40.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:40 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/1268051101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:01:40.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:40 smithi081 ceph-mon[36434]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-15T14:01:40.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:40 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/1268051101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:01:41.069 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24478 quiesce db / --timeout 90 --expiration 144.0 --await-for 45.0' 2024-05-15T14:01:41.069 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24478 quiesce db / --timeout 90 --expiration 144.0 --await-for 45.0 2024-05-15T14:01:41.335 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:01:42.754 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:42 smithi033 ceph-mon[31650]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-15T14:01:42.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:42 smithi052 ceph-mon[36190]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-15T14:01:42.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:42 smithi081 ceph-mon[36434]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-15T14:01:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.016+0000 7fd8c4aef700 1 -- 172.21.15.33:0/53570486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c0103f90 msgr2=0x7fd8c0110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.016+0000 7fd8c4aef700 1 --2- 172.21.15.33:0/53570486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c0103f90 0x7fd8c0110470 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd8b800b0a0 tx=0x7fd8b800b3b0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.016+0000 7fd8c4aef700 1 -- 172.21.15.33:0/53570486 shutdown_connections 2024-05-15T14:01:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.016+0000 7fd8c4aef700 1 --2- 172.21.15.33:0/53570486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c0103f90 0x7fd8c0110470 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.016+0000 7fd8c4aef700 1 --2- 172.21.15.33:0/53570486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8c0108500 0x7fd8c01088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.016+0000 7fd8c4aef700 1 --2- 172.21.15.33:0/53570486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8c01035f0 0x7fd8c0103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.016+0000 7fd8c4aef700 1 -- 172.21.15.33:0/53570486 >> 172.21.15.33:0/53570486 conn(0x7fd8c007a6a0 msgr2=0x7fd8c007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.016+0000 7fd8c4aef700 1 -- 172.21.15.33:0/53570486 shutdown_connections 2024-05-15T14:01:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.016+0000 7fd8c4aef700 1 -- 172.21.15.33:0/53570486 wait complete. 2024-05-15T14:01:43.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.017+0000 7fd8c4aef700 1 Processor -- start 2024-05-15T14:01:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.017+0000 7fd8c4aef700 1 -- start start 2024-05-15T14:01:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.018+0000 7fd8c4aef700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8c01035f0 0x7fd8c019f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.018+0000 7fd8c4aef700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c0103f90 0x7fd8c019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.018+0000 7fd8c4aef700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8c0108500 0x7fd8c0199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.018+0000 7fd8be59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8c01035f0 0x7fd8c019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.018+0000 7fd8be59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8c01035f0 0x7fd8c019f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:47594/0 (socket says 172.21.15.33:47594) 2024-05-15T14:01:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.018+0000 7fd8be59c700 1 -- 172.21.15.33:0/3914785109 learned_addr learned my addr 172.21.15.33:0/3914785109 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.018+0000 7fd8bdd9b700 1 --2- 172.21.15.33:0/3914785109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c0103f90 0x7fd8c019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.018+0000 7fd8bed9d700 1 --2- 172.21.15.33:0/3914785109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8c0108500 0x7fd8c0199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.018+0000 7fd8c4aef700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c01137e0 con 0x7fd8c0103f90 2024-05-15T14:01:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.018+0000 7fd8c4aef700 1 -- 172.21.15.33:0/3914785109 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c0113660 con 0x7fd8c0108500 2024-05-15T14:01:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.018+0000 7fd8c4aef700 1 -- 172.21.15.33:0/3914785109 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c0113960 con 0x7fd8c01035f0 2024-05-15T14:01:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.018+0000 7fd8bdd9b700 1 -- 172.21.15.33:0/3914785109 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8c01035f0 msgr2=0x7fd8c019f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.018+0000 7fd8bdd9b700 1 --2- 172.21.15.33:0/3914785109 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8c01035f0 0x7fd8c019f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.019+0000 7fd8bdd9b700 1 -- 172.21.15.33:0/3914785109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8c0108500 msgr2=0x7fd8c0199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:43.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.019+0000 7fd8bdd9b700 1 --2- 172.21.15.33:0/3914785109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8c0108500 0x7fd8c0199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.019+0000 7fd8bdd9b700 1 -- 172.21.15.33:0/3914785109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c019a080 con 0x7fd8c0103f90 2024-05-15T14:01:43.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.019+0000 7fd8bed9d700 1 --2- 172.21.15.33:0/3914785109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8c0108500 0x7fd8c0199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:01:43.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.019+0000 7fd8bdd9b700 1 --2- 172.21.15.33:0/3914785109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c0103f90 0x7fd8c019fca0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd8b400d750 tx=0x7fd8b400da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:43.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.019+0000 7fd8af7fe700 1 -- 172.21.15.33:0/3914785109 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4014030 con 0x7fd8c0103f90 2024-05-15T14:01:43.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.019+0000 7fd8c4aef700 1 -- 172.21.15.33:0/3914785109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c019a370 con 0x7fd8c0103f90 2024-05-15T14:01:43.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.019+0000 7fd8af7fe700 1 -- 172.21.15.33:0/3914785109 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8b40044d0 con 0x7fd8c0103f90 2024-05-15T14:01:43.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.019+0000 7fd8af7fe700 1 -- 172.21.15.33:0/3914785109 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4005420 con 0x7fd8c0103f90 2024-05-15T14:01:43.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.019+0000 7fd8c4aef700 1 -- 172.21.15.33:0/3914785109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8c01a6630 con 0x7fd8c0103f90 2024-05-15T14:01:43.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.023+0000 7fd8af7fe700 1 -- 172.21.15.33:0/3914785109 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7fd8b4004020 con 0x7fd8c0103f90 2024-05-15T14:01:43.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.024+0000 7fd8cf385b80 1 Processor -- start 2024-05-15T14:01:43.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.024+0000 7fd8cf385b80 1 -- start start 2024-05-15T14:01:43.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.024+0000 7fd8af7fe700 1 --2- 172.21.15.33:0/3914785109 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fd8a80b2bc0 0x7fd8a80b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.024+0000 7fd8cf385b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd36b9b4b0 0x55cd36b9b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.024+0000 7fd8cf385b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd36b9bdd0 0x55cd36b96d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.024+0000 7fd8cf385b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd36b97350 0x55cd36b977b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.024+0000 7fd8cf385b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cd36af4f00 con 0x55cd36b9b4b0 2024-05-15T14:01:43.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.024+0000 7fd8cf385b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cd36ae4aa0 con 0x55cd36b9bdd0 2024-05-15T14:01:43.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.024+0000 7fd8cf385b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55cd36ae8ae0 con 0x55cd36b97350 2024-05-15T14:01:43.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.025+0000 7fd8bed9d700 1 --2- 172.21.15.33:0/3914785109 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fd8a80b2bc0 0x7fd8a80b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:43.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.025+0000 7fd8bed9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd36b97350 0x55cd36b977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:43.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.025+0000 7fd8bed9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd36b97350 0x55cd36b977b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:47600/0 (socket says 172.21.15.33:47600) 2024-05-15T14:01:43.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.025+0000 7fd8bed9d700 1 -- 172.21.15.33:0/3286051934 learned_addr learned my addr 172.21.15.33:0/3286051934 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:43.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.025+0000 7fd8be59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd36b9b4b0 0x55cd36b9b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:43.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.025+0000 7fd8af7fe700 1 -- 172.21.15.33:0/3914785109 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fd8b40e2410 con 0x7fd8c0103f90 2024-05-15T14:01:43.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.025+0000 7fd8bed9d700 1 -- 172.21.15.33:0/3286051934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd36b9bdd0 msgr2=0x55cd36b96d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:43.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.025+0000 7fd8bed9d700 1 --2- 172.21.15.33:0/3286051934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd36b9bdd0 0x55cd36b96d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.025+0000 7fd8bed9d700 1 -- 172.21.15.33:0/3286051934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd36b9b4b0 msgr2=0x55cd36b9b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:43.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.025+0000 7fd8bed9d700 1 --2- 172.21.15.33:0/3286051934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd36b9b4b0 0x55cd36b9b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.025+0000 7fd8bed9d700 1 -- 172.21.15.33:0/3286051934 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd36a8e140 con 0x55cd36b97350 2024-05-15T14:01:43.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.026+0000 7fd8bed9d700 1 --2- 172.21.15.33:0/3914785109 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fd8a80b2bc0 0x7fd8a80b5080 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009e60 tx=0x7fd8b8009df0 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:43.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.026+0000 7fd8bed9d700 1 --2- 172.21.15.33:0/3286051934 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd36b97350 0x55cd36b977b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8015000 tx=0x7fd8b8026000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58315437bc5c6140 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:43.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.026+0000 7fd8acff9700 1 -- 172.21.15.33:0/3286051934 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8007a30 con 0x55cd36b97350 2024-05-15T14:01:43.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.026+0000 7fd8acff9700 1 -- 172.21.15.33:0/3286051934 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8b8007bd0 con 0x55cd36b97350 2024-05-15T14:01:43.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.026+0000 7fd8acff9700 1 -- 172.21.15.33:0/3286051934 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b801b430 con 0x55cd36b97350 2024-05-15T14:01:43.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.027+0000 7fd8cf385b80 1 -- 172.21.15.33:0/3286051934 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd36b97350 msgr2=0x55cd36b977b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:43.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.027+0000 7fd8cf385b80 1 --2- 172.21.15.33:0/3286051934 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd36b97350 0x55cd36b977b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8015000 tx=0x7fd8b8026000 comp rx=0 tx=0).stop 2024-05-15T14:01:43.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.027+0000 7fd8cf385b80 1 -- 172.21.15.33:0/3286051934 shutdown_connections 2024-05-15T14:01:43.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.027+0000 7fd8cf385b80 1 --2- 172.21.15.33:0/3286051934 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd36b97350 0x55cd36b977b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.027+0000 7fd8cf385b80 1 --2- 172.21.15.33:0/3286051934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd36b9b4b0 0x55cd36b9b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.027+0000 7fd8cf385b80 1 --2- 172.21.15.33:0/3286051934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd36b9bdd0 0x55cd36b96d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.027+0000 7fd8cf385b80 1 -- 172.21.15.33:0/3286051934 >> 172.21.15.33:0/3286051934 conn(0x55cd36ac3f40 msgr2=0x55cd36b91530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:43.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.027+0000 7fd8cf385b80 1 -- 172.21.15.33:0/3286051934 shutdown_connections 2024-05-15T14:01:43.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.027+0000 7fd8cf385b80 1 -- 172.21.15.33:0/3286051934 wait complete. 2024-05-15T14:01:43.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.028+0000 7fd8cf385b80 1 -- start start 2024-05-15T14:01:43.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.028+0000 7fd8cf385b80 1 Processor -- start 2024-05-15T14:01:43.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.028+0000 7fd8cf385b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd36b97350 0x55cd36ba2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.028+0000 7fd8cf385b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd36b9b4b0 0x55cd36b9ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.028+0000 7fd8cf385b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd36b9bdd0 0x55cd36ba0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.028+0000 7fd8cf385b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cd369eb130 con 0x55cd36b97350 2024-05-15T14:01:43.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.028+0000 7fd8cf385b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cd36a4cf30 con 0x55cd36b9bdd0 2024-05-15T14:01:43.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.028+0000 7fd8cf385b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55cd36ba7cc0 con 0x55cd36b9b4b0 2024-05-15T14:01:43.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.028+0000 7fd8cf385b80 10 client.0 _ll_register_callbacks cb 0x55cd36a7fae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:01:43.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.028+0000 7fd8be59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd36b97350 0x55cd36ba2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:43.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.028+0000 7fd8be59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd36b97350 0x55cd36ba2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48844/0 (socket says 172.21.15.33:48844) 2024-05-15T14:01:43.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.028+0000 7fd8be59c700 1 -- 172.21.15.33:0/583673413 learned_addr learned my addr 172.21.15.33:0/583673413 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:43.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.028+0000 7fd8bdd9b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd36b9b4b0 0x55cd36b9ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:43.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.028+0000 7fd8bed9d700 1 --2- 172.21.15.33:0/583673413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd36b9bdd0 0x55cd36ba0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:43.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.029+0000 7fd8be59c700 1 -- 172.21.15.33:0/583673413 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd36b9b4b0 msgr2=0x55cd36b9ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:43.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.029+0000 7fd8be59c700 1 --2- 172.21.15.33:0/583673413 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd36b9b4b0 0x55cd36b9ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.029+0000 7fd8be59c700 1 -- 172.21.15.33:0/583673413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd36b9bdd0 msgr2=0x55cd36ba0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:43.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.029+0000 7fd8be59c700 1 --2- 172.21.15.33:0/583673413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd36b9bdd0 0x55cd36ba0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.029+0000 7fd8be59c700 1 -- 172.21.15.33:0/583673413 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd36a34a60 con 0x55cd36b97350 2024-05-15T14:01:43.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.029+0000 7fd8bdd9b700 1 --2- 172.21.15.33:0/583673413 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd36b9b4b0 0x55cd36b9ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:01:43.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.029+0000 7fd8be59c700 1 --2- 172.21.15.33:0/583673413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd36b97350 0x55cd36ba2b90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00126d0 tx=0x7fd8b00129e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=500aa32f14959149 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:43.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.029+0000 7fd8a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:01:43.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.029+0000 7fd8a5ffb700 1 -- 172.21.15.33:0/583673413 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b0010850 con 0x55cd36b97350 2024-05-15T14:01:43.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.029+0000 7fd8cf385b80 1 -- 172.21.15.33:0/583673413 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8b8007bd0 con 0x55cd36b97350 2024-05-15T14:01:43.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.029+0000 7fd8a5ffb700 1 -- 172.21.15.33:0/583673413 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fd8b00109f0 con 0x55cd36b97350 2024-05-15T14:01:43.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.029+0000 7fd8a5ffb700 1 -- 172.21.15.33:0/583673413 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd8b0010bf0 con 0x55cd36b97350 2024-05-15T14:01:43.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.029+0000 7fd8cf385b80 10 client.15492 fetch_fsmap learned FSMap version 24 2024-05-15T14:01:43.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.029+0000 7fd8cf385b80 1 -- 172.21.15.33:0/583673413 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55cd36a9e8f0 con 0x55cd36b97350 2024-05-15T14:01:43.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.030+0000 7fd8a5ffb700 1 -- 172.21.15.33:0/583673413 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8b8007bd0 con 0x55cd36b97350 2024-05-15T14:01:43.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.030+0000 7fd8a5ffb700 1 -- 172.21.15.33:0/583673413 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fd8b000f3c0 con 0x55cd36b97350 2024-05-15T14:01:43.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.030+0000 7fd8cf385b80 10 client.15492 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:01:43.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.030+0000 7fd8cf385b80 10 client.15492 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:01:43.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.030+0000 7fd8cf385b80 20 client.15492 populate_metadata read hostname 'smithi033' 2024-05-15T14:01:43.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.030+0000 7fd8cf385b80 1 --2- 172.21.15.33:0/583673413 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55cd36bb4880 0x55cd36bb6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.030+0000 7fd8cf385b80 4 client.15492 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:01:43.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.030+0000 7fd8cf385b80 1 -- 172.21.15.33:0/583673413 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd36af6e50 con 0x55cd36bb4880 2024-05-15T14:01:43.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.030+0000 7fd8bdd9b700 1 --2- 172.21.15.33:0/583673413 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55cd36bb4880 0x55cd36bb6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:43.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.031+0000 7fd8a5ffb700 10 client.15492 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:01:43.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.031+0000 7fd8bdd9b700 1 --2- 172.21.15.33:0/583673413 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55cd36bb4880 0x55cd36bb6c90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92f741c102212715 server_cookie=7e0cbdbc74c81787 in_seq=0 out_seq=0 2024-05-15T14:01:43.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.031+0000 7fd8a5ffb700 1 -- 172.21.15.33:0/583673413 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cd36af6e50 con 0x55cd36bb4880 2024-05-15T14:01:43.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.031+0000 7fd8a5ffb700 10 client.15492 handle_command_reply: tid=0 2024-05-15T14:01:43.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.031+0000 7fd8cf385b80 1 client.15492 shutdown 2024-05-15T14:01:43.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.032+0000 7fd8cf385b80 1 -- 172.21.15.33:0/583673413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd36b97350 msgr2=0x55cd36ba2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:43.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.032+0000 7fd8cf385b80 1 --2- 172.21.15.33:0/583673413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd36b97350 0x55cd36ba2b90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00126d0 tx=0x7fd8b00129e0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.032+0000 7fd8cf385b80 1 -- 172.21.15.33:0/583673413 shutdown_connections 2024-05-15T14:01:43.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.032+0000 7fd8cf385b80 1 --2- 172.21.15.33:0/583673413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd36b9bdd0 0x55cd36ba0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.032+0000 7fd8cf385b80 1 --2- 172.21.15.33:0/583673413 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55cd36bb4880 0x55cd36bb6c90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.032+0000 7fd8cf385b80 1 --2- 172.21.15.33:0/583673413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd36b97350 0x55cd36ba2b90 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.032+0000 7fd8cf385b80 1 --2- 172.21.15.33:0/583673413 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd36b9b4b0 0x55cd36b9ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.032+0000 7fd8cf385b80 1 -- 172.21.15.33:0/583673413 >> 172.21.15.33:0/583673413 conn(0x55cd36ac3f40 msgr2=0x55cd36b8c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:43.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.032+0000 7fd8a5ffb700 0 client.15492 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:01:43.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.032+0000 7fd8cf385b80 1 -- 172.21.15.33:0/583673413 shutdown_connections 2024-05-15T14:01:43.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.032+0000 7fd8cf385b80 1 -- 172.21.15.33:0/583673413 wait complete. 2024-05-15T14:01:43.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.032+0000 7fd8cf385b80 20 client.15492 trim_cache size 0 max 16384 2024-05-15T14:01:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.052+0000 7f5aceb90640 1 -- 172.21.15.33:0/2869950758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac8107940 msgr2=0x7f5ac8107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.052+0000 7f5aceb90640 1 --2- 172.21.15.33:0/2869950758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac8107940 0x7f5ac8107d20 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5abc009930 tx=0x7f5abc02a420 comp rx=0 tx=0).stop 2024-05-15T14:01:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.052+0000 7f5aceb90640 1 -- 172.21.15.33:0/2869950758 shutdown_connections 2024-05-15T14:01:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.052+0000 7f5aceb90640 1 --2- 172.21.15.33:0/2869950758 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ac81022f0 0x7f5ac810e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.052+0000 7f5aceb90640 1 --2- 172.21.15.33:0/2869950758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ac8101930 0x7f5ac8101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.052+0000 7f5aceb90640 1 --2- 172.21.15.33:0/2869950758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac8107940 0x7f5ac8107d20 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.052+0000 7f5aceb90640 1 -- 172.21.15.33:0/2869950758 >> 172.21.15.33:0/2869950758 conn(0x7f5ac8078a70 msgr2=0x7f5ac8078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.052+0000 7f5aceb90640 1 -- 172.21.15.33:0/2869950758 shutdown_connections 2024-05-15T14:01:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.053+0000 7f5aceb90640 1 -- 172.21.15.33:0/2869950758 wait complete. 2024-05-15T14:01:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.053+0000 7f5aceb90640 1 Processor -- start 2024-05-15T14:01:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.053+0000 7f5aceb90640 1 -- start start 2024-05-15T14:01:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.053+0000 7f5aceb90640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ac8101930 0x7f5ac8199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.053+0000 7f5aceb90640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ac81022f0 0x7f5ac81997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.054+0000 7f5aceb90640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac8107940 0x7f5ac819db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.054+0000 7f5aceb90640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac8111bd0 con 0x7f5ac8107940 2024-05-15T14:01:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.054+0000 7f5aceb90640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac8111a50 con 0x7f5ac81022f0 2024-05-15T14:01:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.054+0000 7f5aceb90640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac8111d50 con 0x7f5ac8101930 2024-05-15T14:01:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.054+0000 7f5acd12d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac8107940 0x7f5ac819db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.054+0000 7f5acd12d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac8107940 0x7f5ac819db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48866/0 (socket says 172.21.15.33:48866) 2024-05-15T14:01:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.054+0000 7f5acd12d640 1 -- 172.21.15.33:0/1918202370 learned_addr learned my addr 172.21.15.33:0/1918202370 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.054+0000 7f5acc92c640 1 --2- 172.21.15.33:0/1918202370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ac8101930 0x7f5ac8199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.054+0000 7f5ac7fff640 1 --2- 172.21.15.33:0/1918202370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ac81022f0 0x7f5ac81997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.054+0000 7f5acd12d640 1 -- 172.21.15.33:0/1918202370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ac8101930 msgr2=0x7f5ac8199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.054+0000 7f5acd12d640 1 --2- 172.21.15.33:0/1918202370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ac8101930 0x7f5ac8199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.054+0000 7f5acd12d640 1 -- 172.21.15.33:0/1918202370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ac81022f0 msgr2=0x7f5ac81997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.054+0000 7f5acd12d640 1 --2- 172.21.15.33:0/1918202370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ac81022f0 0x7f5ac81997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.054+0000 7f5acd12d640 1 -- 172.21.15.33:0/1918202370 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac819e270 con 0x7f5ac8107940 2024-05-15T14:01:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.054+0000 7f5acd12d640 1 --2- 172.21.15.33:0/1918202370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac8107940 0x7f5ac819db40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000eaa0 tx=0x7f5ac000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.054+0000 7f5ac5ffb640 1 -- 172.21.15.33:0/1918202370 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac000cdb0 con 0x7f5ac8107940 2024-05-15T14:01:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.055+0000 7f5aceb90640 1 -- 172.21.15.33:0/1918202370 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac819e560 con 0x7f5ac8107940 2024-05-15T14:01:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.055+0000 7f5ac5ffb640 1 -- 172.21.15.33:0/1918202370 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ac0005020 con 0x7f5ac8107940 2024-05-15T14:01:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.055+0000 7f5ac5ffb640 1 -- 172.21.15.33:0/1918202370 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac00116d0 con 0x7f5ac8107940 2024-05-15T14:01:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.055+0000 7f5aceb90640 1 -- 172.21.15.33:0/1918202370 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac81a5df0 con 0x7f5ac8107940 2024-05-15T14:01:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.057+0000 7f5ad1985740 1 Processor -- start 2024-05-15T14:01:43.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.057+0000 7fd8cf385b80 1 Processor -- start 2024-05-15T14:01:43.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.057+0000 7fd8cf385b80 1 -- start start 2024-05-15T14:01:43.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.058+0000 7fd8cf385b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd36b97350 0x55cd36b8c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.058+0000 7fd8cf385b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd36b9b4b0 0x55cd36ba1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.058+0000 7fd8cf385b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd36b9bdd0 0x55cd36ba2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.058+0000 7fd8cf385b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a000fe40 con 0x55cd36b97350 2024-05-15T14:01:43.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.058+0000 7fd8cf385b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a000d6d0 con 0x55cd36b9b4b0 2024-05-15T14:01:43.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.058+0000 7fd8cf385b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a000b520 con 0x55cd36b9bdd0 2024-05-15T14:01:43.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.058+0000 7fd8be59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd36b97350 0x55cd36b8c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:43.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.058+0000 7fd8be59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd36b97350 0x55cd36b8c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48872/0 (socket says 172.21.15.33:48872) 2024-05-15T14:01:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.058+0000 7fd8be59c700 1 -- 172.21.15.33:0/1616857109 learned_addr learned my addr 172.21.15.33:0/1616857109 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.058+0000 7fd8bdd9b700 1 --2- 172.21.15.33:0/1616857109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd36b9b4b0 0x55cd36ba1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.058+0000 7fd8bed9d700 1 --2- 172.21.15.33:0/1616857109 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd36b9bdd0 0x55cd36ba2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.058+0000 7fd8be59c700 1 -- 172.21.15.33:0/1616857109 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd36b9bdd0 msgr2=0x55cd36ba2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.058+0000 7fd8be59c700 1 --2- 172.21.15.33:0/1616857109 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd36b9bdd0 0x55cd36ba2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.058+0000 7fd8be59c700 1 -- 172.21.15.33:0/1616857109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd36b9b4b0 msgr2=0x55cd36ba1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.058+0000 7fd8be59c700 1 --2- 172.21.15.33:0/1616857109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd36b9b4b0 0x55cd36ba1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.058+0000 7fd8be59c700 1 -- 172.21.15.33:0/1616857109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd36b9e4c0 con 0x55cd36b97350 2024-05-15T14:01:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.058+0000 7fd8be59c700 1 --2- 172.21.15.33:0/1616857109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd36b97350 0x55cd36b8c160 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd8b000a010 tx=0x7fd8b0009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f3d542b2a05c5f server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.059+0000 7f5ac5ffb640 1 -- 172.21.15.33:0/1918202370 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f5ac00051d0 con 0x7f5ac8107940 2024-05-15T14:01:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.059+0000 7f5ad1985740 1 -- start start 2024-05-15T14:01:43.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.059+0000 7fd8a7fff700 1 -- 172.21.15.33:0/1616857109 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b002a070 con 0x55cd36b97350 2024-05-15T14:01:43.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.059+0000 7fd8a7fff700 1 -- 172.21.15.33:0/1616857109 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8b000f3c0 con 0x55cd36b97350 2024-05-15T14:01:43.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.059+0000 7fd8a7fff700 1 -- 172.21.15.33:0/1616857109 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b0005310 con 0x55cd36b97350 2024-05-15T14:01:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.060+0000 7f5ac5ffb640 1 --2- 172.21.15.33:0/1918202370 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f5ab80b2a70 0x7f5ab80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.060+0000 7f5ad1985740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607804478f0 0x560780447cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.060+0000 7f5ad1985740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560780448210 0x560780452590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.060+0000 7f5ad1985740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560780452ad0 0x560780454ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.060+0000 7f5ad1985740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5607803bac30 con 0x560780448210 2024-05-15T14:01:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.060+0000 7f5ad1985740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5607803aa7d0 con 0x5607804478f0 2024-05-15T14:01:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.060+0000 7f5ac7fff640 1 --2- 172.21.15.33:0/1918202370 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f5ab80b2a70 0x7f5ab80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.060+0000 7f5ad1985740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5607803ae810 con 0x560780452ad0 2024-05-15T14:01:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.060+0000 7f5acc92c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607804478f0 0x560780447cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.060+0000 7f5ac5ffb640 1 -- 172.21.15.33:0/1918202370 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f5ac00e2700 con 0x7f5ac8107940 2024-05-15T14:01:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.060+0000 7f5acc92c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607804478f0 0x560780447cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:45114/0 (socket says 172.21.15.33:45114) 2024-05-15T14:01:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.060+0000 7f5acc92c640 1 -- 172.21.15.33:0/1058156793 learned_addr learned my addr 172.21.15.33:0/1058156793 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.060+0000 7f5acd12d640 1 --2- 172.21.15.33:0/1058156793 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560780452ad0 0x560780454ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.060+0000 7f5ac7fff640 1 --2- 172.21.15.33:0/1058156793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560780448210 0x560780452590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:43.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.060+0000 7fd8cf385b80 1 -- 172.21.15.33:0/1616857109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd36b97350 msgr2=0x55cd36b8c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:43.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.060+0000 7fd8cf385b80 1 --2- 172.21.15.33:0/1616857109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd36b97350 0x55cd36b8c160 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd8b000a010 tx=0x7fd8b0009770 comp rx=0 tx=0).stop 2024-05-15T14:01:43.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.060+0000 7fd8cf385b80 1 -- 172.21.15.33:0/1616857109 shutdown_connections 2024-05-15T14:01:43.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.060+0000 7fd8cf385b80 1 --2- 172.21.15.33:0/1616857109 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd36b9bdd0 0x55cd36ba2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.060+0000 7fd8cf385b80 1 --2- 172.21.15.33:0/1616857109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd36b97350 0x55cd36b8c160 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.060+0000 7fd8cf385b80 1 --2- 172.21.15.33:0/1616857109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd36b9b4b0 0x55cd36ba1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.060+0000 7fd8cf385b80 1 -- 172.21.15.33:0/1616857109 >> 172.21.15.33:0/1616857109 conn(0x55cd36ac3f40 msgr2=0x55cd36ba1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:43.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.060+0000 7fd8cf385b80 1 -- 172.21.15.33:0/1616857109 shutdown_connections 2024-05-15T14:01:43.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.060+0000 7fd8cf385b80 1 -- 172.21.15.33:0/1616857109 wait complete. 2024-05-15T14:01:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.061+0000 7f5acd12d640 1 -- 172.21.15.33:0/1058156793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607804478f0 msgr2=0x560780447cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.061+0000 7f5acd12d640 1 --2- 172.21.15.33:0/1058156793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607804478f0 0x560780447cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.061+0000 7f5acd12d640 1 -- 172.21.15.33:0/1058156793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560780448210 msgr2=0x560780452590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.061+0000 7f5acd12d640 1 --2- 172.21.15.33:0/1058156793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560780448210 0x560780452590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.061+0000 7f5acd12d640 1 -- 172.21.15.33:0/1058156793 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607800bf210 con 0x560780452ad0 2024-05-15T14:01:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.061+0000 7f5acc92c640 1 --2- 172.21.15.33:0/1058156793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607804478f0 0x560780447cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:01:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.061+0000 7f5acd12d640 1 --2- 172.21.15.33:0/1058156793 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560780452ad0 0x560780454ec0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00dfce0 tx=0x7f5ac00a93f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95ef8e84ed2fa6c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.061+0000 7f5ac7fff640 1 --2- 172.21.15.33:0/1918202370 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f5ab80b2a70 0x7f5ab80b4f30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8079250 tx=0x7f5ab40098a0 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.061+0000 7f5ab2ffd640 1 -- 172.21.15.33:0/1058156793 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac001f070 con 0x560780452ad0 2024-05-15T14:01:43.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.061+0000 7fd8cf385b80 1 -- start start 2024-05-15T14:01:43.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.061+0000 7f5ab2ffd640 1 -- 172.21.15.33:0/1058156793 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ac00adb50 con 0x560780452ad0 2024-05-15T14:01:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.061+0000 7f5ab2ffd640 1 -- 172.21.15.33:0/1058156793 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac004e110 con 0x560780452ad0 2024-05-15T14:01:43.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.061+0000 7fd8cf385b80 1 Processor -- start 2024-05-15T14:01:43.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.061+0000 7fd8cf385b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd36b97350 0x55cd36bad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.062+0000 7f5ad1985740 1 -- 172.21.15.33:0/1058156793 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560780452ad0 msgr2=0x560780454ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:43.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.062+0000 7f5ad1985740 1 --2- 172.21.15.33:0/1058156793 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560780452ad0 0x560780454ec0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00dfce0 tx=0x7f5ac00a93f0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.062+0000 7fd8cf385b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd36b9b4b0 0x55cd36ba7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.062+0000 7fd8cf385b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd36b9bdd0 0x55cd36ba84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.062+0000 7fd8cf385b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a000bb90 con 0x55cd36b97350 2024-05-15T14:01:43.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.062+0000 7fd8cf385b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a000d000 con 0x55cd36b9bdd0 2024-05-15T14:01:43.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.062+0000 7fd8cf385b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a000c260 con 0x55cd36b9b4b0 2024-05-15T14:01:43.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.062+0000 7fd8cf385b80 10 client.0 _ll_register_callbacks cb 0x55cd36a7fae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:01:43.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.062+0000 7fd8be59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd36b97350 0x55cd36bad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:43.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.062+0000 7fd8be59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd36b97350 0x55cd36bad3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48888/0 (socket says 172.21.15.33:48888) 2024-05-15T14:01:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.062+0000 7f5ad1985740 1 -- 172.21.15.33:0/1058156793 shutdown_connections 2024-05-15T14:01:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.062+0000 7f5ad1985740 1 --2- 172.21.15.33:0/1058156793 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560780452ad0 0x560780454ec0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.062+0000 7f5ad1985740 1 --2- 172.21.15.33:0/1058156793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560780448210 0x560780452590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.062+0000 7f5ad1985740 1 --2- 172.21.15.33:0/1058156793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607804478f0 0x560780447cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.062+0000 7f5ad1985740 1 -- 172.21.15.33:0/1058156793 >> 172.21.15.33:0/1058156793 conn(0x560780443e00 msgr2=0x5607804441e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:43.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.062+0000 7fd8be59c700 1 -- 172.21.15.33:0/2744622228 learned_addr learned my addr 172.21.15.33:0/2744622228 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:43.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.062+0000 7fd8bdd9b700 1 --2- 172.21.15.33:0/2744622228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd36b9b4b0 0x55cd36ba7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:43.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.062+0000 7fd8bed9d700 1 --2- 172.21.15.33:0/2744622228 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd36b9bdd0 0x55cd36ba84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:43.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.062+0000 7fd8be59c700 1 -- 172.21.15.33:0/2744622228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd36b9b4b0 msgr2=0x55cd36ba7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:43.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.062+0000 7fd8be59c700 1 --2- 172.21.15.33:0/2744622228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd36b9b4b0 0x55cd36ba7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.062+0000 7fd8be59c700 1 -- 172.21.15.33:0/2744622228 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd36b9bdd0 msgr2=0x55cd36ba84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:43.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.062+0000 7fd8be59c700 1 --2- 172.21.15.33:0/2744622228 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd36b9bdd0 0x55cd36ba84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.062+0000 7fd8be59c700 1 -- 172.21.15.33:0/2744622228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd36ba8b60 con 0x55cd36b97350 2024-05-15T14:01:43.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.062+0000 7f5ad1985740 1 -- 172.21.15.33:0/1058156793 shutdown_connections 2024-05-15T14:01:43.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.062+0000 7f5ad1985740 1 -- 172.21.15.33:0/1058156793 wait complete. 2024-05-15T14:01:43.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.062+0000 7fd8be59c700 1 --2- 172.21.15.33:0/2744622228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd36b97350 0x55cd36bad3a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00092a0 tx=0x7fd8b0005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a916d76d90da08c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:43.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.063+0000 7fd8a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:01:43.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.063+0000 7fd8a5ffb700 1 -- 172.21.15.33:0/2744622228 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b002a070 con 0x55cd36b97350 2024-05-15T14:01:43.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.063+0000 7fd8cf385b80 1 -- 172.21.15.33:0/2744622228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8b000f3c0 con 0x55cd36b97350 2024-05-15T14:01:43.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.063+0000 7fd8a5ffb700 1 -- 172.21.15.33:0/2744622228 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fd8b00050f0 con 0x55cd36b97350 2024-05-15T14:01:43.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.063+0000 7fd8a5ffb700 1 -- 172.21.15.33:0/2744622228 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd8b0010e50 con 0x55cd36b97350 2024-05-15T14:01:43.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.063+0000 7f5ad1985740 1 -- start start 2024-05-15T14:01:43.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.063+0000 7fd8cf385b80 10 client.15507 fetch_fsmap learned FSMap version 24 2024-05-15T14:01:43.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.063+0000 7fd8cf385b80 1 -- 172.21.15.33:0/2744622228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55cd36c1ffd0 con 0x55cd36b97350 2024-05-15T14:01:43.075 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.063+0000 7f5ad1985740 1 Processor -- start 2024-05-15T14:01:43.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.063+0000 7fd8a5ffb700 1 -- 172.21.15.33:0/2744622228 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8b000f3c0 con 0x55cd36b97350 2024-05-15T14:01:43.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.064+0000 7fd8a5ffb700 1 -- 172.21.15.33:0/2744622228 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fd8b001d450 con 0x55cd36b97350 2024-05-15T14:01:43.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.064+0000 7fd8cf385b80 10 client.15507 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:01:43.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.064+0000 7fd8cf385b80 10 client.15507 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:01:43.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.064+0000 7fd8cf385b80 20 client.15507 populate_metadata read hostname 'smithi033' 2024-05-15T14:01:43.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.064+0000 7fd8cf385b80 1 --2- 172.21.15.33:0/2744622228 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55cd36c20630 0x55cd36c22a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.064+0000 7fd8cf385b80 4 client.15507 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.0, "await_for": 45.0}] 2024-05-15T14:01:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.064+0000 7f5ad1985740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607804478f0 0x5607804524b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.064+0000 7f5ad1985740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560780448210 0x56078044f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.064+0000 7fd8cf385b80 1 -- 172.21.15.33:0/2744622228 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.0, "await_for": 45.0}) -- 0x55cd36af0630 con 0x55cd36c20630 2024-05-15T14:01:43.078 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.064+0000 7f5ad1985740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560780452ad0 0x56078044fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.064+0000 7fd8bdd9b700 1 --2- 172.21.15.33:0/2744622228 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55cd36c20630 0x55cd36c22a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.064+0000 7f5ad1985740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560780090790 con 0x560780452ad0 2024-05-15T14:01:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.064+0000 7f5ad1985740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x560780366890 con 0x5607804478f0 2024-05-15T14:01:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.064+0000 7f5ad1985740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5607804557a0 con 0x560780448210 2024-05-15T14:01:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.064+0000 7f5acd12d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560780452ad0 0x56078044fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:43.079 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.064+0000 7f5ad1985740 10 client.0 _ll_register_callbacks cb 0x560780369d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:01:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.064+0000 7f5acd12d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560780452ad0 0x56078044fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48890/0 (socket says 172.21.15.33:48890) 2024-05-15T14:01:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.064+0000 7f5acd12d640 1 -- 172.21.15.33:0/59825094 learned_addr learned my addr 172.21.15.33:0/59825094 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.064+0000 7f5ac7fff640 1 --2- 172.21.15.33:0/59825094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560780448210 0x56078044f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:43.080 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.064+0000 7f5acc92c640 1 --2- 172.21.15.33:0/59825094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607804478f0 0x5607804524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:43.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.064+0000 7fd8a5ffb700 10 client.15507 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:01:43.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.064+0000 7fd8bdd9b700 1 --2- 172.21.15.33:0/2744622228 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55cd36c20630 0x55cd36c22a20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d59e7db3da420c6 server_cookie=cb10999eb16be419 in_seq=0 out_seq=0 2024-05-15T14:01:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.065+0000 7f5acd12d640 1 -- 172.21.15.33:0/59825094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560780448210 msgr2=0x56078044f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.065+0000 7f5acd12d640 1 --2- 172.21.15.33:0/59825094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560780448210 0x56078044f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.065+0000 7f5acd12d640 1 -- 172.21.15.33:0/59825094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607804478f0 msgr2=0x5607804524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.065+0000 7f5acd12d640 1 --2- 172.21.15.33:0/59825094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607804478f0 0x5607804524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.065+0000 7f5acd12d640 1 -- 172.21.15.33:0/59825094 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56078018b620 con 0x560780452ad0 2024-05-15T14:01:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.065+0000 7f5acc92c640 1 --2- 172.21.15.33:0/59825094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607804478f0 0x5607804524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:01:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.065+0000 7f5ac7fff640 1 --2- 172.21.15.33:0/59825094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560780448210 0x56078044f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:01:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.065+0000 7f5acd12d640 1 --2- 172.21.15.33:0/59825094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560780452ad0 0x56078044fdb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00dfd40 tx=0x7f5ac004e340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b095f5f74839825e server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.065+0000 7f5aa7fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:01:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.065+0000 7f5aa7fff640 1 -- 172.21.15.33:0/59825094 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac001f050 con 0x560780452ad0 2024-05-15T14:01:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.065+0000 7f5aa7fff640 1 -- 172.21.15.33:0/59825094 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f5ac00912c0 con 0x560780452ad0 2024-05-15T14:01:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.065+0000 7f5ad1985740 1 -- 172.21.15.33:0/59825094 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ac00adb50 con 0x560780452ad0 2024-05-15T14:01:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.065+0000 7f5aa7fff640 1 -- 172.21.15.33:0/59825094 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5ac00b2600 con 0x560780452ad0 2024-05-15T14:01:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.065+0000 7f5ad1985740 10 client.15510 fetch_fsmap learned FSMap version 24 2024-05-15T14:01:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.065+0000 7f5ad1985740 1 -- 172.21.15.33:0/59825094 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5607804610a0 con 0x560780452ad0 2024-05-15T14:01:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.065+0000 7f5aa7fff640 1 -- 172.21.15.33:0/59825094 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ac00adb50 con 0x560780452ad0 2024-05-15T14:01:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.066+0000 7f5aa7fff640 1 -- 172.21.15.33:0/59825094 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f5ac0022060 con 0x560780452ad0 2024-05-15T14:01:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.066+0000 7f5ad1985740 10 client.15510 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:01:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.066+0000 7f5ad1985740 10 client.15510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:01:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.066+0000 7f5ad1985740 20 client.15510 populate_metadata read hostname 'smithi033' 2024-05-15T14:01:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.066+0000 7f5ad1985740 1 --2- 172.21.15.33:0/59825094 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5607804623f0 0x5607804647e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.066+0000 7f5ad1985740 4 client.15510 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:01:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.066+0000 7f5ad1985740 1 -- 172.21.15.33:0/59825094 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607803bcb80 con 0x5607804623f0 2024-05-15T14:01:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.066+0000 7f5acc92c640 1 --2- 172.21.15.33:0/59825094 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5607804623f0 0x5607804647e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.066+0000 7f5aa7fff640 10 client.15510 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:01:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.066+0000 7f5acc92c640 1 --2- 172.21.15.33:0/59825094 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5607804623f0 0x5607804647e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53636ae1fa6fe776 server_cookie=b7cd1d03172638f in_seq=0 out_seq=0 2024-05-15T14:01:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.067+0000 7f5aa7fff640 1 -- 172.21.15.33:0/59825094 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5607803bcb80 con 0x5607804623f0 2024-05-15T14:01:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.067+0000 7f5aa7fff640 10 client.15510 handle_command_reply: tid=0 2024-05-15T14:01:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.067+0000 7f5ad1985740 1 client.15510 shutdown 2024-05-15T14:01:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.067+0000 7f5ad1985740 1 -- 172.21.15.33:0/59825094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560780452ad0 msgr2=0x56078044fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.067+0000 7f5ad1985740 1 --2- 172.21.15.33:0/59825094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560780452ad0 0x56078044fdb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5ac00dfd40 tx=0x7f5ac004e340 comp rx=0 tx=0).stop 2024-05-15T14:01:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.067+0000 7f5ad1985740 1 -- 172.21.15.33:0/59825094 shutdown_connections 2024-05-15T14:01:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.067+0000 7f5ad1985740 1 --2- 172.21.15.33:0/59825094 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5607804623f0 0x5607804647e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.068+0000 7f5ad1985740 1 --2- 172.21.15.33:0/59825094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560780452ad0 0x56078044fdb0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.068+0000 7f5aa7fff640 0 client.15510 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:01:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.068+0000 7f5ad1985740 1 --2- 172.21.15.33:0/59825094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560780448210 0x56078044f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.068+0000 7f5ad1985740 1 --2- 172.21.15.33:0/59825094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607804478f0 0x5607804524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.068+0000 7f5ad1985740 1 -- 172.21.15.33:0/59825094 >> 172.21.15.33:0/59825094 conn(0x560780443e00 msgr2=0x560780443100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.068+0000 7f5ad1985740 1 -- 172.21.15.33:0/59825094 shutdown_connections 2024-05-15T14:01:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.068+0000 7f5ad1985740 1 -- 172.21.15.33:0/59825094 wait complete. 2024-05-15T14:01:43.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.068+0000 7f5ad1985740 20 client.15510 trim_cache size 0 max 16384 2024-05-15T14:01:43.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.071+0000 7fd8a5ffb700 1 -- 172.21.15.33:0/2744622228 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55cd36af0630 con 0x55cd36c20630 2024-05-15T14:01:43.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.071+0000 7fd8a5ffb700 10 client.15507 handle_command_reply: tid=0 2024-05-15T14:01:43.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.071+0000 7fd8cf385b80 1 client.15507 shutdown 2024-05-15T14:01:43.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.072+0000 7fd8cf385b80 1 -- 172.21.15.33:0/2744622228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd36b97350 msgr2=0x55cd36bad3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:43.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.072+0000 7fd8cf385b80 1 --2- 172.21.15.33:0/2744622228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd36b97350 0x55cd36bad3a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00092a0 tx=0x7fd8b0005cf0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.072+0000 7fd8cf385b80 1 -- 172.21.15.33:0/2744622228 shutdown_connections 2024-05-15T14:01:43.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.072+0000 7fd8cf385b80 1 --2- 172.21.15.33:0/2744622228 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cd36b9bdd0 0x55cd36ba84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.072+0000 7fd8cf385b80 1 --2- 172.21.15.33:0/2744622228 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55cd36c20630 0x55cd36c22a20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.072+0000 7fd8cf385b80 1 --2- 172.21.15.33:0/2744622228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd36b97350 0x55cd36bad3a0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.072+0000 7fd8a5ffb700 0 client.15507 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:01:43.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.072+0000 7fd8cf385b80 1 --2- 172.21.15.33:0/2744622228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cd36b9b4b0 0x55cd36ba7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.072+0000 7fd8cf385b80 1 -- 172.21.15.33:0/2744622228 >> 172.21.15.33:0/2744622228 conn(0x55cd36ac3f40 msgr2=0x55cd36ba1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:43.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.072+0000 7fd8cf385b80 1 -- 172.21.15.33:0/2744622228 shutdown_connections 2024-05-15T14:01:43.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.072+0000 7fd8cf385b80 1 -- 172.21.15.33:0/2744622228 wait complete. 2024-05-15T14:01:43.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.072+0000 7fd8cf385b80 20 client.15507 trim_cache size 0 max 16384 2024-05-15T14:01:43.089 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T14:01:43.089 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 24, 2024-05-15T14:01:43.089 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24478, 2024-05-15T14:01:43.089 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 6, 2024-05-15T14:01:43.089 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-05-15T14:01:43.089 INFO:teuthology.orchestra.run.smithi033.stdout: "e8d9ebc6": { 2024-05-15T14:01:43.090 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 6, 2024-05-15T14:01:43.090 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-05-15T14:01:43.090 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:01:43.090 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-05-15T14:01:43.090 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:01:43.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:01:43.090 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 90.0, 2024-05-15T14:01:43.090 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 144.0, 2024-05-15T14:01:43.090 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-05-15T14:01:43.090 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-05-15T14:01:43.090 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-05-15T14:01:43.090 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:01:43.090 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-05-15T14:01:43.090 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:01:43.090 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:01:43.091 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:01:43.091 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:01:43.091 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:01:43.091 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:01:43.091 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T14:01:43.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.073+0000 7fd8c4aef700 1 -- 172.21.15.33:0/3914785109 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fd8a80b2bc0 msgr2=0x7fd8a80b5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:43.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.073+0000 7fd8c4aef700 1 --2- 172.21.15.33:0/3914785109 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fd8a80b2bc0 0x7fd8a80b5080 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009e60 tx=0x7fd8b8009df0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.073+0000 7fd8c4aef700 1 -- 172.21.15.33:0/3914785109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c0103f90 msgr2=0x7fd8c019fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:43.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.073+0000 7fd8c4aef700 1 --2- 172.21.15.33:0/3914785109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c0103f90 0x7fd8c019fca0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd8b400d750 tx=0x7fd8b400da60 comp rx=0 tx=0).stop 2024-05-15T14:01:43.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.073+0000 7fd8c4aef700 1 -- 172.21.15.33:0/3914785109 shutdown_connections 2024-05-15T14:01:43.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.073+0000 7fd8c4aef700 1 --2- 172.21.15.33:0/3914785109 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fd8a80b2bc0 0x7fd8a80b5080 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.073+0000 7fd8c4aef700 1 --2- 172.21.15.33:0/3914785109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8c0108500 0x7fd8c0199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.073+0000 7fd8c4aef700 1 --2- 172.21.15.33:0/3914785109 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8c01035f0 0x7fd8c019f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.073+0000 7fd8c4aef700 1 --2- 172.21.15.33:0/3914785109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8c0103f90 0x7fd8c019fca0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.073+0000 7fd8c4aef700 1 -- 172.21.15.33:0/3914785109 >> 172.21.15.33:0/3914785109 conn(0x7fd8c007a6a0 msgr2=0x7fd8c0101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:43.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.073+0000 7fd8c4aef700 1 -- 172.21.15.33:0/3914785109 shutdown_connections 2024-05-15T14:01:43.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:01:43.073+0000 7fd8c4aef700 1 -- 172.21.15.33:0/3914785109 wait complete. 2024-05-15T14:01:43.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.084+0000 7f5ad1985740 1 Processor -- start 2024-05-15T14:01:43.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.085+0000 7f5ad1985740 1 -- start start 2024-05-15T14:01:43.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.085+0000 7f5ad1985740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607804478f0 0x560780469de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.085+0000 7f5ad1985740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560780448210 0x56078046a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.085+0000 7f5ad1985740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560780452ad0 0x56078044ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.085+0000 7f5ad1985740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac81aedc0 con 0x560780448210 2024-05-15T14:01:43.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.085+0000 7f5ad1985740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac81ac690 con 0x5607804478f0 2024-05-15T14:01:43.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.085+0000 7f5ad1985740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac81aa4e0 con 0x560780452ad0 2024-05-15T14:01:43.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.085+0000 7f5ac7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560780448210 0x56078046a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.085+0000 7f5ac7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560780448210 0x56078046a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48894/0 (socket says 172.21.15.33:48894) 2024-05-15T14:01:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.085+0000 7f5ac7fff640 1 -- 172.21.15.33:0/1918961799 learned_addr learned my addr 172.21.15.33:0/1918961799 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.085+0000 7f5acc92c640 1 --2- 172.21.15.33:0/1918961799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607804478f0 0x560780469de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.085+0000 7f5acd12d640 1 --2- 172.21.15.33:0/1918961799 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560780452ad0 0x56078044ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.085+0000 7f5acc92c640 1 -- 172.21.15.33:0/1918961799 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560780452ad0 msgr2=0x56078044ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.085+0000 7f5acc92c640 1 --2- 172.21.15.33:0/1918961799 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560780452ad0 0x56078044ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.085+0000 7f5acc92c640 1 -- 172.21.15.33:0/1918961799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560780448210 msgr2=0x56078046a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.085+0000 7f5acc92c640 1 --2- 172.21.15.33:0/1918961799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560780448210 0x56078046a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.085+0000 7f5acc92c640 1 -- 172.21.15.33:0/1918961799 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56078044dcf0 con 0x5607804478f0 2024-05-15T14:01:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.085+0000 7f5ac7fff640 1 --2- 172.21.15.33:0/1918961799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560780448210 0x56078046a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:01:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.086+0000 7f5acc92c640 1 --2- 172.21.15.33:0/1918961799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607804478f0 0x560780469de0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5abc009900 tx=0x7f5abc005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=924be0ae1dea5de7 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.086+0000 7f5ab27fc640 1 -- 172.21.15.33:0/1918961799 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc04a070 con 0x5607804478f0 2024-05-15T14:01:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.086+0000 7f5ab27fc640 1 -- 172.21.15.33:0/1918961799 <== mon.2 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5abc004400 con 0x5607804478f0 2024-05-15T14:01:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.086+0000 7f5ab27fc640 1 -- 172.21.15.33:0/1918961799 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc035040 con 0x5607804478f0 2024-05-15T14:01:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.087+0000 7f5ad1985740 1 -- 172.21.15.33:0/1918961799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607804478f0 msgr2=0x560780469de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.087+0000 7f5ad1985740 1 --2- 172.21.15.33:0/1918961799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607804478f0 0x560780469de0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5abc009900 tx=0x7f5abc005fd0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.087+0000 7f5ad1985740 1 -- 172.21.15.33:0/1918961799 shutdown_connections 2024-05-15T14:01:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.087+0000 7f5ad1985740 1 --2- 172.21.15.33:0/1918961799 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560780452ad0 0x56078044ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.087+0000 7f5ad1985740 1 --2- 172.21.15.33:0/1918961799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560780448210 0x56078046a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.087+0000 7f5ad1985740 1 --2- 172.21.15.33:0/1918961799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5607804478f0 0x560780469de0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.087+0000 7f5ad1985740 1 -- 172.21.15.33:0/1918961799 >> 172.21.15.33:0/1918961799 conn(0x560780443e00 msgr2=0x56078045d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.087+0000 7f5ad1985740 1 -- 172.21.15.33:0/1918961799 shutdown_connections 2024-05-15T14:01:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.087+0000 7f5ad1985740 1 -- 172.21.15.33:0/1918961799 wait complete. 2024-05-15T14:01:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.087+0000 7f5ad1985740 1 -- start start 2024-05-15T14:01:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.087+0000 7f5ad1985740 1 Processor -- start 2024-05-15T14:01:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.088+0000 7f5ad1985740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607804478f0 0x56078045ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.088+0000 7f5ad1985740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560780448210 0x560780458200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.088+0000 7f5ad1985740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560780452ad0 0x560780458740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.088+0000 7f5ad1985740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac81aab50 con 0x560780448210 2024-05-15T14:01:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.088+0000 7f5ad1985740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac81abfc0 con 0x560780452ad0 2024-05-15T14:01:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.088+0000 7f5ad1985740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac81ab220 con 0x5607804478f0 2024-05-15T14:01:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.088+0000 7f5ad1985740 10 client.0 _ll_register_callbacks cb 0x7f5ac00a5c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:01:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.088+0000 7f5ac7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560780448210 0x560780458200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.088+0000 7f5ac7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560780448210 0x560780458200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48906/0 (socket says 172.21.15.33:48906) 2024-05-15T14:01:43.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.088+0000 7f5ac7fff640 1 -- 172.21.15.33:0/1268059903 learned_addr learned my addr 172.21.15.33:0/1268059903 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.088+0000 7f5acc92c640 1 --2- 172.21.15.33:0/1268059903 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607804478f0 0x56078045ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.088+0000 7f5acd12d640 1 --2- 172.21.15.33:0/1268059903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560780452ad0 0x560780458740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.088+0000 7f5ac7fff640 1 -- 172.21.15.33:0/1268059903 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607804478f0 msgr2=0x56078045ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.088+0000 7f5ac7fff640 1 --2- 172.21.15.33:0/1268059903 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607804478f0 0x56078045ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.088+0000 7f5ac7fff640 1 -- 172.21.15.33:0/1268059903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560780452ad0 msgr2=0x560780458740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.088+0000 7f5ac7fff640 1 --2- 172.21.15.33:0/1268059903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560780452ad0 0x560780458740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.089+0000 7f5ac7fff640 1 -- 172.21.15.33:0/1268059903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560780458e30 con 0x560780448210 2024-05-15T14:01:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.089+0000 7f5acd12d640 1 --2- 172.21.15.33:0/1268059903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560780452ad0 0x560780458740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:01:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.089+0000 7f5ac7fff640 1 --2- 172.21.15.33:0/1268059903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560780448210 0x560780458200 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4011fc0 tx=0x7f5ab401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14dd13ef291b6723 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.089+0000 7f5aa7fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:01:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.089+0000 7f5aa7fff640 1 -- 172.21.15.33:0/1268059903 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab402c400 con 0x560780448210 2024-05-15T14:01:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.089+0000 7f5aa7fff640 1 -- 172.21.15.33:0/1268059903 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f5ab402c5a0 con 0x560780448210 2024-05-15T14:01:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.089+0000 7f5ad1985740 1 -- 172.21.15.33:0/1268059903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5abc004400 con 0x560780448210 2024-05-15T14:01:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.089+0000 7f5aa7fff640 1 -- 172.21.15.33:0/1268059903 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5ab403a050 con 0x560780448210 2024-05-15T14:01:43.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.089+0000 7f5ad1985740 10 client.15516 fetch_fsmap learned FSMap version 24 2024-05-15T14:01:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.089+0000 7f5ad1985740 1 -- 172.21.15.33:0/1268059903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5607804cc8b0 con 0x560780448210 2024-05-15T14:01:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.090+0000 7f5aa7fff640 1 -- 172.21.15.33:0/1268059903 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5abc004400 con 0x560780448210 2024-05-15T14:01:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.090+0000 7f5aa7fff640 1 -- 172.21.15.33:0/1268059903 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f5ab401f070 con 0x560780448210 2024-05-15T14:01:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.090+0000 7f5ad1985740 10 client.15516 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:01:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.090+0000 7f5ad1985740 10 client.15516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:01:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.090+0000 7f5ad1985740 20 client.15516 populate_metadata read hostname 'smithi033' 2024-05-15T14:01:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.090+0000 7f5ad1985740 1 --2- 172.21.15.33:0/1268059903 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5607804623f0 0x5607804cf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.090+0000 7f5ad1985740 4 client.15516 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:01:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.090+0000 7f5ad1985740 1 -- 172.21.15.33:0/1268059903 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5607803b6360 con 0x5607804623f0 2024-05-15T14:01:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.090+0000 7f5acc92c640 1 --2- 172.21.15.33:0/1268059903 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5607804623f0 0x5607804cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.090+0000 7f5aa7fff640 10 client.15516 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:01:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.091+0000 7f5acc92c640 1 --2- 172.21.15.33:0/1268059903 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5607804623f0 0x5607804cf120 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95476fbbb4d5e503 server_cookie=c59c6cbf2ae08766 in_seq=0 out_seq=0 2024-05-15T14:01:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.094+0000 7f5aa7fff640 1 -- 172.21.15.33:0/1268059903 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5607803b6360 con 0x5607804623f0 2024-05-15T14:01:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.094+0000 7f5aa7fff640 10 client.15516 handle_command_reply: tid=0 2024-05-15T14:01:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.094+0000 7f5ad1985740 1 client.15516 shutdown 2024-05-15T14:01:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.095+0000 7f5ad1985740 1 -- 172.21.15.33:0/1268059903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560780448210 msgr2=0x560780458200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:43.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.095+0000 7f5ad1985740 1 --2- 172.21.15.33:0/1268059903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560780448210 0x560780458200 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4011fc0 tx=0x7f5ab401b000 comp rx=0 tx=0).stop 2024-05-15T14:01:43.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.095+0000 7f5ad1985740 1 -- 172.21.15.33:0/1268059903 shutdown_connections 2024-05-15T14:01:43.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.095+0000 7f5ad1985740 1 --2- 172.21.15.33:0/1268059903 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5607804623f0 0x5607804cf120 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.095+0000 7f5ad1985740 1 --2- 172.21.15.33:0/1268059903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560780452ad0 0x560780458740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.095+0000 7f5aa7fff640 0 client.15516 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:01:43.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.095+0000 7f5ad1985740 1 --2- 172.21.15.33:0/1268059903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560780448210 0x560780458200 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.095+0000 7f5ad1985740 1 --2- 172.21.15.33:0/1268059903 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5607804478f0 0x56078045ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.095+0000 7f5ad1985740 1 -- 172.21.15.33:0/1268059903 >> 172.21.15.33:0/1268059903 conn(0x560780443e00 msgr2=0x560780451860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:43.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.095+0000 7f5ad1985740 1 -- 172.21.15.33:0/1268059903 shutdown_connections 2024-05-15T14:01:43.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.095+0000 7f5ad1985740 1 -- 172.21.15.33:0/1268059903 wait complete. 2024-05-15T14:01:43.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.095+0000 7f5ad1985740 20 client.15516 trim_cache size 0 max 16384 2024-05-15T14:01:43.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:01:43.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:01:43.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:01:43.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.096+0000 7f5aceb90640 1 -- 172.21.15.33:0/1918202370 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f5ab80b2a70 msgr2=0x7f5ab80b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.096+0000 7f5aceb90640 1 --2- 172.21.15.33:0/1918202370 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f5ab80b2a70 0x7f5ab80b4f30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8079250 tx=0x7f5ab40098a0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.096+0000 7f5aceb90640 1 -- 172.21.15.33:0/1918202370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac8107940 msgr2=0x7f5ac819db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.096+0000 7f5aceb90640 1 --2- 172.21.15.33:0/1918202370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac8107940 0x7f5ac819db40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000eaa0 tx=0x7f5ac000ef60 comp rx=0 tx=0).stop 2024-05-15T14:01:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.096+0000 7f5aceb90640 1 -- 172.21.15.33:0/1918202370 shutdown_connections 2024-05-15T14:01:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.096+0000 7f5aceb90640 1 --2- 172.21.15.33:0/1918202370 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f5ab80b2a70 0x7f5ab80b4f30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.096+0000 7f5aceb90640 1 --2- 172.21.15.33:0/1918202370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac8107940 0x7f5ac819db40 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.096+0000 7f5aceb90640 1 --2- 172.21.15.33:0/1918202370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ac81022f0 0x7f5ac81997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.096+0000 7f5aceb90640 1 --2- 172.21.15.33:0/1918202370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ac8101930 0x7f5ac8199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.096+0000 7f5aceb90640 1 -- 172.21.15.33:0/1918202370 >> 172.21.15.33:0/1918202370 conn(0x7f5ac8078a70 msgr2=0x7f5ac806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.096+0000 7f5aceb90640 1 -- 172.21.15.33:0/1918202370 shutdown_connections 2024-05-15T14:01:43.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:43.096+0000 7f5aceb90640 1 -- 172.21.15.33:0/1918202370 wait complete. 2024-05-15T14:01:43.922 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'e8d9ebc6', quiesce took 6.0 seconds. Will release after: 18.0 2024-05-15T14:01:44.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:44 smithi052 ceph-mon[36190]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-15T14:01:44.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:44 smithi081 ceph-mon[36434]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-15T14:01:44.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:44 smithi033 ceph-mon[31650]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-15T14:01:46.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:46 smithi052 ceph-mon[36190]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T14:01:46.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:46 smithi081 ceph-mon[36434]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T14:01:46.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:46 smithi033 ceph-mon[31650]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T14:01:48.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:48 smithi052 ceph-mon[36190]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-15T14:01:48.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:48 smithi081 ceph-mon[36434]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-15T14:01:48.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:48 smithi033 ceph-mon[31650]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-15T14:01:50.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:50 smithi052 ceph-mon[36190]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-15T14:01:50.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:50 smithi081 ceph-mon[36434]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-15T14:01:50.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:50 smithi033 ceph-mon[31650]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-15T14:01:52.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:52 smithi052 ceph-mon[36190]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-15T14:01:52.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:52 smithi081 ceph-mon[36434]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-15T14:01:52.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:52 smithi033 ceph-mon[31650]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-15T14:01:54.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:54 smithi052 ceph-mon[36190]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-15T14:01:54.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:54 smithi052 ceph-mon[36190]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-05-15T14:01:54.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:54 smithi081 ceph-mon[36434]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-15T14:01:54.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:54 smithi081 ceph-mon[36434]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-05-15T14:01:54.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:54 smithi033 ceph-mon[31650]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-15T14:01:54.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:54 smithi033 ceph-mon[31650]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-05-15T14:01:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.194+0000 7f3b0674d640 1 -- 172.21.15.33:0/3821504367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b00107940 msgr2=0x7f3b00107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.194+0000 7f3b0674d640 1 --2- 172.21.15.33:0/3821504367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b00107940 0x7f3b00107d20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3af0009930 tx=0x7f3af002a420 comp rx=0 tx=0).stop 2024-05-15T14:01:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.194+0000 7f3b0674d640 1 -- 172.21.15.33:0/3821504367 shutdown_connections 2024-05-15T14:01:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.194+0000 7f3b0674d640 1 --2- 172.21.15.33:0/3821504367 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b001022f0 0x7f3b0010e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.194+0000 7f3b0674d640 1 --2- 172.21.15.33:0/3821504367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b00101930 0x7f3b00101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.194+0000 7f3b0674d640 1 --2- 172.21.15.33:0/3821504367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b00107940 0x7f3b00107d20 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.194+0000 7f3b0674d640 1 -- 172.21.15.33:0/3821504367 >> 172.21.15.33:0/3821504367 conn(0x7f3b00078a70 msgr2=0x7f3b00078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.194+0000 7f3b0674d640 1 -- 172.21.15.33:0/3821504367 shutdown_connections 2024-05-15T14:01:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.194+0000 7f3b0674d640 1 -- 172.21.15.33:0/3821504367 wait complete. 2024-05-15T14:01:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.195+0000 7f3b0674d640 1 Processor -- start 2024-05-15T14:01:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.195+0000 7f3b0674d640 1 -- start start 2024-05-15T14:01:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.195+0000 7f3b0674d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b00101930 0x7f3b00199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.195+0000 7f3b0674d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b001022f0 0x7f3b00199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.195+0000 7f3b0674d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b00107940 0x7f3b0019db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.195+0000 7f3b0674d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3b00111c00 con 0x7f3b00101930 2024-05-15T14:01:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.195+0000 7f3b0674d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3b00111a80 con 0x7f3b001022f0 2024-05-15T14:01:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.195+0000 7f3b0674d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3b00111d80 con 0x7f3b00107940 2024-05-15T14:01:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.195+0000 7f3afffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b00101930 0x7f3b00199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.195+0000 7f3afffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b00101930 0x7f3b00199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43632/0 (socket says 172.21.15.33:43632) 2024-05-15T14:01:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.195+0000 7f3afffff640 1 -- 172.21.15.33:0/445654930 learned_addr learned my addr 172.21.15.33:0/445654930 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.195+0000 7f3aff7fe640 1 --2- 172.21.15.33:0/445654930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b001022f0 0x7f3b00199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.196+0000 7f3b04cea640 1 --2- 172.21.15.33:0/445654930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b00107940 0x7f3b0019db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.196+0000 7f3afffff640 1 -- 172.21.15.33:0/445654930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b00107940 msgr2=0x7f3b0019db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.196+0000 7f3afffff640 1 --2- 172.21.15.33:0/445654930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b00107940 0x7f3b0019db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.196+0000 7f3afffff640 1 -- 172.21.15.33:0/445654930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b001022f0 msgr2=0x7f3b00199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.196+0000 7f3afffff640 1 --2- 172.21.15.33:0/445654930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b001022f0 0x7f3b00199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.196+0000 7f3afffff640 1 -- 172.21.15.33:0/445654930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b0019e230 con 0x7f3b00101930 2024-05-15T14:01:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.196+0000 7f3afffff640 1 --2- 172.21.15.33:0/445654930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b00101930 0x7f3b00199240 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3af0005940 tx=0x7f3af00040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.196+0000 7f3afd7fa640 1 -- 172.21.15.33:0/445654930 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af003f030 con 0x7f3b00101930 2024-05-15T14:01:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.196+0000 7f3b0674d640 1 -- 172.21.15.33:0/445654930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b0019e4c0 con 0x7f3b00101930 2024-05-15T14:01:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.196+0000 7f3afd7fa640 1 -- 172.21.15.33:0/445654930 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3af00044c0 con 0x7f3b00101930 2024-05-15T14:01:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.196+0000 7f3afd7fa640 1 -- 172.21.15.33:0/445654930 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af0035070 con 0x7f3b00101930 2024-05-15T14:01:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.196+0000 7f3b0674d640 1 -- 172.21.15.33:0/445654930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b001a5db0 con 0x7f3b00101930 2024-05-15T14:01:56.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.199+0000 7f3b09461740 1 Processor -- start 2024-05-15T14:01:56.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.199+0000 7f3b09461740 1 -- start start 2024-05-15T14:01:56.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.199+0000 7f3b09461740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55731d2498f0 0x55731d249cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:56.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.199+0000 7f3b09461740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55731d24a210 0x55731d254590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:56.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.199+0000 7f3b09461740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55731d254ad0 0x55731d256ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:56.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.199+0000 7f3b09461740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55731d1bcc30 con 0x55731d254ad0 2024-05-15T14:01:56.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.199+0000 7f3b09461740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55731d1ac7d0 con 0x55731d2498f0 2024-05-15T14:01:56.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.199+0000 7f3b09461740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55731d1b0810 con 0x55731d24a210 2024-05-15T14:01:56.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.200+0000 7f3aff7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55731d254ad0 0x55731d256ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:56.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.200+0000 7f3aff7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55731d254ad0 0x55731d256ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43640/0 (socket says 172.21.15.33:43640) 2024-05-15T14:01:56.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.200+0000 7f3aff7fe640 1 -- 172.21.15.33:0/3389173364 learned_addr learned my addr 172.21.15.33:0/3389173364 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:56.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.200+0000 7f3b04cea640 1 --2- 172.21.15.33:0/3389173364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55731d2498f0 0x55731d249cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:56.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.200+0000 7f3b04cea640 1 -- 172.21.15.33:0/3389173364 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55731d24a210 msgr2=0x55731d254590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-15T14:01:56.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.200+0000 7f3b04cea640 1 --2- 172.21.15.33:0/3389173364 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55731d24a210 0x55731d254590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:56.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.200+0000 7f3b04cea640 1 -- 172.21.15.33:0/3389173364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55731d254ad0 msgr2=0x55731d256ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:56.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.200+0000 7f3b04cea640 1 --2- 172.21.15.33:0/3389173364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55731d254ad0 0x55731d256ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:56.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.200+0000 7f3b04cea640 1 -- 172.21.15.33:0/3389173364 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55731cec1210 con 0x55731d2498f0 2024-05-15T14:01:56.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.200+0000 7f3aff7fe640 1 --2- 172.21.15.33:0/3389173364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55731d254ad0 0x55731d256ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:01:56.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.200+0000 7f3afd7fa640 1 -- 172.21.15.33:0/445654930 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3af0042030 con 0x7f3b00101930 2024-05-15T14:01:56.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.200+0000 7f3b04cea640 1 --2- 172.21.15.33:0/3389173364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55731d2498f0 0x55731d249cd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3af8009f00 tx=0x7f3af8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=204190055fa6d4d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:56.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.201+0000 7f3ae67fc640 1 -- 172.21.15.33:0/3389173364 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af8004d20 con 0x55731d2498f0 2024-05-15T14:01:56.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.201+0000 7f3ae67fc640 1 -- 172.21.15.33:0/3389173364 <== mon.2 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3af8004050 con 0x55731d2498f0 2024-05-15T14:01:56.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.201+0000 7f3ae67fc640 1 -- 172.21.15.33:0/3389173364 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af8005280 con 0x55731d2498f0 2024-05-15T14:01:56.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.201+0000 7f3afd7fa640 1 --2- 172.21.15.33:0/445654930 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f3ae80b2950 0x7f3ae80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:56.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.201+0000 7f3afffff640 1 --2- 172.21.15.33:0/445654930 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f3ae80b2950 0x7f3ae80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:56.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.201+0000 7f3afd7fa640 1 -- 172.21.15.33:0/445654930 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f3af0102920 con 0x7f3b00101930 2024-05-15T14:01:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.201+0000 7f3b09461740 1 -- 172.21.15.33:0/3389173364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55731d2498f0 msgr2=0x55731d249cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.201+0000 7f3b09461740 1 --2- 172.21.15.33:0/3389173364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55731d2498f0 0x55731d249cd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3af8009f00 tx=0x7f3af8011810 comp rx=0 tx=0).stop 2024-05-15T14:01:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.202+0000 7f3b09461740 1 -- 172.21.15.33:0/3389173364 shutdown_connections 2024-05-15T14:01:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.202+0000 7f3b09461740 1 --2- 172.21.15.33:0/3389173364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55731d254ad0 0x55731d256ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.202+0000 7f3b09461740 1 --2- 172.21.15.33:0/3389173364 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55731d24a210 0x55731d254590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.202+0000 7f3b09461740 1 --2- 172.21.15.33:0/3389173364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55731d2498f0 0x55731d249cd0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.202+0000 7f3b09461740 1 -- 172.21.15.33:0/3389173364 >> 172.21.15.33:0/3389173364 conn(0x55731d245e00 msgr2=0x55731d2461e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.202+0000 7f3b09461740 1 -- 172.21.15.33:0/3389173364 shutdown_connections 2024-05-15T14:01:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.202+0000 7f3afffff640 1 --2- 172.21.15.33:0/445654930 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f3ae80b2950 0x7f3ae80b4e10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55731d24ae90 tx=0x7f3af00be540 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:56.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.202+0000 7f3b09461740 1 -- 172.21.15.33:0/3389173364 wait complete. 2024-05-15T14:01:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.202+0000 7f3b09461740 1 -- start start 2024-05-15T14:01:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.203+0000 7f3b09461740 1 Processor -- start 2024-05-15T14:01:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.203+0000 7f3b09461740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55731d2498f0 0x55731d254440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.203+0000 7f3b09461740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55731d24a210 0x55731d251730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.203+0000 7f3b09461740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55731d254ad0 0x55731d251c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.203+0000 7f3b09461740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55731ce92790 con 0x55731d254ad0 2024-05-15T14:01:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.203+0000 7f3b09461740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55731d168890 con 0x55731d24a210 2024-05-15T14:01:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.203+0000 7f3b09461740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55731d25bbb0 con 0x55731d2498f0 2024-05-15T14:01:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.203+0000 7f3b09461740 10 client.0 _ll_register_callbacks cb 0x55731d16bd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:01:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.203+0000 7f3b04cea640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55731d254ad0 0x55731d251c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.203+0000 7f3b04cea640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55731d254ad0 0x55731d251c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43650/0 (socket says 172.21.15.33:43650) 2024-05-15T14:01:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.203+0000 7f3b04cea640 1 -- 172.21.15.33:0/887401792 learned_addr learned my addr 172.21.15.33:0/887401792 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.203+0000 7f3b04cea640 1 --2- 172.21.15.33:0/887401792 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55731d2498f0 0x55731d254440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.203+0000 7f3aff7fe640 1 --2- 172.21.15.33:0/887401792 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55731d24a210 0x55731d251730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.203+0000 7f3b04cea640 1 -- 172.21.15.33:0/887401792 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55731d2498f0 msgr2=0x55731d254440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.203+0000 7f3b04cea640 1 --2- 172.21.15.33:0/887401792 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55731d2498f0 0x55731d254440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.203+0000 7f3b04cea640 1 -- 172.21.15.33:0/887401792 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55731d24a210 msgr2=0x55731d251730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.203+0000 7f3b04cea640 1 --2- 172.21.15.33:0/887401792 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55731d24a210 0x55731d251730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.203+0000 7f3b04cea640 1 -- 172.21.15.33:0/887401792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55731cf8d620 con 0x55731d254ad0 2024-05-15T14:01:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.204+0000 7f3b04cea640 1 --2- 172.21.15.33:0/887401792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55731d254ad0 0x55731d251c90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3af8003eb0 tx=0x7f3af8003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e60e0869ae6bf4f server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.204+0000 7f3adf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:01:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.204+0000 7f3adf7fe640 1 -- 172.21.15.33:0/887401792 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af802d060 con 0x55731d254ad0 2024-05-15T14:01:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.204+0000 7f3b09461740 1 -- 172.21.15.33:0/887401792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3af8004050 con 0x55731d254ad0 2024-05-15T14:01:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.204+0000 7f3adf7fe640 1 -- 172.21.15.33:0/887401792 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f3af8004a40 con 0x55731d254ad0 2024-05-15T14:01:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.204+0000 7f3adf7fe640 1 -- 172.21.15.33:0/887401792 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3af8004c40 con 0x55731d254ad0 2024-05-15T14:01:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.204+0000 7f3b09461740 10 client.15528 fetch_fsmap learned FSMap version 24 2024-05-15T14:01:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.204+0000 7f3b09461740 1 -- 172.21.15.33:0/887401792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55731d267610 con 0x55731d254ad0 2024-05-15T14:01:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.204+0000 7f3adf7fe640 1 -- 172.21.15.33:0/887401792 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3af8004050 con 0x55731d254ad0 2024-05-15T14:01:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.205+0000 7f3adf7fe640 1 -- 172.21.15.33:0/887401792 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f3af8026a00 con 0x55731d254ad0 2024-05-15T14:01:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.205+0000 7f3b09461740 10 client.15528 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:01:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.205+0000 7f3b09461740 10 client.15528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:01:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.205+0000 7f3b09461740 20 client.15528 populate_metadata read hostname 'smithi033' 2024-05-15T14:01:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.205+0000 7f3b09461740 1 --2- 172.21.15.33:0/887401792 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55731d268e90 0x55731d26b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.205+0000 7f3b09461740 4 client.15528 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:01:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.205+0000 7f3b09461740 1 -- 172.21.15.33:0/887401792 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55731d1beb80 con 0x55731d268e90 2024-05-15T14:01:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.206+0000 7f3aff7fe640 1 --2- 172.21.15.33:0/887401792 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55731d268e90 0x55731d26b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.206+0000 7f3aff7fe640 1 --2- 172.21.15.33:0/887401792 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55731d268e90 0x55731d26b2d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30f6ce63f2f58a6e server_cookie=f5fb3279a5749212 in_seq=0 out_seq=0 2024-05-15T14:01:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.206+0000 7f3adf7fe640 10 client.15528 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:01:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.207+0000 7f3adf7fe640 1 -- 172.21.15.33:0/887401792 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55731d1beb80 con 0x55731d268e90 2024-05-15T14:01:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.207+0000 7f3adf7fe640 10 client.15528 handle_command_reply: tid=0 2024-05-15T14:01:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.207+0000 7f3b09461740 1 client.15528 shutdown 2024-05-15T14:01:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.208+0000 7f3b09461740 1 -- 172.21.15.33:0/887401792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55731d254ad0 msgr2=0x55731d251c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.208+0000 7f3b09461740 1 --2- 172.21.15.33:0/887401792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55731d254ad0 0x55731d251c90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f3af8003eb0 tx=0x7f3af8003f90 comp rx=0 tx=0).stop 2024-05-15T14:01:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.208+0000 7f3b09461740 1 -- 172.21.15.33:0/887401792 shutdown_connections 2024-05-15T14:01:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.208+0000 7f3b09461740 1 --2- 172.21.15.33:0/887401792 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55731d268e90 0x55731d26b2d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.208+0000 7f3b09461740 1 --2- 172.21.15.33:0/887401792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55731d254ad0 0x55731d251c90 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.208+0000 7f3b09461740 1 --2- 172.21.15.33:0/887401792 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55731d24a210 0x55731d251730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.208+0000 7f3b09461740 1 --2- 172.21.15.33:0/887401792 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55731d2498f0 0x55731d254440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.208+0000 7f3b09461740 1 -- 172.21.15.33:0/887401792 >> 172.21.15.33:0/887401792 conn(0x55731d245e00 msgr2=0x55731d245100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.208+0000 7f3adf7fe640 0 client.15528 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:01:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.208+0000 7f3b09461740 1 -- 172.21.15.33:0/887401792 shutdown_connections 2024-05-15T14:01:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.208+0000 7f3b09461740 1 -- 172.21.15.33:0/887401792 wait complete. 2024-05-15T14:01:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.208+0000 7f3b09461740 20 client.15528 trim_cache size 0 max 16384 2024-05-15T14:01:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.238+0000 7f3b09461740 1 Processor -- start 2024-05-15T14:01:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.238+0000 7f3b09461740 1 -- start start 2024-05-15T14:01:56.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.238+0000 7f3b09461740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55731d2498f0 0x55731d250890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:56.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.238+0000 7f3b09461740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55731d24a210 0x55731d264a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:56.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.238+0000 7f3b09461740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55731d254ad0 0x55731d24b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:56.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.238+0000 7f3b09461740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3b001aebd0 con 0x55731d2498f0 2024-05-15T14:01:56.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.238+0000 7f3b09461740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3b001ac4a0 con 0x55731d24a210 2024-05-15T14:01:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.238+0000 7f3b09461740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3b001aa3a0 con 0x55731d254ad0 2024-05-15T14:01:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.239+0000 7f3b04cea640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55731d2498f0 0x55731d250890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.239+0000 7f3b04cea640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55731d2498f0 0x55731d250890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43664/0 (socket says 172.21.15.33:43664) 2024-05-15T14:01:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.239+0000 7f3b04cea640 1 -- 172.21.15.33:0/965077480 learned_addr learned my addr 172.21.15.33:0/965077480 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.239+0000 7f3aff7fe640 1 --2- 172.21.15.33:0/965077480 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55731d24a210 0x55731d264a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.239+0000 7f3b04cea640 1 -- 172.21.15.33:0/965077480 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55731d254ad0 msgr2=0x55731d24b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.239+0000 7f3b04cea640 1 --2- 172.21.15.33:0/965077480 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55731d254ad0 0x55731d24b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.239+0000 7f3b04cea640 1 -- 172.21.15.33:0/965077480 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55731d24a210 msgr2=0x55731d264a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.239+0000 7f3b04cea640 1 --2- 172.21.15.33:0/965077480 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55731d24a210 0x55731d264a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.239+0000 7f3b04cea640 1 -- 172.21.15.33:0/965077480 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55731d264fc0 con 0x55731d2498f0 2024-05-15T14:01:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.239+0000 7f3aff7fe640 1 --2- 172.21.15.33:0/965077480 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55731d24a210 0x55731d264a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:01:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.239+0000 7f3b04cea640 1 --2- 172.21.15.33:0/965077480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55731d2498f0 0x55731d250890 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3af8009730 tx=0x7f3af8009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebbbc27fb8d13b4b server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.239+0000 7f3ae5ffb640 1 -- 172.21.15.33:0/965077480 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af802d070 con 0x55731d2498f0 2024-05-15T14:01:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.239+0000 7f3ae5ffb640 1 -- 172.21.15.33:0/965077480 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3af8028450 con 0x55731d2498f0 2024-05-15T14:01:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.239+0000 7f3ae5ffb640 1 -- 172.21.15.33:0/965077480 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af8004a50 con 0x55731d2498f0 2024-05-15T14:01:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.240+0000 7f3b09461740 1 -- 172.21.15.33:0/965077480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55731d2498f0 msgr2=0x55731d250890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.240+0000 7f3b09461740 1 --2- 172.21.15.33:0/965077480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55731d2498f0 0x55731d250890 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3af8009730 tx=0x7f3af8009500 comp rx=0 tx=0).stop 2024-05-15T14:01:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.240+0000 7f3b09461740 1 -- 172.21.15.33:0/965077480 shutdown_connections 2024-05-15T14:01:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.240+0000 7f3b09461740 1 --2- 172.21.15.33:0/965077480 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55731d254ad0 0x55731d24b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.240+0000 7f3b09461740 1 --2- 172.21.15.33:0/965077480 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55731d24a210 0x55731d264a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.240+0000 7f3b09461740 1 --2- 172.21.15.33:0/965077480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55731d2498f0 0x55731d250890 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.240+0000 7f3b09461740 1 -- 172.21.15.33:0/965077480 >> 172.21.15.33:0/965077480 conn(0x55731d245e00 msgr2=0x55731d24b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.241+0000 7f3b09461740 1 -- 172.21.15.33:0/965077480 shutdown_connections 2024-05-15T14:01:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.241+0000 7f3b09461740 1 -- 172.21.15.33:0/965077480 wait complete. 2024-05-15T14:01:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.241+0000 7f3b09461740 1 -- start start 2024-05-15T14:01:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.241+0000 7f3b09461740 1 Processor -- start 2024-05-15T14:01:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.242+0000 7f3b09461740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55731d2498f0 0x55731d261280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.242+0000 7f3b09461740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55731d24a210 0x55731d25bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.242+0000 7f3b09461740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55731d254ad0 0x55731d25c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.242+0000 7f3b04cea640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55731d2498f0 0x55731d261280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.242+0000 7f3b04cea640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55731d2498f0 0x55731d261280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43680/0 (socket says 172.21.15.33:43680) 2024-05-15T14:01:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.242+0000 7f3b04cea640 1 -- 172.21.15.33:0/4046913411 learned_addr learned my addr 172.21.15.33:0/4046913411 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:01:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.242+0000 7f3b09461740 1 -- 172.21.15.33:0/4046913411 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3b001aa960 con 0x55731d2498f0 2024-05-15T14:01:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.242+0000 7f3b09461740 1 -- 172.21.15.33:0/4046913411 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3b001abdd0 con 0x55731d254ad0 2024-05-15T14:01:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.242+0000 7f3b09461740 1 -- 172.21.15.33:0/4046913411 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3b001ab030 con 0x55731d24a210 2024-05-15T14:01:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.242+0000 7f3aff7fe640 1 --2- 172.21.15.33:0/4046913411 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55731d24a210 0x55731d25bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:56.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.242+0000 7f3b04cea640 1 -- 172.21.15.33:0/4046913411 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55731d24a210 msgr2=0x55731d25bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:56.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.242+0000 7f3b04cea640 1 --2- 172.21.15.33:0/4046913411 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55731d24a210 0x55731d25bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:56.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.242+0000 7f3b04cea640 1 -- 172.21.15.33:0/4046913411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55731d254ad0 msgr2=0x55731d25c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:01:56.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.242+0000 7f3b04cea640 1 --2- 172.21.15.33:0/4046913411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55731d254ad0 0x55731d25c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:56.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.242+0000 7f3b04cea640 1 -- 172.21.15.33:0/4046913411 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55731d25c9f0 con 0x55731d2498f0 2024-05-15T14:01:56.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.242+0000 7f3b04cea640 1 --2- 172.21.15.33:0/4046913411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55731d2498f0 0x55731d261280 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3af8026dd0 tx=0x7f3af8028670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d22a3f905ec201a server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:01:56.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.243+0000 7f3adf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:01:56.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.243+0000 7f3adf7fe640 1 -- 172.21.15.33:0/4046913411 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af803e070 con 0x55731d2498f0 2024-05-15T14:01:56.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.243+0000 7f3adf7fe640 1 -- 172.21.15.33:0/4046913411 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f3af80047e0 con 0x55731d2498f0 2024-05-15T14:01:56.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.243+0000 7f3adf7fe640 10 client.0 inactive, discarding osd_map(83..83 src has 1..83) 2024-05-15T14:01:56.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.243+0000 7f3b09461740 10 client.0 _ll_register_callbacks cb 0x7f3af8004290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:01:56.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.243+0000 7f3b09461740 1 -- 172.21.15.33:0/4046913411 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3af8028450 con 0x55731d2498f0 2024-05-15T14:01:56.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.243+0000 7f3adf7fe640 1 -- 172.21.15.33:0/4046913411 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3af8048050 con 0x55731d2498f0 2024-05-15T14:01:56.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.244+0000 7f3b09461740 10 client.15534 fetch_fsmap learned FSMap version 24 2024-05-15T14:01:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.244+0000 7f3b09461740 1 -- 172.21.15.33:0/4046913411 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({fsmap=24}) -- 0x55731d2d2270 con 0x55731d2498f0 2024-05-15T14:01:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.244+0000 7f3adf7fe640 1 -- 172.21.15.33:0/4046913411 <== mon.0 v2:172.21.15.33:3300/0 4 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f3af8039070 con 0x55731d2498f0 2024-05-15T14:01:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.244+0000 7f3b09461740 10 client.15534 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:01:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.244+0000 7f3b09461740 10 client.15534 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:01:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.244+0000 7f3b09461740 20 client.15534 populate_metadata read hostname 'smithi033' 2024-05-15T14:01:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.244+0000 7f3b09461740 1 --2- 172.21.15.33:0/4046913411 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55731d2d2910 0x55731d2d4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:01:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.244+0000 7f3b09461740 4 client.15534 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:01:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.244+0000 7f3b09461740 1 -- 172.21.15.33:0/4046913411 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55731d1b8360 con 0x55731d2d2910 2024-05-15T14:01:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.244+0000 7f3aff7fe640 1 --2- 172.21.15.33:0/4046913411 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55731d2d2910 0x55731d2d4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:01:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.245+0000 7f3adf7fe640 10 client.15534 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:01:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.245+0000 7f3aff7fe640 1 --2- 172.21.15.33:0/4046913411 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55731d2d2910 0x55731d2d4dd0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3132d5d73c4a9fe server_cookie=3a8180d2ae5bf54e in_seq=0 out_seq=0 2024-05-15T14:01:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.250+0000 7f3adf7fe640 1 -- 172.21.15.33:0/4046913411 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55731d1b8360 con 0x55731d2d2910 2024-05-15T14:01:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.250+0000 7f3adf7fe640 10 client.15534 handle_command_reply: tid=0 2024-05-15T14:01:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.250+0000 7f3b09461740 1 client.15534 shutdown 2024-05-15T14:01:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.251+0000 7f3b09461740 1 -- 172.21.15.33:0/4046913411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55731d2498f0 msgr2=0x55731d261280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.251+0000 7f3b09461740 1 --2- 172.21.15.33:0/4046913411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55731d2498f0 0x55731d261280 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3af8026dd0 tx=0x7f3af8028670 comp rx=0 tx=0).stop 2024-05-15T14:01:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.251+0000 7f3b09461740 1 -- 172.21.15.33:0/4046913411 shutdown_connections 2024-05-15T14:01:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.251+0000 7f3b09461740 1 --2- 172.21.15.33:0/4046913411 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55731d2d2910 0x55731d2d4dd0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.251+0000 7f3b09461740 1 --2- 172.21.15.33:0/4046913411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55731d254ad0 0x55731d25c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.251+0000 7f3b09461740 1 --2- 172.21.15.33:0/4046913411 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55731d24a210 0x55731d25bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.251+0000 7f3b09461740 1 --2- 172.21.15.33:0/4046913411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55731d2498f0 0x55731d261280 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.251+0000 7f3b09461740 1 -- 172.21.15.33:0/4046913411 >> 172.21.15.33:0/4046913411 conn(0x55731d245e00 msgr2=0x55731d24bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.251+0000 7f3adf7fe640 0 client.15534 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:01:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.251+0000 7f3b09461740 1 -- 172.21.15.33:0/4046913411 shutdown_connections 2024-05-15T14:01:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.251+0000 7f3b09461740 1 -- 172.21.15.33:0/4046913411 wait complete. 2024-05-15T14:01:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.251+0000 7f3b09461740 20 client.15534 trim_cache size 0 max 16384 2024-05-15T14:01:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:01:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:01:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:01:56.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.252+0000 7f3b0674d640 1 -- 172.21.15.33:0/445654930 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f3ae80b2950 msgr2=0x7f3ae80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:56.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.252+0000 7f3b0674d640 1 --2- 172.21.15.33:0/445654930 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f3ae80b2950 0x7f3ae80b4e10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55731d24ae90 tx=0x7f3af00be540 comp rx=0 tx=0).stop 2024-05-15T14:01:56.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.252+0000 7f3b0674d640 1 -- 172.21.15.33:0/445654930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b00101930 msgr2=0x7f3b00199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:01:56.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.252+0000 7f3b0674d640 1 --2- 172.21.15.33:0/445654930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b00101930 0x7f3b00199240 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3af0005940 tx=0x7f3af00040c0 comp rx=0 tx=0).stop 2024-05-15T14:01:56.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.252+0000 7f3b0674d640 1 -- 172.21.15.33:0/445654930 shutdown_connections 2024-05-15T14:01:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.252+0000 7f3b0674d640 1 --2- 172.21.15.33:0/445654930 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f3ae80b2950 0x7f3ae80b4e10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.252+0000 7f3b0674d640 1 --2- 172.21.15.33:0/445654930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b00107940 0x7f3b0019db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.252+0000 7f3b0674d640 1 --2- 172.21.15.33:0/445654930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b001022f0 0x7f3b00199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.252+0000 7f3b0674d640 1 --2- 172.21.15.33:0/445654930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b00101930 0x7f3b00199240 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:01:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.252+0000 7f3b0674d640 1 -- 172.21.15.33:0/445654930 >> 172.21.15.33:0/445654930 conn(0x7f3b00078a70 msgr2=0x7f3b0006a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:01:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.252+0000 7f3b0674d640 1 -- 172.21.15.33:0/445654930 shutdown_connections 2024-05-15T14:01:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:01:56.252+0000 7f3b0674d640 1 -- 172.21.15.33:0/445654930 wait complete. 2024-05-15T14:01:56.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:56 smithi052 ceph-mon[36190]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-15T14:01:56.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:56 smithi081 ceph-mon[36434]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-15T14:01:56.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:56 smithi033 ceph-mon[31650]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-15T14:01:58.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:01:58 smithi052 ceph-mon[36190]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-05-15T14:01:58.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:01:58 smithi081 ceph-mon[36434]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-05-15T14:01:58.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:01:58 smithi033 ceph-mon[31650]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-05-15T14:02:00.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:00 smithi052 ceph-mon[36190]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-15T14:02:00.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:00 smithi081 ceph-mon[36434]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-15T14:02:00.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:00 smithi033 ceph-mon[31650]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-15T14:02:01.923 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T14:02:02.313 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:02:02.721 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:02 smithi033 ceph-mon[31650]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-15T14:02:02.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:02 smithi052 ceph-mon[36190]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-15T14:02:02.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:02 smithi081 ceph-mon[36434]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-15T14:02:03.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:03 smithi033 ceph-mon[31650]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-15T14:02:03.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:03 smithi052 ceph-mon[36190]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-15T14:02:03.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:03 smithi081 ceph-mon[36434]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-15T14:02:04.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.042+0000 7fd5dbad2700 1 -- 172.21.15.33:0/1592749114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5d4102e00 msgr2=0x7fd5d410f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:04.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.042+0000 7fd5dbad2700 1 --2- 172.21.15.33:0/1592749114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5d4102e00 0x7fd5d410f2f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc00b0a0 tx=0x7fd5cc00b3b0 comp rx=0 tx=0).stop 2024-05-15T14:02:04.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.043+0000 7fd5dbad2700 1 -- 172.21.15.33:0/1592749114 shutdown_connections 2024-05-15T14:02:04.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.043+0000 7fd5dbad2700 1 --2- 172.21.15.33:0/1592749114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5d4102e00 0x7fd5d410f2f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:04.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.043+0000 7fd5dbad2700 1 --2- 172.21.15.33:0/1592749114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd5d4108520 0x7fd5d4108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:04.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.043+0000 7fd5dbad2700 1 --2- 172.21.15.33:0/1592749114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5d4102440 0x7fd5d41028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:04.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.043+0000 7fd5dbad2700 1 -- 172.21.15.33:0/1592749114 >> 172.21.15.33:0/1592749114 conn(0x7fd5d40793f0 msgr2=0x7fd5d4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:02:04.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.043+0000 7fd5dbad2700 1 -- 172.21.15.33:0/1592749114 shutdown_connections 2024-05-15T14:02:04.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.044+0000 7fd5dbad2700 1 -- 172.21.15.33:0/1592749114 wait complete. 2024-05-15T14:02:04.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.044+0000 7fd5dbad2700 1 Processor -- start 2024-05-15T14:02:04.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.044+0000 7fd5dbad2700 1 -- start start 2024-05-15T14:02:04.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.045+0000 7fd5dbad2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd5d4102440 0x7fd5d410fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:04.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.045+0000 7fd5dbad2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5d4102e00 0x7fd5d410ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:04.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.045+0000 7fd5d906d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5d4102e00 0x7fd5d410ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:04.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.045+0000 7fd5d906d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5d4102e00 0x7fd5d410ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57034/0 (socket says 172.21.15.33:57034) 2024-05-15T14:02:04.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.045+0000 7fd5d986e700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd5d4102440 0x7fd5d410fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:04.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.045+0000 7fd5d906d700 1 -- 172.21.15.33:0/2465665927 learned_addr learned my addr 172.21.15.33:0/2465665927 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:02:04.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.045+0000 7fd5dbad2700 1 --2- 172.21.15.33:0/2465665927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5d4108520 0x7fd5d4110500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:04.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.045+0000 7fd5dbad2700 1 -- 172.21.15.33:0/2465665927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d4113790 con 0x7fd5d4102e00 2024-05-15T14:02:04.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.045+0000 7fd5dbad2700 1 -- 172.21.15.33:0/2465665927 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d4113610 con 0x7fd5d4108520 2024-05-15T14:02:04.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.045+0000 7fd5dbad2700 1 -- 172.21.15.33:0/2465665927 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d4113910 con 0x7fd5d4102440 2024-05-15T14:02:04.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.045+0000 7fd5da06f700 1 --2- 172.21.15.33:0/2465665927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5d4108520 0x7fd5d4110500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:04.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.046+0000 7fd5d906d700 1 -- 172.21.15.33:0/2465665927 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd5d4102440 msgr2=0x7fd5d410fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:04.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.046+0000 7fd5d906d700 1 --2- 172.21.15.33:0/2465665927 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd5d4102440 0x7fd5d410fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:04.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.046+0000 7fd5d906d700 1 -- 172.21.15.33:0/2465665927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5d4108520 msgr2=0x7fd5d4110500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:04.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.046+0000 7fd5d906d700 1 --2- 172.21.15.33:0/2465665927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5d4108520 0x7fd5d4110500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:04.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.046+0000 7fd5d906d700 1 -- 172.21.15.33:0/2465665927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d41a80d0 con 0x7fd5d4102e00 2024-05-15T14:02:04.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.046+0000 7fd5d986e700 1 --2- 172.21.15.33:0/2465665927 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd5d4102440 0x7fd5d410fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:02:04.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.046+0000 7fd5d906d700 1 --2- 172.21.15.33:0/2465665927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5d4102e00 0x7fd5d410ffc0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400ed60 tx=0x7fd5c400c5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:02:04.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.046+0000 7fd5caffd700 1 -- 172.21.15.33:0/2465665927 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c400cbc0 con 0x7fd5d4102e00 2024-05-15T14:02:04.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.046+0000 7fd5caffd700 1 -- 172.21.15.33:0/2465665927 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5c400cd60 con 0x7fd5d4102e00 2024-05-15T14:02:04.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.046+0000 7fd5dbad2700 1 -- 172.21.15.33:0/2465665927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5d41a83c0 con 0x7fd5d4102e00 2024-05-15T14:02:04.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.046+0000 7fd5caffd700 1 -- 172.21.15.33:0/2465665927 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4018710 con 0x7fd5d4102e00 2024-05-15T14:02:04.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.046+0000 7fd5dbad2700 1 -- 172.21.15.33:0/2465665927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5d41a8950 con 0x7fd5d4102e00 2024-05-15T14:02:04.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.048+0000 7fd5dbad2700 1 -- 172.21.15.33:0/2465665927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5d410a9b0 con 0x7fd5d4102e00 2024-05-15T14:02:04.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.051+0000 7fd5caffd700 1 -- 172.21.15.33:0/2465665927 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd5c4010430 con 0x7fd5d4102e00 2024-05-15T14:02:04.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.052+0000 7fd5caffd700 1 --2- 172.21.15.33:0/2465665927 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fd5c00b2a70 0x7fd5c00b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:04.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.052+0000 7fd5caffd700 1 -- 172.21.15.33:0/2465665927 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fd5c40e28d0 con 0x7fd5d4102e00 2024-05-15T14:02:04.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.052+0000 7fd5d986e700 1 --2- 172.21.15.33:0/2465665927 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fd5c00b2a70 0x7fd5c00b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:04.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.053+0000 7fd5d986e700 1 --2- 172.21.15.33:0/2465665927 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fd5c00b2a70 0x7fd5c00b4f30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd5d00103d0 tx=0x7fd5d000b3c0 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:02:04.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.058+0000 7fd5caffd700 1 -- 172.21.15.33:0/2465665927 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd5c40aa2c0 con 0x7fd5d4102e00 2024-05-15T14:02:04.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.581+0000 7fd5dbad2700 1 -- 172.21.15.33:0/2465665927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5d4110fb0 con 0x7fd5d4102e00 2024-05-15T14:02:04.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.582+0000 7fd5caffd700 1 -- 172.21.15.33:0/2465665927 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11858 (secure 0 0 0) 0x7fd5c40b2340 con 0x7fd5d4102e00 2024-05-15T14:02:04.583 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T14:02:04.584 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2030117633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2030117633},{"type":"v1","addr":"172.21.15.33:6839","nonce":2030117633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1918979061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1918979061},{"type":"v1","addr":"172.21.15.33:6841","nonce":1918979061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T13:57:19.056417+0000","modified":"2024-05-15T14:00:18.543619+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24463,"mds_2":24461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/1629697841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1629697841},{"type":"v1","addr":"172.21.15.52:6837","nonce":1629697841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/3773839758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3773839758},{"type":"v1","addr":"172.21.15.81:6835","nonce":3773839758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.52:6839/2696044471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2696044471},{"type":"v1","addr":"172.21.15.52:6839","nonce":2696044471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.81:6839/1554395940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":1554395940},{"type":"v1","addr":"172.21.15.81:6839","nonce":1554395940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24463,24478,24461]},"id":1}]} 2024-05-15T14:02:04.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.584+0000 7fd5dbad2700 1 -- 172.21.15.33:0/2465665927 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fd5c00b2a70 msgr2=0x7fd5c00b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:04.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.584+0000 7fd5dbad2700 1 --2- 172.21.15.33:0/2465665927 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fd5c00b2a70 0x7fd5c00b4f30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd5d00103d0 tx=0x7fd5d000b3c0 comp rx=0 tx=0).stop 2024-05-15T14:02:04.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.584+0000 7fd5dbad2700 1 -- 172.21.15.33:0/2465665927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5d4102e00 msgr2=0x7fd5d410ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:04.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.584+0000 7fd5dbad2700 1 --2- 172.21.15.33:0/2465665927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5d4102e00 0x7fd5d410ffc0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400ed60 tx=0x7fd5c400c5b0 comp rx=0 tx=0).stop 2024-05-15T14:02:04.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.585+0000 7fd5dbad2700 1 -- 172.21.15.33:0/2465665927 shutdown_connections 2024-05-15T14:02:04.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.585+0000 7fd5dbad2700 1 --2- 172.21.15.33:0/2465665927 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fd5c00b2a70 0x7fd5c00b4f30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:04.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.585+0000 7fd5dbad2700 1 --2- 172.21.15.33:0/2465665927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5d4108520 0x7fd5d4110500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:04.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.585+0000 7fd5dbad2700 1 --2- 172.21.15.33:0/2465665927 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd5d4102440 0x7fd5d410fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:04.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.585+0000 7fd5dbad2700 1 --2- 172.21.15.33:0/2465665927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5d4102e00 0x7fd5d410ffc0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:04.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.585+0000 7fd5dbad2700 1 -- 172.21.15.33:0/2465665927 >> 172.21.15.33:0/2465665927 conn(0x7fd5d40793f0 msgr2=0x7fd5d4100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:02:04.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.585+0000 7fd5dbad2700 1 -- 172.21.15.33:0/2465665927 shutdown_connections 2024-05-15T14:02:04.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:04.585+0000 7fd5dbad2700 1 -- 172.21.15.33:0/2465665927 wait complete. 2024-05-15T14:02:04.588 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 24 2024-05-15T14:02:04.917 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:04 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/2465665927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:02:05.090 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:04 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/2465665927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:02:05.106 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:04 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/2465665927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:02:05.345 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24478 quiesce db --set-id e8d9ebc6 --release --await' 2024-05-15T14:02:05.346 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24478 quiesce db --set-id e8d9ebc6 --release --await 2024-05-15T14:02:05.702 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:02:05.930 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:05 smithi033 ceph-mon[31650]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-15T14:02:06.090 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:05 smithi052 ceph-mon[36190]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-15T14:02:06.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:05 smithi081 ceph-mon[36434]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-15T14:02:07.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.619+0000 7f0fa1198700 1 -- 172.21.15.33:0/2448868805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f9c102a80 msgr2=0x7f0f9c102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:07.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.619+0000 7f0fa1198700 1 --2- 172.21.15.33:0/2448868805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f9c102a80 0x7f0f9c102e60 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c0099c0 tx=0x7f0f8c009cd0 comp rx=0 tx=0).stop 2024-05-15T14:02:07.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.620+0000 7f0fa1198700 1 -- 172.21.15.33:0/2448868805 shutdown_connections 2024-05-15T14:02:07.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.620+0000 7f0fa1198700 1 --2- 172.21.15.33:0/2448868805 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f9c1077d0 0x7f0f9c10f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:07.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.620+0000 7f0fa1198700 1 --2- 172.21.15.33:0/2448868805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f9c102a80 0x7f0f9c102e60 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:07.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.620+0000 7f0fa1198700 1 --2- 172.21.15.33:0/2448868805 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f9c103430 0x7f0f9c1070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:07.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.620+0000 7f0fa1198700 1 -- 172.21.15.33:0/2448868805 >> 172.21.15.33:0/2448868805 conn(0x7f0f9c0793f0 msgr2=0x7f0f9c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:02:07.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.620+0000 7f0fa1198700 1 -- 172.21.15.33:0/2448868805 shutdown_connections 2024-05-15T14:02:07.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.620+0000 7f0fa1198700 1 -- 172.21.15.33:0/2448868805 wait complete. 2024-05-15T14:02:07.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.621+0000 7f0fa1198700 1 Processor -- start 2024-05-15T14:02:07.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.621+0000 7f0fa1198700 1 -- start start 2024-05-15T14:02:07.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.622+0000 7f0fa1198700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f9c102a80 0x7f0f9c19f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:07.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.622+0000 7f0fa1198700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f9c103430 0x7f0f9c19fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:07.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.622+0000 7f0fa1198700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f9c1077d0 0x7f0f9c199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:07.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.622+0000 7f0fa1198700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c1126a0 con 0x7f0f9c103430 2024-05-15T14:02:07.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.622+0000 7f0fa1198700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c112520 con 0x7f0f9c1077d0 2024-05-15T14:02:07.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.622+0000 7f0fa1198700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c112820 con 0x7f0f9c102a80 2024-05-15T14:02:07.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.622+0000 7f0f9a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f9c103430 0x7f0f9c19fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:07.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.622+0000 7f0f9a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f9c103430 0x7f0f9c19fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57052/0 (socket says 172.21.15.33:57052) 2024-05-15T14:02:07.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.622+0000 7f0f9a59c700 1 -- 172.21.15.33:0/4089196169 learned_addr learned my addr 172.21.15.33:0/4089196169 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:02:07.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.622+0000 7f0f9ad9d700 1 --2- 172.21.15.33:0/4089196169 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f9c102a80 0x7f0f9c19f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:07.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.622+0000 7f0f9b59e700 1 --2- 172.21.15.33:0/4089196169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f9c1077d0 0x7f0f9c199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:07.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.622+0000 7f0f9ad9d700 1 -- 172.21.15.33:0/4089196169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f9c1077d0 msgr2=0x7f0f9c199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:07.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.622+0000 7f0f9ad9d700 1 --2- 172.21.15.33:0/4089196169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f9c1077d0 0x7f0f9c199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:07.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.622+0000 7f0f9ad9d700 1 -- 172.21.15.33:0/4089196169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f9c103430 msgr2=0x7f0f9c19fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:07.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.622+0000 7f0f9ad9d700 1 --2- 172.21.15.33:0/4089196169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f9c103430 0x7f0f9c19fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:07.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.622+0000 7f0f9ad9d700 1 -- 172.21.15.33:0/4089196169 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f9c19a1d0 con 0x7f0f9c102a80 2024-05-15T14:02:07.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.622+0000 7f0f9a59c700 1 --2- 172.21.15.33:0/4089196169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f9c103430 0x7f0f9c19fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:02:07.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.623+0000 7f0f9ad9d700 1 --2- 172.21.15.33:0/4089196169 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f9c102a80 0x7f0f9c19f8b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c0098e0 tx=0x7f0f8c01cdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:02:07.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.624+0000 7f0f83fff700 1 -- 172.21.15.33:0/4089196169 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c009e60 con 0x7f0f9c102a80 2024-05-15T14:02:07.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.624+0000 7f0fa1198700 1 -- 172.21.15.33:0/4089196169 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f9c19a460 con 0x7f0f9c102a80 2024-05-15T14:02:07.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.624+0000 7f0f83fff700 1 -- 172.21.15.33:0/4089196169 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f8c01b070 con 0x7f0f9c102a80 2024-05-15T14:02:07.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.624+0000 7f0fa1198700 1 -- 172.21.15.33:0/4089196169 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f9c104070 con 0x7f0f9c102a80 2024-05-15T14:02:07.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.625+0000 7f0f83fff700 1 -- 172.21.15.33:0/4089196169 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c005290 con 0x7f0f9c102a80 2024-05-15T14:02:07.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.628+0000 7f0f83fff700 1 -- 172.21.15.33:0/4089196169 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0f8c005430 con 0x7f0f9c102a80 2024-05-15T14:02:07.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.629+0000 7f0faba2eb80 1 Processor -- start 2024-05-15T14:02:07.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.629+0000 7f0faba2eb80 1 -- start start 2024-05-15T14:02:07.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.629+0000 7f0faba2eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618dd866420 0x5618dd866800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:07.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.629+0000 7f0faba2eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618dd866d40 0x5618dd861ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:07.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.629+0000 7f0faba2eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618dd8622c0 0x5618dd862720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:07.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.629+0000 7f0faba2eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5618dd7bfe70 con 0x5618dd866d40 2024-05-15T14:02:07.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.629+0000 7f0faba2eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5618dd7afa10 con 0x5618dd866420 2024-05-15T14:02:07.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.629+0000 7f0faba2eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5618dd7b3a50 con 0x5618dd8622c0 2024-05-15T14:02:07.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.630+0000 7f0f9ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618dd866d40 0x5618dd861ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:07.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.630+0000 7f0f9ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618dd866d40 0x5618dd861ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57058/0 (socket says 172.21.15.33:57058) 2024-05-15T14:02:07.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.630+0000 7f0f9ad9d700 1 -- 172.21.15.33:0/613883826 learned_addr learned my addr 172.21.15.33:0/613883826 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:02:07.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.630+0000 7f0f9b59e700 1 --2- 172.21.15.33:0/613883826 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618dd866420 0x5618dd866800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:07.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.630+0000 7f0f83fff700 1 --2- 172.21.15.33:0/4089196169 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0f840b2a70 0x7f0f840b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:07.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.630+0000 7f0f9a59c700 1 --2- 172.21.15.33:0/613883826 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618dd8622c0 0x5618dd862720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:07.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.630+0000 7f0f83fff700 1 -- 172.21.15.33:0/4089196169 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f0f8c0e3280 con 0x7f0f9c102a80 2024-05-15T14:02:07.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.630+0000 7f0f9a59c700 1 -- 172.21.15.33:0/613883826 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618dd866420 msgr2=0x5618dd866800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:07.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.630+0000 7f0f9a59c700 1 --2- 172.21.15.33:0/613883826 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618dd866420 0x5618dd866800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:07.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.630+0000 7f0f9a59c700 1 -- 172.21.15.33:0/613883826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618dd866d40 msgr2=0x5618dd861ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:07.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.630+0000 7f0f9a59c700 1 --2- 172.21.15.33:0/613883826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618dd866d40 0x5618dd861ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:07.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.630+0000 7f0f9a59c700 1 -- 172.21.15.33:0/613883826 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618dd7590b0 con 0x5618dd8622c0 2024-05-15T14:02:07.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.631+0000 7f0f9ad9d700 1 --2- 172.21.15.33:0/613883826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618dd866d40 0x5618dd861ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:02:07.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.631+0000 7f0f9b59e700 1 --2- 172.21.15.33:0/4089196169 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0f840b2a70 0x7f0f840b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:07.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.631+0000 7f0f9a59c700 1 --2- 172.21.15.33:0/613883826 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618dd8622c0 0x5618dd862720 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c108d40 tx=0x7f0f90011820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2253a72d3abb0084 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:02:07.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.631+0000 7f0f817fa700 1 -- 172.21.15.33:0/613883826 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9001e070 con 0x5618dd8622c0 2024-05-15T14:02:07.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.631+0000 7f0f817fa700 1 -- 172.21.15.33:0/613883826 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f90019ae0 con 0x5618dd8622c0 2024-05-15T14:02:07.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.631+0000 7f0f9b59e700 1 --2- 172.21.15.33:0/4089196169 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0f840b2a70 0x7f0f840b4f30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0f9401b860 tx=0x7f0f94004b40 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:02:07.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.631+0000 7f0f817fa700 1 -- 172.21.15.33:0/613883826 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9000ec00 con 0x5618dd8622c0 2024-05-15T14:02:07.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.632+0000 7f0faba2eb80 1 -- 172.21.15.33:0/613883826 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618dd8622c0 msgr2=0x5618dd862720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:07.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.632+0000 7f0faba2eb80 1 --2- 172.21.15.33:0/613883826 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618dd8622c0 0x5618dd862720 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c108d40 tx=0x7f0f90011820 comp rx=0 tx=0).stop 2024-05-15T14:02:07.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.632+0000 7f0faba2eb80 1 -- 172.21.15.33:0/613883826 shutdown_connections 2024-05-15T14:02:07.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.632+0000 7f0faba2eb80 1 --2- 172.21.15.33:0/613883826 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618dd8622c0 0x5618dd862720 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:07.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.632+0000 7f0faba2eb80 1 --2- 172.21.15.33:0/613883826 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618dd866420 0x5618dd866800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:07.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.632+0000 7f0faba2eb80 1 --2- 172.21.15.33:0/613883826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618dd866d40 0x5618dd861ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:07.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.632+0000 7f0faba2eb80 1 -- 172.21.15.33:0/613883826 >> 172.21.15.33:0/613883826 conn(0x5618dd78eeb0 msgr2=0x5618dd85c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:02:07.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.633+0000 7f0faba2eb80 1 -- 172.21.15.33:0/613883826 shutdown_connections 2024-05-15T14:02:07.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.633+0000 7f0faba2eb80 1 -- 172.21.15.33:0/613883826 wait complete. 2024-05-15T14:02:07.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.633+0000 7f0faba2eb80 1 -- start start 2024-05-15T14:02:07.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.633+0000 7f0faba2eb80 1 Processor -- start 2024-05-15T14:02:07.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.634+0000 7f0faba2eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618dd8622c0 0x5618dd86daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:07.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.634+0000 7f0faba2eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618dd866420 0x5618dd86ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:07.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.634+0000 7f0faba2eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618dd866d40 0x5618dd86b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:07.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.634+0000 7f0faba2eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5618dd6b60a0 con 0x5618dd866d40 2024-05-15T14:02:07.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.634+0000 7f0faba2eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5618dd717ea0 con 0x5618dd866420 2024-05-15T14:02:07.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.634+0000 7f0faba2eb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5618dd86e880 con 0x5618dd8622c0 2024-05-15T14:02:07.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.634+0000 7f0f9ad9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618dd8622c0 0x5618dd86daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:07.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.634+0000 7f0f9ad9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618dd8622c0 0x5618dd86daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:58694/0 (socket says 172.21.15.33:58694) 2024-05-15T14:02:07.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.634+0000 7f0f9ad9d700 1 -- 172.21.15.33:0/1816368441 learned_addr learned my addr 172.21.15.33:0/1816368441 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:02:07.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.634+0000 7f0faba2eb80 10 client.0 _ll_register_callbacks cb 0x5618dd74aa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:02:07.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.634+0000 7f0f9b59e700 1 --2- 172.21.15.33:0/1816368441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618dd866d40 0x5618dd86b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:07.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.634+0000 7f0f9a59c700 1 --2- 172.21.15.33:0/1816368441 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618dd866420 0x5618dd86ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:07.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.634+0000 7f0f9ad9d700 1 -- 172.21.15.33:0/1816368441 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618dd866420 msgr2=0x5618dd86ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:07.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.634+0000 7f0f9ad9d700 1 --2- 172.21.15.33:0/1816368441 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618dd866420 0x5618dd86ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:07.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.635+0000 7f0f9ad9d700 1 -- 172.21.15.33:0/1816368441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618dd866d40 msgr2=0x5618dd86b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:07.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.635+0000 7f0f9ad9d700 1 --2- 172.21.15.33:0/1816368441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618dd866d40 0x5618dd86b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:07.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.635+0000 7f0f9ad9d700 1 -- 172.21.15.33:0/1816368441 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618dd6ff9d0 con 0x5618dd8622c0 2024-05-15T14:02:07.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.635+0000 7f0f9b59e700 1 --2- 172.21.15.33:0/1816368441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618dd866d40 0x5618dd86b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:02:07.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.635+0000 7f0f9ad9d700 1 --2- 172.21.15.33:0/1816368441 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618dd8622c0 0x5618dd86daf0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c0a6790 tx=0x7f0f8c009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8817523b46064eb server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:02:07.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.635+0000 7f0f7eff5700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-05-15T14:02:07.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.635+0000 7f0f7eff5700 1 -- 172.21.15.33:0/1816368441 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c0a65d0 con 0x5618dd8622c0 2024-05-15T14:02:07.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.635+0000 7f0f7eff5700 1 -- 172.21.15.33:0/1816368441 <== mon.1 v2:172.21.15.81:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f0f8c0920b0 con 0x5618dd8622c0 2024-05-15T14:02:07.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.635+0000 7f0faba2eb80 1 -- 172.21.15.33:0/1816368441 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f90019ae0 con 0x5618dd8622c0 2024-05-15T14:02:07.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.635+0000 7f0f7eff5700 1 -- 172.21.15.33:0/1816368441 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0f8c0e0e70 con 0x5618dd8622c0 2024-05-15T14:02:07.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.635+0000 7f0faba2eb80 10 client.25240 fetch_fsmap learned FSMap version 24 2024-05-15T14:02:07.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.635+0000 7f0faba2eb80 1 -- 172.21.15.33:0/1816368441 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5618dd769860 con 0x5618dd8622c0 2024-05-15T14:02:07.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.636+0000 7f0f7eff5700 1 -- 172.21.15.33:0/1816368441 <== mon.1 v2:172.21.15.81:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f8c005720 con 0x5618dd8622c0 2024-05-15T14:02:07.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.636+0000 7f0f7eff5700 1 -- 172.21.15.33:0/1816368441 <== mon.1 v2:172.21.15.81:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f0f8c0aa970 con 0x5618dd8622c0 2024-05-15T14:02:07.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.636+0000 7f0faba2eb80 10 client.25240 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:02:07.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.636+0000 7f0faba2eb80 10 client.25240 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:02:07.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.636+0000 7f0faba2eb80 20 client.25240 populate_metadata read hostname 'smithi033' 2024-05-15T14:02:07.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.636+0000 7f0faba2eb80 1 --2- 172.21.15.33:0/1816368441 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5618dd87b5e0 0x5618dd87da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:07.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.636+0000 7f0faba2eb80 4 client.25240 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:02:07.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.636+0000 7f0faba2eb80 1 -- 172.21.15.33:0/1816368441 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0f90019ae0 con 0x5618dd87b5e0 2024-05-15T14:02:07.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.636+0000 7f0f9a59c700 1 --2- 172.21.15.33:0/1816368441 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5618dd87b5e0 0x5618dd87da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:07.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.637+0000 7f0f7eff5700 10 client.25240 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:02:07.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.637+0000 7f0f9a59c700 1 --2- 172.21.15.33:0/1816368441 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5618dd87b5e0 0x5618dd87da00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84feb19cc684c0c server_cookie=9e15b875cfae21cf in_seq=0 out_seq=0 2024-05-15T14:02:07.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.637+0000 7f0f7eff5700 1 -- 172.21.15.33:0/1816368441 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f0f90019ae0 con 0x5618dd87b5e0 2024-05-15T14:02:07.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.637+0000 7f0f7eff5700 10 client.25240 handle_command_reply: tid=0 2024-05-15T14:02:07.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.637+0000 7f0faba2eb80 1 client.25240 shutdown 2024-05-15T14:02:07.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.638+0000 7f0faba2eb80 1 -- 172.21.15.33:0/1816368441 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618dd8622c0 msgr2=0x5618dd86daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:07.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.638+0000 7f0faba2eb80 1 --2- 172.21.15.33:0/1816368441 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618dd8622c0 0x5618dd86daf0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c0a6790 tx=0x7f0f8c009310 comp rx=0 tx=0).stop 2024-05-15T14:02:07.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.638+0000 7f0faba2eb80 1 -- 172.21.15.33:0/1816368441 shutdown_connections 2024-05-15T14:02:07.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.638+0000 7f0faba2eb80 1 --2- 172.21.15.33:0/1816368441 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5618dd87b5e0 0x5618dd87da00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:07.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.638+0000 7f0faba2eb80 1 --2- 172.21.15.33:0/1816368441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618dd866d40 0x5618dd86b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:07.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.638+0000 7f0faba2eb80 1 --2- 172.21.15.33:0/1816368441 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618dd8622c0 0x5618dd86daf0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:07.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.638+0000 7f0faba2eb80 1 --2- 172.21.15.33:0/1816368441 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618dd866420 0x5618dd86ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:07.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.638+0000 7f0faba2eb80 1 -- 172.21.15.33:0/1816368441 >> 172.21.15.33:0/1816368441 conn(0x5618dd78eeb0 msgr2=0x5618dd856fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:02:07.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.638+0000 7f0f7eff5700 0 client.25240 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:02:07.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.638+0000 7f0faba2eb80 1 -- 172.21.15.33:0/1816368441 shutdown_connections 2024-05-15T14:02:07.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.638+0000 7f0faba2eb80 1 -- 172.21.15.33:0/1816368441 wait complete. 2024-05-15T14:02:07.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.638+0000 7f0faba2eb80 20 client.25240 trim_cache size 0 max 16384 2024-05-15T14:02:07.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.679+0000 7f0faba2eb80 1 Processor -- start 2024-05-15T14:02:07.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.679+0000 7f0faba2eb80 1 -- start start 2024-05-15T14:02:07.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.679+0000 7f0faba2eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618dd8622c0 0x5618dd8570d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:07.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.679+0000 7f0faba2eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618dd866420 0x5618dd85fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:07.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.679+0000 7f0faba2eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618dd866d40 0x5618dd8601d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:07.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.679+0000 7f0f9ad9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618dd8622c0 0x5618dd8570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:07.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.679+0000 7f0faba2eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8800fe40 con 0x5618dd866d40 2024-05-15T14:02:07.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.679+0000 7f0f9ad9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618dd8622c0 0x5618dd8570d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:58702/0 (socket says 172.21.15.33:58702) 2024-05-15T14:02:07.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.679+0000 7f0f9ad9d700 1 -- 172.21.15.33:0/1091395636 learned_addr learned my addr 172.21.15.33:0/1091395636 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:02:07.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.679+0000 7f0faba2eb80 1 -- 172.21.15.33:0/1091395636 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8800d6d0 con 0x5618dd866420 2024-05-15T14:02:07.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.679+0000 7f0faba2eb80 1 -- 172.21.15.33:0/1091395636 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8800b520 con 0x5618dd8622c0 2024-05-15T14:02:07.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.679+0000 7f0f9a59c700 1 --2- 172.21.15.33:0/1091395636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618dd866420 0x5618dd85fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:07.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.680+0000 7f0f9b59e700 1 --2- 172.21.15.33:0/1091395636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618dd866d40 0x5618dd8601d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:07.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.680+0000 7f0f9ad9d700 1 -- 172.21.15.33:0/1091395636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618dd866420 msgr2=0x5618dd85fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:07.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.680+0000 7f0f9ad9d700 1 --2- 172.21.15.33:0/1091395636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618dd866420 0x5618dd85fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:07.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.680+0000 7f0f9ad9d700 1 -- 172.21.15.33:0/1091395636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618dd866d40 msgr2=0x5618dd8601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:07.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.680+0000 7f0f9ad9d700 1 --2- 172.21.15.33:0/1091395636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618dd866d40 0x5618dd8601d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:07.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.680+0000 7f0f9ad9d700 1 -- 172.21.15.33:0/1091395636 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618dd85cd90 con 0x5618dd8622c0 2024-05-15T14:02:07.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.680+0000 7f0f9b59e700 1 --2- 172.21.15.33:0/1091395636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618dd866d40 0x5618dd8601d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:02:07.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.680+0000 7f0f9ad9d700 1 --2- 172.21.15.33:0/1091395636 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618dd8622c0 0x5618dd8570d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c0aebc0 tx=0x7f0f8c0b34e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2de8eca8766f5499 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:02:07.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.680+0000 7f0f80ff9700 1 -- 172.21.15.33:0/1091395636 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c0a9e80 con 0x5618dd8622c0 2024-05-15T14:02:07.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.681+0000 7f0f80ff9700 1 -- 172.21.15.33:0/1091395636 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f8c005720 con 0x5618dd8622c0 2024-05-15T14:02:07.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.681+0000 7f0f80ff9700 1 -- 172.21.15.33:0/1091395636 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c0aa060 con 0x5618dd8622c0 2024-05-15T14:02:07.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.681+0000 7f0faba2eb80 1 -- 172.21.15.33:0/1091395636 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618dd8622c0 msgr2=0x5618dd8570d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:07.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.681+0000 7f0faba2eb80 1 --2- 172.21.15.33:0/1091395636 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618dd8622c0 0x5618dd8570d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c0aebc0 tx=0x7f0f8c0b34e0 comp rx=0 tx=0).stop 2024-05-15T14:02:07.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.681+0000 7f0faba2eb80 1 -- 172.21.15.33:0/1091395636 shutdown_connections 2024-05-15T14:02:07.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.681+0000 7f0faba2eb80 1 --2- 172.21.15.33:0/1091395636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618dd866d40 0x5618dd8601d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:07.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.682+0000 7f0faba2eb80 1 --2- 172.21.15.33:0/1091395636 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618dd8622c0 0x5618dd8570d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:07.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.682+0000 7f0faba2eb80 1 --2- 172.21.15.33:0/1091395636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618dd866420 0x5618dd85fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:07.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.682+0000 7f0faba2eb80 1 -- 172.21.15.33:0/1091395636 >> 172.21.15.33:0/1091395636 conn(0x5618dd78eeb0 msgr2=0x5618dd8607f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:02:07.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.682+0000 7f0faba2eb80 1 -- 172.21.15.33:0/1091395636 shutdown_connections 2024-05-15T14:02:07.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.682+0000 7f0faba2eb80 1 -- 172.21.15.33:0/1091395636 wait complete. 2024-05-15T14:02:07.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.682+0000 7f0faba2eb80 1 -- start start 2024-05-15T14:02:07.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.682+0000 7f0faba2eb80 1 Processor -- start 2024-05-15T14:02:07.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.683+0000 7f0faba2eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618dd8622c0 0x5618dd873d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:07.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.683+0000 7f0faba2eb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618dd866420 0x5618dd86eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:07.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.683+0000 7f0faba2eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618dd866d40 0x5618dd86f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:07.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.683+0000 7f0f9a59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618dd866420 0x5618dd86eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:07.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.683+0000 7f0f9a59c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618dd866420 0x5618dd86eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:58706/0 (socket says 172.21.15.33:58706) 2024-05-15T14:02:07.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.683+0000 7f0f9ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618dd8622c0 0x5618dd873d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:07.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.683+0000 7f0f9a59c700 1 -- 172.21.15.33:0/1820966001 learned_addr learned my addr 172.21.15.33:0/1820966001 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:02:07.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.683+0000 7f0faba2eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8800bb90 con 0x5618dd8622c0 2024-05-15T14:02:07.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.683+0000 7f0faba2eb80 1 -- 172.21.15.33:0/1820966001 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8800d000 con 0x5618dd866d40 2024-05-15T14:02:07.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.683+0000 7f0faba2eb80 1 -- 172.21.15.33:0/1820966001 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8800c260 con 0x5618dd866420 2024-05-15T14:02:07.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.683+0000 7f0faba2eb80 10 client.0 _ll_register_callbacks cb 0x5618dd74aa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:02:07.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.683+0000 7f0f9b59e700 1 --2- 172.21.15.33:0/1820966001 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618dd866d40 0x5618dd86f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:07.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.684+0000 7f0f9a59c700 1 -- 172.21.15.33:0/1820966001 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618dd866d40 msgr2=0x5618dd86f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:07.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.684+0000 7f0f9a59c700 1 --2- 172.21.15.33:0/1820966001 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618dd866d40 0x5618dd86f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:07.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.684+0000 7f0f9a59c700 1 -- 172.21.15.33:0/1820966001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618dd8622c0 msgr2=0x5618dd873d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:07.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.684+0000 7f0f9a59c700 1 --2- 172.21.15.33:0/1820966001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618dd8622c0 0x5618dd873d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:07.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.684+0000 7f0f9a59c700 1 -- 172.21.15.33:0/1820966001 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618dd86f5e0 con 0x5618dd866420 2024-05-15T14:02:07.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.684+0000 7f0f9b59e700 1 --2- 172.21.15.33:0/1820966001 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618dd866d40 0x5618dd86f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:02:07.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.684+0000 7f0f9ad9d700 1 --2- 172.21.15.33:0/1820966001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618dd8622c0 0x5618dd873d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:02:07.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.684+0000 7f0f9a59c700 1 --2- 172.21.15.33:0/1820966001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618dd866420 0x5618dd86eb60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0f90011510 tx=0x7f0f90011c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8467ca8962a6f9f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:02:07.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.684+0000 7f0f7eff5700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-05-15T14:02:07.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.685+0000 7f0f7eff5700 1 -- 172.21.15.33:0/1820966001 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f90032020 con 0x5618dd866420 2024-05-15T14:02:07.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.685+0000 7f0f7eff5700 1 -- 172.21.15.33:0/1820966001 <== mon.1 v2:172.21.15.81:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f0f9000e850 con 0x5618dd866420 2024-05-15T14:02:07.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.685+0000 7f0faba2eb80 1 -- 172.21.15.33:0/1820966001 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5618dd7c1dc0 con 0x5618dd866420 2024-05-15T14:02:07.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.685+0000 7f0f7eff5700 1 -- 172.21.15.33:0/1820966001 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f0f9003d050 con 0x5618dd866420 2024-05-15T14:02:07.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.685+0000 7f0faba2eb80 10 client.25246 fetch_fsmap learned FSMap version 24 2024-05-15T14:02:07.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.685+0000 7f0faba2eb80 1 -- 172.21.15.33:0/1820966001 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5618dd8e68e0 con 0x5618dd866420 2024-05-15T14:02:07.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.685+0000 7f0f7eff5700 1 -- 172.21.15.33:0/1820966001 <== mon.1 v2:172.21.15.81:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5618dd7c1dc0 con 0x5618dd866420 2024-05-15T14:02:07.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.686+0000 7f0f7eff5700 1 -- 172.21.15.33:0/1820966001 <== mon.1 v2:172.21.15.81:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f0f900052a0 con 0x5618dd866420 2024-05-15T14:02:07.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.686+0000 7f0faba2eb80 10 client.25246 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:02:07.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.686+0000 7f0faba2eb80 10 client.25246 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:02:07.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.686+0000 7f0faba2eb80 20 client.25246 populate_metadata read hostname 'smithi033' 2024-05-15T14:02:07.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.687+0000 7f0faba2eb80 1 --2- 172.21.15.33:0/1820966001 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5618dd8e6eb0 0x5618dd8e92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:07.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.687+0000 7f0faba2eb80 4 client.25246 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "e8d9ebc6", "release": true, "await": true}] 2024-05-15T14:02:07.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.687+0000 7f0faba2eb80 1 -- 172.21.15.33:0/1820966001 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "quiesce db", "set_id": "e8d9ebc6", "release": true, "await": true}) -- 0x5618dd7bb5a0 con 0x5618dd8e6eb0 2024-05-15T14:02:07.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.687+0000 7f0f9ad9d700 1 --2- 172.21.15.33:0/1820966001 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5618dd8e6eb0 0x5618dd8e92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:07.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.687+0000 7f0f9ad9d700 1 --2- 172.21.15.33:0/1820966001 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5618dd8e6eb0 0x5618dd8e92d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=136eafbf41ee218 server_cookie=bea8a3dae89b3f10 in_seq=0 out_seq=0 2024-05-15T14:02:07.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.687+0000 7f0f7eff5700 10 client.25246 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:02:07.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.689+0000 7f0f7eff5700 1 -- 172.21.15.33:0/1820966001 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5618dd7bb5a0 con 0x5618dd8e6eb0 2024-05-15T14:02:07.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.689+0000 7f0f7eff5700 10 client.25246 handle_command_reply: tid=0 2024-05-15T14:02:07.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.689+0000 7f0faba2eb80 1 client.25246 shutdown 2024-05-15T14:02:07.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.690+0000 7f0faba2eb80 1 -- 172.21.15.33:0/1820966001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618dd866420 msgr2=0x5618dd86eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:07.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.690+0000 7f0faba2eb80 1 --2- 172.21.15.33:0/1820966001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618dd866420 0x5618dd86eb60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0f90011510 tx=0x7f0f90011c10 comp rx=0 tx=0).stop 2024-05-15T14:02:07.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.690+0000 7f0faba2eb80 1 -- 172.21.15.33:0/1820966001 shutdown_connections 2024-05-15T14:02:07.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.690+0000 7f0faba2eb80 1 --2- 172.21.15.33:0/1820966001 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5618dd866d40 0x5618dd86f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:07.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.690+0000 7f0faba2eb80 1 --2- 172.21.15.33:0/1820966001 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5618dd8e6eb0 0x5618dd8e92d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:07.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.690+0000 7f0faba2eb80 1 --2- 172.21.15.33:0/1820966001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5618dd8622c0 0x5618dd873d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:07.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.690+0000 7f0faba2eb80 1 --2- 172.21.15.33:0/1820966001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618dd866420 0x5618dd86eb60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:07.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.690+0000 7f0faba2eb80 1 -- 172.21.15.33:0/1820966001 >> 172.21.15.33:0/1820966001 conn(0x5618dd78eeb0 msgr2=0x5618dd86cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:02:07.694 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T14:02:07.694 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 24, 2024-05-15T14:02:07.694 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24478, 2024-05-15T14:02:07.694 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 8, 2024-05-15T14:02:07.694 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-05-15T14:02:07.694 INFO:teuthology.orchestra.run.smithi033.stdout: "e8d9ebc6": { 2024-05-15T14:02:07.694 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 8, 2024-05-15T14:02:07.694 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-05-15T14:02:07.694 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:02:07.694 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-05-15T14:02:07.694 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:02:07.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:02:07.695 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 90.0, 2024-05-15T14:02:07.695 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 144.0, 2024-05-15T14:02:07.695 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-05-15T14:02:07.695 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-05-15T14:02:07.695 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-05-15T14:02:07.695 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:02:07.695 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-05-15T14:02:07.695 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:02:07.695 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:02:07.695 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:02:07.695 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:02:07.695 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:02:07.695 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:02:07.695 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T14:02:07.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.690+0000 7f0f7eff5700 0 client.25246 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:02:07.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.690+0000 7f0faba2eb80 1 -- 172.21.15.33:0/1820966001 shutdown_connections 2024-05-15T14:02:07.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.690+0000 7f0faba2eb80 1 -- 172.21.15.33:0/1820966001 wait complete. 2024-05-15T14:02:07.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.690+0000 7f0faba2eb80 20 client.25246 trim_cache size 0 max 16384 2024-05-15T14:02:07.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.691+0000 7f0fa1198700 1 -- 172.21.15.33:0/4089196169 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0f840b2a70 msgr2=0x7f0f840b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:07.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.691+0000 7f0fa1198700 1 --2- 172.21.15.33:0/4089196169 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0f840b2a70 0x7f0f840b4f30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0f9401b860 tx=0x7f0f94004b40 comp rx=0 tx=0).stop 2024-05-15T14:02:07.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.691+0000 7f0fa1198700 1 -- 172.21.15.33:0/4089196169 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f9c102a80 msgr2=0x7f0f9c19f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:07.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.691+0000 7f0fa1198700 1 --2- 172.21.15.33:0/4089196169 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f9c102a80 0x7f0f9c19f8b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c0098e0 tx=0x7f0f8c01cdb0 comp rx=0 tx=0).stop 2024-05-15T14:02:07.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.691+0000 7f0fa1198700 1 -- 172.21.15.33:0/4089196169 shutdown_connections 2024-05-15T14:02:07.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.691+0000 7f0fa1198700 1 --2- 172.21.15.33:0/4089196169 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0f840b2a70 0x7f0f840b4f30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:07.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.691+0000 7f0fa1198700 1 --2- 172.21.15.33:0/4089196169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f9c1077d0 0x7f0f9c199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:07.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.691+0000 7f0fa1198700 1 --2- 172.21.15.33:0/4089196169 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f9c102a80 0x7f0f9c19f8b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:07.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.691+0000 7f0fa1198700 1 --2- 172.21.15.33:0/4089196169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f9c103430 0x7f0f9c19fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:07.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.691+0000 7f0fa1198700 1 -- 172.21.15.33:0/4089196169 >> 172.21.15.33:0/4089196169 conn(0x7f0f9c0793f0 msgr2=0x7f0f9c06c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:02:07.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.692+0000 7f0fa1198700 1 -- 172.21.15.33:0/4089196169 shutdown_connections 2024-05-15T14:02:07.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:02:07.692+0000 7f0fa1198700 1 -- 172.21.15.33:0/4089196169 wait complete. 2024-05-15T14:02:08.410 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:08 smithi033 ceph-mon[31650]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-15T14:02:08.449 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'e8d9ebc6', total seconds elapsed: 30.5 2024-05-15T14:02:08.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:08 smithi052 ceph-mon[36190]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-15T14:02:08.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:08 smithi081 ceph-mon[36434]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-15T14:02:09.387 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.385+0000 7fae55e2a640 1 -- 172.21.15.33:0/1285228601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae50101930 msgr2=0x7fae50101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.385+0000 7fae55e2a640 1 --2- 172.21.15.33:0/1285228601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae50101930 0x7fae50101db0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fae44009930 tx=0x7fae4402a420 comp rx=0 tx=0).stop 2024-05-15T14:02:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.385+0000 7fae55e2a640 1 -- 172.21.15.33:0/1285228601 shutdown_connections 2024-05-15T14:02:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.385+0000 7fae55e2a640 1 --2- 172.21.15.33:0/1285228601 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae501022f0 0x7fae5010e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.385+0000 7fae55e2a640 1 --2- 172.21.15.33:0/1285228601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae50101930 0x7fae50101db0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.385+0000 7fae55e2a640 1 --2- 172.21.15.33:0/1285228601 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae50107940 0x7fae50107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.385+0000 7fae55e2a640 1 -- 172.21.15.33:0/1285228601 >> 172.21.15.33:0/1285228601 conn(0x7fae50078a70 msgr2=0x7fae50078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:02:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.385+0000 7fae55e2a640 1 -- 172.21.15.33:0/1285228601 shutdown_connections 2024-05-15T14:02:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.385+0000 7fae55e2a640 1 -- 172.21.15.33:0/1285228601 wait complete. 2024-05-15T14:02:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.385+0000 7fae55e2a640 1 Processor -- start 2024-05-15T14:02:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.386+0000 7fae55e2a640 1 -- start start 2024-05-15T14:02:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.386+0000 7fae55e2a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae50101930 0x7fae50199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.386+0000 7fae55e2a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae501022f0 0x7fae50199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.386+0000 7fae55e2a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae50107940 0x7fae5019da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.386+0000 7fae55e2a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fae50111bd0 con 0x7fae501022f0 2024-05-15T14:02:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.386+0000 7fae55e2a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fae50111a50 con 0x7fae50107940 2024-05-15T14:02:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.386+0000 7fae55e2a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fae50111d50 con 0x7fae50101930 2024-05-15T14:02:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.386+0000 7fae4effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae501022f0 0x7fae50199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.386+0000 7fae4effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae501022f0 0x7fae50199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57108/0 (socket says 172.21.15.33:57108) 2024-05-15T14:02:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.386+0000 7fae4effd640 1 -- 172.21.15.33:0/4250137482 learned_addr learned my addr 172.21.15.33:0/4250137482 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:02:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.386+0000 7fae4ffff640 1 --2- 172.21.15.33:0/4250137482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae50107940 0x7fae5019da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.386+0000 7fae4f7fe640 1 --2- 172.21.15.33:0/4250137482 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae50101930 0x7fae50199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.387+0000 7fae4effd640 1 -- 172.21.15.33:0/4250137482 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae50101930 msgr2=0x7fae50199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.387+0000 7fae4effd640 1 --2- 172.21.15.33:0/4250137482 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae50101930 0x7fae50199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.387+0000 7fae4effd640 1 -- 172.21.15.33:0/4250137482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae50107940 msgr2=0x7fae5019da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.387+0000 7fae4effd640 1 --2- 172.21.15.33:0/4250137482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae50107940 0x7fae5019da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.387+0000 7fae4effd640 1 -- 172.21.15.33:0/4250137482 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae5019e130 con 0x7fae501022f0 2024-05-15T14:02:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.387+0000 7fae4ffff640 1 --2- 172.21.15.33:0/4250137482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae50107940 0x7fae5019da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:02:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.387+0000 7fae4f7fe640 1 --2- 172.21.15.33:0/4250137482 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae50101930 0x7fae50199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:02:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.387+0000 7fae4effd640 1 --2- 172.21.15.33:0/4250137482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae501022f0 0x7fae50199680 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fae44009900 tx=0x7fae44030650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:02:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.387+0000 7fae4cff9640 1 -- 172.21.15.33:0/4250137482 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4403f070 con 0x7fae501022f0 2024-05-15T14:02:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.387+0000 7fae4cff9640 1 -- 172.21.15.33:0/4250137482 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae44004460 con 0x7fae501022f0 2024-05-15T14:02:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.387+0000 7fae4cff9640 1 -- 172.21.15.33:0/4250137482 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae44035030 con 0x7fae501022f0 2024-05-15T14:02:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.387+0000 7fae55e2a640 1 -- 172.21.15.33:0/4250137482 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae5019e3c0 con 0x7fae501022f0 2024-05-15T14:02:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.388+0000 7fae55e2a640 1 -- 172.21.15.33:0/4250137482 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae5019e7a0 con 0x7fae501022f0 2024-05-15T14:02:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.390+0000 7fae4cff9640 1 -- 172.21.15.33:0/4250137482 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fae44042070 con 0x7fae501022f0 2024-05-15T14:02:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.391+0000 7fae58c28740 1 Processor -- start 2024-05-15T14:02:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.391+0000 7fae4cff9640 1 --2- 172.21.15.33:0/4250137482 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fae380b2a50 0x7fae380b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.391+0000 7fae58c28740 1 -- start start 2024-05-15T14:02:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.391+0000 7fae4cff9640 1 -- 172.21.15.33:0/4250137482 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fae44103130 con 0x7fae501022f0 2024-05-15T14:02:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.391+0000 7fae4ffff640 1 --2- 172.21.15.33:0/4250137482 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fae380b2a50 0x7fae380b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.391+0000 7fae58c28740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567cc61100 0x55567cc614e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.391+0000 7fae58c28740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55567cc61a20 0x55567cc6bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.392+0000 7fae58c28740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55567cc6c2e0 0x55567cc6e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.392+0000 7fae58c28740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55567cbcbc30 con 0x55567cc61100 2024-05-15T14:02:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.392+0000 7fae58c28740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55567cbbb7d0 con 0x55567cc6c2e0 2024-05-15T14:02:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.392+0000 7fae58c28740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55567cbbf810 con 0x55567cc61a20 2024-05-15T14:02:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.392+0000 7fae4f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567cc61100 0x55567cc614e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.392+0000 7fae4f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567cc61100 0x55567cc614e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57112/0 (socket says 172.21.15.33:57112) 2024-05-15T14:02:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.392+0000 7fae4f7fe640 1 -- 172.21.15.33:0/3010881171 learned_addr learned my addr 172.21.15.33:0/3010881171 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:02:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.392+0000 7fae4effd640 1 --2- 172.21.15.33:0/3010881171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55567cc61a20 0x55567cc6bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.392+0000 7fae4ffff640 1 --2- 172.21.15.33:0/4250137482 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fae380b2a50 0x7fae380b4f10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fae480099a0 tx=0x7fae48009500 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:02:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.392+0000 7fae4effd640 1 -- 172.21.15.33:0/3010881171 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55567cc6c2e0 msgr2=0x55567cc6e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.392+0000 7fae4effd640 1 --2- 172.21.15.33:0/3010881171 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55567cc6c2e0 0x55567cc6e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.392+0000 7fae4effd640 1 -- 172.21.15.33:0/3010881171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567cc61100 msgr2=0x55567cc614e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.392+0000 7fae4effd640 1 --2- 172.21.15.33:0/3010881171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567cc61100 0x55567cc614e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.392+0000 7fae4effd640 1 -- 172.21.15.33:0/3010881171 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55567c8d0210 con 0x55567cc61a20 2024-05-15T14:02:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.392+0000 7fae4f7fe640 1 --2- 172.21.15.33:0/3010881171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567cc61100 0x55567cc614e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:02:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.392+0000 7fae4effd640 1 --2- 172.21.15.33:0/3010881171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55567cc61a20 0x55567cc6bda0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fae44100830 tx=0x7fae44005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41a095e825256ad server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:02:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.393+0000 7fae35ffb640 1 -- 172.21.15.33:0/3010881171 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae440caac0 con 0x55567cc61a20 2024-05-15T14:02:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.393+0000 7fae35ffb640 1 -- 172.21.15.33:0/3010881171 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae440ca3c0 con 0x55567cc61a20 2024-05-15T14:02:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.393+0000 7fae35ffb640 1 -- 172.21.15.33:0/3010881171 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae44100cb0 con 0x55567cc61a20 2024-05-15T14:02:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.394+0000 7fae58c28740 1 -- 172.21.15.33:0/3010881171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55567cc61a20 msgr2=0x55567cc6bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.394+0000 7fae58c28740 1 --2- 172.21.15.33:0/3010881171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55567cc61a20 0x55567cc6bda0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fae44100830 tx=0x7fae44005fd0 comp rx=0 tx=0).stop 2024-05-15T14:02:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.394+0000 7fae58c28740 1 -- 172.21.15.33:0/3010881171 shutdown_connections 2024-05-15T14:02:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.394+0000 7fae58c28740 1 --2- 172.21.15.33:0/3010881171 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55567cc6c2e0 0x55567cc6e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:09.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.394+0000 7fae58c28740 1 --2- 172.21.15.33:0/3010881171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55567cc61a20 0x55567cc6bda0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.394+0000 7fae58c28740 1 --2- 172.21.15.33:0/3010881171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567cc61100 0x55567cc614e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.394+0000 7fae58c28740 1 -- 172.21.15.33:0/3010881171 >> 172.21.15.33:0/3010881171 conn(0x55567cc5d610 msgr2=0x55567cc5d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:02:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.394+0000 7fae58c28740 1 -- 172.21.15.33:0/3010881171 shutdown_connections 2024-05-15T14:02:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.394+0000 7fae58c28740 1 -- 172.21.15.33:0/3010881171 wait complete. 2024-05-15T14:02:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.394+0000 7fae58c28740 1 -- start start 2024-05-15T14:02:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.395+0000 7fae58c28740 1 Processor -- start 2024-05-15T14:02:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.395+0000 7fae58c28740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55567cc61100 0x55567cc6bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.395+0000 7fae58c28740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567cc61a20 0x55567cc69060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.395+0000 7fae58c28740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55567cc6c2e0 0x55567cc69750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.395+0000 7fae4effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567cc61a20 0x55567cc69060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.395+0000 7fae4effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567cc61a20 0x55567cc69060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57114/0 (socket says 172.21.15.33:57114) 2024-05-15T14:02:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.395+0000 7fae4f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55567cc61100 0x55567cc6bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.395+0000 7fae4effd640 1 -- 172.21.15.33:0/3754576040 learned_addr learned my addr 172.21.15.33:0/3754576040 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:02:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.395+0000 7fae4ffff640 1 --2- 172.21.15.33:0/3754576040 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55567cc6c2e0 0x55567cc69750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.395+0000 7fae58c28740 1 -- 172.21.15.33:0/3754576040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55567c8a1790 con 0x55567cc61a20 2024-05-15T14:02:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.395+0000 7fae58c28740 1 -- 172.21.15.33:0/3754576040 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55567cb77890 con 0x55567cc6c2e0 2024-05-15T14:02:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.395+0000 7fae58c28740 1 -- 172.21.15.33:0/3754576040 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55567cc6ecd0 con 0x55567cc61100 2024-05-15T14:02:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.396+0000 7fae4effd640 1 -- 172.21.15.33:0/3754576040 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55567cc61100 msgr2=0x55567cc6bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.396+0000 7fae4effd640 1 --2- 172.21.15.33:0/3754576040 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55567cc61100 0x55567cc6bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.396+0000 7fae4effd640 1 -- 172.21.15.33:0/3754576040 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55567cc6c2e0 msgr2=0x55567cc69750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.396+0000 7fae4effd640 1 --2- 172.21.15.33:0/3754576040 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55567cc6c2e0 0x55567cc69750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.396+0000 7fae4effd640 1 -- 172.21.15.33:0/3754576040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55567c99c620 con 0x55567cc61a20 2024-05-15T14:02:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.396+0000 7fae4ffff640 1 --2- 172.21.15.33:0/3754576040 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55567cc6c2e0 0x55567cc69750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:02:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.396+0000 7fae4effd640 1 --2- 172.21.15.33:0/3754576040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567cc61a20 0x55567cc69060 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fae44004650 tx=0x7fae44005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aed78722d487a68 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:02:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.396+0000 7fae58c28740 10 client.0 _ll_register_callbacks cb 0x55567cb7ad60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:02:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.396+0000 7fae2effd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:02:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.396+0000 7fae2effd640 1 -- 172.21.15.33:0/3754576040 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae440caac0 con 0x55567cc61a20 2024-05-15T14:02:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.396+0000 7fae2effd640 1 -- 172.21.15.33:0/3754576040 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fae44100ab0 con 0x55567cc61a20 2024-05-15T14:02:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.396+0000 7fae58c28740 1 -- 172.21.15.33:0/3754576040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae440ca3c0 con 0x55567cc61a20 2024-05-15T14:02:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.397+0000 7fae2effd640 1 -- 172.21.15.33:0/3754576040 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fae44100cb0 con 0x55567cc61a20 2024-05-15T14:02:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.397+0000 7fae58c28740 10 client.15567 fetch_fsmap learned FSMap version 24 2024-05-15T14:02:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.397+0000 7fae58c28740 1 -- 172.21.15.33:0/3754576040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({fsmap=24}) -- 0x55567cc7a170 con 0x55567cc61a20 2024-05-15T14:02:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.397+0000 7fae2effd640 1 -- 172.21.15.33:0/3754576040 <== mon.0 v2:172.21.15.33:3300/0 4 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fae440d2d90 con 0x55567cc61a20 2024-05-15T14:02:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.397+0000 7fae58c28740 10 client.15567 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:02:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.397+0000 7fae58c28740 10 client.15567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:02:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.397+0000 7fae58c28740 20 client.15567 populate_metadata read hostname 'smithi033' 2024-05-15T14:02:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.397+0000 7fae58c28740 1 --2- 172.21.15.33:0/3754576040 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55567cc7b6f0 0x55567cc7db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.397+0000 7fae58c28740 4 client.15567 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:02:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.397+0000 7fae58c28740 1 -- 172.21.15.33:0/3754576040 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55567cbcdb80 con 0x55567cc7b6f0 2024-05-15T14:02:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.398+0000 7fae4f7fe640 1 --2- 172.21.15.33:0/3754576040 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55567cc7b6f0 0x55567cc7db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.398+0000 7fae2effd640 10 client.15567 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:02:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.398+0000 7fae4f7fe640 1 --2- 172.21.15.33:0/3754576040 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55567cc7b6f0 0x55567cc7db80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a5cfeca15445744 server_cookie=de0a452bf060f6f7 in_seq=0 out_seq=0 2024-05-15T14:02:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.399+0000 7fae2effd640 1 -- 172.21.15.33:0/3754576040 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55567cbcdb80 con 0x55567cc7b6f0 2024-05-15T14:02:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.399+0000 7fae2effd640 10 client.15567 handle_command_reply: tid=0 2024-05-15T14:02:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.399+0000 7fae58c28740 1 client.15567 shutdown 2024-05-15T14:02:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.399+0000 7fae58c28740 1 -- 172.21.15.33:0/3754576040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567cc61a20 msgr2=0x55567cc69060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.399+0000 7fae58c28740 1 --2- 172.21.15.33:0/3754576040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567cc61a20 0x55567cc69060 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fae44004650 tx=0x7fae44005be0 comp rx=0 tx=0).stop 2024-05-15T14:02:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.399+0000 7fae58c28740 1 -- 172.21.15.33:0/3754576040 shutdown_connections 2024-05-15T14:02:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.399+0000 7fae58c28740 1 --2- 172.21.15.33:0/3754576040 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55567cc7b6f0 0x55567cc7db80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.399+0000 7fae58c28740 1 --2- 172.21.15.33:0/3754576040 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55567cc6c2e0 0x55567cc69750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.399+0000 7fae2effd640 0 client.15567 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:02:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.399+0000 7fae58c28740 1 --2- 172.21.15.33:0/3754576040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567cc61a20 0x55567cc69060 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.399+0000 7fae58c28740 1 --2- 172.21.15.33:0/3754576040 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55567cc61100 0x55567cc6bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.399+0000 7fae58c28740 1 -- 172.21.15.33:0/3754576040 >> 172.21.15.33:0/3754576040 conn(0x55567cc5d610 msgr2=0x55567cc62b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:02:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.399+0000 7fae58c28740 1 -- 172.21.15.33:0/3754576040 shutdown_connections 2024-05-15T14:02:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.399+0000 7fae58c28740 1 -- 172.21.15.33:0/3754576040 wait complete. 2024-05-15T14:02:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.399+0000 7fae58c28740 20 client.15567 trim_cache size 0 max 16384 2024-05-15T14:02:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.430+0000 7fae58c28740 1 Processor -- start 2024-05-15T14:02:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.430+0000 7fae58c28740 1 -- start start 2024-05-15T14:02:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.430+0000 7fae58c28740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567cc61100 0x55567cc77ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.431+0000 7fae4f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567cc61100 0x55567cc77ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.431+0000 7fae4f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567cc61100 0x55567cc77ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57130/0 (socket says 172.21.15.33:57130) 2024-05-15T14:02:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.431+0000 7fae4f7fe640 1 -- 172.21.15.33:0/307382218 learned_addr learned my addr 172.21.15.33:0/307382218 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:02:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.431+0000 7fae58c28740 1 --2- 172.21.15.33:0/307382218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55567cc61a20 0x55567cc681e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.431+0000 7fae58c28740 1 --2- 172.21.15.33:0/307382218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55567cc6c2e0 0x55567cc83440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.431+0000 7fae58c28740 1 -- 172.21.15.33:0/307382218 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fae501aeb50 con 0x55567cc61100 2024-05-15T14:02:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.431+0000 7fae58c28740 1 -- 172.21.15.33:0/307382218 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fae501ac420 con 0x55567cc61a20 2024-05-15T14:02:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.431+0000 7fae58c28740 1 -- 172.21.15.33:0/307382218 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fae501aa270 con 0x55567cc6c2e0 2024-05-15T14:02:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.431+0000 7fae4effd640 1 --2- 172.21.15.33:0/307382218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55567cc61a20 0x55567cc681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.431+0000 7fae4ffff640 1 --2- 172.21.15.33:0/307382218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55567cc6c2e0 0x55567cc83440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.431+0000 7fae4f7fe640 1 -- 172.21.15.33:0/307382218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55567cc6c2e0 msgr2=0x55567cc83440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.431+0000 7fae4f7fe640 1 --2- 172.21.15.33:0/307382218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55567cc6c2e0 0x55567cc83440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.432+0000 7fae4f7fe640 1 -- 172.21.15.33:0/307382218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55567cc61a20 msgr2=0x55567cc681e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.432+0000 7fae4f7fe640 1 --2- 172.21.15.33:0/307382218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55567cc61a20 0x55567cc681e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.432+0000 7fae4f7fe640 1 -- 172.21.15.33:0/307382218 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55567cc50830 con 0x55567cc61100 2024-05-15T14:02:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.432+0000 7fae4f7fe640 1 --2- 172.21.15.33:0/307382218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567cc61100 0x55567cc77ba0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fae40011a30 tx=0x7fae40005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b11a8e27cb0b1e server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:02:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.432+0000 7fae357fa640 1 -- 172.21.15.33:0/307382218 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4000fac0 con 0x55567cc61100 2024-05-15T14:02:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.432+0000 7fae357fa640 1 -- 172.21.15.33:0/307382218 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae40024030 con 0x55567cc61100 2024-05-15T14:02:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.432+0000 7fae357fa640 1 -- 172.21.15.33:0/307382218 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae40004bc0 con 0x55567cc61100 2024-05-15T14:02:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.433+0000 7fae58c28740 1 -- 172.21.15.33:0/307382218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567cc61100 msgr2=0x55567cc77ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.433+0000 7fae58c28740 1 --2- 172.21.15.33:0/307382218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567cc61100 0x55567cc77ba0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fae40011a30 tx=0x7fae40005f90 comp rx=0 tx=0).stop 2024-05-15T14:02:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.433+0000 7fae58c28740 1 -- 172.21.15.33:0/307382218 shutdown_connections 2024-05-15T14:02:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.434+0000 7fae58c28740 1 --2- 172.21.15.33:0/307382218 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55567cc6c2e0 0x55567cc83440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.434+0000 7fae58c28740 1 --2- 172.21.15.33:0/307382218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55567cc61a20 0x55567cc681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.434+0000 7fae58c28740 1 --2- 172.21.15.33:0/307382218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567cc61100 0x55567cc77ba0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.434+0000 7fae58c28740 1 -- 172.21.15.33:0/307382218 >> 172.21.15.33:0/307382218 conn(0x55567cc5d610 msgr2=0x55567cc5d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:02:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.434+0000 7fae58c28740 1 -- 172.21.15.33:0/307382218 shutdown_connections 2024-05-15T14:02:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.434+0000 7fae58c28740 1 -- 172.21.15.33:0/307382218 wait complete. 2024-05-15T14:02:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.434+0000 7fae58c28740 1 -- start start 2024-05-15T14:02:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.435+0000 7fae58c28740 1 Processor -- start 2024-05-15T14:02:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.435+0000 7fae58c28740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55567cc61100 0x55567cc721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.435+0000 7fae58c28740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567cc61a20 0x55567cc72700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.435+0000 7fae58c28740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55567cc6c2e0 0x55567cce5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.435+0000 7fae4effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567cc61a20 0x55567cc72700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.435+0000 7fae4effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567cc61a20 0x55567cc72700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57132/0 (socket says 172.21.15.33:57132) 2024-05-15T14:02:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.435+0000 7fae4f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55567cc61100 0x55567cc721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.436+0000 7fae4f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55567cc61100 0x55567cc721c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:58756/0 (socket says 172.21.15.33:58756) 2024-05-15T14:02:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.436+0000 7fae4effd640 1 -- 172.21.15.33:0/2294513364 learned_addr learned my addr 172.21.15.33:0/2294513364 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:02:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.436+0000 7fae58c28740 1 -- 172.21.15.33:0/2294513364 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fae501aa8e0 con 0x55567cc61a20 2024-05-15T14:02:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.436+0000 7fae4ffff640 1 --2- 172.21.15.33:0/2294513364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55567cc6c2e0 0x55567cce5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.436+0000 7fae58c28740 1 -- 172.21.15.33:0/2294513364 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fae501abd50 con 0x55567cc6c2e0 2024-05-15T14:02:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.436+0000 7fae58c28740 1 -- 172.21.15.33:0/2294513364 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fae501aafb0 con 0x55567cc61100 2024-05-15T14:02:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.436+0000 7fae58c28740 10 client.0 _ll_register_callbacks cb 0x7fae440c6650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:02:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.436+0000 7fae4effd640 1 -- 172.21.15.33:0/2294513364 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55567cc61100 msgr2=0x55567cc721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.436+0000 7fae4effd640 1 --2- 172.21.15.33:0/2294513364 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55567cc61100 0x55567cc721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.436+0000 7fae4effd640 1 -- 172.21.15.33:0/2294513364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55567cc6c2e0 msgr2=0x55567cce5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.436+0000 7fae4effd640 1 --2- 172.21.15.33:0/2294513364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55567cc6c2e0 0x55567cce5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.436+0000 7fae4effd640 1 -- 172.21.15.33:0/2294513364 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55567cce5540 con 0x55567cc61a20 2024-05-15T14:02:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.436+0000 7fae4effd640 1 --2- 172.21.15.33:0/2294513364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567cc61a20 0x55567cc72700 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fae440ca350 tx=0x7fae4402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b645b9c1de38c03c server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:02:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.436+0000 7fae2effd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:02:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.437+0000 7fae2effd640 1 -- 172.21.15.33:0/2294513364 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae440a4c90 con 0x55567cc61a20 2024-05-15T14:02:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.437+0000 7fae2effd640 1 -- 172.21.15.33:0/2294513364 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fae440c9620 con 0x55567cc61a20 2024-05-15T14:02:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.437+0000 7fae58c28740 1 -- 172.21.15.33:0/2294513364 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae40024030 con 0x55567cc61a20 2024-05-15T14:02:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.437+0000 7fae2effd640 1 -- 172.21.15.33:0/2294513364 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fae440c6700 con 0x55567cc61a20 2024-05-15T14:02:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.437+0000 7fae58c28740 10 client.15573 fetch_fsmap learned FSMap version 24 2024-05-15T14:02:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.437+0000 7fae58c28740 1 -- 172.21.15.33:0/2294513364 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55567cce5540 con 0x55567cc61a20 2024-05-15T14:02:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.437+0000 7fae2effd640 1 -- 172.21.15.33:0/2294513364 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae40024030 con 0x55567cc61a20 2024-05-15T14:02:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.438+0000 7fae2effd640 1 -- 172.21.15.33:0/2294513364 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fae440c9090 con 0x55567cc61a20 2024-05-15T14:02:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.438+0000 7fae58c28740 10 client.15573 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:02:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.438+0000 7fae58c28740 10 client.15573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:02:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.438+0000 7fae58c28740 20 client.15573 populate_metadata read hostname 'smithi033' 2024-05-15T14:02:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.438+0000 7fae58c28740 1 --2- 172.21.15.33:0/2294513364 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55567cc7b6f0 0x55567cce8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.438+0000 7fae58c28740 4 client.15573 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:02:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.438+0000 7fae58c28740 1 -- 172.21.15.33:0/2294513364 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55567cbc7360 con 0x55567cc7b6f0 2024-05-15T14:02:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.439+0000 7fae4f7fe640 1 --2- 172.21.15.33:0/2294513364 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55567cc7b6f0 0x55567cce8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.439+0000 7fae4f7fe640 1 --2- 172.21.15.33:0/2294513364 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55567cc7b6f0 0x55567cce8a70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f4864ff369c6282 server_cookie=9afee4ef3cd72643 in_seq=0 out_seq=0 2024-05-15T14:02:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.439+0000 7fae2effd640 10 client.15573 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:02:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.446+0000 7fae2effd640 1 -- 172.21.15.33:0/2294513364 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55567cbc7360 con 0x55567cc7b6f0 2024-05-15T14:02:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.446+0000 7fae2effd640 10 client.15573 handle_command_reply: tid=0 2024-05-15T14:02:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.446+0000 7fae58c28740 1 client.15573 shutdown 2024-05-15T14:02:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.446+0000 7fae58c28740 1 -- 172.21.15.33:0/2294513364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567cc61a20 msgr2=0x55567cc72700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.446+0000 7fae58c28740 1 --2- 172.21.15.33:0/2294513364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567cc61a20 0x55567cc72700 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fae440ca350 tx=0x7fae4402a420 comp rx=0 tx=0).stop 2024-05-15T14:02:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.446+0000 7fae58c28740 1 -- 172.21.15.33:0/2294513364 shutdown_connections 2024-05-15T14:02:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.446+0000 7fae58c28740 1 --2- 172.21.15.33:0/2294513364 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55567cc7b6f0 0x55567cce8a70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.446+0000 7fae58c28740 1 --2- 172.21.15.33:0/2294513364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55567cc6c2e0 0x55567cce5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.447+0000 7fae58c28740 1 --2- 172.21.15.33:0/2294513364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55567cc61a20 0x55567cc72700 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.447+0000 7fae58c28740 1 --2- 172.21.15.33:0/2294513364 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55567cc61100 0x55567cc721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.447+0000 7fae58c28740 1 -- 172.21.15.33:0/2294513364 >> 172.21.15.33:0/2294513364 conn(0x55567cc5d610 msgr2=0x55567cc76b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:02:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.447+0000 7fae2effd640 0 client.15573 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:02:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.447+0000 7fae58c28740 1 -- 172.21.15.33:0/2294513364 shutdown_connections 2024-05-15T14:02:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.447+0000 7fae58c28740 1 -- 172.21.15.33:0/2294513364 wait complete. 2024-05-15T14:02:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.447+0000 7fae58c28740 20 client.15573 trim_cache size 0 max 16384 2024-05-15T14:02:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:02:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:02:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:02:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.448+0000 7fae55e2a640 1 -- 172.21.15.33:0/4250137482 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fae380b2a50 msgr2=0x7fae380b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.448+0000 7fae55e2a640 1 --2- 172.21.15.33:0/4250137482 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fae380b2a50 0x7fae380b4f10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fae480099a0 tx=0x7fae48009500 comp rx=0 tx=0).stop 2024-05-15T14:02:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.448+0000 7fae55e2a640 1 -- 172.21.15.33:0/4250137482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae501022f0 msgr2=0x7fae50199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.448+0000 7fae55e2a640 1 --2- 172.21.15.33:0/4250137482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae501022f0 0x7fae50199680 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fae44009900 tx=0x7fae44030650 comp rx=0 tx=0).stop 2024-05-15T14:02:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.448+0000 7fae55e2a640 1 -- 172.21.15.33:0/4250137482 shutdown_connections 2024-05-15T14:02:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.448+0000 7fae55e2a640 1 --2- 172.21.15.33:0/4250137482 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fae380b2a50 0x7fae380b4f10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.448+0000 7fae55e2a640 1 --2- 172.21.15.33:0/4250137482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae50107940 0x7fae5019da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.448+0000 7fae55e2a640 1 --2- 172.21.15.33:0/4250137482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae501022f0 0x7fae50199680 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.448+0000 7fae55e2a640 1 --2- 172.21.15.33:0/4250137482 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fae50101930 0x7fae50199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.448+0000 7fae55e2a640 1 -- 172.21.15.33:0/4250137482 >> 172.21.15.33:0/4250137482 conn(0x7fae50078a70 msgr2=0x7fae5006a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:02:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.448+0000 7fae55e2a640 1 -- 172.21.15.33:0/4250137482 shutdown_connections 2024-05-15T14:02:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:09.448+0000 7fae55e2a640 1 -- 172.21.15.33:0/4250137482 wait complete. 2024-05-15T14:02:10.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:10 smithi052 ceph-mon[36190]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-05-15T14:02:10.611 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:10 smithi081 ceph-mon[36434]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-05-15T14:02:10.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:10 smithi033 ceph-mon[31650]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-05-15T14:02:12.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:12 smithi052 ceph-mon[36190]: pgmap v58: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-05-15T14:02:12.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:12 smithi081 ceph-mon[36434]: pgmap v58: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-05-15T14:02:12.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:12 smithi033 ceph-mon[31650]: pgmap v58: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-05-15T14:02:14.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:14 smithi052 ceph-mon[36190]: pgmap v59: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-15T14:02:14.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:14 smithi081 ceph-mon[36434]: pgmap v59: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-15T14:02:14.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:14 smithi033 ceph-mon[31650]: pgmap v59: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-05-15T14:02:16.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:16 smithi052 ceph-mon[36190]: pgmap v60: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 5 op/s 2024-05-15T14:02:16.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:16 smithi081 ceph-mon[36434]: pgmap v60: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 5 op/s 2024-05-15T14:02:16.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:16 smithi033 ceph-mon[31650]: pgmap v60: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 5 op/s 2024-05-15T14:02:18.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:18 smithi052 ceph-mon[36190]: pgmap v61: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-15T14:02:18.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:18 smithi081 ceph-mon[36434]: pgmap v61: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-15T14:02:18.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:18 smithi033 ceph-mon[31650]: pgmap v61: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-15T14:02:20.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:20 smithi052 ceph-mon[36190]: pgmap v62: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-15T14:02:20.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:20 smithi081 ceph-mon[36434]: pgmap v62: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-15T14:02:20.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:20 smithi033 ceph-mon[31650]: pgmap v62: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-15T14:02:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.544+0000 7efe9d437640 1 -- 172.21.15.33:0/970725381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe981045e0 msgr2=0x7efe9810cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.544+0000 7efe9d437640 1 --2- 172.21.15.33:0/970725381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe981045e0 0x7efe9810cac0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7efe80009900 tx=0x7efe8002a480 comp rx=0 tx=0).stop 2024-05-15T14:02:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.544+0000 7efe9d437640 1 -- 172.21.15.33:0/970725381 shutdown_connections 2024-05-15T14:02:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.544+0000 7efe9d437640 1 --2- 172.21.15.33:0/970725381 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe9810d090 0x7efe9810f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.544+0000 7efe9d437640 1 --2- 172.21.15.33:0/970725381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe981045e0 0x7efe9810cac0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.544+0000 7efe9d437640 1 --2- 172.21.15.33:0/970725381 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe98103cc0 0x7efe981040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.544+0000 7efe9d437640 1 -- 172.21.15.33:0/970725381 >> 172.21.15.33:0/970725381 conn(0x7efe98078aa0 msgr2=0x7efe98078eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:02:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.544+0000 7efe9d437640 1 -- 172.21.15.33:0/970725381 shutdown_connections 2024-05-15T14:02:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.544+0000 7efe9d437640 1 -- 172.21.15.33:0/970725381 wait complete. 2024-05-15T14:02:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.545+0000 7efe9d437640 1 Processor -- start 2024-05-15T14:02:22.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.545+0000 7efe9d437640 1 -- start start 2024-05-15T14:02:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.545+0000 7efe9d437640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe98103cc0 0x7efe98101910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.545+0000 7efe9d437640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe981045e0 0x7efe98101e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.545+0000 7efe9d437640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe9810d090 0x7efe98102390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.545+0000 7efe9d437640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efe98112ca0 con 0x7efe98103cc0 2024-05-15T14:02:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.545+0000 7efe9d437640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efe98112b20 con 0x7efe9810d090 2024-05-15T14:02:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.545+0000 7efe9d437640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7efe98112e20 con 0x7efe981045e0 2024-05-15T14:02:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.545+0000 7efe96ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe98103cc0 0x7efe98101910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.545+0000 7efe96ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe98103cc0 0x7efe98101910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47930/0 (socket says 172.21.15.33:47930) 2024-05-15T14:02:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.545+0000 7efe96ffd640 1 -- 172.21.15.33:0/2161625734 learned_addr learned my addr 172.21.15.33:0/2161625734 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:02:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.545+0000 7efe8e7fc640 1 --2- 172.21.15.33:0/2161625734 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe981045e0 0x7efe98101e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.546+0000 7efe977fe640 1 --2- 172.21.15.33:0/2161625734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe9810d090 0x7efe98102390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.546+0000 7efe96ffd640 1 -- 172.21.15.33:0/2161625734 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe981045e0 msgr2=0x7efe98101e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.546+0000 7efe96ffd640 1 --2- 172.21.15.33:0/2161625734 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe981045e0 0x7efe98101e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:22.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.546+0000 7efe96ffd640 1 -- 172.21.15.33:0/2161625734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe9810d090 msgr2=0x7efe98102390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:22.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.546+0000 7efe96ffd640 1 --2- 172.21.15.33:0/2161625734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe9810d090 0x7efe98102390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:22.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.546+0000 7efe96ffd640 1 -- 172.21.15.33:0/2161625734 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe981a7640 con 0x7efe98103cc0 2024-05-15T14:02:22.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.546+0000 7efe96ffd640 1 --2- 172.21.15.33:0/2161625734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe98103cc0 0x7efe98101910 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7efe8800c920 tx=0x7efe8800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:02:22.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.546+0000 7efe94ff9640 1 -- 172.21.15.33:0/2161625734 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe88007c70 con 0x7efe98103cc0 2024-05-15T14:02:22.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.546+0000 7efe9d437640 1 -- 172.21.15.33:0/2161625734 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe981a7930 con 0x7efe98103cc0 2024-05-15T14:02:22.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.546+0000 7efe94ff9640 1 -- 172.21.15.33:0/2161625734 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe88007e10 con 0x7efe98103cc0 2024-05-15T14:02:22.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.546+0000 7efe94ff9640 1 -- 172.21.15.33:0/2161625734 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe88018820 con 0x7efe98103cc0 2024-05-15T14:02:22.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.546+0000 7efe9d437640 1 -- 172.21.15.33:0/2161625734 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe981a7ea0 con 0x7efe98103cc0 2024-05-15T14:02:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.549+0000 7efe94ff9640 1 -- 172.21.15.33:0/2161625734 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7efe8800f450 con 0x7efe98103cc0 2024-05-15T14:02:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.549+0000 7efea022f740 1 Processor -- start 2024-05-15T14:02:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.549+0000 7efea022f740 1 -- start start 2024-05-15T14:02:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.549+0000 7efea022f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588e35d28f0 0x5588e35d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.549+0000 7efea022f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588e35d3210 0x5588e35dd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.549+0000 7efea022f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e35ddad0 0x5588e35dfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.549+0000 7efea022f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5588e3545c30 con 0x5588e35d28f0 2024-05-15T14:02:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.549+0000 7efea022f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5588e35357d0 con 0x5588e35ddad0 2024-05-15T14:02:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.549+0000 7efea022f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5588e3539810 con 0x5588e35d3210 2024-05-15T14:02:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.549+0000 7efe977fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588e35d28f0 0x5588e35d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.549+0000 7efe977fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588e35d28f0 0x5588e35d2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47936/0 (socket says 172.21.15.33:47936) 2024-05-15T14:02:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.549+0000 7efe977fe640 1 -- 172.21.15.33:0/3211316376 learned_addr learned my addr 172.21.15.33:0/3211316376 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:02:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.550+0000 7efe8e7fc640 1 --2- 172.21.15.33:0/3211316376 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e35ddad0 0x5588e35dfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.550+0000 7efe96ffd640 1 --2- 172.21.15.33:0/3211316376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588e35d3210 0x5588e35dd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.550+0000 7efe977fe640 1 -- 172.21.15.33:0/3211316376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588e35d3210 msgr2=0x5588e35dd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:22.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.550+0000 7efe977fe640 1 --2- 172.21.15.33:0/3211316376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588e35d3210 0x5588e35dd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.550+0000 7efe977fe640 1 -- 172.21.15.33:0/3211316376 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e35ddad0 msgr2=0x5588e35dfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.550+0000 7efe977fe640 1 --2- 172.21.15.33:0/3211316376 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e35ddad0 0x5588e35dfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.550+0000 7efe977fe640 1 -- 172.21.15.33:0/3211316376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588e324a210 con 0x5588e35d28f0 2024-05-15T14:02:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.550+0000 7efe96ffd640 1 --2- 172.21.15.33:0/3211316376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588e35d3210 0x5588e35dd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:02:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.550+0000 7efe94ff9640 1 --2- 172.21.15.33:0/2161625734 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7efe840b2980 0x7efe840b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.550+0000 7efe977fe640 1 --2- 172.21.15.33:0/3211316376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588e35d28f0 0x5588e35d2cd0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7efe90009e60 tx=0x7efe90010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b91c0120d6b1922 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:02:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.550+0000 7efe96ffd640 1 --2- 172.21.15.33:0/2161625734 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7efe840b2980 0x7efe840b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.550+0000 7efe8d7fa640 1 -- 172.21.15.33:0/3211316376 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe90016070 con 0x5588e35d28f0 2024-05-15T14:02:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.550+0000 7efe8d7fa640 1 -- 172.21.15.33:0/3211316376 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe90004d20 con 0x5588e35d28f0 2024-05-15T14:02:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.550+0000 7efe8d7fa640 1 -- 172.21.15.33:0/3211316376 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe90012770 con 0x5588e35d28f0 2024-05-15T14:02:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.550+0000 7efe94ff9640 1 -- 172.21.15.33:0/2161625734 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7efe880e2710 con 0x7efe98103cc0 2024-05-15T14:02:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.551+0000 7efea022f740 1 -- 172.21.15.33:0/3211316376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588e35d28f0 msgr2=0x5588e35d2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.551+0000 7efe96ffd640 1 --2- 172.21.15.33:0/2161625734 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7efe840b2980 0x7efe840b4e40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efe880dff20 tx=0x7efe88000bc0 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:02:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.551+0000 7efea022f740 1 --2- 172.21.15.33:0/3211316376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588e35d28f0 0x5588e35d2cd0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7efe90009e60 tx=0x7efe90010740 comp rx=0 tx=0).stop 2024-05-15T14:02:22.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.551+0000 7efea022f740 1 -- 172.21.15.33:0/3211316376 shutdown_connections 2024-05-15T14:02:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.551+0000 7efea022f740 1 --2- 172.21.15.33:0/3211316376 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e35ddad0 0x5588e35dfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.551+0000 7efea022f740 1 --2- 172.21.15.33:0/3211316376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588e35d3210 0x5588e35dd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.551+0000 7efea022f740 1 --2- 172.21.15.33:0/3211316376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588e35d28f0 0x5588e35d2cd0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.551+0000 7efea022f740 1 -- 172.21.15.33:0/3211316376 >> 172.21.15.33:0/3211316376 conn(0x5588e35cee00 msgr2=0x5588e35cf1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:02:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.551+0000 7efea022f740 1 -- 172.21.15.33:0/3211316376 shutdown_connections 2024-05-15T14:02:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.551+0000 7efea022f740 1 -- 172.21.15.33:0/3211316376 wait complete. 2024-05-15T14:02:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.552+0000 7efea022f740 1 -- start start 2024-05-15T14:02:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.552+0000 7efea022f740 1 Processor -- start 2024-05-15T14:02:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.552+0000 7efea022f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e35d28f0 0x5588e35dd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.552+0000 7efea022f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588e35d3210 0x5588e35da6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.552+0000 7efea022f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588e35ddad0 0x5588e35dac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.552+0000 7efea022f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5588e321b790 con 0x5588e35d3210 2024-05-15T14:02:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.552+0000 7efea022f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5588e34f1890 con 0x5588e35d28f0 2024-05-15T14:02:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.552+0000 7efea022f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5588e35e07a0 con 0x5588e35ddad0 2024-05-15T14:02:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.552+0000 7efea022f740 10 client.0 _ll_register_callbacks cb 0x5588e34f4d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:02:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.552+0000 7efe8e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588e35d3210 0x5588e35da6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.552+0000 7efe8e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588e35d3210 0x5588e35da6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47946/0 (socket says 172.21.15.33:47946) 2024-05-15T14:02:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.552+0000 7efe8e7fc640 1 -- 172.21.15.33:0/3660970073 learned_addr learned my addr 172.21.15.33:0/3660970073 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:02:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.552+0000 7efe977fe640 1 --2- 172.21.15.33:0/3660970073 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588e35ddad0 0x5588e35dac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.552+0000 7efe8e7fc640 1 -- 172.21.15.33:0/3660970073 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588e35ddad0 msgr2=0x5588e35dac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.552+0000 7efe8e7fc640 1 --2- 172.21.15.33:0/3660970073 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588e35ddad0 0x5588e35dac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.552+0000 7efe8e7fc640 1 -- 172.21.15.33:0/3660970073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e35d28f0 msgr2=0x5588e35dd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.552+0000 7efe8e7fc640 1 --2- 172.21.15.33:0/3660970073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e35d28f0 0x5588e35dd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.552+0000 7efe8e7fc640 1 -- 172.21.15.33:0/3660970073 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588e3316620 con 0x5588e35d3210 2024-05-15T14:02:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.552+0000 7efe8e7fc640 1 --2- 172.21.15.33:0/3660970073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588e35d3210 0x5588e35da6f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7efe80009900 tx=0x7efe800316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac29dc3f2fd95d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:02:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.552+0000 7efe7e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:02:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.553+0000 7efe7e7fc640 1 -- 172.21.15.33:0/3660970073 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe80035040 con 0x5588e35d3210 2024-05-15T14:02:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.553+0000 7efe7e7fc640 1 -- 172.21.15.33:0/3660970073 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7efe80031d10 con 0x5588e35d3210 2024-05-15T14:02:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.553+0000 7efea022f740 1 -- 172.21.15.33:0/3660970073 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe90004d20 con 0x5588e35d3210 2024-05-15T14:02:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.553+0000 7efe7e7fc640 1 -- 172.21.15.33:0/3660970073 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7efe80004040 con 0x5588e35d3210 2024-05-15T14:02:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.553+0000 7efea022f740 10 client.15585 fetch_fsmap learned FSMap version 24 2024-05-15T14:02:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.553+0000 7efea022f740 1 -- 172.21.15.33:0/3660970073 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5588e35ec110 con 0x5588e35d3210 2024-05-15T14:02:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.553+0000 7efe7e7fc640 1 -- 172.21.15.33:0/3660970073 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe90004d20 con 0x5588e35d3210 2024-05-15T14:02:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.553+0000 7efe7e7fc640 1 -- 172.21.15.33:0/3660970073 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7efe800047e0 con 0x5588e35d3210 2024-05-15T14:02:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.553+0000 7efea022f740 10 client.15585 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:02:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.553+0000 7efea022f740 10 client.15585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:02:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.553+0000 7efea022f740 20 client.15585 populate_metadata read hostname 'smithi033' 2024-05-15T14:02:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.553+0000 7efea022f740 1 --2- 172.21.15.33:0/3660970073 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5588e35ed5d0 0x5588e35efa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.553+0000 7efea022f740 4 client.15585 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:02:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.553+0000 7efea022f740 1 -- 172.21.15.33:0/3660970073 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588e3547b80 con 0x5588e35ed5d0 2024-05-15T14:02:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.554+0000 7efe977fe640 1 --2- 172.21.15.33:0/3660970073 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5588e35ed5d0 0x5588e35efa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.554+0000 7efe7e7fc640 10 client.15585 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:02:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.554+0000 7efe977fe640 1 --2- 172.21.15.33:0/3660970073 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5588e35ed5d0 0x5588e35efa70 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac8b663bb59ebde0 server_cookie=62b5eb67a00dafea in_seq=0 out_seq=0 2024-05-15T14:02:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.555+0000 7efe7e7fc640 1 -- 172.21.15.33:0/3660970073 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5588e3547b80 con 0x5588e35ed5d0 2024-05-15T14:02:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.555+0000 7efe7e7fc640 10 client.15585 handle_command_reply: tid=0 2024-05-15T14:02:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.555+0000 7efea022f740 1 client.15585 shutdown 2024-05-15T14:02:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.555+0000 7efea022f740 1 -- 172.21.15.33:0/3660970073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588e35d3210 msgr2=0x5588e35da6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.555+0000 7efea022f740 1 --2- 172.21.15.33:0/3660970073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588e35d3210 0x5588e35da6f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7efe80009900 tx=0x7efe800316d0 comp rx=0 tx=0).stop 2024-05-15T14:02:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.555+0000 7efea022f740 1 -- 172.21.15.33:0/3660970073 shutdown_connections 2024-05-15T14:02:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.555+0000 7efea022f740 1 --2- 172.21.15.33:0/3660970073 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5588e35ed5d0 0x5588e35efa70 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.555+0000 7efea022f740 1 --2- 172.21.15.33:0/3660970073 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588e35ddad0 0x5588e35dac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.555+0000 7efea022f740 1 --2- 172.21.15.33:0/3660970073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588e35d3210 0x5588e35da6f0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.555+0000 7efe7e7fc640 0 client.15585 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:02:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.555+0000 7efea022f740 1 --2- 172.21.15.33:0/3660970073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e35d28f0 0x5588e35dd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.555+0000 7efea022f740 1 -- 172.21.15.33:0/3660970073 >> 172.21.15.33:0/3660970073 conn(0x5588e35cee00 msgr2=0x5588e35ce100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:02:22.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.555+0000 7efea022f740 1 -- 172.21.15.33:0/3660970073 shutdown_connections 2024-05-15T14:02:22.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.555+0000 7efea022f740 1 -- 172.21.15.33:0/3660970073 wait complete. 2024-05-15T14:02:22.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.555+0000 7efea022f740 20 client.15585 trim_cache size 0 max 16384 2024-05-15T14:02:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.570+0000 7efea022f740 1 Processor -- start 2024-05-15T14:02:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.570+0000 7efea022f740 1 -- start start 2024-05-15T14:02:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.571+0000 7efea022f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e35d28f0 0x5588e35d5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.571+0000 7efea022f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588e35d3210 0x5588e35dcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.571+0000 7efea022f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588e35ddad0 0x5588e35d49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.571+0000 7efea022f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efe981b3030 con 0x5588e35d3210 2024-05-15T14:02:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.571+0000 7efea022f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efe981b0900 con 0x5588e35d28f0 2024-05-15T14:02:22.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.571+0000 7efea022f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7efe981ae750 con 0x5588e35ddad0 2024-05-15T14:02:22.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.571+0000 7efe8e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588e35d3210 0x5588e35dcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:22.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.571+0000 7efe8e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588e35d3210 0x5588e35dcf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47956/0 (socket says 172.21.15.33:47956) 2024-05-15T14:02:22.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.571+0000 7efe8e7fc640 1 -- 172.21.15.33:0/2623064273 learned_addr learned my addr 172.21.15.33:0/2623064273 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:02:22.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.571+0000 7efe977fe640 1 --2- 172.21.15.33:0/2623064273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e35d28f0 0x5588e35d5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:22.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.571+0000 7efe977fe640 1 --2- 172.21.15.33:0/2623064273 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588e35ddad0 0x5588e35d49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:22.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.571+0000 7efe8e7fc640 1 -- 172.21.15.33:0/2623064273 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588e35ddad0 msgr2=0x5588e35d49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:22.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.571+0000 7efe8e7fc640 1 --2- 172.21.15.33:0/2623064273 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588e35ddad0 0x5588e35d49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:22.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.571+0000 7efe8e7fc640 1 -- 172.21.15.33:0/2623064273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e35d28f0 msgr2=0x5588e35d5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:22.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.571+0000 7efe8e7fc640 1 --2- 172.21.15.33:0/2623064273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e35d28f0 0x5588e35d5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:22.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.571+0000 7efe8e7fc640 1 -- 172.21.15.33:0/2623064273 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588e35d3040 con 0x5588e35d3210 2024-05-15T14:02:22.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.571+0000 7efe8e7fc640 1 --2- 172.21.15.33:0/2623064273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588e35d3210 0x5588e35dcf30 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7efe8002a9a0 tx=0x7efe80031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea5fe099cfa93b6a server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:02:22.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.572+0000 7efe8cff9640 1 -- 172.21.15.33:0/2623064273 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe80035020 con 0x5588e35d3210 2024-05-15T14:02:22.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.572+0000 7efe8cff9640 1 -- 172.21.15.33:0/2623064273 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe80041070 con 0x5588e35d3210 2024-05-15T14:02:22.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.572+0000 7efe8cff9640 1 -- 172.21.15.33:0/2623064273 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe80003c30 con 0x5588e35d3210 2024-05-15T14:02:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.572+0000 7efea022f740 1 -- 172.21.15.33:0/2623064273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588e35d3210 msgr2=0x5588e35dcf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.572+0000 7efea022f740 1 --2- 172.21.15.33:0/2623064273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588e35d3210 0x5588e35dcf30 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7efe8002a9a0 tx=0x7efe80031a50 comp rx=0 tx=0).stop 2024-05-15T14:02:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.572+0000 7efea022f740 1 -- 172.21.15.33:0/2623064273 shutdown_connections 2024-05-15T14:02:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.572+0000 7efea022f740 1 --2- 172.21.15.33:0/2623064273 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588e35ddad0 0x5588e35d49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.572+0000 7efea022f740 1 --2- 172.21.15.33:0/2623064273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588e35d3210 0x5588e35dcf30 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.572+0000 7efea022f740 1 --2- 172.21.15.33:0/2623064273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e35d28f0 0x5588e35d5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.572+0000 7efea022f740 1 -- 172.21.15.33:0/2623064273 >> 172.21.15.33:0/2623064273 conn(0x5588e35cee00 msgr2=0x5588e35d42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:02:22.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.573+0000 7efea022f740 1 -- 172.21.15.33:0/2623064273 shutdown_connections 2024-05-15T14:02:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.573+0000 7efea022f740 1 -- 172.21.15.33:0/2623064273 wait complete. 2024-05-15T14:02:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.573+0000 7efea022f740 1 -- start start 2024-05-15T14:02:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.573+0000 7efea022f740 1 Processor -- start 2024-05-15T14:02:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.573+0000 7efea022f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e35d28f0 0x5588e35e1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.573+0000 7efea022f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588e35d3210 0x5588e35e1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.573+0000 7efea022f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588e35ddad0 0x5588e35e2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.573+0000 7efea022f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efe981aedc0 con 0x5588e35d3210 2024-05-15T14:02:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.573+0000 7efea022f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efe981b0230 con 0x5588e35d28f0 2024-05-15T14:02:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.573+0000 7efea022f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7efe981af490 con 0x5588e35ddad0 2024-05-15T14:02:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.573+0000 7efea022f740 10 client.0 _ll_register_callbacks cb 0x7efe80032dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:02:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.573+0000 7efe8e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588e35d3210 0x5588e35e1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.573+0000 7efe8e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588e35d3210 0x5588e35e1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47958/0 (socket says 172.21.15.33:47958) 2024-05-15T14:02:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.573+0000 7efe8e7fc640 1 -- 172.21.15.33:0/152861611 learned_addr learned my addr 172.21.15.33:0/152861611 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:02:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.573+0000 7efe977fe640 1 --2- 172.21.15.33:0/152861611 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e35d28f0 0x5588e35e1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.573+0000 7efe977fe640 1 --2- 172.21.15.33:0/152861611 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588e35ddad0 0x5588e35e2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.573+0000 7efe8e7fc640 1 -- 172.21.15.33:0/152861611 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588e35ddad0 msgr2=0x5588e35e2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.574+0000 7efe8e7fc640 1 --2- 172.21.15.33:0/152861611 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588e35ddad0 0x5588e35e2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.574+0000 7efe8e7fc640 1 -- 172.21.15.33:0/152861611 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e35d28f0 msgr2=0x5588e35e1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.574+0000 7efe8e7fc640 1 --2- 172.21.15.33:0/152861611 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e35d28f0 0x5588e35e1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.574+0000 7efe8e7fc640 1 -- 172.21.15.33:0/152861611 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588e3656f40 con 0x5588e35d3210 2024-05-15T14:02:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.574+0000 7efe977fe640 1 --2- 172.21.15.33:0/152861611 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e35d28f0 0x5588e35e1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:02:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.574+0000 7efe8e7fc640 1 --2- 172.21.15.33:0/152861611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588e35d3210 0x5588e35e1bd0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7efe80004be0 tx=0x7efe80000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3a68724255262e server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:02:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.574+0000 7efe7e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:02:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.574+0000 7efe7e7fc640 1 -- 172.21.15.33:0/152861611 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe80003c60 con 0x5588e35d3210 2024-05-15T14:02:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.574+0000 7efe7e7fc640 1 -- 172.21.15.33:0/152861611 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7efe80003e00 con 0x5588e35d3210 2024-05-15T14:02:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.574+0000 7efea022f740 1 -- 172.21.15.33:0/152861611 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe80041070 con 0x5588e35d3210 2024-05-15T14:02:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.574+0000 7efe7e7fc640 1 -- 172.21.15.33:0/152861611 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7efe800044a0 con 0x5588e35d3210 2024-05-15T14:02:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.574+0000 7efea022f740 10 client.15591 fetch_fsmap learned FSMap version 24 2024-05-15T14:02:22.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.574+0000 7efea022f740 1 -- 172.21.15.33:0/152861611 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5588e3658250 con 0x5588e35d3210 2024-05-15T14:02:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.574+0000 7efe7e7fc640 1 -- 172.21.15.33:0/152861611 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe80041070 con 0x5588e35d3210 2024-05-15T14:02:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.574+0000 7efe7e7fc640 1 -- 172.21.15.33:0/152861611 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7efe80068470 con 0x5588e35d3210 2024-05-15T14:02:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.574+0000 7efea022f740 10 client.15591 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:02:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.574+0000 7efea022f740 10 client.15591 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:02:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.574+0000 7efea022f740 20 client.15591 populate_metadata read hostname 'smithi033' 2024-05-15T14:02:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.575+0000 7efea022f740 1 --2- 172.21.15.33:0/152861611 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5588e36588e0 0x5588e365ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.575+0000 7efea022f740 4 client.15591 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:02:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.575+0000 7efea022f740 1 -- 172.21.15.33:0/152861611 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5588e3541360 con 0x5588e36588e0 2024-05-15T14:02:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.575+0000 7efe977fe640 1 --2- 172.21.15.33:0/152861611 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5588e36588e0 0x5588e365ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.575+0000 7efe977fe640 1 --2- 172.21.15.33:0/152861611 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5588e36588e0 0x5588e365ad00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f3eb06a5c67490d server_cookie=b9bbdac6d143de3b in_seq=0 out_seq=0 2024-05-15T14:02:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.575+0000 7efe7e7fc640 10 client.15591 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:02:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.579+0000 7efe7e7fc640 1 -- 172.21.15.33:0/152861611 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5588e3541360 con 0x5588e36588e0 2024-05-15T14:02:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.579+0000 7efe7e7fc640 10 client.15591 handle_command_reply: tid=0 2024-05-15T14:02:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.579+0000 7efea022f740 1 client.15591 shutdown 2024-05-15T14:02:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.579+0000 7efea022f740 1 -- 172.21.15.33:0/152861611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588e35d3210 msgr2=0x5588e35e1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.579+0000 7efea022f740 1 --2- 172.21.15.33:0/152861611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588e35d3210 0x5588e35e1bd0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7efe80004be0 tx=0x7efe80000bc0 comp rx=0 tx=0).stop 2024-05-15T14:02:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.579+0000 7efea022f740 1 -- 172.21.15.33:0/152861611 shutdown_connections 2024-05-15T14:02:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.579+0000 7efea022f740 1 --2- 172.21.15.33:0/152861611 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5588e36588e0 0x5588e365ad00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.579+0000 7efea022f740 1 --2- 172.21.15.33:0/152861611 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588e35ddad0 0x5588e35e2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.579+0000 7efea022f740 1 --2- 172.21.15.33:0/152861611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588e35d3210 0x5588e35e1bd0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.579+0000 7efe7e7fc640 0 client.15591 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:02:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.579+0000 7efea022f740 1 --2- 172.21.15.33:0/152861611 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588e35d28f0 0x5588e35e1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.579+0000 7efea022f740 1 -- 172.21.15.33:0/152861611 >> 172.21.15.33:0/152861611 conn(0x5588e35cee00 msgr2=0x5588e35e8660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:02:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.579+0000 7efea022f740 1 -- 172.21.15.33:0/152861611 shutdown_connections 2024-05-15T14:02:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.579+0000 7efea022f740 1 -- 172.21.15.33:0/152861611 wait complete. 2024-05-15T14:02:22.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.579+0000 7efea022f740 20 client.15591 trim_cache size 0 max 16384 2024-05-15T14:02:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:02:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:02:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:02:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.580+0000 7efe9d437640 1 -- 172.21.15.33:0/2161625734 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7efe840b2980 msgr2=0x7efe840b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.580+0000 7efe9d437640 1 --2- 172.21.15.33:0/2161625734 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7efe840b2980 0x7efe840b4e40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efe880dff20 tx=0x7efe88000bc0 comp rx=0 tx=0).stop 2024-05-15T14:02:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.580+0000 7efe9d437640 1 -- 172.21.15.33:0/2161625734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe98103cc0 msgr2=0x7efe98101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.580+0000 7efe9d437640 1 --2- 172.21.15.33:0/2161625734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe98103cc0 0x7efe98101910 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7efe8800c920 tx=0x7efe8800cde0 comp rx=0 tx=0).stop 2024-05-15T14:02:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.580+0000 7efe9d437640 1 -- 172.21.15.33:0/2161625734 shutdown_connections 2024-05-15T14:02:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.580+0000 7efe9d437640 1 --2- 172.21.15.33:0/2161625734 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7efe840b2980 0x7efe840b4e40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.580+0000 7efe9d437640 1 --2- 172.21.15.33:0/2161625734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe9810d090 0x7efe98102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.580+0000 7efe9d437640 1 --2- 172.21.15.33:0/2161625734 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe981045e0 0x7efe98101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.580+0000 7efe9d437640 1 --2- 172.21.15.33:0/2161625734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe98103cc0 0x7efe98101910 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.580+0000 7efe9d437640 1 -- 172.21.15.33:0/2161625734 >> 172.21.15.33:0/2161625734 conn(0x7efe98078aa0 msgr2=0x7efe980fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:02:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.580+0000 7efe9d437640 1 -- 172.21.15.33:0/2161625734 shutdown_connections 2024-05-15T14:02:22.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:22.580+0000 7efe9d437640 1 -- 172.21.15.33:0/2161625734 wait complete. 2024-05-15T14:02:22.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:22 smithi052 ceph-mon[36190]: pgmap v63: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-15T14:02:22.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:22 smithi081 ceph-mon[36434]: pgmap v63: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-15T14:02:22.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:22 smithi033 ceph-mon[31650]: pgmap v63: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-15T14:02:23.291 INFO:tasks.workunit.client.0.smithi033.stderr:Updating files: 91% (11235/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-15T14:02:23.870 INFO:tasks.workunit.client.0.smithi033.stderr:Note: switching to '9b63866860f0b71216b2c0a73aedd2e0a8a5c98b'. 2024-05-15T14:02:23.870 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-05-15T14:02:23.870 INFO:tasks.workunit.client.0.smithi033.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-15T14:02:23.870 INFO:tasks.workunit.client.0.smithi033.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-15T14:02:23.871 INFO:tasks.workunit.client.0.smithi033.stderr:state without impacting any branches by switching back to a branch. 2024-05-15T14:02:23.871 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-05-15T14:02:23.871 INFO:tasks.workunit.client.0.smithi033.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-15T14:02:23.871 INFO:tasks.workunit.client.0.smithi033.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-15T14:02:23.871 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-05-15T14:02:23.871 INFO:tasks.workunit.client.0.smithi033.stderr: git switch -c 2024-05-15T14:02:23.871 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-05-15T14:02:23.871 INFO:tasks.workunit.client.0.smithi033.stderr:Or undo this operation with: 2024-05-15T14:02:23.871 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-05-15T14:02:23.871 INFO:tasks.workunit.client.0.smithi033.stderr: git switch - 2024-05-15T14:02:23.871 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-05-15T14:02:23.871 INFO:tasks.workunit.client.0.smithi033.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-15T14:02:23.872 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-05-15T14:02:23.872 INFO:tasks.workunit.client.0.smithi033.stderr:HEAD is now at 9b63866860f qa/quiescer: relax some timing requirements in the quiescer 2024-05-15T14:02:23.874 DEBUG:teuthology.orchestra.run.smithi033:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-15T14:02:23.914 INFO:tasks.workunit.client.0.smithi033.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-15T14:02:23.916 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-15T14:02:23.916 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-15T14:02:24.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:24 smithi052 ceph-mon[36190]: pgmap v64: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-15T14:02:24.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:24 smithi081 ceph-mon[36434]: pgmap v64: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-15T14:02:24.660 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-15T14:02:24.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:24 smithi033 ceph-mon[31650]: pgmap v64: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-15T14:02:24.718 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-15T14:02:24.760 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-15T14:02:24.762 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-15T14:02:24.762 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-15T14:02:24.803 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-15T14:02:24.807 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-15T14:02:24.807 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-15T14:02:24.864 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-05-15T14:02:24.866 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-05-15T14:02:24.866 DEBUG:teuthology.orchestra.run.smithi033:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-05-15T14:02:24.930 INFO:tasks.workunit.client.0.smithi033.stderr:+ dbench 1 2024-05-15T14:02:24.938 INFO:tasks.workunit.client.0.smithi033.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-05-15T14:02:24.938 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-05-15T14:02:24.938 INFO:tasks.workunit.client.0.smithi033.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-05-15T14:02:24.945 INFO:tasks.workunit.client.0.smithi033.stdout:0 of 1 processes prepared for launch 0 sec 2024-05-15T14:02:25.046 INFO:tasks.workunit.client.0.smithi033.stdout:1 of 1 processes prepared for launch 0 sec 2024-05-15T14:02:25.046 INFO:tasks.workunit.client.0.smithi033.stdout:releasing clients 2024-05-15T14:02:26.046 INFO:tasks.workunit.client.0.smithi033.stdout: 1 3379 35.46 MB/sec warmup 1 sec latency 12.600 ms 2024-05-15T14:02:26.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:26 smithi052 ceph-mon[36190]: pgmap v65: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T14:02:26.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:26 smithi081 ceph-mon[36434]: pgmap v65: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T14:02:26.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:26 smithi033 ceph-mon[31650]: pgmap v65: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-15T14:02:27.046 INFO:tasks.workunit.client.0.smithi033.stdout: 1 6324 25.54 MB/sec warmup 2 sec latency 22.891 ms 2024-05-15T14:02:28.046 INFO:tasks.workunit.client.0.smithi033.stdout: 1 9243 21.64 MB/sec warmup 3 sec latency 21.918 ms 2024-05-15T14:02:28.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:28 smithi052 ceph-mon[36190]: pgmap v66: 97 pgs: 97 active+clean; 4.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 338 KiB/s wr, 46 op/s 2024-05-15T14:02:28.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:28 smithi081 ceph-mon[36434]: pgmap v66: 97 pgs: 97 active+clean; 4.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 338 KiB/s wr, 46 op/s 2024-05-15T14:02:28.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:28 smithi033 ceph-mon[31650]: pgmap v66: 97 pgs: 97 active+clean; 4.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 338 KiB/s wr, 46 op/s 2024-05-15T14:02:29.046 INFO:tasks.workunit.client.0.smithi033.stdout: 1 11812 20.09 MB/sec warmup 4 sec latency 21.340 ms 2024-05-15T14:02:30.046 INFO:tasks.workunit.client.0.smithi033.stdout: 1 14762 19.49 MB/sec warmup 5 sec latency 10.519 ms 2024-05-15T14:02:30.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:30 smithi052 ceph-mon[36190]: pgmap v67: 97 pgs: 97 active+clean; 4.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 338 KiB/s wr, 43 op/s 2024-05-15T14:02:30.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:30 smithi081 ceph-mon[36434]: pgmap v67: 97 pgs: 97 active+clean; 4.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 338 KiB/s wr, 43 op/s 2024-05-15T14:02:30.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:30 smithi033 ceph-mon[31650]: pgmap v67: 97 pgs: 97 active+clean; 4.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 338 KiB/s wr, 43 op/s 2024-05-15T14:02:31.046 INFO:tasks.workunit.client.0.smithi033.stdout: 1 18057 19.37 MB/sec warmup 6 sec latency 19.399 ms 2024-05-15T14:02:31.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:31 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:02:31.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:31 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:02:31.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:31 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:02:32.046 INFO:tasks.workunit.client.0.smithi033.stdout: 1 20977 18.56 MB/sec warmup 7 sec latency 85.044 ms 2024-05-15T14:02:32.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:32 smithi052 ceph-mon[36190]: pgmap v68: 97 pgs: 97 active+clean; 4.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 338 KiB/s wr, 43 op/s 2024-05-15T14:02:32.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:32 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:02:32.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:32 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:02:32.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:32 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:02:32.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:32 smithi081 ceph-mon[36434]: pgmap v68: 97 pgs: 97 active+clean; 4.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 338 KiB/s wr, 43 op/s 2024-05-15T14:02:32.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:32 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:02:32.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:32 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:02:32.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:32 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:02:32.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:32 smithi033 ceph-mon[31650]: pgmap v68: 97 pgs: 97 active+clean; 4.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 338 KiB/s wr, 43 op/s 2024-05-15T14:02:32.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:32 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:02:32.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:32 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:02:32.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:32 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:02:33.046 INFO:tasks.workunit.client.0.smithi033.stdout: 1 23673 17.83 MB/sec warmup 8 sec latency 27.478 ms 2024-05-15T14:02:34.046 INFO:tasks.workunit.client.0.smithi033.stdout: 1 26095 17.44 MB/sec warmup 9 sec latency 12.265 ms 2024-05-15T14:02:34.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:34 smithi052 ceph-mon[36190]: pgmap v69: 97 pgs: 97 active+clean; 12 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 1.0 MiB/s wr, 128 op/s 2024-05-15T14:02:34.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:34 smithi081 ceph-mon[36434]: pgmap v69: 97 pgs: 97 active+clean; 12 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 1.0 MiB/s wr, 128 op/s 2024-05-15T14:02:34.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:34 smithi033 ceph-mon[31650]: pgmap v69: 97 pgs: 97 active+clean; 12 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 1.0 MiB/s wr, 128 op/s 2024-05-15T14:02:35.046 INFO:tasks.workunit.client.0.smithi033.stdout: 1 29185 17.44 MB/sec warmup 10 sec latency 12.069 ms 2024-05-15T14:02:35.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.675+0000 7f42fc8c8640 1 -- 172.21.15.33:0/3048943895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f8105870 msgr2=0x7f42f81098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.675+0000 7f42fc8c8640 1 --2- 172.21.15.33:0/3048943895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f8105870 0x7f42f81098b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f42e0009900 tx=0x7f42e002a410 comp rx=0 tx=0).stop 2024-05-15T14:02:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.675+0000 7f42fc8c8640 1 -- 172.21.15.33:0/3048943895 shutdown_connections 2024-05-15T14:02:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.675+0000 7f42fc8c8640 1 --2- 172.21.15.33:0/3048943895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42f8109f90 0x7f42f8111b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:35.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.675+0000 7f42fc8c8640 1 --2- 172.21.15.33:0/3048943895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f8105870 0x7f42f81098b0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.675+0000 7f42fc8c8640 1 --2- 172.21.15.33:0/3048943895 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42f8104ec0 0x7f42f81052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.675+0000 7f42fc8c8640 1 -- 172.21.15.33:0/3048943895 >> 172.21.15.33:0/3048943895 conn(0x7f42f8100740 msgr2=0x7f42f8102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:02:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.675+0000 7f42fc8c8640 1 -- 172.21.15.33:0/3048943895 shutdown_connections 2024-05-15T14:02:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.675+0000 7f42fc8c8640 1 -- 172.21.15.33:0/3048943895 wait complete. 2024-05-15T14:02:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.676+0000 7f42fc8c8640 1 Processor -- start 2024-05-15T14:02:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.676+0000 7f42fc8c8640 1 -- start start 2024-05-15T14:02:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.676+0000 7f42fc8c8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f8104ec0 0x7f42f819b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.676+0000 7f42fc8c8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42f8109f90 0x7f42f819ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.676+0000 7f42fc8c8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42f819fd90 0x7f42f81a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.676+0000 7f42fc8c8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f42f8114ef0 con 0x7f42f8104ec0 2024-05-15T14:02:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.676+0000 7f42fc8c8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f42f8114d70 con 0x7f42f819fd90 2024-05-15T14:02:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.676+0000 7f42fc8c8640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f42f8115070 con 0x7f42f8109f90 2024-05-15T14:02:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.676+0000 7f42f659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f8104ec0 0x7f42f819b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.677+0000 7f42f5d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42f8109f90 0x7f42f819ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.677+0000 7f42f659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f8104ec0 0x7f42f819b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46080/0 (socket says 172.21.15.33:46080) 2024-05-15T14:02:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.677+0000 7f42f659c640 1 -- 172.21.15.33:0/3441176392 learned_addr learned my addr 172.21.15.33:0/3441176392 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:02:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.677+0000 7f42f6d9d640 1 --2- 172.21.15.33:0/3441176392 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42f819fd90 0x7f42f81a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.677+0000 7f42f659c640 1 -- 172.21.15.33:0/3441176392 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42f8109f90 msgr2=0x7f42f819ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.677+0000 7f42f659c640 1 --2- 172.21.15.33:0/3441176392 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42f8109f90 0x7f42f819ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.677+0000 7f42f659c640 1 -- 172.21.15.33:0/3441176392 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42f819fd90 msgr2=0x7f42f81a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:35.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.677+0000 7f42f659c640 1 --2- 172.21.15.33:0/3441176392 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42f819fd90 0x7f42f81a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.677+0000 7f42f659c640 1 -- 172.21.15.33:0/3441176392 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f81a0880 con 0x7f42f8104ec0 2024-05-15T14:02:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.677+0000 7f42f5d9b640 1 --2- 172.21.15.33:0/3441176392 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42f8109f90 0x7f42f819ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:02:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.677+0000 7f42f659c640 1 --2- 172.21.15.33:0/3441176392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f8104ec0 0x7f42f819b4d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f42e800eb90 tx=0x7f42e800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:02:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.677+0000 7f42ef7fe640 1 -- 172.21.15.33:0/3441176392 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e8004d20 con 0x7f42f8104ec0 2024-05-15T14:02:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.677+0000 7f42fc8c8640 1 -- 172.21.15.33:0/3441176392 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42f81a0b70 con 0x7f42f8104ec0 2024-05-15T14:02:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.677+0000 7f42ef7fe640 1 -- 172.21.15.33:0/3441176392 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42e800ee90 con 0x7f42f8104ec0 2024-05-15T14:02:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.677+0000 7f42ef7fe640 1 -- 172.21.15.33:0/3441176392 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e8010670 con 0x7f42f8104ec0 2024-05-15T14:02:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.677+0000 7f42fc8c8640 1 -- 172.21.15.33:0/3441176392 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42f81a8330 con 0x7f42f8104ec0 2024-05-15T14:02:35.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.680+0000 7f42ef7fe640 1 -- 172.21.15.33:0/3441176392 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f42e8005020 con 0x7f42f8104ec0 2024-05-15T14:02:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.680+0000 7f42ff5dd740 1 Processor -- start 2024-05-15T14:02:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.680+0000 7f42ff5dd740 1 -- start start 2024-05-15T14:02:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.681+0000 7f42ff5dd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f2e158f0 0x5613f2e15cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.681+0000 7f42ff5dd740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5613f2e16210 0x5613f2e20590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.681+0000 7f42ff5dd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5613f2e20ad0 0x5613f2e22ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.681+0000 7f42ff5dd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5613f2d88c30 con 0x5613f2e158f0 2024-05-15T14:02:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.681+0000 7f42ff5dd740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5613f2d787d0 con 0x5613f2e20ad0 2024-05-15T14:02:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.681+0000 7f42ff5dd740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5613f2d7c810 con 0x5613f2e16210 2024-05-15T14:02:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.681+0000 7f42f6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f2e158f0 0x5613f2e15cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.681+0000 7f42f659c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5613f2e16210 0x5613f2e20590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.681+0000 7f42f6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f2e158f0 0x5613f2e15cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46094/0 (socket says 172.21.15.33:46094) 2024-05-15T14:02:35.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.681+0000 7f42f6d9d640 1 -- 172.21.15.33:0/2945802389 learned_addr learned my addr 172.21.15.33:0/2945802389 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:02:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.681+0000 7f42ef7fe640 1 --2- 172.21.15.33:0/3441176392 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f42e40b2950 0x7f42e40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.681+0000 7f42f5d9b640 1 --2- 172.21.15.33:0/2945802389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5613f2e20ad0 0x5613f2e22ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.681+0000 7f42f6d9d640 1 -- 172.21.15.33:0/2945802389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5613f2e16210 msgr2=0x5613f2e20590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.681+0000 7f42f6d9d640 1 --2- 172.21.15.33:0/2945802389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5613f2e16210 0x5613f2e20590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.681+0000 7f42f6d9d640 1 -- 172.21.15.33:0/2945802389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5613f2e20ad0 msgr2=0x5613f2e22ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.681+0000 7f42f6d9d640 1 --2- 172.21.15.33:0/2945802389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5613f2e20ad0 0x5613f2e22ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.681+0000 7f42f6d9d640 1 -- 172.21.15.33:0/2945802389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613f2a8d210 con 0x5613f2e158f0 2024-05-15T14:02:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.681+0000 7f42ef7fe640 1 -- 172.21.15.33:0/3441176392 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f42e80e2c20 con 0x7f42f8104ec0 2024-05-15T14:02:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.681+0000 7f42f6d9d640 1 --2- 172.21.15.33:0/3441176392 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f42e40b2950 0x7f42e40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.681+0000 7f42f6d9d640 1 --2- 172.21.15.33:0/2945802389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f2e158f0 0x5613f2e15cd0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f42f0009da0 tx=0x7f42f0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e911e79b08b50503 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:02:35.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.682+0000 7f42ecff9640 1 -- 172.21.15.33:0/2945802389 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f0017050 con 0x5613f2e158f0 2024-05-15T14:02:35.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.682+0000 7f42ecff9640 1 -- 172.21.15.33:0/2945802389 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42f00268f0 con 0x5613f2e158f0 2024-05-15T14:02:35.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.682+0000 7f42ecff9640 1 -- 172.21.15.33:0/2945802389 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f000f6c0 con 0x5613f2e158f0 2024-05-15T14:02:35.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.682+0000 7f42ff5dd740 1 -- 172.21.15.33:0/2945802389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f2e158f0 msgr2=0x5613f2e15cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:35.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.682+0000 7f42ff5dd740 1 --2- 172.21.15.33:0/2945802389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f2e158f0 0x5613f2e15cd0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f42f0009da0 tx=0x7f42f0011810 comp rx=0 tx=0).stop 2024-05-15T14:02:35.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.682+0000 7f42f6d9d640 1 --2- 172.21.15.33:0/3441176392 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f42e40b2950 0x7f42e40b4e10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f42f0023000 tx=0x7f42f0024000 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:02:35.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.682+0000 7f42ff5dd740 1 -- 172.21.15.33:0/2945802389 shutdown_connections 2024-05-15T14:02:35.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.682+0000 7f42ff5dd740 1 --2- 172.21.15.33:0/2945802389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5613f2e20ad0 0x5613f2e22ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:35.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.682+0000 7f42ff5dd740 1 --2- 172.21.15.33:0/2945802389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5613f2e16210 0x5613f2e20590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:35.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.682+0000 7f42ff5dd740 1 --2- 172.21.15.33:0/2945802389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f2e158f0 0x5613f2e15cd0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:35.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.682+0000 7f42ff5dd740 1 -- 172.21.15.33:0/2945802389 >> 172.21.15.33:0/2945802389 conn(0x5613f2e11e00 msgr2=0x5613f2e121e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:02:35.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.683+0000 7f42ff5dd740 1 -- 172.21.15.33:0/2945802389 shutdown_connections 2024-05-15T14:02:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.683+0000 7f42ff5dd740 1 -- 172.21.15.33:0/2945802389 wait complete. 2024-05-15T14:02:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.683+0000 7f42ff5dd740 1 -- start start 2024-05-15T14:02:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.683+0000 7f42ff5dd740 1 Processor -- start 2024-05-15T14:02:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.683+0000 7f42ff5dd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f2e158f0 0x5613f2e20420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:35.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.683+0000 7f42ff5dd740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5613f2e16210 0x5613f2e1d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:35.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.683+0000 7f42ff5dd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5613f2e20ad0 0x5613f2e1dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:35.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.683+0000 7f42ff5dd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5613f2a5e790 con 0x5613f2e158f0 2024-05-15T14:02:35.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.683+0000 7f42ff5dd740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5613f2d34890 con 0x5613f2e20ad0 2024-05-15T14:02:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.683+0000 7f42ff5dd740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5613f2e237a0 con 0x5613f2e16210 2024-05-15T14:02:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.683+0000 7f42ff5dd740 10 client.0 _ll_register_callbacks cb 0x5613f2d37d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:02:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.683+0000 7f42f659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f2e158f0 0x5613f2e20420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.683+0000 7f42f659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f2e158f0 0x5613f2e20420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46104/0 (socket says 172.21.15.33:46104) 2024-05-15T14:02:35.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.683+0000 7f42f659c640 1 -- 172.21.15.33:0/3666163440 learned_addr learned my addr 172.21.15.33:0/3666163440 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:02:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.683+0000 7f42f5d9b640 1 --2- 172.21.15.33:0/3666163440 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5613f2e16210 0x5613f2e1d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.683+0000 7f42f6d9d640 1 --2- 172.21.15.33:0/3666163440 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5613f2e20ad0 0x5613f2e1dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.684+0000 7f42f659c640 1 -- 172.21.15.33:0/3666163440 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5613f2e16210 msgr2=0x5613f2e1d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.684+0000 7f42f659c640 1 --2- 172.21.15.33:0/3666163440 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5613f2e16210 0x5613f2e1d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.684+0000 7f42f659c640 1 -- 172.21.15.33:0/3666163440 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5613f2e20ad0 msgr2=0x5613f2e1dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:35.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.684+0000 7f42f659c640 1 --2- 172.21.15.33:0/3666163440 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5613f2e20ad0 0x5613f2e1dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.684+0000 7f42f659c640 1 -- 172.21.15.33:0/3666163440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613f2b59620 con 0x5613f2e158f0 2024-05-15T14:02:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.684+0000 7f42f6d9d640 1 --2- 172.21.15.33:0/3666163440 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5613f2e20ad0 0x5613f2e1dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:02:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.684+0000 7f42f659c640 1 --2- 172.21.15.33:0/3666163440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f2e158f0 0x5613f2e20420 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f42e80e0980 tx=0x7f42e80e6360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8780c3f70f8260ef server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:02:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.684+0000 7f42ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:02:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.684+0000 7f42ddffb640 1 -- 172.21.15.33:0/3666163440 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e80a9ce0 con 0x5613f2e158f0 2024-05-15T14:02:35.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.684+0000 7f42ddffb640 1 -- 172.21.15.33:0/3666163440 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f42e80b2af0 con 0x5613f2e158f0 2024-05-15T14:02:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.684+0000 7f42ff5dd740 1 -- 172.21.15.33:0/3666163440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42f00268f0 con 0x5613f2e158f0 2024-05-15T14:02:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.684+0000 7f42ddffb640 1 -- 172.21.15.33:0/3666163440 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f42e80a9680 con 0x5613f2e158f0 2024-05-15T14:02:35.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.684+0000 7f42ff5dd740 10 client.15603 fetch_fsmap learned FSMap version 24 2024-05-15T14:02:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.684+0000 7f42ff5dd740 1 -- 172.21.15.33:0/3666163440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5613f2e2f220 con 0x5613f2e158f0 2024-05-15T14:02:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.684+0000 7f42ddffb640 1 -- 172.21.15.33:0/3666163440 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42f00268f0 con 0x5613f2e158f0 2024-05-15T14:02:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.685+0000 7f42ddffb640 1 -- 172.21.15.33:0/3666163440 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f42e80e8030 con 0x5613f2e158f0 2024-05-15T14:02:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.685+0000 7f42ff5dd740 10 client.15603 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:02:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.685+0000 7f42ff5dd740 10 client.15603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:02:35.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.685+0000 7f42ff5dd740 20 client.15603 populate_metadata read hostname 'smithi033' 2024-05-15T14:02:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.685+0000 7f42ff5dd740 1 --2- 172.21.15.33:0/3666163440 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5613f2e30690 0x5613f2e32a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.685+0000 7f42ff5dd740 4 client.15603 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:02:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.685+0000 7f42ff5dd740 1 -- 172.21.15.33:0/3666163440 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613f2d8ab80 con 0x5613f2e30690 2024-05-15T14:02:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.685+0000 7f42f5d9b640 1 --2- 172.21.15.33:0/3666163440 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5613f2e30690 0x5613f2e32a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.685+0000 7f42ddffb640 10 client.15603 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:02:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.685+0000 7f42f5d9b640 1 --2- 172.21.15.33:0/3666163440 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5613f2e30690 0x5613f2e32a80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fba5d000176235e9 server_cookie=7ab1438aaccf6d47 in_seq=0 out_seq=0 2024-05-15T14:02:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.687+0000 7f42ddffb640 1 -- 172.21.15.33:0/3666163440 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5613f2d8ab80 con 0x5613f2e30690 2024-05-15T14:02:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.687+0000 7f42ddffb640 10 client.15603 handle_command_reply: tid=0 2024-05-15T14:02:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.687+0000 7f42ff5dd740 1 client.15603 shutdown 2024-05-15T14:02:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.687+0000 7f42ff5dd740 1 -- 172.21.15.33:0/3666163440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f2e158f0 msgr2=0x5613f2e20420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:35.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.687+0000 7f42ff5dd740 1 --2- 172.21.15.33:0/3666163440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f2e158f0 0x5613f2e20420 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f42e80e0980 tx=0x7f42e80e6360 comp rx=0 tx=0).stop 2024-05-15T14:02:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.687+0000 7f42ff5dd740 1 -- 172.21.15.33:0/3666163440 shutdown_connections 2024-05-15T14:02:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.687+0000 7f42ff5dd740 1 --2- 172.21.15.33:0/3666163440 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5613f2e30690 0x5613f2e32a80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:35.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.688+0000 7f42ff5dd740 1 --2- 172.21.15.33:0/3666163440 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5613f2e20ad0 0x5613f2e1dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:35.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.688+0000 7f42ddffb640 0 client.15603 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:02:35.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.688+0000 7f42ff5dd740 1 --2- 172.21.15.33:0/3666163440 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5613f2e16210 0x5613f2e1d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:35.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.688+0000 7f42ff5dd740 1 --2- 172.21.15.33:0/3666163440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f2e158f0 0x5613f2e20420 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:35.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.688+0000 7f42ff5dd740 1 -- 172.21.15.33:0/3666163440 >> 172.21.15.33:0/3666163440 conn(0x5613f2e11e00 msgr2=0x5613f2e11100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:02:35.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.688+0000 7f42ff5dd740 1 -- 172.21.15.33:0/3666163440 shutdown_connections 2024-05-15T14:02:35.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.688+0000 7f42ff5dd740 1 -- 172.21.15.33:0/3666163440 wait complete. 2024-05-15T14:02:35.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.688+0000 7f42ff5dd740 20 client.15603 trim_cache size 0 max 16384 2024-05-15T14:02:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.704+0000 7f42ff5dd740 1 Processor -- start 2024-05-15T14:02:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.704+0000 7f42ff5dd740 1 -- start start 2024-05-15T14:02:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.704+0000 7f42ff5dd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f2e158f0 0x5613f2e17d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.704+0000 7f42ff5dd740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5613f2e16210 0x5613f2e1ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.704+0000 7f42ff5dd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5613f2e20ad0 0x5613f2e38310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.704+0000 7f42ff5dd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f42f81b31b0 con 0x5613f2e158f0 2024-05-15T14:02:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.704+0000 7f42ff5dd740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f42f81b0a80 con 0x5613f2e20ad0 2024-05-15T14:02:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.704+0000 7f42ff5dd740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f42f81ae8d0 con 0x5613f2e16210 2024-05-15T14:02:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.704+0000 7f42f659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f2e158f0 0x5613f2e17d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.704+0000 7f42f659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f2e158f0 0x5613f2e17d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46112/0 (socket says 172.21.15.33:46112) 2024-05-15T14:02:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.704+0000 7f42f659c640 1 -- 172.21.15.33:0/4230232258 learned_addr learned my addr 172.21.15.33:0/4230232258 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:02:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.704+0000 7f42f5d9b640 1 --2- 172.21.15.33:0/4230232258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5613f2e16210 0x5613f2e1ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.704+0000 7f42f6d9d640 1 --2- 172.21.15.33:0/4230232258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5613f2e20ad0 0x5613f2e38310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:35.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.704+0000 7f42f659c640 1 -- 172.21.15.33:0/4230232258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5613f2e16210 msgr2=0x5613f2e1ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.704+0000 7f42f659c640 1 --2- 172.21.15.33:0/4230232258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5613f2e16210 0x5613f2e1ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.704+0000 7f42f659c640 1 -- 172.21.15.33:0/4230232258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5613f2e20ad0 msgr2=0x5613f2e38310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.704+0000 7f42f659c640 1 --2- 172.21.15.33:0/4230232258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5613f2e20ad0 0x5613f2e38310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.704+0000 7f42f659c640 1 -- 172.21.15.33:0/4230232258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613f2e11c30 con 0x5613f2e158f0 2024-05-15T14:02:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.704+0000 7f42f659c640 1 --2- 172.21.15.33:0/4230232258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f2e158f0 0x5613f2e17d70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f42e80108e0 tx=0x7f42e8009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df06d93079220ec7 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:02:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.705+0000 7f42dffff640 1 -- 172.21.15.33:0/4230232258 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e8089070 con 0x5613f2e158f0 2024-05-15T14:02:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.705+0000 7f42dffff640 1 -- 172.21.15.33:0/4230232258 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42e80a9a20 con 0x5613f2e158f0 2024-05-15T14:02:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.705+0000 7f42dffff640 1 -- 172.21.15.33:0/4230232258 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e80a9860 con 0x5613f2e158f0 2024-05-15T14:02:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.705+0000 7f42ff5dd740 1 -- 172.21.15.33:0/4230232258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f2e158f0 msgr2=0x5613f2e17d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.705+0000 7f42ff5dd740 1 --2- 172.21.15.33:0/4230232258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f2e158f0 0x5613f2e17d70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f42e80108e0 tx=0x7f42e8009200 comp rx=0 tx=0).stop 2024-05-15T14:02:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.705+0000 7f42ff5dd740 1 -- 172.21.15.33:0/4230232258 shutdown_connections 2024-05-15T14:02:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.705+0000 7f42ff5dd740 1 --2- 172.21.15.33:0/4230232258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5613f2e20ad0 0x5613f2e38310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.705+0000 7f42ff5dd740 1 --2- 172.21.15.33:0/4230232258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5613f2e16210 0x5613f2e1ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.705+0000 7f42ff5dd740 1 --2- 172.21.15.33:0/4230232258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f2e158f0 0x5613f2e17d70 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:35.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.705+0000 7f42ff5dd740 1 -- 172.21.15.33:0/4230232258 >> 172.21.15.33:0/4230232258 conn(0x5613f2e11e00 msgr2=0x5613f2e18ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:02:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.705+0000 7f42ff5dd740 1 -- 172.21.15.33:0/4230232258 shutdown_connections 2024-05-15T14:02:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.705+0000 7f42ff5dd740 1 -- 172.21.15.33:0/4230232258 wait complete. 2024-05-15T14:02:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.706+0000 7f42ff5dd740 1 -- start start 2024-05-15T14:02:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.706+0000 7f42ff5dd740 1 Processor -- start 2024-05-15T14:02:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.706+0000 7f42ff5dd740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5613f2e158f0 0x5613f2e244e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.706+0000 7f42ff5dd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f2e16210 0x5613f2e24a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.706+0000 7f42ff5dd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5613f2e20ad0 0x5613f2e24f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.706+0000 7f42ff5dd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f42f81aef40 con 0x5613f2e16210 2024-05-15T14:02:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.706+0000 7f42ff5dd740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f42f81b03b0 con 0x5613f2e20ad0 2024-05-15T14:02:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.706+0000 7f42ff5dd740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f42f81af610 con 0x5613f2e158f0 2024-05-15T14:02:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.706+0000 7f42ff5dd740 10 client.0 _ll_register_callbacks cb 0x7f42e80e6460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:02:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.706+0000 7f42f5d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f2e16210 0x5613f2e24a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.706+0000 7f42f5d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f2e16210 0x5613f2e24a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46114/0 (socket says 172.21.15.33:46114) 2024-05-15T14:02:35.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.706+0000 7f42f5d9b640 1 -- 172.21.15.33:0/2622228619 learned_addr learned my addr 172.21.15.33:0/2622228619 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:02:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.706+0000 7f42f659c640 1 --2- 172.21.15.33:0/2622228619 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5613f2e158f0 0x5613f2e244e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.706+0000 7f42f6d9d640 1 --2- 172.21.15.33:0/2622228619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5613f2e20ad0 0x5613f2e24f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.706+0000 7f42f5d9b640 1 -- 172.21.15.33:0/2622228619 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5613f2e158f0 msgr2=0x5613f2e244e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.706+0000 7f42f5d9b640 1 --2- 172.21.15.33:0/2622228619 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5613f2e158f0 0x5613f2e244e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.706+0000 7f42f5d9b640 1 -- 172.21.15.33:0/2622228619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5613f2e20ad0 msgr2=0x5613f2e24f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.706+0000 7f42f5d9b640 1 --2- 172.21.15.33:0/2622228619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5613f2e20ad0 0x5613f2e24f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.706+0000 7f42f5d9b640 1 -- 172.21.15.33:0/2622228619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613f2e99ec0 con 0x5613f2e16210 2024-05-15T14:02:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.706+0000 7f42f5d9b640 1 --2- 172.21.15.33:0/2622228619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f2e16210 0x5613f2e24a20 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f42e0009900 tx=0x7f42e00043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=323a0d0939745811 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:02:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.706+0000 7f42ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:02:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.707+0000 7f42ddffb640 1 -- 172.21.15.33:0/2622228619 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e0035040 con 0x5613f2e16210 2024-05-15T14:02:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.707+0000 7f42ddffb640 1 -- 172.21.15.33:0/2622228619 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f42e0005450 con 0x5613f2e16210 2024-05-15T14:02:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.707+0000 7f42ff5dd740 1 -- 172.21.15.33:0/2622228619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42e80a9a20 con 0x5613f2e16210 2024-05-15T14:02:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.707+0000 7f42ddffb640 1 -- 172.21.15.33:0/2622228619 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f42e0054050 con 0x5613f2e16210 2024-05-15T14:02:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.707+0000 7f42ff5dd740 10 client.15609 fetch_fsmap learned FSMap version 24 2024-05-15T14:02:35.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.707+0000 7f42ff5dd740 1 -- 172.21.15.33:0/2622228619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5613f2e9b1a0 con 0x5613f2e16210 2024-05-15T14:02:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.707+0000 7f42ddffb640 1 -- 172.21.15.33:0/2622228619 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42e80a9a20 con 0x5613f2e16210 2024-05-15T14:02:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.707+0000 7f42ddffb640 1 -- 172.21.15.33:0/2622228619 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f42e0045620 con 0x5613f2e16210 2024-05-15T14:02:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.707+0000 7f42ff5dd740 10 client.15609 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:02:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.707+0000 7f42ff5dd740 10 client.15609 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:02:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.707+0000 7f42ff5dd740 20 client.15609 populate_metadata read hostname 'smithi033' 2024-05-15T14:02:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.707+0000 7f42ff5dd740 1 --2- 172.21.15.33:0/2622228619 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5613f2e9b9a0 0x5613f2e9ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.707+0000 7f42ff5dd740 4 client.15609 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:02:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.707+0000 7f42ff5dd740 1 -- 172.21.15.33:0/2622228619 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5613f2d84360 con 0x5613f2e9b9a0 2024-05-15T14:02:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.707+0000 7f42f659c640 1 --2- 172.21.15.33:0/2622228619 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5613f2e9b9a0 0x5613f2e9ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.708+0000 7f42ddffb640 10 client.15609 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:02:35.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.708+0000 7f42f659c640 1 --2- 172.21.15.33:0/2622228619 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5613f2e9b9a0 0x5613f2e9ddc0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f467b1ceb5fa1fac server_cookie=1080d6b8b31ecd69 in_seq=0 out_seq=0 2024-05-15T14:02:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.825+0000 7f42ddffb640 1 -- 172.21.15.33:0/2622228619 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5613f2d84360 con 0x5613f2e9b9a0 2024-05-15T14:02:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.825+0000 7f42ddffb640 10 client.15609 handle_command_reply: tid=0 2024-05-15T14:02:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.825+0000 7f42ff5dd740 1 client.15609 shutdown 2024-05-15T14:02:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.825+0000 7f42ff5dd740 1 -- 172.21.15.33:0/2622228619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f2e16210 msgr2=0x5613f2e24a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.825+0000 7f42ff5dd740 1 --2- 172.21.15.33:0/2622228619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f2e16210 0x5613f2e24a20 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f42e0009900 tx=0x7f42e00043b0 comp rx=0 tx=0).stop 2024-05-15T14:02:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.826+0000 7f42ff5dd740 1 -- 172.21.15.33:0/2622228619 shutdown_connections 2024-05-15T14:02:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.826+0000 7f42ff5dd740 1 --2- 172.21.15.33:0/2622228619 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5613f2e9b9a0 0x5613f2e9ddc0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.826+0000 7f42ff5dd740 1 --2- 172.21.15.33:0/2622228619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5613f2e20ad0 0x5613f2e24f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.826+0000 7f42ff5dd740 1 --2- 172.21.15.33:0/2622228619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f2e16210 0x5613f2e24a20 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.826+0000 7f42ddffb640 0 client.15609 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:02:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.826+0000 7f42ff5dd740 1 --2- 172.21.15.33:0/2622228619 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5613f2e158f0 0x5613f2e244e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.826+0000 7f42ff5dd740 1 -- 172.21.15.33:0/2622228619 >> 172.21.15.33:0/2622228619 conn(0x5613f2e11e00 msgr2=0x5613f2e17d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:02:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.826+0000 7f42ff5dd740 1 -- 172.21.15.33:0/2622228619 shutdown_connections 2024-05-15T14:02:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.826+0000 7f42ff5dd740 1 -- 172.21.15.33:0/2622228619 wait complete. 2024-05-15T14:02:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.826+0000 7f42ff5dd740 20 client.15609 trim_cache size 0 max 16384 2024-05-15T14:02:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:02:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:02:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:02:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.826+0000 7f42fc8c8640 1 -- 172.21.15.33:0/3441176392 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f42e40b2950 msgr2=0x7f42e40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.826+0000 7f42fc8c8640 1 --2- 172.21.15.33:0/3441176392 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f42e40b2950 0x7f42e40b4e10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f42f0023000 tx=0x7f42f0024000 comp rx=0 tx=0).stop 2024-05-15T14:02:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.826+0000 7f42fc8c8640 1 -- 172.21.15.33:0/3441176392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f8104ec0 msgr2=0x7f42f819b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.826+0000 7f42fc8c8640 1 --2- 172.21.15.33:0/3441176392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f8104ec0 0x7f42f819b4d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f42e800eb90 tx=0x7f42e800c6a0 comp rx=0 tx=0).stop 2024-05-15T14:02:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.826+0000 7f42fc8c8640 1 -- 172.21.15.33:0/3441176392 shutdown_connections 2024-05-15T14:02:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.826+0000 7f42fc8c8640 1 --2- 172.21.15.33:0/3441176392 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f42e40b2950 0x7f42e40b4e10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.826+0000 7f42fc8c8640 1 --2- 172.21.15.33:0/3441176392 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f42f819fd90 0x7f42f81a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.826+0000 7f42fc8c8640 1 --2- 172.21.15.33:0/3441176392 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42f8109f90 0x7f42f819ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:35.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.826+0000 7f42fc8c8640 1 --2- 172.21.15.33:0/3441176392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f8104ec0 0x7f42f819b4d0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:35.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.826+0000 7f42fc8c8640 1 -- 172.21.15.33:0/3441176392 >> 172.21.15.33:0/3441176392 conn(0x7f42f8100740 msgr2=0x7f42f8101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:02:35.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.826+0000 7f42fc8c8640 1 -- 172.21.15.33:0/3441176392 shutdown_connections 2024-05-15T14:02:35.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:35.826+0000 7f42fc8c8640 1 -- 172.21.15.33:0/3441176392 wait complete. 2024-05-15T14:02:36.046 INFO:tasks.workunit.client.0.smithi033.stdout: 1 32218 17.53 MB/sec warmup 11 sec latency 80.718 ms 2024-05-15T14:02:36.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:36 smithi052 ceph-mon[36190]: pgmap v70: 97 pgs: 97 active+clean; 12 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 1.0 MiB/s wr, 125 op/s 2024-05-15T14:02:36.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:36 smithi081 ceph-mon[36434]: pgmap v70: 97 pgs: 97 active+clean; 12 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 1.0 MiB/s wr, 125 op/s 2024-05-15T14:02:36.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:36 smithi033 ceph-mon[31650]: pgmap v70: 97 pgs: 97 active+clean; 12 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 1.0 MiB/s wr, 125 op/s 2024-05-15T14:02:37.046 INFO:tasks.workunit.client.0.smithi033.stdout: 1 35313 17.29 MB/sec warmup 12 sec latency 14.310 ms 2024-05-15T14:02:38.046 INFO:tasks.workunit.client.0.smithi033.stdout: 1 37892 16.86 MB/sec warmup 13 sec latency 37.276 ms 2024-05-15T14:02:38.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:38 smithi052 ceph-mon[36190]: pgmap v71: 97 pgs: 97 active+clean; 10 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 1.7 MiB/s wr, 313 op/s 2024-05-15T14:02:38.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:38 smithi081 ceph-mon[36434]: pgmap v71: 97 pgs: 97 active+clean; 10 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 1.7 MiB/s wr, 313 op/s 2024-05-15T14:02:38.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:38 smithi033 ceph-mon[31650]: pgmap v71: 97 pgs: 97 active+clean; 10 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 1.7 MiB/s wr, 313 op/s 2024-05-15T14:02:39.046 INFO:tasks.workunit.client.0.smithi033.stdout: 1 40552 16.89 MB/sec warmup 14 sec latency 14.226 ms 2024-05-15T14:02:40.046 INFO:tasks.workunit.client.0.smithi033.stdout: 1 43481 16.74 MB/sec warmup 15 sec latency 13.783 ms 2024-05-15T14:02:40.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:40 smithi052 ceph-mon[36190]: pgmap v72: 97 pgs: 97 active+clean; 10 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 1.4 MiB/s wr, 272 op/s 2024-05-15T14:02:40.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:40 smithi081 ceph-mon[36434]: pgmap v72: 97 pgs: 97 active+clean; 10 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 1.4 MiB/s wr, 272 op/s 2024-05-15T14:02:40.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:40 smithi033 ceph-mon[31650]: pgmap v72: 97 pgs: 97 active+clean; 10 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 1.4 MiB/s wr, 272 op/s 2024-05-15T14:02:41.046 INFO:tasks.workunit.client.0.smithi033.stdout: 1 46616 16.85 MB/sec warmup 16 sec latency 18.035 ms 2024-05-15T14:02:42.047 INFO:tasks.workunit.client.0.smithi033.stdout: 1 49923 16.96 MB/sec warmup 17 sec latency 12.551 ms 2024-05-15T14:02:42.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:42 smithi052 ceph-mon[36190]: pgmap v73: 97 pgs: 97 active+clean; 10 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 1.4 MiB/s wr, 272 op/s 2024-05-15T14:02:42.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:42 smithi081 ceph-mon[36434]: pgmap v73: 97 pgs: 97 active+clean; 10 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 1.4 MiB/s wr, 272 op/s 2024-05-15T14:02:42.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:42 smithi033 ceph-mon[31650]: pgmap v73: 97 pgs: 97 active+clean; 10 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 1.4 MiB/s wr, 272 op/s 2024-05-15T14:02:43.047 INFO:tasks.workunit.client.0.smithi033.stdout: 1 52896 16.78 MB/sec warmup 18 sec latency 15.787 ms 2024-05-15T14:02:44.046 INFO:tasks.workunit.client.0.smithi033.stdout: 1 55606 16.57 MB/sec warmup 19 sec latency 23.877 ms 2024-05-15T14:02:44.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:44 smithi052 ceph-mon[36190]: pgmap v74: 97 pgs: 97 active+clean; 17 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 363 op/s 2024-05-15T14:02:44.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:44 smithi081 ceph-mon[36434]: pgmap v74: 97 pgs: 97 active+clean; 17 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 363 op/s 2024-05-15T14:02:44.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:44 smithi033 ceph-mon[31650]: pgmap v74: 97 pgs: 97 active+clean; 17 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 363 op/s 2024-05-15T14:02:45.047 INFO:tasks.workunit.client.0.smithi033.stdout: 1 57944 16.45 MB/sec warmup 20 sec latency 11.803 ms 2024-05-15T14:02:46.046 INFO:tasks.workunit.client.0.smithi033.stdout: 1 60889 16.48 MB/sec warmup 21 sec latency 10.792 ms 2024-05-15T14:02:46.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:46 smithi052 ceph-mon[36190]: pgmap v75: 97 pgs: 97 active+clean; 17 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 1.4 MiB/s wr, 279 op/s 2024-05-15T14:02:46.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:46 smithi081 ceph-mon[36434]: pgmap v75: 97 pgs: 97 active+clean; 17 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 1.4 MiB/s wr, 279 op/s 2024-05-15T14:02:46.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:46 smithi033 ceph-mon[31650]: pgmap v75: 97 pgs: 97 active+clean; 17 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 1.4 MiB/s wr, 279 op/s 2024-05-15T14:02:47.047 INFO:tasks.workunit.client.0.smithi033.stdout: 1 64082 16.58 MB/sec warmup 22 sec latency 16.925 ms 2024-05-15T14:02:48.047 INFO:tasks.workunit.client.0.smithi033.stdout: 1 66850 16.42 MB/sec warmup 23 sec latency 13.568 ms 2024-05-15T14:02:48.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:48 smithi052 ceph-mon[36190]: pgmap v76: 97 pgs: 97 active+clean; 23 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 366 op/s 2024-05-15T14:02:48.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:48 smithi081 ceph-mon[36434]: pgmap v76: 97 pgs: 97 active+clean; 23 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 366 op/s 2024-05-15T14:02:48.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:48 smithi033 ceph-mon[31650]: pgmap v76: 97 pgs: 97 active+clean; 23 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 366 op/s 2024-05-15T14:02:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.929+0000 7f334f5f0640 1 -- 172.21.15.33:0/2553720421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33481045e0 msgr2=0x7f334810cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.929+0000 7f334f5f0640 1 --2- 172.21.15.33:0/2553720421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33481045e0 0x7f334810cac0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3338009900 tx=0x7f333802a480 comp rx=0 tx=0).stop 2024-05-15T14:02:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.929+0000 7f334f5f0640 1 -- 172.21.15.33:0/2553720421 shutdown_connections 2024-05-15T14:02:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.929+0000 7f334f5f0640 1 --2- 172.21.15.33:0/2553720421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f334810d090 0x7f334810f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.929+0000 7f334f5f0640 1 --2- 172.21.15.33:0/2553720421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33481045e0 0x7f334810cac0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.929+0000 7f334f5f0640 1 --2- 172.21.15.33:0/2553720421 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3348103cc0 0x7f33481040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.929+0000 7f334f5f0640 1 -- 172.21.15.33:0/2553720421 >> 172.21.15.33:0/2553720421 conn(0x7f3348078a70 msgr2=0x7f3348078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:02:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.929+0000 7f334f5f0640 1 -- 172.21.15.33:0/2553720421 shutdown_connections 2024-05-15T14:02:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.929+0000 7f334f5f0640 1 -- 172.21.15.33:0/2553720421 wait complete. 2024-05-15T14:02:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.930+0000 7f334f5f0640 1 Processor -- start 2024-05-15T14:02:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.930+0000 7f334f5f0640 1 -- start start 2024-05-15T14:02:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.930+0000 7f334f5f0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3348103cc0 0x7f3348199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.930+0000 7f334f5f0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33481045e0 0x7f33481996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.930+0000 7f334f5f0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f334810d090 0x7f334819da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.930+0000 7f334f5f0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3348112d20 con 0x7f33481045e0 2024-05-15T14:02:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.930+0000 7f334f5f0640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3348112ba0 con 0x7f3348103cc0 2024-05-15T14:02:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.930+0000 7f334f5f0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3348112ea0 con 0x7f334810d090 2024-05-15T14:02:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.930+0000 7f334cb8b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33481045e0 0x7f33481996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.930+0000 7f334db8d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f334810d090 0x7f334819da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.930+0000 7f334cb8b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33481045e0 0x7f33481996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51848/0 (socket says 172.21.15.33:51848) 2024-05-15T14:02:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.930+0000 7f334cb8b640 1 -- 172.21.15.33:0/474400661 learned_addr learned my addr 172.21.15.33:0/474400661 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:02:48.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.930+0000 7f334d38c640 1 --2- 172.21.15.33:0/474400661 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3348103cc0 0x7f3348199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.930+0000 7f334db8d640 1 -- 172.21.15.33:0/474400661 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3348103cc0 msgr2=0x7f3348199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.930+0000 7f334db8d640 1 --2- 172.21.15.33:0/474400661 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3348103cc0 0x7f3348199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.930+0000 7f334db8d640 1 -- 172.21.15.33:0/474400661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33481045e0 msgr2=0x7f33481996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.930+0000 7f334db8d640 1 --2- 172.21.15.33:0/474400661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33481045e0 0x7f33481996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.931+0000 7f334db8d640 1 -- 172.21.15.33:0/474400661 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f334819e170 con 0x7f334810d090 2024-05-15T14:02:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.931+0000 7f334db8d640 1 --2- 172.21.15.33:0/474400661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f334810d090 0x7f334819da40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f334000e940 tx=0x7f334000ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:02:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.931+0000 7f333e7fc640 1 -- 172.21.15.33:0/474400661 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334000ccc0 con 0x7f334810d090 2024-05-15T14:02:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.931+0000 7f333e7fc640 1 -- 172.21.15.33:0/474400661 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3340005020 con 0x7f334810d090 2024-05-15T14:02:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.931+0000 7f334f5f0640 1 -- 172.21.15.33:0/474400661 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f334819e460 con 0x7f334810d090 2024-05-15T14:02:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.931+0000 7f333e7fc640 1 -- 172.21.15.33:0/474400661 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3340004a00 con 0x7f334810d090 2024-05-15T14:02:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.931+0000 7f334f5f0640 1 -- 172.21.15.33:0/474400661 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33481a5cf0 con 0x7f334810d090 2024-05-15T14:02:48.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.933+0000 7f333e7fc640 1 -- 172.21.15.33:0/474400661 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3340004ba0 con 0x7f334810d090 2024-05-15T14:02:48.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.934+0000 7f33523e5740 1 Processor -- start 2024-05-15T14:02:48.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.934+0000 7f33523e5740 1 -- start start 2024-05-15T14:02:48.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.934+0000 7f33523e5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e5cf8c8f0 0x555e5cf8ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:48.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.934+0000 7f33523e5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555e5cf8d210 0x555e5cf97590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:48.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.934+0000 7f33523e5740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555e5cf97ad0 0x555e5cf99ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:48.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.934+0000 7f33523e5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555e5ceffc30 con 0x555e5cf8c8f0 2024-05-15T14:02:48.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.934+0000 7f33523e5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555e5ceef7d0 con 0x555e5cf8d210 2024-05-15T14:02:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.934+0000 7f33523e5740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x555e5cef3810 con 0x555e5cf97ad0 2024-05-15T14:02:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.934+0000 7f334cb8b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e5cf8c8f0 0x555e5cf8ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.934+0000 7f334d38c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555e5cf8d210 0x555e5cf97590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.934+0000 7f334cb8b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e5cf8c8f0 0x555e5cf8ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51862/0 (socket says 172.21.15.33:51862) 2024-05-15T14:02:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.934+0000 7f334d38c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555e5cf8d210 0x555e5cf97590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:40114/0 (socket says 172.21.15.33:40114) 2024-05-15T14:02:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.934+0000 7f334cb8b640 1 -- 172.21.15.33:0/3118055271 learned_addr learned my addr 172.21.15.33:0/3118055271 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:02:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.934+0000 7f334cb8b640 1 --2- 172.21.15.33:0/3118055271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555e5cf97ad0 0x555e5cf99ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.935+0000 7f334d38c640 1 -- 172.21.15.33:0/3118055271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555e5cf97ad0 msgr2=0x555e5cf99ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.935+0000 7f334d38c640 1 --2- 172.21.15.33:0/3118055271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555e5cf97ad0 0x555e5cf99ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.935+0000 7f334d38c640 1 -- 172.21.15.33:0/3118055271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e5cf8c8f0 msgr2=0x555e5cf8ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.935+0000 7f334d38c640 1 --2- 172.21.15.33:0/3118055271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e5cf8c8f0 0x555e5cf8ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.935+0000 7f334d38c640 1 -- 172.21.15.33:0/3118055271 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e5cc04210 con 0x555e5cf8d210 2024-05-15T14:02:48.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.935+0000 7f333e7fc640 1 --2- 172.21.15.33:0/474400661 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f33340b2950 0x7f33340b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.935+0000 7f334cb8b640 1 --2- 172.21.15.33:0/474400661 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f33340b2950 0x7f33340b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.935+0000 7f334d38c640 1 --2- 172.21.15.33:0/3118055271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555e5cf8d210 0x555e5cf97590 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3344007ed0 tx=0x7f334400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4f0baf8dd45ebb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:02:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.935+0000 7f333e7fc640 1 -- 172.21.15.33:0/474400661 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f33400e24f0 con 0x7f334810d090 2024-05-15T14:02:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.935+0000 7f33337fe640 1 -- 172.21.15.33:0/3118055271 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3344004d20 con 0x555e5cf8d210 2024-05-15T14:02:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.935+0000 7f33337fe640 1 -- 172.21.15.33:0/3118055271 <== mon.2 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3344010af0 con 0x555e5cf8d210 2024-05-15T14:02:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.935+0000 7f33337fe640 1 -- 172.21.15.33:0/3118055271 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3344005690 con 0x555e5cf8d210 2024-05-15T14:02:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.936+0000 7f33523e5740 1 -- 172.21.15.33:0/3118055271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555e5cf8d210 msgr2=0x555e5cf97590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.936+0000 7f334cb8b640 1 --2- 172.21.15.33:0/474400661 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f33340b2950 0x7f33340b4e10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x555e5cf8a220 tx=0x7f3338005a20 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:02:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.936+0000 7f33523e5740 1 --2- 172.21.15.33:0/3118055271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555e5cf8d210 0x555e5cf97590 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3344007ed0 tx=0x7f334400f790 comp rx=0 tx=0).stop 2024-05-15T14:02:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.936+0000 7f33523e5740 1 -- 172.21.15.33:0/3118055271 shutdown_connections 2024-05-15T14:02:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.936+0000 7f33523e5740 1 --2- 172.21.15.33:0/3118055271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555e5cf97ad0 0x555e5cf99ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.936+0000 7f33523e5740 1 --2- 172.21.15.33:0/3118055271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555e5cf8d210 0x555e5cf97590 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.936+0000 7f33523e5740 1 --2- 172.21.15.33:0/3118055271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e5cf8c8f0 0x555e5cf8ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.936+0000 7f33523e5740 1 -- 172.21.15.33:0/3118055271 >> 172.21.15.33:0/3118055271 conn(0x555e5cf88e00 msgr2=0x555e5cf891e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:02:48.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.936+0000 7f33523e5740 1 -- 172.21.15.33:0/3118055271 shutdown_connections 2024-05-15T14:02:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.936+0000 7f33523e5740 1 -- 172.21.15.33:0/3118055271 wait complete. 2024-05-15T14:02:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.936+0000 7f33523e5740 1 -- start start 2024-05-15T14:02:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.936+0000 7f33523e5740 1 Processor -- start 2024-05-15T14:02:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.937+0000 7f33523e5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555e5cf8c8f0 0x555e5cf973c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.937+0000 7f33523e5740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555e5cf8d210 0x555e5cf946b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.937+0000 7f33523e5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e5cf97ad0 0x555e5cf94bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.937+0000 7f33523e5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555e5cbd5790 con 0x555e5cf97ad0 2024-05-15T14:02:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.937+0000 7f33523e5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555e5ceab890 con 0x555e5cf8c8f0 2024-05-15T14:02:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.937+0000 7f33523e5740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x555e5cf9a7a0 con 0x555e5cf8d210 2024-05-15T14:02:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.937+0000 7f33523e5740 10 client.0 _ll_register_callbacks cb 0x555e5ceaed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:02:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.937+0000 7f334db8d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e5cf97ad0 0x555e5cf94bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.937+0000 7f334cb8b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555e5cf8d210 0x555e5cf946b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.937+0000 7f334db8d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e5cf97ad0 0x555e5cf94bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51878/0 (socket says 172.21.15.33:51878) 2024-05-15T14:02:48.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.937+0000 7f334db8d640 1 -- 172.21.15.33:0/1333660508 learned_addr learned my addr 172.21.15.33:0/1333660508 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:02:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.937+0000 7f334d38c640 1 --2- 172.21.15.33:0/1333660508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555e5cf8c8f0 0x555e5cf973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.937+0000 7f334db8d640 1 -- 172.21.15.33:0/1333660508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555e5cf8d210 msgr2=0x555e5cf946b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.937+0000 7f334db8d640 1 --2- 172.21.15.33:0/1333660508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555e5cf8d210 0x555e5cf946b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.937+0000 7f334db8d640 1 -- 172.21.15.33:0/1333660508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555e5cf8c8f0 msgr2=0x555e5cf973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.937+0000 7f334db8d640 1 --2- 172.21.15.33:0/1333660508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555e5cf8c8f0 0x555e5cf973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.937+0000 7f334db8d640 1 -- 172.21.15.33:0/1333660508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e5ccd0620 con 0x555e5cf97ad0 2024-05-15T14:02:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.937+0000 7f334db8d640 1 --2- 172.21.15.33:0/1333660508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e5cf97ad0 0x555e5cf94bf0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3340009750 tx=0x7f3340005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fbb39451a6d9134 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:02:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.937+0000 7f3330ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:02:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.938+0000 7f3330ff9640 1 -- 172.21.15.33:0/1333660508 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334001c070 con 0x555e5cf97ad0 2024-05-15T14:02:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.938+0000 7f3330ff9640 1 -- 172.21.15.33:0/1333660508 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f33400910e0 con 0x555e5cf97ad0 2024-05-15T14:02:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.938+0000 7f33523e5740 1 -- 172.21.15.33:0/1333660508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3344010af0 con 0x555e5cf97ad0 2024-05-15T14:02:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.938+0000 7f3330ff9640 1 -- 172.21.15.33:0/1333660508 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3340091320 con 0x555e5cf97ad0 2024-05-15T14:02:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.938+0000 7f33523e5740 10 client.15621 fetch_fsmap learned FSMap version 24 2024-05-15T14:02:48.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.938+0000 7f33523e5740 1 -- 172.21.15.33:0/1333660508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555e5cfa61b0 con 0x555e5cf97ad0 2024-05-15T14:02:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.938+0000 7f3330ff9640 1 -- 172.21.15.33:0/1333660508 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3344010af0 con 0x555e5cf97ad0 2024-05-15T14:02:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.938+0000 7f3330ff9640 1 -- 172.21.15.33:0/1333660508 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f33400bebd0 con 0x555e5cf97ad0 2024-05-15T14:02:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.938+0000 7f33523e5740 10 client.15621 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:02:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.938+0000 7f33523e5740 10 client.15621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:02:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.938+0000 7f33523e5740 20 client.15621 populate_metadata read hostname 'smithi033' 2024-05-15T14:02:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.938+0000 7f33523e5740 1 --2- 172.21.15.33:0/1333660508 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x555e5cfa7670 0x555e5cfa9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.938+0000 7f33523e5740 4 client.15621 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:02:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.938+0000 7f33523e5740 1 -- 172.21.15.33:0/1333660508 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e5cf01b80 con 0x555e5cfa7670 2024-05-15T14:02:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.938+0000 7f334d38c640 1 --2- 172.21.15.33:0/1333660508 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x555e5cfa7670 0x555e5cfa9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.939+0000 7f334d38c640 1 --2- 172.21.15.33:0/1333660508 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x555e5cfa7670 0x555e5cfa9a60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2443c8372b0cad65 server_cookie=38319e56225ec572 in_seq=0 out_seq=0 2024-05-15T14:02:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.939+0000 7f3330ff9640 10 client.15621 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:02:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.939+0000 7f3330ff9640 1 -- 172.21.15.33:0/1333660508 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555e5cf01b80 con 0x555e5cfa7670 2024-05-15T14:02:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.939+0000 7f3330ff9640 10 client.15621 handle_command_reply: tid=0 2024-05-15T14:02:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.939+0000 7f33523e5740 1 client.15621 shutdown 2024-05-15T14:02:48.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.940+0000 7f33523e5740 1 -- 172.21.15.33:0/1333660508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e5cf97ad0 msgr2=0x555e5cf94bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.940+0000 7f33523e5740 1 --2- 172.21.15.33:0/1333660508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e5cf97ad0 0x555e5cf94bf0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3340009750 tx=0x7f3340005720 comp rx=0 tx=0).stop 2024-05-15T14:02:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.940+0000 7f33523e5740 1 -- 172.21.15.33:0/1333660508 shutdown_connections 2024-05-15T14:02:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.940+0000 7f33523e5740 1 --2- 172.21.15.33:0/1333660508 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x555e5cfa7670 0x555e5cfa9a60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.940+0000 7f33523e5740 1 --2- 172.21.15.33:0/1333660508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e5cf97ad0 0x555e5cf94bf0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.940+0000 7f33523e5740 1 --2- 172.21.15.33:0/1333660508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555e5cf8d210 0x555e5cf946b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.940+0000 7f33523e5740 1 --2- 172.21.15.33:0/1333660508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555e5cf8c8f0 0x555e5cf973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.940+0000 7f33523e5740 1 -- 172.21.15.33:0/1333660508 >> 172.21.15.33:0/1333660508 conn(0x555e5cf88e00 msgr2=0x555e5cf88100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:02:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.940+0000 7f3330ff9640 0 client.15621 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:02:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.940+0000 7f33523e5740 1 -- 172.21.15.33:0/1333660508 shutdown_connections 2024-05-15T14:02:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.940+0000 7f33523e5740 1 -- 172.21.15.33:0/1333660508 wait complete. 2024-05-15T14:02:48.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.940+0000 7f33523e5740 20 client.15621 trim_cache size 0 max 16384 2024-05-15T14:02:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.957+0000 7f33523e5740 1 Processor -- start 2024-05-15T14:02:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.957+0000 7f33523e5740 1 -- start start 2024-05-15T14:02:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.957+0000 7f33523e5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e5cf8c8f0 0x555e5cfaf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.957+0000 7f33523e5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555e5cf8d210 0x555e5cfaf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.957+0000 7f33523e5740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555e5cf97ad0 0x555e5cf91910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.957+0000 7f33523e5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f33481b0ee0 con 0x555e5cf8c8f0 2024-05-15T14:02:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.957+0000 7f33523e5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f33481ae7b0 con 0x555e5cf8d210 2024-05-15T14:02:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.957+0000 7f33523e5740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f33481ac600 con 0x555e5cf97ad0 2024-05-15T14:02:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.957+0000 7f334d38c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e5cf8c8f0 0x555e5cfaf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.957+0000 7f334db8d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555e5cf97ad0 0x555e5cf91910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.957+0000 7f334cb8b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555e5cf8d210 0x555e5cfaf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.957+0000 7f334d38c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e5cf8c8f0 0x555e5cfaf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51892/0 (socket says 172.21.15.33:51892) 2024-05-15T14:02:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.957+0000 7f334d38c640 1 -- 172.21.15.33:0/636228084 learned_addr learned my addr 172.21.15.33:0/636228084 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:02:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.958+0000 7f334d38c640 1 -- 172.21.15.33:0/636228084 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555e5cf97ad0 msgr2=0x555e5cf91910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.958+0000 7f334d38c640 1 --2- 172.21.15.33:0/636228084 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555e5cf97ad0 0x555e5cf91910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.958+0000 7f334d38c640 1 -- 172.21.15.33:0/636228084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555e5cf8d210 msgr2=0x555e5cfaf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.958+0000 7f334d38c640 1 --2- 172.21.15.33:0/636228084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555e5cf8d210 0x555e5cfaf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.958+0000 7f334d38c640 1 -- 172.21.15.33:0/636228084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e5cf88c30 con 0x555e5cf8c8f0 2024-05-15T14:02:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.958+0000 7f334d38c640 1 --2- 172.21.15.33:0/636228084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e5cf8c8f0 0x555e5cfaf2e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3344005340 tx=0x7f33440073c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b4e0b0bf45fcb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:02:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.958+0000 7f3332ffd640 1 -- 172.21.15.33:0/636228084 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334402d070 con 0x555e5cf8c8f0 2024-05-15T14:02:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.958+0000 7f3332ffd640 1 -- 172.21.15.33:0/636228084 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3344007910 con 0x555e5cf8c8f0 2024-05-15T14:02:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.958+0000 7f3332ffd640 1 -- 172.21.15.33:0/636228084 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334401d040 con 0x555e5cf8c8f0 2024-05-15T14:02:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.959+0000 7f33523e5740 1 -- 172.21.15.33:0/636228084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e5cf8c8f0 msgr2=0x555e5cfaf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.959+0000 7f33523e5740 1 --2- 172.21.15.33:0/636228084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e5cf8c8f0 0x555e5cfaf2e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3344005340 tx=0x7f33440073c0 comp rx=0 tx=0).stop 2024-05-15T14:02:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.959+0000 7f33523e5740 1 -- 172.21.15.33:0/636228084 shutdown_connections 2024-05-15T14:02:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.959+0000 7f33523e5740 1 --2- 172.21.15.33:0/636228084 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555e5cf97ad0 0x555e5cf91910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.959+0000 7f33523e5740 1 --2- 172.21.15.33:0/636228084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555e5cf8d210 0x555e5cfaf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.959+0000 7f33523e5740 1 --2- 172.21.15.33:0/636228084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e5cf8c8f0 0x555e5cfaf2e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.959+0000 7f33523e5740 1 -- 172.21.15.33:0/636228084 >> 172.21.15.33:0/636228084 conn(0x555e5cf88e00 msgr2=0x555e5cfa2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:02:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.959+0000 7f33523e5740 1 -- 172.21.15.33:0/636228084 shutdown_connections 2024-05-15T14:02:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.959+0000 7f33523e5740 1 -- 172.21.15.33:0/636228084 wait complete. 2024-05-15T14:02:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.959+0000 7f33523e5740 1 -- start start 2024-05-15T14:02:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.959+0000 7f33523e5740 1 Processor -- start 2024-05-15T14:02:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.960+0000 7f33523e5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e5cf8c8f0 0x555e5cf9fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.960+0000 7f33523e5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555e5cf8d210 0x555e5cf9d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.960+0000 7f33523e5740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555e5cf97ad0 0x555e5cf9d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.960+0000 7f33523e5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f33481acc70 con 0x555e5cf8c8f0 2024-05-15T14:02:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.960+0000 7f33523e5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f33481ae0e0 con 0x555e5cf8d210 2024-05-15T14:02:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.960+0000 7f33523e5740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f33481ad340 con 0x555e5cf97ad0 2024-05-15T14:02:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.960+0000 7f33523e5740 10 client.0 _ll_register_callbacks cb 0x7f33400a95c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:02:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.960+0000 7f334d38c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e5cf8c8f0 0x555e5cf9fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.960+0000 7f334d38c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e5cf8c8f0 0x555e5cf9fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51900/0 (socket says 172.21.15.33:51900) 2024-05-15T14:02:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.960+0000 7f334db8d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555e5cf97ad0 0x555e5cf9d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.960+0000 7f334d38c640 1 -- 172.21.15.33:0/3900427165 learned_addr learned my addr 172.21.15.33:0/3900427165 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:02:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.960+0000 7f334cb8b640 1 --2- 172.21.15.33:0/3900427165 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555e5cf8d210 0x555e5cf9d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.960+0000 7f334d38c640 1 -- 172.21.15.33:0/3900427165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555e5cf97ad0 msgr2=0x555e5cf9d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.960+0000 7f334d38c640 1 --2- 172.21.15.33:0/3900427165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555e5cf97ad0 0x555e5cf9d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.960+0000 7f334d38c640 1 -- 172.21.15.33:0/3900427165 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555e5cf8d210 msgr2=0x555e5cf9d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:02:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.960+0000 7f334d38c640 1 --2- 172.21.15.33:0/3900427165 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555e5cf8d210 0x555e5cf9d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.960+0000 7f334d38c640 1 -- 172.21.15.33:0/3900427165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e5cf9dc90 con 0x555e5cf8c8f0 2024-05-15T14:02:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.960+0000 7f334cb8b640 1 --2- 172.21.15.33:0/3900427165 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555e5cf8d210 0x555e5cf9d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:02:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.960+0000 7f334db8d640 1 --2- 172.21.15.33:0/3900427165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555e5cf97ad0 0x555e5cf9d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:02:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.960+0000 7f334d38c640 1 --2- 172.21.15.33:0/3900427165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e5cf8c8f0 0x555e5cf9fd80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f33440106c0 tx=0x7f33440106f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b64c09599acdb86 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:02:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.960+0000 7f3330ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:02:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.961+0000 7f3330ff9640 1 -- 172.21.15.33:0/3900427165 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334402d030 con 0x555e5cf8c8f0 2024-05-15T14:02:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.961+0000 7f33523e5740 1 -- 172.21.15.33:0/3900427165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3344007910 con 0x555e5cf8c8f0 2024-05-15T14:02:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.961+0000 7f3330ff9640 1 -- 172.21.15.33:0/3900427165 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f3344010950 con 0x555e5cf8c8f0 2024-05-15T14:02:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.961+0000 7f3330ff9640 1 -- 172.21.15.33:0/3900427165 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3344004c60 con 0x555e5cf8c8f0 2024-05-15T14:02:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.961+0000 7f33523e5740 10 client.15627 fetch_fsmap learned FSMap version 24 2024-05-15T14:02:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.961+0000 7f33523e5740 1 -- 172.21.15.33:0/3900427165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555e5d011ce0 con 0x555e5cf8c8f0 2024-05-15T14:02:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.961+0000 7f3330ff9640 1 -- 172.21.15.33:0/3900427165 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3344007910 con 0x555e5cf8c8f0 2024-05-15T14:02:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.961+0000 7f3330ff9640 1 -- 172.21.15.33:0/3900427165 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f3344027210 con 0x555e5cf8c8f0 2024-05-15T14:02:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.961+0000 7f33523e5740 10 client.15627 fetch_fsmap finished waiting for FSMap version 24 2024-05-15T14:02:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.961+0000 7f33523e5740 10 client.15627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:02:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.961+0000 7f33523e5740 20 client.15627 populate_metadata read hostname 'smithi033' 2024-05-15T14:02:48.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.961+0000 7f33523e5740 1 --2- 172.21.15.33:0/3900427165 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x555e5d0120b0 0x555e5d0144d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:02:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.961+0000 7f33523e5740 4 client.15627 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:02:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.961+0000 7f33523e5740 1 -- 172.21.15.33:0/3900427165 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555e5cefb360 con 0x555e5d0120b0 2024-05-15T14:02:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.961+0000 7f334cb8b640 1 --2- 172.21.15.33:0/3900427165 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x555e5d0120b0 0x555e5d0144d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:02:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.962+0000 7f334cb8b640 1 --2- 172.21.15.33:0/3900427165 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x555e5d0120b0 0x555e5d0144d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bc437770f1a9d97 server_cookie=23745bbeca95a83b in_seq=0 out_seq=0 2024-05-15T14:02:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:48.962+0000 7f3330ff9640 10 client.15627 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:02:49.047 INFO:tasks.workunit.client.0.smithi033.stdout: 1 68908 16.22 MB/sec warmup 24 sec latency 38.407 ms 2024-05-15T14:02:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:49.083+0000 7f3330ff9640 1 -- 172.21.15.33:0/3900427165 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555e5cefb360 con 0x555e5d0120b0 2024-05-15T14:02:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:49.083+0000 7f3330ff9640 10 client.15627 handle_command_reply: tid=0 2024-05-15T14:02:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:49.083+0000 7f33523e5740 1 client.15627 shutdown 2024-05-15T14:02:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:49.083+0000 7f33523e5740 1 -- 172.21.15.33:0/3900427165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e5cf8c8f0 msgr2=0x555e5cf9fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:49.083+0000 7f33523e5740 1 --2- 172.21.15.33:0/3900427165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e5cf8c8f0 0x555e5cf9fd80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f33440106c0 tx=0x7f33440106f0 comp rx=0 tx=0).stop 2024-05-15T14:02:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:49.083+0000 7f33523e5740 1 -- 172.21.15.33:0/3900427165 shutdown_connections 2024-05-15T14:02:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:49.083+0000 7f33523e5740 1 --2- 172.21.15.33:0/3900427165 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x555e5d0120b0 0x555e5d0144d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:49.083+0000 7f33523e5740 1 --2- 172.21.15.33:0/3900427165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555e5cf97ad0 0x555e5cf9d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:49.083+0000 7f33523e5740 1 --2- 172.21.15.33:0/3900427165 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555e5cf8d210 0x555e5cf9d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:49.083+0000 7f33523e5740 1 --2- 172.21.15.33:0/3900427165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e5cf8c8f0 0x555e5cf9fd80 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:49.083+0000 7f3330ff9640 0 client.15627 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:02:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:49.083+0000 7f33523e5740 1 -- 172.21.15.33:0/3900427165 >> 172.21.15.33:0/3900427165 conn(0x555e5cf88e00 msgr2=0x555e5cfa2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:02:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:49.083+0000 7f33523e5740 1 -- 172.21.15.33:0/3900427165 shutdown_connections 2024-05-15T14:02:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:49.083+0000 7f33523e5740 1 -- 172.21.15.33:0/3900427165 wait complete. 2024-05-15T14:02:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:49.084+0000 7f33523e5740 20 client.15627 trim_cache size 0 max 16384 2024-05-15T14:02:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:02:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:02:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:02:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:49.084+0000 7f334f5f0640 1 -- 172.21.15.33:0/474400661 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f33340b2950 msgr2=0x7f33340b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:49.084+0000 7f334f5f0640 1 --2- 172.21.15.33:0/474400661 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f33340b2950 0x7f33340b4e10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x555e5cf8a220 tx=0x7f3338005a20 comp rx=0 tx=0).stop 2024-05-15T14:02:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:49.084+0000 7f334f5f0640 1 -- 172.21.15.33:0/474400661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f334810d090 msgr2=0x7f334819da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:02:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:49.084+0000 7f334f5f0640 1 --2- 172.21.15.33:0/474400661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f334810d090 0x7f334819da40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f334000e940 tx=0x7f334000ee00 comp rx=0 tx=0).stop 2024-05-15T14:02:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:49.084+0000 7f334f5f0640 1 -- 172.21.15.33:0/474400661 shutdown_connections 2024-05-15T14:02:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:49.084+0000 7f334f5f0640 1 --2- 172.21.15.33:0/474400661 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f33340b2950 0x7f33340b4e10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:49.084+0000 7f334f5f0640 1 --2- 172.21.15.33:0/474400661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f334810d090 0x7f334819da40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:49.084+0000 7f334f5f0640 1 --2- 172.21.15.33:0/474400661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33481045e0 0x7f33481996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:49.084+0000 7f334f5f0640 1 --2- 172.21.15.33:0/474400661 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3348103cc0 0x7f3348199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:02:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:49.084+0000 7f334f5f0640 1 -- 172.21.15.33:0/474400661 >> 172.21.15.33:0/474400661 conn(0x7f3348078a70 msgr2=0x7f33480fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:02:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:49.085+0000 7f334f5f0640 1 -- 172.21.15.33:0/474400661 shutdown_connections 2024-05-15T14:02:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:02:49.085+0000 7f334f5f0640 1 -- 172.21.15.33:0/474400661 wait complete. 2024-05-15T14:02:50.047 INFO:tasks.workunit.client.0.smithi033.stdout: 1 71202 16.12 MB/sec warmup 25 sec latency 292.973 ms 2024-05-15T14:02:50.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:50 smithi052 ceph-mon[36190]: pgmap v77: 97 pgs: 97 active+clean; 23 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-05-15T14:02:50.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:50 smithi052 ceph-mon[36190]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:02:50.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:50 smithi081 ceph-mon[36434]: pgmap v77: 97 pgs: 97 active+clean; 23 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-05-15T14:02:50.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:50 smithi081 ceph-mon[36434]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:02:50.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:50 smithi033 ceph-mon[31650]: pgmap v77: 97 pgs: 97 active+clean; 23 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-05-15T14:02:50.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:50 smithi033 ceph-mon[31650]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:02:51.047 INFO:tasks.workunit.client.0.smithi033.stdout: 1 71202 15.50 MB/sec warmup 26 sec latency 1293.016 ms 2024-05-15T14:02:52.047 INFO:tasks.workunit.client.0.smithi033.stdout: 1 71202 14.93 MB/sec warmup 27 sec latency 2293.102 ms 2024-05-15T14:02:52.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:52 smithi052 ceph-mon[36190]: pgmap v78: 97 pgs: 97 active+clean; 23 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-05-15T14:02:52.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:52 smithi081 ceph-mon[36434]: pgmap v78: 97 pgs: 97 active+clean; 23 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-05-15T14:02:52.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:52 smithi033 ceph-mon[31650]: pgmap v78: 97 pgs: 97 active+clean; 23 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-05-15T14:02:53.047 INFO:tasks.workunit.client.0.smithi033.stdout: 1 71202 14.40 MB/sec warmup 28 sec latency 3293.183 ms 2024-05-15T14:02:54.047 INFO:tasks.workunit.client.0.smithi033.stdout: 1 71206 13.90 MB/sec warmup 29 sec latency 4060.862 ms 2024-05-15T14:02:54.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:54 smithi052 ceph-mon[36190]: pgmap v79: 97 pgs: 97 active+clean; 27 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.1 MiB/s wr, 514 op/s 2024-05-15T14:02:54.693 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:54 smithi033 ceph-mon[31650]: pgmap v79: 97 pgs: 97 active+clean; 27 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.1 MiB/s wr, 514 op/s 2024-05-15T14:02:54.694 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:54 smithi081 ceph-mon[36434]: pgmap v79: 97 pgs: 97 active+clean; 27 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.1 MiB/s wr, 514 op/s 2024-05-15T14:02:55.047 INFO:tasks.workunit.client.0.smithi033.stdout: 1 71206 13.44 MB/sec warmup 30 sec latency 1230.468 ms 2024-05-15T14:02:56.048 INFO:tasks.workunit.client.0.smithi033.stdout: 1 71206 13.00 MB/sec warmup 31 sec latency 2230.574 ms 2024-05-15T14:02:56.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:56 smithi052 ceph-mon[36190]: pgmap v80: 97 pgs: 97 active+clean; 27 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 7.4 MiB/s wr, 422 op/s 2024-05-15T14:02:56.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:56 smithi081 ceph-mon[36434]: pgmap v80: 97 pgs: 97 active+clean; 27 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 7.4 MiB/s wr, 422 op/s 2024-05-15T14:02:56.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:56 smithi033 ceph-mon[31650]: pgmap v80: 97 pgs: 97 active+clean; 27 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 7.4 MiB/s wr, 422 op/s 2024-05-15T14:02:57.048 INFO:tasks.workunit.client.0.smithi033.stdout: 1 71206 12.60 MB/sec warmup 32 sec latency 3230.677 ms 2024-05-15T14:02:58.048 INFO:tasks.workunit.client.0.smithi033.stdout: 1 71206 12.21 MB/sec warmup 33 sec latency 4230.792 ms 2024-05-15T14:02:58.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:02:58 smithi052 ceph-mon[36190]: pgmap v81: 97 pgs: 97 active+clean; 27 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 7.5 MiB/s wr, 426 op/s 2024-05-15T14:02:58.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:02:58 smithi081 ceph-mon[36434]: pgmap v81: 97 pgs: 97 active+clean; 27 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 7.5 MiB/s wr, 426 op/s 2024-05-15T14:02:58.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:02:58 smithi033 ceph-mon[31650]: pgmap v81: 97 pgs: 97 active+clean; 27 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 7.5 MiB/s wr, 426 op/s 2024-05-15T14:02:59.048 INFO:tasks.workunit.client.0.smithi033.stdout: 1 71368 11.86 MB/sec warmup 34 sec latency 4999.678 ms 2024-05-15T14:03:00.048 INFO:tasks.workunit.client.0.smithi033.stdout: 1 73924 11.89 MB/sec warmup 35 sec latency 15.064 ms 2024-05-15T14:03:00.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:00 smithi052 ceph-mon[36190]: pgmap v82: 97 pgs: 97 active+clean; 27 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 6.8 MiB/s wr, 339 op/s 2024-05-15T14:03:00.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:00 smithi081 ceph-mon[36434]: pgmap v82: 97 pgs: 97 active+clean; 27 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 6.8 MiB/s wr, 339 op/s 2024-05-15T14:03:00.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:00 smithi033 ceph-mon[31650]: pgmap v82: 97 pgs: 97 active+clean; 27 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 6.8 MiB/s wr, 339 op/s 2024-05-15T14:03:01.048 INFO:tasks.workunit.client.0.smithi033.stdout: 1 75551 11.77 MB/sec warmup 36 sec latency 15.461 ms 2024-05-15T14:03:02.048 INFO:tasks.workunit.client.0.smithi033.stdout: 1 78032 11.89 MB/sec warmup 37 sec latency 23.371 ms 2024-05-15T14:03:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.176+0000 7fa77f978640 1 -- 172.21.15.33:0/3027179012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa77810d090 msgr2=0x7fa77810f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.176+0000 7fa77f978640 1 --2- 172.21.15.33:0/3027179012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa77810d090 0x7fa77810f4c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa770009ca0 tx=0x7fa77002a950 comp rx=0 tx=0).stop 2024-05-15T14:03:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.177+0000 7fa77f978640 1 -- 172.21.15.33:0/3027179012 shutdown_connections 2024-05-15T14:03:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.177+0000 7fa77f978640 1 --2- 172.21.15.33:0/3027179012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa77810d090 0x7fa77810f4c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.177+0000 7fa77f978640 1 --2- 172.21.15.33:0/3027179012 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7781045e0 0x7fa77810cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.177+0000 7fa77f978640 1 --2- 172.21.15.33:0/3027179012 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa778103cc0 0x7fa7781040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.177+0000 7fa77f978640 1 -- 172.21.15.33:0/3027179012 >> 172.21.15.33:0/3027179012 conn(0x7fa778078a70 msgr2=0x7fa778078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.177+0000 7fa77f978640 1 -- 172.21.15.33:0/3027179012 shutdown_connections 2024-05-15T14:03:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.177+0000 7fa77f978640 1 -- 172.21.15.33:0/3027179012 wait complete. 2024-05-15T14:03:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.177+0000 7fa77f978640 1 Processor -- start 2024-05-15T14:03:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.177+0000 7fa77f978640 1 -- start start 2024-05-15T14:03:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.177+0000 7fa77f978640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa778103cc0 0x7fa778199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.177+0000 7fa77f978640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7781045e0 0x7fa778199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.177+0000 7fa77f978640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa77810d090 0x7fa77819dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.177+0000 7fa77f978640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa778112cd0 con 0x7fa77810d090 2024-05-15T14:03:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.178+0000 7fa77f978640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa778112b50 con 0x7fa7781045e0 2024-05-15T14:03:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.178+0000 7fa77f978640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa778112e50 con 0x7fa778103cc0 2024-05-15T14:03:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.178+0000 7fa77df15640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa77810d090 0x7fa77819dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.178+0000 7fa77df15640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa77810d090 0x7fa77819dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49700/0 (socket says 172.21.15.33:49700) 2024-05-15T14:03:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.178+0000 7fa77df15640 1 -- 172.21.15.33:0/843196179 learned_addr learned my addr 172.21.15.33:0/843196179 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.178+0000 7fa77cf13640 1 --2- 172.21.15.33:0/843196179 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7781045e0 0x7fa778199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.178+0000 7fa77d714640 1 --2- 172.21.15.33:0/843196179 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa778103cc0 0x7fa778199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.178+0000 7fa77cf13640 1 -- 172.21.15.33:0/843196179 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa778103cc0 msgr2=0x7fa778199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.178+0000 7fa77cf13640 1 --2- 172.21.15.33:0/843196179 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa778103cc0 0x7fa778199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.178+0000 7fa77cf13640 1 -- 172.21.15.33:0/843196179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa77810d090 msgr2=0x7fa77819dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.178+0000 7fa77cf13640 1 --2- 172.21.15.33:0/843196179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa77810d090 0x7fa77819dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.178+0000 7fa77cf13640 1 -- 172.21.15.33:0/843196179 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77819e200 con 0x7fa7781045e0 2024-05-15T14:03:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.178+0000 7fa77df15640 1 --2- 172.21.15.33:0/843196179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa77810d090 0x7fa77819dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:03:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.178+0000 7fa77cf13640 1 --2- 172.21.15.33:0/843196179 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7781045e0 0x7fa778199750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa76800e9f0 tx=0x7fa76800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.178+0000 7fa76e7fc640 1 -- 172.21.15.33:0/843196179 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76800ccc0 con 0x7fa7781045e0 2024-05-15T14:03:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.178+0000 7fa76e7fc640 1 -- 172.21.15.33:0/843196179 <== mon.2 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa768005020 con 0x7fa7781045e0 2024-05-15T14:03:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.178+0000 7fa77f978640 1 -- 172.21.15.33:0/843196179 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa77819e4f0 con 0x7fa7781045e0 2024-05-15T14:03:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.178+0000 7fa77f978640 1 -- 172.21.15.33:0/843196179 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7781a5d80 con 0x7fa7781045e0 2024-05-15T14:03:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.178+0000 7fa76e7fc640 1 -- 172.21.15.33:0/843196179 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa768004a00 con 0x7fa7781045e0 2024-05-15T14:03:02.190 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.181+0000 7fa78276a740 1 Processor -- start 2024-05-15T14:03:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.181+0000 7fa78276a740 1 -- start start 2024-05-15T14:03:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.181+0000 7fa76e7fc640 1 -- 172.21.15.33:0/843196179 <== mon.2 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa768004ba0 con 0x7fa7781045e0 2024-05-15T14:03:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.181+0000 7fa78276a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55feb3fc38f0 0x55feb3fc3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.181+0000 7fa78276a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55feb3fc4210 0x55feb3fce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.181+0000 7fa78276a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55feb3fcead0 0x55feb3fd0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.181+0000 7fa78276a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55feb3f36c30 con 0x55feb3fcead0 2024-05-15T14:03:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.181+0000 7fa78276a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55feb3f267d0 con 0x55feb3fc4210 2024-05-15T14:03:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.181+0000 7fa78276a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55feb3f2a810 con 0x55feb3fc38f0 2024-05-15T14:03:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.181+0000 7fa77cf13640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55feb3fcead0 0x55feb3fd0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.181+0000 7fa77df15640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55feb3fc38f0 0x55feb3fc3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.181+0000 7fa77cf13640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55feb3fcead0 0x55feb3fd0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49712/0 (socket says 172.21.15.33:49712) 2024-05-15T14:03:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.181+0000 7fa77df15640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55feb3fc38f0 0x55feb3fc3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:38844/0 (socket says 172.21.15.33:38844) 2024-05-15T14:03:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.181+0000 7fa77cf13640 1 -- 172.21.15.33:0/3362990808 learned_addr learned my addr 172.21.15.33:0/3362990808 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.181+0000 7fa77d714640 1 --2- 172.21.15.33:0/3362990808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55feb3fc4210 0x55feb3fce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.182+0000 7fa77cf13640 1 -- 172.21.15.33:0/3362990808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55feb3fc38f0 msgr2=0x55feb3fc3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.182+0000 7fa77cf13640 1 --2- 172.21.15.33:0/3362990808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55feb3fc38f0 0x55feb3fc3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.182+0000 7fa77cf13640 1 -- 172.21.15.33:0/3362990808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55feb3fc4210 msgr2=0x55feb3fce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.182+0000 7fa77cf13640 1 --2- 172.21.15.33:0/3362990808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55feb3fc4210 0x55feb3fce590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.182+0000 7fa77cf13640 1 -- 172.21.15.33:0/3362990808 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55feb3c3b210 con 0x55feb3fcead0 2024-05-15T14:03:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.182+0000 7fa77d714640 1 --2- 172.21.15.33:0/3362990808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55feb3fc4210 0x55feb3fce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:03:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.182+0000 7fa77df15640 1 --2- 172.21.15.33:0/3362990808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55feb3fc38f0 0x55feb3fc3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:03:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.182+0000 7fa76e7fc640 1 --2- 172.21.15.33:0/843196179 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fa7640b2950 0x7fa7640b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.182+0000 7fa77cf13640 1 --2- 172.21.15.33:0/3362990808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55feb3fcead0 0x55feb3fd0ec0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa7680dfc00 tx=0x7fa7680e87b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c383466630fbe485 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.182+0000 7fa77df15640 1 --2- 172.21.15.33:0/843196179 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fa7640b2950 0x7fa7640b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.182+0000 7fa7637fe640 1 -- 172.21.15.33:0/3362990808 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7680f5070 con 0x55feb3fcead0 2024-05-15T14:03:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.182+0000 7fa76e7fc640 1 -- 172.21.15.33:0/843196179 <== mon.2 v2:172.21.15.52:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fa7680e2500 con 0x7fa7781045e0 2024-05-15T14:03:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.182+0000 7fa7637fe640 1 -- 172.21.15.33:0/3362990808 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7680a9300 con 0x55feb3fcead0 2024-05-15T14:03:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.182+0000 7fa7637fe640 1 -- 172.21.15.33:0/3362990808 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7680a95c0 con 0x55feb3fcead0 2024-05-15T14:03:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.183+0000 7fa78276a740 1 -- 172.21.15.33:0/3362990808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55feb3fcead0 msgr2=0x55feb3fd0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.183+0000 7fa78276a740 1 --2- 172.21.15.33:0/3362990808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55feb3fcead0 0x55feb3fd0ec0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa7680dfc00 tx=0x7fa7680e87b0 comp rx=0 tx=0).stop 2024-05-15T14:03:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.183+0000 7fa77df15640 1 --2- 172.21.15.33:0/843196179 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fa7640b2950 0x7fa7640b4e10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa77002fcc0 tx=0x7fa770005f50 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.183+0000 7fa78276a740 1 -- 172.21.15.33:0/3362990808 shutdown_connections 2024-05-15T14:03:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.183+0000 7fa78276a740 1 --2- 172.21.15.33:0/3362990808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55feb3fcead0 0x55feb3fd0ec0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.183+0000 7fa78276a740 1 --2- 172.21.15.33:0/3362990808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55feb3fc4210 0x55feb3fce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.183+0000 7fa78276a740 1 --2- 172.21.15.33:0/3362990808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55feb3fc38f0 0x55feb3fc3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.183+0000 7fa78276a740 1 -- 172.21.15.33:0/3362990808 >> 172.21.15.33:0/3362990808 conn(0x55feb3fbfe00 msgr2=0x55feb3fc01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.183+0000 7fa78276a740 1 -- 172.21.15.33:0/3362990808 shutdown_connections 2024-05-15T14:03:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.184+0000 7fa78276a740 1 -- 172.21.15.33:0/3362990808 wait complete. 2024-05-15T14:03:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.184+0000 7fa78276a740 1 -- start start 2024-05-15T14:03:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.184+0000 7fa78276a740 1 Processor -- start 2024-05-15T14:03:02.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.184+0000 7fa78276a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55feb3fc38f0 0x55feb3fce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.184+0000 7fa78276a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55feb3fc4210 0x55feb3fcb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.184+0000 7fa78276a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55feb3fcead0 0x55feb3fcbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.184+0000 7fa78276a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55feb3c0c790 con 0x55feb3fc38f0 2024-05-15T14:03:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.184+0000 7fa78276a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55feb3ee2890 con 0x55feb3fc4210 2024-05-15T14:03:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.184+0000 7fa78276a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55feb3fd17a0 con 0x55feb3fcead0 2024-05-15T14:03:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.184+0000 7fa78276a740 10 client.0 _ll_register_callbacks cb 0x55feb3ee5d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:03:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.184+0000 7fa77d714640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55feb3fc38f0 0x55feb3fce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.185+0000 7fa77d714640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55feb3fc38f0 0x55feb3fce3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49726/0 (socket says 172.21.15.33:49726) 2024-05-15T14:03:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.185+0000 7fa77d714640 1 -- 172.21.15.33:0/1055076189 learned_addr learned my addr 172.21.15.33:0/1055076189 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.185+0000 7fa77cf13640 1 --2- 172.21.15.33:0/1055076189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55feb3fc4210 0x55feb3fcb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.185+0000 7fa77df15640 1 --2- 172.21.15.33:0/1055076189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55feb3fcead0 0x55feb3fcbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.185+0000 7fa77d714640 1 -- 172.21.15.33:0/1055076189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55feb3fcead0 msgr2=0x55feb3fcbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.185+0000 7fa77d714640 1 --2- 172.21.15.33:0/1055076189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55feb3fcead0 0x55feb3fcbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.185+0000 7fa77d714640 1 -- 172.21.15.33:0/1055076189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55feb3fc4210 msgr2=0x55feb3fcb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.185+0000 7fa77d714640 1 --2- 172.21.15.33:0/1055076189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55feb3fc4210 0x55feb3fcb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.185+0000 7fa77d714640 1 -- 172.21.15.33:0/1055076189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55feb3d07620 con 0x55feb3fc38f0 2024-05-15T14:03:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.185+0000 7fa77d714640 1 --2- 172.21.15.33:0/1055076189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55feb3fc38f0 0x55feb3fce3d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa774012740 tx=0x7fa774012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5caa612478f727b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.185+0000 7fa760ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:03:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.185+0000 7fa760ff9640 1 -- 172.21.15.33:0/1055076189 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa774004030 con 0x55feb3fc38f0 2024-05-15T14:03:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.185+0000 7fa760ff9640 1 -- 172.21.15.33:0/1055076189 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fa7740041d0 con 0x55feb3fc38f0 2024-05-15T14:03:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.185+0000 7fa78276a740 1 -- 172.21.15.33:0/1055076189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7680a9300 con 0x55feb3fc38f0 2024-05-15T14:03:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.185+0000 7fa760ff9640 1 -- 172.21.15.33:0/1055076189 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa7740043d0 con 0x55feb3fc38f0 2024-05-15T14:03:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.185+0000 7fa78276a740 10 client.15639 fetch_fsmap learned FSMap version 25 2024-05-15T14:03:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.185+0000 7fa78276a740 1 -- 172.21.15.33:0/1055076189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55feb3fdd200 con 0x55feb3fc38f0 2024-05-15T14:03:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.186+0000 7fa760ff9640 1 -- 172.21.15.33:0/1055076189 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7680a9300 con 0x55feb3fc38f0 2024-05-15T14:03:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.186+0000 7fa760ff9640 1 -- 172.21.15.33:0/1055076189 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa774003b50 con 0x55feb3fc38f0 2024-05-15T14:03:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.186+0000 7fa78276a740 10 client.15639 fetch_fsmap finished waiting for FSMap version 25 2024-05-15T14:03:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.186+0000 7fa78276a740 10 client.15639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:03:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.186+0000 7fa78276a740 20 client.15639 populate_metadata read hostname 'smithi033' 2024-05-15T14:03:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.186+0000 7fa78276a740 1 --2- 172.21.15.33:0/1055076189 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55feb3fde760 0x55feb3fe0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.186+0000 7fa78276a740 4 client.15639 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:03:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.186+0000 7fa78276a740 1 -- 172.21.15.33:0/1055076189 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55feb3f38b80 con 0x55feb3fde760 2024-05-15T14:03:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.186+0000 7fa77cf13640 1 --2- 172.21.15.33:0/1055076189 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55feb3fde760 0x55feb3fe0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.186+0000 7fa760ff9640 10 client.15639 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:03:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.186+0000 7fa77cf13640 1 --2- 172.21.15.33:0/1055076189 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55feb3fde760 0x55feb3fe0bb0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce2aac936ea8c11a server_cookie=fbf514573e447c60 in_seq=0 out_seq=0 2024-05-15T14:03:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.187+0000 7fa760ff9640 1 -- 172.21.15.33:0/1055076189 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55feb3f38b80 con 0x55feb3fde760 2024-05-15T14:03:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.187+0000 7fa760ff9640 10 client.15639 handle_command_reply: tid=0 2024-05-15T14:03:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.187+0000 7fa78276a740 1 client.15639 shutdown 2024-05-15T14:03:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.187+0000 7fa78276a740 1 -- 172.21.15.33:0/1055076189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55feb3fc38f0 msgr2=0x55feb3fce3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.187+0000 7fa78276a740 1 --2- 172.21.15.33:0/1055076189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55feb3fc38f0 0x55feb3fce3d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa774012740 tx=0x7fa774012c00 comp rx=0 tx=0).stop 2024-05-15T14:03:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.187+0000 7fa78276a740 1 -- 172.21.15.33:0/1055076189 shutdown_connections 2024-05-15T14:03:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.187+0000 7fa78276a740 1 --2- 172.21.15.33:0/1055076189 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55feb3fde760 0x55feb3fe0bb0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.187+0000 7fa78276a740 1 --2- 172.21.15.33:0/1055076189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55feb3fcead0 0x55feb3fcbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.187+0000 7fa78276a740 1 --2- 172.21.15.33:0/1055076189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55feb3fc4210 0x55feb3fcb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.187+0000 7fa760ff9640 0 client.15639 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:03:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.187+0000 7fa78276a740 1 --2- 172.21.15.33:0/1055076189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55feb3fc38f0 0x55feb3fce3d0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.188+0000 7fa78276a740 1 -- 172.21.15.33:0/1055076189 >> 172.21.15.33:0/1055076189 conn(0x55feb3fbfe00 msgr2=0x55feb3fbf100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.188+0000 7fa78276a740 1 -- 172.21.15.33:0/1055076189 shutdown_connections 2024-05-15T14:03:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.188+0000 7fa78276a740 1 -- 172.21.15.33:0/1055076189 wait complete. 2024-05-15T14:03:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.188+0000 7fa78276a740 20 client.15639 trim_cache size 0 max 16384 2024-05-15T14:03:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.205+0000 7fa78276a740 1 Processor -- start 2024-05-15T14:03:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.205+0000 7fa78276a740 1 -- start start 2024-05-15T14:03:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.205+0000 7fa78276a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55feb3fc38f0 0x55feb3fc6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.205+0000 7fa78276a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55feb3fc4210 0x55feb3fcdf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.205+0000 7fa78276a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55feb3fcead0 0x55feb3fbef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.205+0000 7fa78276a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa7781b0f80 con 0x55feb3fc38f0 2024-05-15T14:03:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.205+0000 7fa78276a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa7781ae850 con 0x55feb3fcead0 2024-05-15T14:03:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.205+0000 7fa78276a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa7781ac6a0 con 0x55feb3fc4210 2024-05-15T14:03:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.205+0000 7fa77d714640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55feb3fc38f0 0x55feb3fc6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.205+0000 7fa77d714640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55feb3fc38f0 0x55feb3fc6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49738/0 (socket says 172.21.15.33:49738) 2024-05-15T14:03:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.205+0000 7fa77cf13640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55feb3fc4210 0x55feb3fcdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.205+0000 7fa77d714640 1 -- 172.21.15.33:0/2043601625 learned_addr learned my addr 172.21.15.33:0/2043601625 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.205+0000 7fa77df15640 1 --2- 172.21.15.33:0/2043601625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55feb3fcead0 0x55feb3fbef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.205+0000 7fa77cf13640 1 -- 172.21.15.33:0/2043601625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55feb3fcead0 msgr2=0x55feb3fbef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.205+0000 7fa77cf13640 1 --2- 172.21.15.33:0/2043601625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55feb3fcead0 0x55feb3fbef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.205+0000 7fa77cf13640 1 -- 172.21.15.33:0/2043601625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55feb3fc38f0 msgr2=0x55feb3fc6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.205+0000 7fa77cf13640 1 --2- 172.21.15.33:0/2043601625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55feb3fc38f0 0x55feb3fc6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.205+0000 7fa77cf13640 1 -- 172.21.15.33:0/2043601625 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55feb3fbfc30 con 0x55feb3fc4210 2024-05-15T14:03:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.205+0000 7fa77d714640 1 --2- 172.21.15.33:0/2043601625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55feb3fc38f0 0x55feb3fc6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:03:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.206+0000 7fa77cf13640 1 --2- 172.21.15.33:0/2043601625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55feb3fc4210 0x55feb3fcdf00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa76800e9c0 tx=0x7fa7680a5040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf081441fab705ee server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.206+0000 7fa762ffd640 1 -- 172.21.15.33:0/2043601625 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76801c070 con 0x55feb3fc4210 2024-05-15T14:03:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.206+0000 7fa762ffd640 1 -- 172.21.15.33:0/2043601625 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7680a9300 con 0x55feb3fc4210 2024-05-15T14:03:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.206+0000 7fa762ffd640 1 -- 172.21.15.33:0/2043601625 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7680a8f60 con 0x55feb3fc4210 2024-05-15T14:03:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.206+0000 7fa78276a740 1 -- 172.21.15.33:0/2043601625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55feb3fc4210 msgr2=0x55feb3fcdf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.206+0000 7fa78276a740 1 --2- 172.21.15.33:0/2043601625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55feb3fc4210 0x55feb3fcdf00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa76800e9c0 tx=0x7fa7680a5040 comp rx=0 tx=0).stop 2024-05-15T14:03:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.206+0000 7fa78276a740 1 -- 172.21.15.33:0/2043601625 shutdown_connections 2024-05-15T14:03:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.206+0000 7fa78276a740 1 --2- 172.21.15.33:0/2043601625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55feb3fcead0 0x55feb3fbef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.206+0000 7fa78276a740 1 --2- 172.21.15.33:0/2043601625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55feb3fc4210 0x55feb3fcdf00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.206+0000 7fa78276a740 1 --2- 172.21.15.33:0/2043601625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55feb3fc38f0 0x55feb3fc6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.206+0000 7fa78276a740 1 -- 172.21.15.33:0/2043601625 >> 172.21.15.33:0/2043601625 conn(0x55feb3fbfe00 msgr2=0x55feb3fbe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.207+0000 7fa78276a740 1 -- 172.21.15.33:0/2043601625 shutdown_connections 2024-05-15T14:03:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.207+0000 7fa78276a740 1 -- 172.21.15.33:0/2043601625 wait complete. 2024-05-15T14:03:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.207+0000 7fa78276a740 1 -- start start 2024-05-15T14:03:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.207+0000 7fa78276a740 1 Processor -- start 2024-05-15T14:03:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.207+0000 7fa78276a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55feb3fc38f0 0x55feb3fd6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.207+0000 7fa78276a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55feb3fc4210 0x55feb3fd21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.207+0000 7fa78276a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55feb3fcead0 0x55feb3fd26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.207+0000 7fa78276a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa7781acd10 con 0x55feb3fc4210 2024-05-15T14:03:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.207+0000 7fa78276a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa7781ae180 con 0x55feb3fc38f0 2024-05-15T14:03:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.207+0000 7fa78276a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa7781ad3e0 con 0x55feb3fcead0 2024-05-15T14:03:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.207+0000 7fa78276a740 10 client.0 _ll_register_callbacks cb 0x7fa774009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:03:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.207+0000 7fa77cf13640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55feb3fc4210 0x55feb3fd21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.207+0000 7fa77cf13640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55feb3fc4210 0x55feb3fd21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49740/0 (socket says 172.21.15.33:49740) 2024-05-15T14:03:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.207+0000 7fa77cf13640 1 -- 172.21.15.33:0/3785555273 learned_addr learned my addr 172.21.15.33:0/3785555273 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.207+0000 7fa77d714640 1 --2- 172.21.15.33:0/3785555273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55feb3fc38f0 0x55feb3fd6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.207+0000 7fa77df15640 1 --2- 172.21.15.33:0/3785555273 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55feb3fcead0 0x55feb3fd26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.208+0000 7fa77cf13640 1 -- 172.21.15.33:0/3785555273 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55feb3fcead0 msgr2=0x55feb3fd26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.208+0000 7fa77cf13640 1 --2- 172.21.15.33:0/3785555273 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55feb3fcead0 0x55feb3fd26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.208+0000 7fa77cf13640 1 -- 172.21.15.33:0/3785555273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55feb3fc38f0 msgr2=0x55feb3fd6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.208+0000 7fa77cf13640 1 --2- 172.21.15.33:0/3785555273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55feb3fc38f0 0x55feb3fd6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.208+0000 7fa77cf13640 1 -- 172.21.15.33:0/3785555273 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55feb3fd2d80 con 0x55feb3fc4210 2024-05-15T14:03:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.208+0000 7fa77d714640 1 --2- 172.21.15.33:0/3785555273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55feb3fc38f0 0x55feb3fd6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:03:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.208+0000 7fa77cf13640 1 --2- 172.21.15.33:0/3785555273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55feb3fc4210 0x55feb3fd21a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa7680a5010 tx=0x7fa7680096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc97faeae96373d server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.208+0000 7fa760ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:03:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.208+0000 7fa760ff9640 1 -- 172.21.15.33:0/3785555273 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76801c070 con 0x55feb3fc4210 2024-05-15T14:03:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.208+0000 7fa760ff9640 1 -- 172.21.15.33:0/3785555273 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fa7680051d0 con 0x55feb3fc4210 2024-05-15T14:03:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.208+0000 7fa78276a740 1 -- 172.21.15.33:0/3785555273 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7680a9300 con 0x55feb3fc4210 2024-05-15T14:03:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.208+0000 7fa760ff9640 1 -- 172.21.15.33:0/3785555273 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa7680a9140 con 0x55feb3fc4210 2024-05-15T14:03:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.208+0000 7fa78276a740 10 client.15645 fetch_fsmap learned FSMap version 25 2024-05-15T14:03:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.208+0000 7fa78276a740 1 -- 172.21.15.33:0/3785555273 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55feb4048f10 con 0x55feb3fc4210 2024-05-15T14:03:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.208+0000 7fa760ff9640 1 -- 172.21.15.33:0/3785555273 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7680a9300 con 0x55feb3fc4210 2024-05-15T14:03:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.209+0000 7fa760ff9640 1 -- 172.21.15.33:0/3785555273 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa7680dfee0 con 0x55feb3fc4210 2024-05-15T14:03:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.209+0000 7fa78276a740 10 client.15645 fetch_fsmap finished waiting for FSMap version 25 2024-05-15T14:03:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.209+0000 7fa78276a740 10 client.15645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:03:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.209+0000 7fa78276a740 20 client.15645 populate_metadata read hostname 'smithi033' 2024-05-15T14:03:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.209+0000 7fa78276a740 1 --2- 172.21.15.33:0/3785555273 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55feb4048f60 0x55feb404b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.209+0000 7fa78276a740 4 client.15645 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:03:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.209+0000 7fa78276a740 1 -- 172.21.15.33:0/3785555273 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55feb3f32360 con 0x55feb4048f60 2024-05-15T14:03:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.209+0000 7fa77d714640 1 --2- 172.21.15.33:0/3785555273 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55feb4048f60 0x55feb404b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.209+0000 7fa760ff9640 10 client.15645 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:03:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.209+0000 7fa77d714640 1 --2- 172.21.15.33:0/3785555273 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55feb4048f60 0x55feb404b380 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2deb8583cb67d68f server_cookie=e9491ab4d7cfd71a in_seq=0 out_seq=0 2024-05-15T14:03:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.272+0000 7fa760ff9640 1 -- 172.21.15.33:0/3785555273 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55feb3f32360 con 0x55feb4048f60 2024-05-15T14:03:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.272+0000 7fa760ff9640 10 client.15645 handle_command_reply: tid=0 2024-05-15T14:03:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.272+0000 7fa78276a740 1 client.15645 shutdown 2024-05-15T14:03:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.272+0000 7fa78276a740 1 -- 172.21.15.33:0/3785555273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55feb3fc4210 msgr2=0x55feb3fd21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.272+0000 7fa78276a740 1 --2- 172.21.15.33:0/3785555273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55feb3fc4210 0x55feb3fd21a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa7680a5010 tx=0x7fa7680096e0 comp rx=0 tx=0).stop 2024-05-15T14:03:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.272+0000 7fa78276a740 1 -- 172.21.15.33:0/3785555273 shutdown_connections 2024-05-15T14:03:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.272+0000 7fa78276a740 1 --2- 172.21.15.33:0/3785555273 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55feb4048f60 0x55feb404b380 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.272+0000 7fa78276a740 1 --2- 172.21.15.33:0/3785555273 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55feb3fcead0 0x55feb3fd26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.272+0000 7fa78276a740 1 --2- 172.21.15.33:0/3785555273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55feb3fc4210 0x55feb3fd21a0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.272+0000 7fa78276a740 1 --2- 172.21.15.33:0/3785555273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55feb3fc38f0 0x55feb3fd6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.272+0000 7fa78276a740 1 -- 172.21.15.33:0/3785555273 >> 172.21.15.33:0/3785555273 conn(0x55feb3fbfe00 msgr2=0x55feb3fd9660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.272+0000 7fa760ff9640 0 client.15645 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:03:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.272+0000 7fa78276a740 1 -- 172.21.15.33:0/3785555273 shutdown_connections 2024-05-15T14:03:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.272+0000 7fa78276a740 1 -- 172.21.15.33:0/3785555273 wait complete. 2024-05-15T14:03:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.272+0000 7fa78276a740 20 client.15645 trim_cache size 0 max 16384 2024-05-15T14:03:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:03:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:03:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:03:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.273+0000 7fa77f978640 1 -- 172.21.15.33:0/843196179 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fa7640b2950 msgr2=0x7fa7640b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.273+0000 7fa77f978640 1 --2- 172.21.15.33:0/843196179 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fa7640b2950 0x7fa7640b4e10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa77002fcc0 tx=0x7fa770005f50 comp rx=0 tx=0).stop 2024-05-15T14:03:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.273+0000 7fa77f978640 1 -- 172.21.15.33:0/843196179 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7781045e0 msgr2=0x7fa778199750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.273+0000 7fa77f978640 1 --2- 172.21.15.33:0/843196179 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7781045e0 0x7fa778199750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa76800e9f0 tx=0x7fa76800eeb0 comp rx=0 tx=0).stop 2024-05-15T14:03:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.273+0000 7fa77f978640 1 -- 172.21.15.33:0/843196179 shutdown_connections 2024-05-15T14:03:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.273+0000 7fa77f978640 1 --2- 172.21.15.33:0/843196179 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fa7640b2950 0x7fa7640b4e10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:02.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.273+0000 7fa77f978640 1 --2- 172.21.15.33:0/843196179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa77810d090 0x7fa77819dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.273+0000 7fa77f978640 1 --2- 172.21.15.33:0/843196179 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7781045e0 0x7fa778199750 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.273+0000 7fa77f978640 1 --2- 172.21.15.33:0/843196179 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa778103cc0 0x7fa778199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.273+0000 7fa77f978640 1 -- 172.21.15.33:0/843196179 >> 172.21.15.33:0/843196179 conn(0x7fa778078a70 msgr2=0x7fa7780fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.273+0000 7fa77f978640 1 -- 172.21.15.33:0/843196179 shutdown_connections 2024-05-15T14:03:02.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:02.273+0000 7fa77f978640 1 -- 172.21.15.33:0/843196179 wait complete. 2024-05-15T14:03:02.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:02 smithi052 ceph-mon[36190]: pgmap v83: 97 pgs: 97 active+clean; 27 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 6.8 MiB/s wr, 339 op/s 2024-05-15T14:03:02.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:02 smithi081 ceph-mon[36434]: pgmap v83: 97 pgs: 97 active+clean; 27 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 6.8 MiB/s wr, 339 op/s 2024-05-15T14:03:02.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:02 smithi033 ceph-mon[31650]: pgmap v83: 97 pgs: 97 active+clean; 27 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 6.8 MiB/s wr, 339 op/s 2024-05-15T14:03:03.048 INFO:tasks.workunit.client.0.smithi033.stdout: 1 79304 11.68 MB/sec warmup 38 sec latency 51.152 ms 2024-05-15T14:03:04.048 INFO:tasks.workunit.client.0.smithi033.stdout: 1 82013 11.82 MB/sec warmup 39 sec latency 12.472 ms 2024-05-15T14:03:04.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:04 smithi052 ceph-mon[36190]: pgmap v84: 97 pgs: 97 active+clean; 31 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.3 MiB/s wr, 442 op/s 2024-05-15T14:03:04.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:04 smithi081 ceph-mon[36434]: pgmap v84: 97 pgs: 97 active+clean; 31 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.3 MiB/s wr, 442 op/s 2024-05-15T14:03:04.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:04 smithi033 ceph-mon[31650]: pgmap v84: 97 pgs: 97 active+clean; 31 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.3 MiB/s wr, 442 op/s 2024-05-15T14:03:05.048 INFO:tasks.workunit.client.0.smithi033.stdout: 1 84429 11.80 MB/sec warmup 40 sec latency 12.783 ms 2024-05-15T14:03:06.048 INFO:tasks.workunit.client.0.smithi033.stdout: 1 86156 11.73 MB/sec warmup 41 sec latency 17.514 ms 2024-05-15T14:03:06.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:06 smithi052 ceph-mon[36190]: pgmap v85: 97 pgs: 97 active+clean; 31 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 534 KiB/s wr, 106 op/s 2024-05-15T14:03:06.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:06 smithi081 ceph-mon[36434]: pgmap v85: 97 pgs: 97 active+clean; 31 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 534 KiB/s wr, 106 op/s 2024-05-15T14:03:06.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:06 smithi033 ceph-mon[31650]: pgmap v85: 97 pgs: 97 active+clean; 31 MiB data, 530 MiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 534 KiB/s wr, 106 op/s 2024-05-15T14:03:07.048 INFO:tasks.workunit.client.0.smithi033.stdout: 1 88996 11.87 MB/sec warmup 42 sec latency 11.184 ms 2024-05-15T14:03:08.048 INFO:tasks.workunit.client.0.smithi033.stdout: 1 91116 11.77 MB/sec warmup 43 sec latency 14.140 ms 2024-05-15T14:03:08.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:08 smithi052 ceph-mon[36190]: pgmap v86: 97 pgs: 97 active+clean; 31 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 246 op/s 2024-05-15T14:03:08.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:08 smithi081 ceph-mon[36434]: pgmap v86: 97 pgs: 97 active+clean; 31 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 246 op/s 2024-05-15T14:03:08.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:08 smithi033 ceph-mon[31650]: pgmap v86: 97 pgs: 97 active+clean; 31 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 246 op/s 2024-05-15T14:03:09.048 INFO:tasks.workunit.client.0.smithi033.stdout: 1 92991 11.78 MB/sec warmup 44 sec latency 19.649 ms 2024-05-15T14:03:10.048 INFO:tasks.workunit.client.0.smithi033.stdout: 1 95936 11.91 MB/sec warmup 45 sec latency 10.703 ms 2024-05-15T14:03:10.592 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:10 smithi052 ceph-mon[36190]: pgmap v87: 97 pgs: 97 active+clean; 31 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.1 MiB/s wr, 242 op/s 2024-05-15T14:03:10.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:10 smithi081 ceph-mon[36434]: pgmap v87: 97 pgs: 97 active+clean; 31 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.1 MiB/s wr, 242 op/s 2024-05-15T14:03:10.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:10 smithi033 ceph-mon[31650]: pgmap v87: 97 pgs: 97 active+clean; 31 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.1 MiB/s wr, 242 op/s 2024-05-15T14:03:11.049 INFO:tasks.workunit.client.0.smithi033.stdout: 1 97407 11.79 MB/sec warmup 46 sec latency 15.928 ms 2024-05-15T14:03:12.048 INFO:tasks.workunit.client.0.smithi033.stdout: 1 99819 11.84 MB/sec warmup 47 sec latency 8.868 ms 2024-05-15T14:03:12.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:12 smithi052 ceph-mon[36190]: pgmap v88: 97 pgs: 97 active+clean; 31 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.1 MiB/s wr, 242 op/s 2024-05-15T14:03:12.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:12 smithi081 ceph-mon[36434]: pgmap v88: 97 pgs: 97 active+clean; 31 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.1 MiB/s wr, 242 op/s 2024-05-15T14:03:12.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:12 smithi033 ceph-mon[31650]: pgmap v88: 97 pgs: 97 active+clean; 31 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.1 MiB/s wr, 242 op/s 2024-05-15T14:03:13.048 INFO:tasks.workunit.client.0.smithi033.stdout: 1 102357 11.85 MB/sec warmup 48 sec latency 12.204 ms 2024-05-15T14:03:14.050 INFO:tasks.workunit.client.0.smithi033.stdout: 1 104045 11.79 MB/sec warmup 49 sec latency 19.194 ms 2024-05-15T14:03:14.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:14 smithi081 ceph-mon[36434]: pgmap v89: 97 pgs: 97 active+clean; 37 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.8 MiB/s wr, 347 op/s 2024-05-15T14:03:14.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:14 smithi033 ceph-mon[31650]: pgmap v89: 97 pgs: 97 active+clean; 37 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.8 MiB/s wr, 347 op/s 2024-05-15T14:03:14.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:14 smithi052 ceph-mon[36190]: pgmap v89: 97 pgs: 97 active+clean; 37 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.8 MiB/s wr, 347 op/s 2024-05-15T14:03:15.049 INFO:tasks.workunit.client.0.smithi033.stdout: 1 106763 11.89 MB/sec warmup 50 sec latency 16.951 ms 2024-05-15T14:03:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.367+0000 7fc7732e8640 1 -- 172.21.15.33:0/427312674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc76c101930 msgr2=0x7fc76c101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.367+0000 7fc7732e8640 1 --2- 172.21.15.33:0/427312674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc76c101930 0x7fc76c101db0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc760009930 tx=0x7fc76002a420 comp rx=0 tx=0).stop 2024-05-15T14:03:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.367+0000 7fc7732e8640 1 -- 172.21.15.33:0/427312674 shutdown_connections 2024-05-15T14:03:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.367+0000 7fc7732e8640 1 --2- 172.21.15.33:0/427312674 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc76c1022f0 0x7fc76c10e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.367+0000 7fc7732e8640 1 --2- 172.21.15.33:0/427312674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc76c101930 0x7fc76c101db0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.367+0000 7fc7732e8640 1 --2- 172.21.15.33:0/427312674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc76c107940 0x7fc76c107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.367+0000 7fc7732e8640 1 -- 172.21.15.33:0/427312674 >> 172.21.15.33:0/427312674 conn(0x7fc76c078a70 msgr2=0x7fc76c078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.367+0000 7fc7732e8640 1 -- 172.21.15.33:0/427312674 shutdown_connections 2024-05-15T14:03:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.367+0000 7fc7732e8640 1 -- 172.21.15.33:0/427312674 wait complete. 2024-05-15T14:03:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.367+0000 7fc7732e8640 1 Processor -- start 2024-05-15T14:03:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.367+0000 7fc7732e8640 1 -- start start 2024-05-15T14:03:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.368+0000 7fc7732e8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc76c101930 0x7fc76c0fe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.368+0000 7fc7732e8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc76c1022f0 0x7fc76c0fee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.368+0000 7fc7732e8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc76c107940 0x7fc76c0ff370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.368+0000 7fc7732e8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c111bd0 con 0x7fc76c101930 2024-05-15T14:03:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.368+0000 7fc7732e8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c111a50 con 0x7fc76c107940 2024-05-15T14:03:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.368+0000 7fc7732e8640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c111d50 con 0x7fc76c1022f0 2024-05-15T14:03:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.368+0000 7fc771084640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc76c101930 0x7fc76c0fe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.368+0000 7fc771084640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc76c101930 0x7fc76c0fe8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58596/0 (socket says 172.21.15.33:58596) 2024-05-15T14:03:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.368+0000 7fc771084640 1 -- 172.21.15.33:0/2240430463 learned_addr learned my addr 172.21.15.33:0/2240430463 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.368+0000 7fc770883640 1 --2- 172.21.15.33:0/2240430463 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc76c1022f0 0x7fc76c0fee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.368+0000 7fc771885640 1 --2- 172.21.15.33:0/2240430463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc76c107940 0x7fc76c0ff370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.368+0000 7fc771084640 1 -- 172.21.15.33:0/2240430463 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc76c1022f0 msgr2=0x7fc76c0fee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.368+0000 7fc771084640 1 --2- 172.21.15.33:0/2240430463 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc76c1022f0 0x7fc76c0fee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.368+0000 7fc771084640 1 -- 172.21.15.33:0/2240430463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc76c107940 msgr2=0x7fc76c0ff370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.368+0000 7fc771084640 1 --2- 172.21.15.33:0/2240430463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc76c107940 0x7fc76c0ff370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.368+0000 7fc771084640 1 -- 172.21.15.33:0/2240430463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76c1a7760 con 0x7fc76c101930 2024-05-15T14:03:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.368+0000 7fc771084640 1 --2- 172.21.15.33:0/2240430463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc76c101930 0x7fc76c0fe8f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc75c009d50 tx=0x7fc75c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.368+0000 7fc76a7fc640 1 -- 172.21.15.33:0/2240430463 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c003b70 con 0x7fc76c101930 2024-05-15T14:03:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.368+0000 7fc76a7fc640 1 -- 172.21.15.33:0/2240430463 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc75c003d10 con 0x7fc76c101930 2024-05-15T14:03:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.368+0000 7fc7732e8640 1 -- 172.21.15.33:0/2240430463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc76c1a7a50 con 0x7fc76c101930 2024-05-15T14:03:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.368+0000 7fc76a7fc640 1 -- 172.21.15.33:0/2240430463 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c0108a0 con 0x7fc76c101930 2024-05-15T14:03:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.368+0000 7fc7732e8640 1 -- 172.21.15.33:0/2240430463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc76c1a7f70 con 0x7fc76c101930 2024-05-15T14:03:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.371+0000 7fc76a7fc640 1 -- 172.21.15.33:0/2240430463 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc75c004000 con 0x7fc76c101930 2024-05-15T14:03:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.371+0000 7fc776008740 1 Processor -- start 2024-05-15T14:03:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.371+0000 7fc776008740 1 -- start start 2024-05-15T14:03:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.371+0000 7fc776008740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596e12258f0 0x5596e1225cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.371+0000 7fc776008740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596e1226210 0x5596e1230590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.371+0000 7fc776008740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596e1230ad0 0x5596e1232ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.371+0000 7fc776008740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5596e1198c30 con 0x5596e12258f0 2024-05-15T14:03:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.371+0000 7fc776008740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5596e11887d0 con 0x5596e1226210 2024-05-15T14:03:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.371+0000 7fc776008740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5596e118c810 con 0x5596e1230ad0 2024-05-15T14:03:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.371+0000 7fc771885640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596e12258f0 0x5596e1225cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.371+0000 7fc771885640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596e12258f0 0x5596e1225cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58602/0 (socket says 172.21.15.33:58602) 2024-05-15T14:03:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.371+0000 7fc771885640 1 -- 172.21.15.33:0/781765463 learned_addr learned my addr 172.21.15.33:0/781765463 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.371+0000 7fc770883640 1 --2- 172.21.15.33:0/781765463 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596e1230ad0 0x5596e1232ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.372+0000 7fc771084640 1 --2- 172.21.15.33:0/781765463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596e1226210 0x5596e1230590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.372+0000 7fc771885640 1 -- 172.21.15.33:0/781765463 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596e1230ad0 msgr2=0x5596e1232ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.372+0000 7fc771885640 1 --2- 172.21.15.33:0/781765463 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596e1230ad0 0x5596e1232ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.372+0000 7fc771885640 1 -- 172.21.15.33:0/781765463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596e1226210 msgr2=0x5596e1230590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.372+0000 7fc771885640 1 --2- 172.21.15.33:0/781765463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596e1226210 0x5596e1230590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.372+0000 7fc771885640 1 -- 172.21.15.33:0/781765463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596e0e9d210 con 0x5596e12258f0 2024-05-15T14:03:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.372+0000 7fc771084640 1 --2- 172.21.15.33:0/781765463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596e1226210 0x5596e1230590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:03:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.372+0000 7fc76a7fc640 1 --2- 172.21.15.33:0/2240430463 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc7540b2950 0x7fc7540b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.372+0000 7fc771885640 1 --2- 172.21.15.33:0/781765463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596e12258f0 0x5596e1225cd0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc764009ca0 tx=0x7fc764011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ecd754f4a6c5b33 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.372+0000 7fc771084640 1 --2- 172.21.15.33:0/2240430463 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc7540b2950 0x7fc7540b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.372+0000 7fc7537fe640 1 -- 172.21.15.33:0/781765463 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76401e070 con 0x5596e12258f0 2024-05-15T14:03:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.372+0000 7fc7537fe640 1 -- 172.21.15.33:0/781765463 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc764009e50 con 0x5596e12258f0 2024-05-15T14:03:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.372+0000 7fc7537fe640 1 -- 172.21.15.33:0/781765463 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76400e9f0 con 0x5596e12258f0 2024-05-15T14:03:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.372+0000 7fc76a7fc640 1 -- 172.21.15.33:0/2240430463 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fc75c0e2c10 con 0x7fc76c101930 2024-05-15T14:03:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.373+0000 7fc776008740 1 -- 172.21.15.33:0/781765463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596e12258f0 msgr2=0x5596e1225cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.373+0000 7fc771084640 1 --2- 172.21.15.33:0/2240430463 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc7540b2950 0x7fc7540b4e10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc75c0e0420 tx=0x7fc75c004930 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.373+0000 7fc776008740 1 --2- 172.21.15.33:0/781765463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596e12258f0 0x5596e1225cd0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc764009ca0 tx=0x7fc764011810 comp rx=0 tx=0).stop 2024-05-15T14:03:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.373+0000 7fc776008740 1 -- 172.21.15.33:0/781765463 shutdown_connections 2024-05-15T14:03:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.373+0000 7fc776008740 1 --2- 172.21.15.33:0/781765463 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596e1230ad0 0x5596e1232ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.373+0000 7fc776008740 1 --2- 172.21.15.33:0/781765463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596e1226210 0x5596e1230590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.373+0000 7fc776008740 1 --2- 172.21.15.33:0/781765463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596e12258f0 0x5596e1225cd0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.373+0000 7fc776008740 1 -- 172.21.15.33:0/781765463 >> 172.21.15.33:0/781765463 conn(0x5596e1221e00 msgr2=0x5596e12221e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.373+0000 7fc776008740 1 -- 172.21.15.33:0/781765463 shutdown_connections 2024-05-15T14:03:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.373+0000 7fc776008740 1 -- 172.21.15.33:0/781765463 wait complete. 2024-05-15T14:03:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.373+0000 7fc776008740 1 -- start start 2024-05-15T14:03:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.373+0000 7fc776008740 1 Processor -- start 2024-05-15T14:03:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.374+0000 7fc776008740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596e12258f0 0x5596e1230460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.374+0000 7fc776008740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596e1226210 0x5596e122d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.374+0000 7fc776008740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596e1230ad0 0x5596e122dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.374+0000 7fc776008740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5596e0e6e790 con 0x5596e1230ad0 2024-05-15T14:03:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.374+0000 7fc776008740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5596e1144890 con 0x5596e1226210 2024-05-15T14:03:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.374+0000 7fc776008740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5596e12337a0 con 0x5596e12258f0 2024-05-15T14:03:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.374+0000 7fc776008740 10 client.0 _ll_register_callbacks cb 0x5596e1147d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:03:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.374+0000 7fc771885640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596e12258f0 0x5596e1230460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.374+0000 7fc771885640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596e1230ad0 0x5596e122dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.374+0000 7fc771885640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596e1230ad0 0x5596e122dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58608/0 (socket says 172.21.15.33:58608) 2024-05-15T14:03:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.374+0000 7fc771885640 1 -- 172.21.15.33:0/234993495 learned_addr learned my addr 172.21.15.33:0/234993495 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.374+0000 7fc770883640 1 --2- 172.21.15.33:0/234993495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596e1226210 0x5596e122d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.374+0000 7fc771885640 1 -- 172.21.15.33:0/234993495 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596e12258f0 msgr2=0x5596e1230460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.374+0000 7fc771885640 1 --2- 172.21.15.33:0/234993495 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596e12258f0 0x5596e1230460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.374+0000 7fc771885640 1 -- 172.21.15.33:0/234993495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596e1226210 msgr2=0x5596e122d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.374+0000 7fc771885640 1 --2- 172.21.15.33:0/234993495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596e1226210 0x5596e122d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.374+0000 7fc771885640 1 -- 172.21.15.33:0/234993495 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596e0f69620 con 0x5596e1230ad0 2024-05-15T14:03:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.374+0000 7fc771885640 1 --2- 172.21.15.33:0/234993495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596e1230ad0 0x5596e122dcd0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc76401c040 tx=0x7fc764000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd2630dcf80eaa37 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.374+0000 7fc750ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:03:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.375+0000 7fc750ff9640 1 -- 172.21.15.33:0/234993495 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76402d060 con 0x5596e1230ad0 2024-05-15T14:03:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.375+0000 7fc750ff9640 1 -- 172.21.15.33:0/234993495 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fc764004460 con 0x5596e1230ad0 2024-05-15T14:03:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.375+0000 7fc776008740 1 -- 172.21.15.33:0/234993495 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc764009e50 con 0x5596e1230ad0 2024-05-15T14:03:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.375+0000 7fc750ff9640 1 -- 172.21.15.33:0/234993495 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc76401e040 con 0x5596e1230ad0 2024-05-15T14:03:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.375+0000 7fc776008740 10 client.15657 fetch_fsmap learned FSMap version 25 2024-05-15T14:03:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.375+0000 7fc776008740 1 -- 172.21.15.33:0/234993495 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5596e123f1e0 con 0x5596e1230ad0 2024-05-15T14:03:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.375+0000 7fc750ff9640 1 -- 172.21.15.33:0/234993495 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc764009e50 con 0x5596e1230ad0 2024-05-15T14:03:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.375+0000 7fc750ff9640 1 -- 172.21.15.33:0/234993495 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc764004ce0 con 0x5596e1230ad0 2024-05-15T14:03:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.375+0000 7fc776008740 10 client.15657 fetch_fsmap finished waiting for FSMap version 25 2024-05-15T14:03:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.375+0000 7fc776008740 10 client.15657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:03:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.375+0000 7fc776008740 20 client.15657 populate_metadata read hostname 'smithi033' 2024-05-15T14:03:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.375+0000 7fc776008740 1 --2- 172.21.15.33:0/234993495 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5596e1240640 0x5596e1242a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.375+0000 7fc776008740 4 client.15657 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:03:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.375+0000 7fc776008740 1 -- 172.21.15.33:0/234993495 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596e119ab80 con 0x5596e1240640 2024-05-15T14:03:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.375+0000 7fc770883640 1 --2- 172.21.15.33:0/234993495 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5596e1240640 0x5596e1242a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.376+0000 7fc750ff9640 10 client.15657 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:03:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.376+0000 7fc770883640 1 --2- 172.21.15.33:0/234993495 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5596e1240640 0x5596e1242a30 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=715222a4c97a2f2e server_cookie=2828353faedf8f58 in_seq=0 out_seq=0 2024-05-15T14:03:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.377+0000 7fc750ff9640 1 -- 172.21.15.33:0/234993495 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5596e119ab80 con 0x5596e1240640 2024-05-15T14:03:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.377+0000 7fc750ff9640 10 client.15657 handle_command_reply: tid=0 2024-05-15T14:03:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.377+0000 7fc776008740 1 client.15657 shutdown 2024-05-15T14:03:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.377+0000 7fc776008740 1 -- 172.21.15.33:0/234993495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596e1230ad0 msgr2=0x5596e122dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.377+0000 7fc776008740 1 --2- 172.21.15.33:0/234993495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596e1230ad0 0x5596e122dcd0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc76401c040 tx=0x7fc764000bc0 comp rx=0 tx=0).stop 2024-05-15T14:03:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.377+0000 7fc776008740 1 -- 172.21.15.33:0/234993495 shutdown_connections 2024-05-15T14:03:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.377+0000 7fc776008740 1 --2- 172.21.15.33:0/234993495 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5596e1240640 0x5596e1242a30 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.377+0000 7fc776008740 1 --2- 172.21.15.33:0/234993495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596e1230ad0 0x5596e122dcd0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.377+0000 7fc776008740 1 --2- 172.21.15.33:0/234993495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596e1226210 0x5596e122d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.377+0000 7fc750ff9640 0 client.15657 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:03:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.377+0000 7fc776008740 1 --2- 172.21.15.33:0/234993495 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596e12258f0 0x5596e1230460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.377+0000 7fc776008740 1 -- 172.21.15.33:0/234993495 >> 172.21.15.33:0/234993495 conn(0x5596e1221e00 msgr2=0x5596e1221100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.377+0000 7fc776008740 1 -- 172.21.15.33:0/234993495 shutdown_connections 2024-05-15T14:03:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.377+0000 7fc776008740 1 -- 172.21.15.33:0/234993495 wait complete. 2024-05-15T14:03:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.378+0000 7fc776008740 20 client.15657 trim_cache size 0 max 16384 2024-05-15T14:03:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.393+0000 7fc776008740 1 Processor -- start 2024-05-15T14:03:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.393+0000 7fc776008740 1 -- start start 2024-05-15T14:03:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.393+0000 7fc776008740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596e1221e00 0x5596e123c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.393+0000 7fc776008740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596e12258f0 0x5596e122ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.393+0000 7fc776008740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596e1226210 0x5596e1248170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.393+0000 7fc776008740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c1b3260 con 0x5596e1226210 2024-05-15T14:03:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.394+0000 7fc776008740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c1b0b30 con 0x5596e1221e00 2024-05-15T14:03:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.394+0000 7fc776008740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c1ae980 con 0x5596e12258f0 2024-05-15T14:03:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.394+0000 7fc771885640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596e1226210 0x5596e1248170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.394+0000 7fc771885640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596e1226210 0x5596e1248170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58624/0 (socket says 172.21.15.33:58624) 2024-05-15T14:03:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.394+0000 7fc771885640 1 -- 172.21.15.33:0/4133342144 learned_addr learned my addr 172.21.15.33:0/4133342144 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.394+0000 7fc770883640 1 --2- 172.21.15.33:0/4133342144 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596e12258f0 0x5596e122ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.394+0000 7fc771885640 1 --2- 172.21.15.33:0/4133342144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596e1221e00 0x5596e123c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.394+0000 7fc771885640 1 -- 172.21.15.33:0/4133342144 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596e12258f0 msgr2=0x5596e122ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.394+0000 7fc771885640 1 --2- 172.21.15.33:0/4133342144 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596e12258f0 0x5596e122ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.394+0000 7fc771885640 1 -- 172.21.15.33:0/4133342144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596e1221e00 msgr2=0x5596e123c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.394+0000 7fc771885640 1 --2- 172.21.15.33:0/4133342144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596e1221e00 0x5596e123c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.394+0000 7fc771885640 1 -- 172.21.15.33:0/4133342144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596e123cbb0 con 0x5596e1226210 2024-05-15T14:03:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.394+0000 7fc771885640 1 --2- 172.21.15.33:0/4133342144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596e1226210 0x5596e1248170 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fc7640051f0 tx=0x7fc764014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe7d2a819d0e9a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.394+0000 7fc752ffd640 1 -- 172.21.15.33:0/4133342144 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76402d040 con 0x5596e1226210 2024-05-15T14:03:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.394+0000 7fc752ffd640 1 -- 172.21.15.33:0/4133342144 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc764009e50 con 0x5596e1226210 2024-05-15T14:03:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.394+0000 7fc752ffd640 1 -- 172.21.15.33:0/4133342144 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76400e960 con 0x5596e1226210 2024-05-15T14:03:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.395+0000 7fc776008740 1 -- 172.21.15.33:0/4133342144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596e1226210 msgr2=0x5596e1248170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.395+0000 7fc776008740 1 --2- 172.21.15.33:0/4133342144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596e1226210 0x5596e1248170 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fc7640051f0 tx=0x7fc764014040 comp rx=0 tx=0).stop 2024-05-15T14:03:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.395+0000 7fc776008740 1 -- 172.21.15.33:0/4133342144 shutdown_connections 2024-05-15T14:03:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.395+0000 7fc776008740 1 --2- 172.21.15.33:0/4133342144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596e1226210 0x5596e1248170 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.395+0000 7fc776008740 1 --2- 172.21.15.33:0/4133342144 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596e12258f0 0x5596e122ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.395+0000 7fc776008740 1 --2- 172.21.15.33:0/4133342144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596e1221e00 0x5596e123c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.395+0000 7fc776008740 1 -- 172.21.15.33:0/4133342144 >> 172.21.15.33:0/4133342144 conn(0x5596e121d400 msgr2=0x5596e1227b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.395+0000 7fc776008740 1 -- 172.21.15.33:0/4133342144 shutdown_connections 2024-05-15T14:03:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.395+0000 7fc776008740 1 -- 172.21.15.33:0/4133342144 wait complete. 2024-05-15T14:03:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.395+0000 7fc776008740 1 -- start start 2024-05-15T14:03:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.395+0000 7fc776008740 1 Processor -- start 2024-05-15T14:03:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.396+0000 7fc776008740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596e1221e00 0x5596e12390b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.396+0000 7fc776008740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596e12258f0 0x5596e1234420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.396+0000 7fc776008740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596e1226210 0x5596e1234b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.396+0000 7fc776008740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c1aeff0 con 0x5596e1221e00 2024-05-15T14:03:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.396+0000 7fc776008740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c1b0460 con 0x5596e1226210 2024-05-15T14:03:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.396+0000 7fc776008740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c1af6c0 con 0x5596e12258f0 2024-05-15T14:03:15.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.396+0000 7fc776008740 10 client.0 _ll_register_callbacks cb 0x7fc764005650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:03:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.396+0000 7fc771885640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596e1221e00 0x5596e12390b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.396+0000 7fc771885640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596e1221e00 0x5596e12390b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58634/0 (socket says 172.21.15.33:58634) 2024-05-15T14:03:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.396+0000 7fc771885640 1 -- 172.21.15.33:0/2989652774 learned_addr learned my addr 172.21.15.33:0/2989652774 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.396+0000 7fc770883640 1 --2- 172.21.15.33:0/2989652774 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596e12258f0 0x5596e1234420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.396+0000 7fc771885640 1 --2- 172.21.15.33:0/2989652774 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596e1226210 0x5596e1234b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.396+0000 7fc771885640 1 -- 172.21.15.33:0/2989652774 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596e12258f0 msgr2=0x5596e1234420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.396+0000 7fc771885640 1 --2- 172.21.15.33:0/2989652774 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596e12258f0 0x5596e1234420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.396+0000 7fc771885640 1 -- 172.21.15.33:0/2989652774 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596e1226210 msgr2=0x5596e1234b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.396+0000 7fc771885640 1 --2- 172.21.15.33:0/2989652774 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596e1226210 0x5596e1234b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.396+0000 7fc771885640 1 -- 172.21.15.33:0/2989652774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596e12a9d20 con 0x5596e1221e00 2024-05-15T14:03:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.396+0000 7fc771885640 1 --2- 172.21.15.33:0/2989652774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596e1221e00 0x5596e12390b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc764000c00 tx=0x7fc7640038d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d685c3237588ff4c server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.396+0000 7fc750ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:03:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.397+0000 7fc750ff9640 1 -- 172.21.15.33:0/2989652774 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc764046020 con 0x5596e1221e00 2024-05-15T14:03:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.397+0000 7fc750ff9640 1 -- 172.21.15.33:0/2989652774 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7fc7640039b0 con 0x5596e1221e00 2024-05-15T14:03:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.397+0000 7fc776008740 1 -- 172.21.15.33:0/2989652774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc764009e50 con 0x5596e1221e00 2024-05-15T14:03:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.397+0000 7fc750ff9640 1 -- 172.21.15.33:0/2989652774 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc76402d040 con 0x5596e1221e00 2024-05-15T14:03:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.397+0000 7fc776008740 10 client.15663 fetch_fsmap learned FSMap version 25 2024-05-15T14:03:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.397+0000 7fc776008740 1 -- 172.21.15.33:0/2989652774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5596e12aaeb0 con 0x5596e1221e00 2024-05-15T14:03:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.397+0000 7fc750ff9640 1 -- 172.21.15.33:0/2989652774 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc764009e50 con 0x5596e1221e00 2024-05-15T14:03:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.397+0000 7fc750ff9640 1 -- 172.21.15.33:0/2989652774 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc7640048c0 con 0x5596e1221e00 2024-05-15T14:03:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.397+0000 7fc776008740 10 client.15663 fetch_fsmap finished waiting for FSMap version 25 2024-05-15T14:03:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.397+0000 7fc776008740 10 client.15663 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:03:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.397+0000 7fc776008740 20 client.15663 populate_metadata read hostname 'smithi033' 2024-05-15T14:03:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.397+0000 7fc776008740 1 --2- 172.21.15.33:0/2989652774 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5596e1230ad0 0x5596e12ad4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.397+0000 7fc776008740 4 client.15663 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:03:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.397+0000 7fc776008740 1 -- 172.21.15.33:0/2989652774 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5596e1194360 con 0x5596e1230ad0 2024-05-15T14:03:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.397+0000 7fc770883640 1 --2- 172.21.15.33:0/2989652774 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5596e1230ad0 0x5596e12ad4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.398+0000 7fc770883640 1 --2- 172.21.15.33:0/2989652774 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5596e1230ad0 0x5596e12ad4e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3d4cd5f609e01c2 server_cookie=4b399b7fb93a851d in_seq=0 out_seq=0 2024-05-15T14:03:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.398+0000 7fc750ff9640 10 client.15663 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:03:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.459+0000 7fc750ff9640 1 -- 172.21.15.33:0/2989652774 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5596e1194360 con 0x5596e1230ad0 2024-05-15T14:03:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.459+0000 7fc750ff9640 10 client.15663 handle_command_reply: tid=0 2024-05-15T14:03:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.459+0000 7fc776008740 1 client.15663 shutdown 2024-05-15T14:03:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.459+0000 7fc776008740 1 -- 172.21.15.33:0/2989652774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596e1221e00 msgr2=0x5596e12390b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.459+0000 7fc776008740 1 --2- 172.21.15.33:0/2989652774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596e1221e00 0x5596e12390b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc764000c00 tx=0x7fc7640038d0 comp rx=0 tx=0).stop 2024-05-15T14:03:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.460+0000 7fc776008740 1 -- 172.21.15.33:0/2989652774 shutdown_connections 2024-05-15T14:03:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.460+0000 7fc776008740 1 --2- 172.21.15.33:0/2989652774 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5596e1230ad0 0x5596e12ad4e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.460+0000 7fc776008740 1 --2- 172.21.15.33:0/2989652774 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596e1226210 0x5596e1234b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.460+0000 7fc776008740 1 --2- 172.21.15.33:0/2989652774 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5596e12258f0 0x5596e1234420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.460+0000 7fc776008740 1 --2- 172.21.15.33:0/2989652774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5596e1221e00 0x5596e12390b0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.460+0000 7fc776008740 1 -- 172.21.15.33:0/2989652774 >> 172.21.15.33:0/2989652774 conn(0x5596e121d400 msgr2=0x5596e1227210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.460+0000 7fc750ff9640 0 client.15663 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:03:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.460+0000 7fc776008740 1 -- 172.21.15.33:0/2989652774 shutdown_connections 2024-05-15T14:03:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.460+0000 7fc776008740 1 -- 172.21.15.33:0/2989652774 wait complete. 2024-05-15T14:03:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.460+0000 7fc776008740 20 client.15663 trim_cache size 0 max 16384 2024-05-15T14:03:15.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:03:15.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:03:15.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:03:15.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.460+0000 7fc7732e8640 1 -- 172.21.15.33:0/2240430463 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc7540b2950 msgr2=0x7fc7540b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:15.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.460+0000 7fc7732e8640 1 --2- 172.21.15.33:0/2240430463 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc7540b2950 0x7fc7540b4e10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc75c0e0420 tx=0x7fc75c004930 comp rx=0 tx=0).stop 2024-05-15T14:03:15.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.460+0000 7fc7732e8640 1 -- 172.21.15.33:0/2240430463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc76c101930 msgr2=0x7fc76c0fe8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:15.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.460+0000 7fc7732e8640 1 --2- 172.21.15.33:0/2240430463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc76c101930 0x7fc76c0fe8f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc75c009d50 tx=0x7fc75c00c6a0 comp rx=0 tx=0).stop 2024-05-15T14:03:15.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.460+0000 7fc7732e8640 1 -- 172.21.15.33:0/2240430463 shutdown_connections 2024-05-15T14:03:15.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.460+0000 7fc7732e8640 1 --2- 172.21.15.33:0/2240430463 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc7540b2950 0x7fc7540b4e10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:15.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.460+0000 7fc7732e8640 1 --2- 172.21.15.33:0/2240430463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc76c107940 0x7fc76c0ff370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:15.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.460+0000 7fc7732e8640 1 --2- 172.21.15.33:0/2240430463 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc76c1022f0 0x7fc76c0fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:15.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.460+0000 7fc7732e8640 1 --2- 172.21.15.33:0/2240430463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc76c101930 0x7fc76c0fe8f0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:15.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.460+0000 7fc7732e8640 1 -- 172.21.15.33:0/2240430463 >> 172.21.15.33:0/2240430463 conn(0x7fc76c078a70 msgr2=0x7fc76c06a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:15.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.461+0000 7fc7732e8640 1 -- 172.21.15.33:0/2240430463 shutdown_connections 2024-05-15T14:03:15.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:15.461+0000 7fc7732e8640 1 -- 172.21.15.33:0/2240430463 wait complete. 2024-05-15T14:03:16.049 INFO:tasks.workunit.client.0.smithi033.stdout: 1 108835 11.80 MB/sec warmup 51 sec latency 21.759 ms 2024-05-15T14:03:16.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:16 smithi081 ceph-mon[36434]: pgmap v90: 97 pgs: 97 active+clean; 37 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 244 op/s 2024-05-15T14:03:16.642 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:16 smithi052 ceph-mon[36190]: pgmap v90: 97 pgs: 97 active+clean; 37 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 244 op/s 2024-05-15T14:03:16.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:16 smithi033 ceph-mon[31650]: pgmap v90: 97 pgs: 97 active+clean; 37 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 244 op/s 2024-05-15T14:03:17.049 INFO:tasks.workunit.client.0.smithi033.stdout: 1 110508 11.80 MB/sec warmup 52 sec latency 13.002 ms 2024-05-15T14:03:18.049 INFO:tasks.workunit.client.0.smithi033.stdout: 1 113385 11.86 MB/sec warmup 53 sec latency 13.012 ms 2024-05-15T14:03:18.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:18 smithi081 ceph-mon[36434]: pgmap v91: 97 pgs: 97 active+clean; 39 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.9 MiB/s wr, 395 op/s 2024-05-15T14:03:18.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:18 smithi033 ceph-mon[31650]: pgmap v91: 97 pgs: 97 active+clean; 39 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.9 MiB/s wr, 395 op/s 2024-05-15T14:03:18.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:18 smithi052 ceph-mon[36190]: pgmap v91: 97 pgs: 97 active+clean; 39 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.9 MiB/s wr, 395 op/s 2024-05-15T14:03:19.049 INFO:tasks.workunit.client.0.smithi033.stdout: 1 115073 11.82 MB/sec warmup 54 sec latency 13.039 ms 2024-05-15T14:03:20.049 INFO:tasks.workunit.client.0.smithi033.stdout: 1 117518 11.86 MB/sec warmup 55 sec latency 16.682 ms 2024-05-15T14:03:20.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:20 smithi081 ceph-mon[36434]: pgmap v92: 97 pgs: 97 active+clean; 39 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.3 MiB/s wr, 255 op/s 2024-05-15T14:03:20.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:20 smithi033 ceph-mon[31650]: pgmap v92: 97 pgs: 97 active+clean; 39 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.3 MiB/s wr, 255 op/s 2024-05-15T14:03:20.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:20 smithi052 ceph-mon[36190]: pgmap v92: 97 pgs: 97 active+clean; 39 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.3 MiB/s wr, 255 op/s 2024-05-15T14:03:21.049 INFO:tasks.workunit.client.0.smithi033.stdout: 1 119869 11.84 MB/sec warmup 56 sec latency 27.335 ms 2024-05-15T14:03:22.049 INFO:tasks.workunit.client.0.smithi033.stdout: 1 121613 11.79 MB/sec warmup 57 sec latency 15.248 ms 2024-05-15T14:03:22.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:22 smithi081 ceph-mon[36434]: pgmap v93: 97 pgs: 97 active+clean; 39 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.3 MiB/s wr, 255 op/s 2024-05-15T14:03:22.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:22 smithi033 ceph-mon[31650]: pgmap v93: 97 pgs: 97 active+clean; 39 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.3 MiB/s wr, 255 op/s 2024-05-15T14:03:22.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:22 smithi052 ceph-mon[36190]: pgmap v93: 97 pgs: 97 active+clean; 39 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.3 MiB/s wr, 255 op/s 2024-05-15T14:03:23.049 INFO:tasks.workunit.client.0.smithi033.stdout: 1 124457 11.90 MB/sec warmup 58 sec latency 13.233 ms 2024-05-15T14:03:24.049 INFO:tasks.workunit.client.0.smithi033.stdout: 1 126604 11.82 MB/sec warmup 59 sec latency 13.086 ms 2024-05-15T14:03:24.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:24 smithi081 ceph-mon[36434]: pgmap v94: 97 pgs: 97 active+clean; 45 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.0 MiB/s wr, 366 op/s 2024-05-15T14:03:24.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:24 smithi033 ceph-mon[31650]: pgmap v94: 97 pgs: 97 active+clean; 45 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.0 MiB/s wr, 366 op/s 2024-05-15T14:03:24.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:24 smithi052 ceph-mon[36190]: pgmap v94: 97 pgs: 97 active+clean; 45 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.0 MiB/s wr, 366 op/s 2024-05-15T14:03:25.049 INFO:tasks.workunit.client.0.smithi033.stdout: 1 128443 11.83 MB/sec warmup 60 sec latency 18.943 ms 2024-05-15T14:03:26.049 INFO:tasks.workunit.client.0.smithi033.stdout: 1 131391 11.92 MB/sec warmup 61 sec latency 11.309 ms 2024-05-15T14:03:26.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:26 smithi081 ceph-mon[36434]: pgmap v95: 97 pgs: 97 active+clean; 45 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 261 op/s 2024-05-15T14:03:26.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:26 smithi033 ceph-mon[31650]: pgmap v95: 97 pgs: 97 active+clean; 45 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 261 op/s 2024-05-15T14:03:26.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:26 smithi052 ceph-mon[36190]: pgmap v95: 97 pgs: 97 active+clean; 45 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 261 op/s 2024-05-15T14:03:27.049 INFO:tasks.workunit.client.0.smithi033.stdout: 1 133052 11.84 MB/sec warmup 62 sec latency 13.185 ms 2024-05-15T14:03:28.049 INFO:tasks.workunit.client.0.smithi033.stdout: 1 135290 11.87 MB/sec warmup 63 sec latency 10.293 ms 2024-05-15T14:03:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.555+0000 7f0878338640 1 -- 172.21.15.33:0/919539263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0870108a50 msgr2=0x7f0870108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.555+0000 7f0878338640 1 --2- 172.21.15.33:0/919539263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0870108a50 0x7f0870108e30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f086c009930 tx=0x7f086c02a420 comp rx=0 tx=0).stop 2024-05-15T14:03:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.555+0000 7f0878338640 1 -- 172.21.15.33:0/919539263 shutdown_connections 2024-05-15T14:03:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.555+0000 7f0878338640 1 --2- 172.21.15.33:0/919539263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0870103400 0x7f087010f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.555+0000 7f0878338640 1 --2- 172.21.15.33:0/919539263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0870102a40 0x7f0870102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.555+0000 7f0878338640 1 --2- 172.21.15.33:0/919539263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0870108a50 0x7f0870108e30 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.555+0000 7f0878338640 1 -- 172.21.15.33:0/919539263 >> 172.21.15.33:0/919539263 conn(0x7f08700fe510 msgr2=0x7f0870100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.556+0000 7f0878338640 1 -- 172.21.15.33:0/919539263 shutdown_connections 2024-05-15T14:03:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.556+0000 7f0878338640 1 -- 172.21.15.33:0/919539263 wait complete. 2024-05-15T14:03:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.556+0000 7f0878338640 1 Processor -- start 2024-05-15T14:03:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.556+0000 7f0878338640 1 -- start start 2024-05-15T14:03:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.556+0000 7f0878338640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0870102a40 0x7f08701992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.556+0000 7f0878338640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0870103400 0x7f08701997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.556+0000 7f0878338640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0870108a50 0x7f087019db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.556+0000 7f0878338640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0870112d10 con 0x7f0870102a40 2024-05-15T14:03:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.556+0000 7f0878338640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0870112b90 con 0x7f0870108a50 2024-05-15T14:03:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.556+0000 7f0878338640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0870112e90 con 0x7f0870103400 2024-05-15T14:03:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.556+0000 7f08760d4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0870102a40 0x7f08701992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.556+0000 7f08760d4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0870102a40 0x7f08701992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57340/0 (socket says 172.21.15.33:57340) 2024-05-15T14:03:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.556+0000 7f08760d4640 1 -- 172.21.15.33:0/2817652714 learned_addr learned my addr 172.21.15.33:0/2817652714 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.557+0000 7f08758d3640 1 --2- 172.21.15.33:0/2817652714 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0870103400 0x7f08701997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.557+0000 7f08768d5640 1 --2- 172.21.15.33:0/2817652714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0870108a50 0x7f087019db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.557+0000 7f08760d4640 1 -- 172.21.15.33:0/2817652714 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0870103400 msgr2=0x7f08701997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.557+0000 7f08760d4640 1 --2- 172.21.15.33:0/2817652714 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0870103400 0x7f08701997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.557+0000 7f08760d4640 1 -- 172.21.15.33:0/2817652714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0870108a50 msgr2=0x7f087019db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.557+0000 7f08760d4640 1 --2- 172.21.15.33:0/2817652714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0870108a50 0x7f087019db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.557+0000 7f08760d4640 1 -- 172.21.15.33:0/2817652714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f087019e2a0 con 0x7f0870102a40 2024-05-15T14:03:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.557+0000 7f08760d4640 1 --2- 172.21.15.33:0/2817652714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0870102a40 0x7f08701992b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f086c005940 tx=0x7f086c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.557+0000 7f08677fe640 1 -- 172.21.15.33:0/2817652714 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c03f030 con 0x7f0870102a40 2024-05-15T14:03:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.557+0000 7f08677fe640 1 -- 172.21.15.33:0/2817652714 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f086c003cb0 con 0x7f0870102a40 2024-05-15T14:03:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.557+0000 7f0878338640 1 -- 172.21.15.33:0/2817652714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f087019e530 con 0x7f0870102a40 2024-05-15T14:03:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.557+0000 7f08677fe640 1 -- 172.21.15.33:0/2817652714 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c035070 con 0x7f0870102a40 2024-05-15T14:03:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.557+0000 7f0878338640 1 -- 172.21.15.33:0/2817652714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08701a5e20 con 0x7f0870102a40 2024-05-15T14:03:28.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.560+0000 7f087b051740 1 Processor -- start 2024-05-15T14:03:28.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.560+0000 7f087b051740 1 -- start start 2024-05-15T14:03:28.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.560+0000 7f08677fe640 1 -- 172.21.15.33:0/2817652714 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f086c042030 con 0x7f0870102a40 2024-05-15T14:03:28.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.560+0000 7f087b051740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d7599b88f0 0x55d7599b8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:28.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.560+0000 7f087b051740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7599b9210 0x55d7599c3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:28.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.560+0000 7f087b051740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d7599c3ad0 0x55d7599c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:28.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.560+0000 7f087b051740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d75992bc30 con 0x55d7599b9210 2024-05-15T14:03:28.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.560+0000 7f087b051740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d75991b7d0 con 0x55d7599b88f0 2024-05-15T14:03:28.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.560+0000 7f087b051740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55d75991f810 con 0x55d7599c3ad0 2024-05-15T14:03:28.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.561+0000 7f08760d4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7599b9210 0x55d7599c3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:28.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.561+0000 7f08758d3640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d7599c3ad0 0x55d7599c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:28.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.561+0000 7f08758d3640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d7599c3ad0 0x55d7599c5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:48488/0 (socket says 172.21.15.33:48488) 2024-05-15T14:03:28.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.561+0000 7f08768d5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d7599b88f0 0x55d7599b8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:28.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.561+0000 7f08758d3640 1 -- 172.21.15.33:0/4057304741 learned_addr learned my addr 172.21.15.33:0/4057304741 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:28.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.561+0000 7f08758d3640 1 -- 172.21.15.33:0/4057304741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d7599b88f0 msgr2=0x55d7599b8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:28.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.561+0000 7f08758d3640 1 --2- 172.21.15.33:0/4057304741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d7599b88f0 0x55d7599b8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:28.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.561+0000 7f08758d3640 1 -- 172.21.15.33:0/4057304741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7599b9210 msgr2=0x55d7599c3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:28.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.561+0000 7f08758d3640 1 --2- 172.21.15.33:0/4057304741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7599b9210 0x55d7599c3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:28.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.561+0000 7f08758d3640 1 -- 172.21.15.33:0/4057304741 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d759630210 con 0x55d7599c3ad0 2024-05-15T14:03:28.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.561+0000 7f08758d3640 1 --2- 172.21.15.33:0/4057304741 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d7599c3ad0 0x55d7599c5ec0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f086000bcb0 tx=0x7f08600107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4667e93f0665f807 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:28.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.561+0000 7f0864bfe640 1 -- 172.21.15.33:0/4057304741 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0860004030 con 0x55d7599c3ad0 2024-05-15T14:03:28.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.561+0000 7f0864bfe640 1 -- 172.21.15.33:0/4057304741 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08600041d0 con 0x55d7599c3ad0 2024-05-15T14:03:28.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.561+0000 7f0864bfe640 1 -- 172.21.15.33:0/4057304741 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08600117b0 con 0x55d7599c3ad0 2024-05-15T14:03:28.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.561+0000 7f08677fe640 1 --2- 172.21.15.33:0/2817652714 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f085c0b29d0 0x7f085c0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:28.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.562+0000 7f08768d5640 1 --2- 172.21.15.33:0/2817652714 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f085c0b29d0 0x7f085c0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:28.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.562+0000 7f08677fe640 1 -- 172.21.15.33:0/2817652714 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f086c102d10 con 0x7f0870102a40 2024-05-15T14:03:28.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.562+0000 7f087b051740 1 -- 172.21.15.33:0/4057304741 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d7599c3ad0 msgr2=0x55d7599c5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:28.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.562+0000 7f087b051740 1 --2- 172.21.15.33:0/4057304741 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d7599c3ad0 0x55d7599c5ec0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f086000bcb0 tx=0x7f08600107a0 comp rx=0 tx=0).stop 2024-05-15T14:03:28.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.562+0000 7f08768d5640 1 --2- 172.21.15.33:0/2817652714 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f085c0b29d0 0x7f085c0b4e90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0868005fd0 tx=0x7f0868009500 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:28.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.562+0000 7f087b051740 1 -- 172.21.15.33:0/4057304741 shutdown_connections 2024-05-15T14:03:28.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.562+0000 7f087b051740 1 --2- 172.21.15.33:0/4057304741 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d7599c3ad0 0x55d7599c5ec0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:28.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.562+0000 7f087b051740 1 --2- 172.21.15.33:0/4057304741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7599b9210 0x55d7599c3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:28.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.562+0000 7f087b051740 1 --2- 172.21.15.33:0/4057304741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d7599b88f0 0x55d7599b8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:28.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.562+0000 7f087b051740 1 -- 172.21.15.33:0/4057304741 >> 172.21.15.33:0/4057304741 conn(0x55d7599b4e00 msgr2=0x55d7599b51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.562+0000 7f087b051740 1 -- 172.21.15.33:0/4057304741 shutdown_connections 2024-05-15T14:03:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.563+0000 7f087b051740 1 -- 172.21.15.33:0/4057304741 wait complete. 2024-05-15T14:03:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.563+0000 7f087b051740 1 -- start start 2024-05-15T14:03:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.563+0000 7f087b051740 1 Processor -- start 2024-05-15T14:03:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.563+0000 7f087b051740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7599b88f0 0x55d7599c3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.563+0000 7f087b051740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d7599b9210 0x55d7599c0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.563+0000 7f087b051740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d7599c3ad0 0x55d7599c0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.563+0000 7f087b051740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d759601790 con 0x55d7599b88f0 2024-05-15T14:03:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.563+0000 7f087b051740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d7598d7890 con 0x55d7599b9210 2024-05-15T14:03:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.563+0000 7f087b051740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55d7599c67a0 con 0x55d7599c3ad0 2024-05-15T14:03:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.563+0000 7f08760d4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7599b88f0 0x55d7599c3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.563+0000 7f087b051740 10 client.0 _ll_register_callbacks cb 0x55d7598dad60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:03:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.563+0000 7f08758d3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d7599b9210 0x55d7599c0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.563+0000 7f08760d4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7599b88f0 0x55d7599c3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57362/0 (socket says 172.21.15.33:57362) 2024-05-15T14:03:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.563+0000 7f08760d4640 1 -- 172.21.15.33:0/3743333358 learned_addr learned my addr 172.21.15.33:0/3743333358 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.563+0000 7f08768d5640 1 --2- 172.21.15.33:0/3743333358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d7599c3ad0 0x55d7599c0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.564+0000 7f08758d3640 1 -- 172.21.15.33:0/3743333358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d7599c3ad0 msgr2=0x55d7599c0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.564+0000 7f08758d3640 1 --2- 172.21.15.33:0/3743333358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d7599c3ad0 0x55d7599c0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.564+0000 7f08758d3640 1 -- 172.21.15.33:0/3743333358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7599b88f0 msgr2=0x55d7599c3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.564+0000 7f08758d3640 1 --2- 172.21.15.33:0/3743333358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7599b88f0 0x55d7599c3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.564+0000 7f08758d3640 1 -- 172.21.15.33:0/3743333358 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7596fc620 con 0x55d7599b9210 2024-05-15T14:03:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.564+0000 7f08758d3640 1 --2- 172.21.15.33:0/3743333358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d7599b9210 0x55d7599c0730 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0860011010 tx=0x7f086000b970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60c01501424740c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.564+0000 7f0859ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-05-15T14:03:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.564+0000 7f0859ffb640 1 -- 172.21.15.33:0/3743333358 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086002a060 con 0x55d7599b9210 2024-05-15T14:03:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.564+0000 7f0859ffb640 1 -- 172.21.15.33:0/3743333358 <== mon.2 v2:172.21.15.52:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f0860005040 con 0x55d7599b9210 2024-05-15T14:03:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.564+0000 7f087b051740 1 -- 172.21.15.33:0/3743333358 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08600041d0 con 0x55d7599b9210 2024-05-15T14:03:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.564+0000 7f0859ffb640 1 -- 172.21.15.33:0/3743333358 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f086000bea0 con 0x55d7599b9210 2024-05-15T14:03:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.564+0000 7f087b051740 10 client.25286 fetch_fsmap learned FSMap version 25 2024-05-15T14:03:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.564+0000 7f087b051740 1 -- 172.21.15.33:0/3743333358 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d7599d21c0 con 0x55d7599b9210 2024-05-15T14:03:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.565+0000 7f0859ffb640 1 -- 172.21.15.33:0/3743333358 <== mon.2 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08600041d0 con 0x55d7599b9210 2024-05-15T14:03:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.565+0000 7f0859ffb640 1 -- 172.21.15.33:0/3743333358 <== mon.2 v2:172.21.15.52:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f08600046c0 con 0x55d7599b9210 2024-05-15T14:03:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.565+0000 7f087b051740 10 client.25286 fetch_fsmap finished waiting for FSMap version 25 2024-05-15T14:03:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.565+0000 7f087b051740 10 client.25286 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:03:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.565+0000 7f087b051740 20 client.25286 populate_metadata read hostname 'smithi033' 2024-05-15T14:03:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.565+0000 7f087b051740 1 --2- 172.21.15.33:0/3743333358 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55d7599d35c0 0x55d7599d59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.565+0000 7f087b051740 4 client.25286 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:03:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.565+0000 7f087b051740 1 -- 172.21.15.33:0/3743333358 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d75992db80 con 0x55d7599d35c0 2024-05-15T14:03:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.565+0000 7f08760d4640 1 --2- 172.21.15.33:0/3743333358 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55d7599d35c0 0x55d7599d59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.566+0000 7f08760d4640 1 --2- 172.21.15.33:0/3743333358 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55d7599d35c0 0x55d7599d59b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93419d22a4627610 server_cookie=c2ee81bae55747d9 in_seq=0 out_seq=0 2024-05-15T14:03:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.566+0000 7f0859ffb640 10 client.25286 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:03:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.567+0000 7f0859ffb640 1 -- 172.21.15.33:0/3743333358 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d75992db80 con 0x55d7599d35c0 2024-05-15T14:03:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.567+0000 7f0859ffb640 10 client.25286 handle_command_reply: tid=0 2024-05-15T14:03:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.567+0000 7f087b051740 1 client.25286 shutdown 2024-05-15T14:03:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.567+0000 7f087b051740 1 -- 172.21.15.33:0/3743333358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d7599b9210 msgr2=0x55d7599c0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.567+0000 7f087b051740 1 --2- 172.21.15.33:0/3743333358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d7599b9210 0x55d7599c0730 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0860011010 tx=0x7f086000b970 comp rx=0 tx=0).stop 2024-05-15T14:03:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.567+0000 7f087b051740 1 -- 172.21.15.33:0/3743333358 shutdown_connections 2024-05-15T14:03:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.567+0000 7f087b051740 1 --2- 172.21.15.33:0/3743333358 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55d7599d35c0 0x55d7599d59b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.567+0000 7f087b051740 1 --2- 172.21.15.33:0/3743333358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d7599c3ad0 0x55d7599c0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.567+0000 7f087b051740 1 --2- 172.21.15.33:0/3743333358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d7599b9210 0x55d7599c0730 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.567+0000 7f0859ffb640 0 client.25286 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:03:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.567+0000 7f087b051740 1 --2- 172.21.15.33:0/3743333358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7599b88f0 0x55d7599c3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.567+0000 7f087b051740 1 -- 172.21.15.33:0/3743333358 >> 172.21.15.33:0/3743333358 conn(0x55d7599b4e00 msgr2=0x55d7599b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.567+0000 7f087b051740 1 -- 172.21.15.33:0/3743333358 shutdown_connections 2024-05-15T14:03:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.567+0000 7f087b051740 1 -- 172.21.15.33:0/3743333358 wait complete. 2024-05-15T14:03:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.567+0000 7f087b051740 20 client.25286 trim_cache size 0 max 16384 2024-05-15T14:03:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.583+0000 7f087b051740 1 Processor -- start 2024-05-15T14:03:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.583+0000 7f087b051740 1 -- start start 2024-05-15T14:03:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.583+0000 7f087b051740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d7599b88f0 0x55d7599baa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.583+0000 7f087b051740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7599c3ad0 0x55d7599c2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.583+0000 7f087b051740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d7599b03f0 0x55d7599db000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.583+0000 7f087b051740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f08701b0fe0 con 0x55d7599c3ad0 2024-05-15T14:03:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.583+0000 7f087b051740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f08701ae8b0 con 0x55d7599b88f0 2024-05-15T14:03:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.583+0000 7f087b051740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f08701ac700 con 0x55d7599b03f0 2024-05-15T14:03:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.583+0000 7f08758d3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7599c3ad0 0x55d7599c2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.583+0000 7f08758d3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7599c3ad0 0x55d7599c2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57368/0 (socket says 172.21.15.33:57368) 2024-05-15T14:03:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.583+0000 7f08758d3640 1 -- 172.21.15.33:0/1596325781 learned_addr learned my addr 172.21.15.33:0/1596325781 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.583+0000 7f08768d5640 1 --2- 172.21.15.33:0/1596325781 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d7599b03f0 0x55d7599db000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.583+0000 7f08760d4640 1 --2- 172.21.15.33:0/1596325781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d7599b88f0 0x55d7599baa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.584+0000 7f08758d3640 1 -- 172.21.15.33:0/1596325781 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d7599b03f0 msgr2=0x55d7599db000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.584+0000 7f08758d3640 1 --2- 172.21.15.33:0/1596325781 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d7599b03f0 0x55d7599db000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.584+0000 7f08758d3640 1 -- 172.21.15.33:0/1596325781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d7599b88f0 msgr2=0x55d7599baa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.584+0000 7f08758d3640 1 --2- 172.21.15.33:0/1596325781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d7599b88f0 0x55d7599baa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.584+0000 7f08758d3640 1 -- 172.21.15.33:0/1596325781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7599b4880 con 0x55d7599c3ad0 2024-05-15T14:03:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.584+0000 7f08760d4640 1 --2- 172.21.15.33:0/1596325781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d7599b88f0 0x55d7599baa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:03:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.584+0000 7f08758d3640 1 --2- 172.21.15.33:0/1596325781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7599c3ad0 0x55d7599c2f70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f086000bbd0 tx=0x7f0860011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3998b16bca5e08 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.584+0000 7f085bfff640 1 -- 172.21.15.33:0/1596325781 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086002a030 con 0x55d7599c3ad0 2024-05-15T14:03:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.584+0000 7f085bfff640 1 -- 172.21.15.33:0/1596325781 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08600035f0 con 0x55d7599c3ad0 2024-05-15T14:03:28.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.584+0000 7f085bfff640 1 -- 172.21.15.33:0/1596325781 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08600117b0 con 0x55d7599c3ad0 2024-05-15T14:03:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.585+0000 7f087b051740 1 -- 172.21.15.33:0/1596325781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7599c3ad0 msgr2=0x55d7599c2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.585+0000 7f087b051740 1 --2- 172.21.15.33:0/1596325781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7599c3ad0 0x55d7599c2f70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f086000bbd0 tx=0x7f0860011440 comp rx=0 tx=0).stop 2024-05-15T14:03:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.585+0000 7f087b051740 1 -- 172.21.15.33:0/1596325781 shutdown_connections 2024-05-15T14:03:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.585+0000 7f087b051740 1 --2- 172.21.15.33:0/1596325781 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d7599b03f0 0x55d7599db000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.585+0000 7f087b051740 1 --2- 172.21.15.33:0/1596325781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7599c3ad0 0x55d7599c2f70 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.585+0000 7f087b051740 1 --2- 172.21.15.33:0/1596325781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d7599b88f0 0x55d7599baa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.585+0000 7f087b051740 1 -- 172.21.15.33:0/1596325781 >> 172.21.15.33:0/1596325781 conn(0x55d7599b4e00 msgr2=0x55d7599ba2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.585+0000 7f087b051740 1 -- 172.21.15.33:0/1596325781 shutdown_connections 2024-05-15T14:03:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.585+0000 7f087b051740 1 -- 172.21.15.33:0/1596325781 wait complete. 2024-05-15T14:03:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.585+0000 7f087b051740 1 -- start start 2024-05-15T14:03:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.586+0000 7f087b051740 1 Processor -- start 2024-05-15T14:03:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.586+0000 7f087b051740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d7599b88f0 0x55d7599c7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.586+0000 7f087b051740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d7599c3ad0 0x55d7599c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.586+0000 7f087b051740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7599c80b0 0x55d759a40fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.586+0000 7f087b051740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f08701acd70 con 0x55d7599c80b0 2024-05-15T14:03:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.586+0000 7f087b051740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f08701ae1e0 con 0x55d7599c3ad0 2024-05-15T14:03:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.586+0000 7f087b051740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f08701ad440 con 0x55d7599b88f0 2024-05-15T14:03:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.586+0000 7f087b051740 10 client.0 _ll_register_callbacks cb 0x7f086000bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:03:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.586+0000 7f08768d5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7599c80b0 0x55d759a40fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.586+0000 7f08760d4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d7599b88f0 0x55d7599c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.586+0000 7f08758d3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d7599c3ad0 0x55d7599c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.586+0000 7f08768d5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7599c80b0 0x55d759a40fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57382/0 (socket says 172.21.15.33:57382) 2024-05-15T14:03:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.586+0000 7f08760d4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d7599b88f0 0x55d7599c7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:48514/0 (socket says 172.21.15.33:48514) 2024-05-15T14:03:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.586+0000 7f08768d5640 1 -- 172.21.15.33:0/1747895109 learned_addr learned my addr 172.21.15.33:0/1747895109 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.586+0000 7f08768d5640 1 -- 172.21.15.33:0/1747895109 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d7599b88f0 msgr2=0x55d7599c7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.586+0000 7f08768d5640 1 --2- 172.21.15.33:0/1747895109 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d7599b88f0 0x55d7599c7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.586+0000 7f08768d5640 1 -- 172.21.15.33:0/1747895109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d7599c3ad0 msgr2=0x55d7599c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.586+0000 7f08768d5640 1 --2- 172.21.15.33:0/1747895109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d7599c3ad0 0x55d7599c7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.586+0000 7f08768d5640 1 -- 172.21.15.33:0/1747895109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d759a41500 con 0x55d7599c80b0 2024-05-15T14:03:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.586+0000 7f08760d4640 1 --2- 172.21.15.33:0/1747895109 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d7599b88f0 0x55d7599c7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:03:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.587+0000 7f08768d5640 1 --2- 172.21.15.33:0/1747895109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7599c80b0 0x55d759a40fc0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f08680040f0 tx=0x7f0868004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4491d7e073dcd22 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.587+0000 7f0859ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:03:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.587+0000 7f0859ffb640 1 -- 172.21.15.33:0/1747895109 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086802e060 con 0x55d7599c80b0 2024-05-15T14:03:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.587+0000 7f087b051740 1 -- 172.21.15.33:0/1747895109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08600035f0 con 0x55d7599c80b0 2024-05-15T14:03:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.587+0000 7f0859ffb640 1 -- 172.21.15.33:0/1747895109 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f086800f890 con 0x55d7599c80b0 2024-05-15T14:03:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.587+0000 7f0859ffb640 1 -- 172.21.15.33:0/1747895109 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f086800fa90 con 0x55d7599c80b0 2024-05-15T14:03:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.587+0000 7f087b051740 10 client.15681 fetch_fsmap learned FSMap version 25 2024-05-15T14:03:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.587+0000 7f087b051740 1 -- 172.21.15.33:0/1747895109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d759a426f0 con 0x55d7599c80b0 2024-05-15T14:03:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.587+0000 7f0859ffb640 1 -- 172.21.15.33:0/1747895109 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08600035f0 con 0x55d7599c80b0 2024-05-15T14:03:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.587+0000 7f0859ffb640 1 -- 172.21.15.33:0/1747895109 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f086801d420 con 0x55d7599c80b0 2024-05-15T14:03:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.587+0000 7f087b051740 10 client.15681 fetch_fsmap finished waiting for FSMap version 25 2024-05-15T14:03:28.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.587+0000 7f087b051740 10 client.15681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:03:28.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.587+0000 7f087b051740 20 client.15681 populate_metadata read hostname 'smithi033' 2024-05-15T14:03:28.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.587+0000 7f087b051740 1 --2- 172.21.15.33:0/1747895109 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55d759a42f00 0x55d759a45320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:28.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.587+0000 7f087b051740 4 client.15681 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:03:28.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.587+0000 7f087b051740 1 -- 172.21.15.33:0/1747895109 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d759927360 con 0x55d759a42f00 2024-05-15T14:03:28.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.588+0000 7f08760d4640 1 --2- 172.21.15.33:0/1747895109 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55d759a42f00 0x55d759a45320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:28.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.588+0000 7f0859ffb640 10 client.15681 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:03:28.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.588+0000 7f08760d4640 1 --2- 172.21.15.33:0/1747895109 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55d759a42f00 0x55d759a45320 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a91ff572bdd94544 server_cookie=5e87bed9c9e82094 in_seq=0 out_seq=0 2024-05-15T14:03:28.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:28 smithi081 ceph-mon[36434]: pgmap v96: 97 pgs: 97 active+clean; 51 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.0 MiB/s wr, 370 op/s 2024-05-15T14:03:28.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.653+0000 7f0859ffb640 1 -- 172.21.15.33:0/1747895109 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d759927360 con 0x55d759a42f00 2024-05-15T14:03:28.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.653+0000 7f0859ffb640 10 client.15681 handle_command_reply: tid=0 2024-05-15T14:03:28.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.653+0000 7f087b051740 1 client.15681 shutdown 2024-05-15T14:03:28.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.653+0000 7f087b051740 1 -- 172.21.15.33:0/1747895109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7599c80b0 msgr2=0x55d759a40fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:28.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.653+0000 7f087b051740 1 --2- 172.21.15.33:0/1747895109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7599c80b0 0x55d759a40fc0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f08680040f0 tx=0x7f0868004a40 comp rx=0 tx=0).stop 2024-05-15T14:03:28.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.653+0000 7f087b051740 1 -- 172.21.15.33:0/1747895109 shutdown_connections 2024-05-15T14:03:28.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.653+0000 7f087b051740 1 --2- 172.21.15.33:0/1747895109 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55d759a42f00 0x55d759a45320 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:28.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.653+0000 7f087b051740 1 --2- 172.21.15.33:0/1747895109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7599c80b0 0x55d759a40fc0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:28.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.653+0000 7f087b051740 1 --2- 172.21.15.33:0/1747895109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d7599c3ad0 0x55d7599c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:28.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.653+0000 7f0859ffb640 0 client.15681 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:03:28.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.653+0000 7f087b051740 1 --2- 172.21.15.33:0/1747895109 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d7599b88f0 0x55d7599c7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:28.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.653+0000 7f087b051740 1 -- 172.21.15.33:0/1747895109 >> 172.21.15.33:0/1747895109 conn(0x55d7599b4e00 msgr2=0x55d7599b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:28.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.654+0000 7f087b051740 1 -- 172.21.15.33:0/1747895109 shutdown_connections 2024-05-15T14:03:28.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.654+0000 7f087b051740 1 -- 172.21.15.33:0/1747895109 wait complete. 2024-05-15T14:03:28.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.654+0000 7f087b051740 20 client.15681 trim_cache size 0 max 16384 2024-05-15T14:03:28.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:03:28.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:03:28.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:03:28.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.654+0000 7f0878338640 1 -- 172.21.15.33:0/2817652714 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f085c0b29d0 msgr2=0x7f085c0b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:28.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.654+0000 7f0878338640 1 --2- 172.21.15.33:0/2817652714 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f085c0b29d0 0x7f085c0b4e90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0868005fd0 tx=0x7f0868009500 comp rx=0 tx=0).stop 2024-05-15T14:03:28.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.654+0000 7f0878338640 1 -- 172.21.15.33:0/2817652714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0870102a40 msgr2=0x7f08701992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:28.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.654+0000 7f0878338640 1 --2- 172.21.15.33:0/2817652714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0870102a40 0x7f08701992b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f086c005940 tx=0x7f086c0042f0 comp rx=0 tx=0).stop 2024-05-15T14:03:28.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.654+0000 7f0878338640 1 -- 172.21.15.33:0/2817652714 shutdown_connections 2024-05-15T14:03:28.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.654+0000 7f0878338640 1 --2- 172.21.15.33:0/2817652714 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f085c0b29d0 0x7f085c0b4e90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:28.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.654+0000 7f0878338640 1 --2- 172.21.15.33:0/2817652714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0870108a50 0x7f087019db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:28.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.654+0000 7f0878338640 1 --2- 172.21.15.33:0/2817652714 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0870103400 0x7f08701997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:28.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.654+0000 7f0878338640 1 --2- 172.21.15.33:0/2817652714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0870102a40 0x7f08701992b0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:28.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.654+0000 7f0878338640 1 -- 172.21.15.33:0/2817652714 >> 172.21.15.33:0/2817652714 conn(0x7f08700fe510 msgr2=0x7f08700fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:28.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.654+0000 7f0878338640 1 -- 172.21.15.33:0/2817652714 shutdown_connections 2024-05-15T14:03:28.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:28.654+0000 7f0878338640 1 -- 172.21.15.33:0/2817652714 wait complete. 2024-05-15T14:03:28.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:28 smithi033 ceph-mon[31650]: pgmap v96: 97 pgs: 97 active+clean; 51 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.0 MiB/s wr, 370 op/s 2024-05-15T14:03:28.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:28 smithi052 ceph-mon[36190]: pgmap v96: 97 pgs: 97 active+clean; 51 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.0 MiB/s wr, 370 op/s 2024-05-15T14:03:29.049 INFO:tasks.workunit.client.0.smithi033.stdout: 1 137693 11.86 MB/sec warmup 64 sec latency 82.144 ms 2024-05-15T14:03:30.049 INFO:tasks.workunit.client.0.smithi033.stdout: 1 139280 11.81 MB/sec warmup 65 sec latency 13.162 ms 2024-05-15T14:03:30.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:30 smithi033 ceph-mon[31650]: pgmap v97: 97 pgs: 97 active+clean; 51 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.3 MiB/s wr, 219 op/s 2024-05-15T14:03:30.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:30 smithi052 ceph-mon[36190]: pgmap v97: 97 pgs: 97 active+clean; 51 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.3 MiB/s wr, 219 op/s 2024-05-15T14:03:30.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:30 smithi081 ceph-mon[36434]: pgmap v97: 97 pgs: 97 active+clean; 51 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.3 MiB/s wr, 219 op/s 2024-05-15T14:03:31.050 INFO:tasks.workunit.client.0.smithi033.stdout: 1 142044 11.89 MB/sec warmup 66 sec latency 33.417 ms 2024-05-15T14:03:32.050 INFO:tasks.workunit.client.0.smithi033.stdout: 1 143597 11.81 MB/sec warmup 67 sec latency 20.136 ms 2024-05-15T14:03:32.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:32 smithi033 ceph-mon[31650]: pgmap v98: 97 pgs: 97 active+clean; 51 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.3 MiB/s wr, 219 op/s 2024-05-15T14:03:32.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:32 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:03:32.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:32 smithi052 ceph-mon[36190]: pgmap v98: 97 pgs: 97 active+clean; 51 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.3 MiB/s wr, 219 op/s 2024-05-15T14:03:32.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:32 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:03:32.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:32 smithi081 ceph-mon[36434]: pgmap v98: 97 pgs: 97 active+clean; 51 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.3 MiB/s wr, 219 op/s 2024-05-15T14:03:32.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:32 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:03:33.049 INFO:tasks.workunit.client.0.smithi033.stdout: 1 145920 11.84 MB/sec warmup 68 sec latency 12.598 ms 2024-05-15T14:03:33.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:33 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:03:33.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:33 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:03:33.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:33 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:03:33.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:33 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:03:33.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:33 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:03:33.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:33 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:03:33.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:33 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:03:33.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:33 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:03:33.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:33 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:03:34.049 INFO:tasks.workunit.client.0.smithi033.stdout: 1 148481 11.85 MB/sec warmup 69 sec latency 14.939 ms 2024-05-15T14:03:34.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:34 smithi033 ceph-mon[31650]: pgmap v99: 97 pgs: 97 active+clean; 49 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.2 MiB/s wr, 386 op/s 2024-05-15T14:03:34.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:34 smithi052 ceph-mon[36190]: pgmap v99: 97 pgs: 97 active+clean; 49 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.2 MiB/s wr, 386 op/s 2024-05-15T14:03:34.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:34 smithi081 ceph-mon[36434]: pgmap v99: 97 pgs: 97 active+clean; 49 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.2 MiB/s wr, 386 op/s 2024-05-15T14:03:35.049 INFO:tasks.workunit.client.0.smithi033.stdout: 1 150218 11.80 MB/sec warmup 70 sec latency 17.009 ms 2024-05-15T14:03:36.050 INFO:tasks.workunit.client.0.smithi033.stdout: 1 152792 11.87 MB/sec warmup 71 sec latency 36.788 ms 2024-05-15T14:03:36.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:36 smithi033 ceph-mon[31650]: pgmap v100: 97 pgs: 97 active+clean; 49 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 276 op/s 2024-05-15T14:03:36.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:36 smithi052 ceph-mon[36190]: pgmap v100: 97 pgs: 97 active+clean; 49 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 276 op/s 2024-05-15T14:03:36.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:36 smithi081 ceph-mon[36434]: pgmap v100: 97 pgs: 97 active+clean; 49 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 276 op/s 2024-05-15T14:03:37.050 INFO:tasks.workunit.client.0.smithi033.stdout: 1 154800 11.79 MB/sec warmup 72 sec latency 42.167 ms 2024-05-15T14:03:37.592 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-05-15T14:03:37.593 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T14:03:37.857 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:03:38.050 INFO:tasks.workunit.client.0.smithi033.stdout: 1 156669 11.81 MB/sec warmup 73 sec latency 15.764 ms 2024-05-15T14:03:38.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:38 smithi033 ceph-mon[31650]: pgmap v101: 97 pgs: 97 active+clean; 54 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.9 MiB/s wr, 407 op/s 2024-05-15T14:03:38.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:38 smithi052 ceph-mon[36190]: pgmap v101: 97 pgs: 97 active+clean; 54 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.9 MiB/s wr, 407 op/s 2024-05-15T14:03:38.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:38 smithi081 ceph-mon[36434]: pgmap v101: 97 pgs: 97 active+clean; 54 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.9 MiB/s wr, 407 op/s 2024-05-15T14:03:39.050 INFO:tasks.workunit.client.0.smithi033.stdout: 1 159468 11.85 MB/sec warmup 74 sec latency 16.914 ms 2024-05-15T14:03:39.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.728+0000 7f4c154e3700 1 -- 172.21.15.33:0/4263904665 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c101041a0 msgr2=0x7f4c101081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:39.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.728+0000 7f4c154e3700 1 --2- 172.21.15.33:0/4263904665 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c101041a0 0x7f4c101081c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4c040098f0 tx=0x7f4c04009c00 comp rx=0 tx=0).stop 2024-05-15T14:03:39.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.728+0000 7f4c154e3700 1 -- 172.21.15.33:0/4263904665 shutdown_connections 2024-05-15T14:03:39.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.728+0000 7f4c154e3700 1 --2- 172.21.15.33:0/4263904665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c101088f0 0x7f4c10110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:39.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.728+0000 7f4c154e3700 1 --2- 172.21.15.33:0/4263904665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c101037f0 0x7f4c10103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:39.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.728+0000 7f4c154e3700 1 --2- 172.21.15.33:0/4263904665 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c101041a0 0x7f4c101081c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:39.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.728+0000 7f4c154e3700 1 -- 172.21.15.33:0/4263904665 >> 172.21.15.33:0/4263904665 conn(0x7f4c1007a6a0 msgr2=0x7f4c1007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:39.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.728+0000 7f4c154e3700 1 -- 172.21.15.33:0/4263904665 shutdown_connections 2024-05-15T14:03:39.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.728+0000 7f4c154e3700 1 -- 172.21.15.33:0/4263904665 wait complete. 2024-05-15T14:03:39.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.729+0000 7f4c154e3700 1 Processor -- start 2024-05-15T14:03:39.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.729+0000 7f4c154e3700 1 -- start start 2024-05-15T14:03:39.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.729+0000 7f4c154e3700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c101037f0 0x7f4c1019bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:39.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.729+0000 7f4c154e3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c101041a0 0x7f4c1019c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:39.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.729+0000 7f4c154e3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c101088f0 0x7f4c101a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:39.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.729+0000 7f4c154e3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4c101136d0 con 0x7f4c101088f0 2024-05-15T14:03:39.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.729+0000 7f4c154e3700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4c10113550 con 0x7f4c101041a0 2024-05-15T14:03:39.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.729+0000 7f4c154e3700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4c10113850 con 0x7f4c101037f0 2024-05-15T14:03:39.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.730+0000 7f4c0f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c101088f0 0x7f4c101a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:39.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.730+0000 7f4c0f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c101088f0 0x7f4c101a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45688/0 (socket says 172.21.15.33:45688) 2024-05-15T14:03:39.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.730+0000 7f4c0effd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c101037f0 0x7f4c1019bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:39.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.730+0000 7f4c0f7fe700 1 -- 172.21.15.33:0/2363244038 learned_addr learned my addr 172.21.15.33:0/2363244038 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:39.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.730+0000 7f4c0e7fc700 1 --2- 172.21.15.33:0/2363244038 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c101041a0 0x7f4c1019c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:39.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.730+0000 7f4c0f7fe700 1 -- 172.21.15.33:0/2363244038 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c101037f0 msgr2=0x7f4c1019bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:39.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.730+0000 7f4c0f7fe700 1 --2- 172.21.15.33:0/2363244038 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c101037f0 0x7f4c1019bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:39.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.730+0000 7f4c0f7fe700 1 -- 172.21.15.33:0/2363244038 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c101041a0 msgr2=0x7f4c1019c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:39.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.730+0000 7f4c0f7fe700 1 --2- 172.21.15.33:0/2363244038 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c101041a0 0x7f4c1019c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:39.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.730+0000 7f4c0f7fe700 1 -- 172.21.15.33:0/2363244038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c101a0d90 con 0x7f4c101088f0 2024-05-15T14:03:39.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.730+0000 7f4c0e7fc700 1 --2- 172.21.15.33:0/2363244038 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c101041a0 0x7f4c1019c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:03:39.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.730+0000 7f4c0effd700 1 --2- 172.21.15.33:0/2363244038 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c101037f0 0x7f4c1019bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:03:39.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.730+0000 7f4c0f7fe700 1 --2- 172.21.15.33:0/2363244038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c101088f0 0x7f4c101a0650 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800e3c0 tx=0x7f4c0800e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:39.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.730+0000 7f4bf7fff700 1 -- 172.21.15.33:0/2363244038 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c080092d0 con 0x7f4c101088f0 2024-05-15T14:03:39.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.730+0000 7f4bf7fff700 1 -- 172.21.15.33:0/2363244038 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c0800ee70 con 0x7f4c101088f0 2024-05-15T14:03:39.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.730+0000 7f4c154e3700 1 -- 172.21.15.33:0/2363244038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c101a1080 con 0x7f4c101088f0 2024-05-15T14:03:39.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.730+0000 7f4bf7fff700 1 -- 172.21.15.33:0/2363244038 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c08004030 con 0x7f4c101088f0 2024-05-15T14:03:39.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.731+0000 7f4c154e3700 1 -- 172.21.15.33:0/2363244038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c101a8900 con 0x7f4c101088f0 2024-05-15T14:03:39.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.731+0000 7f4c154e3700 1 -- 172.21.15.33:0/2363244038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c1006a230 con 0x7f4c101088f0 2024-05-15T14:03:39.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.733+0000 7f4bf7fff700 1 -- 172.21.15.33:0/2363244038 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4c08012050 con 0x7f4c101088f0 2024-05-15T14:03:39.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.734+0000 7f4bf7fff700 1 --2- 172.21.15.33:0/2363244038 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f4bf80b29f0 0x7f4bf80b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:39.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.734+0000 7f4bf7fff700 1 -- 172.21.15.33:0/2363244038 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f4c080e2600 con 0x7f4c101088f0 2024-05-15T14:03:39.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.734+0000 7f4c0effd700 1 --2- 172.21.15.33:0/2363244038 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f4bf80b29f0 0x7f4bf80b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:39.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.734+0000 7f4c0effd700 1 --2- 172.21.15.33:0/2363244038 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f4bf80b29f0 0x7f4bf80b4eb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4c00011490 tx=0x7f4c00009260 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:39.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:39.737+0000 7f4bf7fff700 1 -- 172.21.15.33:0/2363244038 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4c08022020 con 0x7f4c101088f0 2024-05-15T14:03:40.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:40.025+0000 7f4c154e3700 1 -- 172.21.15.33:0/2363244038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c101a8b00 con 0x7f4c101088f0 2024-05-15T14:03:40.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:40.025+0000 7f4bf7fff700 1 -- 172.21.15.33:0/2363244038 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11861 (secure 0 0 0) 0x7f4c08010040 con 0x7f4c101088f0 2024-05-15T14:03:40.028 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T14:03:40.028 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2030117633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2030117633},{"type":"v1","addr":"172.21.15.33:6839","nonce":2030117633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1918979061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1918979061},{"type":"v1","addr":"172.21.15.33:6841","nonce":1918979061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T13:57:19.056417+0000","modified":"2024-05-15T14:02:48.675249+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24463,"mds_2":24461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/1629697841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1629697841},{"type":"v1","addr":"172.21.15.52:6837","nonce":1629697841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/3773839758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3773839758},{"type":"v1","addr":"172.21.15.81:6835","nonce":3773839758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.52:6839/2696044471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2696044471},{"type":"v1","addr":"172.21.15.52:6839","nonce":2696044471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.81:6839/1554395940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":1554395940},{"type":"v1","addr":"172.21.15.81:6839","nonce":1554395940}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24461,24463,24478]},"id":1}]} 2024-05-15T14:03:40.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:40.028+0000 7f4c154e3700 1 -- 172.21.15.33:0/2363244038 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f4bf80b29f0 msgr2=0x7f4bf80b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:40.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:40.028+0000 7f4c154e3700 1 --2- 172.21.15.33:0/2363244038 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f4bf80b29f0 0x7f4bf80b4eb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4c00011490 tx=0x7f4c00009260 comp rx=0 tx=0).stop 2024-05-15T14:03:40.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:40.028+0000 7f4c154e3700 1 -- 172.21.15.33:0/2363244038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c101088f0 msgr2=0x7f4c101a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:40.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:40.028+0000 7f4c154e3700 1 --2- 172.21.15.33:0/2363244038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c101088f0 0x7f4c101a0650 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800e3c0 tx=0x7f4c0800e6d0 comp rx=0 tx=0).stop 2024-05-15T14:03:40.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:40.028+0000 7f4c154e3700 1 -- 172.21.15.33:0/2363244038 shutdown_connections 2024-05-15T14:03:40.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:40.028+0000 7f4c154e3700 1 --2- 172.21.15.33:0/2363244038 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f4bf80b29f0 0x7f4bf80b4eb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:40.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:40.028+0000 7f4c154e3700 1 --2- 172.21.15.33:0/2363244038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c101088f0 0x7f4c101a0650 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:40.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:40.028+0000 7f4c154e3700 1 --2- 172.21.15.33:0/2363244038 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c101037f0 0x7f4c1019bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:40.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:40.028+0000 7f4c154e3700 1 --2- 172.21.15.33:0/2363244038 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c101041a0 0x7f4c1019c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:40.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:40.028+0000 7f4c154e3700 1 -- 172.21.15.33:0/2363244038 >> 172.21.15.33:0/2363244038 conn(0x7f4c1007a6a0 msgr2=0x7f4c10101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:40.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:40.028+0000 7f4c154e3700 1 -- 172.21.15.33:0/2363244038 shutdown_connections 2024-05-15T14:03:40.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:40.028+0000 7f4c154e3700 1 -- 172.21.15.33:0/2363244038 wait complete. 2024-05-15T14:03:40.032 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 25 2024-05-15T14:03:40.050 INFO:tasks.workunit.client.0.smithi033.stdout: 1 160823 11.78 MB/sec warmup 75 sec latency 37.670 ms 2024-05-15T14:03:40.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:40 smithi033 ceph-mon[31650]: pgmap v102: 97 pgs: 97 active+clean; 54 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 298 op/s 2024-05-15T14:03:40.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:40 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/2363244038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:03:40.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:40 smithi052 ceph-mon[36190]: pgmap v102: 97 pgs: 97 active+clean; 54 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 298 op/s 2024-05-15T14:03:40.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:40 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/2363244038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:03:40.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:40 smithi081 ceph-mon[36434]: pgmap v102: 97 pgs: 97 active+clean; 54 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 298 op/s 2024-05-15T14:03:40.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:40 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/2363244038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:03:40.908 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24478 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-05-15T14:03:40.908 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24478 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-05-15T14:03:41.050 INFO:tasks.workunit.client.0.smithi033.stdout: 1 163439 11.85 MB/sec warmup 76 sec latency 42.051 ms 2024-05-15T14:03:41.175 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:03:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.749+0000 7f40d2bf7640 1 -- 172.21.15.33:0/3680230550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40cc107960 msgr2=0x7f40cc107d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.749+0000 7f40d2bf7640 1 --2- 172.21.15.33:0/3680230550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40cc107960 0x7f40cc107d40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f40c0009930 tx=0x7f40c002a420 comp rx=0 tx=0).stop 2024-05-15T14:03:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.749+0000 7f40d2bf7640 1 -- 172.21.15.33:0/3680230550 shutdown_connections 2024-05-15T14:03:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.749+0000 7f40d2bf7640 1 --2- 172.21.15.33:0/3680230550 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40cc102310 0x7f40cc10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.749+0000 7f40d2bf7640 1 --2- 172.21.15.33:0/3680230550 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40cc101950 0x7f40cc101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.749+0000 7f40d2bf7640 1 --2- 172.21.15.33:0/3680230550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40cc107960 0x7f40cc107d40 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.749+0000 7f40d2bf7640 1 -- 172.21.15.33:0/3680230550 >> 172.21.15.33:0/3680230550 conn(0x7f40cc078a50 msgr2=0x7f40cc078e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.750+0000 7f40d2bf7640 1 -- 172.21.15.33:0/3680230550 shutdown_connections 2024-05-15T14:03:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.750+0000 7f40d2bf7640 1 -- 172.21.15.33:0/3680230550 wait complete. 2024-05-15T14:03:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.750+0000 7f40d2bf7640 1 Processor -- start 2024-05-15T14:03:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.750+0000 7f40d2bf7640 1 -- start start 2024-05-15T14:03:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.750+0000 7f40d2bf7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40cc101950 0x7f40cc1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.750+0000 7f40d2bf7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40cc102310 0x7f40cc1997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.750+0000 7f40d2bf7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40cc107960 0x7f40cc19db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.750+0000 7f40d2bf7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc111c20 con 0x7f40cc102310 2024-05-15T14:03:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.750+0000 7f40d2bf7640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc111aa0 con 0x7f40cc101950 2024-05-15T14:03:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.750+0000 7f40d2bf7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc111da0 con 0x7f40cc107960 2024-05-15T14:03:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.751+0000 7f40cbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40cc102310 0x7f40cc1997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.751+0000 7f40d0993640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40cc101950 0x7f40cc1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.751+0000 7f40d1194640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40cc107960 0x7f40cc19db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.751+0000 7f40d0993640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40cc101950 0x7f40cc1992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:57944/0 (socket says 172.21.15.33:57944) 2024-05-15T14:03:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.751+0000 7f40cbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40cc102310 0x7f40cc1997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45692/0 (socket says 172.21.15.33:45692) 2024-05-15T14:03:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.751+0000 7f40cbfff640 1 -- 172.21.15.33:0/4093704769 learned_addr learned my addr 172.21.15.33:0/4093704769 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.751+0000 7f40d1194640 1 -- 172.21.15.33:0/4093704769 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40cc101950 msgr2=0x7f40cc1992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.751+0000 7f40d1194640 1 --2- 172.21.15.33:0/4093704769 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40cc101950 0x7f40cc1992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.751+0000 7f40d1194640 1 -- 172.21.15.33:0/4093704769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40cc102310 msgr2=0x7f40cc1997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.751+0000 7f40d1194640 1 --2- 172.21.15.33:0/4093704769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40cc102310 0x7f40cc1997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.751+0000 7f40d1194640 1 -- 172.21.15.33:0/4093704769 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40cc19e2a0 con 0x7f40cc107960 2024-05-15T14:03:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.751+0000 7f40cbfff640 1 --2- 172.21.15.33:0/4093704769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40cc102310 0x7f40cc1997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:03:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.751+0000 7f40d1194640 1 --2- 172.21.15.33:0/4093704769 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40cc107960 0x7f40cc19db70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f40c4009fd0 tx=0x7f40c400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.751+0000 7f40c9ffb640 1 -- 172.21.15.33:0/4093704769 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c400ce20 con 0x7f40cc107960 2024-05-15T14:03:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.751+0000 7f40c9ffb640 1 -- 172.21.15.33:0/4093704769 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40c4005020 con 0x7f40cc107960 2024-05-15T14:03:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.751+0000 7f40d2bf7640 1 -- 172.21.15.33:0/4093704769 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40cc19e590 con 0x7f40cc107960 2024-05-15T14:03:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.751+0000 7f40c9ffb640 1 -- 172.21.15.33:0/4093704769 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c4010690 con 0x7f40cc107960 2024-05-15T14:03:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.752+0000 7f40d2bf7640 1 -- 172.21.15.33:0/4093704769 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40cc1a5e20 con 0x7f40cc107960 2024-05-15T14:03:41.768 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.755+0000 7f40c9ffb640 1 -- 172.21.15.33:0/4093704769 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f40c40054d0 con 0x7f40cc107960 2024-05-15T14:03:41.768 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.755+0000 7f40d59ee740 1 Processor -- start 2024-05-15T14:03:41.768 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.755+0000 7f40d59ee740 1 -- start start 2024-05-15T14:03:41.768 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.755+0000 7f40d59ee740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2491da7f0 0x55e2491dabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:41.768 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.755+0000 7f40d59ee740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2491db110 0x55e2491e99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:41.768 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.755+0000 7f40d59ee740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2491e9ee0 0x55e2491ec2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:41.768 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.755+0000 7f40d59ee740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e24914dc30 con 0x55e2491e9ee0 2024-05-15T14:03:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.755+0000 7f40d59ee740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e24913d7d0 con 0x55e2491da7f0 2024-05-15T14:03:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.755+0000 7f40d59ee740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55e249141810 con 0x55e2491db110 2024-05-15T14:03:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.755+0000 7f40d0993640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2491db110 0x55e2491e99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.755+0000 7f40cbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2491e9ee0 0x55e2491ec2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.755+0000 7f40d0993640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2491db110 0x55e2491e99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:50438/0 (socket says 172.21.15.33:50438) 2024-05-15T14:03:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.755+0000 7f40d0993640 1 -- 172.21.15.33:0/3159223265 learned_addr learned my addr 172.21.15.33:0/3159223265 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.756+0000 7f40cbfff640 1 --2- 172.21.15.33:0/3159223265 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2491da7f0 0x55e2491dabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.756+0000 7f40d0993640 1 -- 172.21.15.33:0/3159223265 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2491da7f0 msgr2=0x55e2491dabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:41.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.756+0000 7f40d0993640 1 --2- 172.21.15.33:0/3159223265 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2491da7f0 0x55e2491dabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:41.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.756+0000 7f40d0993640 1 -- 172.21.15.33:0/3159223265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2491e9ee0 msgr2=0x55e2491ec2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:41.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.756+0000 7f40d0993640 1 --2- 172.21.15.33:0/3159223265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2491e9ee0 0x55e2491ec2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:41.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.756+0000 7f40c9ffb640 1 --2- 172.21.15.33:0/4093704769 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f40bc0b2950 0x7f40bc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:41.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.756+0000 7f40d0993640 1 -- 172.21.15.33:0/3159223265 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e248e52210 con 0x55e2491db110 2024-05-15T14:03:41.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.756+0000 7f40cbfff640 1 --2- 172.21.15.33:0/4093704769 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f40bc0b2950 0x7f40bc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:41.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.756+0000 7f40c9ffb640 1 -- 172.21.15.33:0/4093704769 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f40c40e2c70 con 0x7f40cc107960 2024-05-15T14:03:41.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.756+0000 7f40d0993640 1 --2- 172.21.15.33:0/3159223265 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2491db110 0x55e2491e99a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f40c002a940 tx=0x7f40c0005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4cc4b988dc04e59 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:41.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.756+0000 7f40b6ffd640 1 -- 172.21.15.33:0/3159223265 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c0040070 con 0x55e2491db110 2024-05-15T14:03:41.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.756+0000 7f40b6ffd640 1 -- 172.21.15.33:0/3159223265 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40c0004030 con 0x55e2491db110 2024-05-15T14:03:41.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.756+0000 7f40b6ffd640 1 -- 172.21.15.33:0/3159223265 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c0035040 con 0x55e2491db110 2024-05-15T14:03:41.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.757+0000 7f40d59ee740 1 -- 172.21.15.33:0/3159223265 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2491db110 msgr2=0x55e2491e99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:41.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.757+0000 7f40cbfff640 1 --2- 172.21.15.33:0/4093704769 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f40bc0b2950 0x7f40bc0b4e10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55e2491dc3b0 tx=0x7f40b8009c80 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:41.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.757+0000 7f40d59ee740 1 --2- 172.21.15.33:0/3159223265 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2491db110 0x55e2491e99a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f40c002a940 tx=0x7f40c0005be0 comp rx=0 tx=0).stop 2024-05-15T14:03:41.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.757+0000 7f40d59ee740 1 -- 172.21.15.33:0/3159223265 shutdown_connections 2024-05-15T14:03:41.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.757+0000 7f40d59ee740 1 --2- 172.21.15.33:0/3159223265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2491e9ee0 0x55e2491ec2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:41.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.757+0000 7f40d59ee740 1 --2- 172.21.15.33:0/3159223265 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2491db110 0x55e2491e99a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:41.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.757+0000 7f40d59ee740 1 --2- 172.21.15.33:0/3159223265 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2491da7f0 0x55e2491dabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.757+0000 7f40d59ee740 1 -- 172.21.15.33:0/3159223265 >> 172.21.15.33:0/3159223265 conn(0x55e2491d6e00 msgr2=0x55e2491d71e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.757+0000 7f40d59ee740 1 -- 172.21.15.33:0/3159223265 shutdown_connections 2024-05-15T14:03:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.757+0000 7f40d59ee740 1 -- 172.21.15.33:0/3159223265 wait complete. 2024-05-15T14:03:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.757+0000 7f40d59ee740 1 -- start start 2024-05-15T14:03:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.758+0000 7f40d59ee740 1 Processor -- start 2024-05-15T14:03:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.758+0000 7f40d59ee740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2491da7f0 0x55e2491e97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.758+0000 7f40d59ee740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2491db110 0x55e2491e6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.758+0000 7f40d59ee740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2491e9ee0 0x55e2491e6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.758+0000 7f40d59ee740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e248e23790 con 0x55e2491da7f0 2024-05-15T14:03:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.758+0000 7f40d59ee740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e2490f9890 con 0x55e2491e9ee0 2024-05-15T14:03:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.758+0000 7f40d59ee740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55e2491ecb50 con 0x55e2491db110 2024-05-15T14:03:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.758+0000 7f40d59ee740 10 client.0 _ll_register_callbacks cb 0x55e2490fcd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:03:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.758+0000 7f40cbfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2491db110 0x55e2491e6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.758+0000 7f40d0993640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2491da7f0 0x55e2491e97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.758+0000 7f40d0993640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2491da7f0 0x55e2491e97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45716/0 (socket says 172.21.15.33:45716) 2024-05-15T14:03:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.758+0000 7f40d1194640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2491e9ee0 0x55e2491e6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.758+0000 7f40d0993640 1 -- 172.21.15.33:0/1031472086 learned_addr learned my addr 172.21.15.33:0/1031472086 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.758+0000 7f40d0993640 1 -- 172.21.15.33:0/1031472086 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2491db110 msgr2=0x55e2491e6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.758+0000 7f40d0993640 1 --2- 172.21.15.33:0/1031472086 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2491db110 0x55e2491e6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.758+0000 7f40d0993640 1 -- 172.21.15.33:0/1031472086 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2491e9ee0 msgr2=0x55e2491e6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.758+0000 7f40d0993640 1 --2- 172.21.15.33:0/1031472086 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2491e9ee0 0x55e2491e6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.758+0000 7f40d0993640 1 -- 172.21.15.33:0/1031472086 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e248f1e620 con 0x55e2491da7f0 2024-05-15T14:03:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.758+0000 7f40d1194640 1 --2- 172.21.15.33:0/1031472086 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2491e9ee0 0x55e2491e6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:03:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.758+0000 7f40cbfff640 1 --2- 172.21.15.33:0/1031472086 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2491db110 0x55e2491e6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:03:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.759+0000 7f40d0993640 1 --2- 172.21.15.33:0/1031472086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2491da7f0 0x55e2491e97b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f40c0000c00 tx=0x7f40c002ab70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a64847dd8dadd648 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.759+0000 7f40abfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:03:41.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.759+0000 7f40abfff640 1 -- 172.21.15.33:0/1031472086 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c0040070 con 0x55e2491da7f0 2024-05-15T14:03:41.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.759+0000 7f40abfff640 1 -- 172.21.15.33:0/1031472086 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f40c00043f0 con 0x55e2491da7f0 2024-05-15T14:03:41.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.759+0000 7f40d59ee740 1 -- 172.21.15.33:0/1031472086 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40c0004030 con 0x55e2491da7f0 2024-05-15T14:03:41.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.759+0000 7f40abfff640 1 -- 172.21.15.33:0/1031472086 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f40c0035070 con 0x55e2491da7f0 2024-05-15T14:03:41.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.759+0000 7f40d59ee740 10 client.15699 fetch_fsmap learned FSMap version 25 2024-05-15T14:03:41.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.759+0000 7f40d59ee740 1 -- 172.21.15.33:0/1031472086 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e2491f8570 con 0x55e2491da7f0 2024-05-15T14:03:41.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.759+0000 7f40abfff640 1 -- 172.21.15.33:0/1031472086 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40c0004030 con 0x55e2491da7f0 2024-05-15T14:03:41.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.759+0000 7f40abfff640 1 -- 172.21.15.33:0/1031472086 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f40c0004930 con 0x55e2491da7f0 2024-05-15T14:03:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.760+0000 7f40d59ee740 10 client.15699 fetch_fsmap finished waiting for FSMap version 25 2024-05-15T14:03:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.760+0000 7f40d59ee740 10 client.15699 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:03:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.760+0000 7f40d59ee740 20 client.15699 populate_metadata read hostname 'smithi033' 2024-05-15T14:03:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.760+0000 7f40d59ee740 1 --2- 172.21.15.33:0/1031472086 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e2491f98c0 0x55e2491fbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.760+0000 7f40d59ee740 4 client.15699 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:03:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.760+0000 7f40d59ee740 1 -- 172.21.15.33:0/1031472086 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e24914fb80 con 0x55e2491f98c0 2024-05-15T14:03:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.760+0000 7f40cbfff640 1 --2- 172.21.15.33:0/1031472086 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e2491f98c0 0x55e2491fbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.760+0000 7f40abfff640 10 client.15699 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:03:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.760+0000 7f40cbfff640 1 --2- 172.21.15.33:0/1031472086 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e2491f98c0 0x55e2491fbcb0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d74cd7500e679a75 server_cookie=4bfe0d58784579a1 in_seq=0 out_seq=0 2024-05-15T14:03:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.761+0000 7f40abfff640 1 -- 172.21.15.33:0/1031472086 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e24914fb80 con 0x55e2491f98c0 2024-05-15T14:03:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.761+0000 7f40abfff640 10 client.15699 handle_command_reply: tid=0 2024-05-15T14:03:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.761+0000 7f40d59ee740 1 client.15699 shutdown 2024-05-15T14:03:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.761+0000 7f40d59ee740 1 -- 172.21.15.33:0/1031472086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2491da7f0 msgr2=0x55e2491e97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.761+0000 7f40d59ee740 1 --2- 172.21.15.33:0/1031472086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2491da7f0 0x55e2491e97b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f40c0000c00 tx=0x7f40c002ab70 comp rx=0 tx=0).stop 2024-05-15T14:03:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.761+0000 7f40d59ee740 1 -- 172.21.15.33:0/1031472086 shutdown_connections 2024-05-15T14:03:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.761+0000 7f40d59ee740 1 --2- 172.21.15.33:0/1031472086 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e2491f98c0 0x55e2491fbcb0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.761+0000 7f40d59ee740 1 --2- 172.21.15.33:0/1031472086 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2491e9ee0 0x55e2491e6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.761+0000 7f40d59ee740 1 --2- 172.21.15.33:0/1031472086 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2491db110 0x55e2491e6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.761+0000 7f40d59ee740 1 --2- 172.21.15.33:0/1031472086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2491da7f0 0x55e2491e97b0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.761+0000 7f40d59ee740 1 -- 172.21.15.33:0/1031472086 >> 172.21.15.33:0/1031472086 conn(0x55e2491d6e00 msgr2=0x55e2491dcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.761+0000 7f40abfff640 0 client.15699 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:03:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.761+0000 7f40d59ee740 1 -- 172.21.15.33:0/1031472086 shutdown_connections 2024-05-15T14:03:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.761+0000 7f40d59ee740 1 -- 172.21.15.33:0/1031472086 wait complete. 2024-05-15T14:03:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.761+0000 7f40d59ee740 20 client.15699 trim_cache size 0 max 16384 2024-05-15T14:03:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.778+0000 7f40d59ee740 1 Processor -- start 2024-05-15T14:03:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.778+0000 7f40d59ee740 1 -- start start 2024-05-15T14:03:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.778+0000 7f40d59ee740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2491d6e00 0x55e2491e21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.778+0000 7f40d59ee740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2491da7f0 0x55e2491e5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.778+0000 7f40d59ee740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2491db110 0x55e2491dcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.778+0000 7f40d59ee740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc1aecf0 con 0x55e2491db110 2024-05-15T14:03:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.778+0000 7f40d59ee740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc1ac5c0 con 0x55e2491d6e00 2024-05-15T14:03:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.778+0000 7f40d59ee740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc1aa410 con 0x55e2491da7f0 2024-05-15T14:03:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.778+0000 7f40d1194640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2491db110 0x55e2491dcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.778+0000 7f40cbfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2491da7f0 0x55e2491e5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.778+0000 7f40d1194640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2491db110 0x55e2491dcaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45726/0 (socket says 172.21.15.33:45726) 2024-05-15T14:03:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.778+0000 7f40d1194640 1 -- 172.21.15.33:0/737834775 learned_addr learned my addr 172.21.15.33:0/737834775 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.778+0000 7f40d0993640 1 --2- 172.21.15.33:0/737834775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2491d6e00 0x55e2491e21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.778+0000 7f40d1194640 1 -- 172.21.15.33:0/737834775 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2491da7f0 msgr2=0x55e2491e5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.778+0000 7f40d1194640 1 --2- 172.21.15.33:0/737834775 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2491da7f0 0x55e2491e5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.778+0000 7f40d1194640 1 -- 172.21.15.33:0/737834775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2491d6e00 msgr2=0x55e2491e21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.778+0000 7f40d1194640 1 --2- 172.21.15.33:0/737834775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2491d6e00 0x55e2491e21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.778+0000 7f40d1194640 1 -- 172.21.15.33:0/737834775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2491e5a50 con 0x55e2491db110 2024-05-15T14:03:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.779+0000 7f40d1194640 1 --2- 172.21.15.33:0/737834775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2491db110 0x55e2491dcaf0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f40c40a9c60 tx=0x7f40c40aa720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82189e2d341110ff server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.779+0000 7f40b67fc640 1 -- 172.21.15.33:0/737834775 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c40b2dc0 con 0x55e2491db110 2024-05-15T14:03:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.779+0000 7f40b67fc640 1 -- 172.21.15.33:0/737834775 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40c40a61f0 con 0x55e2491db110 2024-05-15T14:03:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.779+0000 7f40b67fc640 1 -- 172.21.15.33:0/737834775 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c40a58d0 con 0x55e2491db110 2024-05-15T14:03:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.779+0000 7f40d59ee740 1 -- 172.21.15.33:0/737834775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2491db110 msgr2=0x55e2491dcaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.779+0000 7f40d59ee740 1 --2- 172.21.15.33:0/737834775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2491db110 0x55e2491dcaf0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f40c40a9c60 tx=0x7f40c40aa720 comp rx=0 tx=0).stop 2024-05-15T14:03:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.779+0000 7f40d59ee740 1 -- 172.21.15.33:0/737834775 shutdown_connections 2024-05-15T14:03:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.779+0000 7f40d59ee740 1 --2- 172.21.15.33:0/737834775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2491db110 0x55e2491dcaf0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.779+0000 7f40d59ee740 1 --2- 172.21.15.33:0/737834775 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2491da7f0 0x55e2491e5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.779+0000 7f40d59ee740 1 --2- 172.21.15.33:0/737834775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2491d6e00 0x55e2491e21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.779+0000 7f40d59ee740 1 -- 172.21.15.33:0/737834775 >> 172.21.15.33:0/737834775 conn(0x55e2491e7140 msgr2=0x55e2491dc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.780+0000 7f40d59ee740 1 -- 172.21.15.33:0/737834775 shutdown_connections 2024-05-15T14:03:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.780+0000 7f40d59ee740 1 -- 172.21.15.33:0/737834775 wait complete. 2024-05-15T14:03:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.780+0000 7f40d59ee740 1 -- start start 2024-05-15T14:03:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.780+0000 7f40d59ee740 1 Processor -- start 2024-05-15T14:03:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.780+0000 7f40d59ee740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2491d6e00 0x55e2491f1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.780+0000 7f40d59ee740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2491da7f0 0x55e2491f2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.780+0000 7f40d59ee740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2491db110 0x55e2491ed490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.780+0000 7f40d59ee740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc1aaa80 con 0x55e2491d6e00 2024-05-15T14:03:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.780+0000 7f40d59ee740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc1abef0 con 0x55e2491da7f0 2024-05-15T14:03:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.780+0000 7f40d59ee740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc1ab150 con 0x55e2491db110 2024-05-15T14:03:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.780+0000 7f40d59ee740 10 client.0 _ll_register_callbacks cb 0x7f40c0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:03:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.780+0000 7f40d0993640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2491d6e00 0x55e2491f1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.780+0000 7f40d0993640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2491d6e00 0x55e2491f1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45728/0 (socket says 172.21.15.33:45728) 2024-05-15T14:03:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.780+0000 7f40d0993640 1 -- 172.21.15.33:0/2914012722 learned_addr learned my addr 172.21.15.33:0/2914012722 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.780+0000 7f40d1194640 1 --2- 172.21.15.33:0/2914012722 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2491db110 0x55e2491ed490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.780+0000 7f40cbfff640 1 --2- 172.21.15.33:0/2914012722 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2491da7f0 0x55e2491f2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.781+0000 7f40d0993640 1 -- 172.21.15.33:0/2914012722 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2491db110 msgr2=0x55e2491ed490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.781+0000 7f40d0993640 1 --2- 172.21.15.33:0/2914012722 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2491db110 0x55e2491ed490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.781+0000 7f40d0993640 1 -- 172.21.15.33:0/2914012722 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2491da7f0 msgr2=0x55e2491f2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.781+0000 7f40d0993640 1 --2- 172.21.15.33:0/2914012722 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2491da7f0 0x55e2491f2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.781+0000 7f40d0993640 1 -- 172.21.15.33:0/2914012722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2491dbe60 con 0x55e2491d6e00 2024-05-15T14:03:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.781+0000 7f40d0993640 1 --2- 172.21.15.33:0/2914012722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2491d6e00 0x55e2491f1f10 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f40c0004d30 tx=0x7f40c0001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=371e316d62060dfd server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.781+0000 7f40abfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:03:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.781+0000 7f40abfff640 1 -- 172.21.15.33:0/2914012722 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c0040040 con 0x55e2491d6e00 2024-05-15T14:03:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.781+0000 7f40abfff640 1 -- 172.21.15.33:0/2914012722 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8498+0+0 (secure 0 0 0) 0x7f40c004a070 con 0x55e2491d6e00 2024-05-15T14:03:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.781+0000 7f40d59ee740 1 -- 172.21.15.33:0/2914012722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40c40a61f0 con 0x55e2491d6e00 2024-05-15T14:03:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.781+0000 7f40abfff640 1 -- 172.21.15.33:0/2914012722 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f40c0065050 con 0x55e2491d6e00 2024-05-15T14:03:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.781+0000 7f40d59ee740 10 client.15705 fetch_fsmap learned FSMap version 25 2024-05-15T14:03:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.781+0000 7f40d59ee740 1 -- 172.21.15.33:0/2914012722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e2492638b0 con 0x55e2491d6e00 2024-05-15T14:03:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.781+0000 7f40abfff640 1 -- 172.21.15.33:0/2914012722 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40c40a61f0 con 0x55e2491d6e00 2024-05-15T14:03:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.782+0000 7f40abfff640 1 -- 172.21.15.33:0/2914012722 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f40c0057500 con 0x55e2491d6e00 2024-05-15T14:03:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.782+0000 7f40d59ee740 10 client.15705 fetch_fsmap finished waiting for FSMap version 25 2024-05-15T14:03:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.782+0000 7f40d59ee740 10 client.15705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:03:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.782+0000 7f40d59ee740 20 client.15705 populate_metadata read hostname 'smithi033' 2024-05-15T14:03:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.782+0000 7f40d59ee740 1 --2- 172.21.15.33:0/2914012722 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e2491e9ee0 0x55e249266060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.782+0000 7f40d59ee740 4 client.15705 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:03:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.782+0000 7f40d59ee740 1 -- 172.21.15.33:0/2914012722 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e249149360 con 0x55e2491e9ee0 2024-05-15T14:03:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.782+0000 7f40cbfff640 1 --2- 172.21.15.33:0/2914012722 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e2491e9ee0 0x55e249266060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.782+0000 7f40cbfff640 1 --2- 172.21.15.33:0/2914012722 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e2491e9ee0 0x55e249266060 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a94a2497e044bba server_cookie=8f77ad80b6634759 in_seq=0 out_seq=0 2024-05-15T14:03:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.782+0000 7f40abfff640 10 client.15705 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:03:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.840+0000 7f40abfff640 1 -- 172.21.15.33:0/2914012722 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e249149360 con 0x55e2491e9ee0 2024-05-15T14:03:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.840+0000 7f40abfff640 10 client.15705 handle_command_reply: tid=0 2024-05-15T14:03:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.840+0000 7f40d59ee740 1 client.15705 shutdown 2024-05-15T14:03:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.840+0000 7f40d59ee740 1 -- 172.21.15.33:0/2914012722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2491d6e00 msgr2=0x55e2491f1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.840+0000 7f40d59ee740 1 --2- 172.21.15.33:0/2914012722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2491d6e00 0x55e2491f1f10 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f40c0004d30 tx=0x7f40c0001e50 comp rx=0 tx=0).stop 2024-05-15T14:03:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.840+0000 7f40d59ee740 1 -- 172.21.15.33:0/2914012722 shutdown_connections 2024-05-15T14:03:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.840+0000 7f40d59ee740 1 --2- 172.21.15.33:0/2914012722 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e2491e9ee0 0x55e249266060 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.840+0000 7f40d59ee740 1 --2- 172.21.15.33:0/2914012722 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2491db110 0x55e2491ed490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.840+0000 7f40d59ee740 1 --2- 172.21.15.33:0/2914012722 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2491da7f0 0x55e2491f2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.840+0000 7f40d59ee740 1 --2- 172.21.15.33:0/2914012722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2491d6e00 0x55e2491f1f10 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:41.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.841+0000 7f40d59ee740 1 -- 172.21.15.33:0/2914012722 >> 172.21.15.33:0/2914012722 conn(0x55e2491e7140 msgr2=0x55e2491dc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:41.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.840+0000 7f40abfff640 0 client.15705 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:03:41.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.841+0000 7f40d59ee740 1 -- 172.21.15.33:0/2914012722 shutdown_connections 2024-05-15T14:03:41.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.841+0000 7f40d59ee740 1 -- 172.21.15.33:0/2914012722 wait complete. 2024-05-15T14:03:41.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.841+0000 7f40d59ee740 20 client.15705 trim_cache size 0 max 16384 2024-05-15T14:03:41.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:03:41.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:03:41.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:03:41.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.842+0000 7f40d2bf7640 1 -- 172.21.15.33:0/4093704769 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f40bc0b2950 msgr2=0x7f40bc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:41.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.842+0000 7f40d2bf7640 1 --2- 172.21.15.33:0/4093704769 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f40bc0b2950 0x7f40bc0b4e10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55e2491dc3b0 tx=0x7f40b8009c80 comp rx=0 tx=0).stop 2024-05-15T14:03:41.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.842+0000 7f40d2bf7640 1 -- 172.21.15.33:0/4093704769 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40cc107960 msgr2=0x7f40cc19db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:41.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.842+0000 7f40d2bf7640 1 --2- 172.21.15.33:0/4093704769 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40cc107960 0x7f40cc19db70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f40c4009fd0 tx=0x7f40c400ef10 comp rx=0 tx=0).stop 2024-05-15T14:03:41.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.842+0000 7f40d2bf7640 1 -- 172.21.15.33:0/4093704769 shutdown_connections 2024-05-15T14:03:41.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.842+0000 7f40d2bf7640 1 --2- 172.21.15.33:0/4093704769 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f40bc0b2950 0x7f40bc0b4e10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:41.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.842+0000 7f40d2bf7640 1 --2- 172.21.15.33:0/4093704769 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f40cc107960 0x7f40cc19db70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:41.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.842+0000 7f40d2bf7640 1 --2- 172.21.15.33:0/4093704769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40cc102310 0x7f40cc1997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:41.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.842+0000 7f40d2bf7640 1 --2- 172.21.15.33:0/4093704769 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40cc101950 0x7f40cc1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:41.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.842+0000 7f40d2bf7640 1 -- 172.21.15.33:0/4093704769 >> 172.21.15.33:0/4093704769 conn(0x7f40cc078a50 msgr2=0x7f40cc100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:41.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.842+0000 7f40d2bf7640 1 -- 172.21.15.33:0/4093704769 shutdown_connections 2024-05-15T14:03:41.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:41.842+0000 7f40d2bf7640 1 -- 172.21.15.33:0/4093704769 wait complete. 2024-05-15T14:03:42.050 INFO:tasks.workunit.client.0.smithi033.stdout: 1 165098 11.77 MB/sec warmup 77 sec latency 32.132 ms 2024-05-15T14:03:42.357 INFO:journalctl@ceph.mds.f.smithi081.stdout:May 15 14:03:41 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-f[66264]: ignoring --setuser ceph since I am not root 2024-05-15T14:03:42.357 INFO:journalctl@ceph.mds.f.smithi081.stdout:May 15 14:03:41 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-f[66264]: ignoring --setgroup ceph since I am not root 2024-05-15T14:03:42.357 INFO:journalctl@ceph.mds.f.smithi081.stdout:May 15 14:03:41 smithi081 ceph-mds[66268]: -- 172.21.15.81:0/1816926317 <== mon.1 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56320e473a40 con 0x56320e446c00 2024-05-15T14:03:42.357 INFO:journalctl@ceph.mds.f.smithi081.stdout:May 15 14:03:41 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-f[66264]: starting mds.f at 2024-05-15T14:03:42.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:42 smithi033 ceph-mon[31650]: pgmap v103: 97 pgs: 97 active+clean; 54 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 298 op/s 2024-05-15T14:03:42.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:42 smithi033 ceph-mon[31650]: daemon mds.f restarted 2024-05-15T14:03:42.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:42 smithi052 ceph-mon[36190]: pgmap v103: 97 pgs: 97 active+clean; 54 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 298 op/s 2024-05-15T14:03:42.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:42 smithi052 ceph-mon[36190]: daemon mds.f restarted 2024-05-15T14:03:42.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:42 smithi081 ceph-mon[36434]: pgmap v103: 97 pgs: 97 active+clean; 54 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 298 op/s 2024-05-15T14:03:42.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:42 smithi081 ceph-mon[36434]: daemon mds.f restarted 2024-05-15T14:03:43.050 INFO:tasks.workunit.client.0.smithi033.stdout: 1 167185 11.79 MB/sec warmup 78 sec latency 19.529 ms 2024-05-15T14:03:43.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.071+0000 7f34033d6700 1 -- 172.21.15.33:0/2034897384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33fc103690 msgr2=0x7f33fc103af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:43.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.071+0000 7f34033d6700 1 --2- 172.21.15.33:0/2034897384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33fc103690 0x7f33fc103af0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f33ec009990 tx=0x7f33ec009ca0 comp rx=0 tx=0).stop 2024-05-15T14:03:43.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.071+0000 7f34033d6700 1 -- 172.21.15.33:0/2034897384 shutdown_connections 2024-05-15T14:03:43.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.071+0000 7f34033d6700 1 --2- 172.21.15.33:0/2034897384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33fc104030 0x7f33fc110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:43.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.071+0000 7f34033d6700 1 --2- 172.21.15.33:0/2034897384 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33fc1084e0 0x7f33fc1088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:43.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.071+0000 7f34033d6700 1 --2- 172.21.15.33:0/2034897384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33fc103690 0x7f33fc103af0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:43.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.071+0000 7f34033d6700 1 -- 172.21.15.33:0/2034897384 >> 172.21.15.33:0/2034897384 conn(0x7f33fc07a730 msgr2=0x7f33fc07ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:43.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.071+0000 7f34033d6700 1 -- 172.21.15.33:0/2034897384 shutdown_connections 2024-05-15T14:03:43.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.071+0000 7f34033d6700 1 -- 172.21.15.33:0/2034897384 wait complete. 2024-05-15T14:03:43.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.072+0000 7f34033d6700 1 Processor -- start 2024-05-15T14:03:43.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.072+0000 7f34033d6700 1 -- start start 2024-05-15T14:03:43.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.072+0000 7f34033d6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33fc103690 0x7f33fc19f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:43.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.072+0000 7f34033d6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33fc104030 0x7f33fc19fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:43.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.072+0000 7f34033d6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33fc1084e0 0x7f33fc199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:43.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.072+0000 7f34033d6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc113770 con 0x7f33fc1084e0 2024-05-15T14:03:43.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.072+0000 7f34033d6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc1135f0 con 0x7f33fc104030 2024-05-15T14:03:43.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.072+0000 7f34033d6700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc1138f0 con 0x7f33fc103690 2024-05-15T14:03:43.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.072+0000 7f3401172700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33fc103690 0x7f33fc19f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:43.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.072+0000 7f3401973700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33fc1084e0 0x7f33fc199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:43.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.072+0000 7f3401172700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33fc103690 0x7f33fc19f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:60132/0 (socket says 172.21.15.33:60132) 2024-05-15T14:03:43.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.072+0000 7f3401172700 1 -- 172.21.15.33:0/1965155910 learned_addr learned my addr 172.21.15.33:0/1965155910 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:43.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.072+0000 7f3400971700 1 --2- 172.21.15.33:0/1965155910 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33fc104030 0x7f33fc19fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:43.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.072+0000 7f3401172700 1 -- 172.21.15.33:0/1965155910 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33fc104030 msgr2=0x7f33fc19fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:43.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.072+0000 7f3401172700 1 --2- 172.21.15.33:0/1965155910 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33fc104030 0x7f33fc19fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:43.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.072+0000 7f3401172700 1 -- 172.21.15.33:0/1965155910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33fc1084e0 msgr2=0x7f33fc199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:43.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.072+0000 7f3401172700 1 --2- 172.21.15.33:0/1965155910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33fc1084e0 0x7f33fc199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:43.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.072+0000 7f3401172700 1 -- 172.21.15.33:0/1965155910 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33fc19a0d0 con 0x7f33fc103690 2024-05-15T14:03:43.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.073+0000 7f3401172700 1 --2- 172.21.15.33:0/1965155910 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33fc103690 0x7f33fc19f7b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f33f800e970 tx=0x7f33f800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:43.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.073+0000 7f33f27fc700 1 -- 172.21.15.33:0/1965155910 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f8004220 con 0x7f33fc103690 2024-05-15T14:03:43.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.073+0000 7f33f27fc700 1 -- 172.21.15.33:0/1965155910 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33f80043c0 con 0x7f33fc103690 2024-05-15T14:03:43.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.073+0000 7f34033d6700 1 -- 172.21.15.33:0/1965155910 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33fc19a3c0 con 0x7f33fc103690 2024-05-15T14:03:43.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.073+0000 7f33f27fc700 1 -- 172.21.15.33:0/1965155910 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f8003ce0 con 0x7f33fc103690 2024-05-15T14:03:43.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.073+0000 7f34033d6700 1 -- 172.21.15.33:0/1965155910 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33fc1a6680 con 0x7f33fc103690 2024-05-15T14:03:43.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.075+0000 7f33f27fc700 1 -- 172.21.15.33:0/1965155910 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f33f800c7e0 con 0x7f33fc103690 2024-05-15T14:03:43.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.076+0000 7f33f27fc700 1 --2- 172.21.15.33:0/1965155910 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f33e80b2aa0 0x7f33e80b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:43.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.076+0000 7f3400971700 1 --2- 172.21.15.33:0/1965155910 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f33e80b2aa0 0x7f33e80b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:43.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.076+0000 7f33f27fc700 1 -- 172.21.15.33:0/1965155910 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f33f80e27b0 con 0x7f33fc103690 2024-05-15T14:03:43.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.077+0000 7f3400971700 1 --2- 172.21.15.33:0/1965155910 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f33e80b2aa0 0x7f33e80b4f60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f33ec00b2e0 tx=0x7f33ec005960 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:43.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.077+0000 7f340dc6cb80 1 Processor -- start 2024-05-15T14:03:43.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.077+0000 7f340dc6cb80 1 -- start start 2024-05-15T14:03:43.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.077+0000 7f340dc6cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9ac80f4b0 0x55e9ac80f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:43.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.077+0000 7f340dc6cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ac80fdd0 0x55e9ac80ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:43.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.077+0000 7f340dc6cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9ac80b350 0x55e9ac80b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:43.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.077+0000 7f340dc6cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e9ac768f00 con 0x55e9ac80f4b0 2024-05-15T14:03:43.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.077+0000 7f340dc6cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e9ac758aa0 con 0x55e9ac80fdd0 2024-05-15T14:03:43.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.077+0000 7f340dc6cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55e9ac75cae0 con 0x55e9ac80b350 2024-05-15T14:03:43.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.077+0000 7f3401172700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9ac80f4b0 0x55e9ac80f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:43.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.077+0000 7f3401973700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9ac80b350 0x55e9ac80b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:43.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.077+0000 7f3400971700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ac80fdd0 0x55e9ac80ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:43.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.078+0000 7f3401172700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9ac80f4b0 0x55e9ac80f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48498/0 (socket says 172.21.15.33:48498) 2024-05-15T14:03:43.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.077+0000 7f3401973700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9ac80b350 0x55e9ac80b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:60134/0 (socket says 172.21.15.33:60134) 2024-05-15T14:03:43.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.078+0000 7f3401172700 1 -- 172.21.15.33:0/2721229696 learned_addr learned my addr 172.21.15.33:0/2721229696 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:43.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.078+0000 7f3401973700 1 -- 172.21.15.33:0/2721229696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ac80fdd0 msgr2=0x55e9ac80ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:43.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.078+0000 7f3401973700 1 --2- 172.21.15.33:0/2721229696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ac80fdd0 0x55e9ac80ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:43.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.078+0000 7f3401973700 1 -- 172.21.15.33:0/2721229696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9ac80f4b0 msgr2=0x55e9ac80f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:43.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.078+0000 7f3401973700 1 --2- 172.21.15.33:0/2721229696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9ac80f4b0 0x55e9ac80f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:43.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.078+0000 7f3401973700 1 -- 172.21.15.33:0/2721229696 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9ac702140 con 0x55e9ac80b350 2024-05-15T14:03:43.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.078+0000 7f3401172700 1 --2- 172.21.15.33:0/2721229696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9ac80f4b0 0x55e9ac80f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:03:43.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.078+0000 7f3401973700 1 --2- 172.21.15.33:0/2721229696 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9ac80b350 0x55e9ac80b7b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f33f4011580 tx=0x7f33f4009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7cd6f7e21726dbc server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:43.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.078+0000 7f33e77fe700 1 -- 172.21.15.33:0/2721229696 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f4011e80 con 0x55e9ac80b350 2024-05-15T14:03:43.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.078+0000 7f33e77fe700 1 -- 172.21.15.33:0/2721229696 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33f4004bb0 con 0x55e9ac80b350 2024-05-15T14:03:43.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.078+0000 7f33e77fe700 1 -- 172.21.15.33:0/2721229696 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f40046c0 con 0x55e9ac80b350 2024-05-15T14:03:43.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.079+0000 7f340dc6cb80 1 -- 172.21.15.33:0/2721229696 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9ac80b350 msgr2=0x55e9ac80b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:43.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.079+0000 7f340dc6cb80 1 --2- 172.21.15.33:0/2721229696 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9ac80b350 0x55e9ac80b7b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f33f4011580 tx=0x7f33f4009920 comp rx=0 tx=0).stop 2024-05-15T14:03:43.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.079+0000 7f340dc6cb80 1 -- 172.21.15.33:0/2721229696 shutdown_connections 2024-05-15T14:03:43.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.079+0000 7f340dc6cb80 1 --2- 172.21.15.33:0/2721229696 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9ac80b350 0x55e9ac80b7b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:43.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.079+0000 7f340dc6cb80 1 --2- 172.21.15.33:0/2721229696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9ac80f4b0 0x55e9ac80f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:43.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.079+0000 7f340dc6cb80 1 --2- 172.21.15.33:0/2721229696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ac80fdd0 0x55e9ac80ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:43.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.079+0000 7f340dc6cb80 1 -- 172.21.15.33:0/2721229696 >> 172.21.15.33:0/2721229696 conn(0x55e9ac737f40 msgr2=0x55e9ac805530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:43.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.079+0000 7f340dc6cb80 1 -- 172.21.15.33:0/2721229696 shutdown_connections 2024-05-15T14:03:43.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.079+0000 7f340dc6cb80 1 -- 172.21.15.33:0/2721229696 wait complete. 2024-05-15T14:03:43.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.079+0000 7f340dc6cb80 1 -- start start 2024-05-15T14:03:43.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.080+0000 7f340dc6cb80 1 Processor -- start 2024-05-15T14:03:43.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.080+0000 7f340dc6cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9ac80b350 0x55e9ac816c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:43.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.080+0000 7f340dc6cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9ac80f4b0 0x55e9ac813fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:43.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.080+0000 7f340dc6cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ac80fdd0 0x55e9ac814500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:43.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.080+0000 7f340dc6cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e9ac65f130 con 0x55e9ac80f4b0 2024-05-15T14:03:43.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.080+0000 7f340dc6cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e9ac6c0f30 con 0x55e9ac80fdd0 2024-05-15T14:03:43.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.080+0000 7f340dc6cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55e9ac817910 con 0x55e9ac80b350 2024-05-15T14:03:43.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.080+0000 7f340dc6cb80 10 client.0 _ll_register_callbacks cb 0x55e9ac6f3ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:03:43.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.080+0000 7f3401172700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9ac80b350 0x55e9ac816c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:43.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.080+0000 7f3400971700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9ac80f4b0 0x55e9ac813fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:43.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.080+0000 7f3401973700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ac80fdd0 0x55e9ac814500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:43.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.080+0000 7f3400971700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9ac80f4b0 0x55e9ac813fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48506/0 (socket says 172.21.15.33:48506) 2024-05-15T14:03:43.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.080+0000 7f3401172700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9ac80b350 0x55e9ac816c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:60136/0 (socket says 172.21.15.33:60136) 2024-05-15T14:03:43.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.080+0000 7f3400971700 1 -- 172.21.15.33:0/1410293533 learned_addr learned my addr 172.21.15.33:0/1410293533 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:43.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.080+0000 7f3401172700 1 -- 172.21.15.33:0/1410293533 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ac80fdd0 msgr2=0x55e9ac814500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:43.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.080+0000 7f3401172700 1 --2- 172.21.15.33:0/1410293533 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ac80fdd0 0x55e9ac814500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:43.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.080+0000 7f3401172700 1 -- 172.21.15.33:0/1410293533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9ac80f4b0 msgr2=0x55e9ac813fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:43.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.080+0000 7f3401172700 1 --2- 172.21.15.33:0/1410293533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9ac80f4b0 0x55e9ac813fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:43.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.081+0000 7f3401172700 1 -- 172.21.15.33:0/1410293533 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9ac6a8a60 con 0x55e9ac80b350 2024-05-15T14:03:43.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.081+0000 7f3401172700 1 --2- 172.21.15.33:0/1410293533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9ac80b350 0x55e9ac816c20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f33f80a5b20 tx=0x7f33f8004570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f4140feed20af04 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:43.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.081+0000 7f33e4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-05-15T14:03:43.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.081+0000 7f33e4ff9700 1 -- 172.21.15.33:0/1410293533 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f80c3060 con 0x55e9ac80b350 2024-05-15T14:03:43.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.081+0000 7f340dc6cb80 1 -- 172.21.15.33:0/1410293533 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33f4004bb0 con 0x55e9ac80b350 2024-05-15T14:03:43.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.081+0000 7f33e4ff9700 1 -- 172.21.15.33:0/1410293533 <== mon.1 v2:172.21.15.81:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f33f8016330 con 0x55e9ac80b350 2024-05-15T14:03:43.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.081+0000 7f33e4ff9700 1 -- 172.21.15.33:0/1410293533 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f33f80a95b0 con 0x55e9ac80b350 2024-05-15T14:03:43.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.081+0000 7f340dc6cb80 10 client.25405 fetch_fsmap learned FSMap version 27 2024-05-15T14:03:43.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.081+0000 7f340dc6cb80 1 -- 172.21.15.33:0/1410293533 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e9ac7128f0 con 0x55e9ac80b350 2024-05-15T14:03:43.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.081+0000 7f33e4ff9700 1 -- 172.21.15.33:0/1410293533 <== mon.1 v2:172.21.15.81:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33f4004bb0 con 0x55e9ac80b350 2024-05-15T14:03:43.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.081+0000 7f33e4ff9700 1 -- 172.21.15.33:0/1410293533 <== mon.1 v2:172.21.15.81:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f33f80a9d00 con 0x55e9ac80b350 2024-05-15T14:03:43.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.081+0000 7f340dc6cb80 10 client.25405 fetch_fsmap finished waiting for FSMap version 27 2024-05-15T14:03:43.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.081+0000 7f340dc6cb80 10 client.25405 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:03:43.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.081+0000 7f340dc6cb80 20 client.25405 populate_metadata read hostname 'smithi033' 2024-05-15T14:03:43.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.081+0000 7f340dc6cb80 1 --2- 172.21.15.33:0/1410293533 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e9ac824500 0x55e9ac8268f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:43.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.081+0000 7f340dc6cb80 4 client.25405 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:03:43.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.082+0000 7f340dc6cb80 1 -- 172.21.15.33:0/1410293533 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9ac76ae50 con 0x55e9ac824500 2024-05-15T14:03:43.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.082+0000 7f3400971700 1 --2- 172.21.15.33:0/1410293533 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e9ac824500 0x55e9ac8268f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:43.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.082+0000 7f33e4ff9700 10 client.25405 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:03:43.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.082+0000 7f3400971700 1 --2- 172.21.15.33:0/1410293533 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e9ac824500 0x55e9ac8268f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8dc4c772ad20bdb server_cookie=3aca65dc90a206d7 in_seq=0 out_seq=0 2024-05-15T14:03:43.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.083+0000 7f33e4ff9700 1 -- 172.21.15.33:0/1410293533 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e9ac76ae50 con 0x55e9ac824500 2024-05-15T14:03:43.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.083+0000 7f33e4ff9700 10 client.25405 handle_command_reply: tid=0 2024-05-15T14:03:43.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.083+0000 7f340dc6cb80 1 client.25405 shutdown 2024-05-15T14:03:43.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.083+0000 7f340dc6cb80 1 -- 172.21.15.33:0/1410293533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9ac80b350 msgr2=0x55e9ac816c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:43.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.083+0000 7f340dc6cb80 1 --2- 172.21.15.33:0/1410293533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9ac80b350 0x55e9ac816c20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f33f80a5b20 tx=0x7f33f8004570 comp rx=0 tx=0).stop 2024-05-15T14:03:43.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.083+0000 7f340dc6cb80 1 -- 172.21.15.33:0/1410293533 shutdown_connections 2024-05-15T14:03:43.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.083+0000 7f340dc6cb80 1 --2- 172.21.15.33:0/1410293533 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ac80fdd0 0x55e9ac814500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:43.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.083+0000 7f340dc6cb80 1 --2- 172.21.15.33:0/1410293533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9ac80b350 0x55e9ac816c20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:43.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.083+0000 7f340dc6cb80 1 --2- 172.21.15.33:0/1410293533 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e9ac824500 0x55e9ac8268f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:43.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.083+0000 7f340dc6cb80 1 --2- 172.21.15.33:0/1410293533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9ac80f4b0 0x55e9ac813fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:43.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.083+0000 7f340dc6cb80 1 -- 172.21.15.33:0/1410293533 >> 172.21.15.33:0/1410293533 conn(0x55e9ac737f40 msgr2=0x55e9ac800050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:43.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.083+0000 7f33e4ff9700 0 client.25405 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:03:43.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.083+0000 7f340dc6cb80 1 -- 172.21.15.33:0/1410293533 shutdown_connections 2024-05-15T14:03:43.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.083+0000 7f340dc6cb80 1 -- 172.21.15.33:0/1410293533 wait complete. 2024-05-15T14:03:43.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.083+0000 7f340dc6cb80 20 client.25405 trim_cache size 0 max 16384 2024-05-15T14:03:43.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.108+0000 7f340dc6cb80 1 Processor -- start 2024-05-15T14:03:43.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.108+0000 7f340dc6cb80 1 -- start start 2024-05-15T14:03:43.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.108+0000 7f340dc6cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ac80b350 0x55e9ac800160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:43.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.108+0000 7f340dc6cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9ac80f4b0 0x55e9ac808d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:43.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.108+0000 7f340dc6cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9ac80fdd0 0x55e9ac809260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:43.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.108+0000 7f340dc6cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc1b0700 con 0x55e9ac80f4b0 2024-05-15T14:03:43.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.108+0000 7f340dc6cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc1adf30 con 0x55e9ac80b350 2024-05-15T14:03:43.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.108+0000 7f340dc6cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc1abd80 con 0x55e9ac80fdd0 2024-05-15T14:03:43.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.108+0000 7f3400971700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9ac80f4b0 0x55e9ac808d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:43.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.108+0000 7f3400971700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9ac80f4b0 0x55e9ac808d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48522/0 (socket says 172.21.15.33:48522) 2024-05-15T14:03:43.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.108+0000 7f3400971700 1 -- 172.21.15.33:0/3990893908 learned_addr learned my addr 172.21.15.33:0/3990893908 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:43.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.108+0000 7f3401973700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9ac80fdd0 0x55e9ac809260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:43.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.108+0000 7f3401172700 1 --2- 172.21.15.33:0/3990893908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ac80b350 0x55e9ac800160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:43.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.108+0000 7f3401973700 1 -- 172.21.15.33:0/3990893908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ac80b350 msgr2=0x55e9ac800160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:43.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.108+0000 7f3401973700 1 --2- 172.21.15.33:0/3990893908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ac80b350 0x55e9ac800160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:43.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.108+0000 7f3401973700 1 -- 172.21.15.33:0/3990893908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9ac80f4b0 msgr2=0x55e9ac808d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:43.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.108+0000 7f3401973700 1 --2- 172.21.15.33:0/3990893908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9ac80f4b0 0x55e9ac808d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:43.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.108+0000 7f3401973700 1 -- 172.21.15.33:0/3990893908 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9ac814ef0 con 0x55e9ac80fdd0 2024-05-15T14:03:43.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.108+0000 7f3400971700 1 --2- 172.21.15.33:0/3990893908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9ac80f4b0 0x55e9ac808d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:03:43.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.109+0000 7f3401973700 1 --2- 172.21.15.33:0/3990893908 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9ac80fdd0 0x55e9ac809260 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f33f4009690 tx=0x7f33f4005b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=589fe1e66ed1a1a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:43.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.109+0000 7f33e6ffd700 1 -- 172.21.15.33:0/3990893908 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f40093e0 con 0x55e9ac80fdd0 2024-05-15T14:03:43.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.109+0000 7f33e6ffd700 1 -- 172.21.15.33:0/3990893908 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33f4003a00 con 0x55e9ac80fdd0 2024-05-15T14:03:43.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.109+0000 7f33e6ffd700 1 -- 172.21.15.33:0/3990893908 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f4015690 con 0x55e9ac80fdd0 2024-05-15T14:03:43.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.109+0000 7f340dc6cb80 1 -- 172.21.15.33:0/3990893908 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9ac80fdd0 msgr2=0x55e9ac809260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:43.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.109+0000 7f340dc6cb80 1 --2- 172.21.15.33:0/3990893908 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9ac80fdd0 0x55e9ac809260 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f33f4009690 tx=0x7f33f4005b00 comp rx=0 tx=0).stop 2024-05-15T14:03:43.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.110+0000 7f340dc6cb80 1 -- 172.21.15.33:0/3990893908 shutdown_connections 2024-05-15T14:03:43.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.110+0000 7f340dc6cb80 1 --2- 172.21.15.33:0/3990893908 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9ac80fdd0 0x55e9ac809260 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:43.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.110+0000 7f340dc6cb80 1 --2- 172.21.15.33:0/3990893908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ac80b350 0x55e9ac800160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:43.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.110+0000 7f340dc6cb80 1 --2- 172.21.15.33:0/3990893908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9ac80f4b0 0x55e9ac808d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:43.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.110+0000 7f340dc6cb80 1 -- 172.21.15.33:0/3990893908 >> 172.21.15.33:0/3990893908 conn(0x55e9ac737f40 msgr2=0x55e9ac809980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:43.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.110+0000 7f340dc6cb80 1 -- 172.21.15.33:0/3990893908 shutdown_connections 2024-05-15T14:03:43.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.110+0000 7f340dc6cb80 1 -- 172.21.15.33:0/3990893908 wait complete. 2024-05-15T14:03:43.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.110+0000 7f340dc6cb80 1 -- start start 2024-05-15T14:03:43.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.110+0000 7f340dc6cb80 1 Processor -- start 2024-05-15T14:03:43.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.110+0000 7f340dc6cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ac80b350 0x55e9ac817b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:43.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.110+0000 7f340dc6cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9ac80f4b0 0x55e9ac8180a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:43.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.110+0000 7f340dc6cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9ac80fdd0 0x55e9ac8185e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:43.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.110+0000 7f340dc6cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc1ac3f0 con 0x55e9ac80f4b0 2024-05-15T14:03:43.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.110+0000 7f340dc6cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc1ad860 con 0x55e9ac80b350 2024-05-15T14:03:43.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.110+0000 7f340dc6cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc1acac0 con 0x55e9ac80fdd0 2024-05-15T14:03:43.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.111+0000 7f340dc6cb80 10 client.0 _ll_register_callbacks cb 0x55e9ac6f3ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:03:43.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.111+0000 7f3400971700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9ac80f4b0 0x55e9ac8180a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:43.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.111+0000 7f3401973700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9ac80fdd0 0x55e9ac8185e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:43.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.111+0000 7f3401172700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ac80b350 0x55e9ac817b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:43.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.111+0000 7f3401973700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9ac80fdd0 0x55e9ac8185e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:60160/0 (socket says 172.21.15.33:60160) 2024-05-15T14:03:43.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.111+0000 7f3401973700 1 -- 172.21.15.33:0/1950082504 learned_addr learned my addr 172.21.15.33:0/1950082504 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:43.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.111+0000 7f3400971700 1 -- 172.21.15.33:0/1950082504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9ac80fdd0 msgr2=0x55e9ac8185e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:43.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.111+0000 7f3400971700 1 --2- 172.21.15.33:0/1950082504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9ac80fdd0 0x55e9ac8185e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:43.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.111+0000 7f3400971700 1 -- 172.21.15.33:0/1950082504 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ac80b350 msgr2=0x55e9ac817b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:43.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.111+0000 7f3400971700 1 --2- 172.21.15.33:0/1950082504 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ac80b350 0x55e9ac817b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:43.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.111+0000 7f3400971700 1 -- 172.21.15.33:0/1950082504 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9ac88ece0 con 0x55e9ac80f4b0 2024-05-15T14:03:43.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.111+0000 7f3401172700 1 --2- 172.21.15.33:0/1950082504 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ac80b350 0x55e9ac817b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:03:43.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.111+0000 7f3400971700 1 --2- 172.21.15.33:0/1950082504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9ac80f4b0 0x55e9ac8180a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f33ec006010 tx=0x7f33ec01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67806aae21b792b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:43.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.111+0000 7f33e4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:03:43.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.111+0000 7f33e4ff9700 1 -- 172.21.15.33:0/1950082504 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec003890 con 0x55e9ac80f4b0 2024-05-15T14:03:43.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.111+0000 7f33e4ff9700 1 -- 172.21.15.33:0/1950082504 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f33ec003a30 con 0x55e9ac80f4b0 2024-05-15T14:03:43.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.111+0000 7f340dc6cb80 1 -- 172.21.15.33:0/1950082504 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33f4003a00 con 0x55e9ac80f4b0 2024-05-15T14:03:43.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.112+0000 7f33e4ff9700 1 -- 172.21.15.33:0/1950082504 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f33ec043050 con 0x55e9ac80f4b0 2024-05-15T14:03:43.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.112+0000 7f340dc6cb80 10 client.15729 fetch_fsmap learned FSMap version 27 2024-05-15T14:03:43.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.112+0000 7f340dc6cb80 1 -- 172.21.15.33:0/1950082504 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e9ac88ffe0 con 0x55e9ac80f4b0 2024-05-15T14:03:43.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.112+0000 7f33e4ff9700 1 -- 172.21.15.33:0/1950082504 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33f4003a00 con 0x55e9ac80f4b0 2024-05-15T14:03:43.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.112+0000 7f33e4ff9700 1 -- 172.21.15.33:0/1950082504 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f33ec00fbb0 con 0x55e9ac80f4b0 2024-05-15T14:03:43.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.112+0000 7f340dc6cb80 10 client.15729 fetch_fsmap finished waiting for FSMap version 27 2024-05-15T14:03:43.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.112+0000 7f340dc6cb80 10 client.15729 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:03:43.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.112+0000 7f340dc6cb80 20 client.15729 populate_metadata read hostname 'smithi033' 2024-05-15T14:03:43.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.112+0000 7f340dc6cb80 1 --2- 172.21.15.33:0/1950082504 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e9ac8907c0 0x55e9ac892be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:43.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.112+0000 7f340dc6cb80 4 client.15729 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-05-15T14:03:43.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.112+0000 7f340dc6cb80 1 -- 172.21.15.33:0/1950082504 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x55e9ac764630 con 0x55e9ac8907c0 2024-05-15T14:03:43.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.112+0000 7f3401172700 1 --2- 172.21.15.33:0/1950082504 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e9ac8907c0 0x55e9ac892be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:43.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.112+0000 7f33e4ff9700 10 client.15729 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:03:43.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.112+0000 7f3401172700 1 --2- 172.21.15.33:0/1950082504 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e9ac8907c0 0x55e9ac892be0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33ab02fbf1f68924 server_cookie=fe50ecd56c998d37 in_seq=0 out_seq=0 2024-05-15T14:03:43.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.199+0000 7f33e4ff9700 1 -- 172.21.15.33:0/1950082504 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e9ac764630 con 0x55e9ac8907c0 2024-05-15T14:03:43.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.199+0000 7f33e4ff9700 10 client.15729 handle_command_reply: tid=0 2024-05-15T14:03:43.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.199+0000 7f340dc6cb80 1 client.15729 shutdown 2024-05-15T14:03:43.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.199+0000 7f340dc6cb80 1 -- 172.21.15.33:0/1950082504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9ac80f4b0 msgr2=0x55e9ac8180a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:43.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.199+0000 7f340dc6cb80 1 --2- 172.21.15.33:0/1950082504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9ac80f4b0 0x55e9ac8180a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f33ec006010 tx=0x7f33ec01f000 comp rx=0 tx=0).stop 2024-05-15T14:03:43.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.199+0000 7f340dc6cb80 1 -- 172.21.15.33:0/1950082504 shutdown_connections 2024-05-15T14:03:43.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.199+0000 7f340dc6cb80 1 --2- 172.21.15.33:0/1950082504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e9ac80fdd0 0x55e9ac8185e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:43.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.199+0000 7f340dc6cb80 1 --2- 172.21.15.33:0/1950082504 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ac80b350 0x55e9ac817b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:43.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.199+0000 7f340dc6cb80 1 --2- 172.21.15.33:0/1950082504 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e9ac8907c0 0x55e9ac892be0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:43.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.199+0000 7f340dc6cb80 1 --2- 172.21.15.33:0/1950082504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9ac80f4b0 0x55e9ac8180a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:43.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.199+0000 7f340dc6cb80 1 -- 172.21.15.33:0/1950082504 >> 172.21.15.33:0/1950082504 conn(0x55e9ac737f40 msgr2=0x55e9ac80a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:43.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.199+0000 7f33e4ff9700 0 client.15729 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:03:43.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.199+0000 7f340dc6cb80 1 -- 172.21.15.33:0/1950082504 shutdown_connections 2024-05-15T14:03:43.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.199+0000 7f340dc6cb80 1 -- 172.21.15.33:0/1950082504 wait complete. 2024-05-15T14:03:43.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.199+0000 7f340dc6cb80 20 client.15729 trim_cache size 0 max 16384 2024-05-15T14:03:43.205 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T14:03:43.205 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 27, 2024-05-15T14:03:43.205 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24478, 2024-05-15T14:03:43.205 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 10, 2024-05-15T14:03:43.205 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-05-15T14:03:43.205 INFO:teuthology.orchestra.run.smithi033.stdout: "504787be": { 2024-05-15T14:03:43.205 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 10, 2024-05-15T14:03:43.205 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-05-15T14:03:43.205 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:03:43.205 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-05-15T14:03:43.205 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:03:43.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:03:43.206 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 90.0, 2024-05-15T14:03:43.206 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 130.0, 2024-05-15T14:03:43.206 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-05-15T14:03:43.206 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-05-15T14:03:43.206 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-05-15T14:03:43.206 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:03:43.206 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-05-15T14:03:43.206 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:03:43.206 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:03:43.206 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:03:43.206 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:03:43.206 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:03:43.207 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:03:43.207 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T14:03:43.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.200+0000 7f34033d6700 1 -- 172.21.15.33:0/1965155910 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f33e80b2aa0 msgr2=0x7f33e80b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:43.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.200+0000 7f34033d6700 1 --2- 172.21.15.33:0/1965155910 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f33e80b2aa0 0x7f33e80b4f60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f33ec00b2e0 tx=0x7f33ec005960 comp rx=0 tx=0).stop 2024-05-15T14:03:43.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.200+0000 7f34033d6700 1 -- 172.21.15.33:0/1965155910 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33fc103690 msgr2=0x7f33fc19f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:43.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.200+0000 7f34033d6700 1 --2- 172.21.15.33:0/1965155910 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33fc103690 0x7f33fc19f7b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f33f800e970 tx=0x7f33f800ec80 comp rx=0 tx=0).stop 2024-05-15T14:03:43.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.200+0000 7f34033d6700 1 -- 172.21.15.33:0/1965155910 shutdown_connections 2024-05-15T14:03:43.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.200+0000 7f34033d6700 1 --2- 172.21.15.33:0/1965155910 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f33e80b2aa0 0x7f33e80b4f60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:43.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.200+0000 7f34033d6700 1 --2- 172.21.15.33:0/1965155910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33fc1084e0 0x7f33fc199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:43.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.200+0000 7f34033d6700 1 --2- 172.21.15.33:0/1965155910 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33fc103690 0x7f33fc19f7b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:43.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.200+0000 7f34033d6700 1 --2- 172.21.15.33:0/1965155910 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33fc104030 0x7f33fc19fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:43.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.200+0000 7f34033d6700 1 -- 172.21.15.33:0/1965155910 >> 172.21.15.33:0/1965155910 conn(0x7f33fc07a730 msgr2=0x7f33fc101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:43.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.200+0000 7f34033d6700 1 -- 172.21.15.33:0/1965155910 shutdown_connections 2024-05-15T14:03:43.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:43.200+0000 7f34033d6700 1 -- 172.21.15.33:0/1965155910 wait complete. 2024-05-15T14:03:43.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:43 smithi033 ceph-mon[31650]: osdmap e84: 12 total, 12 up, 12 in 2024-05-15T14:03:43.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:43 smithi033 ceph-mon[31650]: mds.? [v2:172.21.15.81:6834/727032511,v1:172.21.15.81:6835/727032511] up:boot 2024-05-15T14:03:43.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:43 smithi033 ceph-mon[31650]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 2 up:standby-replay 7 up:standby 2024-05-15T14:03:43.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:43 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-15T14:03:43.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:43 smithi033 ceph-mon[31650]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:03:43.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:43 smithi052 ceph-mon[36190]: osdmap e84: 12 total, 12 up, 12 in 2024-05-15T14:03:43.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:43 smithi052 ceph-mon[36190]: mds.? [v2:172.21.15.81:6834/727032511,v1:172.21.15.81:6835/727032511] up:boot 2024-05-15T14:03:43.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:43 smithi052 ceph-mon[36190]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 2 up:standby-replay 7 up:standby 2024-05-15T14:03:43.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:43 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-15T14:03:43.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:43 smithi052 ceph-mon[36190]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:03:43.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:43 smithi081 ceph-mon[36434]: osdmap e84: 12 total, 12 up, 12 in 2024-05-15T14:03:43.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:43 smithi081 ceph-mon[36434]: mds.? [v2:172.21.15.81:6834/727032511,v1:172.21.15.81:6835/727032511] up:boot 2024-05-15T14:03:43.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:43 smithi081 ceph-mon[36434]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 2 up:standby-replay 7 up:standby 2024-05-15T14:03:43.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:43 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-15T14:03:43.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:43 smithi081 ceph-mon[36434]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:03:43.988 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '504787be', quiesce took 6.4 seconds. Will release after: 3.5999999999999996 2024-05-15T14:03:44.050 INFO:tasks.workunit.client.0.smithi033.stdout: 1 167286 11.64 MB/sec warmup 79 sec latency 872.487 ms 2024-05-15T14:03:44.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:44 smithi033 ceph-mon[31650]: pgmap v105: 97 pgs: 97 active+clean; 50 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 361 op/s 2024-05-15T14:03:44.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:44 smithi052 ceph-mon[36190]: pgmap v105: 97 pgs: 97 active+clean; 50 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 361 op/s 2024-05-15T14:03:44.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:44 smithi081 ceph-mon[36434]: pgmap v105: 97 pgs: 97 active+clean; 50 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 361 op/s 2024-05-15T14:03:45.050 INFO:tasks.workunit.client.0.smithi033.stdout: 1 167286 11.50 MB/sec warmup 80 sec latency 1872.614 ms 2024-05-15T14:03:46.050 INFO:tasks.workunit.client.0.smithi033.stdout: 1 167286 11.35 MB/sec warmup 81 sec latency 2872.660 ms 2024-05-15T14:03:46.642 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:46 smithi052 ceph-mon[36190]: pgmap v106: 97 pgs: 97 active+clean; 50 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 361 op/s 2024-05-15T14:03:46.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:46 smithi033 ceph-mon[31650]: pgmap v106: 97 pgs: 97 active+clean; 50 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 361 op/s 2024-05-15T14:03:46.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:46 smithi081 ceph-mon[36434]: pgmap v106: 97 pgs: 97 active+clean; 50 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 361 op/s 2024-05-15T14:03:47.050 INFO:tasks.workunit.client.0.smithi033.stdout: 1 167286 11.22 MB/sec warmup 82 sec latency 3872.757 ms 2024-05-15T14:03:47.589 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T14:03:47.855 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:03:48.050 INFO:tasks.workunit.client.0.smithi033.stdout: 1 167286 11.08 MB/sec warmup 83 sec latency 4872.852 ms 2024-05-15T14:03:48.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:48 smithi033 ceph-mon[31650]: pgmap v107: 97 pgs: 97 active+clean; 59 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 249 op/s 2024-05-15T14:03:48.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:48 smithi052 ceph-mon[36190]: pgmap v107: 97 pgs: 97 active+clean; 59 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 249 op/s 2024-05-15T14:03:48.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:48 smithi081 ceph-mon[36434]: pgmap v107: 97 pgs: 97 active+clean; 59 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 249 op/s 2024-05-15T14:03:49.051 INFO:tasks.workunit.client.0.smithi033.stdout: 1 167286 10.95 MB/sec warmup 84 sec latency 5872.951 ms 2024-05-15T14:03:49.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.599+0000 7fcb431a3700 1 -- 172.21.15.33:0/3964893591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb3c102e20 msgr2=0x7fcb3c10f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:49.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.599+0000 7fcb431a3700 1 --2- 172.21.15.33:0/3964893591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb3c102e20 0x7fcb3c10f330 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fcb3400b0a0 tx=0x7fcb3400b3b0 comp rx=0 tx=0).stop 2024-05-15T14:03:49.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.599+0000 7fcb431a3700 1 -- 172.21.15.33:0/3964893591 shutdown_connections 2024-05-15T14:03:49.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.599+0000 7fcb431a3700 1 --2- 172.21.15.33:0/3964893591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb3c102e20 0x7fcb3c10f330 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:49.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.599+0000 7fcb431a3700 1 --2- 172.21.15.33:0/3964893591 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb3c1084f0 0x7fcb3c1088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:49.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.599+0000 7fcb431a3700 1 --2- 172.21.15.33:0/3964893591 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb3c102460 0x7fcb3c1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:49.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.599+0000 7fcb431a3700 1 -- 172.21.15.33:0/3964893591 >> 172.21.15.33:0/3964893591 conn(0x7fcb3c0794e0 msgr2=0x7fcb3c0798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:49.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.599+0000 7fcb431a3700 1 -- 172.21.15.33:0/3964893591 shutdown_connections 2024-05-15T14:03:49.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.599+0000 7fcb431a3700 1 -- 172.21.15.33:0/3964893591 wait complete. 2024-05-15T14:03:49.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.600+0000 7fcb431a3700 1 Processor -- start 2024-05-15T14:03:49.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.600+0000 7fcb431a3700 1 -- start start 2024-05-15T14:03:49.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.600+0000 7fcb431a3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb3c102460 0x7fcb3c0ff490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:49.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.600+0000 7fcb431a3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb3c102e20 0x7fcb3c0ff9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:49.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.600+0000 7fcb431a3700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb3c1084f0 0x7fcb3c0fff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:49.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.600+0000 7fcb431a3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c112600 con 0x7fcb3c102e20 2024-05-15T14:03:49.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.600+0000 7fcb431a3700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c112480 con 0x7fcb3c102460 2024-05-15T14:03:49.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.600+0000 7fcb431a3700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fcb3c112780 con 0x7fcb3c1084f0 2024-05-15T14:03:49.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.600+0000 7fcb3bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb3c102e20 0x7fcb3c0ff9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:49.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.600+0000 7fcb3bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb3c102e20 0x7fcb3c0ff9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48550/0 (socket says 172.21.15.33:48550) 2024-05-15T14:03:49.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.600+0000 7fcb3bfff700 1 -- 172.21.15.33:0/444297488 learned_addr learned my addr 172.21.15.33:0/444297488 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:49.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.601+0000 7fcb41740700 1 --2- 172.21.15.33:0/444297488 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb3c1084f0 0x7fcb3c0fff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:49.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.601+0000 7fcb40f3f700 1 --2- 172.21.15.33:0/444297488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb3c102460 0x7fcb3c0ff490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:49.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.601+0000 7fcb3bfff700 1 -- 172.21.15.33:0/444297488 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb3c1084f0 msgr2=0x7fcb3c0fff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:49.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.601+0000 7fcb3bfff700 1 --2- 172.21.15.33:0/444297488 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb3c1084f0 0x7fcb3c0fff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:49.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.601+0000 7fcb3bfff700 1 -- 172.21.15.33:0/444297488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb3c102460 msgr2=0x7fcb3c0ff490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:49.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.601+0000 7fcb3bfff700 1 --2- 172.21.15.33:0/444297488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb3c102460 0x7fcb3c0ff490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:49.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.601+0000 7fcb3bfff700 1 -- 172.21.15.33:0/444297488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb3c1a8150 con 0x7fcb3c102e20 2024-05-15T14:03:49.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.601+0000 7fcb40f3f700 1 --2- 172.21.15.33:0/444297488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb3c102460 0x7fcb3c0ff490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:03:49.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.601+0000 7fcb41740700 1 --2- 172.21.15.33:0/444297488 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb3c1084f0 0x7fcb3c0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:03:49.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.601+0000 7fcb3bfff700 1 --2- 172.21.15.33:0/444297488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb3c102e20 0x7fcb3c0ff9d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fcb3000d980 tx=0x7fcb3000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:49.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.601+0000 7fcb39ffb700 1 -- 172.21.15.33:0/444297488 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb30016070 con 0x7fcb3c102e20 2024-05-15T14:03:49.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.601+0000 7fcb39ffb700 1 -- 172.21.15.33:0/444297488 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb300044d0 con 0x7fcb3c102e20 2024-05-15T14:03:49.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.601+0000 7fcb431a3700 1 -- 172.21.15.33:0/444297488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb3c1a8440 con 0x7fcb3c102e20 2024-05-15T14:03:49.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.601+0000 7fcb39ffb700 1 -- 172.21.15.33:0/444297488 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb30004d00 con 0x7fcb3c102e20 2024-05-15T14:03:49.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.601+0000 7fcb431a3700 1 -- 172.21.15.33:0/444297488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb3c1a8980 con 0x7fcb3c102e20 2024-05-15T14:03:49.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.602+0000 7fcb431a3700 1 -- 172.21.15.33:0/444297488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb3c06a230 con 0x7fcb3c102e20 2024-05-15T14:03:49.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.604+0000 7fcb39ffb700 1 -- 172.21.15.33:0/444297488 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fcb30004020 con 0x7fcb3c102e20 2024-05-15T14:03:49.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.604+0000 7fcb39ffb700 1 --2- 172.21.15.33:0/444297488 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fcb240b29f0 0x7fcb240b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:49.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.604+0000 7fcb39ffb700 1 -- 172.21.15.33:0/444297488 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7fcb300e24e0 con 0x7fcb3c102e20 2024-05-15T14:03:49.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.604+0000 7fcb40f3f700 1 --2- 172.21.15.33:0/444297488 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fcb240b29f0 0x7fcb240b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:49.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.605+0000 7fcb40f3f700 1 --2- 172.21.15.33:0/444297488 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fcb240b29f0 0x7fcb240b4eb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c00baa0 tx=0x7fcb2c005db0 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:49.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.609+0000 7fcb39ffb700 1 -- 172.21.15.33:0/444297488 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcb3001f020 con 0x7fcb3c102e20 2024-05-15T14:03:49.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.884+0000 7fcb431a3700 1 -- 172.21.15.33:0/444297488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb3c02e560 con 0x7fcb3c102e20 2024-05-15T14:03:49.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.884+0000 7fcb39ffb700 1 -- 172.21.15.33:0/444297488 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11858 (secure 0 0 0) 0x7fcb300a9ed0 con 0x7fcb3c102e20 2024-05-15T14:03:49.887 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T14:03:49.887 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2030117633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2030117633},{"type":"v1","addr":"172.21.15.33:6839","nonce":2030117633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1918979061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1918979061},{"type":"v1","addr":"172.21.15.33:6841","nonce":1918979061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T13:57:19.056417+0000","modified":"2024-05-15T14:03:42.376493+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24463,"mds_2":24461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/1629697841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1629697841},{"type":"v1","addr":"172.21.15.52:6837","nonce":1629697841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.52:6839/2696044471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2696044471},{"type":"v1","addr":"172.21.15.52:6839","nonce":2696044471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.81:6839/1554395940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":1554395940},{"type":"v1","addr":"172.21.15.81:6839","nonce":1554395940}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25393":{"gid":25393,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/727032511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":727032511},{"type":"v1","addr":"172.21.15.81:6835","nonce":727032511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24463,24478,24461]},"id":1}]} 2024-05-15T14:03:49.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.887+0000 7fcb431a3700 1 -- 172.21.15.33:0/444297488 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fcb240b29f0 msgr2=0x7fcb240b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:49.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.887+0000 7fcb431a3700 1 --2- 172.21.15.33:0/444297488 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fcb240b29f0 0x7fcb240b4eb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c00baa0 tx=0x7fcb2c005db0 comp rx=0 tx=0).stop 2024-05-15T14:03:49.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.887+0000 7fcb431a3700 1 -- 172.21.15.33:0/444297488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb3c102e20 msgr2=0x7fcb3c0ff9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:49.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.887+0000 7fcb431a3700 1 --2- 172.21.15.33:0/444297488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb3c102e20 0x7fcb3c0ff9d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fcb3000d980 tx=0x7fcb3000dd40 comp rx=0 tx=0).stop 2024-05-15T14:03:49.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.887+0000 7fcb431a3700 1 -- 172.21.15.33:0/444297488 shutdown_connections 2024-05-15T14:03:49.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.887+0000 7fcb431a3700 1 --2- 172.21.15.33:0/444297488 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fcb240b29f0 0x7fcb240b4eb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:49.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.887+0000 7fcb431a3700 1 --2- 172.21.15.33:0/444297488 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb3c1084f0 0x7fcb3c0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:49.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.887+0000 7fcb431a3700 1 --2- 172.21.15.33:0/444297488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcb3c102460 0x7fcb3c0ff490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:49.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.887+0000 7fcb431a3700 1 --2- 172.21.15.33:0/444297488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb3c102e20 0x7fcb3c0ff9d0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:49.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.887+0000 7fcb431a3700 1 -- 172.21.15.33:0/444297488 >> 172.21.15.33:0/444297488 conn(0x7fcb3c0794e0 msgr2=0x7fcb3c06c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:49.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.887+0000 7fcb431a3700 1 -- 172.21.15.33:0/444297488 shutdown_connections 2024-05-15T14:03:49.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:49.887+0000 7fcb431a3700 1 -- 172.21.15.33:0/444297488 wait complete. 2024-05-15T14:03:49.891 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-05-15T14:03:50.051 INFO:tasks.workunit.client.0.smithi033.stdout: 1 167286 10.82 MB/sec warmup 85 sec latency 6873.040 ms 2024-05-15T14:03:50.658 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:50 smithi081 ceph-mon[36434]: pgmap v108: 97 pgs: 97 active+clean; 59 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 249 op/s 2024-05-15T14:03:50.659 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:50 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/444297488' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:03:50.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:50 smithi033 ceph-mon[31650]: pgmap v108: 97 pgs: 97 active+clean; 59 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 249 op/s 2024-05-15T14:03:50.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:50 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/444297488' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:03:50.772 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24478 quiesce db --set-id 504787be --release --await' 2024-05-15T14:03:50.773 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24478 quiesce db --set-id 504787be --release --await 2024-05-15T14:03:50.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:50 smithi052 ceph-mon[36190]: pgmap v108: 97 pgs: 97 active+clean; 59 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 249 op/s 2024-05-15T14:03:50.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:50 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/444297488' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:03:51.030 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:03:51.051 INFO:tasks.workunit.client.0.smithi033.stdout: 1 167286 10.69 MB/sec warmup 86 sec latency 7873.080 ms 2024-05-15T14:03:52.051 INFO:tasks.workunit.client.0.smithi033.stdout: 1 167286 10.57 MB/sec warmup 87 sec latency 8873.156 ms 2024-05-15T14:03:52.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:52 smithi033 ceph-mon[31650]: pgmap v109: 97 pgs: 97 active+clean; 59 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 249 op/s 2024-05-15T14:03:52.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.733+0000 7fc4f84ab700 1 -- 172.21.15.33:0/164173770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4f01041a0 msgr2=0x7fc4f01081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:52.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.733+0000 7fc4f84ab700 1 --2- 172.21.15.33:0/164173770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4f01041a0 0x7fc4f01081c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc4e00099c0 tx=0x7fc4e0009cd0 comp rx=0 tx=0).stop 2024-05-15T14:03:52.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.734+0000 7fc4f84ab700 1 -- 172.21.15.33:0/164173770 shutdown_connections 2024-05-15T14:03:52.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.734+0000 7fc4f84ab700 1 --2- 172.21.15.33:0/164173770 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc4f01088f0 0x7fc4f0110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:52.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.734+0000 7fc4f84ab700 1 --2- 172.21.15.33:0/164173770 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4f01037f0 0x7fc4f0103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:52.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.734+0000 7fc4f84ab700 1 --2- 172.21.15.33:0/164173770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4f01041a0 0x7fc4f01081c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:52.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.734+0000 7fc4f84ab700 1 -- 172.21.15.33:0/164173770 >> 172.21.15.33:0/164173770 conn(0x7fc4f007a6a0 msgr2=0x7fc4f007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:52.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.734+0000 7fc4f84ab700 1 -- 172.21.15.33:0/164173770 shutdown_connections 2024-05-15T14:03:52.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.734+0000 7fc4f84ab700 1 -- 172.21.15.33:0/164173770 wait complete. 2024-05-15T14:03:52.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.734+0000 7fc4f84ab700 1 Processor -- start 2024-05-15T14:03:52.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.734+0000 7fc4f84ab700 1 -- start start 2024-05-15T14:03:52.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.735+0000 7fc4f84ab700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4f01037f0 0x7fc4f019f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:52.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.735+0000 7fc4f84ab700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4f01041a0 0x7fc4f019fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:52.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.735+0000 7fc4f84ab700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc4f01088f0 0x7fc4f0199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:52.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.735+0000 7fc4f84ab700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f01137b0 con 0x7fc4f01037f0 2024-05-15T14:03:52.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.735+0000 7fc4f84ab700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f0113630 con 0x7fc4f01041a0 2024-05-15T14:03:52.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.735+0000 7fc4f84ab700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f0113930 con 0x7fc4f01088f0 2024-05-15T14:03:52.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.735+0000 7fc4f6247700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4f01037f0 0x7fc4f019f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:52.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.735+0000 7fc4f6247700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4f01037f0 0x7fc4f019f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56900/0 (socket says 172.21.15.33:56900) 2024-05-15T14:03:52.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.735+0000 7fc4f6247700 1 -- 172.21.15.33:0/3910073941 learned_addr learned my addr 172.21.15.33:0/3910073941 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:52.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.735+0000 7fc4f5a46700 1 --2- 172.21.15.33:0/3910073941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4f01041a0 0x7fc4f019fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:52.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.735+0000 7fc4f6247700 1 -- 172.21.15.33:0/3910073941 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc4f01088f0 msgr2=0x7fc4f0199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:52.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.735+0000 7fc4f6a48700 1 --2- 172.21.15.33:0/3910073941 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc4f01088f0 0x7fc4f0199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:52.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.735+0000 7fc4f6247700 1 --2- 172.21.15.33:0/3910073941 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc4f01088f0 0x7fc4f0199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:52.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.736+0000 7fc4f6247700 1 -- 172.21.15.33:0/3910073941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4f01041a0 msgr2=0x7fc4f019fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:52.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.736+0000 7fc4f6247700 1 --2- 172.21.15.33:0/3910073941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4f01041a0 0x7fc4f019fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:52.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.736+0000 7fc4f6247700 1 -- 172.21.15.33:0/3910073941 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f019a1f0 con 0x7fc4f01037f0 2024-05-15T14:03:52.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.736+0000 7fc4f6a48700 1 --2- 172.21.15.33:0/3910073941 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc4f01088f0 0x7fc4f0199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:03:52.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.736+0000 7fc4f6247700 1 --2- 172.21.15.33:0/3910073941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4f01037f0 0x7fc4f019f8d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec00b530 tx=0x7fc4ec00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:52.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.736+0000 7fc4e77fe700 1 -- 172.21.15.33:0/3910073941 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec015020 con 0x7fc4f01037f0 2024-05-15T14:03:52.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.736+0000 7fc4f84ab700 1 -- 172.21.15.33:0/3910073941 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f019a4e0 con 0x7fc4f01037f0 2024-05-15T14:03:52.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.736+0000 7fc4e77fe700 1 -- 172.21.15.33:0/3910073941 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4ec011ce0 con 0x7fc4f01037f0 2024-05-15T14:03:52.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.736+0000 7fc4e77fe700 1 -- 172.21.15.33:0/3910073941 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec010e40 con 0x7fc4f01037f0 2024-05-15T14:03:52.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.736+0000 7fc4f84ab700 1 -- 172.21.15.33:0/3910073941 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f01a67a0 con 0x7fc4f01037f0 2024-05-15T14:03:52.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.739+0000 7fc4e77fe700 1 -- 172.21.15.33:0/3910073941 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc4ec018020 con 0x7fc4f01037f0 2024-05-15T14:03:52.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.739+0000 7fc4e77fe700 1 --2- 172.21.15.33:0/3910073941 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc4dc0b29f0 0x7fc4dc0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:52.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.739+0000 7fc4f5a46700 1 --2- 172.21.15.33:0/3910073941 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc4dc0b29f0 0x7fc4dc0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:52.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.739+0000 7fc4e77fe700 1 -- 172.21.15.33:0/3910073941 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7fc4ec0e1d30 con 0x7fc4f01037f0 2024-05-15T14:03:52.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.740+0000 7fc502d41b80 1 Processor -- start 2024-05-15T14:03:52.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.740+0000 7fc4f5a46700 1 --2- 172.21.15.33:0/3910073941 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc4dc0b29f0 0x7fc4dc0b4eb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0009990 tx=0x7fc4e00095f0 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:52.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.740+0000 7fc502d41b80 1 -- start start 2024-05-15T14:03:52.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.740+0000 7fc502d41b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56462d660420 0x56462d660800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:52.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.740+0000 7fc502d41b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56462d660d40 0x56462d65bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:52.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.740+0000 7fc502d41b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56462d65c2c0 0x56462d65c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:52.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.740+0000 7fc502d41b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56462d5b9e70 con 0x56462d660d40 2024-05-15T14:03:52.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.740+0000 7fc502d41b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56462d5a9a10 con 0x56462d65c2c0 2024-05-15T14:03:52.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.740+0000 7fc502d41b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56462d5ada50 con 0x56462d660420 2024-05-15T14:03:52.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.740+0000 7fc4f5a46700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56462d660d40 0x56462d65bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:52.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.741+0000 7fc4f5a46700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56462d660d40 0x56462d65bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56904/0 (socket says 172.21.15.33:56904) 2024-05-15T14:03:52.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.741+0000 7fc4f5a46700 1 -- 172.21.15.33:0/719220753 learned_addr learned my addr 172.21.15.33:0/719220753 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:52.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.741+0000 7fc4f6a48700 1 --2- 172.21.15.33:0/719220753 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56462d65c2c0 0x56462d65c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:52.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.741+0000 7fc4f6247700 1 --2- 172.21.15.33:0/719220753 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56462d660420 0x56462d660800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:52.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.741+0000 7fc4f5a46700 1 -- 172.21.15.33:0/719220753 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56462d660420 msgr2=0x56462d660800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:52.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.741+0000 7fc4f5a46700 1 --2- 172.21.15.33:0/719220753 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56462d660420 0x56462d660800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:52.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.741+0000 7fc4f5a46700 1 -- 172.21.15.33:0/719220753 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56462d65c2c0 msgr2=0x56462d65c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:52.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.741+0000 7fc4f5a46700 1 --2- 172.21.15.33:0/719220753 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56462d65c2c0 0x56462d65c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:52.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.741+0000 7fc4f5a46700 1 -- 172.21.15.33:0/719220753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56462d5530b0 con 0x56462d660d40 2024-05-15T14:03:52.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.741+0000 7fc4f6a48700 1 --2- 172.21.15.33:0/719220753 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56462d65c2c0 0x56462d65c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:03:52.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.741+0000 7fc4f5a46700 1 --2- 172.21.15.33:0/719220753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56462d660d40 0x56462d65bca0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0015000 tx=0x7fc4e001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73baf951a1a28503 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:52.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.741+0000 7fc4e4d70700 1 -- 172.21.15.33:0/719220753 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0035070 con 0x56462d660d40 2024-05-15T14:03:52.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.741+0000 7fc4e4d70700 1 -- 172.21.15.33:0/719220753 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4e00259c0 con 0x56462d660d40 2024-05-15T14:03:52.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.741+0000 7fc4e4d70700 1 -- 172.21.15.33:0/719220753 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e000fb30 con 0x56462d660d40 2024-05-15T14:03:52.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.742+0000 7fc502d41b80 1 -- 172.21.15.33:0/719220753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56462d660d40 msgr2=0x56462d65bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:52.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.742+0000 7fc502d41b80 1 --2- 172.21.15.33:0/719220753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56462d660d40 0x56462d65bca0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0015000 tx=0x7fc4e001f000 comp rx=0 tx=0).stop 2024-05-15T14:03:52.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.742+0000 7fc502d41b80 1 -- 172.21.15.33:0/719220753 shutdown_connections 2024-05-15T14:03:52.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.742+0000 7fc502d41b80 1 --2- 172.21.15.33:0/719220753 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56462d65c2c0 0x56462d65c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:52.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.742+0000 7fc502d41b80 1 --2- 172.21.15.33:0/719220753 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56462d660420 0x56462d660800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:52.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.742+0000 7fc502d41b80 1 --2- 172.21.15.33:0/719220753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56462d660d40 0x56462d65bca0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:52.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.742+0000 7fc502d41b80 1 -- 172.21.15.33:0/719220753 >> 172.21.15.33:0/719220753 conn(0x56462d588eb0 msgr2=0x56462d6564a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:52.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.742+0000 7fc502d41b80 1 -- 172.21.15.33:0/719220753 shutdown_connections 2024-05-15T14:03:52.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.742+0000 7fc502d41b80 1 -- 172.21.15.33:0/719220753 wait complete. 2024-05-15T14:03:52.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.743+0000 7fc502d41b80 1 -- start start 2024-05-15T14:03:52.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.743+0000 7fc502d41b80 1 Processor -- start 2024-05-15T14:03:52.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.743+0000 7fc502d41b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56462d65c2c0 0x56462d667b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:52.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.743+0000 7fc502d41b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56462d660420 0x56462d664ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:52.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.743+0000 7fc502d41b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56462d660d40 0x56462d6653e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:52.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.743+0000 7fc502d41b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56462d4b00a0 con 0x56462d660420 2024-05-15T14:03:52.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.743+0000 7fc502d41b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56462d511ea0 con 0x56462d660d40 2024-05-15T14:03:52.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.743+0000 7fc502d41b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56462d668820 con 0x56462d65c2c0 2024-05-15T14:03:52.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.743+0000 7fc502d41b80 10 client.0 _ll_register_callbacks cb 0x56462d544a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:03:52.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.743+0000 7fc4f5a46700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56462d660420 0x56462d664ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:52.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.743+0000 7fc4f5a46700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56462d660420 0x56462d664ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56920/0 (socket says 172.21.15.33:56920) 2024-05-15T14:03:52.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.743+0000 7fc4f5a46700 1 -- 172.21.15.33:0/284633670 learned_addr learned my addr 172.21.15.33:0/284633670 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:52.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.743+0000 7fc4f6247700 1 --2- 172.21.15.33:0/284633670 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56462d65c2c0 0x56462d667b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:52.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.743+0000 7fc4f6a48700 1 --2- 172.21.15.33:0/284633670 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56462d660d40 0x56462d6653e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:52.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.743+0000 7fc4f5a46700 1 -- 172.21.15.33:0/284633670 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56462d65c2c0 msgr2=0x56462d667b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:52.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.743+0000 7fc4f5a46700 1 --2- 172.21.15.33:0/284633670 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56462d65c2c0 0x56462d667b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:52.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.743+0000 7fc4f5a46700 1 -- 172.21.15.33:0/284633670 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56462d660d40 msgr2=0x56462d6653e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:52.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.743+0000 7fc4f5a46700 1 --2- 172.21.15.33:0/284633670 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56462d660d40 0x56462d6653e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:52.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.743+0000 7fc4f5a46700 1 -- 172.21.15.33:0/284633670 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56462d4f99d0 con 0x56462d660420 2024-05-15T14:03:52.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.743+0000 7fc4f6a48700 1 --2- 172.21.15.33:0/284633670 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56462d660d40 0x56462d6653e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:03:52.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.744+0000 7fc4f5a46700 1 --2- 172.21.15.33:0/284633670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56462d660420 0x56462d664ea0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0006010 tx=0x7fc4e0025f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1e6d809cd620d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:52.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.744+0000 7fc4d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:03:52.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.744+0000 7fc4d9ffb700 1 -- 172.21.15.33:0/284633670 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0035040 con 0x56462d660420 2024-05-15T14:03:52.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.744+0000 7fc4d9ffb700 1 -- 172.21.15.33:0/284633670 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7fc4e0025510 con 0x56462d660420 2024-05-15T14:03:52.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.744+0000 7fc502d41b80 1 -- 172.21.15.33:0/284633670 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4e00259c0 con 0x56462d660420 2024-05-15T14:03:52.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.744+0000 7fc4d9ffb700 1 -- 172.21.15.33:0/284633670 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc4e0048050 con 0x56462d660420 2024-05-15T14:03:52.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.744+0000 7fc502d41b80 10 client.15747 fetch_fsmap learned FSMap version 27 2024-05-15T14:03:52.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.744+0000 7fc502d41b80 1 -- 172.21.15.33:0/284633670 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56462d563860 con 0x56462d660420 2024-05-15T14:03:52.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.744+0000 7fc4d9ffb700 1 -- 172.21.15.33:0/284633670 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4e00259c0 con 0x56462d660420 2024-05-15T14:03:52.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.744+0000 7fc4d9ffb700 1 -- 172.21.15.33:0/284633670 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc4e0043030 con 0x56462d660420 2024-05-15T14:03:52.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.744+0000 7fc502d41b80 10 client.15747 fetch_fsmap finished waiting for FSMap version 27 2024-05-15T14:03:52.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.744+0000 7fc502d41b80 10 client.15747 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:03:52.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.745+0000 7fc502d41b80 20 client.15747 populate_metadata read hostname 'smithi033' 2024-05-15T14:03:52.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.745+0000 7fc502d41b80 1 --2- 172.21.15.33:0/284633670 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56462d675210 0x56462d677600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:52.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.745+0000 7fc502d41b80 4 client.15747 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:03:52.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.745+0000 7fc502d41b80 1 -- 172.21.15.33:0/284633670 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56462d5bbdc0 con 0x56462d675210 2024-05-15T14:03:52.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.745+0000 7fc4f6247700 1 --2- 172.21.15.33:0/284633670 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56462d675210 0x56462d677600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:52.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.745+0000 7fc4f6247700 1 --2- 172.21.15.33:0/284633670 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56462d675210 0x56462d677600 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=367541e5be9e79a5 server_cookie=62728a6411eeda22 in_seq=0 out_seq=0 2024-05-15T14:03:52.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.745+0000 7fc4d9ffb700 10 client.15747 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:03:52.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.746+0000 7fc4d9ffb700 1 -- 172.21.15.33:0/284633670 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56462d5bbdc0 con 0x56462d675210 2024-05-15T14:03:52.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.746+0000 7fc4d9ffb700 10 client.15747 handle_command_reply: tid=0 2024-05-15T14:03:52.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.746+0000 7fc502d41b80 1 client.15747 shutdown 2024-05-15T14:03:52.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.746+0000 7fc502d41b80 1 -- 172.21.15.33:0/284633670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56462d660420 msgr2=0x56462d664ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:52.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.746+0000 7fc502d41b80 1 --2- 172.21.15.33:0/284633670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56462d660420 0x56462d664ea0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0006010 tx=0x7fc4e0025f50 comp rx=0 tx=0).stop 2024-05-15T14:03:52.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.746+0000 7fc502d41b80 1 -- 172.21.15.33:0/284633670 shutdown_connections 2024-05-15T14:03:52.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.746+0000 7fc502d41b80 1 --2- 172.21.15.33:0/284633670 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56462d660d40 0x56462d6653e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:52.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.746+0000 7fc502d41b80 1 --2- 172.21.15.33:0/284633670 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56462d65c2c0 0x56462d667b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:52.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.746+0000 7fc502d41b80 1 --2- 172.21.15.33:0/284633670 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56462d675210 0x56462d677600 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:52.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.746+0000 7fc502d41b80 1 --2- 172.21.15.33:0/284633670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56462d660420 0x56462d664ea0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:52.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.746+0000 7fc502d41b80 1 -- 172.21.15.33:0/284633670 >> 172.21.15.33:0/284633670 conn(0x56462d588eb0 msgr2=0x56462d650fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:52.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.746+0000 7fc4d9ffb700 0 client.15747 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:03:52.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.746+0000 7fc502d41b80 1 -- 172.21.15.33:0/284633670 shutdown_connections 2024-05-15T14:03:52.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.746+0000 7fc502d41b80 1 -- 172.21.15.33:0/284633670 wait complete. 2024-05-15T14:03:52.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.747+0000 7fc502d41b80 20 client.15747 trim_cache size 0 max 16384 2024-05-15T14:03:52.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.767+0000 7fc502d41b80 1 Processor -- start 2024-05-15T14:03:52.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.767+0000 7fc502d41b80 1 -- start start 2024-05-15T14:03:52.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.767+0000 7fc502d41b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56462d65c2c0 0x56462d65b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:52.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.767+0000 7fc502d41b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56462d660420 0x56462d6510d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:52.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.767+0000 7fc502d41b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56462d660d40 0x56462d65f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:52.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.767+0000 7fc502d41b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f01b0680 con 0x56462d660420 2024-05-15T14:03:52.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.767+0000 7fc502d41b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f01adeb0 con 0x56462d660d40 2024-05-15T14:03:52.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.767+0000 7fc502d41b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f01abd00 con 0x56462d65c2c0 2024-05-15T14:03:52.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.767+0000 7fc4f5a46700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56462d660420 0x56462d6510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:52.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.767+0000 7fc4f5a46700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56462d660420 0x56462d6510d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56924/0 (socket says 172.21.15.33:56924) 2024-05-15T14:03:52.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.767+0000 7fc4f5a46700 1 -- 172.21.15.33:0/370010241 learned_addr learned my addr 172.21.15.33:0/370010241 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:52.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.767+0000 7fc4f6247700 1 --2- 172.21.15.33:0/370010241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56462d65c2c0 0x56462d65b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:52.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.767+0000 7fc4f6a48700 1 --2- 172.21.15.33:0/370010241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56462d660d40 0x56462d65f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:52.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.767+0000 7fc4f5a46700 1 -- 172.21.15.33:0/370010241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56462d65c2c0 msgr2=0x56462d65b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:52.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.767+0000 7fc4f5a46700 1 --2- 172.21.15.33:0/370010241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56462d65c2c0 0x56462d65b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:52.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.767+0000 7fc4f5a46700 1 -- 172.21.15.33:0/370010241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56462d660d40 msgr2=0x56462d65f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:52.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.767+0000 7fc4f5a46700 1 --2- 172.21.15.33:0/370010241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56462d660d40 0x56462d65f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:52.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.767+0000 7fc4f5a46700 1 -- 172.21.15.33:0/370010241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56462d6674b0 con 0x56462d660420 2024-05-15T14:03:52.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.767+0000 7fc4f6a48700 1 --2- 172.21.15.33:0/370010241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56462d660d40 0x56462d65f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:03:52.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.768+0000 7fc4f5a46700 1 --2- 172.21.15.33:0/370010241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56462d660420 0x56462d6510d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc4e003b890 tx=0x7fc4e0005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5fd9c374242045a server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:52.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.768+0000 7fc4dbfff700 1 -- 172.21.15.33:0/370010241 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0035070 con 0x56462d660420 2024-05-15T14:03:52.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.768+0000 7fc4dbfff700 1 -- 172.21.15.33:0/370010241 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4e000fc30 con 0x56462d660420 2024-05-15T14:03:52.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.768+0000 7fc4dbfff700 1 -- 172.21.15.33:0/370010241 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0025240 con 0x56462d660420 2024-05-15T14:03:52.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.768+0000 7fc502d41b80 1 -- 172.21.15.33:0/370010241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56462d660420 msgr2=0x56462d6510d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:52.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.768+0000 7fc502d41b80 1 --2- 172.21.15.33:0/370010241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56462d660420 0x56462d6510d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc4e003b890 tx=0x7fc4e0005ea0 comp rx=0 tx=0).stop 2024-05-15T14:03:52.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.768+0000 7fc502d41b80 1 -- 172.21.15.33:0/370010241 shutdown_connections 2024-05-15T14:03:52.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.768+0000 7fc502d41b80 1 --2- 172.21.15.33:0/370010241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56462d660d40 0x56462d65f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:52.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.768+0000 7fc502d41b80 1 --2- 172.21.15.33:0/370010241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56462d65c2c0 0x56462d65b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:52.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.768+0000 7fc502d41b80 1 --2- 172.21.15.33:0/370010241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56462d660420 0x56462d6510d0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:52.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.768+0000 7fc502d41b80 1 -- 172.21.15.33:0/370010241 >> 172.21.15.33:0/370010241 conn(0x56462d588eb0 msgr2=0x56462d65f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:52.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.769+0000 7fc502d41b80 1 -- 172.21.15.33:0/370010241 shutdown_connections 2024-05-15T14:03:52.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.769+0000 7fc502d41b80 1 -- 172.21.15.33:0/370010241 wait complete. 2024-05-15T14:03:52.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.769+0000 7fc502d41b80 1 -- start start 2024-05-15T14:03:52.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.769+0000 7fc502d41b80 1 Processor -- start 2024-05-15T14:03:52.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.769+0000 7fc502d41b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56462d65c2c0 0x56462d67a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:52.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.769+0000 7fc502d41b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56462d660420 0x56462d678920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:52.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.769+0000 7fc502d41b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56462d660d40 0x56462d678e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:52.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.769+0000 7fc502d41b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f01ac370 con 0x56462d660420 2024-05-15T14:03:52.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.769+0000 7fc502d41b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f01ad7e0 con 0x56462d660d40 2024-05-15T14:03:52.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.769+0000 7fc502d41b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f01aca40 con 0x56462d65c2c0 2024-05-15T14:03:52.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.769+0000 7fc502d41b80 10 client.0 _ll_register_callbacks cb 0x56462d544a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:03:52.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.769+0000 7fc4f5a46700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56462d660420 0x56462d678920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:52.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.769+0000 7fc4f5a46700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56462d660420 0x56462d678920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56926/0 (socket says 172.21.15.33:56926) 2024-05-15T14:03:52.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.769+0000 7fc4f5a46700 1 -- 172.21.15.33:0/1215087046 learned_addr learned my addr 172.21.15.33:0/1215087046 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:52.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.770+0000 7fc4f6247700 1 --2- 172.21.15.33:0/1215087046 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56462d65c2c0 0x56462d67a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:52.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.770+0000 7fc4f5a46700 1 -- 172.21.15.33:0/1215087046 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56462d65c2c0 msgr2=0x56462d67a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:52.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.770+0000 7fc4f6a48700 1 --2- 172.21.15.33:0/1215087046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56462d660d40 0x56462d678e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:52.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.770+0000 7fc4f5a46700 1 --2- 172.21.15.33:0/1215087046 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56462d65c2c0 0x56462d67a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:52.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.770+0000 7fc4f5a46700 1 -- 172.21.15.33:0/1215087046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56462d660d40 msgr2=0x56462d678e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:52.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.770+0000 7fc4f5a46700 1 --2- 172.21.15.33:0/1215087046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56462d660d40 0x56462d678e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:52.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.770+0000 7fc4f5a46700 1 -- 172.21.15.33:0/1215087046 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56462d679420 con 0x56462d660420 2024-05-15T14:03:52.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.770+0000 7fc4f5a46700 1 --2- 172.21.15.33:0/1215087046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56462d660420 0x56462d678920 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0027010 tx=0x7fc4e000fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e6a9dec79aa104 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:52.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.770+0000 7fc4d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:03:52.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.770+0000 7fc4d9ffb700 1 -- 172.21.15.33:0/1215087046 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0035070 con 0x56462d660420 2024-05-15T14:03:52.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.770+0000 7fc4d9ffb700 1 -- 172.21.15.33:0/1215087046 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7fc4e003bc90 con 0x56462d660420 2024-05-15T14:03:52.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.770+0000 7fc502d41b80 1 -- 172.21.15.33:0/1215087046 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4e000fc30 con 0x56462d660420 2024-05-15T14:03:52.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.770+0000 7fc4d9ffb700 1 -- 172.21.15.33:0/1215087046 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc4e005a050 con 0x56462d660420 2024-05-15T14:03:52.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.770+0000 7fc502d41b80 10 client.15753 fetch_fsmap learned FSMap version 27 2024-05-15T14:03:52.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.770+0000 7fc502d41b80 1 -- 172.21.15.33:0/1215087046 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56462d6e48f0 con 0x56462d660420 2024-05-15T14:03:52.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.770+0000 7fc4d9ffb700 1 -- 172.21.15.33:0/1215087046 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4e000fc30 con 0x56462d660420 2024-05-15T14:03:52.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.771+0000 7fc4d9ffb700 1 -- 172.21.15.33:0/1215087046 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc4e0043070 con 0x56462d660420 2024-05-15T14:03:52.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.771+0000 7fc502d41b80 10 client.15753 fetch_fsmap finished waiting for FSMap version 27 2024-05-15T14:03:52.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.771+0000 7fc502d41b80 10 client.15753 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:03:52.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.771+0000 7fc502d41b80 20 client.15753 populate_metadata read hostname 'smithi033' 2024-05-15T14:03:52.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.771+0000 7fc502d41b80 1 --2- 172.21.15.33:0/1215087046 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56462d6e4bf0 0x56462d6e7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:52.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.771+0000 7fc502d41b80 4 client.15753 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "504787be", "release": true, "await": true}] 2024-05-15T14:03:52.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.771+0000 7fc502d41b80 1 -- 172.21.15.33:0/1215087046 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "quiesce db", "set_id": "504787be", "release": true, "await": true}) -- 0x56462d5b55a0 con 0x56462d6e4bf0 2024-05-15T14:03:52.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.771+0000 7fc4f6247700 1 --2- 172.21.15.33:0/1215087046 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56462d6e4bf0 0x56462d6e7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:52.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.771+0000 7fc4f6247700 1 --2- 172.21.15.33:0/1215087046 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56462d6e4bf0 0x56462d6e7030 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96f327660f74fbdd server_cookie=11bcdb61c4dae42d in_seq=0 out_seq=0 2024-05-15T14:03:52.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.771+0000 7fc4d9ffb700 10 client.15753 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:03:52.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.779+0000 7fc4d9ffb700 1 -- 172.21.15.33:0/1215087046 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56462d5b55a0 con 0x56462d6e4bf0 2024-05-15T14:03:52.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.779+0000 7fc4d9ffb700 10 client.15753 handle_command_reply: tid=0 2024-05-15T14:03:52.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.779+0000 7fc502d41b80 1 client.15753 shutdown 2024-05-15T14:03:52.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.779+0000 7fc502d41b80 1 -- 172.21.15.33:0/1215087046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56462d660420 msgr2=0x56462d678920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:52.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.779+0000 7fc502d41b80 1 --2- 172.21.15.33:0/1215087046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56462d660420 0x56462d678920 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0027010 tx=0x7fc4e000fa40 comp rx=0 tx=0).stop 2024-05-15T14:03:52.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.779+0000 7fc502d41b80 1 -- 172.21.15.33:0/1215087046 shutdown_connections 2024-05-15T14:03:52.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.779+0000 7fc502d41b80 1 --2- 172.21.15.33:0/1215087046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56462d660d40 0x56462d678e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:52.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.779+0000 7fc502d41b80 1 --2- 172.21.15.33:0/1215087046 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56462d65c2c0 0x56462d67a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:52.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.779+0000 7fc502d41b80 1 --2- 172.21.15.33:0/1215087046 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56462d6e4bf0 0x56462d6e7030 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:52.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.779+0000 7fc502d41b80 1 --2- 172.21.15.33:0/1215087046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56462d660420 0x56462d678920 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:52.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.779+0000 7fc4d9ffb700 0 client.15753 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:03:52.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.779+0000 7fc502d41b80 1 -- 172.21.15.33:0/1215087046 >> 172.21.15.33:0/1215087046 conn(0x56462d588eb0 msgr2=0x56462d65b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:52.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.779+0000 7fc502d41b80 1 -- 172.21.15.33:0/1215087046 shutdown_connections 2024-05-15T14:03:52.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.779+0000 7fc502d41b80 1 -- 172.21.15.33:0/1215087046 wait complete. 2024-05-15T14:03:52.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.779+0000 7fc502d41b80 20 client.15753 trim_cache size 0 max 16384 2024-05-15T14:03:52.783 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T14:03:52.783 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 27, 2024-05-15T14:03:52.783 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24478, 2024-05-15T14:03:52.783 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 12, 2024-05-15T14:03:52.783 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-05-15T14:03:52.783 INFO:teuthology.orchestra.run.smithi033.stdout: "504787be": { 2024-05-15T14:03:52.783 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 12, 2024-05-15T14:03:52.783 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-05-15T14:03:52.783 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:03:52.783 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-05-15T14:03:52.783 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:03:52.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:03:52.784 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 90.0, 2024-05-15T14:03:52.784 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 130.0, 2024-05-15T14:03:52.784 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-05-15T14:03:52.784 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-05-15T14:03:52.784 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-05-15T14:03:52.784 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:03:52.784 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-05-15T14:03:52.784 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:03:52.784 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:03:52.784 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:03:52.784 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:03:52.784 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:03:52.784 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:03:52.784 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T14:03:52.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.780+0000 7fc4f84ab700 1 -- 172.21.15.33:0/3910073941 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc4dc0b29f0 msgr2=0x7fc4dc0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:52.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.780+0000 7fc4f84ab700 1 --2- 172.21.15.33:0/3910073941 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc4dc0b29f0 0x7fc4dc0b4eb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0009990 tx=0x7fc4e00095f0 comp rx=0 tx=0).stop 2024-05-15T14:03:52.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.780+0000 7fc4f84ab700 1 -- 172.21.15.33:0/3910073941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4f01037f0 msgr2=0x7fc4f019f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:52.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.780+0000 7fc4f84ab700 1 --2- 172.21.15.33:0/3910073941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4f01037f0 0x7fc4f019f8d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec00b530 tx=0x7fc4ec00b840 comp rx=0 tx=0).stop 2024-05-15T14:03:52.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.780+0000 7fc4f84ab700 1 -- 172.21.15.33:0/3910073941 shutdown_connections 2024-05-15T14:03:52.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.780+0000 7fc4f84ab700 1 --2- 172.21.15.33:0/3910073941 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc4dc0b29f0 0x7fc4dc0b4eb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:52.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.780+0000 7fc4f84ab700 1 --2- 172.21.15.33:0/3910073941 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc4f01088f0 0x7fc4f0199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:52.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.780+0000 7fc4f84ab700 1 --2- 172.21.15.33:0/3910073941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4f01037f0 0x7fc4f019f8d0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:52.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.780+0000 7fc4f84ab700 1 --2- 172.21.15.33:0/3910073941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4f01041a0 0x7fc4f019fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:52.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.780+0000 7fc4f84ab700 1 -- 172.21.15.33:0/3910073941 >> 172.21.15.33:0/3910073941 conn(0x7fc4f007a6a0 msgr2=0x7fc4f01017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:52.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.780+0000 7fc4f84ab700 1 -- 172.21.15.33:0/3910073941 shutdown_connections 2024-05-15T14:03:52.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:03:52.780+0000 7fc4f84ab700 1 -- 172.21.15.33:0/3910073941 wait complete. 2024-05-15T14:03:52.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:52 smithi052 ceph-mon[36190]: pgmap v109: 97 pgs: 97 active+clean; 59 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 249 op/s 2024-05-15T14:03:52.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:52 smithi081 ceph-mon[36434]: pgmap v109: 97 pgs: 97 active+clean; 59 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 249 op/s 2024-05-15T14:03:53.051 INFO:tasks.workunit.client.0.smithi033.stdout: 1 167656 10.46 MB/sec warmup 88 sec latency 9601.467 ms 2024-05-15T14:03:53.721 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '504787be', total seconds elapsed: 16.1 2024-05-15T14:03:54.051 INFO:tasks.workunit.client.0.smithi033.stdout: 1 170498 10.55 MB/sec warmup 89 sec latency 25.340 ms 2024-05-15T14:03:54.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:54 smithi033 ceph-mon[31650]: pgmap v110: 97 pgs: 97 active+clean; 58 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 1.7 MiB/s wr, 77 op/s 2024-05-15T14:03:54.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:54 smithi052 ceph-mon[36190]: pgmap v110: 97 pgs: 97 active+clean; 58 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 1.7 MiB/s wr, 77 op/s 2024-05-15T14:03:54.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:54 smithi081 ceph-mon[36434]: pgmap v110: 97 pgs: 97 active+clean; 58 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 1.7 MiB/s wr, 77 op/s 2024-05-15T14:03:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.935+0000 7eff9b054640 1 -- 172.21.15.33:0/1901250430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff941022f0 msgr2=0x7eff9410e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.935+0000 7eff9b054640 1 --2- 172.21.15.33:0/1901250430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff941022f0 0x7eff9410e800 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7eff8c009cd0 tx=0x7eff8c02a8e0 comp rx=0 tx=0).stop 2024-05-15T14:03:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.935+0000 7eff9b054640 1 -- 172.21.15.33:0/1901250430 shutdown_connections 2024-05-15T14:03:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.935+0000 7eff9b054640 1 --2- 172.21.15.33:0/1901250430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff941022f0 0x7eff9410e800 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.935+0000 7eff9b054640 1 --2- 172.21.15.33:0/1901250430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff94101930 0x7eff94101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.935+0000 7eff9b054640 1 --2- 172.21.15.33:0/1901250430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff94107940 0x7eff94107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.935+0000 7eff9b054640 1 -- 172.21.15.33:0/1901250430 >> 172.21.15.33:0/1901250430 conn(0x7eff94078a70 msgr2=0x7eff94078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.935+0000 7eff9b054640 1 -- 172.21.15.33:0/1901250430 shutdown_connections 2024-05-15T14:03:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.935+0000 7eff9b054640 1 -- 172.21.15.33:0/1901250430 wait complete. 2024-05-15T14:03:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.936+0000 7eff9b054640 1 Processor -- start 2024-05-15T14:03:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.936+0000 7eff9b054640 1 -- start start 2024-05-15T14:03:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.936+0000 7eff9b054640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff94101930 0x7eff9419a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.936+0000 7eff9b054640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff941022f0 0x7eff9419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.936+0000 7eff9b054640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff94107940 0x7eff941a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.936+0000 7eff9b054640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7eff94111c00 con 0x7eff94107940 2024-05-15T14:03:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.936+0000 7eff9b054640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7eff94111a80 con 0x7eff94101930 2024-05-15T14:03:54.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.936+0000 7eff9b054640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7eff94111d80 con 0x7eff941022f0 2024-05-15T14:03:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.936+0000 7eff995f1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff94107940 0x7eff941a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.936+0000 7eff98df0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff94101930 0x7eff9419a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.936+0000 7eff8bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff941022f0 0x7eff9419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.936+0000 7eff98df0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff94101930 0x7eff9419a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:58436/0 (socket says 172.21.15.33:58436) 2024-05-15T14:03:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.936+0000 7eff98df0640 1 -- 172.21.15.33:0/333119719 learned_addr learned my addr 172.21.15.33:0/333119719 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.936+0000 7eff995f1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff94107940 0x7eff941a5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56944/0 (socket says 172.21.15.33:56944) 2024-05-15T14:03:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.936+0000 7eff8bfff640 1 -- 172.21.15.33:0/333119719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff94101930 msgr2=0x7eff9419a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.936+0000 7eff8bfff640 1 --2- 172.21.15.33:0/333119719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff94101930 0x7eff9419a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.937+0000 7eff8bfff640 1 -- 172.21.15.33:0/333119719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff94107940 msgr2=0x7eff941a5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.937+0000 7eff8bfff640 1 --2- 172.21.15.33:0/333119719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff94107940 0x7eff941a5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.937+0000 7eff8bfff640 1 -- 172.21.15.33:0/333119719 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff941a5640 con 0x7eff941022f0 2024-05-15T14:03:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.937+0000 7eff8bfff640 1 --2- 172.21.15.33:0/333119719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff941022f0 0x7eff9419a5c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7eff8400e9f0 tx=0x7eff8400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.937+0000 7eff89ffb640 1 -- 172.21.15.33:0/333119719 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8400ccc0 con 0x7eff941022f0 2024-05-15T14:03:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.937+0000 7eff9b054640 1 -- 172.21.15.33:0/333119719 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff941a5810 con 0x7eff941022f0 2024-05-15T14:03:54.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.937+0000 7eff89ffb640 1 -- 172.21.15.33:0/333119719 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff84005020 con 0x7eff941022f0 2024-05-15T14:03:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.937+0000 7eff89ffb640 1 -- 172.21.15.33:0/333119719 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8401c070 con 0x7eff941022f0 2024-05-15T14:03:54.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.937+0000 7eff9b054640 1 -- 172.21.15.33:0/333119719 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff941a5cd0 con 0x7eff941022f0 2024-05-15T14:03:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.939+0000 7eff9dd6f740 1 Processor -- start 2024-05-15T14:03:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.940+0000 7eff9dd6f740 1 -- start start 2024-05-15T14:03:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.940+0000 7eff89ffb640 1 -- 172.21.15.33:0/333119719 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7eff840049a0 con 0x7eff941022f0 2024-05-15T14:03:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.940+0000 7eff9dd6f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b1727927f0 0x55b172792bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.940+0000 7eff9dd6f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b172793110 0x55b1727a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.940+0000 7eff9dd6f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1727a1ee0 0x55b1727a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.940+0000 7eff9dd6f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b172705c30 con 0x55b1727a1ee0 2024-05-15T14:03:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.940+0000 7eff9dd6f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b1726f57d0 con 0x55b172793110 2024-05-15T14:03:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.940+0000 7eff9dd6f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55b1726f9810 con 0x55b1727927f0 2024-05-15T14:03:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.940+0000 7eff995f1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b1727927f0 0x55b172792bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.940+0000 7eff995f1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b1727927f0 0x55b172792bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:37596/0 (socket says 172.21.15.33:37596) 2024-05-15T14:03:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.940+0000 7eff8bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1727a1ee0 0x55b1727a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.940+0000 7eff98df0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b172793110 0x55b1727a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.940+0000 7eff995f1640 1 -- 172.21.15.33:0/177468734 learned_addr learned my addr 172.21.15.33:0/177468734 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.940+0000 7eff995f1640 1 -- 172.21.15.33:0/177468734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b172793110 msgr2=0x55b1727a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.940+0000 7eff995f1640 1 --2- 172.21.15.33:0/177468734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b172793110 0x55b1727a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.940+0000 7eff995f1640 1 -- 172.21.15.33:0/177468734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1727a1ee0 msgr2=0x55b1727a42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.940+0000 7eff995f1640 1 --2- 172.21.15.33:0/177468734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1727a1ee0 0x55b1727a42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.940+0000 7eff995f1640 1 -- 172.21.15.33:0/177468734 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b17240a210 con 0x55b1727927f0 2024-05-15T14:03:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.940+0000 7eff8bfff640 1 --2- 172.21.15.33:0/177468734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1727a1ee0 0x55b1727a42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:03:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.940+0000 7eff98df0640 1 --2- 172.21.15.33:0/177468734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b172793110 0x55b1727a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:03:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.941+0000 7eff995f1640 1 --2- 172.21.15.33:0/177468734 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b1727927f0 0x55b172792bd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7eff8c009cd0 tx=0x7eff8c009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c2c1638741f1d72 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.941+0000 7eff89ffb640 1 --2- 172.21.15.33:0/333119719 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7eff7c0b2950 0x7eff7c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.941+0000 7eff7affd640 1 -- 172.21.15.33:0/177468734 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c049070 con 0x55b1727927f0 2024-05-15T14:03:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.941+0000 7eff7affd640 1 -- 172.21.15.33:0/177468734 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff8c036040 con 0x55b1727927f0 2024-05-15T14:03:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.941+0000 7eff7affd640 1 -- 172.21.15.33:0/177468734 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c004240 con 0x55b1727927f0 2024-05-15T14:03:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.941+0000 7eff98df0640 1 --2- 172.21.15.33:0/333119719 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7eff7c0b2950 0x7eff7c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.941+0000 7eff89ffb640 1 -- 172.21.15.33:0/333119719 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7eff840e2570 con 0x7eff941022f0 2024-05-15T14:03:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.941+0000 7eff9dd6f740 1 -- 172.21.15.33:0/177468734 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b1727927f0 msgr2=0x55b172792bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.941+0000 7eff98df0640 1 --2- 172.21.15.33:0/333119719 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7eff7c0b2950 0x7eff7c0b4e10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55b1727954e0 tx=0x7eff9000a380 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.941+0000 7eff9dd6f740 1 --2- 172.21.15.33:0/177468734 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b1727927f0 0x55b172792bd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7eff8c009cd0 tx=0x7eff8c009a20 comp rx=0 tx=0).stop 2024-05-15T14:03:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.942+0000 7eff9dd6f740 1 -- 172.21.15.33:0/177468734 shutdown_connections 2024-05-15T14:03:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.942+0000 7eff9dd6f740 1 --2- 172.21.15.33:0/177468734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1727a1ee0 0x55b1727a42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.942+0000 7eff9dd6f740 1 --2- 172.21.15.33:0/177468734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b172793110 0x55b1727a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.942+0000 7eff9dd6f740 1 --2- 172.21.15.33:0/177468734 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b1727927f0 0x55b172792bd0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.942+0000 7eff9dd6f740 1 -- 172.21.15.33:0/177468734 >> 172.21.15.33:0/177468734 conn(0x55b17278ee00 msgr2=0x55b17278f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.942+0000 7eff9dd6f740 1 -- 172.21.15.33:0/177468734 shutdown_connections 2024-05-15T14:03:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.942+0000 7eff9dd6f740 1 -- 172.21.15.33:0/177468734 wait complete. 2024-05-15T14:03:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.942+0000 7eff9dd6f740 1 -- start start 2024-05-15T14:03:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.942+0000 7eff9dd6f740 1 Processor -- start 2024-05-15T14:03:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.942+0000 7eff9dd6f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1727927f0 0x55b1727a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.942+0000 7eff9dd6f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b172793110 0x55b17279eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.942+0000 7eff9dd6f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b1727a1ee0 0x55b17279f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.942+0000 7eff9dd6f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b1723db790 con 0x55b1727927f0 2024-05-15T14:03:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.942+0000 7eff9dd6f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b1726b1890 con 0x55b172793110 2024-05-15T14:03:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.942+0000 7eff9dd6f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55b1727a4b50 con 0x55b1727a1ee0 2024-05-15T14:03:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.942+0000 7eff9dd6f740 10 client.0 _ll_register_callbacks cb 0x55b1726b4d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:03:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.943+0000 7eff995f1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b1727a1ee0 0x55b17279f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.943+0000 7eff98df0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1727927f0 0x55b1727a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.943+0000 7eff995f1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b1727a1ee0 0x55b17279f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:37600/0 (socket says 172.21.15.33:37600) 2024-05-15T14:03:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.943+0000 7eff995f1640 1 -- 172.21.15.33:0/237824076 learned_addr learned my addr 172.21.15.33:0/237824076 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.943+0000 7eff8bfff640 1 --2- 172.21.15.33:0/237824076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b172793110 0x55b17279eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.943+0000 7eff995f1640 1 -- 172.21.15.33:0/237824076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b172793110 msgr2=0x55b17279eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.943+0000 7eff995f1640 1 --2- 172.21.15.33:0/237824076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b172793110 0x55b17279eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.943+0000 7eff995f1640 1 -- 172.21.15.33:0/237824076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1727927f0 msgr2=0x55b1727a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.943+0000 7eff995f1640 1 --2- 172.21.15.33:0/237824076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1727927f0 0x55b1727a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.943+0000 7eff995f1640 1 -- 172.21.15.33:0/237824076 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1724d6620 con 0x55b1727a1ee0 2024-05-15T14:03:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.943+0000 7eff8bfff640 1 --2- 172.21.15.33:0/237824076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b172793110 0x55b17279eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:03:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.943+0000 7eff98df0640 1 --2- 172.21.15.33:0/237824076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1727927f0 0x55b1727a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:03:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.943+0000 7eff995f1640 1 --2- 172.21.15.33:0/237824076 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b1727a1ee0 0x55b17279f1b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7eff8c047040 tx=0x7eff8c003d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6877b50ff4433a07 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.943+0000 7eff73fff640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-05-15T14:03:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.943+0000 7eff73fff640 1 -- 172.21.15.33:0/237824076 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c049040 con 0x55b1727a1ee0 2024-05-15T14:03:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.943+0000 7eff9dd6f740 1 -- 172.21.15.33:0/237824076 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff8c036040 con 0x55b1727a1ee0 2024-05-15T14:03:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.943+0000 7eff73fff640 1 -- 172.21.15.33:0/237824076 <== mon.1 v2:172.21.15.81:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7eff8c0319d0 con 0x55b1727a1ee0 2024-05-15T14:03:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.943+0000 7eff73fff640 1 -- 172.21.15.33:0/237824076 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7eff8c05d050 con 0x55b1727a1ee0 2024-05-15T14:03:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.943+0000 7eff9dd6f740 10 client.25438 fetch_fsmap learned FSMap version 27 2024-05-15T14:03:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.943+0000 7eff9dd6f740 1 -- 172.21.15.33:0/237824076 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b1727b0650 con 0x55b1727a1ee0 2024-05-15T14:03:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.944+0000 7eff73fff640 1 -- 172.21.15.33:0/237824076 <== mon.1 v2:172.21.15.81:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff8c036040 con 0x55b1727a1ee0 2024-05-15T14:03:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.944+0000 7eff73fff640 1 -- 172.21.15.33:0/237824076 <== mon.1 v2:172.21.15.81:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7eff8c030070 con 0x55b1727a1ee0 2024-05-15T14:03:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.944+0000 7eff9dd6f740 10 client.25438 fetch_fsmap finished waiting for FSMap version 27 2024-05-15T14:03:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.944+0000 7eff9dd6f740 10 client.25438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:03:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.944+0000 7eff9dd6f740 20 client.25438 populate_metadata read hostname 'smithi033' 2024-05-15T14:03:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.944+0000 7eff9dd6f740 1 --2- 172.21.15.33:0/237824076 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55b1727b19a0 0x55b1727b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.944+0000 7eff9dd6f740 4 client.25438 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:03:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.944+0000 7eff9dd6f740 1 -- 172.21.15.33:0/237824076 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b172707b80 con 0x55b1727b19a0 2024-05-15T14:03:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.944+0000 7eff98df0640 1 --2- 172.21.15.33:0/237824076 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55b1727b19a0 0x55b1727b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.944+0000 7eff98df0640 1 --2- 172.21.15.33:0/237824076 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55b1727b19a0 0x55b1727b3d90 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=770493f4bf0ae626 server_cookie=3d4cbcfc7ce498aa in_seq=0 out_seq=0 2024-05-15T14:03:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.944+0000 7eff73fff640 10 client.25438 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:03:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.945+0000 7eff73fff640 1 -- 172.21.15.33:0/237824076 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b172707b80 con 0x55b1727b19a0 2024-05-15T14:03:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.945+0000 7eff73fff640 10 client.25438 handle_command_reply: tid=0 2024-05-15T14:03:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.945+0000 7eff9dd6f740 1 client.25438 shutdown 2024-05-15T14:03:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.945+0000 7eff9dd6f740 1 -- 172.21.15.33:0/237824076 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b1727a1ee0 msgr2=0x55b17279f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.945+0000 7eff9dd6f740 1 --2- 172.21.15.33:0/237824076 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b1727a1ee0 0x55b17279f1b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7eff8c047040 tx=0x7eff8c003d50 comp rx=0 tx=0).stop 2024-05-15T14:03:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.945+0000 7eff9dd6f740 1 -- 172.21.15.33:0/237824076 shutdown_connections 2024-05-15T14:03:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.945+0000 7eff9dd6f740 1 --2- 172.21.15.33:0/237824076 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55b1727b19a0 0x55b1727b3d90 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.945+0000 7eff9dd6f740 1 --2- 172.21.15.33:0/237824076 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b1727a1ee0 0x55b17279f1b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.945+0000 7eff9dd6f740 1 --2- 172.21.15.33:0/237824076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b172793110 0x55b17279eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.945+0000 7eff9dd6f740 1 --2- 172.21.15.33:0/237824076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1727927f0 0x55b1727a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.945+0000 7eff73fff640 0 client.25438 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:03:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.946+0000 7eff9dd6f740 1 -- 172.21.15.33:0/237824076 >> 172.21.15.33:0/237824076 conn(0x55b17278ee00 msgr2=0x55b172794c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.946+0000 7eff9dd6f740 1 -- 172.21.15.33:0/237824076 shutdown_connections 2024-05-15T14:03:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.946+0000 7eff9dd6f740 1 -- 172.21.15.33:0/237824076 wait complete. 2024-05-15T14:03:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.946+0000 7eff9dd6f740 20 client.25438 trim_cache size 0 max 16384 2024-05-15T14:03:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.963+0000 7eff9dd6f740 1 Processor -- start 2024-05-15T14:03:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.963+0000 7eff9dd6f740 1 -- start start 2024-05-15T14:03:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.963+0000 7eff9dd6f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1727927f0 0x55b172799940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.963+0000 7eff9dd6f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b172793110 0x55b1727ada20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.963+0000 7eff9dd6f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1727a1ee0 0x55b1727a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.963+0000 7eff9dd6f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7eff941aec10 con 0x55b1727a1ee0 2024-05-15T14:03:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.963+0000 7eff9dd6f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7eff941ac480 con 0x55b1727927f0 2024-05-15T14:03:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.963+0000 7eff9dd6f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7eff941aa2d0 con 0x55b172793110 2024-05-15T14:03:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.963+0000 7eff995f1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1727a1ee0 0x55b1727a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.963+0000 7eff98df0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1727927f0 0x55b172799940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.963+0000 7eff995f1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1727a1ee0 0x55b1727a13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56974/0 (socket says 172.21.15.33:56974) 2024-05-15T14:03:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.963+0000 7eff995f1640 1 -- 172.21.15.33:0/1608136432 learned_addr learned my addr 172.21.15.33:0/1608136432 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.963+0000 7eff8bfff640 1 --2- 172.21.15.33:0/1608136432 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b172793110 0x55b1727ada20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.963+0000 7eff995f1640 1 -- 172.21.15.33:0/1608136432 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b172793110 msgr2=0x55b1727ada20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.963+0000 7eff995f1640 1 --2- 172.21.15.33:0/1608136432 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b172793110 0x55b1727ada20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.963+0000 7eff995f1640 1 -- 172.21.15.33:0/1608136432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1727927f0 msgr2=0x55b172799940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.963+0000 7eff995f1640 1 --2- 172.21.15.33:0/1608136432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1727927f0 0x55b172799940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.963+0000 7eff995f1640 1 -- 172.21.15.33:0/1608136432 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b17279d0f0 con 0x55b1727a1ee0 2024-05-15T14:03:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.963+0000 7eff98df0640 1 --2- 172.21.15.33:0/1608136432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1727927f0 0x55b172799940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:03:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.964+0000 7eff995f1640 1 --2- 172.21.15.33:0/1608136432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1727a1ee0 0x55b1727a13e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7eff8c02ae00 tx=0x7eff8c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e905ff346d5763d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.964+0000 7eff7a7fc640 1 -- 172.21.15.33:0/1608136432 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c049070 con 0x55b1727a1ee0 2024-05-15T14:03:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.964+0000 7eff7a7fc640 1 -- 172.21.15.33:0/1608136432 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff8c036050 con 0x55b1727a1ee0 2024-05-15T14:03:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.964+0000 7eff7a7fc640 1 -- 172.21.15.33:0/1608136432 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c030960 con 0x55b1727a1ee0 2024-05-15T14:03:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.964+0000 7eff9dd6f740 1 -- 172.21.15.33:0/1608136432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1727a1ee0 msgr2=0x55b1727a13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.964+0000 7eff9dd6f740 1 --2- 172.21.15.33:0/1608136432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1727a1ee0 0x55b1727a13e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7eff8c02ae00 tx=0x7eff8c009a20 comp rx=0 tx=0).stop 2024-05-15T14:03:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.964+0000 7eff9dd6f740 1 -- 172.21.15.33:0/1608136432 shutdown_connections 2024-05-15T14:03:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.964+0000 7eff9dd6f740 1 --2- 172.21.15.33:0/1608136432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1727a1ee0 0x55b1727a13e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.964+0000 7eff9dd6f740 1 --2- 172.21.15.33:0/1608136432 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b172793110 0x55b1727ada20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.964+0000 7eff9dd6f740 1 --2- 172.21.15.33:0/1608136432 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b1727927f0 0x55b172799940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.964+0000 7eff9dd6f740 1 -- 172.21.15.33:0/1608136432 >> 172.21.15.33:0/1608136432 conn(0x55b17278ee00 msgr2=0x55b17279dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.965+0000 7eff9dd6f740 1 -- 172.21.15.33:0/1608136432 shutdown_connections 2024-05-15T14:03:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.965+0000 7eff9dd6f740 1 -- 172.21.15.33:0/1608136432 wait complete. 2024-05-15T14:03:54.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.965+0000 7eff9dd6f740 1 -- start start 2024-05-15T14:03:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.965+0000 7eff9dd6f740 1 Processor -- start 2024-05-15T14:03:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.965+0000 7eff9dd6f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1727927f0 0x55b172814270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.965+0000 7eff9dd6f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b172793110 0x55b1728147b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.965+0000 7eff9dd6f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b1727a1ee0 0x55b1727a5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.965+0000 7eff9dd6f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7eff941aa940 con 0x55b1727927f0 2024-05-15T14:03:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.965+0000 7eff9dd6f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7eff941abdb0 con 0x55b172793110 2024-05-15T14:03:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.965+0000 7eff9dd6f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7eff941ab010 con 0x55b1727a1ee0 2024-05-15T14:03:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.965+0000 7eff9dd6f740 10 client.0 _ll_register_callbacks cb 0x7eff9000a6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:03:54.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.965+0000 7eff98df0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1727927f0 0x55b172814270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.966+0000 7eff98df0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1727927f0 0x55b172814270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56990/0 (socket says 172.21.15.33:56990) 2024-05-15T14:03:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.966+0000 7eff98df0640 1 -- 172.21.15.33:0/414396165 learned_addr learned my addr 172.21.15.33:0/414396165 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:03:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.966+0000 7eff995f1640 1 --2- 172.21.15.33:0/414396165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b1727a1ee0 0x55b1727a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.966+0000 7eff8bfff640 1 --2- 172.21.15.33:0/414396165 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b172793110 0x55b1728147b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.966+0000 7eff98df0640 1 -- 172.21.15.33:0/414396165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b1727a1ee0 msgr2=0x55b1727a5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.966+0000 7eff98df0640 1 --2- 172.21.15.33:0/414396165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b1727a1ee0 0x55b1727a5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.966+0000 7eff98df0640 1 -- 172.21.15.33:0/414396165 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b172793110 msgr2=0x55b1728147b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:03:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.966+0000 7eff98df0640 1 --2- 172.21.15.33:0/414396165 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b172793110 0x55b1728147b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.966+0000 7eff98df0640 1 -- 172.21.15.33:0/414396165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1727a5860 con 0x55b1727927f0 2024-05-15T14:03:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.966+0000 7eff98df0640 1 --2- 172.21.15.33:0/414396165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1727927f0 0x55b172814270 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7eff9000afd0 tx=0x7eff900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6724dc512d66d475 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:03:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.966+0000 7eff73fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:03:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.966+0000 7eff73fff640 1 -- 172.21.15.33:0/414396165 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff90033070 con 0x55b1727927f0 2024-05-15T14:03:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.966+0000 7eff73fff640 1 -- 172.21.15.33:0/414396165 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7eff900053d0 con 0x55b1727927f0 2024-05-15T14:03:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.966+0000 7eff9dd6f740 1 -- 172.21.15.33:0/414396165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff8c036050 con 0x55b1727927f0 2024-05-15T14:03:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.966+0000 7eff73fff640 1 -- 172.21.15.33:0/414396165 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7eff9003f050 con 0x55b1727927f0 2024-05-15T14:03:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.966+0000 7eff9dd6f740 10 client.15771 fetch_fsmap learned FSMap version 27 2024-05-15T14:03:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.966+0000 7eff9dd6f740 1 -- 172.21.15.33:0/414396165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b17281ba80 con 0x55b1727927f0 2024-05-15T14:03:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.967+0000 7eff73fff640 1 -- 172.21.15.33:0/414396165 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff8c036050 con 0x55b1727927f0 2024-05-15T14:03:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.967+0000 7eff73fff640 1 -- 172.21.15.33:0/414396165 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7eff9002c270 con 0x55b1727927f0 2024-05-15T14:03:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.967+0000 7eff9dd6f740 10 client.15771 fetch_fsmap finished waiting for FSMap version 27 2024-05-15T14:03:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.967+0000 7eff9dd6f740 10 client.15771 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:03:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.967+0000 7eff9dd6f740 20 client.15771 populate_metadata read hostname 'smithi033' 2024-05-15T14:03:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.967+0000 7eff9dd6f740 1 --2- 172.21.15.33:0/414396165 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55b17281c210 0x55b17281e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:03:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.967+0000 7eff9dd6f740 4 client.15771 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:03:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.967+0000 7eff9dd6f740 1 -- 172.21.15.33:0/414396165 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b172701360 con 0x55b17281c210 2024-05-15T14:03:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.967+0000 7eff8bfff640 1 --2- 172.21.15.33:0/414396165 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55b17281c210 0x55b17281e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:03:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.967+0000 7eff73fff640 10 client.15771 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:03:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:54.967+0000 7eff8bfff640 1 --2- 172.21.15.33:0/414396165 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55b17281c210 0x55b17281e630 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea956a8f44bbbd18 server_cookie=5f3f42e1678421fb in_seq=0 out_seq=0 2024-05-15T14:03:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:55.003+0000 7eff73fff640 1 -- 172.21.15.33:0/414396165 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b172701360 con 0x55b17281c210 2024-05-15T14:03:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:55.003+0000 7eff73fff640 10 client.15771 handle_command_reply: tid=0 2024-05-15T14:03:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:55.003+0000 7eff9dd6f740 1 client.15771 shutdown 2024-05-15T14:03:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:55.003+0000 7eff9dd6f740 1 -- 172.21.15.33:0/414396165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1727927f0 msgr2=0x55b172814270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:55.003+0000 7eff9dd6f740 1 --2- 172.21.15.33:0/414396165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1727927f0 0x55b172814270 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7eff9000afd0 tx=0x7eff900046c0 comp rx=0 tx=0).stop 2024-05-15T14:03:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:55.003+0000 7eff9dd6f740 1 -- 172.21.15.33:0/414396165 shutdown_connections 2024-05-15T14:03:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:55.003+0000 7eff9dd6f740 1 --2- 172.21.15.33:0/414396165 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55b17281c210 0x55b17281e630 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:55.003+0000 7eff9dd6f740 1 --2- 172.21.15.33:0/414396165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b1727a1ee0 0x55b1727a5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:55.003+0000 7eff9dd6f740 1 --2- 172.21.15.33:0/414396165 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b172793110 0x55b1728147b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:55.003+0000 7eff73fff640 0 client.15771 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:03:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:55.003+0000 7eff9dd6f740 1 --2- 172.21.15.33:0/414396165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1727927f0 0x55b172814270 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:55.003+0000 7eff9dd6f740 1 -- 172.21.15.33:0/414396165 >> 172.21.15.33:0/414396165 conn(0x55b17278ee00 msgr2=0x55b172794340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:55.003+0000 7eff9dd6f740 1 -- 172.21.15.33:0/414396165 shutdown_connections 2024-05-15T14:03:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:55.003+0000 7eff9dd6f740 1 -- 172.21.15.33:0/414396165 wait complete. 2024-05-15T14:03:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:55.003+0000 7eff9dd6f740 20 client.15771 trim_cache size 0 max 16384 2024-05-15T14:03:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:03:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:03:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:03:55.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:55.004+0000 7eff9b054640 1 -- 172.21.15.33:0/333119719 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7eff7c0b2950 msgr2=0x7eff7c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:55.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:55.004+0000 7eff9b054640 1 --2- 172.21.15.33:0/333119719 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7eff7c0b2950 0x7eff7c0b4e10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55b1727954e0 tx=0x7eff9000a380 comp rx=0 tx=0).stop 2024-05-15T14:03:55.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:55.004+0000 7eff9b054640 1 -- 172.21.15.33:0/333119719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff941022f0 msgr2=0x7eff9419a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:03:55.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:55.004+0000 7eff9b054640 1 --2- 172.21.15.33:0/333119719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff941022f0 0x7eff9419a5c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7eff8400e9f0 tx=0x7eff8400eeb0 comp rx=0 tx=0).stop 2024-05-15T14:03:55.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:55.004+0000 7eff9b054640 1 -- 172.21.15.33:0/333119719 shutdown_connections 2024-05-15T14:03:55.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:55.004+0000 7eff9b054640 1 --2- 172.21.15.33:0/333119719 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7eff7c0b2950 0x7eff7c0b4e10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:55.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:55.004+0000 7eff9b054640 1 --2- 172.21.15.33:0/333119719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff94107940 0x7eff941a5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:55.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:55.004+0000 7eff9b054640 1 --2- 172.21.15.33:0/333119719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7eff941022f0 0x7eff9419a5c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:55.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:55.004+0000 7eff9b054640 1 --2- 172.21.15.33:0/333119719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff94101930 0x7eff9419a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:03:55.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:55.004+0000 7eff9b054640 1 -- 172.21.15.33:0/333119719 >> 172.21.15.33:0/333119719 conn(0x7eff94078a70 msgr2=0x7eff941044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:03:55.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:55.004+0000 7eff9b054640 1 -- 172.21.15.33:0/333119719 shutdown_connections 2024-05-15T14:03:55.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:03:55.004+0000 7eff9b054640 1 -- 172.21.15.33:0/333119719 wait complete. 2024-05-15T14:03:55.051 INFO:tasks.workunit.client.0.smithi033.stdout: 1 171977 10.49 MB/sec warmup 90 sec latency 18.592 ms 2024-05-15T14:03:56.051 INFO:tasks.workunit.client.0.smithi033.stdout: 1 174249 10.53 MB/sec warmup 91 sec latency 43.534 ms 2024-05-15T14:03:56.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:56 smithi033 ceph-mon[31650]: pgmap v111: 97 pgs: 97 active+clean; 58 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 1.5 MiB/s wr, 70 op/s 2024-05-15T14:03:56.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:56 smithi052 ceph-mon[36190]: pgmap v111: 97 pgs: 97 active+clean; 58 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 1.5 MiB/s wr, 70 op/s 2024-05-15T14:03:56.914 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:56 smithi081 ceph-mon[36434]: pgmap v111: 97 pgs: 97 active+clean; 58 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 1.5 MiB/s wr, 70 op/s 2024-05-15T14:03:57.051 INFO:tasks.workunit.client.0.smithi033.stdout: 1 176540 10.53 MB/sec warmup 92 sec latency 21.376 ms 2024-05-15T14:03:58.051 INFO:tasks.workunit.client.0.smithi033.stdout: 1 178178 10.51 MB/sec warmup 93 sec latency 13.209 ms 2024-05-15T14:03:58.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:03:58 smithi033 ceph-mon[31650]: pgmap v112: 97 pgs: 97 active+clean; 52 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 225 op/s 2024-05-15T14:03:58.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:03:58 smithi052 ceph-mon[36190]: pgmap v112: 97 pgs: 97 active+clean; 52 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 225 op/s 2024-05-15T14:03:58.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:03:58 smithi081 ceph-mon[36434]: pgmap v112: 97 pgs: 97 active+clean; 52 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 225 op/s 2024-05-15T14:03:59.051 INFO:tasks.workunit.client.0.smithi033.stdout: 1 181146 10.59 MB/sec warmup 94 sec latency 11.542 ms 2024-05-15T14:04:00.051 INFO:tasks.workunit.client.0.smithi033.stdout: 1 182688 10.54 MB/sec warmup 95 sec latency 15.843 ms 2024-05-15T14:04:00.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:00 smithi033 ceph-mon[31650]: pgmap v113: 97 pgs: 97 active+clean; 52 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 187 op/s 2024-05-15T14:04:00.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:00 smithi052 ceph-mon[36190]: pgmap v113: 97 pgs: 97 active+clean; 52 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 187 op/s 2024-05-15T14:04:00.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:00 smithi081 ceph-mon[36434]: pgmap v113: 97 pgs: 97 active+clean; 52 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 187 op/s 2024-05-15T14:04:01.051 INFO:tasks.workunit.client.0.smithi033.stdout: 1 184834 10.58 MB/sec warmup 96 sec latency 122.840 ms 2024-05-15T14:04:02.051 INFO:tasks.workunit.client.0.smithi033.stdout: 1 184834 10.47 MB/sec warmup 97 sec latency 1122.885 ms 2024-05-15T14:04:02.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:02 smithi033 ceph-mon[31650]: pgmap v114: 97 pgs: 97 active+clean; 52 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 187 op/s 2024-05-15T14:04:02.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:02 smithi052 ceph-mon[36190]: pgmap v114: 97 pgs: 97 active+clean; 52 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 187 op/s 2024-05-15T14:04:02.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:02 smithi081 ceph-mon[36434]: pgmap v114: 97 pgs: 97 active+clean; 52 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 187 op/s 2024-05-15T14:04:03.051 INFO:tasks.workunit.client.0.smithi033.stdout: 1 184834 10.36 MB/sec warmup 98 sec latency 2122.967 ms 2024-05-15T14:04:04.051 INFO:tasks.workunit.client.0.smithi033.stdout: 1 185035 10.26 MB/sec warmup 99 sec latency 2886.626 ms 2024-05-15T14:04:04.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:04 smithi033 ceph-mon[31650]: pgmap v115: 97 pgs: 97 active+clean; 55 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 295 op/s 2024-05-15T14:04:04.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:04 smithi052 ceph-mon[36190]: pgmap v115: 97 pgs: 97 active+clean; 55 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 295 op/s 2024-05-15T14:04:04.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:04 smithi081 ceph-mon[36434]: pgmap v115: 97 pgs: 97 active+clean; 55 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 295 op/s 2024-05-15T14:04:05.051 INFO:tasks.workunit.client.0.smithi033.stdout: 1 187984 10.33 MB/sec warmup 100 sec latency 11.298 ms 2024-05-15T14:04:06.051 INFO:tasks.workunit.client.0.smithi033.stdout: 1 189185 10.26 MB/sec warmup 101 sec latency 48.640 ms 2024-05-15T14:04:06.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:06 smithi033 ceph-mon[31650]: pgmap v116: 97 pgs: 97 active+clean; 55 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 262 op/s 2024-05-15T14:04:06.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:06 smithi052 ceph-mon[36190]: pgmap v116: 97 pgs: 97 active+clean; 55 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 262 op/s 2024-05-15T14:04:06.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:06 smithi081 ceph-mon[36434]: pgmap v116: 97 pgs: 97 active+clean; 55 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 262 op/s 2024-05-15T14:04:07.051 INFO:tasks.workunit.client.0.smithi033.stdout: 1 191831 10.33 MB/sec warmup 102 sec latency 34.787 ms 2024-05-15T14:04:08.051 INFO:tasks.workunit.client.0.smithi033.stdout: 1 193989 10.30 MB/sec warmup 103 sec latency 16.169 ms 2024-05-15T14:04:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.096+0000 7f31a3237640 1 -- 172.21.15.33:0/1617696318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f319c103cc0 msgr2=0x7f319c1040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.096+0000 7f31a3237640 1 --2- 172.21.15.33:0/1617696318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f319c103cc0 0x7f319c1040a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3190009900 tx=0x7f319002a480 comp rx=0 tx=0).stop 2024-05-15T14:04:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.096+0000 7f31a3237640 1 -- 172.21.15.33:0/1617696318 shutdown_connections 2024-05-15T14:04:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.096+0000 7f31a3237640 1 --2- 172.21.15.33:0/1617696318 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f319c10d090 0x7f319c10f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.096+0000 7f31a3237640 1 --2- 172.21.15.33:0/1617696318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f319c1045e0 0x7f319c10cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.096+0000 7f31a3237640 1 --2- 172.21.15.33:0/1617696318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f319c103cc0 0x7f319c1040a0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.096+0000 7f31a3237640 1 -- 172.21.15.33:0/1617696318 >> 172.21.15.33:0/1617696318 conn(0x7f319c078a70 msgr2=0x7f319c078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.097+0000 7f31a3237640 1 -- 172.21.15.33:0/1617696318 shutdown_connections 2024-05-15T14:04:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.097+0000 7f31a3237640 1 -- 172.21.15.33:0/1617696318 wait complete. 2024-05-15T14:04:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.097+0000 7f31a3237640 1 Processor -- start 2024-05-15T14:04:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.097+0000 7f31a3237640 1 -- start start 2024-05-15T14:04:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.097+0000 7f31a3237640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f319c103cc0 0x7f319c1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.097+0000 7f31a3237640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f319c1045e0 0x7f319c1997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.097+0000 7f31a3237640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f319c10d090 0x7f319c19db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.097+0000 7f31a3237640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f319c112cd0 con 0x7f319c10d090 2024-05-15T14:04:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.097+0000 7f31a3237640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f319c112b50 con 0x7f319c103cc0 2024-05-15T14:04:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.097+0000 7f31a3237640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f319c112e50 con 0x7f319c1045e0 2024-05-15T14:04:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.098+0000 7f31a17d4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f319c10d090 0x7f319c19db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.098+0000 7f31a17d4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f319c10d090 0x7f319c19db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57984/0 (socket says 172.21.15.33:57984) 2024-05-15T14:04:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.098+0000 7f31a0fd3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f319c103cc0 0x7f319c1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.098+0000 7f31a17d4640 1 -- 172.21.15.33:0/2811685229 learned_addr learned my addr 172.21.15.33:0/2811685229 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:04:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.098+0000 7f319bfff640 1 --2- 172.21.15.33:0/2811685229 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f319c1045e0 0x7f319c1997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.098+0000 7f31a17d4640 1 -- 172.21.15.33:0/2811685229 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f319c1045e0 msgr2=0x7f319c1997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.098+0000 7f31a17d4640 1 --2- 172.21.15.33:0/2811685229 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f319c1045e0 0x7f319c1997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.098+0000 7f31a17d4640 1 -- 172.21.15.33:0/2811685229 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f319c103cc0 msgr2=0x7f319c1992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.098+0000 7f31a17d4640 1 --2- 172.21.15.33:0/2811685229 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f319c103cc0 0x7f319c1992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.098+0000 7f31a17d4640 1 -- 172.21.15.33:0/2811685229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f319c19e2a0 con 0x7f319c10d090 2024-05-15T14:04:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.098+0000 7f31a0fd3640 1 --2- 172.21.15.33:0/2811685229 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f319c103cc0 0x7f319c1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:04:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.098+0000 7f31a17d4640 1 --2- 172.21.15.33:0/2811685229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f319c10d090 0x7f319c19db70 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f319400e9d0 tx=0x7f319400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.098+0000 7f3199ffb640 1 -- 172.21.15.33:0/2811685229 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319400cd80 con 0x7f319c10d090 2024-05-15T14:04:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.098+0000 7f3199ffb640 1 -- 172.21.15.33:0/2811685229 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3194005020 con 0x7f319c10d090 2024-05-15T14:04:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.098+0000 7f31a3237640 1 -- 172.21.15.33:0/2811685229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f319c19e590 con 0x7f319c10d090 2024-05-15T14:04:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.098+0000 7f3199ffb640 1 -- 172.21.15.33:0/2811685229 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31940116d0 con 0x7f319c10d090 2024-05-15T14:04:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.099+0000 7f31a3237640 1 -- 172.21.15.33:0/2811685229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f319c1a5e20 con 0x7f319c10d090 2024-05-15T14:04:08.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.101+0000 7f3199ffb640 1 -- 172.21.15.33:0/2811685229 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f31940051d0 con 0x7f319c10d090 2024-05-15T14:04:08.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.101+0000 7f31a602f740 1 Processor -- start 2024-05-15T14:04:08.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.101+0000 7f31a602f740 1 -- start start 2024-05-15T14:04:08.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.101+0000 7f31a602f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f4e5918f0 0x555f4e591cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:08.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.101+0000 7f31a602f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555f4e592210 0x555f4e59c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:08.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.101+0000 7f31a602f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f4e59cad0 0x555f4e59eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:08.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.101+0000 7f31a602f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555f4e504c30 con 0x555f4e5918f0 2024-05-15T14:04:08.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.101+0000 7f31a602f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555f4e4f47d0 con 0x555f4e59cad0 2024-05-15T14:04:08.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.101+0000 7f31a602f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x555f4e4f8810 con 0x555f4e592210 2024-05-15T14:04:08.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.101+0000 7f3199ffb640 1 --2- 172.21.15.33:0/2811685229 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f318c0b29d0 0x7f318c0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:08.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.102+0000 7f319bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f4e5918f0 0x555f4e591cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:08.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.102+0000 7f31a0fd3640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555f4e592210 0x555f4e59c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:08.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.102+0000 7f31a0fd3640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555f4e592210 0x555f4e59c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:42522/0 (socket says 172.21.15.33:42522) 2024-05-15T14:04:08.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.102+0000 7f319bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f4e5918f0 0x555f4e591cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57988/0 (socket says 172.21.15.33:57988) 2024-05-15T14:04:08.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.102+0000 7f31a0fd3640 1 -- 172.21.15.33:0/1622008887 learned_addr learned my addr 172.21.15.33:0/1622008887 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:04:08.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.102+0000 7f319bfff640 1 --2- 172.21.15.33:0/1622008887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f4e59cad0 0x555f4e59eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:08.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.102+0000 7f31a17d4640 1 --2- 172.21.15.33:0/2811685229 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f318c0b29d0 0x7f318c0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:08.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.102+0000 7f3199ffb640 1 -- 172.21.15.33:0/2811685229 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f31940e29a0 con 0x7f319c10d090 2024-05-15T14:04:08.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.102+0000 7f319bfff640 1 -- 172.21.15.33:0/1622008887 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555f4e592210 msgr2=0x555f4e59c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:08.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.102+0000 7f319bfff640 1 --2- 172.21.15.33:0/1622008887 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555f4e592210 0x555f4e59c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:08.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.102+0000 7f319bfff640 1 -- 172.21.15.33:0/1622008887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f4e59cad0 msgr2=0x555f4e59eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:08.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.102+0000 7f319bfff640 1 --2- 172.21.15.33:0/1622008887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f4e59cad0 0x555f4e59eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:08.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.102+0000 7f319bfff640 1 -- 172.21.15.33:0/1622008887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f4e209210 con 0x555f4e5918f0 2024-05-15T14:04:08.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.102+0000 7f319bfff640 1 --2- 172.21.15.33:0/1622008887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f4e5918f0 0x555f4e591cd0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3188009fd0 tx=0x7f3188005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77000f676e1c7d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:08.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.102+0000 7f31a17d4640 1 --2- 172.21.15.33:0/2811685229 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f318c0b29d0 0x7f318c0b4e90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f31940a5dc0 tx=0x7f3194076670 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:08.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.102+0000 7f3186ffd640 1 -- 172.21.15.33:0/1622008887 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3188013cb0 con 0x555f4e5918f0 2024-05-15T14:04:08.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.102+0000 7f3186ffd640 1 -- 172.21.15.33:0/1622008887 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3188013e50 con 0x555f4e5918f0 2024-05-15T14:04:08.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.102+0000 7f3186ffd640 1 -- 172.21.15.33:0/1622008887 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3188004e50 con 0x555f4e5918f0 2024-05-15T14:04:08.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.103+0000 7f31a602f740 1 -- 172.21.15.33:0/1622008887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f4e5918f0 msgr2=0x555f4e591cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:08.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.103+0000 7f31a602f740 1 --2- 172.21.15.33:0/1622008887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f4e5918f0 0x555f4e591cd0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3188009fd0 tx=0x7f3188005850 comp rx=0 tx=0).stop 2024-05-15T14:04:08.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.103+0000 7f31a602f740 1 -- 172.21.15.33:0/1622008887 shutdown_connections 2024-05-15T14:04:08.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.103+0000 7f31a602f740 1 --2- 172.21.15.33:0/1622008887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f4e59cad0 0x555f4e59eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:08.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.103+0000 7f31a602f740 1 --2- 172.21.15.33:0/1622008887 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555f4e592210 0x555f4e59c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:08.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.103+0000 7f31a602f740 1 --2- 172.21.15.33:0/1622008887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f4e5918f0 0x555f4e591cd0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:08.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.103+0000 7f31a602f740 1 -- 172.21.15.33:0/1622008887 >> 172.21.15.33:0/1622008887 conn(0x555f4e58de00 msgr2=0x555f4e58e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:08.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.103+0000 7f31a602f740 1 -- 172.21.15.33:0/1622008887 shutdown_connections 2024-05-15T14:04:08.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.103+0000 7f31a602f740 1 -- 172.21.15.33:0/1622008887 wait complete. 2024-05-15T14:04:08.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.103+0000 7f31a602f740 1 -- start start 2024-05-15T14:04:08.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.104+0000 7f31a602f740 1 Processor -- start 2024-05-15T14:04:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.104+0000 7f31a602f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f4e5918f0 0x555f4e59c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.104+0000 7f31a602f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555f4e592210 0x555f4e5996b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.104+0000 7f31a602f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f4e59cad0 0x555f4e599bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.104+0000 7f31a602f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555f4e1da790 con 0x555f4e59cad0 2024-05-15T14:04:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.104+0000 7f31a602f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555f4e4b0890 con 0x555f4e5918f0 2024-05-15T14:04:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.104+0000 7f31a602f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x555f4e59f740 con 0x555f4e592210 2024-05-15T14:04:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.104+0000 7f31a602f740 10 client.0 _ll_register_callbacks cb 0x555f4e4b3d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:04:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.104+0000 7f319bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f4e59cad0 0x555f4e599bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.104+0000 7f319bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f4e59cad0 0x555f4e599bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58002/0 (socket says 172.21.15.33:58002) 2024-05-15T14:04:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.104+0000 7f319bfff640 1 -- 172.21.15.33:0/3002238578 learned_addr learned my addr 172.21.15.33:0/3002238578 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:04:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.104+0000 7f31a0fd3640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555f4e592210 0x555f4e5996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.104+0000 7f319bfff640 1 --2- 172.21.15.33:0/3002238578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f4e5918f0 0x555f4e59c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.104+0000 7f319bfff640 1 -- 172.21.15.33:0/3002238578 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555f4e592210 msgr2=0x555f4e5996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.104+0000 7f319bfff640 1 --2- 172.21.15.33:0/3002238578 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555f4e592210 0x555f4e5996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.104+0000 7f319bfff640 1 -- 172.21.15.33:0/3002238578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f4e5918f0 msgr2=0x555f4e59c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.104+0000 7f319bfff640 1 --2- 172.21.15.33:0/3002238578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f4e5918f0 0x555f4e59c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.104+0000 7f319bfff640 1 -- 172.21.15.33:0/3002238578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f4e2d5620 con 0x555f4e59cad0 2024-05-15T14:04:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.104+0000 7f31a0fd3640 1 --2- 172.21.15.33:0/3002238578 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555f4e592210 0x555f4e5996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:04:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.105+0000 7f319bfff640 1 --2- 172.21.15.33:0/3002238578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f4e59cad0 0x555f4e599bf0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3188009b20 tx=0x7f318801c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6876fa111486ff5 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.105+0000 7f317bfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:04:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.105+0000 7f317bfff640 1 -- 172.21.15.33:0/3002238578 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3188013cb0 con 0x555f4e59cad0 2024-05-15T14:04:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.105+0000 7f317bfff640 1 -- 172.21.15.33:0/3002238578 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f3188005120 con 0x555f4e59cad0 2024-05-15T14:04:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.105+0000 7f31a602f740 1 -- 172.21.15.33:0/3002238578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3188013e50 con 0x555f4e59cad0 2024-05-15T14:04:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.105+0000 7f317bfff640 1 -- 172.21.15.33:0/3002238578 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3188005320 con 0x555f4e59cad0 2024-05-15T14:04:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.105+0000 7f31a602f740 10 client.15783 fetch_fsmap learned FSMap version 27 2024-05-15T14:04:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.105+0000 7f31a602f740 1 -- 172.21.15.33:0/3002238578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555f4e5ab140 con 0x555f4e59cad0 2024-05-15T14:04:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.105+0000 7f317bfff640 1 -- 172.21.15.33:0/3002238578 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3188013e50 con 0x555f4e59cad0 2024-05-15T14:04:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.105+0000 7f317bfff640 1 -- 172.21.15.33:0/3002238578 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f3188020740 con 0x555f4e59cad0 2024-05-15T14:04:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.105+0000 7f31a602f740 10 client.15783 fetch_fsmap finished waiting for FSMap version 27 2024-05-15T14:04:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.105+0000 7f31a602f740 10 client.15783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:04:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.105+0000 7f31a602f740 20 client.15783 populate_metadata read hostname 'smithi033' 2024-05-15T14:04:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.105+0000 7f31a602f740 1 --2- 172.21.15.33:0/3002238578 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x555f4e5ac490 0x555f4e5ae880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.105+0000 7f31a602f740 4 client.15783 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:04:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.105+0000 7f31a602f740 1 -- 172.21.15.33:0/3002238578 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f4e506b80 con 0x555f4e5ac490 2024-05-15T14:04:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.106+0000 7f31a0fd3640 1 --2- 172.21.15.33:0/3002238578 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x555f4e5ac490 0x555f4e5ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.106+0000 7f31a0fd3640 1 --2- 172.21.15.33:0/3002238578 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x555f4e5ac490 0x555f4e5ae880 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64a2a532518e05ec server_cookie=e6292c0b99745c0c in_seq=0 out_seq=0 2024-05-15T14:04:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.106+0000 7f317bfff640 10 client.15783 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:04:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.107+0000 7f317bfff640 1 -- 172.21.15.33:0/3002238578 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555f4e506b80 con 0x555f4e5ac490 2024-05-15T14:04:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.107+0000 7f317bfff640 10 client.15783 handle_command_reply: tid=0 2024-05-15T14:04:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.107+0000 7f31a602f740 1 client.15783 shutdown 2024-05-15T14:04:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.107+0000 7f31a602f740 1 -- 172.21.15.33:0/3002238578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f4e59cad0 msgr2=0x555f4e599bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.107+0000 7f31a602f740 1 --2- 172.21.15.33:0/3002238578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f4e59cad0 0x555f4e599bf0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3188009b20 tx=0x7f318801c040 comp rx=0 tx=0).stop 2024-05-15T14:04:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.107+0000 7f31a602f740 1 -- 172.21.15.33:0/3002238578 shutdown_connections 2024-05-15T14:04:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.107+0000 7f31a602f740 1 --2- 172.21.15.33:0/3002238578 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x555f4e5ac490 0x555f4e5ae880 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.107+0000 7f31a602f740 1 --2- 172.21.15.33:0/3002238578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f4e59cad0 0x555f4e599bf0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.107+0000 7f31a602f740 1 --2- 172.21.15.33:0/3002238578 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555f4e592210 0x555f4e5996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.107+0000 7f317bfff640 0 client.15783 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:04:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.107+0000 7f31a602f740 1 --2- 172.21.15.33:0/3002238578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f4e5918f0 0x555f4e59c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.107+0000 7f31a602f740 1 -- 172.21.15.33:0/3002238578 >> 172.21.15.33:0/3002238578 conn(0x555f4e58de00 msgr2=0x555f4e58d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.107+0000 7f31a602f740 1 -- 172.21.15.33:0/3002238578 shutdown_connections 2024-05-15T14:04:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.107+0000 7f31a602f740 1 -- 172.21.15.33:0/3002238578 wait complete. 2024-05-15T14:04:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.107+0000 7f31a602f740 20 client.15783 trim_cache size 0 max 16384 2024-05-15T14:04:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.122+0000 7f31a602f740 1 Processor -- start 2024-05-15T14:04:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.122+0000 7f31a602f740 1 -- start start 2024-05-15T14:04:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.122+0000 7f31a602f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555f4e5918f0 0x555f4e5a8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.122+0000 7f31a602f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f4e592210 0x555f4e59bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.122+0000 7f31a602f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f4e59cad0 0x555f4e5b4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.122+0000 7f31a602f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1b1060 con 0x555f4e59cad0 2024-05-15T14:04:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.122+0000 7f31a602f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1ae8d0 con 0x555f4e592210 2024-05-15T14:04:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.122+0000 7f31a602f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1ac720 con 0x555f4e5918f0 2024-05-15T14:04:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.123+0000 7f319bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555f4e5918f0 0x555f4e5a8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.123+0000 7f31a0fd3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f4e592210 0x555f4e59bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.123+0000 7f31a0fd3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f4e592210 0x555f4e59bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:52304/0 (socket says 172.21.15.33:52304) 2024-05-15T14:04:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.123+0000 7f319bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555f4e5918f0 0x555f4e5a8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:42548/0 (socket says 172.21.15.33:42548) 2024-05-15T14:04:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.123+0000 7f31a0fd3640 1 -- 172.21.15.33:0/3130116088 learned_addr learned my addr 172.21.15.33:0/3130116088 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:04:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.123+0000 7f319bfff640 1 --2- 172.21.15.33:0/3130116088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f4e59cad0 0x555f4e5b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.123+0000 7f31a0fd3640 1 -- 172.21.15.33:0/3130116088 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555f4e5918f0 msgr2=0x555f4e5a8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.123+0000 7f31a0fd3640 1 --2- 172.21.15.33:0/3130116088 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555f4e5918f0 0x555f4e5a8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.123+0000 7f31a0fd3640 1 -- 172.21.15.33:0/3130116088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f4e59cad0 msgr2=0x555f4e5b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.123+0000 7f31a0fd3640 1 --2- 172.21.15.33:0/3130116088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f4e59cad0 0x555f4e5b4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.123+0000 7f31a0fd3640 1 -- 172.21.15.33:0/3130116088 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f4e59a1f0 con 0x555f4e592210 2024-05-15T14:04:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.123+0000 7f31a0fd3640 1 --2- 172.21.15.33:0/3130116088 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f4e592210 0x555f4e59bef0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3190032fd0 tx=0x7f3190005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9303bdc4e9330c4a server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.123+0000 7f31867fc640 1 -- 172.21.15.33:0/3130116088 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319004b070 con 0x555f4e592210 2024-05-15T14:04:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.123+0000 7f31867fc640 1 -- 172.21.15.33:0/3130116088 <== mon.2 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3190005040 con 0x555f4e592210 2024-05-15T14:04:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.123+0000 7f31867fc640 1 -- 172.21.15.33:0/3130116088 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3190035040 con 0x555f4e592210 2024-05-15T14:04:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.124+0000 7f31a602f740 1 -- 172.21.15.33:0/3130116088 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f4e592210 msgr2=0x555f4e59bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.124+0000 7f31a602f740 1 --2- 172.21.15.33:0/3130116088 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f4e592210 0x555f4e59bef0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3190032fd0 tx=0x7f3190005b50 comp rx=0 tx=0).stop 2024-05-15T14:04:08.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.124+0000 7f31a602f740 1 -- 172.21.15.33:0/3130116088 shutdown_connections 2024-05-15T14:04:08.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.124+0000 7f31a602f740 1 --2- 172.21.15.33:0/3130116088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f4e59cad0 0x555f4e5b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:08.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.124+0000 7f31a602f740 1 --2- 172.21.15.33:0/3130116088 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f4e592210 0x555f4e59bef0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:08.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.124+0000 7f31a602f740 1 --2- 172.21.15.33:0/3130116088 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555f4e5918f0 0x555f4e5a8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:08.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.124+0000 7f31a602f740 1 -- 172.21.15.33:0/3130116088 >> 172.21.15.33:0/3130116088 conn(0x555f4e58de00 msgr2=0x555f4e594ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:08.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.124+0000 7f31a602f740 1 -- 172.21.15.33:0/3130116088 shutdown_connections 2024-05-15T14:04:08.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.124+0000 7f31a602f740 1 -- 172.21.15.33:0/3130116088 wait complete. 2024-05-15T14:04:08.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.124+0000 7f31a602f740 1 -- start start 2024-05-15T14:04:08.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.124+0000 7f31a602f740 1 Processor -- start 2024-05-15T14:04:08.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.125+0000 7f31a602f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f4e5918f0 0x555f4e59ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:08.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.125+0000 7f31a602f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f4e592210 0x555f4e5a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:08.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.125+0000 7f31a602f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555f4e59cad0 0x555f4e60ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:08.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.125+0000 7f31a602f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1acd90 con 0x555f4e5918f0 2024-05-15T14:04:08.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.125+0000 7f31a602f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1ae200 con 0x555f4e592210 2024-05-15T14:04:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.125+0000 7f31a602f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1ad460 con 0x555f4e59cad0 2024-05-15T14:04:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.125+0000 7f31a602f740 10 client.0 _ll_register_callbacks cb 0x7f3188013660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:04:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.125+0000 7f319bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f4e5918f0 0x555f4e59ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.125+0000 7f31a0fd3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f4e592210 0x555f4e5a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.125+0000 7f31a0fd3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f4e592210 0x555f4e5a04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:52308/0 (socket says 172.21.15.33:52308) 2024-05-15T14:04:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.125+0000 7f319bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f4e5918f0 0x555f4e59ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58034/0 (socket says 172.21.15.33:58034) 2024-05-15T14:04:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.125+0000 7f31a0fd3640 1 -- 172.21.15.33:0/2440087750 learned_addr learned my addr 172.21.15.33:0/2440087750 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:04:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.125+0000 7f319bfff640 1 --2- 172.21.15.33:0/2440087750 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555f4e59cad0 0x555f4e60ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.125+0000 7f31a0fd3640 1 -- 172.21.15.33:0/2440087750 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555f4e59cad0 msgr2=0x555f4e60ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.125+0000 7f31a0fd3640 1 --2- 172.21.15.33:0/2440087750 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555f4e59cad0 0x555f4e60ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.125+0000 7f31a0fd3640 1 -- 172.21.15.33:0/2440087750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f4e5918f0 msgr2=0x555f4e59ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.125+0000 7f31a0fd3640 1 --2- 172.21.15.33:0/2440087750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f4e5918f0 0x555f4e59ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.125+0000 7f31a0fd3640 1 -- 172.21.15.33:0/2440087750 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f4e60f1e0 con 0x555f4e592210 2024-05-15T14:04:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.125+0000 7f31a0fd3640 1 --2- 172.21.15.33:0/2440087750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f4e592210 0x555f4e5a04b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f319002a9a0 tx=0x7f3190005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=428a6d1b91d9e74d server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.125+0000 7f317bfff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-05-15T14:04:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.125+0000 7f317bfff640 1 -- 172.21.15.33:0/2440087750 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3190035050 con 0x555f4e592210 2024-05-15T14:04:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.125+0000 7f31a602f740 1 -- 172.21.15.33:0/2440087750 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3190005040 con 0x555f4e592210 2024-05-15T14:04:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.125+0000 7f317bfff640 1 -- 172.21.15.33:0/2440087750 <== mon.2 v2:172.21.15.52:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f31900051f0 con 0x555f4e592210 2024-05-15T14:04:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.126+0000 7f317bfff640 1 -- 172.21.15.33:0/2440087750 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f319005c050 con 0x555f4e592210 2024-05-15T14:04:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.126+0000 7f31a602f740 10 client.25382 fetch_fsmap learned FSMap version 27 2024-05-15T14:04:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.126+0000 7f31a602f740 1 -- 172.21.15.33:0/2440087750 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555f4e6165c0 con 0x555f4e592210 2024-05-15T14:04:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.126+0000 7f317bfff640 1 -- 172.21.15.33:0/2440087750 <== mon.2 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3190005040 con 0x555f4e592210 2024-05-15T14:04:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.126+0000 7f317bfff640 1 -- 172.21.15.33:0/2440087750 <== mon.2 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f31900463d0 con 0x555f4e592210 2024-05-15T14:04:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.126+0000 7f31a602f740 10 client.25382 fetch_fsmap finished waiting for FSMap version 27 2024-05-15T14:04:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.126+0000 7f31a602f740 10 client.25382 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:04:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.126+0000 7f31a602f740 20 client.25382 populate_metadata read hostname 'smithi033' 2024-05-15T14:04:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.126+0000 7f31a602f740 1 --2- 172.21.15.33:0/2440087750 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x555f4e616c90 0x555f4e6190b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.126+0000 7f31a602f740 4 client.25382 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:04:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.126+0000 7f31a602f740 1 -- 172.21.15.33:0/2440087750 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555f4e500360 con 0x555f4e616c90 2024-05-15T14:04:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.126+0000 7f319bfff640 1 --2- 172.21.15.33:0/2440087750 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x555f4e616c90 0x555f4e6190b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.126+0000 7f319bfff640 1 --2- 172.21.15.33:0/2440087750 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x555f4e616c90 0x555f4e6190b0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73fe2df81900f0e9 server_cookie=1091ed20f686c130 in_seq=0 out_seq=0 2024-05-15T14:04:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.126+0000 7f317bfff640 10 client.25382 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:04:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.191+0000 7f317bfff640 1 -- 172.21.15.33:0/2440087750 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555f4e500360 con 0x555f4e616c90 2024-05-15T14:04:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.191+0000 7f317bfff640 10 client.25382 handle_command_reply: tid=0 2024-05-15T14:04:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.191+0000 7f31a602f740 1 client.25382 shutdown 2024-05-15T14:04:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.191+0000 7f31a602f740 1 -- 172.21.15.33:0/2440087750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f4e592210 msgr2=0x555f4e5a04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.191+0000 7f31a602f740 1 --2- 172.21.15.33:0/2440087750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f4e592210 0x555f4e5a04b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f319002a9a0 tx=0x7f3190005850 comp rx=0 tx=0).stop 2024-05-15T14:04:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.191+0000 7f31a602f740 1 -- 172.21.15.33:0/2440087750 shutdown_connections 2024-05-15T14:04:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.191+0000 7f31a602f740 1 --2- 172.21.15.33:0/2440087750 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x555f4e616c90 0x555f4e6190b0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.191+0000 7f31a602f740 1 --2- 172.21.15.33:0/2440087750 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555f4e59cad0 0x555f4e60ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.191+0000 7f31a602f740 1 --2- 172.21.15.33:0/2440087750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f4e592210 0x555f4e5a04b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.191+0000 7f31a602f740 1 --2- 172.21.15.33:0/2440087750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555f4e5918f0 0x555f4e59ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.191+0000 7f317bfff640 0 client.25382 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:04:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.191+0000 7f31a602f740 1 -- 172.21.15.33:0/2440087750 >> 172.21.15.33:0/2440087750 conn(0x555f4e58de00 msgr2=0x555f4e5932a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.191+0000 7f31a602f740 1 -- 172.21.15.33:0/2440087750 shutdown_connections 2024-05-15T14:04:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.191+0000 7f31a602f740 1 -- 172.21.15.33:0/2440087750 wait complete. 2024-05-15T14:04:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.191+0000 7f31a602f740 20 client.25382 trim_cache size 0 max 16384 2024-05-15T14:04:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:04:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:04:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:04:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.192+0000 7f31a3237640 1 -- 172.21.15.33:0/2811685229 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f318c0b29d0 msgr2=0x7f318c0b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.192+0000 7f31a3237640 1 --2- 172.21.15.33:0/2811685229 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f318c0b29d0 0x7f318c0b4e90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f31940a5dc0 tx=0x7f3194076670 comp rx=0 tx=0).stop 2024-05-15T14:04:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.192+0000 7f31a3237640 1 -- 172.21.15.33:0/2811685229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f319c10d090 msgr2=0x7f319c19db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.192+0000 7f31a3237640 1 --2- 172.21.15.33:0/2811685229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f319c10d090 0x7f319c19db70 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f319400e9d0 tx=0x7f319400ee90 comp rx=0 tx=0).stop 2024-05-15T14:04:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.192+0000 7f31a3237640 1 -- 172.21.15.33:0/2811685229 shutdown_connections 2024-05-15T14:04:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.192+0000 7f31a3237640 1 --2- 172.21.15.33:0/2811685229 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f318c0b29d0 0x7f318c0b4e90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.192+0000 7f31a3237640 1 --2- 172.21.15.33:0/2811685229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f319c10d090 0x7f319c19db70 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.192+0000 7f31a3237640 1 --2- 172.21.15.33:0/2811685229 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f319c1045e0 0x7f319c1997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.192+0000 7f31a3237640 1 --2- 172.21.15.33:0/2811685229 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f319c103cc0 0x7f319c1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.192+0000 7f31a3237640 1 -- 172.21.15.33:0/2811685229 >> 172.21.15.33:0/2811685229 conn(0x7f319c078a70 msgr2=0x7f319c0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.192+0000 7f31a3237640 1 -- 172.21.15.33:0/2811685229 shutdown_connections 2024-05-15T14:04:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:08.192+0000 7f31a3237640 1 -- 172.21.15.33:0/2811685229 wait complete. 2024-05-15T14:04:08.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:08 smithi033 ceph-mon[31650]: pgmap v117: 97 pgs: 97 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.0 MiB/s wr, 372 op/s 2024-05-15T14:04:08.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:08 smithi052 ceph-mon[36190]: pgmap v117: 97 pgs: 97 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.0 MiB/s wr, 372 op/s 2024-05-15T14:04:08.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:08 smithi081 ceph-mon[36434]: pgmap v117: 97 pgs: 97 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.0 MiB/s wr, 372 op/s 2024-05-15T14:04:09.052 INFO:tasks.workunit.client.0.smithi033.stdout: 1 195780 10.32 MB/sec warmup 104 sec latency 29.876 ms 2024-05-15T14:04:10.051 INFO:tasks.workunit.client.0.smithi033.stdout: 1 198760 10.39 MB/sec warmup 105 sec latency 17.256 ms 2024-05-15T14:04:10.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:10 smithi033 ceph-mon[31650]: pgmap v118: 97 pgs: 97 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 217 op/s 2024-05-15T14:04:10.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:10 smithi052 ceph-mon[36190]: pgmap v118: 97 pgs: 97 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 217 op/s 2024-05-15T14:04:10.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:10 smithi081 ceph-mon[36434]: pgmap v118: 97 pgs: 97 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 217 op/s 2024-05-15T14:04:11.052 INFO:tasks.workunit.client.0.smithi033.stdout: 1 200158 10.35 MB/sec warmup 106 sec latency 29.050 ms 2024-05-15T14:04:12.052 INFO:tasks.workunit.client.0.smithi033.stdout: 1 202559 10.38 MB/sec warmup 107 sec latency 44.617 ms 2024-05-15T14:04:12.294 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.5 2024-05-15T14:04:12.295 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T14:04:12.561 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:04:12.669 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:12 smithi033 ceph-mon[31650]: pgmap v119: 97 pgs: 97 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 217 op/s 2024-05-15T14:04:12.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:12 smithi052 ceph-mon[36190]: pgmap v119: 97 pgs: 97 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 217 op/s 2024-05-15T14:04:12.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:12 smithi081 ceph-mon[36434]: pgmap v119: 97 pgs: 97 active+clean; 60 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 217 op/s 2024-05-15T14:04:13.052 INFO:tasks.workunit.client.0.smithi033.stdout: 1 204673 10.36 MB/sec warmup 108 sec latency 13.759 ms 2024-05-15T14:04:14.052 INFO:tasks.workunit.client.0.smithi033.stdout: 1 206519 10.37 MB/sec warmup 109 sec latency 12.128 ms 2024-05-15T14:04:14.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.292+0000 7ff5d5d01700 1 -- 172.21.15.33:0/3790414361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5d01095b0 msgr2=0x7ff5d0109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:14.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.292+0000 7ff5d5d01700 1 --2- 172.21.15.33:0/3790414361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5d01095b0 0x7ff5d0109990 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0009990 tx=0x7ff5c0009ca0 comp rx=0 tx=0).stop 2024-05-15T14:04:14.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.293+0000 7ff5d5d01700 1 -- 172.21.15.33:0/3790414361 shutdown_connections 2024-05-15T14:04:14.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.293+0000 7ff5d5d01700 1 --2- 172.21.15.33:0/3790414361 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff5d006bbe0 0x7ff5d01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:14.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.293+0000 7ff5d5d01700 1 --2- 172.21.15.33:0/3790414361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5d01095b0 0x7ff5d0109990 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:14.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.293+0000 7ff5d5d01700 1 --2- 172.21.15.33:0/3790414361 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5d006b220 0x7ff5d006b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:14.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.293+0000 7ff5d5d01700 1 -- 172.21.15.33:0/3790414361 >> 172.21.15.33:0/3790414361 conn(0x7ff5d00793f0 msgr2=0x7ff5d0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:14.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.293+0000 7ff5d5d01700 1 -- 172.21.15.33:0/3790414361 shutdown_connections 2024-05-15T14:04:14.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.293+0000 7ff5d5d01700 1 -- 172.21.15.33:0/3790414361 wait complete. 2024-05-15T14:04:14.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.293+0000 7ff5d5d01700 1 Processor -- start 2024-05-15T14:04:14.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.294+0000 7ff5d5d01700 1 -- start start 2024-05-15T14:04:14.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.294+0000 7ff5d5d01700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff5d006b220 0x7ff5d019bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:14.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.294+0000 7ff5d5d01700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5d006bbe0 0x7ff5d019c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:14.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.294+0000 7ff5d5d01700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5d01095b0 0x7ff5d01a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:14.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.294+0000 7ff5d5d01700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d0113740 con 0x7ff5d006bbe0 2024-05-15T14:04:14.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.294+0000 7ff5d5d01700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d01135c0 con 0x7ff5d01095b0 2024-05-15T14:04:14.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.294+0000 7ff5d5d01700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d01138c0 con 0x7ff5d006b220 2024-05-15T14:04:14.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.294+0000 7ff5ceffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5d006bbe0 0x7ff5d019c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:14.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.294+0000 7ff5ceffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5d006bbe0 0x7ff5d019c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39182/0 (socket says 172.21.15.33:39182) 2024-05-15T14:04:14.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.294+0000 7ff5ceffd700 1 -- 172.21.15.33:0/444204274 learned_addr learned my addr 172.21.15.33:0/444204274 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:04:14.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.294+0000 7ff5cf7fe700 1 --2- 172.21.15.33:0/444204274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff5d006b220 0x7ff5d019bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:14.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.294+0000 7ff5cffff700 1 --2- 172.21.15.33:0/444204274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5d01095b0 0x7ff5d01a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:14.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.294+0000 7ff5ceffd700 1 -- 172.21.15.33:0/444204274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff5d006b220 msgr2=0x7ff5d019bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:14.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.294+0000 7ff5ceffd700 1 --2- 172.21.15.33:0/444204274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff5d006b220 0x7ff5d019bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:14.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.294+0000 7ff5ceffd700 1 -- 172.21.15.33:0/444204274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5d01095b0 msgr2=0x7ff5d01a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:14.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.294+0000 7ff5ceffd700 1 --2- 172.21.15.33:0/444204274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5d01095b0 0x7ff5d01a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:14.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.294+0000 7ff5ceffd700 1 -- 172.21.15.33:0/444204274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d01a0f20 con 0x7ff5d006bbe0 2024-05-15T14:04:14.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.294+0000 7ff5cf7fe700 1 --2- 172.21.15.33:0/444204274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff5d006b220 0x7ff5d019bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:04:14.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.294+0000 7ff5cffff700 1 --2- 172.21.15.33:0/444204274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5d01095b0 0x7ff5d01a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:04:14.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.295+0000 7ff5ceffd700 1 --2- 172.21.15.33:0/444204274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5d006bbe0 0x7ff5d019c450 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff5c400e9a0 tx=0x7ff5c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:14.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.295+0000 7ff5ccff9700 1 -- 172.21.15.33:0/444204274 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c400cbd0 con 0x7ff5d006bbe0 2024-05-15T14:04:14.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.295+0000 7ff5ccff9700 1 -- 172.21.15.33:0/444204274 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5c400cd70 con 0x7ff5d006bbe0 2024-05-15T14:04:14.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.295+0000 7ff5d5d01700 1 -- 172.21.15.33:0/444204274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5d01a1210 con 0x7ff5d006bbe0 2024-05-15T14:04:14.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.295+0000 7ff5ccff9700 1 -- 172.21.15.33:0/444204274 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4018680 con 0x7ff5d006bbe0 2024-05-15T14:04:14.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.295+0000 7ff5d5d01700 1 -- 172.21.15.33:0/444204274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5d01a8a90 con 0x7ff5d006bbe0 2024-05-15T14:04:14.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.296+0000 7ff5d5d01700 1 -- 172.21.15.33:0/444204274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5d006a230 con 0x7ff5d006bbe0 2024-05-15T14:04:14.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.297+0000 7ff5ccff9700 1 -- 172.21.15.33:0/444204274 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff5c4010430 con 0x7ff5d006bbe0 2024-05-15T14:04:14.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.298+0000 7ff5ccff9700 1 --2- 172.21.15.33:0/444204274 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ff5b80b29f0 0x7ff5b80b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:14.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.298+0000 7ff5cf7fe700 1 --2- 172.21.15.33:0/444204274 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ff5b80b29f0 0x7ff5b80b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:14.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.298+0000 7ff5ccff9700 1 -- 172.21.15.33:0/444204274 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7ff5c40e2890 con 0x7ff5d006bbe0 2024-05-15T14:04:14.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.299+0000 7ff5cf7fe700 1 --2- 172.21.15.33:0/444204274 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ff5b80b29f0 0x7ff5b80b4eb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000b2e0 tx=0x7ff5c001a040 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:14.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.303+0000 7ff5ccff9700 1 -- 172.21.15.33:0/444204274 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff5c40aa280 con 0x7ff5d006bbe0 2024-05-15T14:04:14.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:14 smithi033 ceph-mon[31650]: pgmap v120: 97 pgs: 97 active+clean; 56 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.1 MiB/s wr, 390 op/s 2024-05-15T14:04:14.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.583+0000 7ff5d5d01700 1 -- 172.21.15.33:0/444204274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5d01a8c60 con 0x7ff5d006bbe0 2024-05-15T14:04:14.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.584+0000 7ff5ccff9700 1 -- 172.21.15.33:0/444204274 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11858 (secure 0 0 0) 0x7ff5c40a9b80 con 0x7ff5d006bbe0 2024-05-15T14:04:14.586 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T14:04:14.586 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2030117633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2030117633},{"type":"v1","addr":"172.21.15.33:6839","nonce":2030117633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1918979061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1918979061},{"type":"v1","addr":"172.21.15.33:6841","nonce":1918979061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T13:57:19.056417+0000","modified":"2024-05-15T14:03:42.376493+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24463,"mds_2":24461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/1629697841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1629697841},{"type":"v1","addr":"172.21.15.52:6837","nonce":1629697841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.52:6839/2696044471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2696044471},{"type":"v1","addr":"172.21.15.52:6839","nonce":2696044471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.81:6839/1554395940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":1554395940},{"type":"v1","addr":"172.21.15.81:6839","nonce":1554395940}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25393":{"gid":25393,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/727032511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":727032511},{"type":"v1","addr":"172.21.15.81:6835","nonce":727032511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24463,24478,24461]},"id":1}]} 2024-05-15T14:04:14.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.586+0000 7ff5d5d01700 1 -- 172.21.15.33:0/444204274 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ff5b80b29f0 msgr2=0x7ff5b80b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:14.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.586+0000 7ff5d5d01700 1 --2- 172.21.15.33:0/444204274 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ff5b80b29f0 0x7ff5b80b4eb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000b2e0 tx=0x7ff5c001a040 comp rx=0 tx=0).stop 2024-05-15T14:04:14.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.586+0000 7ff5d5d01700 1 -- 172.21.15.33:0/444204274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5d006bbe0 msgr2=0x7ff5d019c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:14.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.586+0000 7ff5d5d01700 1 --2- 172.21.15.33:0/444204274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5d006bbe0 0x7ff5d019c450 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff5c400e9a0 tx=0x7ff5c400ecb0 comp rx=0 tx=0).stop 2024-05-15T14:04:14.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.587+0000 7ff5d5d01700 1 -- 172.21.15.33:0/444204274 shutdown_connections 2024-05-15T14:04:14.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.587+0000 7ff5d5d01700 1 --2- 172.21.15.33:0/444204274 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ff5b80b29f0 0x7ff5b80b4eb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:14.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.587+0000 7ff5d5d01700 1 --2- 172.21.15.33:0/444204274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff5d01095b0 0x7ff5d01a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:14.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.587+0000 7ff5d5d01700 1 --2- 172.21.15.33:0/444204274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff5d006b220 0x7ff5d019bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:14.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.587+0000 7ff5d5d01700 1 --2- 172.21.15.33:0/444204274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5d006bbe0 0x7ff5d019c450 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:14.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.587+0000 7ff5d5d01700 1 -- 172.21.15.33:0/444204274 >> 172.21.15.33:0/444204274 conn(0x7ff5d00793f0 msgr2=0x7ff5d0100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:14.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.587+0000 7ff5d5d01700 1 -- 172.21.15.33:0/444204274 shutdown_connections 2024-05-15T14:04:14.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:14.587+0000 7ff5d5d01700 1 -- 172.21.15.33:0/444204274 wait complete. 2024-05-15T14:04:14.589 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-05-15T14:04:14.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:14 smithi052 ceph-mon[36190]: pgmap v120: 97 pgs: 97 active+clean; 56 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.1 MiB/s wr, 390 op/s 2024-05-15T14:04:14.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:14 smithi081 ceph-mon[36434]: pgmap v120: 97 pgs: 97 active+clean; 56 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.1 MiB/s wr, 390 op/s 2024-05-15T14:04:15.052 INFO:tasks.workunit.client.0.smithi033.stdout: 1 209358 10.43 MB/sec warmup 110 sec latency 38.098 ms 2024-05-15T14:04:15.341 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24478 quiesce db / --timeout 90 --expiration 146.5 --await-for 45.0' 2024-05-15T14:04:15.341 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24478 quiesce db / --timeout 90 --expiration 146.5 --await-for 45.0 2024-05-15T14:04:15.605 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:04:15.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:15 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/444204274' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:04:15.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:15 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/444204274' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:04:15.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:15 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/444204274' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:04:16.052 INFO:tasks.workunit.client.0.smithi033.stdout: 1 210869 10.40 MB/sec warmup 111 sec latency 14.284 ms 2024-05-15T14:04:16.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:16 smithi033 ceph-mon[31650]: pgmap v121: 97 pgs: 97 active+clean; 56 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 282 op/s 2024-05-15T14:04:16.741 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:16 smithi052 ceph-mon[36190]: pgmap v121: 97 pgs: 97 active+clean; 56 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 282 op/s 2024-05-15T14:04:16.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:16 smithi081 ceph-mon[36434]: pgmap v121: 97 pgs: 97 active+clean; 56 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 282 op/s 2024-05-15T14:04:17.052 INFO:tasks.workunit.client.0.smithi033.stdout: 1 213258 10.43 MB/sec warmup 112 sec latency 18.491 ms 2024-05-15T14:04:17.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.448+0000 7f03a355b700 1 -- 172.21.15.33:0/2936439360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f039c108840 msgr2=0x7f039c110450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:17.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.448+0000 7f03a355b700 1 --2- 172.21.15.33:0/2936439360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f039c108840 0x7f039c110450 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f039400b0d0 tx=0x7f039400b3e0 comp rx=0 tx=0).stop 2024-05-15T14:04:17.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.448+0000 7f03a355b700 1 -- 172.21.15.33:0/2936439360 shutdown_connections 2024-05-15T14:04:17.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.448+0000 7f03a355b700 1 --2- 172.21.15.33:0/2936439360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f039c108840 0x7f039c110450 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:17.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.448+0000 7f03a355b700 1 --2- 172.21.15.33:0/2936439360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f039c103850 0x7f039c103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:17.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.448+0000 7f03a355b700 1 --2- 172.21.15.33:0/2936439360 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f039c104200 0x7f039c108160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:17.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.448+0000 7f03a355b700 1 -- 172.21.15.33:0/2936439360 >> 172.21.15.33:0/2936439360 conn(0x7f039c07a610 msgr2=0x7f039c07aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:17.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.448+0000 7f03a355b700 1 -- 172.21.15.33:0/2936439360 shutdown_connections 2024-05-15T14:04:17.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.448+0000 7f03a355b700 1 -- 172.21.15.33:0/2936439360 wait complete. 2024-05-15T14:04:17.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.449+0000 7f03a355b700 1 Processor -- start 2024-05-15T14:04:17.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.449+0000 7f03a355b700 1 -- start start 2024-05-15T14:04:17.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.449+0000 7f03a355b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f039c103850 0x7f039c075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:17.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.449+0000 7f03a355b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f039c104200 0x7f039c075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:17.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.449+0000 7f03a355b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f039c108840 0x7f039c0762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:17.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.449+0000 7f03a355b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f039c113710 con 0x7f039c103850 2024-05-15T14:04:17.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.449+0000 7f03a355b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f039c113590 con 0x7f039c104200 2024-05-15T14:04:17.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.449+0000 7f03a355b700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f039c113890 con 0x7f039c108840 2024-05-15T14:04:17.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.449+0000 7f03a12f7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f039c103850 0x7f039c075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:17.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.449+0000 7f03a12f7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f039c103850 0x7f039c075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39194/0 (socket says 172.21.15.33:39194) 2024-05-15T14:04:17.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.449+0000 7f03a12f7700 1 -- 172.21.15.33:0/2497360160 learned_addr learned my addr 172.21.15.33:0/2497360160 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:04:17.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.449+0000 7f03a0af6700 1 --2- 172.21.15.33:0/2497360160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f039c104200 0x7f039c075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:17.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.449+0000 7f03a1af8700 1 --2- 172.21.15.33:0/2497360160 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f039c108840 0x7f039c0762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:17.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.450+0000 7f03a12f7700 1 -- 172.21.15.33:0/2497360160 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f039c108840 msgr2=0x7f039c0762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:17.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.450+0000 7f03a12f7700 1 --2- 172.21.15.33:0/2497360160 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f039c108840 0x7f039c0762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:17.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.450+0000 7f03a12f7700 1 -- 172.21.15.33:0/2497360160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f039c104200 msgr2=0x7f039c075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:17.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.450+0000 7f03a12f7700 1 --2- 172.21.15.33:0/2497360160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f039c104200 0x7f039c075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:17.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.450+0000 7f03a12f7700 1 -- 172.21.15.33:0/2497360160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039c1a8050 con 0x7f039c103850 2024-05-15T14:04:17.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.450+0000 7f03a0af6700 1 --2- 172.21.15.33:0/2497360160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f039c104200 0x7f039c075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:04:17.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.450+0000 7f03a12f7700 1 --2- 172.21.15.33:0/2497360160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f039c103850 0x7f039c075840 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f039800e9a0 tx=0x7f039800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:17.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.450+0000 7f03927fc700 1 -- 172.21.15.33:0/2497360160 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039800ca10 con 0x7f039c103850 2024-05-15T14:04:17.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.450+0000 7f03a355b700 1 -- 172.21.15.33:0/2497360160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f039c1a8340 con 0x7f039c103850 2024-05-15T14:04:17.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.450+0000 7f03927fc700 1 -- 172.21.15.33:0/2497360160 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f039800cbb0 con 0x7f039c103850 2024-05-15T14:04:17.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.450+0000 7f03927fc700 1 -- 172.21.15.33:0/2497360160 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0398018680 con 0x7f039c103850 2024-05-15T14:04:17.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.450+0000 7f03a355b700 1 -- 172.21.15.33:0/2497360160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f039c1a8880 con 0x7f039c103850 2024-05-15T14:04:17.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.453+0000 7f03927fc700 1 -- 172.21.15.33:0/2497360160 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0398010430 con 0x7f039c103850 2024-05-15T14:04:17.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.454+0000 7f03927fc700 1 --2- 172.21.15.33:0/2497360160 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f03880b2aa0 0x7f03880b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:17.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.454+0000 7f03a0af6700 1 --2- 172.21.15.33:0/2497360160 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f03880b2aa0 0x7f03880b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:17.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.454+0000 7f03927fc700 1 -- 172.21.15.33:0/2497360160 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f03980e27c0 con 0x7f039c103850 2024-05-15T14:04:17.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.454+0000 7f03addf1b80 1 Processor -- start 2024-05-15T14:04:17.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.454+0000 7f03addf1b80 1 -- start start 2024-05-15T14:04:17.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.454+0000 7f03a0af6700 1 --2- 172.21.15.33:0/2497360160 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f03880b2aa0 0x7f03880b4f60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f038c009680 tx=0x7f038c009260 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:17.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.455+0000 7f03addf1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5627e96b9e30 0x5627e96ba210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:17.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.455+0000 7f03addf1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e96ba750 0x5627e96b4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:17.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.455+0000 7f03addf1b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627e96b4b60 0x5627e96b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:17.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.455+0000 7f03addf1b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5627e9612e20 con 0x5627e96b4b60 2024-05-15T14:04:17.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.455+0000 7f03addf1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5627e96029c0 con 0x5627e96b9e30 2024-05-15T14:04:17.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.455+0000 7f03addf1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5627e9606a00 con 0x5627e96ba750 2024-05-15T14:04:17.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.455+0000 7f03a0af6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e96ba750 0x5627e96b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:17.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.455+0000 7f03a12f7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5627e96b9e30 0x5627e96ba210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:17.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.455+0000 7f03a0af6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e96ba750 0x5627e96b4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:41646/0 (socket says 172.21.15.33:41646) 2024-05-15T14:04:17.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.455+0000 7f03a12f7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5627e96b9e30 0x5627e96ba210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:53052/0 (socket says 172.21.15.33:53052) 2024-05-15T14:04:17.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.455+0000 7f03a0af6700 1 -- 172.21.15.33:0/2429995647 learned_addr learned my addr 172.21.15.33:0/2429995647 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:04:17.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.455+0000 7f03a1af8700 1 --2- 172.21.15.33:0/2429995647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627e96b4b60 0x5627e96b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:17.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.455+0000 7f03a0af6700 1 -- 172.21.15.33:0/2429995647 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5627e96b9e30 msgr2=0x5627e96ba210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:17.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.455+0000 7f03a0af6700 1 --2- 172.21.15.33:0/2429995647 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5627e96b9e30 0x5627e96ba210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:17.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.455+0000 7f03a0af6700 1 -- 172.21.15.33:0/2429995647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627e96b4b60 msgr2=0x5627e96b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:17.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.455+0000 7f03a0af6700 1 --2- 172.21.15.33:0/2429995647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627e96b4b60 0x5627e96b4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:17.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.455+0000 7f03a0af6700 1 -- 172.21.15.33:0/2429995647 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627e9552a60 con 0x5627e96ba750 2024-05-15T14:04:17.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.455+0000 7f03a0af6700 1 --2- 172.21.15.33:0/2429995647 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e96ba750 0x5627e96b4540 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f038c00e8d0 tx=0x7f038c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2ea046ef7ede064 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:17.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.456+0000 7f03877fe700 1 -- 172.21.15.33:0/2429995647 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c029070 con 0x5627e96ba750 2024-05-15T14:04:17.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.456+0000 7f03877fe700 1 -- 172.21.15.33:0/2429995647 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f038c01c930 con 0x5627e96ba750 2024-05-15T14:04:17.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.456+0000 7f03877fe700 1 -- 172.21.15.33:0/2429995647 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c01aac0 con 0x5627e96ba750 2024-05-15T14:04:17.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.456+0000 7f03addf1b80 1 -- 172.21.15.33:0/2429995647 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e96ba750 msgr2=0x5627e96b4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:17.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.456+0000 7f03addf1b80 1 --2- 172.21.15.33:0/2429995647 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e96ba750 0x5627e96b4540 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f038c00e8d0 tx=0x7f038c014000 comp rx=0 tx=0).stop 2024-05-15T14:04:17.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.456+0000 7f03addf1b80 1 -- 172.21.15.33:0/2429995647 shutdown_connections 2024-05-15T14:04:17.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.456+0000 7f03addf1b80 1 --2- 172.21.15.33:0/2429995647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627e96b4b60 0x5627e96b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:17.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.456+0000 7f03addf1b80 1 --2- 172.21.15.33:0/2429995647 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5627e96b9e30 0x5627e96ba210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:17.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.456+0000 7f03addf1b80 1 --2- 172.21.15.33:0/2429995647 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e96ba750 0x5627e96b4540 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:17.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.456+0000 7f03addf1b80 1 -- 172.21.15.33:0/2429995647 >> 172.21.15.33:0/2429995647 conn(0x5627e95df3e0 msgr2=0x5627e96aed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:17.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.457+0000 7f03addf1b80 1 -- 172.21.15.33:0/2429995647 shutdown_connections 2024-05-15T14:04:17.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.457+0000 7f03addf1b80 1 -- 172.21.15.33:0/2429995647 wait complete. 2024-05-15T14:04:17.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.457+0000 7f03addf1b80 1 -- start start 2024-05-15T14:04:17.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.457+0000 7f03addf1b80 1 Processor -- start 2024-05-15T14:04:17.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.458+0000 7f03addf1b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627e96b4b60 0x5627e96c0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:17.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.458+0000 7f03addf1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e96b9e30 0x5627e96bd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:17.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.458+0000 7f03addf1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5627e96ba750 0x5627e96bdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:17.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.458+0000 7f03addf1b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5627e9509130 con 0x5627e96b4b60 2024-05-15T14:04:17.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.458+0000 7f03addf1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5627e95a4950 con 0x5627e96ba750 2024-05-15T14:04:17.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.458+0000 7f03addf1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5627e96c1190 con 0x5627e96b9e30 2024-05-15T14:04:17.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.458+0000 7f03addf1b80 10 client.0 _ll_register_callbacks cb 0x5627e9599710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:04:17.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.458+0000 7f03a12f7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627e96b4b60 0x5627e96c0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:17.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.458+0000 7f03a1af8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5627e96ba750 0x5627e96bdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:17.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.458+0000 7f03a12f7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627e96b4b60 0x5627e96c0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39202/0 (socket says 172.21.15.33:39202) 2024-05-15T14:04:17.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.458+0000 7f03a12f7700 1 -- 172.21.15.33:0/585731725 learned_addr learned my addr 172.21.15.33:0/585731725 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:04:17.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.458+0000 7f03a0af6700 1 --2- 172.21.15.33:0/585731725 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e96b9e30 0x5627e96bd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:17.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.458+0000 7f03a12f7700 1 -- 172.21.15.33:0/585731725 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e96b9e30 msgr2=0x5627e96bd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:17.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.458+0000 7f03a12f7700 1 --2- 172.21.15.33:0/585731725 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e96b9e30 0x5627e96bd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:17.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.458+0000 7f03a12f7700 1 -- 172.21.15.33:0/585731725 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5627e96ba750 msgr2=0x5627e96bdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:17.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.458+0000 7f03a12f7700 1 --2- 172.21.15.33:0/585731725 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5627e96ba750 0x5627e96bdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:17.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.458+0000 7f03a12f7700 1 -- 172.21.15.33:0/585731725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627e95a1820 con 0x5627e96b4b60 2024-05-15T14:04:17.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.458+0000 7f03a12f7700 1 --2- 172.21.15.33:0/585731725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627e96b4b60 0x5627e96c0410 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f039800c2d0 tx=0x7f03980a9f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa91d80defd3ff1 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:17.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.458+0000 7f0384ff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:04:17.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.459+0000 7f0384ff9700 1 -- 172.21.15.33:0/585731725 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03980a5040 con 0x5627e96b4b60 2024-05-15T14:04:17.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.459+0000 7f0384ff9700 1 -- 172.21.15.33:0/585731725 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f03980cc2d0 con 0x5627e96b4b60 2024-05-15T14:04:17.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.459+0000 7f03addf1b80 1 -- 172.21.15.33:0/585731725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f038c01c930 con 0x5627e96b4b60 2024-05-15T14:04:17.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.459+0000 7f0384ff9700 1 -- 172.21.15.33:0/585731725 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f03980a9560 con 0x5627e96b4b60 2024-05-15T14:04:17.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.459+0000 7f03addf1b80 10 client.15807 fetch_fsmap learned FSMap version 27 2024-05-15T14:04:17.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.459+0000 7f03addf1b80 1 -- 172.21.15.33:0/585731725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5627e96cb8a0 con 0x5627e96b4b60 2024-05-15T14:04:17.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.459+0000 7f0384ff9700 1 -- 172.21.15.33:0/585731725 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f038c01c930 con 0x5627e96b4b60 2024-05-15T14:04:17.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.459+0000 7f0384ff9700 1 -- 172.21.15.33:0/585731725 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f039806f2a0 con 0x5627e96b4b60 2024-05-15T14:04:17.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.459+0000 7f03addf1b80 10 client.15807 fetch_fsmap finished waiting for FSMap version 27 2024-05-15T14:04:17.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.459+0000 7f03addf1b80 10 client.15807 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:04:17.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.459+0000 7f03addf1b80 20 client.15807 populate_metadata read hostname 'smithi033' 2024-05-15T14:04:17.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.459+0000 7f03addf1b80 1 --2- 172.21.15.33:0/585731725 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5627e96cdee0 0x5627e96d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:17.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.459+0000 7f03addf1b80 4 client.15807 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:04:17.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.459+0000 7f03addf1b80 1 -- 172.21.15.33:0/585731725 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627e96804f0 con 0x5627e96cdee0 2024-05-15T14:04:17.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.459+0000 7f03a0af6700 1 --2- 172.21.15.33:0/585731725 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5627e96cdee0 0x5627e96d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:17.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.460+0000 7f03a0af6700 1 --2- 172.21.15.33:0/585731725 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5627e96cdee0 0x5627e96d02d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc89e3214f193ee4 server_cookie=4cdad318b147e403 in_seq=0 out_seq=0 2024-05-15T14:04:17.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.460+0000 7f0384ff9700 10 client.15807 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:04:17.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.461+0000 7f0384ff9700 1 -- 172.21.15.33:0/585731725 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5627e96804f0 con 0x5627e96cdee0 2024-05-15T14:04:17.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.461+0000 7f0384ff9700 10 client.15807 handle_command_reply: tid=0 2024-05-15T14:04:17.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.461+0000 7f03addf1b80 1 client.15807 shutdown 2024-05-15T14:04:17.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.461+0000 7f03addf1b80 1 -- 172.21.15.33:0/585731725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627e96b4b60 msgr2=0x5627e96c0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:17.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.461+0000 7f03addf1b80 1 --2- 172.21.15.33:0/585731725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627e96b4b60 0x5627e96c0410 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f039800c2d0 tx=0x7f03980a9f90 comp rx=0 tx=0).stop 2024-05-15T14:04:17.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.461+0000 7f03addf1b80 1 -- 172.21.15.33:0/585731725 shutdown_connections 2024-05-15T14:04:17.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.461+0000 7f03addf1b80 1 --2- 172.21.15.33:0/585731725 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5627e96ba750 0x5627e96bdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:17.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.461+0000 7f03addf1b80 1 --2- 172.21.15.33:0/585731725 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5627e96cdee0 0x5627e96d02d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:17.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.461+0000 7f03addf1b80 1 --2- 172.21.15.33:0/585731725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627e96b4b60 0x5627e96c0410 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:17.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.461+0000 7f03addf1b80 1 --2- 172.21.15.33:0/585731725 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e96b9e30 0x5627e96bd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:17.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.461+0000 7f03addf1b80 1 -- 172.21.15.33:0/585731725 >> 172.21.15.33:0/585731725 conn(0x5627e95df3e0 msgr2=0x5627e96a9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:17.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.461+0000 7f0384ff9700 0 client.15807 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:04:17.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.461+0000 7f03addf1b80 1 -- 172.21.15.33:0/585731725 shutdown_connections 2024-05-15T14:04:17.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.461+0000 7f03addf1b80 1 -- 172.21.15.33:0/585731725 wait complete. 2024-05-15T14:04:17.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.461+0000 7f03addf1b80 20 client.15807 trim_cache size 0 max 16384 2024-05-15T14:04:17.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.482+0000 7f03addf1b80 1 Processor -- start 2024-05-15T14:04:17.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.482+0000 7f03addf1b80 1 -- start start 2024-05-15T14:04:17.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.483+0000 7f03addf1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e96b9e30 0x5627e96ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:17.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.483+0000 7f03addf1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5627e96ba750 0x5627e96a9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:17.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.483+0000 7f03addf1b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627e96bf570 0x5627e96bf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:17.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.483+0000 7f03addf1b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f039c1b2930 con 0x5627e96bf570 2024-05-15T14:04:17.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.483+0000 7f03addf1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f039c1b0160 con 0x5627e96ba750 2024-05-15T14:04:17.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.483+0000 7f03addf1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f039c1adfb0 con 0x5627e96b9e30 2024-05-15T14:04:17.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.483+0000 7f03a1af8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627e96bf570 0x5627e96bf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:17.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.483+0000 7f03a1af8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627e96bf570 0x5627e96bf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39212/0 (socket says 172.21.15.33:39212) 2024-05-15T14:04:17.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.483+0000 7f03a1af8700 1 -- 172.21.15.33:0/450978199 learned_addr learned my addr 172.21.15.33:0/450978199 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:04:17.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.483+0000 7f03a12f7700 1 --2- 172.21.15.33:0/450978199 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e96b9e30 0x5627e96ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:17.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.483+0000 7f03a0af6700 1 --2- 172.21.15.33:0/450978199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5627e96ba750 0x5627e96a9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:17.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.483+0000 7f03a1af8700 1 -- 172.21.15.33:0/450978199 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e96b9e30 msgr2=0x5627e96ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:17.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.483+0000 7f03a1af8700 1 --2- 172.21.15.33:0/450978199 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e96b9e30 0x5627e96ca0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:17.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.483+0000 7f03a1af8700 1 -- 172.21.15.33:0/450978199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5627e96ba750 msgr2=0x5627e96a9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:17.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.483+0000 7f03a1af8700 1 --2- 172.21.15.33:0/450978199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5627e96ba750 0x5627e96a9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:17.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.483+0000 7f03a1af8700 1 -- 172.21.15.33:0/450978199 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627e96afa70 con 0x5627e96bf570 2024-05-15T14:04:17.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.483+0000 7f03a0af6700 1 --2- 172.21.15.33:0/450978199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5627e96ba750 0x5627e96a9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:04:17.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.483+0000 7f03a1af8700 1 --2- 172.21.15.33:0/450978199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627e96bf570 0x5627e96bf9f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0394009520 tx=0x7f0394009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc9f5b12b1b38fe server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:17.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.483+0000 7f0386ffd700 1 -- 172.21.15.33:0/450978199 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039400bca0 con 0x5627e96bf570 2024-05-15T14:04:17.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.483+0000 7f0386ffd700 1 -- 172.21.15.33:0/450978199 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f039400be40 con 0x5627e96bf570 2024-05-15T14:04:17.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.483+0000 7f0386ffd700 1 -- 172.21.15.33:0/450978199 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0394005110 con 0x5627e96bf570 2024-05-15T14:04:17.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.484+0000 7f03addf1b80 1 -- 172.21.15.33:0/450978199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627e96bf570 msgr2=0x5627e96bf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:17.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.484+0000 7f03addf1b80 1 --2- 172.21.15.33:0/450978199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627e96bf570 0x5627e96bf9f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0394009520 tx=0x7f0394009550 comp rx=0 tx=0).stop 2024-05-15T14:04:17.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.484+0000 7f03addf1b80 1 -- 172.21.15.33:0/450978199 shutdown_connections 2024-05-15T14:04:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.484+0000 7f03addf1b80 1 --2- 172.21.15.33:0/450978199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627e96bf570 0x5627e96bf9f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.484+0000 7f03addf1b80 1 --2- 172.21.15.33:0/450978199 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e96b9e30 0x5627e96ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.484+0000 7f03addf1b80 1 --2- 172.21.15.33:0/450978199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5627e96ba750 0x5627e96a9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.484+0000 7f03addf1b80 1 -- 172.21.15.33:0/450978199 >> 172.21.15.33:0/450978199 conn(0x5627e95df3e0 msgr2=0x5627e96bee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.484+0000 7f03addf1b80 1 -- 172.21.15.33:0/450978199 shutdown_connections 2024-05-15T14:04:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.484+0000 7f03addf1b80 1 -- 172.21.15.33:0/450978199 wait complete. 2024-05-15T14:04:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.485+0000 7f03addf1b80 1 -- start start 2024-05-15T14:04:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.485+0000 7f03addf1b80 1 Processor -- start 2024-05-15T14:04:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.485+0000 7f03addf1b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627e96b9e30 0x5627e96c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.485+0000 7f03addf1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5627e96ba750 0x5627e96c33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.485+0000 7f03addf1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e96bf570 0x5627e96c3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.485+0000 7f03addf1b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f039c1ae620 con 0x5627e96b9e30 2024-05-15T14:04:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.485+0000 7f03addf1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f039c1afa90 con 0x5627e96ba750 2024-05-15T14:04:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.485+0000 7f03addf1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f039c1aecf0 con 0x5627e96bf570 2024-05-15T14:04:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.485+0000 7f03addf1b80 10 client.0 _ll_register_callbacks cb 0x5627e9599710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:04:17.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.485+0000 7f03a12f7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627e96b9e30 0x5627e96c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:17.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.485+0000 7f03a12f7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627e96b9e30 0x5627e96c69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39214/0 (socket says 172.21.15.33:39214) 2024-05-15T14:04:17.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.485+0000 7f03a12f7700 1 -- 172.21.15.33:0/3966660356 learned_addr learned my addr 172.21.15.33:0/3966660356 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:04:17.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.485+0000 7f03a1af8700 1 --2- 172.21.15.33:0/3966660356 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e96bf570 0x5627e96c3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:17.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.486+0000 7f03a0af6700 1 --2- 172.21.15.33:0/3966660356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5627e96ba750 0x5627e96c33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:17.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.486+0000 7f03a12f7700 1 -- 172.21.15.33:0/3966660356 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e96bf570 msgr2=0x5627e96c3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:17.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.486+0000 7f03a12f7700 1 --2- 172.21.15.33:0/3966660356 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e96bf570 0x5627e96c3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:17.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.486+0000 7f03a12f7700 1 -- 172.21.15.33:0/3966660356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5627e96ba750 msgr2=0x5627e96c33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:17.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.486+0000 7f03a12f7700 1 --2- 172.21.15.33:0/3966660356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5627e96ba750 0x5627e96c33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:17.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.486+0000 7f03a12f7700 1 -- 172.21.15.33:0/3966660356 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627e96c4200 con 0x5627e96b9e30 2024-05-15T14:04:17.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.486+0000 7f03a12f7700 1 --2- 172.21.15.33:0/3966660356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627e96b9e30 0x5627e96c69f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f03980098c0 tx=0x7f0398009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff4a22a0bb51fd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:17.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.486+0000 7f0384ff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:04:17.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.486+0000 7f0384ff9700 1 -- 172.21.15.33:0/3966660356 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03980a5040 con 0x5627e96b9e30 2024-05-15T14:04:17.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.486+0000 7f0384ff9700 1 -- 172.21.15.33:0/3966660356 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f039806f340 con 0x5627e96b9e30 2024-05-15T14:04:17.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.486+0000 7f03addf1b80 1 -- 172.21.15.33:0/3966660356 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5627e9614d70 con 0x5627e96b9e30 2024-05-15T14:04:17.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.486+0000 7f0384ff9700 1 -- 172.21.15.33:0/3966660356 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f03980aa150 con 0x5627e96b9e30 2024-05-15T14:04:17.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.486+0000 7f03addf1b80 10 client.15813 fetch_fsmap learned FSMap version 27 2024-05-15T14:04:17.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.486+0000 7f03addf1b80 1 -- 172.21.15.33:0/3966660356 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5627e97397f0 con 0x5627e96b9e30 2024-05-15T14:04:17.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.487+0000 7f0384ff9700 1 -- 172.21.15.33:0/3966660356 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5627e9614d70 con 0x5627e96b9e30 2024-05-15T14:04:17.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.487+0000 7f0384ff9700 1 -- 172.21.15.33:0/3966660356 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f03980aef90 con 0x5627e96b9e30 2024-05-15T14:04:17.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.487+0000 7f03addf1b80 10 client.15813 fetch_fsmap finished waiting for FSMap version 27 2024-05-15T14:04:17.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.487+0000 7f03addf1b80 10 client.15813 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:04:17.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.487+0000 7f03addf1b80 20 client.15813 populate_metadata read hostname 'smithi033' 2024-05-15T14:04:17.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.487+0000 7f03addf1b80 1 --2- 172.21.15.33:0/3966660356 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5627e9739fd0 0x5627e973c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:17.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.487+0000 7f03addf1b80 4 client.15813 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.5, "await_for": 45.0}] 2024-05-15T14:04:17.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.487+0000 7f03addf1b80 1 -- 172.21.15.33:0/3966660356 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.5, "await_for": 45.0}) -- 0x5627e960e550 con 0x5627e9739fd0 2024-05-15T14:04:17.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.487+0000 7f03a0af6700 1 --2- 172.21.15.33:0/3966660356 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5627e9739fd0 0x5627e973c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:17.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.487+0000 7f03a0af6700 1 --2- 172.21.15.33:0/3966660356 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5627e9739fd0 0x5627e973c3f0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa9ffa474f54c262 server_cookie=7d56a6691235dfcf in_seq=0 out_seq=0 2024-05-15T14:04:17.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.487+0000 7f0384ff9700 10 client.15813 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:04:17.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.551+0000 7f0384ff9700 1 -- 172.21.15.33:0/3966660356 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5627e960e550 con 0x5627e9739fd0 2024-05-15T14:04:17.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.551+0000 7f0384ff9700 10 client.15813 handle_command_reply: tid=0 2024-05-15T14:04:17.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.551+0000 7f03addf1b80 1 client.15813 shutdown 2024-05-15T14:04:17.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.551+0000 7f03addf1b80 1 -- 172.21.15.33:0/3966660356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627e96b9e30 msgr2=0x5627e96c69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:17.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.551+0000 7f03addf1b80 1 --2- 172.21.15.33:0/3966660356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627e96b9e30 0x5627e96c69f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f03980098c0 tx=0x7f0398009790 comp rx=0 tx=0).stop 2024-05-15T14:04:17.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.551+0000 7f03addf1b80 1 -- 172.21.15.33:0/3966660356 shutdown_connections 2024-05-15T14:04:17.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.551+0000 7f03addf1b80 1 --2- 172.21.15.33:0/3966660356 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5627e96bf570 0x5627e96c3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:17.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.551+0000 7f03addf1b80 1 --2- 172.21.15.33:0/3966660356 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5627e9739fd0 0x5627e973c3f0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:17.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.551+0000 7f03addf1b80 1 --2- 172.21.15.33:0/3966660356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627e96b9e30 0x5627e96c69f0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:17.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.551+0000 7f0384ff9700 0 client.15813 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:04:17.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.551+0000 7f03addf1b80 1 --2- 172.21.15.33:0/3966660356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5627e96ba750 0x5627e96c33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:17.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.551+0000 7f03addf1b80 1 -- 172.21.15.33:0/3966660356 >> 172.21.15.33:0/3966660356 conn(0x5627e95df3e0 msgr2=0x5627e96b8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:17.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.551+0000 7f03addf1b80 1 -- 172.21.15.33:0/3966660356 shutdown_connections 2024-05-15T14:04:17.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.551+0000 7f03addf1b80 1 -- 172.21.15.33:0/3966660356 wait complete. 2024-05-15T14:04:17.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.551+0000 7f03addf1b80 20 client.15813 trim_cache size 0 max 16384 2024-05-15T14:04:17.554 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T14:04:17.554 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 27, 2024-05-15T14:04:17.554 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24478, 2024-05-15T14:04:17.554 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 14, 2024-05-15T14:04:17.554 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-05-15T14:04:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: "518e8a95": { 2024-05-15T14:04:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 14, 2024-05-15T14:04:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-05-15T14:04:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:04:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-05-15T14:04:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:04:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:04:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 90.0, 2024-05-15T14:04:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 146.5, 2024-05-15T14:04:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-05-15T14:04:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-05-15T14:04:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-05-15T14:04:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:04:17.555 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-05-15T14:04:17.556 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:04:17.556 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:04:17.556 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:04:17.556 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:04:17.556 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:04:17.556 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:04:17.556 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T14:04:17.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.552+0000 7f03a355b700 1 -- 172.21.15.33:0/2497360160 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f03880b2aa0 msgr2=0x7f03880b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:17.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.552+0000 7f03a355b700 1 --2- 172.21.15.33:0/2497360160 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f03880b2aa0 0x7f03880b4f60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f038c009680 tx=0x7f038c009260 comp rx=0 tx=0).stop 2024-05-15T14:04:17.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.552+0000 7f03a355b700 1 -- 172.21.15.33:0/2497360160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f039c103850 msgr2=0x7f039c075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:17.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.552+0000 7f03a355b700 1 --2- 172.21.15.33:0/2497360160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f039c103850 0x7f039c075840 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f039800e9a0 tx=0x7f039800ecb0 comp rx=0 tx=0).stop 2024-05-15T14:04:17.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.552+0000 7f03a355b700 1 -- 172.21.15.33:0/2497360160 shutdown_connections 2024-05-15T14:04:17.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.552+0000 7f03a355b700 1 --2- 172.21.15.33:0/2497360160 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f03880b2aa0 0x7f03880b4f60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:17.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.552+0000 7f03a355b700 1 --2- 172.21.15.33:0/2497360160 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f039c108840 0x7f039c0762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:17.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.552+0000 7f03a355b700 1 --2- 172.21.15.33:0/2497360160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f039c103850 0x7f039c075840 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:17.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.552+0000 7f03a355b700 1 --2- 172.21.15.33:0/2497360160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f039c104200 0x7f039c075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:17.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.552+0000 7f03a355b700 1 -- 172.21.15.33:0/2497360160 >> 172.21.15.33:0/2497360160 conn(0x7f039c07a610 msgr2=0x7f039c1015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:17.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.552+0000 7f03a355b700 1 -- 172.21.15.33:0/2497360160 shutdown_connections 2024-05-15T14:04:17.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:17.552+0000 7f03a355b700 1 -- 172.21.15.33:0/2497360160 wait complete. 2024-05-15T14:04:18.052 INFO:tasks.workunit.client.0.smithi033.stdout: 1 213945 10.36 MB/sec warmup 113 sec latency 502.023 ms 2024-05-15T14:04:18.228 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '518e8a95', quiesce took 5.9 seconds. Will release after: 20.6 2024-05-15T14:04:18.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:18 smithi033 ceph-mon[31650]: pgmap v122: 97 pgs: 97 active+clean; 62 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 395 op/s 2024-05-15T14:04:18.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:18 smithi052 ceph-mon[36190]: pgmap v122: 97 pgs: 97 active+clean; 62 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 395 op/s 2024-05-15T14:04:18.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:18 smithi081 ceph-mon[36434]: pgmap v122: 97 pgs: 97 active+clean; 62 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 395 op/s 2024-05-15T14:04:19.052 INFO:tasks.workunit.client.0.smithi033.stdout: 1 213945 10.27 MB/sec warmup 114 sec latency 1502.061 ms 2024-05-15T14:04:20.052 INFO:tasks.workunit.client.0.smithi033.stdout: 1 213945 10.18 MB/sec warmup 115 sec latency 2502.117 ms 2024-05-15T14:04:20.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:20 smithi033 ceph-mon[31650]: pgmap v123: 97 pgs: 97 active+clean; 62 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 286 op/s 2024-05-15T14:04:20.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:20 smithi052 ceph-mon[36190]: pgmap v123: 97 pgs: 97 active+clean; 62 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 286 op/s 2024-05-15T14:04:20.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:20 smithi081 ceph-mon[36434]: pgmap v123: 97 pgs: 97 active+clean; 62 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 286 op/s 2024-05-15T14:04:21.052 INFO:tasks.workunit.client.0.smithi033.stdout: 1 213945 10.09 MB/sec warmup 116 sec latency 3502.167 ms 2024-05-15T14:04:21.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.302+0000 7fbd5b971640 1 -- 172.21.15.33:0/557532874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd54102a50 msgr2=0x7fbd54102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:21.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.302+0000 7fbd5b971640 1 --2- 172.21.15.33:0/557532874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd54102a50 0x7fbd54102ed0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009930 tx=0x7fbd4402a420 comp rx=0 tx=0).stop 2024-05-15T14:04:21.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.302+0000 7fbd5b971640 1 -- 172.21.15.33:0/557532874 shutdown_connections 2024-05-15T14:04:21.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.302+0000 7fbd5b971640 1 --2- 172.21.15.33:0/557532874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd54103410 0x7fbd5410f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:21.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.302+0000 7fbd5b971640 1 --2- 172.21.15.33:0/557532874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd54102a50 0x7fbd54102ed0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:21.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.302+0000 7fbd5b971640 1 --2- 172.21.15.33:0/557532874 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd54108a60 0x7fbd54108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:21.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.302+0000 7fbd5b971640 1 -- 172.21.15.33:0/557532874 >> 172.21.15.33:0/557532874 conn(0x7fbd540fe540 msgr2=0x7fbd54100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:21.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.303+0000 7fbd5b971640 1 -- 172.21.15.33:0/557532874 shutdown_connections 2024-05-15T14:04:21.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.303+0000 7fbd5b971640 1 -- 172.21.15.33:0/557532874 wait complete. 2024-05-15T14:04:21.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.303+0000 7fbd5b971640 1 Processor -- start 2024-05-15T14:04:21.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.303+0000 7fbd5b971640 1 -- start start 2024-05-15T14:04:21.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.303+0000 7fbd5b971640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd54102a50 0x7fbd54199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:21.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.303+0000 7fbd5b971640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd54103410 0x7fbd54199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:21.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.303+0000 7fbd5b971640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd54108a60 0x7fbd5419dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:21.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.303+0000 7fbd5b971640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbd54112cf0 con 0x7fbd54103410 2024-05-15T14:04:21.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.303+0000 7fbd5b971640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbd54112b70 con 0x7fbd54108a60 2024-05-15T14:04:21.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.303+0000 7fbd5b971640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbd54112e70 con 0x7fbd54102a50 2024-05-15T14:04:21.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.304+0000 7fbd58f0c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd54103410 0x7fbd54199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:21.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.304+0000 7fbd58f0c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd54103410 0x7fbd54199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39234/0 (socket says 172.21.15.33:39234) 2024-05-15T14:04:21.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.304+0000 7fbd58f0c640 1 -- 172.21.15.33:0/2823121425 learned_addr learned my addr 172.21.15.33:0/2823121425 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:04:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.304+0000 7fbd59f0e640 1 --2- 172.21.15.33:0/2823121425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd54108a60 0x7fbd5419dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.304+0000 7fbd5970d640 1 --2- 172.21.15.33:0/2823121425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd54102a50 0x7fbd54199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.304+0000 7fbd58f0c640 1 -- 172.21.15.33:0/2823121425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd54102a50 msgr2=0x7fbd54199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.304+0000 7fbd58f0c640 1 --2- 172.21.15.33:0/2823121425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd54102a50 0x7fbd54199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.304+0000 7fbd58f0c640 1 -- 172.21.15.33:0/2823121425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd54108a60 msgr2=0x7fbd5419dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.304+0000 7fbd58f0c640 1 --2- 172.21.15.33:0/2823121425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd54108a60 0x7fbd5419dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.304+0000 7fbd58f0c640 1 -- 172.21.15.33:0/2823121425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd5419e200 con 0x7fbd54103410 2024-05-15T14:04:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.304+0000 7fbd5970d640 1 --2- 172.21.15.33:0/2823121425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd54102a50 0x7fbd54199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:04:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.304+0000 7fbd59f0e640 1 --2- 172.21.15.33:0/2823121425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd54108a60 0x7fbd5419dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:04:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.304+0000 7fbd58f0c640 1 --2- 172.21.15.33:0/2823121425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd54103410 0x7fbd54199750 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fbd44005940 tx=0x7fbd44004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.304+0000 7fbd4a7fc640 1 -- 172.21.15.33:0/2823121425 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4403f030 con 0x7fbd54103410 2024-05-15T14:04:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.304+0000 7fbd4a7fc640 1 -- 172.21.15.33:0/2823121425 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd44003aa0 con 0x7fbd54103410 2024-05-15T14:04:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.304+0000 7fbd5b971640 1 -- 172.21.15.33:0/2823121425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd5419e490 con 0x7fbd54103410 2024-05-15T14:04:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.304+0000 7fbd4a7fc640 1 -- 172.21.15.33:0/2823121425 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd44035070 con 0x7fbd54103410 2024-05-15T14:04:21.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.304+0000 7fbd5b971640 1 -- 172.21.15.33:0/2823121425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd541a5d80 con 0x7fbd54103410 2024-05-15T14:04:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.307+0000 7fbd5e764740 1 Processor -- start 2024-05-15T14:04:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.307+0000 7fbd4a7fc640 1 -- 172.21.15.33:0/2823121425 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbd44042030 con 0x7fbd54103410 2024-05-15T14:04:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.307+0000 7fbd5e764740 1 -- start start 2024-05-15T14:04:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.308+0000 7fbd5e764740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac72f717f0 0x55ac72f71bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.308+0000 7fbd5e764740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac72f72110 0x55ac72f809a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.308+0000 7fbd5e764740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ac72f80ee0 0x55ac72f832d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.308+0000 7fbd5e764740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ac72ee4c30 con 0x55ac72f72110 2024-05-15T14:04:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.308+0000 7fbd5e764740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ac72ed47d0 con 0x55ac72f717f0 2024-05-15T14:04:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.308+0000 7fbd5e764740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55ac72ed8810 con 0x55ac72f80ee0 2024-05-15T14:04:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.308+0000 7fbd5970d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac72f72110 0x55ac72f809a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.308+0000 7fbd5970d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac72f72110 0x55ac72f809a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39248/0 (socket says 172.21.15.33:39248) 2024-05-15T14:04:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.308+0000 7fbd5970d640 1 -- 172.21.15.33:0/916292734 learned_addr learned my addr 172.21.15.33:0/916292734 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:04:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.308+0000 7fbd59f0e640 1 --2- 172.21.15.33:0/916292734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac72f717f0 0x55ac72f71bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.308+0000 7fbd58f0c640 1 --2- 172.21.15.33:0/916292734 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ac72f80ee0 0x55ac72f832d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.308+0000 7fbd5970d640 1 -- 172.21.15.33:0/916292734 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ac72f80ee0 msgr2=0x55ac72f832d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.308+0000 7fbd5970d640 1 --2- 172.21.15.33:0/916292734 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ac72f80ee0 0x55ac72f832d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.308+0000 7fbd5970d640 1 -- 172.21.15.33:0/916292734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac72f717f0 msgr2=0x55ac72f71bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.308+0000 7fbd5970d640 1 --2- 172.21.15.33:0/916292734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac72f717f0 0x55ac72f71bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.308+0000 7fbd5970d640 1 -- 172.21.15.33:0/916292734 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac72be9210 con 0x55ac72f72110 2024-05-15T14:04:21.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.308+0000 7fbd4a7fc640 1 --2- 172.21.15.33:0/2823121425 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbd400b29d0 0x7fbd400b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:21.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.308+0000 7fbd5970d640 1 --2- 172.21.15.33:0/916292734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac72f72110 0x55ac72f809a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000bba0 tx=0x7fbd5000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e120a02b4568067d server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:21.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.309+0000 7fbd59f0e640 1 --2- 172.21.15.33:0/2823121425 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbd400b29d0 0x7fbd400b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:21.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.309+0000 7fbd4a7fc640 1 -- 172.21.15.33:0/2823121425 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7fbd44102aa0 con 0x7fbd54103410 2024-05-15T14:04:21.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.309+0000 7fbd3f7fe640 1 -- 172.21.15.33:0/916292734 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd50017620 con 0x55ac72f72110 2024-05-15T14:04:21.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.309+0000 7fbd3f7fe640 1 -- 172.21.15.33:0/916292734 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd50017c70 con 0x55ac72f72110 2024-05-15T14:04:21.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.309+0000 7fbd3f7fe640 1 -- 172.21.15.33:0/916292734 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5000fb40 con 0x55ac72f72110 2024-05-15T14:04:21.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.309+0000 7fbd5e764740 1 -- 172.21.15.33:0/916292734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac72f72110 msgr2=0x55ac72f809a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:21.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.309+0000 7fbd59f0e640 1 --2- 172.21.15.33:0/2823121425 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbd400b29d0 0x7fbd400b4e90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c005fd0 tx=0x7fbd4c008630 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:21.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.309+0000 7fbd5e764740 1 --2- 172.21.15.33:0/916292734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac72f72110 0x55ac72f809a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000bba0 tx=0x7fbd5000d710 comp rx=0 tx=0).stop 2024-05-15T14:04:21.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.310+0000 7fbd5e764740 1 -- 172.21.15.33:0/916292734 shutdown_connections 2024-05-15T14:04:21.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.310+0000 7fbd5e764740 1 --2- 172.21.15.33:0/916292734 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ac72f80ee0 0x55ac72f832d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:21.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.310+0000 7fbd5e764740 1 --2- 172.21.15.33:0/916292734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac72f72110 0x55ac72f809a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:21.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.310+0000 7fbd5e764740 1 --2- 172.21.15.33:0/916292734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac72f717f0 0x55ac72f71bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:21.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.310+0000 7fbd5e764740 1 -- 172.21.15.33:0/916292734 >> 172.21.15.33:0/916292734 conn(0x55ac72f6de00 msgr2=0x55ac72f6e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:21.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.310+0000 7fbd5e764740 1 -- 172.21.15.33:0/916292734 shutdown_connections 2024-05-15T14:04:21.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.310+0000 7fbd5e764740 1 -- 172.21.15.33:0/916292734 wait complete. 2024-05-15T14:04:21.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.310+0000 7fbd5e764740 1 -- start start 2024-05-15T14:04:21.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.310+0000 7fbd5e764740 1 Processor -- start 2024-05-15T14:04:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.311+0000 7fbd5e764740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac72f717f0 0x55ac72f807d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.311+0000 7fbd5e764740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac72f72110 0x55ac72f7dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.311+0000 7fbd5e764740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ac72f80ee0 0x55ac72f7e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.311+0000 7fbd5e764740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ac72bba790 con 0x55ac72f717f0 2024-05-15T14:04:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.311+0000 7fbd5e764740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ac72e90890 con 0x55ac72f72110 2024-05-15T14:04:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.311+0000 7fbd5e764740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55ac72f83bb0 con 0x55ac72f80ee0 2024-05-15T14:04:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.311+0000 7fbd5e764740 10 client.0 _ll_register_callbacks cb 0x55ac72e93d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:04:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.311+0000 7fbd5970d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac72f717f0 0x55ac72f807d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.311+0000 7fbd5970d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac72f717f0 0x55ac72f807d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39254/0 (socket says 172.21.15.33:39254) 2024-05-15T14:04:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.311+0000 7fbd5970d640 1 -- 172.21.15.33:0/1500210562 learned_addr learned my addr 172.21.15.33:0/1500210562 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:04:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.311+0000 7fbd58f0c640 1 --2- 172.21.15.33:0/1500210562 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac72f72110 0x55ac72f7dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.311+0000 7fbd59f0e640 1 --2- 172.21.15.33:0/1500210562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ac72f80ee0 0x55ac72f7e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.311+0000 7fbd5970d640 1 -- 172.21.15.33:0/1500210562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ac72f80ee0 msgr2=0x55ac72f7e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.311+0000 7fbd5970d640 1 --2- 172.21.15.33:0/1500210562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ac72f80ee0 0x55ac72f7e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.311+0000 7fbd5970d640 1 -- 172.21.15.33:0/1500210562 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac72f72110 msgr2=0x55ac72f7dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.311+0000 7fbd5970d640 1 --2- 172.21.15.33:0/1500210562 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac72f72110 0x55ac72f7dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.311+0000 7fbd5970d640 1 -- 172.21.15.33:0/1500210562 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac72cb5620 con 0x55ac72f717f0 2024-05-15T14:04:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.311+0000 7fbd59f0e640 1 --2- 172.21.15.33:0/1500210562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ac72f80ee0 0x55ac72f7e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:04:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.311+0000 7fbd58f0c640 1 --2- 172.21.15.33:0/1500210562 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac72f72110 0x55ac72f7dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:04:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.311+0000 7fbd5970d640 1 --2- 172.21.15.33:0/1500210562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac72f717f0 0x55ac72f807d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbd50019040 tx=0x7fbd5000b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea54a43a8c852ae9 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.311+0000 7fbd3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:04:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.311+0000 7fbd3cff9640 1 -- 172.21.15.33:0/1500210562 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd50026020 con 0x55ac72f717f0 2024-05-15T14:04:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.311+0000 7fbd3cff9640 1 -- 172.21.15.33:0/1500210562 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7fbd50005040 con 0x55ac72f717f0 2024-05-15T14:04:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.311+0000 7fbd5e764740 1 -- 172.21.15.33:0/1500210562 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd50017c70 con 0x55ac72f717f0 2024-05-15T14:04:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.312+0000 7fbd3cff9640 1 -- 172.21.15.33:0/1500210562 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbd50031050 con 0x55ac72f717f0 2024-05-15T14:04:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.312+0000 7fbd5e764740 10 client.15825 fetch_fsmap learned FSMap version 27 2024-05-15T14:04:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.312+0000 7fbd5e764740 1 -- 172.21.15.33:0/1500210562 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ac72f8f5f0 con 0x55ac72f717f0 2024-05-15T14:04:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.312+0000 7fbd3cff9640 1 -- 172.21.15.33:0/1500210562 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd50017c70 con 0x55ac72f717f0 2024-05-15T14:04:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.312+0000 7fbd3cff9640 1 -- 172.21.15.33:0/1500210562 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fbd500178b0 con 0x55ac72f717f0 2024-05-15T14:04:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.313+0000 7fbd5e764740 10 client.15825 fetch_fsmap finished waiting for FSMap version 27 2024-05-15T14:04:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.313+0000 7fbd5e764740 10 client.15825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:04:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.313+0000 7fbd5e764740 20 client.15825 populate_metadata read hostname 'smithi033' 2024-05-15T14:04:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.313+0000 7fbd5e764740 1 --2- 172.21.15.33:0/1500210562 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55ac72f90ab0 0x55ac72f92ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.313+0000 7fbd5e764740 4 client.15825 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:04:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.313+0000 7fbd5e764740 1 -- 172.21.15.33:0/1500210562 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac72ee6b80 con 0x55ac72f90ab0 2024-05-15T14:04:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.313+0000 7fbd58f0c640 1 --2- 172.21.15.33:0/1500210562 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55ac72f90ab0 0x55ac72f92ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.313+0000 7fbd3cff9640 10 client.15825 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:04:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.313+0000 7fbd58f0c640 1 --2- 172.21.15.33:0/1500210562 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55ac72f90ab0 0x55ac72f92ea0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eccfbbe2c145659 server_cookie=4c2c797e4707ed27 in_seq=0 out_seq=0 2024-05-15T14:04:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.314+0000 7fbd3cff9640 1 -- 172.21.15.33:0/1500210562 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ac72ee6b80 con 0x55ac72f90ab0 2024-05-15T14:04:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.314+0000 7fbd3cff9640 10 client.15825 handle_command_reply: tid=0 2024-05-15T14:04:21.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.314+0000 7fbd5e764740 1 client.15825 shutdown 2024-05-15T14:04:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.314+0000 7fbd5e764740 1 -- 172.21.15.33:0/1500210562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac72f717f0 msgr2=0x55ac72f807d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.314+0000 7fbd5e764740 1 --2- 172.21.15.33:0/1500210562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac72f717f0 0x55ac72f807d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbd50019040 tx=0x7fbd5000b650 comp rx=0 tx=0).stop 2024-05-15T14:04:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.314+0000 7fbd5e764740 1 -- 172.21.15.33:0/1500210562 shutdown_connections 2024-05-15T14:04:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.314+0000 7fbd5e764740 1 --2- 172.21.15.33:0/1500210562 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55ac72f90ab0 0x55ac72f92ea0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.314+0000 7fbd5e764740 1 --2- 172.21.15.33:0/1500210562 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ac72f80ee0 0x55ac72f7e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.314+0000 7fbd3cff9640 0 client.15825 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:04:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.314+0000 7fbd5e764740 1 --2- 172.21.15.33:0/1500210562 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac72f72110 0x55ac72f7dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.314+0000 7fbd5e764740 1 --2- 172.21.15.33:0/1500210562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac72f717f0 0x55ac72f807d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.314+0000 7fbd5e764740 1 -- 172.21.15.33:0/1500210562 >> 172.21.15.33:0/1500210562 conn(0x55ac72f6de00 msgr2=0x55ac72f73c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.314+0000 7fbd5e764740 1 -- 172.21.15.33:0/1500210562 shutdown_connections 2024-05-15T14:04:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.314+0000 7fbd5e764740 1 -- 172.21.15.33:0/1500210562 wait complete. 2024-05-15T14:04:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.315+0000 7fbd5e764740 20 client.15825 trim_cache size 0 max 16384 2024-05-15T14:04:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.330+0000 7fbd5e764740 1 Processor -- start 2024-05-15T14:04:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.330+0000 7fbd5e764740 1 -- start start 2024-05-15T14:04:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.330+0000 7fbd5e764740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac72f717f0 0x55ac72f791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.330+0000 7fbd5e764740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ac72f72110 0x55ac72f73a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.330+0000 7fbd5e764740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac72f80ee0 0x55ac72f80300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.330+0000 7fbd5e764740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbd541aed70 con 0x55ac72f717f0 2024-05-15T14:04:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.330+0000 7fbd5e764740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbd541ac5e0 con 0x55ac72f80ee0 2024-05-15T14:04:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.330+0000 7fbd5e764740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbd541aa430 con 0x55ac72f72110 2024-05-15T14:04:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.330+0000 7fbd5970d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac72f717f0 0x55ac72f791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.330+0000 7fbd5970d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac72f717f0 0x55ac72f791d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39256/0 (socket says 172.21.15.33:39256) 2024-05-15T14:04:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.330+0000 7fbd5970d640 1 -- 172.21.15.33:0/2846758956 learned_addr learned my addr 172.21.15.33:0/2846758956 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:04:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.330+0000 7fbd59f0e640 1 --2- 172.21.15.33:0/2846758956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac72f80ee0 0x55ac72f80300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.330+0000 7fbd58f0c640 1 --2- 172.21.15.33:0/2846758956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ac72f72110 0x55ac72f73a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.330+0000 7fbd5970d640 1 -- 172.21.15.33:0/2846758956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ac72f72110 msgr2=0x55ac72f73a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.330+0000 7fbd5970d640 1 --2- 172.21.15.33:0/2846758956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ac72f72110 0x55ac72f73a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.330+0000 7fbd5970d640 1 -- 172.21.15.33:0/2846758956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac72f80ee0 msgr2=0x55ac72f80300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.330+0000 7fbd5970d640 1 --2- 172.21.15.33:0/2846758956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac72f80ee0 0x55ac72f80300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.330+0000 7fbd5970d640 1 -- 172.21.15.33:0/2846758956 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac72f6dc30 con 0x55ac72f717f0 2024-05-15T14:04:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.331+0000 7fbd5970d640 1 --2- 172.21.15.33:0/2846758956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac72f717f0 0x55ac72f791d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000bba0 tx=0x7fbd5000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a20d5c422e15247e server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.331+0000 7fbd3effd640 1 -- 172.21.15.33:0/2846758956 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5003a070 con 0x55ac72f717f0 2024-05-15T14:04:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.331+0000 7fbd3effd640 1 -- 172.21.15.33:0/2846758956 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd500179f0 con 0x55ac72f717f0 2024-05-15T14:04:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.331+0000 7fbd3effd640 1 -- 172.21.15.33:0/2846758956 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd50026040 con 0x55ac72f717f0 2024-05-15T14:04:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.331+0000 7fbd5e764740 1 -- 172.21.15.33:0/2846758956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac72f717f0 msgr2=0x55ac72f791d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.331+0000 7fbd5e764740 1 --2- 172.21.15.33:0/2846758956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac72f717f0 0x55ac72f791d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000bba0 tx=0x7fbd5000dd40 comp rx=0 tx=0).stop 2024-05-15T14:04:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.331+0000 7fbd5e764740 1 -- 172.21.15.33:0/2846758956 shutdown_connections 2024-05-15T14:04:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.331+0000 7fbd5e764740 1 --2- 172.21.15.33:0/2846758956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac72f80ee0 0x55ac72f80300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.331+0000 7fbd5e764740 1 --2- 172.21.15.33:0/2846758956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ac72f72110 0x55ac72f73a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.331+0000 7fbd5e764740 1 --2- 172.21.15.33:0/2846758956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac72f717f0 0x55ac72f791d0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.331+0000 7fbd5e764740 1 -- 172.21.15.33:0/2846758956 >> 172.21.15.33:0/2846758956 conn(0x55ac72f6de00 msgr2=0x55ac72f73340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.331+0000 7fbd5e764740 1 -- 172.21.15.33:0/2846758956 shutdown_connections 2024-05-15T14:04:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.331+0000 7fbd5e764740 1 -- 172.21.15.33:0/2846758956 wait complete. 2024-05-15T14:04:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.332+0000 7fbd5e764740 1 -- start start 2024-05-15T14:04:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.332+0000 7fbd5e764740 1 Processor -- start 2024-05-15T14:04:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.332+0000 7fbd5e764740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ac72f717f0 0x55ac72f892e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.332+0000 7fbd5e764740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac72f72110 0x55ac72f846f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.332+0000 7fbd5e764740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac72f80ee0 0x55ac72f84c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.332+0000 7fbd5e764740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbd541aaaa0 con 0x55ac72f72110 2024-05-15T14:04:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.332+0000 7fbd5e764740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbd541abf10 con 0x55ac72f80ee0 2024-05-15T14:04:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.332+0000 7fbd5e764740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbd541ab170 con 0x55ac72f717f0 2024-05-15T14:04:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.332+0000 7fbd5e764740 10 client.0 _ll_register_callbacks cb 0x7fbd5000fc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:04:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.332+0000 7fbd58f0c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac72f72110 0x55ac72f846f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.332+0000 7fbd58f0c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac72f72110 0x55ac72f846f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39260/0 (socket says 172.21.15.33:39260) 2024-05-15T14:04:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.332+0000 7fbd58f0c640 1 -- 172.21.15.33:0/1345064545 learned_addr learned my addr 172.21.15.33:0/1345064545 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:04:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.332+0000 7fbd5970d640 1 --2- 172.21.15.33:0/1345064545 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ac72f717f0 0x55ac72f892e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.333+0000 7fbd59f0e640 1 --2- 172.21.15.33:0/1345064545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac72f80ee0 0x55ac72f84c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.333+0000 7fbd58f0c640 1 -- 172.21.15.33:0/1345064545 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ac72f717f0 msgr2=0x55ac72f892e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.333+0000 7fbd58f0c640 1 --2- 172.21.15.33:0/1345064545 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ac72f717f0 0x55ac72f892e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.333+0000 7fbd58f0c640 1 -- 172.21.15.33:0/1345064545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac72f80ee0 msgr2=0x55ac72f84c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.333+0000 7fbd58f0c640 1 --2- 172.21.15.33:0/1345064545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac72f80ee0 0x55ac72f84c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.333+0000 7fbd58f0c640 1 -- 172.21.15.33:0/1345064545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac72f72650 con 0x55ac72f72110 2024-05-15T14:04:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.333+0000 7fbd58f0c640 1 --2- 172.21.15.33:0/1345064545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac72f72110 0x55ac72f846f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fbd44100800 tx=0x7fbd4403b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96473d5964feee10 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.333+0000 7fbd3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:04:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.333+0000 7fbd3cff9640 1 -- 172.21.15.33:0/1345064545 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd44108030 con 0x55ac72f72110 2024-05-15T14:04:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.333+0000 7fbd3cff9640 1 -- 172.21.15.33:0/1345064545 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7fbd440c6380 con 0x55ac72f72110 2024-05-15T14:04:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.333+0000 7fbd5e764740 1 -- 172.21.15.33:0/1345064545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd500179f0 con 0x55ac72f72110 2024-05-15T14:04:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.333+0000 7fbd3cff9640 1 -- 172.21.15.33:0/1345064545 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbd44049070 con 0x55ac72f72110 2024-05-15T14:04:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.333+0000 7fbd5e764740 10 client.15831 fetch_fsmap learned FSMap version 27 2024-05-15T14:04:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.333+0000 7fbd5e764740 1 -- 172.21.15.33:0/1345064545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ac72ffaf40 con 0x55ac72f72110 2024-05-15T14:04:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.333+0000 7fbd3cff9640 1 -- 172.21.15.33:0/1345064545 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd500179f0 con 0x55ac72f72110 2024-05-15T14:04:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.333+0000 7fbd3cff9640 1 -- 172.21.15.33:0/1345064545 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fbd44100480 con 0x55ac72f72110 2024-05-15T14:04:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.333+0000 7fbd5e764740 10 client.15831 fetch_fsmap finished waiting for FSMap version 27 2024-05-15T14:04:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.334+0000 7fbd5e764740 10 client.15831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:04:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.334+0000 7fbd5e764740 20 client.15831 populate_metadata read hostname 'smithi033' 2024-05-15T14:04:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.334+0000 7fbd5e764740 1 --2- 172.21.15.33:0/1345064545 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55ac72ffb720 0x55ac72ffdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.334+0000 7fbd5e764740 4 client.15831 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:04:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.334+0000 7fbd5e764740 1 -- 172.21.15.33:0/1345064545 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ac72ee0360 con 0x55ac72ffb720 2024-05-15T14:04:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.334+0000 7fbd5970d640 1 --2- 172.21.15.33:0/1345064545 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55ac72ffb720 0x55ac72ffdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.334+0000 7fbd3cff9640 10 client.15831 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:04:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.334+0000 7fbd5970d640 1 --2- 172.21.15.33:0/1345064545 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55ac72ffb720 0x55ac72ffdb40 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0234c747a577fa2 server_cookie=8763a70ea14d0c1c in_seq=0 out_seq=0 2024-05-15T14:04:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.386+0000 7fbd3cff9640 1 -- 172.21.15.33:0/1345064545 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ac72ee0360 con 0x55ac72ffb720 2024-05-15T14:04:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.386+0000 7fbd3cff9640 10 client.15831 handle_command_reply: tid=0 2024-05-15T14:04:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.386+0000 7fbd5e764740 1 client.15831 shutdown 2024-05-15T14:04:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.387+0000 7fbd5e764740 1 -- 172.21.15.33:0/1345064545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac72f72110 msgr2=0x55ac72f846f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.387+0000 7fbd5e764740 1 --2- 172.21.15.33:0/1345064545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac72f72110 0x55ac72f846f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fbd44100800 tx=0x7fbd4403b720 comp rx=0 tx=0).stop 2024-05-15T14:04:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.387+0000 7fbd5e764740 1 -- 172.21.15.33:0/1345064545 shutdown_connections 2024-05-15T14:04:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.387+0000 7fbd5e764740 1 --2- 172.21.15.33:0/1345064545 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55ac72ffb720 0x55ac72ffdb40 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.387+0000 7fbd5e764740 1 --2- 172.21.15.33:0/1345064545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ac72f80ee0 0x55ac72f84c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.387+0000 7fbd5e764740 1 --2- 172.21.15.33:0/1345064545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac72f72110 0x55ac72f846f0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.387+0000 7fbd3cff9640 0 client.15831 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:04:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.387+0000 7fbd5e764740 1 --2- 172.21.15.33:0/1345064545 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ac72f717f0 0x55ac72f892e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.387+0000 7fbd5e764740 1 -- 172.21.15.33:0/1345064545 >> 172.21.15.33:0/1345064545 conn(0x55ac72f6de00 msgr2=0x55ac72f6d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.387+0000 7fbd5e764740 1 -- 172.21.15.33:0/1345064545 shutdown_connections 2024-05-15T14:04:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.387+0000 7fbd5e764740 1 -- 172.21.15.33:0/1345064545 wait complete. 2024-05-15T14:04:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.387+0000 7fbd5e764740 20 client.15831 trim_cache size 0 max 16384 2024-05-15T14:04:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:04:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:04:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:04:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.387+0000 7fbd5b971640 1 -- 172.21.15.33:0/2823121425 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbd400b29d0 msgr2=0x7fbd400b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.387+0000 7fbd5b971640 1 --2- 172.21.15.33:0/2823121425 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbd400b29d0 0x7fbd400b4e90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c005fd0 tx=0x7fbd4c008630 comp rx=0 tx=0).stop 2024-05-15T14:04:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.387+0000 7fbd5b971640 1 -- 172.21.15.33:0/2823121425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd54103410 msgr2=0x7fbd54199750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.387+0000 7fbd5b971640 1 --2- 172.21.15.33:0/2823121425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd54103410 0x7fbd54199750 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fbd44005940 tx=0x7fbd44004270 comp rx=0 tx=0).stop 2024-05-15T14:04:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.388+0000 7fbd5b971640 1 -- 172.21.15.33:0/2823121425 shutdown_connections 2024-05-15T14:04:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.388+0000 7fbd5b971640 1 --2- 172.21.15.33:0/2823121425 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbd400b29d0 0x7fbd400b4e90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.388+0000 7fbd5b971640 1 --2- 172.21.15.33:0/2823121425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbd54108a60 0x7fbd5419dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.388+0000 7fbd5b971640 1 --2- 172.21.15.33:0/2823121425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd54103410 0x7fbd54199750 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.388+0000 7fbd5b971640 1 --2- 172.21.15.33:0/2823121425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd54102a50 0x7fbd54199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.388+0000 7fbd5b971640 1 -- 172.21.15.33:0/2823121425 >> 172.21.15.33:0/2823121425 conn(0x7fbd540fe540 msgr2=0x7fbd540fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.388+0000 7fbd5b971640 1 -- 172.21.15.33:0/2823121425 shutdown_connections 2024-05-15T14:04:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:21.388+0000 7fbd5b971640 1 -- 172.21.15.33:0/2823121425 wait complete. 2024-05-15T14:04:22.052 INFO:tasks.workunit.client.0.smithi033.stdout: 1 213945 10.00 MB/sec warmup 117 sec latency 4502.291 ms 2024-05-15T14:04:22.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:22 smithi033 ceph-mon[31650]: pgmap v124: 97 pgs: 97 active+clean; 62 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 286 op/s 2024-05-15T14:04:22.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:22 smithi052 ceph-mon[36190]: pgmap v124: 97 pgs: 97 active+clean; 62 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 286 op/s 2024-05-15T14:04:22.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:22 smithi081 ceph-mon[36434]: pgmap v124: 97 pgs: 97 active+clean; 62 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 286 op/s 2024-05-15T14:04:23.052 INFO:tasks.workunit.client.0.smithi033.stdout: 1 213945 9.92 MB/sec warmup 118 sec latency 5502.387 ms 2024-05-15T14:04:24.052 INFO:tasks.workunit.client.0.smithi033.stdout: 1 213945 9.83 MB/sec warmup 119 sec latency 6502.499 ms 2024-05-15T14:04:24.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:24 smithi033 ceph-mon[31650]: pgmap v125: 97 pgs: 97 active+clean; 61 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 338 op/s 2024-05-15T14:04:24.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:24 smithi052 ceph-mon[36190]: pgmap v125: 97 pgs: 97 active+clean; 61 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 338 op/s 2024-05-15T14:04:24.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:24 smithi081 ceph-mon[36434]: pgmap v125: 97 pgs: 97 active+clean; 61 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 338 op/s 2024-05-15T14:04:26.053 INFO:tasks.workunit.client.0.smithi033.stdout: 1 213945 0.00 MB/sec execute 1 sec latency 8502.705 ms 2024-05-15T14:04:26.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:26 smithi033 ceph-mon[31650]: pgmap v126: 97 pgs: 97 active+clean; 61 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 165 op/s 2024-05-15T14:04:26.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:26 smithi052 ceph-mon[36190]: pgmap v126: 97 pgs: 97 active+clean; 61 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 165 op/s 2024-05-15T14:04:26.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:26 smithi081 ceph-mon[36434]: pgmap v126: 97 pgs: 97 active+clean; 61 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 165 op/s 2024-05-15T14:04:27.053 INFO:tasks.workunit.client.0.smithi033.stdout: 1 213945 0.00 MB/sec execute 2 sec latency 9502.843 ms 2024-05-15T14:04:28.053 INFO:tasks.workunit.client.0.smithi033.stdout: 1 213945 0.00 MB/sec execute 3 sec latency 10502.958 ms 2024-05-15T14:04:28.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:28 smithi033 ceph-mon[31650]: pgmap v127: 97 pgs: 97 active+clean; 67 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 179 op/s 2024-05-15T14:04:28.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:28 smithi052 ceph-mon[36190]: pgmap v127: 97 pgs: 97 active+clean; 67 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 179 op/s 2024-05-15T14:04:28.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:28 smithi081 ceph-mon[36434]: pgmap v127: 97 pgs: 97 active+clean; 67 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 179 op/s 2024-05-15T14:04:29.053 INFO:tasks.workunit.client.0.smithi033.stdout: 1 213945 0.00 MB/sec execute 4 sec latency 11503.010 ms 2024-05-15T14:04:30.053 INFO:tasks.workunit.client.0.smithi033.stdout: 1 213945 0.00 MB/sec execute 5 sec latency 12503.102 ms 2024-05-15T14:04:30.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:30 smithi052 ceph-mon[36190]: pgmap v128: 97 pgs: 97 active+clean; 67 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 817 KiB/s wr, 65 op/s 2024-05-15T14:04:30.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:30 smithi081 ceph-mon[36434]: pgmap v128: 97 pgs: 97 active+clean; 67 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 817 KiB/s wr, 65 op/s 2024-05-15T14:04:30.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:30 smithi033 ceph-mon[31650]: pgmap v128: 97 pgs: 97 active+clean; 67 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 817 KiB/s wr, 65 op/s 2024-05-15T14:04:31.053 INFO:tasks.workunit.client.0.smithi033.stdout: 1 213945 0.00 MB/sec execute 6 sec latency 13503.151 ms 2024-05-15T14:04:32.053 INFO:tasks.workunit.client.0.smithi033.stdout: 1 213945 0.00 MB/sec execute 7 sec latency 14503.238 ms 2024-05-15T14:04:32.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:32 smithi052 ceph-mon[36190]: pgmap v129: 97 pgs: 97 active+clean; 67 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 817 KiB/s wr, 65 op/s 2024-05-15T14:04:32.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:32 smithi081 ceph-mon[36434]: pgmap v129: 97 pgs: 97 active+clean; 67 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 817 KiB/s wr, 65 op/s 2024-05-15T14:04:32.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:32 smithi033 ceph-mon[31650]: pgmap v129: 97 pgs: 97 active+clean; 67 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 817 KiB/s wr, 65 op/s 2024-05-15T14:04:33.053 INFO:tasks.workunit.client.0.smithi033.stdout: 1 213945 0.00 MB/sec execute 8 sec latency 15503.290 ms 2024-05-15T14:04:33.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:33 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:04:33.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:33 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:04:33.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:33 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:04:33.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:33 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:04:33.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:33 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:04:33.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:33 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:04:33.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:33 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:04:33.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:33 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:04:33.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:33 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:04:33.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:33 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:04:33.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:33 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:04:33.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:33 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:04:34.053 INFO:tasks.workunit.client.0.smithi033.stdout: 1 213945 0.00 MB/sec execute 9 sec latency 16503.365 ms 2024-05-15T14:04:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.487+0000 7f930b912640 1 -- 172.21.15.33:0/845758125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9304101930 msgr2=0x7f9304101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.487+0000 7f930b912640 1 --2- 172.21.15.33:0/845758125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9304101930 0x7f9304101db0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f92f4009930 tx=0x7f92f402a420 comp rx=0 tx=0).stop 2024-05-15T14:04:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.487+0000 7f930b912640 1 -- 172.21.15.33:0/845758125 shutdown_connections 2024-05-15T14:04:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.487+0000 7f930b912640 1 --2- 172.21.15.33:0/845758125 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93041022f0 0x7f930410e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.487+0000 7f930b912640 1 --2- 172.21.15.33:0/845758125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9304101930 0x7f9304101db0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.487+0000 7f930b912640 1 --2- 172.21.15.33:0/845758125 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9304107940 0x7f9304107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.487+0000 7f930b912640 1 -- 172.21.15.33:0/845758125 >> 172.21.15.33:0/845758125 conn(0x7f9304078a70 msgr2=0x7f9304078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.487+0000 7f930b912640 1 -- 172.21.15.33:0/845758125 shutdown_connections 2024-05-15T14:04:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.488+0000 7f930b912640 1 -- 172.21.15.33:0/845758125 wait complete. 2024-05-15T14:04:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.488+0000 7f930b912640 1 Processor -- start 2024-05-15T14:04:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.488+0000 7f930b912640 1 -- start start 2024-05-15T14:04:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.488+0000 7f930b912640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9304101930 0x7f930419a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.488+0000 7f930b912640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93041022f0 0x7f930419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.488+0000 7f930b912640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9304107940 0x7f93041a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.488+0000 7f930b912640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9304111c50 con 0x7f9304101930 2024-05-15T14:04:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.488+0000 7f930b912640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9304111ad0 con 0x7f9304107940 2024-05-15T14:04:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.488+0000 7f930b912640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9304111dd0 con 0x7f93041022f0 2024-05-15T14:04:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.488+0000 7f93096ae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9304101930 0x7f930419a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.489+0000 7f93096ae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9304101930 0x7f930419a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51974/0 (socket says 172.21.15.33:51974) 2024-05-15T14:04:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.489+0000 7f93096ae640 1 -- 172.21.15.33:0/1635664370 learned_addr learned my addr 172.21.15.33:0/1635664370 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:04:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.489+0000 7f9309eaf640 1 --2- 172.21.15.33:0/1635664370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9304107940 0x7f93041a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.489+0000 7f9308ead640 1 --2- 172.21.15.33:0/1635664370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93041022f0 0x7f930419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.489+0000 7f93096ae640 1 -- 172.21.15.33:0/1635664370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93041022f0 msgr2=0x7f930419a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.489+0000 7f93096ae640 1 --2- 172.21.15.33:0/1635664370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93041022f0 0x7f930419a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.489+0000 7f93096ae640 1 -- 172.21.15.33:0/1635664370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9304107940 msgr2=0x7f93041a5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.489+0000 7f93096ae640 1 --2- 172.21.15.33:0/1635664370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9304107940 0x7f93041a5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.489+0000 7f93096ae640 1 -- 172.21.15.33:0/1635664370 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93041a5750 con 0x7f9304101930 2024-05-15T14:04:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.489+0000 7f93096ae640 1 --2- 172.21.15.33:0/1635664370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9304101930 0x7f930419a190 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f930000d950 tx=0x7f930000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.489+0000 7f92fa7fc640 1 -- 172.21.15.33:0/1635664370 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9300017070 con 0x7f9304101930 2024-05-15T14:04:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.489+0000 7f92fa7fc640 1 -- 172.21.15.33:0/1635664370 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9300009c30 con 0x7f9304101930 2024-05-15T14:04:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.489+0000 7f92fa7fc640 1 -- 172.21.15.33:0/1635664370 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930000f620 con 0x7f9304101930 2024-05-15T14:04:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.489+0000 7f930b912640 1 -- 172.21.15.33:0/1635664370 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93041a5920 con 0x7f9304101930 2024-05-15T14:04:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.489+0000 7f930b912640 1 -- 172.21.15.33:0/1635664370 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93041a5e40 con 0x7f9304101930 2024-05-15T14:04:34.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.493+0000 7f930e62a740 1 Processor -- start 2024-05-15T14:04:34.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.493+0000 7f930e62a740 1 -- start start 2024-05-15T14:04:34.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.493+0000 7f92fa7fc640 1 -- 172.21.15.33:0/1635664370 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9300010460 con 0x7f9304101930 2024-05-15T14:04:34.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.493+0000 7f930e62a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266e2cb8f0 0x56266e2cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:34.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.493+0000 7f930e62a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56266e2cc210 0x56266e2d6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:34.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.493+0000 7f930e62a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56266e2d6ad0 0x56266e2d8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:34.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.493+0000 7f930e62a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56266e23ec30 con 0x56266e2cb8f0 2024-05-15T14:04:34.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.493+0000 7f930e62a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56266e22e7d0 con 0x56266e2d6ad0 2024-05-15T14:04:34.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.493+0000 7f930e62a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56266e232810 con 0x56266e2cc210 2024-05-15T14:04:34.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.493+0000 7f9309eaf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266e2cb8f0 0x56266e2cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:34.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.493+0000 7f9309eaf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266e2cb8f0 0x56266e2cbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51980/0 (socket says 172.21.15.33:51980) 2024-05-15T14:04:34.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.493+0000 7f9309eaf640 1 -- 172.21.15.33:0/3064200111 learned_addr learned my addr 172.21.15.33:0/3064200111 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:04:34.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.493+0000 7f93096ae640 1 --2- 172.21.15.33:0/3064200111 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56266e2cc210 0x56266e2d6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:34.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.493+0000 7f9308ead640 1 --2- 172.21.15.33:0/3064200111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56266e2d6ad0 0x56266e2d8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.494+0000 7f9309eaf640 1 -- 172.21.15.33:0/3064200111 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56266e2cc210 msgr2=0x56266e2d6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.494+0000 7f9309eaf640 1 --2- 172.21.15.33:0/3064200111 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56266e2cc210 0x56266e2d6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.494+0000 7f9309eaf640 1 -- 172.21.15.33:0/3064200111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56266e2d6ad0 msgr2=0x56266e2d8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.494+0000 7f9309eaf640 1 --2- 172.21.15.33:0/3064200111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56266e2d6ad0 0x56266e2d8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.494+0000 7f9309eaf640 1 -- 172.21.15.33:0/3064200111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56266df43210 con 0x56266e2cb8f0 2024-05-15T14:04:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.494+0000 7f9308ead640 1 --2- 172.21.15.33:0/3064200111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56266e2d6ad0 0x56266e2d8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:04:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.494+0000 7f9309eaf640 1 --2- 172.21.15.33:0/3064200111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266e2cb8f0 0x56266e2cbcd0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f92fc009fa0 tx=0x7f92fc00f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f9b9148fe65a29 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.494+0000 7f92fa7fc640 1 --2- 172.21.15.33:0/1635664370 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f92f00b2950 0x7f92f00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.494+0000 7f92ef7fe640 1 -- 172.21.15.33:0/3064200111 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc015070 con 0x56266e2cb8f0 2024-05-15T14:04:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.494+0000 7f92ef7fe640 1 -- 172.21.15.33:0/3064200111 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92fc01ab10 con 0x56266e2cb8f0 2024-05-15T14:04:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.494+0000 7f92ef7fe640 1 -- 172.21.15.33:0/3064200111 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc011d00 con 0x56266e2cb8f0 2024-05-15T14:04:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.494+0000 7f93096ae640 1 --2- 172.21.15.33:0/1635664370 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f92f00b2950 0x7f92f00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:34.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.494+0000 7f92fa7fc640 1 -- 172.21.15.33:0/1635664370 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f93000e2260 con 0x7f9304101930 2024-05-15T14:04:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.495+0000 7f930e62a740 1 -- 172.21.15.33:0/3064200111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266e2cb8f0 msgr2=0x56266e2cbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.495+0000 7f93096ae640 1 --2- 172.21.15.33:0/1635664370 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f92f00b2950 0x7f92f00b4e10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f93000dfa70 tx=0x7f93000e5db0 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.495+0000 7f930e62a740 1 --2- 172.21.15.33:0/3064200111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266e2cb8f0 0x56266e2cbcd0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f92fc009fa0 tx=0x7f92fc00f7f0 comp rx=0 tx=0).stop 2024-05-15T14:04:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.495+0000 7f930e62a740 1 -- 172.21.15.33:0/3064200111 shutdown_connections 2024-05-15T14:04:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.495+0000 7f930e62a740 1 --2- 172.21.15.33:0/3064200111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56266e2d6ad0 0x56266e2d8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.495+0000 7f930e62a740 1 --2- 172.21.15.33:0/3064200111 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56266e2cc210 0x56266e2d6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.495+0000 7f930e62a740 1 --2- 172.21.15.33:0/3064200111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266e2cb8f0 0x56266e2cbcd0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.495+0000 7f930e62a740 1 -- 172.21.15.33:0/3064200111 >> 172.21.15.33:0/3064200111 conn(0x56266e2c7e00 msgr2=0x56266e2c81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.495+0000 7f930e62a740 1 -- 172.21.15.33:0/3064200111 shutdown_connections 2024-05-15T14:04:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.495+0000 7f930e62a740 1 -- 172.21.15.33:0/3064200111 wait complete. 2024-05-15T14:04:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.495+0000 7f930e62a740 1 -- start start 2024-05-15T14:04:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.495+0000 7f930e62a740 1 Processor -- start 2024-05-15T14:04:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.496+0000 7f930e62a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56266e2cb8f0 0x56266e2d6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.496+0000 7f930e62a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266e2cc210 0x56266e2d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.496+0000 7f930e62a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56266e2d6ad0 0x56266e2d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.496+0000 7f930e62a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56266df14790 con 0x56266e2cc210 2024-05-15T14:04:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.496+0000 7f930e62a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56266e1ea890 con 0x56266e2cb8f0 2024-05-15T14:04:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.496+0000 7f930e62a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56266e2d97a0 con 0x56266e2d6ad0 2024-05-15T14:04:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.496+0000 7f930e62a740 10 client.0 _ll_register_callbacks cb 0x56266e1edd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:04:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.496+0000 7f9308ead640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266e2cc210 0x56266e2d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.496+0000 7f9308ead640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266e2cc210 0x56266e2d36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51992/0 (socket says 172.21.15.33:51992) 2024-05-15T14:04:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.496+0000 7f9308ead640 1 -- 172.21.15.33:0/4027381415 learned_addr learned my addr 172.21.15.33:0/4027381415 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:04:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.496+0000 7f9309eaf640 1 --2- 172.21.15.33:0/4027381415 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56266e2d6ad0 0x56266e2d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.496+0000 7f9309eaf640 1 --2- 172.21.15.33:0/4027381415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56266e2cb8f0 0x56266e2d6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.496+0000 7f9308ead640 1 -- 172.21.15.33:0/4027381415 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56266e2d6ad0 msgr2=0x56266e2d3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.496+0000 7f9308ead640 1 --2- 172.21.15.33:0/4027381415 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56266e2d6ad0 0x56266e2d3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.496+0000 7f9308ead640 1 -- 172.21.15.33:0/4027381415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56266e2cb8f0 msgr2=0x56266e2d6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.496+0000 7f9308ead640 1 --2- 172.21.15.33:0/4027381415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56266e2cb8f0 0x56266e2d6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.496+0000 7f9308ead640 1 -- 172.21.15.33:0/4027381415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56266e00f620 con 0x56266e2cc210 2024-05-15T14:04:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.496+0000 7f9309eaf640 1 --2- 172.21.15.33:0/4027381415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56266e2cb8f0 0x56266e2d6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:04:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.496+0000 7f9308ead640 1 --2- 172.21.15.33:0/4027381415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266e2cc210 0x56266e2d36f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f92f4031440 tx=0x7f92f4005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=180f704adaed3103 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.496+0000 7f92ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:04:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.497+0000 7f92ecff9640 1 -- 172.21.15.33:0/4027381415 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f4035030 con 0x56266e2cc210 2024-05-15T14:04:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.497+0000 7f92ecff9640 1 -- 172.21.15.33:0/4027381415 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f92f40048c0 con 0x56266e2cc210 2024-05-15T14:04:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.497+0000 7f930e62a740 1 -- 172.21.15.33:0/4027381415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92fc01ab10 con 0x56266e2cc210 2024-05-15T14:04:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.497+0000 7f92ecff9640 1 -- 172.21.15.33:0/4027381415 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f92f4005d50 con 0x56266e2cc210 2024-05-15T14:04:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.497+0000 7f930e62a740 10 client.15843 fetch_fsmap learned FSMap version 27 2024-05-15T14:04:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.497+0000 7f930e62a740 1 -- 172.21.15.33:0/4027381415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56266e2e5110 con 0x56266e2cc210 2024-05-15T14:04:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.497+0000 7f92ecff9640 1 -- 172.21.15.33:0/4027381415 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92fc01ab10 con 0x56266e2cc210 2024-05-15T14:04:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.497+0000 7f92ecff9640 1 -- 172.21.15.33:0/4027381415 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f92f40314a0 con 0x56266e2cc210 2024-05-15T14:04:34.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.497+0000 7f930e62a740 10 client.15843 fetch_fsmap finished waiting for FSMap version 27 2024-05-15T14:04:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.497+0000 7f930e62a740 10 client.15843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:04:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.497+0000 7f930e62a740 20 client.15843 populate_metadata read hostname 'smithi033' 2024-05-15T14:04:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.497+0000 7f930e62a740 1 --2- 172.21.15.33:0/4027381415 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56266e2e6670 0x56266e2e8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.497+0000 7f930e62a740 4 client.15843 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:04:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.497+0000 7f930e62a740 1 -- 172.21.15.33:0/4027381415 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56266e240b80 con 0x56266e2e6670 2024-05-15T14:04:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.498+0000 7f9309eaf640 1 --2- 172.21.15.33:0/4027381415 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56266e2e6670 0x56266e2e8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.498+0000 7f9309eaf640 1 --2- 172.21.15.33:0/4027381415 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56266e2e6670 0x56266e2e8a80 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3269fa804dd2e1a8 server_cookie=8be3bad7f872e579 in_seq=0 out_seq=0 2024-05-15T14:04:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.498+0000 7f92ecff9640 10 client.15843 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:04:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.499+0000 7f92ecff9640 1 -- 172.21.15.33:0/4027381415 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56266e240b80 con 0x56266e2e6670 2024-05-15T14:04:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.499+0000 7f92ecff9640 10 client.15843 handle_command_reply: tid=0 2024-05-15T14:04:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.499+0000 7f930e62a740 1 client.15843 shutdown 2024-05-15T14:04:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.499+0000 7f930e62a740 1 -- 172.21.15.33:0/4027381415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266e2cc210 msgr2=0x56266e2d36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.499+0000 7f930e62a740 1 --2- 172.21.15.33:0/4027381415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266e2cc210 0x56266e2d36f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f92f4031440 tx=0x7f92f4005250 comp rx=0 tx=0).stop 2024-05-15T14:04:34.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.499+0000 7f930e62a740 1 -- 172.21.15.33:0/4027381415 shutdown_connections 2024-05-15T14:04:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.499+0000 7f930e62a740 1 --2- 172.21.15.33:0/4027381415 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56266e2e6670 0x56266e2e8a80 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.499+0000 7f930e62a740 1 --2- 172.21.15.33:0/4027381415 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56266e2d6ad0 0x56266e2d3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.499+0000 7f930e62a740 1 --2- 172.21.15.33:0/4027381415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266e2cc210 0x56266e2d36f0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.499+0000 7f92ecff9640 0 client.15843 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:04:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.499+0000 7f930e62a740 1 --2- 172.21.15.33:0/4027381415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56266e2cb8f0 0x56266e2d6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.499+0000 7f930e62a740 1 -- 172.21.15.33:0/4027381415 >> 172.21.15.33:0/4027381415 conn(0x56266e2c7e00 msgr2=0x56266e2c7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.499+0000 7f930e62a740 1 -- 172.21.15.33:0/4027381415 shutdown_connections 2024-05-15T14:04:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.499+0000 7f930e62a740 1 -- 172.21.15.33:0/4027381415 wait complete. 2024-05-15T14:04:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.499+0000 7f930e62a740 20 client.15843 trim_cache size 0 max 16384 2024-05-15T14:04:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.514+0000 7f930e62a740 1 Processor -- start 2024-05-15T14:04:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.514+0000 7f930e62a740 1 -- start start 2024-05-15T14:04:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.515+0000 7f930e62a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266e2cb8f0 0x56266e2e2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.515+0000 7f930e62a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56266e2cc210 0x56266e2d5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.515+0000 7f930e62a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56266e2d6ad0 0x56266e2ee390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.515+0000 7f930e62a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f93041b1090 con 0x56266e2cb8f0 2024-05-15T14:04:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.515+0000 7f930e62a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f93041ae900 con 0x56266e2cc210 2024-05-15T14:04:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.515+0000 7f930e62a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f93041ac750 con 0x56266e2d6ad0 2024-05-15T14:04:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.515+0000 7f9309eaf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266e2cb8f0 0x56266e2e2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.515+0000 7f9309eaf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266e2cb8f0 0x56266e2e2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51994/0 (socket says 172.21.15.33:51994) 2024-05-15T14:04:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.515+0000 7f9309eaf640 1 -- 172.21.15.33:0/1600503140 learned_addr learned my addr 172.21.15.33:0/1600503140 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:04:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.515+0000 7f9309eaf640 1 -- 172.21.15.33:0/1600503140 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56266e2d6ad0 msgr2=0x56266e2ee390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.515+0000 7f9309eaf640 1 --2- 172.21.15.33:0/1600503140 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56266e2d6ad0 0x56266e2ee390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.515+0000 7f9308ead640 1 --2- 172.21.15.33:0/1600503140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56266e2cc210 0x56266e2d5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.515+0000 7f9309eaf640 1 -- 172.21.15.33:0/1600503140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56266e2cc210 msgr2=0x56266e2d5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.515+0000 7f9309eaf640 1 --2- 172.21.15.33:0/1600503140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56266e2cc210 0x56266e2d5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.515+0000 7f9309eaf640 1 -- 172.21.15.33:0/1600503140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56266e2cc040 con 0x56266e2cb8f0 2024-05-15T14:04:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.515+0000 7f9308ead640 1 --2- 172.21.15.33:0/1600503140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56266e2cc210 0x56266e2d5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:04:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.515+0000 7f9309eaf640 1 --2- 172.21.15.33:0/1600503140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266e2cb8f0 0x56266e2e2670 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f92fc0119e0 tx=0x7f92fc000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=380603d445b395b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.515+0000 7f92eeffd640 1 -- 172.21.15.33:0/1600503140 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc02c070 con 0x56266e2cb8f0 2024-05-15T14:04:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.515+0000 7f92eeffd640 1 -- 172.21.15.33:0/1600503140 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92fc009ba0 con 0x56266e2cb8f0 2024-05-15T14:04:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.515+0000 7f92eeffd640 1 -- 172.21.15.33:0/1600503140 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc015040 con 0x56266e2cb8f0 2024-05-15T14:04:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.516+0000 7f930e62a740 1 -- 172.21.15.33:0/1600503140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266e2cb8f0 msgr2=0x56266e2e2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.516+0000 7f930e62a740 1 --2- 172.21.15.33:0/1600503140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266e2cb8f0 0x56266e2e2670 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f92fc0119e0 tx=0x7f92fc000f40 comp rx=0 tx=0).stop 2024-05-15T14:04:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.516+0000 7f930e62a740 1 -- 172.21.15.33:0/1600503140 shutdown_connections 2024-05-15T14:04:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.516+0000 7f930e62a740 1 --2- 172.21.15.33:0/1600503140 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56266e2d6ad0 0x56266e2ee390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.516+0000 7f930e62a740 1 --2- 172.21.15.33:0/1600503140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56266e2cc210 0x56266e2d5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.516+0000 7f930e62a740 1 --2- 172.21.15.33:0/1600503140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266e2cb8f0 0x56266e2e2670 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.516+0000 7f930e62a740 1 -- 172.21.15.33:0/1600503140 >> 172.21.15.33:0/1600503140 conn(0x56266e2c7e00 msgr2=0x56266e2ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.516+0000 7f930e62a740 1 -- 172.21.15.33:0/1600503140 shutdown_connections 2024-05-15T14:04:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.516+0000 7f930e62a740 1 -- 172.21.15.33:0/1600503140 wait complete. 2024-05-15T14:04:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.516+0000 7f930e62a740 1 -- start start 2024-05-15T14:04:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.516+0000 7f930e62a740 1 Processor -- start 2024-05-15T14:04:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.517+0000 7f930e62a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56266e2cb8f0 0x56266e2df0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.517+0000 7f930e62a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56266e2cc210 0x56266e2d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.517+0000 7f930e62a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266e2d6ad0 0x56266e2d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.517+0000 7f930e62a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f93041acdc0 con 0x56266e2d6ad0 2024-05-15T14:04:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.517+0000 7f930e62a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f93041ae230 con 0x56266e2cc210 2024-05-15T14:04:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.517+0000 7f930e62a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f93041ad490 con 0x56266e2cb8f0 2024-05-15T14:04:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.517+0000 7f930e62a740 10 client.0 _ll_register_callbacks cb 0x7f92f4004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:04:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.517+0000 7f9309eaf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266e2d6ad0 0x56266e2d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.517+0000 7f9309eaf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266e2d6ad0 0x56266e2d0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52006/0 (socket says 172.21.15.33:52006) 2024-05-15T14:04:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.517+0000 7f9309eaf640 1 -- 172.21.15.33:0/564040296 learned_addr learned my addr 172.21.15.33:0/564040296 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:04:34.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.517+0000 7f9309eaf640 1 --2- 172.21.15.33:0/564040296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56266e2cb8f0 0x56266e2df0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:34.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.517+0000 7f9308ead640 1 --2- 172.21.15.33:0/564040296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56266e2cc210 0x56266e2d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:34.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.517+0000 7f9309eaf640 1 -- 172.21.15.33:0/564040296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56266e2cb8f0 msgr2=0x56266e2df0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:34.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.517+0000 7f9309eaf640 1 --2- 172.21.15.33:0/564040296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56266e2cb8f0 0x56266e2df0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:34.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.517+0000 7f9309eaf640 1 -- 172.21.15.33:0/564040296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56266e2cc210 msgr2=0x56266e2d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:34.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.517+0000 7f9309eaf640 1 --2- 172.21.15.33:0/564040296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56266e2cc210 0x56266e2d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:34.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.517+0000 7f9309eaf640 1 -- 172.21.15.33:0/564040296 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56266e34ff40 con 0x56266e2d6ad0 2024-05-15T14:04:34.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.517+0000 7f9308ead640 1 --2- 172.21.15.33:0/564040296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56266e2cc210 0x56266e2d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:04:34.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.518+0000 7f9309eaf640 1 --2- 172.21.15.33:0/564040296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266e2d6ad0 0x56266e2d0a90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f92fc003c30 tx=0x7f92fc009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46fe2e0c970b0488 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:34.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.518+0000 7f92ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:04:34.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.518+0000 7f92ecff9640 1 -- 172.21.15.33:0/564040296 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92fc02c070 con 0x56266e2d6ad0 2024-05-15T14:04:34.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.518+0000 7f92ecff9640 1 -- 172.21.15.33:0/564040296 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f92fc01a660 con 0x56266e2d6ad0 2024-05-15T14:04:34.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.518+0000 7f930e62a740 1 -- 172.21.15.33:0/564040296 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92fc009ba0 con 0x56266e2d6ad0 2024-05-15T14:04:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.518+0000 7f92ecff9640 1 -- 172.21.15.33:0/564040296 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f92fc03a050 con 0x56266e2d6ad0 2024-05-15T14:04:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.518+0000 7f930e62a740 10 client.15849 fetch_fsmap learned FSMap version 27 2024-05-15T14:04:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.518+0000 7f930e62a740 1 -- 172.21.15.33:0/564040296 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56266e350fe0 con 0x56266e2d6ad0 2024-05-15T14:04:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.518+0000 7f92ecff9640 1 -- 172.21.15.33:0/564040296 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92fc009ba0 con 0x56266e2d6ad0 2024-05-15T14:04:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.518+0000 7f92ecff9640 1 -- 172.21.15.33:0/564040296 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f92fc01cce0 con 0x56266e2d6ad0 2024-05-15T14:04:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.518+0000 7f930e62a740 10 client.15849 fetch_fsmap finished waiting for FSMap version 27 2024-05-15T14:04:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.518+0000 7f930e62a740 10 client.15849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:04:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.518+0000 7f930e62a740 20 client.15849 populate_metadata read hostname 'smithi033' 2024-05-15T14:04:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.518+0000 7f930e62a740 1 --2- 172.21.15.33:0/564040296 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56266e351570 0x56266e353990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.518+0000 7f930e62a740 4 client.15849 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:04:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.518+0000 7f930e62a740 1 -- 172.21.15.33:0/564040296 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56266e23a360 con 0x56266e351570 2024-05-15T14:04:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.519+0000 7f9308ead640 1 --2- 172.21.15.33:0/564040296 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56266e351570 0x56266e353990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.519+0000 7f92ecff9640 10 client.15849 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:04:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.519+0000 7f9308ead640 1 --2- 172.21.15.33:0/564040296 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56266e351570 0x56266e353990 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2caf84c73baee7a8 server_cookie=2230e6b0682f653e in_seq=0 out_seq=0 2024-05-15T14:04:34.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.543+0000 7f92ecff9640 1 -- 172.21.15.33:0/564040296 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56266e23a360 con 0x56266e351570 2024-05-15T14:04:34.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.543+0000 7f92ecff9640 10 client.15849 handle_command_reply: tid=0 2024-05-15T14:04:34.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.543+0000 7f930e62a740 1 client.15849 shutdown 2024-05-15T14:04:34.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.543+0000 7f930e62a740 1 -- 172.21.15.33:0/564040296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266e2d6ad0 msgr2=0x56266e2d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.543+0000 7f930e62a740 1 --2- 172.21.15.33:0/564040296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266e2d6ad0 0x56266e2d0a90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f92fc003c30 tx=0x7f92fc009740 comp rx=0 tx=0).stop 2024-05-15T14:04:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.543+0000 7f930e62a740 1 -- 172.21.15.33:0/564040296 shutdown_connections 2024-05-15T14:04:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.543+0000 7f930e62a740 1 --2- 172.21.15.33:0/564040296 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56266e351570 0x56266e353990 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.543+0000 7f930e62a740 1 --2- 172.21.15.33:0/564040296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266e2d6ad0 0x56266e2d0a90 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.543+0000 7f930e62a740 1 --2- 172.21.15.33:0/564040296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56266e2cc210 0x56266e2d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.543+0000 7f92ecff9640 0 client.15849 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:04:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.543+0000 7f930e62a740 1 --2- 172.21.15.33:0/564040296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56266e2cb8f0 0x56266e2df0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.543+0000 7f930e62a740 1 -- 172.21.15.33:0/564040296 >> 172.21.15.33:0/564040296 conn(0x56266e2c7e00 msgr2=0x56266e2ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.544+0000 7f930e62a740 1 -- 172.21.15.33:0/564040296 shutdown_connections 2024-05-15T14:04:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.544+0000 7f930e62a740 1 -- 172.21.15.33:0/564040296 wait complete. 2024-05-15T14:04:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.544+0000 7f930e62a740 20 client.15849 trim_cache size 0 max 16384 2024-05-15T14:04:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:04:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:04:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:04:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.544+0000 7f930b912640 1 -- 172.21.15.33:0/1635664370 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f92f00b2950 msgr2=0x7f92f00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.544+0000 7f930b912640 1 --2- 172.21.15.33:0/1635664370 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f92f00b2950 0x7f92f00b4e10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f93000dfa70 tx=0x7f93000e5db0 comp rx=0 tx=0).stop 2024-05-15T14:04:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.544+0000 7f930b912640 1 -- 172.21.15.33:0/1635664370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9304101930 msgr2=0x7f930419a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.544+0000 7f930b912640 1 --2- 172.21.15.33:0/1635664370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9304101930 0x7f930419a190 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f930000d950 tx=0x7f930000de10 comp rx=0 tx=0).stop 2024-05-15T14:04:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.544+0000 7f930b912640 1 -- 172.21.15.33:0/1635664370 shutdown_connections 2024-05-15T14:04:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.544+0000 7f930b912640 1 --2- 172.21.15.33:0/1635664370 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f92f00b2950 0x7f92f00b4e10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.544+0000 7f930b912640 1 --2- 172.21.15.33:0/1635664370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9304107940 0x7f93041a5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.544+0000 7f930b912640 1 --2- 172.21.15.33:0/1635664370 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93041022f0 0x7f930419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.544+0000 7f930b912640 1 --2- 172.21.15.33:0/1635664370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9304101930 0x7f930419a190 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.544+0000 7f930b912640 1 -- 172.21.15.33:0/1635664370 >> 172.21.15.33:0/1635664370 conn(0x7f9304078a70 msgr2=0x7f9304104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.544+0000 7f930b912640 1 -- 172.21.15.33:0/1635664370 shutdown_connections 2024-05-15T14:04:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:34.544+0000 7f930b912640 1 -- 172.21.15.33:0/1635664370 wait complete. 2024-05-15T14:04:34.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:34 smithi052 ceph-mon[36190]: pgmap v130: 97 pgs: 97 active+clean; 67 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 1.4 MiB/s wr, 91 op/s 2024-05-15T14:04:34.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:34 smithi081 ceph-mon[36434]: pgmap v130: 97 pgs: 97 active+clean; 67 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 1.4 MiB/s wr, 91 op/s 2024-05-15T14:04:34.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:34 smithi033 ceph-mon[31650]: pgmap v130: 97 pgs: 97 active+clean; 67 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 1.4 MiB/s wr, 91 op/s 2024-05-15T14:04:35.053 INFO:tasks.workunit.client.0.smithi033.stdout: 1 213945 0.00 MB/sec execute 10 sec latency 17503.453 ms 2024-05-15T14:04:36.054 INFO:tasks.workunit.client.0.smithi033.stdout: 1 213945 0.00 MB/sec execute 11 sec latency 18503.545 ms 2024-05-15T14:04:36.742 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:36 smithi033 ceph-mon[31650]: pgmap v131: 97 pgs: 97 active+clean; 67 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 1.0 MiB/s wr, 39 op/s 2024-05-15T14:04:36.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:36 smithi052 ceph-mon[36190]: pgmap v131: 97 pgs: 97 active+clean; 67 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 1.0 MiB/s wr, 39 op/s 2024-05-15T14:04:36.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:36 smithi081 ceph-mon[36434]: pgmap v131: 97 pgs: 97 active+clean; 67 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 1.0 MiB/s wr, 39 op/s 2024-05-15T14:04:37.053 INFO:tasks.workunit.client.0.smithi033.stdout: 1 213945 0.00 MB/sec execute 12 sec latency 19503.661 ms 2024-05-15T14:04:38.053 INFO:tasks.workunit.client.0.smithi033.stdout: 1 213945 0.00 MB/sec execute 13 sec latency 20503.711 ms 2024-05-15T14:04:38.829 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T14:04:38.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:38 smithi052 ceph-mon[36190]: pgmap v132: 97 pgs: 97 active+clean; 65 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 1.5 MiB/s wr, 66 op/s 2024-05-15T14:04:38.853 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:38 smithi033 ceph-mon[31650]: pgmap v132: 97 pgs: 97 active+clean; 65 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 1.5 MiB/s wr, 66 op/s 2024-05-15T14:04:38.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:38 smithi081 ceph-mon[36434]: pgmap v132: 97 pgs: 97 active+clean; 65 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 1.5 MiB/s wr, 66 op/s 2024-05-15T14:04:39.054 INFO:tasks.workunit.client.0.smithi033.stdout: 1 213945 0.00 MB/sec execute 14 sec latency 21503.745 ms 2024-05-15T14:04:39.081 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:04:40.054 INFO:tasks.workunit.client.0.smithi033.stdout: 1 213945 0.00 MB/sec execute 15 sec latency 22503.836 ms 2024-05-15T14:04:40.555 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:40 smithi033 ceph-mon[31650]: pgmap v133: 97 pgs: 97 active+clean; 65 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.0 MiB/s wr, 53 op/s 2024-05-15T14:04:40.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.829+0000 7fb4172de700 1 -- 172.21.15.33:0/1118407714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb410102e00 msgr2=0x7fb41010f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:40.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.829+0000 7fb4172de700 1 --2- 172.21.15.33:0/1118407714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb410102e00 0x7fb41010f2f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb40800b0a0 tx=0x7fb40800b3b0 comp rx=0 tx=0).stop 2024-05-15T14:04:40.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.829+0000 7fb4172de700 1 -- 172.21.15.33:0/1118407714 shutdown_connections 2024-05-15T14:04:40.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.829+0000 7fb4172de700 1 --2- 172.21.15.33:0/1118407714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb410102e00 0x7fb41010f2f0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:40.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.829+0000 7fb4172de700 1 --2- 172.21.15.33:0/1118407714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb410108520 0x7fb410108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:40.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.829+0000 7fb4172de700 1 --2- 172.21.15.33:0/1118407714 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb410102440 0x7fb4101028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:40.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.829+0000 7fb4172de700 1 -- 172.21.15.33:0/1118407714 >> 172.21.15.33:0/1118407714 conn(0x7fb4100793f0 msgr2=0x7fb410079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:40.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.829+0000 7fb4172de700 1 -- 172.21.15.33:0/1118407714 shutdown_connections 2024-05-15T14:04:40.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.829+0000 7fb4172de700 1 -- 172.21.15.33:0/1118407714 wait complete. 2024-05-15T14:04:40.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.830+0000 7fb4172de700 1 Processor -- start 2024-05-15T14:04:40.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.830+0000 7fb4172de700 1 -- start start 2024-05-15T14:04:40.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.830+0000 7fb4172de700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb410102440 0x7fb410104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:40.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.830+0000 7fb4172de700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb410102e00 0x7fb4101045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:40.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.830+0000 7fb4172de700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb410108520 0x7fb410104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:40.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.830+0000 7fb4172de700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb410112670 con 0x7fb410108520 2024-05-15T14:04:40.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.830+0000 7fb4172de700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb4101124f0 con 0x7fb410102e00 2024-05-15T14:04:40.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.830+0000 7fb4172de700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb4101127f0 con 0x7fb410102440 2024-05-15T14:04:40.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.831+0000 7fb41587b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb410108520 0x7fb410104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:40.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.831+0000 7fb41587b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb410108520 0x7fb410104b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52030/0 (socket says 172.21.15.33:52030) 2024-05-15T14:04:40.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.831+0000 7fb41587b700 1 -- 172.21.15.33:0/3898895316 learned_addr learned my addr 172.21.15.33:0/3898895316 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:04:40.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.831+0000 7fb414879700 1 --2- 172.21.15.33:0/3898895316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb410102e00 0x7fb4101045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:40.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.831+0000 7fb41507a700 1 --2- 172.21.15.33:0/3898895316 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb410102440 0x7fb410104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:40.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.831+0000 7fb41587b700 1 -- 172.21.15.33:0/3898895316 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb410102440 msgr2=0x7fb410104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:40.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.831+0000 7fb41587b700 1 --2- 172.21.15.33:0/3898895316 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb410102440 0x7fb410104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:40.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.831+0000 7fb41587b700 1 -- 172.21.15.33:0/3898895316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb410102e00 msgr2=0x7fb4101045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:40.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.831+0000 7fb41587b700 1 --2- 172.21.15.33:0/3898895316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb410102e00 0x7fb4101045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:40.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.831+0000 7fb41587b700 1 -- 172.21.15.33:0/3898895316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4101a8180 con 0x7fb410108520 2024-05-15T14:04:40.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.831+0000 7fb41587b700 1 --2- 172.21.15.33:0/3898895316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb410108520 0x7fb410104b00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb40800b830 tx=0x7fb408004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:40.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.831+0000 7fb4067fc700 1 -- 172.21.15.33:0/3898895316 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb408021070 con 0x7fb410108520 2024-05-15T14:04:40.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.831+0000 7fb4067fc700 1 -- 172.21.15.33:0/3898895316 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb408007e30 con 0x7fb410108520 2024-05-15T14:04:40.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.831+0000 7fb4172de700 1 -- 172.21.15.33:0/3898895316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4101a8410 con 0x7fb410108520 2024-05-15T14:04:40.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.831+0000 7fb4067fc700 1 -- 172.21.15.33:0/3898895316 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4080047d0 con 0x7fb410108520 2024-05-15T14:04:40.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.831+0000 7fb4172de700 1 -- 172.21.15.33:0/3898895316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4101a88c0 con 0x7fb410108520 2024-05-15T14:04:40.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.832+0000 7fb4172de700 1 -- 172.21.15.33:0/3898895316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb41006a230 con 0x7fb410108520 2024-05-15T14:04:40.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.834+0000 7fb4067fc700 1 -- 172.21.15.33:0/3898895316 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb408007980 con 0x7fb410108520 2024-05-15T14:04:40.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.834+0000 7fb4067fc700 1 --2- 172.21.15.33:0/3898895316 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fb3fc0b29f0 0x7fb3fc0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:40.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.835+0000 7fb41507a700 1 --2- 172.21.15.33:0/3898895316 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fb3fc0b29f0 0x7fb3fc0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:40.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.835+0000 7fb4067fc700 1 -- 172.21.15.33:0/3898895316 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7fb40800f020 con 0x7fb410108520 2024-05-15T14:04:40.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.835+0000 7fb41507a700 1 --2- 172.21.15.33:0/3898895316 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fb3fc0b29f0 0x7fb3fc0b4eb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb40c006fd0 tx=0x7fb40c008040 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:40.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:40.838+0000 7fb4067fc700 1 -- 172.21.15.33:0/3898895316 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb40802a020 con 0x7fb410108520 2024-05-15T14:04:40.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:40 smithi052 ceph-mon[36190]: pgmap v133: 97 pgs: 97 active+clean; 65 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.0 MiB/s wr, 53 op/s 2024-05-15T14:04:40.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:40 smithi081 ceph-mon[36434]: pgmap v133: 97 pgs: 97 active+clean; 65 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.0 MiB/s wr, 53 op/s 2024-05-15T14:04:41.054 INFO:tasks.workunit.client.0.smithi033.stdout: 1 213945 0.00 MB/sec execute 16 sec latency 23503.910 ms 2024-05-15T14:04:41.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:41.107+0000 7fb4172de700 1 -- 172.21.15.33:0/3898895316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb41002e560 con 0x7fb410108520 2024-05-15T14:04:41.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:41.108+0000 7fb4067fc700 1 -- 172.21.15.33:0/3898895316 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11858 (secure 0 0 0) 0x7fb40802e3e0 con 0x7fb410108520 2024-05-15T14:04:41.110 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T14:04:41.110 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2030117633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2030117633},{"type":"v1","addr":"172.21.15.33:6839","nonce":2030117633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1918979061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1918979061},{"type":"v1","addr":"172.21.15.33:6841","nonce":1918979061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T13:57:19.056417+0000","modified":"2024-05-15T14:03:42.376493+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24463,"mds_2":24461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/1629697841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1629697841},{"type":"v1","addr":"172.21.15.52:6837","nonce":1629697841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.52:6839/2696044471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2696044471},{"type":"v1","addr":"172.21.15.52:6839","nonce":2696044471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.81:6839/1554395940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":1554395940},{"type":"v1","addr":"172.21.15.81:6839","nonce":1554395940}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25393":{"gid":25393,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/727032511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":727032511},{"type":"v1","addr":"172.21.15.81:6835","nonce":727032511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24463,24478,24461]},"id":1}]} 2024-05-15T14:04:41.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:41.110+0000 7fb4172de700 1 -- 172.21.15.33:0/3898895316 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fb3fc0b29f0 msgr2=0x7fb3fc0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:41.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:41.110+0000 7fb4172de700 1 --2- 172.21.15.33:0/3898895316 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fb3fc0b29f0 0x7fb3fc0b4eb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb40c006fd0 tx=0x7fb40c008040 comp rx=0 tx=0).stop 2024-05-15T14:04:41.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:41.110+0000 7fb4172de700 1 -- 172.21.15.33:0/3898895316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb410108520 msgr2=0x7fb410104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:41.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:41.110+0000 7fb4172de700 1 --2- 172.21.15.33:0/3898895316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb410108520 0x7fb410104b00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb40800b830 tx=0x7fb408004ce0 comp rx=0 tx=0).stop 2024-05-15T14:04:41.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:41.110+0000 7fb4172de700 1 -- 172.21.15.33:0/3898895316 shutdown_connections 2024-05-15T14:04:41.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:41.110+0000 7fb4172de700 1 --2- 172.21.15.33:0/3898895316 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fb3fc0b29f0 0x7fb3fc0b4eb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:41.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:41.110+0000 7fb4172de700 1 --2- 172.21.15.33:0/3898895316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb410108520 0x7fb410104b00 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:41.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:41.110+0000 7fb4172de700 1 --2- 172.21.15.33:0/3898895316 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb410102440 0x7fb410104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:41.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:41.110+0000 7fb4172de700 1 --2- 172.21.15.33:0/3898895316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb410102e00 0x7fb4101045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:41.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:41.110+0000 7fb4172de700 1 -- 172.21.15.33:0/3898895316 >> 172.21.15.33:0/3898895316 conn(0x7fb4100793f0 msgr2=0x7fb41006b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:41.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:41.110+0000 7fb4172de700 1 -- 172.21.15.33:0/3898895316 shutdown_connections 2024-05-15T14:04:41.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:41.111+0000 7fb4172de700 1 -- 172.21.15.33:0/3898895316 wait complete. 2024-05-15T14:04:41.113 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-05-15T14:04:41.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:41 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/3898895316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:04:41.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:41 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/3898895316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:04:41.912 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:41 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/3898895316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:04:41.940 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24478 quiesce db --set-id 518e8a95 --release --await' 2024-05-15T14:04:41.941 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24478 quiesce db --set-id 518e8a95 --release --await 2024-05-15T14:04:42.054 INFO:tasks.workunit.client.0.smithi033.stdout: 1 213945 0.00 MB/sec execute 17 sec latency 24503.938 ms 2024-05-15T14:04:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:04:42.665 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:42 smithi033 ceph-mon[31650]: pgmap v134: 97 pgs: 97 active+clean; 65 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.0 MiB/s wr, 53 op/s 2024-05-15T14:04:42.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:42 smithi052 ceph-mon[36190]: pgmap v134: 97 pgs: 97 active+clean; 65 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.0 MiB/s wr, 53 op/s 2024-05-15T14:04:42.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:42 smithi081 ceph-mon[36434]: pgmap v134: 97 pgs: 97 active+clean; 65 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.0 MiB/s wr, 53 op/s 2024-05-15T14:04:43.054 INFO:tasks.workunit.client.0.smithi033.stdout: 1 213945 0.00 MB/sec execute 18 sec latency 25503.982 ms 2024-05-15T14:04:43.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.873+0000 7fabd004c700 1 -- 172.21.15.33:0/893399674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabc8108910 msgr2=0x7fabc8110490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:43.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.873+0000 7fabd004c700 1 --2- 172.21.15.33:0/893399674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabc8108910 0x7fabc8110490 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fabc000b0d0 tx=0x7fabc000b3e0 comp rx=0 tx=0).stop 2024-05-15T14:04:43.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.873+0000 7fabd004c700 1 -- 172.21.15.33:0/893399674 shutdown_connections 2024-05-15T14:04:43.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.873+0000 7fabd004c700 1 --2- 172.21.15.33:0/893399674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabc8108910 0x7fabc8110490 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.873+0000 7fabd004c700 1 --2- 172.21.15.33:0/893399674 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabc8103820 0x7fabc8103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.873+0000 7fabd004c700 1 --2- 172.21.15.33:0/893399674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabc81041d0 0x7fabc81081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.873+0000 7fabd004c700 1 -- 172.21.15.33:0/893399674 >> 172.21.15.33:0/893399674 conn(0x7fabc807a6b0 msgr2=0x7fabc807aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.873+0000 7fabd004c700 1 -- 172.21.15.33:0/893399674 shutdown_connections 2024-05-15T14:04:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.873+0000 7fabd004c700 1 -- 172.21.15.33:0/893399674 wait complete. 2024-05-15T14:04:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.873+0000 7fabd004c700 1 Processor -- start 2024-05-15T14:04:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.874+0000 7fabd004c700 1 -- start start 2024-05-15T14:04:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.874+0000 7fabd004c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabc8103820 0x7fabc8075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.874+0000 7fabd004c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabc81041d0 0x7fabc8075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.874+0000 7fabd004c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabc8108910 0x7fabc80762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.874+0000 7fabd004c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fabc81136e0 con 0x7fabc81041d0 2024-05-15T14:04:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.874+0000 7fabd004c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fabc8113560 con 0x7fabc8103820 2024-05-15T14:04:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.874+0000 7fabd004c700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fabc8113860 con 0x7fabc8108910 2024-05-15T14:04:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.874+0000 7fabcd5e7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabc81041d0 0x7fabc8075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.874+0000 7fabcd5e7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabc81041d0 0x7fabc8075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44688/0 (socket says 172.21.15.33:44688) 2024-05-15T14:04:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.874+0000 7fabcd5e7700 1 -- 172.21.15.33:0/22542149 learned_addr learned my addr 172.21.15.33:0/22542149 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:04:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.874+0000 7fabce5e9700 1 --2- 172.21.15.33:0/22542149 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabc8108910 0x7fabc80762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.874+0000 7fabcdde8700 1 --2- 172.21.15.33:0/22542149 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabc8103820 0x7fabc8075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.874+0000 7fabcd5e7700 1 -- 172.21.15.33:0/22542149 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabc8108910 msgr2=0x7fabc80762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.874+0000 7fabcd5e7700 1 --2- 172.21.15.33:0/22542149 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabc8108910 0x7fabc80762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:43.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.874+0000 7fabcd5e7700 1 -- 172.21.15.33:0/22542149 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabc8103820 msgr2=0x7fabc8075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:43.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.874+0000 7fabcd5e7700 1 --2- 172.21.15.33:0/22542149 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabc8103820 0x7fabc8075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:43.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.874+0000 7fabcd5e7700 1 -- 172.21.15.33:0/22542149 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc81a8040 con 0x7fabc81041d0 2024-05-15T14:04:43.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.874+0000 7fabcd5e7700 1 --2- 172.21.15.33:0/22542149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabc81041d0 0x7fabc8075d80 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fabb800d950 tx=0x7fabb800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:43.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.875+0000 7fabbeffd700 1 -- 172.21.15.33:0/22542149 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb8016070 con 0x7fabc81041d0 2024-05-15T14:04:43.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.875+0000 7fabbeffd700 1 -- 172.21.15.33:0/22542149 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabb800fd40 con 0x7fabc81041d0 2024-05-15T14:04:43.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.875+0000 7fabd004c700 1 -- 172.21.15.33:0/22542149 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc81a8330 con 0x7fabc81041d0 2024-05-15T14:04:43.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.875+0000 7fabbeffd700 1 -- 172.21.15.33:0/22542149 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb800e3b0 con 0x7fabc81041d0 2024-05-15T14:04:43.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.875+0000 7fabd004c700 1 -- 172.21.15.33:0/22542149 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc81a88c0 con 0x7fabc81041d0 2024-05-15T14:04:43.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.877+0000 7fabbeffd700 1 -- 172.21.15.33:0/22542149 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fabb8019070 con 0x7fabc81041d0 2024-05-15T14:04:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.878+0000 7fabbeffd700 1 --2- 172.21.15.33:0/22542149 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fabb40b2a70 0x7fabb40b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.878+0000 7fabbeffd700 1 -- 172.21.15.33:0/22542149 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7fabb80e1fd0 con 0x7fabc81041d0 2024-05-15T14:04:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.878+0000 7fabcdde8700 1 --2- 172.21.15.33:0/22542149 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fabb40b2a70 0x7fabb40b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.879+0000 7fabcdde8700 1 --2- 172.21.15.33:0/22542149 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fabb40b2a70 0x7fabb40b4f30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fabc4007be0 tx=0x7fabc40073d0 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.879+0000 7fabda8e2b80 1 Processor -- start 2024-05-15T14:04:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.879+0000 7fabda8e2b80 1 -- start start 2024-05-15T14:04:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.879+0000 7fabda8e2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a60097da0 0x563a60098180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.879+0000 7fabda8e2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a600986c0 0x563a600924b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.879+0000 7fabda8e2b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563a60092ad0 0x563a60092f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.879+0000 7fabda8e2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563a5fff0d90 con 0x563a600986c0 2024-05-15T14:04:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.879+0000 7fabda8e2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563a5ffe0930 con 0x563a60097da0 2024-05-15T14:04:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.879+0000 7fabda8e2b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x563a5ffe4970 con 0x563a60092ad0 2024-05-15T14:04:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.879+0000 7fabcdde8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a60097da0 0x563a60098180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:43.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.879+0000 7fabcd5e7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a600986c0 0x563a600924b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:43.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.879+0000 7fabcdde8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a60097da0 0x563a60098180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:53098/0 (socket says 172.21.15.33:53098) 2024-05-15T14:04:43.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.879+0000 7fabcd5e7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a600986c0 0x563a600924b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44692/0 (socket says 172.21.15.33:44692) 2024-05-15T14:04:43.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.879+0000 7fabcdde8700 1 -- 172.21.15.33:0/3395040099 learned_addr learned my addr 172.21.15.33:0/3395040099 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:04:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.880+0000 7fabce5e9700 1 --2- 172.21.15.33:0/3395040099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563a60092ad0 0x563a60092f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.880+0000 7fabcd5e7700 1 -- 172.21.15.33:0/3395040099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563a60092ad0 msgr2=0x563a60092f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.880+0000 7fabcd5e7700 1 --2- 172.21.15.33:0/3395040099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563a60092ad0 0x563a60092f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.880+0000 7fabcd5e7700 1 -- 172.21.15.33:0/3395040099 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a60097da0 msgr2=0x563a60098180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.880+0000 7fabcd5e7700 1 --2- 172.21.15.33:0/3395040099 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a60097da0 0x563a60098180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.880+0000 7fabcd5e7700 1 -- 172.21.15.33:0/3395040099 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a5ff309d0 con 0x563a600986c0 2024-05-15T14:04:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.880+0000 7fabce5e9700 1 --2- 172.21.15.33:0/3395040099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563a60092ad0 0x563a60092f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:04:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.880+0000 7fabcd5e7700 1 --2- 172.21.15.33:0/3395040099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a600986c0 0x563a600924b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fabb80df730 tx=0x7fabb8005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=718d7b743913c840 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.880+0000 7fabb3fff700 1 -- 172.21.15.33:0/3395040099 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb8020040 con 0x563a600986c0 2024-05-15T14:04:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.880+0000 7fabb3fff700 1 -- 172.21.15.33:0/3395040099 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabb80b2120 con 0x563a600986c0 2024-05-15T14:04:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.880+0000 7fabb3fff700 1 -- 172.21.15.33:0/3395040099 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb80b1ef0 con 0x563a600986c0 2024-05-15T14:04:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.881+0000 7fabda8e2b80 1 -- 172.21.15.33:0/3395040099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a600986c0 msgr2=0x563a600924b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:43.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.881+0000 7fabda8e2b80 1 --2- 172.21.15.33:0/3395040099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a600986c0 0x563a600924b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fabb80df730 tx=0x7fabb8005910 comp rx=0 tx=0).stop 2024-05-15T14:04:43.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.881+0000 7fabda8e2b80 1 -- 172.21.15.33:0/3395040099 shutdown_connections 2024-05-15T14:04:43.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.881+0000 7fabda8e2b80 1 --2- 172.21.15.33:0/3395040099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563a60092ad0 0x563a60092f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:43.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.881+0000 7fabda8e2b80 1 --2- 172.21.15.33:0/3395040099 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a60097da0 0x563a60098180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:43.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.881+0000 7fabda8e2b80 1 --2- 172.21.15.33:0/3395040099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a600986c0 0x563a600924b0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:43.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.881+0000 7fabda8e2b80 1 -- 172.21.15.33:0/3395040099 >> 172.21.15.33:0/3395040099 conn(0x563a5ffbd350 msgr2=0x563a6008cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:43.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.881+0000 7fabda8e2b80 1 -- 172.21.15.33:0/3395040099 shutdown_connections 2024-05-15T14:04:43.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.881+0000 7fabda8e2b80 1 -- 172.21.15.33:0/3395040099 wait complete. 2024-05-15T14:04:43.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.882+0000 7fabda8e2b80 1 -- start start 2024-05-15T14:04:43.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.882+0000 7fabda8e2b80 1 Processor -- start 2024-05-15T14:04:43.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.882+0000 7fabda8e2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a60092ad0 0x563a6009e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:43.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.882+0000 7fabda8e2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a60097da0 0x563a6009b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:43.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.882+0000 7fabda8e2b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563a600986c0 0x563a6009bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:43.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.882+0000 7fabda8e2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563a5fee70a0 con 0x563a60092ad0 2024-05-15T14:04:43.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.882+0000 7fabda8e2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563a5ff828c0 con 0x563a60097da0 2024-05-15T14:04:43.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.882+0000 7fabda8e2b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x563a6009f0a0 con 0x563a600986c0 2024-05-15T14:04:43.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.882+0000 7fabda8e2b80 10 client.0 _ll_register_callbacks cb 0x563a5ff77680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:04:43.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.882+0000 7fabcdde8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a60092ad0 0x563a6009e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:43.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.882+0000 7fabcdde8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a60092ad0 0x563a6009e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44696/0 (socket says 172.21.15.33:44696) 2024-05-15T14:04:43.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.882+0000 7fabcdde8700 1 -- 172.21.15.33:0/1150777810 learned_addr learned my addr 172.21.15.33:0/1150777810 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:04:43.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.882+0000 7fabcd5e7700 1 --2- 172.21.15.33:0/1150777810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a60097da0 0x563a6009b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:43.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.883+0000 7fabce5e9700 1 --2- 172.21.15.33:0/1150777810 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563a600986c0 0x563a6009bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:43.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.883+0000 7fabcdde8700 1 -- 172.21.15.33:0/1150777810 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563a600986c0 msgr2=0x563a6009bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:43.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.883+0000 7fabcdde8700 1 --2- 172.21.15.33:0/1150777810 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563a600986c0 0x563a6009bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:43.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.883+0000 7fabcdde8700 1 -- 172.21.15.33:0/1150777810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a60097da0 msgr2=0x563a6009b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:43.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.883+0000 7fabcdde8700 1 --2- 172.21.15.33:0/1150777810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a60097da0 0x563a6009b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:43.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.883+0000 7fabcdde8700 1 -- 172.21.15.33:0/1150777810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a5ff7f790 con 0x563a60092ad0 2024-05-15T14:04:43.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.883+0000 7fabce5e9700 1 --2- 172.21.15.33:0/1150777810 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563a600986c0 0x563a6009bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:04:43.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.883+0000 7fabcd5e7700 1 --2- 172.21.15.33:0/1150777810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a60097da0 0x563a6009b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:04:43.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.883+0000 7fabcdde8700 1 --2- 172.21.15.33:0/1150777810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a60092ad0 0x563a6009e4b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fabc40042b0 tx=0x7fabc40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d7276095598276 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:43.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.883+0000 7fabb17fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:04:43.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.883+0000 7fabb17fa700 1 -- 172.21.15.33:0/1150777810 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc4024070 con 0x563a60092ad0 2024-05-15T14:04:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.883+0000 7fabb17fa700 1 -- 172.21.15.33:0/1150777810 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7fabc4003a90 con 0x563a60092ad0 2024-05-15T14:04:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.883+0000 7fabda8e2b80 1 -- 172.21.15.33:0/1150777810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabb80b2120 con 0x563a60092ad0 2024-05-15T14:04:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.883+0000 7fabb17fa700 1 -- 172.21.15.33:0/1150777810 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fabc4030050 con 0x563a60092ad0 2024-05-15T14:04:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.883+0000 7fabda8e2b80 10 client.15867 fetch_fsmap learned FSMap version 27 2024-05-15T14:04:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.883+0000 7fabda8e2b80 1 -- 172.21.15.33:0/1150777810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563a600a9910 con 0x563a60092ad0 2024-05-15T14:04:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.884+0000 7fabb17fa700 1 -- 172.21.15.33:0/1150777810 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabc4015070 con 0x563a60092ad0 2024-05-15T14:04:43.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.884+0000 7fabb17fa700 1 -- 172.21.15.33:0/1150777810 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fabc4020410 con 0x563a60092ad0 2024-05-15T14:04:43.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.884+0000 7fabda8e2b80 10 client.15867 fetch_fsmap finished waiting for FSMap version 27 2024-05-15T14:04:43.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.884+0000 7fabda8e2b80 10 client.15867 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:04:43.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.884+0000 7fabda8e2b80 20 client.15867 populate_metadata read hostname 'smithi033' 2024-05-15T14:04:43.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.884+0000 7fabda8e2b80 1 --2- 172.21.15.33:0/1150777810 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x563a600abf40 0x563a600ae380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:43.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.884+0000 7fabda8e2b80 4 client.15867 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:04:43.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.884+0000 7fabda8e2b80 1 -- 172.21.15.33:0/1150777810 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fabb80b2120 con 0x563a600abf40 2024-05-15T14:04:43.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.884+0000 7fabcd5e7700 1 --2- 172.21.15.33:0/1150777810 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x563a600abf40 0x563a600ae380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:43.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.885+0000 7fabcd5e7700 1 --2- 172.21.15.33:0/1150777810 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x563a600abf40 0x563a600ae380 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a54f654969ca58db server_cookie=f8cf351bf4167f72 in_seq=0 out_seq=0 2024-05-15T14:04:43.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.885+0000 7fabb17fa700 10 client.15867 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:04:43.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.885+0000 7fabb17fa700 1 -- 172.21.15.33:0/1150777810 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fabb80b2120 con 0x563a600abf40 2024-05-15T14:04:43.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.885+0000 7fabb17fa700 10 client.15867 handle_command_reply: tid=0 2024-05-15T14:04:43.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.885+0000 7fabda8e2b80 1 client.15867 shutdown 2024-05-15T14:04:43.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.885+0000 7fabda8e2b80 1 -- 172.21.15.33:0/1150777810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a60092ad0 msgr2=0x563a6009e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:43.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.885+0000 7fabda8e2b80 1 --2- 172.21.15.33:0/1150777810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a60092ad0 0x563a6009e4b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fabc40042b0 tx=0x7fabc40047c0 comp rx=0 tx=0).stop 2024-05-15T14:04:43.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.885+0000 7fabda8e2b80 1 -- 172.21.15.33:0/1150777810 shutdown_connections 2024-05-15T14:04:43.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.885+0000 7fabda8e2b80 1 --2- 172.21.15.33:0/1150777810 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563a600986c0 0x563a6009bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:43.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.885+0000 7fabda8e2b80 1 --2- 172.21.15.33:0/1150777810 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x563a600abf40 0x563a600ae380 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:43.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.885+0000 7fabda8e2b80 1 --2- 172.21.15.33:0/1150777810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a60092ad0 0x563a6009e4b0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:43.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.885+0000 7fabda8e2b80 1 --2- 172.21.15.33:0/1150777810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a60097da0 0x563a6009b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:43.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.886+0000 7fabb17fa700 0 client.15867 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:04:43.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.886+0000 7fabda8e2b80 1 -- 172.21.15.33:0/1150777810 >> 172.21.15.33:0/1150777810 conn(0x563a5ffbd350 msgr2=0x563a600877d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:43.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.886+0000 7fabda8e2b80 1 -- 172.21.15.33:0/1150777810 shutdown_connections 2024-05-15T14:04:43.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.886+0000 7fabda8e2b80 1 -- 172.21.15.33:0/1150777810 wait complete. 2024-05-15T14:04:43.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.886+0000 7fabda8e2b80 20 client.15867 trim_cache size 0 max 16384 2024-05-15T14:04:43.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.906+0000 7fabda8e2b80 1 Processor -- start 2024-05-15T14:04:43.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.906+0000 7fabda8e2b80 1 -- start start 2024-05-15T14:04:43.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.906+0000 7fabda8e2b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563a60092ad0 0x563a60095fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:43.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.906+0000 7fabda8e2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a60097da0 0x563a600878e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:43.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.906+0000 7fabda8e2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a600986c0 0x563a600a7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:43.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.906+0000 7fabda8e2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fabc81ae560 con 0x563a600986c0 2024-05-15T14:04:43.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.906+0000 7fabda8e2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fabc81abd90 con 0x563a60097da0 2024-05-15T14:04:43.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.906+0000 7fabda8e2b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fabc81a9d30 con 0x563a60092ad0 2024-05-15T14:04:43.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.906+0000 7fabce5e9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a600986c0 0x563a600a7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:43.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.906+0000 7fabce5e9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a600986c0 0x563a600a7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44702/0 (socket says 172.21.15.33:44702) 2024-05-15T14:04:43.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.906+0000 7fabce5e9700 1 -- 172.21.15.33:0/2885985045 learned_addr learned my addr 172.21.15.33:0/2885985045 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:04:43.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.906+0000 7fabcd5e7700 1 --2- 172.21.15.33:0/2885985045 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a60097da0 0x563a600878e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:43.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.906+0000 7fabcdde8700 1 --2- 172.21.15.33:0/2885985045 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563a60092ad0 0x563a60095fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:43.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.907+0000 7fabce5e9700 1 -- 172.21.15.33:0/2885985045 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563a60092ad0 msgr2=0x563a60095fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:43.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.907+0000 7fabce5e9700 1 --2- 172.21.15.33:0/2885985045 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563a60092ad0 0x563a60095fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:43.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.907+0000 7fabce5e9700 1 -- 172.21.15.33:0/2885985045 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a60097da0 msgr2=0x563a600878e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:43.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.907+0000 7fabce5e9700 1 --2- 172.21.15.33:0/2885985045 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a60097da0 0x563a600878e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:43.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.907+0000 7fabce5e9700 1 -- 172.21.15.33:0/2885985045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a600983d0 con 0x563a600986c0 2024-05-15T14:04:43.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.907+0000 7fabcd5e7700 1 --2- 172.21.15.33:0/2885985045 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a60097da0 0x563a600878e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:04:43.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.907+0000 7fabcdde8700 1 --2- 172.21.15.33:0/2885985045 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563a60092ad0 0x563a60095fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:04:43.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.907+0000 7fabce5e9700 1 --2- 172.21.15.33:0/2885985045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a600986c0 0x563a600a7fb0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fabc0015000 tx=0x7fabc000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fadfc1aa84cc0e41 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:43.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.907+0000 7fabb37fe700 1 -- 172.21.15.33:0/2885985045 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc0003c00 con 0x563a600986c0 2024-05-15T14:04:43.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.907+0000 7fabb37fe700 1 -- 172.21.15.33:0/2885985045 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabc0003da0 con 0x563a600986c0 2024-05-15T14:04:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.907+0000 7fabb37fe700 1 -- 172.21.15.33:0/2885985045 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc00052d0 con 0x563a600986c0 2024-05-15T14:04:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.908+0000 7fabda8e2b80 1 -- 172.21.15.33:0/2885985045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a600986c0 msgr2=0x563a600a7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.908+0000 7fabda8e2b80 1 --2- 172.21.15.33:0/2885985045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a600986c0 0x563a600a7fb0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fabc0015000 tx=0x7fabc000b040 comp rx=0 tx=0).stop 2024-05-15T14:04:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.908+0000 7fabda8e2b80 1 -- 172.21.15.33:0/2885985045 shutdown_connections 2024-05-15T14:04:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.908+0000 7fabda8e2b80 1 --2- 172.21.15.33:0/2885985045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a600986c0 0x563a600a7fb0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.908+0000 7fabda8e2b80 1 --2- 172.21.15.33:0/2885985045 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563a60092ad0 0x563a60095fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.908+0000 7fabda8e2b80 1 --2- 172.21.15.33:0/2885985045 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a60097da0 0x563a600878e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.908+0000 7fabda8e2b80 1 -- 172.21.15.33:0/2885985045 >> 172.21.15.33:0/2885985045 conn(0x563a5ffbd350 msgr2=0x563a60097240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.908+0000 7fabda8e2b80 1 -- 172.21.15.33:0/2885985045 shutdown_connections 2024-05-15T14:04:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.908+0000 7fabda8e2b80 1 -- 172.21.15.33:0/2885985045 wait complete. 2024-05-15T14:04:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.908+0000 7fabda8e2b80 1 -- start start 2024-05-15T14:04:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.908+0000 7fabda8e2b80 1 Processor -- start 2024-05-15T14:04:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.909+0000 7fabda8e2b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563a60092ad0 0x563a600a1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.909+0000 7fabda8e2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a60097da0 0x563a600a1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:43.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.909+0000 7fabda8e2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a600986c0 0x563a60116810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:43.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.909+0000 7fabda8e2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fabc81aa250 con 0x563a60097da0 2024-05-15T14:04:43.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.909+0000 7fabda8e2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fabc81ab6c0 con 0x563a600986c0 2024-05-15T14:04:43.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.909+0000 7fabda8e2b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fabc81aa920 con 0x563a60092ad0 2024-05-15T14:04:43.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.909+0000 7fabcd5e7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a60097da0 0x563a600a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:43.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.909+0000 7fabda8e2b80 10 client.0 _ll_register_callbacks cb 0x563a5ff77680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:04:43.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.909+0000 7fabcd5e7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a60097da0 0x563a600a1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44704/0 (socket says 172.21.15.33:44704) 2024-05-15T14:04:43.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.909+0000 7fabcd5e7700 1 -- 172.21.15.33:0/2953535134 learned_addr learned my addr 172.21.15.33:0/2953535134 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:04:43.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.909+0000 7fabcdde8700 1 --2- 172.21.15.33:0/2953535134 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563a60092ad0 0x563a600a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:43.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.909+0000 7fabce5e9700 1 --2- 172.21.15.33:0/2953535134 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a600986c0 0x563a60116810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:43.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.909+0000 7fabcd5e7700 1 -- 172.21.15.33:0/2953535134 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563a60092ad0 msgr2=0x563a600a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:43.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.909+0000 7fabcd5e7700 1 --2- 172.21.15.33:0/2953535134 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563a60092ad0 0x563a600a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:43.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.909+0000 7fabcd5e7700 1 -- 172.21.15.33:0/2953535134 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a600986c0 msgr2=0x563a60116810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:43.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.909+0000 7fabcd5e7700 1 --2- 172.21.15.33:0/2953535134 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a600986c0 0x563a60116810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:43.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.909+0000 7fabcd5e7700 1 -- 172.21.15.33:0/2953535134 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a60116d50 con 0x563a60097da0 2024-05-15T14:04:43.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.909+0000 7fabce5e9700 1 --2- 172.21.15.33:0/2953535134 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a600986c0 0x563a60116810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:04:43.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.909+0000 7fabcd5e7700 1 --2- 172.21.15.33:0/2953535134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a60097da0 0x563a600a1840 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fabb80a8fc0 tx=0x7fabb8010a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b24aee91e1c07047 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:43.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.909+0000 7fabb17fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:04:43.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.909+0000 7fabb17fa700 1 -- 172.21.15.33:0/2953535134 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb8020040 con 0x563a60097da0 2024-05-15T14:04:43.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.910+0000 7fabb17fa700 1 -- 172.21.15.33:0/2953535134 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7fabb80b1c90 con 0x563a60097da0 2024-05-15T14:04:43.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.910+0000 7fabda8e2b80 1 -- 172.21.15.33:0/2953535134 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabc0003da0 con 0x563a60097da0 2024-05-15T14:04:43.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.910+0000 7fabb17fa700 1 -- 172.21.15.33:0/2953535134 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fabb80b2120 con 0x563a60097da0 2024-05-15T14:04:43.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.910+0000 7fabda8e2b80 10 client.15873 fetch_fsmap learned FSMap version 27 2024-05-15T14:04:43.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.910+0000 7fabda8e2b80 1 -- 172.21.15.33:0/2953535134 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563a60117ec0 con 0x563a60097da0 2024-05-15T14:04:43.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.910+0000 7fabb17fa700 1 -- 172.21.15.33:0/2953535134 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabc0003da0 con 0x563a60097da0 2024-05-15T14:04:43.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.910+0000 7fabb17fa700 1 -- 172.21.15.33:0/2953535134 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fabb80a8520 con 0x563a60097da0 2024-05-15T14:04:43.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.910+0000 7fabda8e2b80 10 client.15873 fetch_fsmap finished waiting for FSMap version 27 2024-05-15T14:04:43.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.910+0000 7fabda8e2b80 10 client.15873 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:04:43.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.910+0000 7fabda8e2b80 20 client.15873 populate_metadata read hostname 'smithi033' 2024-05-15T14:04:43.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.910+0000 7fabda8e2b80 1 --2- 172.21.15.33:0/2953535134 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x563a601181b0 0x563a6011a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:43.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.910+0000 7fabda8e2b80 4 client.15873 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "518e8a95", "release": true, "await": true}] 2024-05-15T14:04:43.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.910+0000 7fabda8e2b80 1 -- 172.21.15.33:0/2953535134 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "quiesce db", "set_id": "518e8a95", "release": true, "await": true}) -- 0x563a6005e460 con 0x563a601181b0 2024-05-15T14:04:43.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.910+0000 7fabcdde8700 1 --2- 172.21.15.33:0/2953535134 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x563a601181b0 0x563a6011a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:43.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.911+0000 7fabb17fa700 10 client.15873 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:04:43.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.911+0000 7fabcdde8700 1 --2- 172.21.15.33:0/2953535134 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x563a601181b0 0x563a6011a5a0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc0e65a1da2ea8a1 server_cookie=66497b772b6d4690 in_seq=0 out_seq=0 2024-05-15T14:04:43.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.918+0000 7fabb17fa700 1 -- 172.21.15.33:0/2953535134 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563a6005e460 con 0x563a601181b0 2024-05-15T14:04:43.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.918+0000 7fabb17fa700 10 client.15873 handle_command_reply: tid=0 2024-05-15T14:04:43.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.918+0000 7fabda8e2b80 1 client.15873 shutdown 2024-05-15T14:04:43.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.918+0000 7fabda8e2b80 1 -- 172.21.15.33:0/2953535134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a60097da0 msgr2=0x563a600a1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:43.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.918+0000 7fabda8e2b80 1 --2- 172.21.15.33:0/2953535134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a60097da0 0x563a600a1840 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fabb80a8fc0 tx=0x7fabb8010a40 comp rx=0 tx=0).stop 2024-05-15T14:04:43.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.918+0000 7fabda8e2b80 1 -- 172.21.15.33:0/2953535134 shutdown_connections 2024-05-15T14:04:43.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.918+0000 7fabda8e2b80 1 --2- 172.21.15.33:0/2953535134 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a600986c0 0x563a60116810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:43.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.918+0000 7fabda8e2b80 1 --2- 172.21.15.33:0/2953535134 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563a60092ad0 0x563a600a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:43.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.918+0000 7fabda8e2b80 1 --2- 172.21.15.33:0/2953535134 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x563a601181b0 0x563a6011a5a0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:43.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.918+0000 7fabda8e2b80 1 --2- 172.21.15.33:0/2953535134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a60097da0 0x563a600a1840 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:43.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.918+0000 7fabb17fa700 0 client.15873 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:04:43.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.918+0000 7fabda8e2b80 1 -- 172.21.15.33:0/2953535134 >> 172.21.15.33:0/2953535134 conn(0x563a5ffbd350 msgr2=0x563a600914a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:43.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.918+0000 7fabda8e2b80 1 -- 172.21.15.33:0/2953535134 shutdown_connections 2024-05-15T14:04:43.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.918+0000 7fabda8e2b80 1 -- 172.21.15.33:0/2953535134 wait complete. 2024-05-15T14:04:43.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.918+0000 7fabda8e2b80 20 client.15873 trim_cache size 0 max 16384 2024-05-15T14:04:43.924 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T14:04:43.924 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 27, 2024-05-15T14:04:43.924 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24478, 2024-05-15T14:04:43.924 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 16, 2024-05-15T14:04:43.924 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-05-15T14:04:43.924 INFO:teuthology.orchestra.run.smithi033.stdout: "518e8a95": { 2024-05-15T14:04:43.924 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 16, 2024-05-15T14:04:43.924 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-05-15T14:04:43.924 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:04:43.924 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-05-15T14:04:43.924 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:04:43.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:04:43.924 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 90.0, 2024-05-15T14:04:43.924 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 146.5, 2024-05-15T14:04:43.925 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-05-15T14:04:43.925 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-05-15T14:04:43.925 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-05-15T14:04:43.925 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:04:43.925 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-05-15T14:04:43.925 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:04:43.925 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:04:43.925 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:04:43.925 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:04:43.925 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:04:43.925 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:04:43.925 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T14:04:43.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.919+0000 7fabd004c700 1 -- 172.21.15.33:0/22542149 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fabb40b2a70 msgr2=0x7fabb40b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:43.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.919+0000 7fabd004c700 1 --2- 172.21.15.33:0/22542149 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fabb40b2a70 0x7fabb40b4f30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fabc4007be0 tx=0x7fabc40073d0 comp rx=0 tx=0).stop 2024-05-15T14:04:43.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.919+0000 7fabd004c700 1 -- 172.21.15.33:0/22542149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabc81041d0 msgr2=0x7fabc8075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:43.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.919+0000 7fabd004c700 1 --2- 172.21.15.33:0/22542149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabc81041d0 0x7fabc8075d80 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fabb800d950 tx=0x7fabb800dc60 comp rx=0 tx=0).stop 2024-05-15T14:04:43.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.919+0000 7fabd004c700 1 -- 172.21.15.33:0/22542149 shutdown_connections 2024-05-15T14:04:43.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.919+0000 7fabd004c700 1 --2- 172.21.15.33:0/22542149 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fabb40b2a70 0x7fabb40b4f30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:43.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.919+0000 7fabd004c700 1 --2- 172.21.15.33:0/22542149 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabc8108910 0x7fabc80762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:43.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.919+0000 7fabd004c700 1 --2- 172.21.15.33:0/22542149 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fabc8103820 0x7fabc8075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:43.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.919+0000 7fabd004c700 1 --2- 172.21.15.33:0/22542149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabc81041d0 0x7fabc8075d80 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:43.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.919+0000 7fabd004c700 1 -- 172.21.15.33:0/22542149 >> 172.21.15.33:0/22542149 conn(0x7fabc807a6b0 msgr2=0x7fabc81016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:43.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.919+0000 7fabd004c700 1 -- 172.21.15.33:0/22542149 shutdown_connections 2024-05-15T14:04:43.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:04:43.919+0000 7fabd004c700 1 -- 172.21.15.33:0/22542149 wait complete. 2024-05-15T14:04:44.054 INFO:tasks.workunit.client.0.smithi033.stdout: 1 214308 0.16 MB/sec execute 19 sec latency 26371.832 ms 2024-05-15T14:04:44.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:44 smithi033 ceph-mon[31650]: pgmap v135: 97 pgs: 97 active+clean; 68 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 1.8 MiB/s wr, 79 op/s 2024-05-15T14:04:44.759 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '518e8a95', total seconds elapsed: 32.5 2024-05-15T14:04:44.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:44 smithi052 ceph-mon[36190]: pgmap v135: 97 pgs: 97 active+clean; 68 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 1.8 MiB/s wr, 79 op/s 2024-05-15T14:04:44.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:44 smithi081 ceph-mon[36434]: pgmap v135: 97 pgs: 97 active+clean; 68 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 1.8 MiB/s wr, 79 op/s 2024-05-15T14:04:45.054 INFO:tasks.workunit.client.0.smithi033.stdout: 1 216782 0.86 MB/sec execute 20 sec latency 12.760 ms 2024-05-15T14:04:46.054 INFO:tasks.workunit.client.0.smithi033.stdout: 1 218871 1.16 MB/sec execute 21 sec latency 11.418 ms 2024-05-15T14:04:46.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:46 smithi052 ceph-mon[36190]: pgmap v136: 97 pgs: 97 active+clean; 68 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 1.2 MiB/s wr, 53 op/s 2024-05-15T14:04:46.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:46 smithi081 ceph-mon[36434]: pgmap v136: 97 pgs: 97 active+clean; 68 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 1.2 MiB/s wr, 53 op/s 2024-05-15T14:04:46.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:46 smithi033 ceph-mon[31650]: pgmap v136: 97 pgs: 97 active+clean; 68 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 1.2 MiB/s wr, 53 op/s 2024-05-15T14:04:47.054 INFO:tasks.workunit.client.0.smithi033.stdout: 1 220585 1.65 MB/sec execute 22 sec latency 22.833 ms 2024-05-15T14:04:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.641+0000 7f632e245640 1 -- 172.21.15.33:0/948735080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6328103cc0 msgr2=0x7f63281040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:47.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.641+0000 7f632e245640 1 --2- 172.21.15.33:0/948735080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6328103cc0 0x7f63281040a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f6318009900 tx=0x7f631802a480 comp rx=0 tx=0).stop 2024-05-15T14:04:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.641+0000 7f632e245640 1 -- 172.21.15.33:0/948735080 shutdown_connections 2024-05-15T14:04:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.641+0000 7f632e245640 1 --2- 172.21.15.33:0/948735080 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f632810d090 0x7f632810f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.641+0000 7f632e245640 1 --2- 172.21.15.33:0/948735080 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63281045e0 0x7f632810cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.641+0000 7f632e245640 1 --2- 172.21.15.33:0/948735080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6328103cc0 0x7f63281040a0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.641+0000 7f632e245640 1 -- 172.21.15.33:0/948735080 >> 172.21.15.33:0/948735080 conn(0x7f6328078a70 msgr2=0x7f6328078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.641+0000 7f632e245640 1 -- 172.21.15.33:0/948735080 shutdown_connections 2024-05-15T14:04:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.641+0000 7f632e245640 1 -- 172.21.15.33:0/948735080 wait complete. 2024-05-15T14:04:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.641+0000 7f632e245640 1 Processor -- start 2024-05-15T14:04:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.641+0000 7f632e245640 1 -- start start 2024-05-15T14:04:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.642+0000 7f632e245640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6328103cc0 0x7f63281992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.642+0000 7f632e245640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63281045e0 0x7f63281997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.642+0000 7f632e245640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f632810d090 0x7f632819db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.642+0000 7f632e245640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6328112cd0 con 0x7f6328103cc0 2024-05-15T14:04:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.642+0000 7f632e245640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6328112b50 con 0x7f63281045e0 2024-05-15T14:04:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.642+0000 7f632e245640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f6328112e50 con 0x7f632810d090 2024-05-15T14:04:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.642+0000 7f63277fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6328103cc0 0x7f63281992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.642+0000 7f6326ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63281045e0 0x7f63281997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.642+0000 7f6326ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63281045e0 0x7f63281997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:53138/0 (socket says 172.21.15.33:53138) 2024-05-15T14:04:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.642+0000 7f63277fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6328103cc0 0x7f63281992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44722/0 (socket says 172.21.15.33:44722) 2024-05-15T14:04:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.642+0000 7f6327fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f632810d090 0x7f632819db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.642+0000 7f6326ffd640 1 -- 172.21.15.33:0/2625634025 learned_addr learned my addr 172.21.15.33:0/2625634025 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:04:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.642+0000 7f6326ffd640 1 -- 172.21.15.33:0/2625634025 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f632810d090 msgr2=0x7f632819db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.642+0000 7f6326ffd640 1 --2- 172.21.15.33:0/2625634025 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f632810d090 0x7f632819db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.642+0000 7f6326ffd640 1 -- 172.21.15.33:0/2625634025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6328103cc0 msgr2=0x7f63281992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.642+0000 7f6326ffd640 1 --2- 172.21.15.33:0/2625634025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6328103cc0 0x7f63281992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.642+0000 7f6326ffd640 1 -- 172.21.15.33:0/2625634025 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f632819e2a0 con 0x7f63281045e0 2024-05-15T14:04:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.642+0000 7f63277fe640 1 --2- 172.21.15.33:0/2625634025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6328103cc0 0x7f63281992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:04:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.642+0000 7f6326ffd640 1 --2- 172.21.15.33:0/2625634025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63281045e0 0x7f63281997f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f631c00d950 tx=0x7f631c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.642+0000 7f6324ff9640 1 -- 172.21.15.33:0/2625634025 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c017070 con 0x7f63281045e0 2024-05-15T14:04:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.643+0000 7f632e245640 1 -- 172.21.15.33:0/2625634025 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f632819e590 con 0x7f63281045e0 2024-05-15T14:04:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.643+0000 7f6324ff9640 1 -- 172.21.15.33:0/2625634025 <== mon.2 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f631c009c30 con 0x7f63281045e0 2024-05-15T14:04:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.643+0000 7f6324ff9640 1 -- 172.21.15.33:0/2625634025 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c00f620 con 0x7f63281045e0 2024-05-15T14:04:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.643+0000 7f632e245640 1 -- 172.21.15.33:0/2625634025 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63281a5e20 con 0x7f63281045e0 2024-05-15T14:04:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.645+0000 7f6324ff9640 1 -- 172.21.15.33:0/2625634025 <== mon.2 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f631c010460 con 0x7f63281045e0 2024-05-15T14:04:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.645+0000 7f6331037740 1 Processor -- start 2024-05-15T14:04:47.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.645+0000 7f6331037740 1 -- start start 2024-05-15T14:04:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.646+0000 7f6331037740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8a67948f0 0x55b8a6794cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.646+0000 7f6331037740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b8a6795210 0x55b8a679f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.646+0000 7f6331037740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8a679fad0 0x55b8a67a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.646+0000 7f6331037740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b8a6707c30 con 0x55b8a679fad0 2024-05-15T14:04:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.646+0000 7f6331037740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b8a66f77d0 con 0x55b8a6795210 2024-05-15T14:04:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.646+0000 7f6331037740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55b8a66fb810 con 0x55b8a67948f0 2024-05-15T14:04:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.646+0000 7f6327fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8a67948f0 0x55b8a6794cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.646+0000 7f6326ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8a679fad0 0x55b8a67a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.646+0000 7f63277fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b8a6795210 0x55b8a679f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.646+0000 7f6327fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8a67948f0 0x55b8a6794cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:60570/0 (socket says 172.21.15.33:60570) 2024-05-15T14:04:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.646+0000 7f6327fff640 1 -- 172.21.15.33:0/3605045560 learned_addr learned my addr 172.21.15.33:0/3605045560 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:04:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.646+0000 7f6324ff9640 1 --2- 172.21.15.33:0/2625634025 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f63100b2950 0x7f63100b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:47.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.646+0000 7f6327fff640 1 -- 172.21.15.33:0/3605045560 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b8a6795210 msgr2=0x55b8a679f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.646+0000 7f6327fff640 1 --2- 172.21.15.33:0/3605045560 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b8a6795210 0x55b8a679f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.646+0000 7f6327fff640 1 -- 172.21.15.33:0/3605045560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8a679fad0 msgr2=0x55b8a67a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.646+0000 7f6327fff640 1 --2- 172.21.15.33:0/3605045560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8a679fad0 0x55b8a67a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.646+0000 7f6327fff640 1 -- 172.21.15.33:0/3605045560 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8a640c210 con 0x55b8a67948f0 2024-05-15T14:04:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.646+0000 7f6324ff9640 1 -- 172.21.15.33:0/2625634025 <== mon.2 v2:172.21.15.52:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f631c0e20f0 con 0x7f63281045e0 2024-05-15T14:04:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.646+0000 7f6327fff640 1 --2- 172.21.15.33:0/2625634025 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f63100b2950 0x7f63100b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.646+0000 7f6327fff640 1 --2- 172.21.15.33:0/3605045560 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8a67948f0 0x55b8a6794cd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6320009f30 tx=0x7f63200118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f21225ed42adda2 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.647+0000 7f630dffb640 1 -- 172.21.15.33:0/3605045560 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6320003bb0 con 0x55b8a67948f0 2024-05-15T14:04:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.647+0000 7f630dffb640 1 -- 172.21.15.33:0/3605045560 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63200045c0 con 0x55b8a67948f0 2024-05-15T14:04:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.647+0000 7f630dffb640 1 -- 172.21.15.33:0/3605045560 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6320019400 con 0x55b8a67948f0 2024-05-15T14:04:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.647+0000 7f6327fff640 1 --2- 172.21.15.33:0/2625634025 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f63100b2950 0x7f63100b4e10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f632001d000 tx=0x7f6320019750 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.647+0000 7f6331037740 1 -- 172.21.15.33:0/3605045560 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8a67948f0 msgr2=0x55b8a6794cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.647+0000 7f6331037740 1 --2- 172.21.15.33:0/3605045560 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8a67948f0 0x55b8a6794cd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6320009f30 tx=0x7f63200118d0 comp rx=0 tx=0).stop 2024-05-15T14:04:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.647+0000 7f6331037740 1 -- 172.21.15.33:0/3605045560 shutdown_connections 2024-05-15T14:04:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.647+0000 7f6331037740 1 --2- 172.21.15.33:0/3605045560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8a679fad0 0x55b8a67a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.647+0000 7f6331037740 1 --2- 172.21.15.33:0/3605045560 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b8a6795210 0x55b8a679f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.647+0000 7f6331037740 1 --2- 172.21.15.33:0/3605045560 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8a67948f0 0x55b8a6794cd0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.647+0000 7f6331037740 1 -- 172.21.15.33:0/3605045560 >> 172.21.15.33:0/3605045560 conn(0x55b8a6790e00 msgr2=0x55b8a67911e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.647+0000 7f6331037740 1 -- 172.21.15.33:0/3605045560 shutdown_connections 2024-05-15T14:04:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.647+0000 7f6331037740 1 -- 172.21.15.33:0/3605045560 wait complete. 2024-05-15T14:04:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.648+0000 7f6331037740 1 -- start start 2024-05-15T14:04:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.648+0000 7f6331037740 1 Processor -- start 2024-05-15T14:04:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.648+0000 7f6331037740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b8a67948f0 0x55b8a679f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.648+0000 7f6331037740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8a6795210 0x55b8a679c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.648+0000 7f6331037740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8a679fad0 0x55b8a679cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.648+0000 7f6331037740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b8a63dd790 con 0x55b8a6795210 2024-05-15T14:04:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.648+0000 7f6331037740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b8a66b3890 con 0x55b8a67948f0 2024-05-15T14:04:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.648+0000 7f6331037740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55b8a67a27a0 con 0x55b8a679fad0 2024-05-15T14:04:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.648+0000 7f6331037740 10 client.0 _ll_register_callbacks cb 0x55b8a66b6d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:04:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.648+0000 7f6326ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8a6795210 0x55b8a679c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.648+0000 7f6326ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8a6795210 0x55b8a679c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44736/0 (socket says 172.21.15.33:44736) 2024-05-15T14:04:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.648+0000 7f6326ffd640 1 -- 172.21.15.33:0/816824069 learned_addr learned my addr 172.21.15.33:0/816824069 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:04:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.648+0000 7f6327fff640 1 --2- 172.21.15.33:0/816824069 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8a679fad0 0x55b8a679cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.649+0000 7f63277fe640 1 --2- 172.21.15.33:0/816824069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b8a67948f0 0x55b8a679f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.649+0000 7f6326ffd640 1 -- 172.21.15.33:0/816824069 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8a679fad0 msgr2=0x55b8a679cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.649+0000 7f6326ffd640 1 --2- 172.21.15.33:0/816824069 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8a679fad0 0x55b8a679cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.649+0000 7f6326ffd640 1 -- 172.21.15.33:0/816824069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b8a67948f0 msgr2=0x55b8a679f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.649+0000 7f6326ffd640 1 --2- 172.21.15.33:0/816824069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b8a67948f0 0x55b8a679f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.649+0000 7f6326ffd640 1 -- 172.21.15.33:0/816824069 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8a64d8620 con 0x55b8a6795210 2024-05-15T14:04:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.649+0000 7f63277fe640 1 --2- 172.21.15.33:0/816824069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b8a67948f0 0x55b8a679f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:04:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.649+0000 7f6326ffd640 1 --2- 172.21.15.33:0/816824069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8a6795210 0x55b8a679c610 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f631c0df950 tx=0x7f631c0e5800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29d3ad6f692c1bc server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.649+0000 7f6306ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:04:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.649+0000 7f6306ffd640 1 -- 172.21.15.33:0/816824069 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c020070 con 0x55b8a6795210 2024-05-15T14:04:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.649+0000 7f6306ffd640 1 -- 172.21.15.33:0/816824069 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f631c0a8790 con 0x55b8a6795210 2024-05-15T14:04:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.649+0000 7f6331037740 1 -- 172.21.15.33:0/816824069 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63200045c0 con 0x55b8a6795210 2024-05-15T14:04:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.649+0000 7f6306ffd640 1 -- 172.21.15.33:0/816824069 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f631c0a8e80 con 0x55b8a6795210 2024-05-15T14:04:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.649+0000 7f6331037740 10 client.15885 fetch_fsmap learned FSMap version 27 2024-05-15T14:04:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.649+0000 7f6331037740 1 -- 172.21.15.33:0/816824069 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b8a67ae0b0 con 0x55b8a6795210 2024-05-15T14:04:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.649+0000 7f6306ffd640 1 -- 172.21.15.33:0/816824069 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63200045c0 con 0x55b8a6795210 2024-05-15T14:04:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.650+0000 7f6306ffd640 1 -- 172.21.15.33:0/816824069 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f631c010750 con 0x55b8a6795210 2024-05-15T14:04:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.650+0000 7f6331037740 10 client.15885 fetch_fsmap finished waiting for FSMap version 27 2024-05-15T14:04:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.650+0000 7f6331037740 10 client.15885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:04:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.650+0000 7f6331037740 20 client.15885 populate_metadata read hostname 'smithi033' 2024-05-15T14:04:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.650+0000 7f6331037740 1 --2- 172.21.15.33:0/816824069 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55b8a67af570 0x55b8a67b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.650+0000 7f6331037740 4 client.15885 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:04:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.650+0000 7f6331037740 1 -- 172.21.15.33:0/816824069 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8a6709b80 con 0x55b8a67af570 2024-05-15T14:04:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.650+0000 7f63277fe640 1 --2- 172.21.15.33:0/816824069 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55b8a67af570 0x55b8a67b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.650+0000 7f6306ffd640 10 client.15885 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:04:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.650+0000 7f63277fe640 1 --2- 172.21.15.33:0/816824069 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55b8a67af570 0x55b8a67b19a0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b9204af509c9e2b server_cookie=7463afc4a7c41bee in_seq=0 out_seq=0 2024-05-15T14:04:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.651+0000 7f6306ffd640 1 -- 172.21.15.33:0/816824069 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b8a6709b80 con 0x55b8a67af570 2024-05-15T14:04:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.651+0000 7f6306ffd640 10 client.15885 handle_command_reply: tid=0 2024-05-15T14:04:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.651+0000 7f6331037740 1 client.15885 shutdown 2024-05-15T14:04:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.651+0000 7f6331037740 1 -- 172.21.15.33:0/816824069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8a6795210 msgr2=0x55b8a679c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.651+0000 7f6331037740 1 --2- 172.21.15.33:0/816824069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8a6795210 0x55b8a679c610 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f631c0df950 tx=0x7f631c0e5800 comp rx=0 tx=0).stop 2024-05-15T14:04:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.651+0000 7f6331037740 1 -- 172.21.15.33:0/816824069 shutdown_connections 2024-05-15T14:04:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.651+0000 7f6331037740 1 --2- 172.21.15.33:0/816824069 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55b8a67af570 0x55b8a67b19a0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.651+0000 7f6331037740 1 --2- 172.21.15.33:0/816824069 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8a679fad0 0x55b8a679cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.651+0000 7f6331037740 1 --2- 172.21.15.33:0/816824069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8a6795210 0x55b8a679c610 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.651+0000 7f6306ffd640 0 client.15885 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:04:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.651+0000 7f6331037740 1 --2- 172.21.15.33:0/816824069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b8a67948f0 0x55b8a679f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.651+0000 7f6331037740 1 -- 172.21.15.33:0/816824069 >> 172.21.15.33:0/816824069 conn(0x55b8a6790e00 msgr2=0x55b8a6790100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:47.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.651+0000 7f6331037740 1 -- 172.21.15.33:0/816824069 shutdown_connections 2024-05-15T14:04:47.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.651+0000 7f6331037740 1 -- 172.21.15.33:0/816824069 wait complete. 2024-05-15T14:04:47.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.651+0000 7f6331037740 20 client.15885 trim_cache size 0 max 16384 2024-05-15T14:04:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.667+0000 7f6331037740 1 Processor -- start 2024-05-15T14:04:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.667+0000 7f6331037740 1 -- start start 2024-05-15T14:04:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.667+0000 7f6331037740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8a67948f0 0x55b8a6797dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.667+0000 7f6331037740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b8a6795210 0x55b8a678bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.667+0000 7f6331037740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8a679fad0 0x55b8a6796a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.667+0000 7f6331037740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f63281b1010 con 0x55b8a67948f0 2024-05-15T14:04:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.667+0000 7f6331037740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f63281ae880 con 0x55b8a6795210 2024-05-15T14:04:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.667+0000 7f6331037740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f63281ac6d0 con 0x55b8a679fad0 2024-05-15T14:04:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.668+0000 7f63277fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8a67948f0 0x55b8a6797dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.668+0000 7f63277fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8a67948f0 0x55b8a6797dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44750/0 (socket says 172.21.15.33:44750) 2024-05-15T14:04:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.668+0000 7f63277fe640 1 -- 172.21.15.33:0/2266403660 learned_addr learned my addr 172.21.15.33:0/2266403660 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:04:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.668+0000 7f6326ffd640 1 --2- 172.21.15.33:0/2266403660 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b8a6795210 0x55b8a678bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.668+0000 7f6327fff640 1 --2- 172.21.15.33:0/2266403660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8a679fad0 0x55b8a6796a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.668+0000 7f63277fe640 1 -- 172.21.15.33:0/2266403660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8a679fad0 msgr2=0x55b8a6796a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.668+0000 7f63277fe640 1 --2- 172.21.15.33:0/2266403660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8a679fad0 0x55b8a6796a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.668+0000 7f63277fe640 1 -- 172.21.15.33:0/2266403660 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b8a6795210 msgr2=0x55b8a678bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.668+0000 7f63277fe640 1 --2- 172.21.15.33:0/2266403660 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b8a6795210 0x55b8a678bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.668+0000 7f63277fe640 1 -- 172.21.15.33:0/2266403660 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8a679b5c0 con 0x55b8a67948f0 2024-05-15T14:04:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.668+0000 7f6327fff640 1 --2- 172.21.15.33:0/2266403660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8a679fad0 0x55b8a6796a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:04:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.668+0000 7f63277fe640 1 --2- 172.21.15.33:0/2266403660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8a67948f0 0x55b8a6797dc0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6318031440 tx=0x7f6318005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc985660f84c7dc server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.668+0000 7f630d7fa640 1 -- 172.21.15.33:0/2266403660 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6318035040 con 0x55b8a67948f0 2024-05-15T14:04:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.668+0000 7f630d7fa640 1 -- 172.21.15.33:0/2266403660 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63180054f0 con 0x55b8a67948f0 2024-05-15T14:04:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.668+0000 7f630d7fa640 1 -- 172.21.15.33:0/2266403660 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6318004b60 con 0x55b8a67948f0 2024-05-15T14:04:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.669+0000 7f6331037740 1 -- 172.21.15.33:0/2266403660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8a67948f0 msgr2=0x55b8a6797dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.669+0000 7f6331037740 1 --2- 172.21.15.33:0/2266403660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8a67948f0 0x55b8a6797dc0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6318031440 tx=0x7f6318005b50 comp rx=0 tx=0).stop 2024-05-15T14:04:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.669+0000 7f6331037740 1 -- 172.21.15.33:0/2266403660 shutdown_connections 2024-05-15T14:04:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.669+0000 7f6331037740 1 --2- 172.21.15.33:0/2266403660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8a679fad0 0x55b8a6796a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.669+0000 7f6331037740 1 --2- 172.21.15.33:0/2266403660 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b8a6795210 0x55b8a678bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.669+0000 7f6331037740 1 --2- 172.21.15.33:0/2266403660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8a67948f0 0x55b8a6797dc0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.669+0000 7f6331037740 1 -- 172.21.15.33:0/2266403660 >> 172.21.15.33:0/2266403660 conn(0x55b8a6790e00 msgr2=0x55b8a67962f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.669+0000 7f6331037740 1 -- 172.21.15.33:0/2266403660 shutdown_connections 2024-05-15T14:04:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.669+0000 7f6331037740 1 -- 172.21.15.33:0/2266403660 wait complete. 2024-05-15T14:04:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.669+0000 7f6331037740 1 -- start start 2024-05-15T14:04:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.670+0000 7f6331037740 1 Processor -- start 2024-05-15T14:04:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.670+0000 7f6331037740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8a67948f0 0x55b8a67995a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.670+0000 7f6331037740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8a6795210 0x55b8a6799ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.670+0000 7f6331037740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b8a679fad0 0x55b8a679a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.670+0000 7f6331037740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f63281acd40 con 0x55b8a6795210 2024-05-15T14:04:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.670+0000 7f6331037740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f63281ae1b0 con 0x55b8a679fad0 2024-05-15T14:04:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.670+0000 7f6331037740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f63281ad410 con 0x55b8a67948f0 2024-05-15T14:04:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.670+0000 7f6331037740 10 client.0 _ll_register_callbacks cb 0x7f631c0e5e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:04:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.670+0000 7f6326ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8a6795210 0x55b8a6799ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.670+0000 7f6326ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8a6795210 0x55b8a6799ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44764/0 (socket says 172.21.15.33:44764) 2024-05-15T14:04:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.670+0000 7f6326ffd640 1 -- 172.21.15.33:0/916211010 learned_addr learned my addr 172.21.15.33:0/916211010 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:04:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.670+0000 7f63277fe640 1 --2- 172.21.15.33:0/916211010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8a67948f0 0x55b8a67995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.670+0000 7f6327fff640 1 --2- 172.21.15.33:0/916211010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b8a679fad0 0x55b8a679a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.670+0000 7f6326ffd640 1 -- 172.21.15.33:0/916211010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8a67948f0 msgr2=0x55b8a67995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.670+0000 7f6326ffd640 1 --2- 172.21.15.33:0/916211010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8a67948f0 0x55b8a67995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.670+0000 7f6326ffd640 1 -- 172.21.15.33:0/916211010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b8a679fad0 msgr2=0x55b8a679a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:04:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.670+0000 7f6326ffd640 1 --2- 172.21.15.33:0/916211010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b8a679fad0 0x55b8a679a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.670+0000 7f6326ffd640 1 -- 172.21.15.33:0/916211010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8a6819000 con 0x55b8a6795210 2024-05-15T14:04:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.670+0000 7f6326ffd640 1 --2- 172.21.15.33:0/916211010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8a6795210 0x55b8a6799ae0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f631c010cf0 tx=0x7f631c0a8a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f93290e568c72497 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:04:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.670+0000 7f6306ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:04:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.671+0000 7f6306ffd640 1 -- 172.21.15.33:0/916211010 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c020040 con 0x55b8a6795210 2024-05-15T14:04:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.671+0000 7f6306ffd640 1 -- 172.21.15.33:0/916211010 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f631c0a50b0 con 0x55b8a6795210 2024-05-15T14:04:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.671+0000 7f6331037740 1 -- 172.21.15.33:0/916211010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63180054f0 con 0x55b8a6795210 2024-05-15T14:04:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.671+0000 7f6306ffd640 1 -- 172.21.15.33:0/916211010 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f631c0a8ca0 con 0x55b8a6795210 2024-05-15T14:04:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.671+0000 7f6331037740 10 client.15891 fetch_fsmap learned FSMap version 27 2024-05-15T14:04:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.671+0000 7f6331037740 1 -- 172.21.15.33:0/916211010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b8a681a2b0 con 0x55b8a6795210 2024-05-15T14:04:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.671+0000 7f6306ffd640 1 -- 172.21.15.33:0/916211010 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63180054f0 con 0x55b8a6795210 2024-05-15T14:04:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.671+0000 7f6306ffd640 1 -- 172.21.15.33:0/916211010 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f631c0ae9c0 con 0x55b8a6795210 2024-05-15T14:04:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.671+0000 7f6331037740 10 client.15891 fetch_fsmap finished waiting for FSMap version 27 2024-05-15T14:04:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.671+0000 7f6331037740 10 client.15891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:04:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.671+0000 7f6331037740 20 client.15891 populate_metadata read hostname 'smithi033' 2024-05-15T14:04:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.671+0000 7f6331037740 1 --2- 172.21.15.33:0/916211010 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55b8a681aa30 0x55b8a681ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:04:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.671+0000 7f6331037740 4 client.15891 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:04:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.671+0000 7f6331037740 1 -- 172.21.15.33:0/916211010 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b8a6703360 con 0x55b8a681aa30 2024-05-15T14:04:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.671+0000 7f63277fe640 1 --2- 172.21.15.33:0/916211010 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55b8a681aa30 0x55b8a681ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:04:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.672+0000 7f63277fe640 1 --2- 172.21.15.33:0/916211010 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55b8a681aa30 0x55b8a681ce50 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5860950ef85db64 server_cookie=41cc7b332827ffd5 in_seq=0 out_seq=0 2024-05-15T14:04:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.672+0000 7f6306ffd640 10 client.15891 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:04:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.728+0000 7f6306ffd640 1 -- 172.21.15.33:0/916211010 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b8a6703360 con 0x55b8a681aa30 2024-05-15T14:04:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.729+0000 7f6306ffd640 10 client.15891 handle_command_reply: tid=0 2024-05-15T14:04:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.729+0000 7f6331037740 1 client.15891 shutdown 2024-05-15T14:04:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.729+0000 7f6331037740 1 -- 172.21.15.33:0/916211010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8a6795210 msgr2=0x55b8a6799ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.729+0000 7f6331037740 1 --2- 172.21.15.33:0/916211010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8a6795210 0x55b8a6799ae0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f631c010cf0 tx=0x7f631c0a8a40 comp rx=0 tx=0).stop 2024-05-15T14:04:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.729+0000 7f6331037740 1 -- 172.21.15.33:0/916211010 shutdown_connections 2024-05-15T14:04:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.729+0000 7f6331037740 1 --2- 172.21.15.33:0/916211010 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55b8a681aa30 0x55b8a681ce50 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.729+0000 7f6331037740 1 --2- 172.21.15.33:0/916211010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b8a679fad0 0x55b8a679a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.729+0000 7f6306ffd640 0 client.15891 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:04:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.729+0000 7f6331037740 1 --2- 172.21.15.33:0/916211010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8a6795210 0x55b8a6799ae0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.729+0000 7f6331037740 1 --2- 172.21.15.33:0/916211010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b8a67948f0 0x55b8a67995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.729+0000 7f6331037740 1 -- 172.21.15.33:0/916211010 >> 172.21.15.33:0/916211010 conn(0x55b8a6790e00 msgr2=0x55b8a6796b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.729+0000 7f6331037740 1 -- 172.21.15.33:0/916211010 shutdown_connections 2024-05-15T14:04:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.729+0000 7f6331037740 1 -- 172.21.15.33:0/916211010 wait complete. 2024-05-15T14:04:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.729+0000 7f6331037740 20 client.15891 trim_cache size 0 max 16384 2024-05-15T14:04:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:04:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:04:47.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:04:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.730+0000 7f632e245640 1 -- 172.21.15.33:0/2625634025 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f63100b2950 msgr2=0x7f63100b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.730+0000 7f632e245640 1 --2- 172.21.15.33:0/2625634025 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f63100b2950 0x7f63100b4e10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f632001d000 tx=0x7f6320019750 comp rx=0 tx=0).stop 2024-05-15T14:04:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.730+0000 7f632e245640 1 -- 172.21.15.33:0/2625634025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63281045e0 msgr2=0x7f63281997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:04:47.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.730+0000 7f632e245640 1 --2- 172.21.15.33:0/2625634025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63281045e0 0x7f63281997f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f631c00d950 tx=0x7f631c00de10 comp rx=0 tx=0).stop 2024-05-15T14:04:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.730+0000 7f632e245640 1 -- 172.21.15.33:0/2625634025 shutdown_connections 2024-05-15T14:04:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.730+0000 7f632e245640 1 --2- 172.21.15.33:0/2625634025 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f63100b2950 0x7f63100b4e10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.730+0000 7f632e245640 1 --2- 172.21.15.33:0/2625634025 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f632810d090 0x7f632819db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.730+0000 7f632e245640 1 --2- 172.21.15.33:0/2625634025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f63281045e0 0x7f63281997f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.730+0000 7f632e245640 1 --2- 172.21.15.33:0/2625634025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6328103cc0 0x7f63281992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:04:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.730+0000 7f632e245640 1 -- 172.21.15.33:0/2625634025 >> 172.21.15.33:0/2625634025 conn(0x7f6328078a70 msgr2=0x7f63280ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:04:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.730+0000 7f632e245640 1 -- 172.21.15.33:0/2625634025 shutdown_connections 2024-05-15T14:04:47.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:04:47.730+0000 7f632e245640 1 -- 172.21.15.33:0/2625634025 wait complete. 2024-05-15T14:04:48.054 INFO:tasks.workunit.client.0.smithi033.stdout: 1 223474 2.33 MB/sec execute 23 sec latency 37.570 ms 2024-05-15T14:04:48.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:48 smithi052 ceph-mon[36190]: pgmap v137: 97 pgs: 97 active+clean; 71 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 150 op/s 2024-05-15T14:04:48.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:48 smithi081 ceph-mon[36434]: pgmap v137: 97 pgs: 97 active+clean; 71 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 150 op/s 2024-05-15T14:04:48.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:48 smithi033 ceph-mon[31650]: pgmap v137: 97 pgs: 97 active+clean; 71 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 150 op/s 2024-05-15T14:04:49.054 INFO:tasks.workunit.client.0.smithi033.stdout: 1 225071 2.52 MB/sec execute 24 sec latency 11.654 ms 2024-05-15T14:04:50.054 INFO:tasks.workunit.client.0.smithi033.stdout: 1 227478 2.98 MB/sec execute 25 sec latency 13.178 ms 2024-05-15T14:04:50.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:50 smithi052 ceph-mon[36190]: pgmap v138: 97 pgs: 97 active+clean; 71 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 122 op/s 2024-05-15T14:04:50.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:50 smithi081 ceph-mon[36434]: pgmap v138: 97 pgs: 97 active+clean; 71 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 122 op/s 2024-05-15T14:04:50.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:50 smithi033 ceph-mon[31650]: pgmap v138: 97 pgs: 97 active+clean; 71 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 122 op/s 2024-05-15T14:04:51.054 INFO:tasks.workunit.client.0.smithi033.stdout: 1 229850 3.29 MB/sec execute 26 sec latency 14.518 ms 2024-05-15T14:04:52.054 INFO:tasks.workunit.client.0.smithi033.stdout: 1 231394 3.48 MB/sec execute 27 sec latency 19.071 ms 2024-05-15T14:04:52.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:52 smithi052 ceph-mon[36190]: pgmap v139: 97 pgs: 97 active+clean; 71 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 122 op/s 2024-05-15T14:04:52.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:52 smithi081 ceph-mon[36434]: pgmap v139: 97 pgs: 97 active+clean; 71 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 122 op/s 2024-05-15T14:04:52.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:52 smithi033 ceph-mon[31650]: pgmap v139: 97 pgs: 97 active+clean; 71 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 122 op/s 2024-05-15T14:04:53.054 INFO:tasks.workunit.client.0.smithi033.stdout: 1 234351 4.02 MB/sec execute 28 sec latency 13.810 ms 2024-05-15T14:04:54.054 INFO:tasks.workunit.client.0.smithi033.stdout: 1 235851 4.07 MB/sec execute 29 sec latency 13.994 ms 2024-05-15T14:04:54.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:54 smithi052 ceph-mon[36190]: pgmap v140: 97 pgs: 97 active+clean; 64 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 260 op/s 2024-05-15T14:04:54.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:54 smithi081 ceph-mon[36434]: pgmap v140: 97 pgs: 97 active+clean; 64 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 260 op/s 2024-05-15T14:04:54.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:54 smithi033 ceph-mon[31650]: pgmap v140: 97 pgs: 97 active+clean; 64 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 260 op/s 2024-05-15T14:04:55.054 INFO:tasks.workunit.client.0.smithi033.stdout: 1 238146 4.40 MB/sec execute 30 sec latency 12.588 ms 2024-05-15T14:04:56.054 INFO:tasks.workunit.client.0.smithi033.stdout: 1 240610 4.66 MB/sec execute 31 sec latency 15.059 ms 2024-05-15T14:04:56.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:56 smithi052 ceph-mon[36190]: pgmap v141: 97 pgs: 97 active+clean; 64 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 234 op/s 2024-05-15T14:04:56.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:56 smithi081 ceph-mon[36434]: pgmap v141: 97 pgs: 97 active+clean; 64 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 234 op/s 2024-05-15T14:04:56.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:56 smithi033 ceph-mon[31650]: pgmap v141: 97 pgs: 97 active+clean; 64 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.9 MiB/s wr, 234 op/s 2024-05-15T14:04:57.054 INFO:tasks.workunit.client.0.smithi033.stdout: 1 242152 4.74 MB/sec execute 32 sec latency 18.380 ms 2024-05-15T14:04:58.054 INFO:tasks.workunit.client.0.smithi033.stdout: 1 245000 5.15 MB/sec execute 33 sec latency 12.094 ms 2024-05-15T14:04:58.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:04:58 smithi052 ceph-mon[36190]: pgmap v142: 97 pgs: 97 active+clean; 70 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 332 op/s 2024-05-15T14:04:58.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:04:58 smithi081 ceph-mon[36434]: pgmap v142: 97 pgs: 97 active+clean; 70 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 332 op/s 2024-05-15T14:04:58.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:04:58 smithi033 ceph-mon[31650]: pgmap v142: 97 pgs: 97 active+clean; 70 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 332 op/s 2024-05-15T14:04:59.054 INFO:tasks.workunit.client.0.smithi033.stdout: 1 246647 5.16 MB/sec execute 34 sec latency 22.828 ms 2024-05-15T14:05:00.055 INFO:tasks.workunit.client.0.smithi033.stdout: 1 248838 5.41 MB/sec execute 35 sec latency 10.920 ms 2024-05-15T14:05:00.843 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:00 smithi052 ceph-mon[36190]: pgmap v143: 97 pgs: 97 active+clean; 70 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 236 op/s 2024-05-15T14:05:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.826+0000 7f276e2f8640 1 -- 172.21.15.33:0/1654933567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2768103cc0 msgr2=0x7f27681040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.826+0000 7f276e2f8640 1 --2- 172.21.15.33:0/1654933567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2768103cc0 0x7f27681040a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2758009900 tx=0x7f275802a480 comp rx=0 tx=0).stop 2024-05-15T14:05:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.826+0000 7f276e2f8640 1 -- 172.21.15.33:0/1654933567 shutdown_connections 2024-05-15T14:05:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.826+0000 7f276e2f8640 1 --2- 172.21.15.33:0/1654933567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f276810d090 0x7f276810f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.826+0000 7f276e2f8640 1 --2- 172.21.15.33:0/1654933567 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27681045e0 0x7f276810cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.826+0000 7f276e2f8640 1 --2- 172.21.15.33:0/1654933567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2768103cc0 0x7f27681040a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.826+0000 7f276e2f8640 1 -- 172.21.15.33:0/1654933567 >> 172.21.15.33:0/1654933567 conn(0x7f2768078a70 msgr2=0x7f2768078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.826+0000 7f276e2f8640 1 -- 172.21.15.33:0/1654933567 shutdown_connections 2024-05-15T14:05:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.826+0000 7f276e2f8640 1 -- 172.21.15.33:0/1654933567 wait complete. 2024-05-15T14:05:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.826+0000 7f276e2f8640 1 Processor -- start 2024-05-15T14:05:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.826+0000 7f276e2f8640 1 -- start start 2024-05-15T14:05:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.827+0000 7f276e2f8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2768103cc0 0x7f2768199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.827+0000 7f276e2f8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27681045e0 0x7f27681997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.827+0000 7f276e2f8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f276810d090 0x7f276819db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.827+0000 7f276e2f8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2768112cd0 con 0x7f276810d090 2024-05-15T14:05:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.827+0000 7f276e2f8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2768112b50 con 0x7f2768103cc0 2024-05-15T14:05:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.827+0000 7f276e2f8640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2768112e50 con 0x7f27681045e0 2024-05-15T14:05:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.827+0000 7f276c895640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f276810d090 0x7f276819db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.827+0000 7f27677fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27681045e0 0x7f27681997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.827+0000 7f276c895640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f276810d090 0x7f276819db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48340/0 (socket says 172.21.15.33:48340) 2024-05-15T14:05:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.827+0000 7f276c895640 1 -- 172.21.15.33:0/3421370875 learned_addr learned my addr 172.21.15.33:0/3421370875 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.827+0000 7f2767fff640 1 --2- 172.21.15.33:0/3421370875 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2768103cc0 0x7f2768199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.827+0000 7f276c895640 1 -- 172.21.15.33:0/3421370875 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27681045e0 msgr2=0x7f27681997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.827+0000 7f276c895640 1 --2- 172.21.15.33:0/3421370875 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27681045e0 0x7f27681997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.827+0000 7f276c895640 1 -- 172.21.15.33:0/3421370875 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2768103cc0 msgr2=0x7f2768199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.827+0000 7f276c895640 1 --2- 172.21.15.33:0/3421370875 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2768103cc0 0x7f2768199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.827+0000 7f276c895640 1 -- 172.21.15.33:0/3421370875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f276819e230 con 0x7f276810d090 2024-05-15T14:05:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.827+0000 7f27677fe640 1 --2- 172.21.15.33:0/3421370875 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27681045e0 0x7f27681997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:05:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.828+0000 7f276c895640 1 --2- 172.21.15.33:0/3421370875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f276810d090 0x7f276819db00 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f276000eb40 tx=0x7f276000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.828+0000 7f27657fa640 1 -- 172.21.15.33:0/3421370875 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2760004b20 con 0x7f276810d090 2024-05-15T14:05:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.828+0000 7f276e2f8640 1 -- 172.21.15.33:0/3421370875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f276819e520 con 0x7f276810d090 2024-05-15T14:05:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.828+0000 7f27657fa640 1 -- 172.21.15.33:0/3421370875 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2760004cc0 con 0x7f276810d090 2024-05-15T14:05:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.828+0000 7f27657fa640 1 -- 172.21.15.33:0/3421370875 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2760010720 con 0x7f276810d090 2024-05-15T14:05:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.828+0000 7f276e2f8640 1 -- 172.21.15.33:0/3421370875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27681a5db0 con 0x7f276810d090 2024-05-15T14:05:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.830+0000 7f277100f740 1 Processor -- start 2024-05-15T14:05:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.830+0000 7f27657fa640 1 -- 172.21.15.33:0/3421370875 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2760005560 con 0x7f276810d090 2024-05-15T14:05:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.830+0000 7f277100f740 1 -- start start 2024-05-15T14:05:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.831+0000 7f277100f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458ba4e8f0 0x56458ba4ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.831+0000 7f277100f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56458ba4f210 0x56458ba59590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.831+0000 7f277100f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56458ba59ad0 0x56458ba5bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.831+0000 7f277100f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56458b9c1c30 con 0x56458ba4e8f0 2024-05-15T14:05:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.831+0000 7f277100f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56458b9b17d0 con 0x56458ba59ad0 2024-05-15T14:05:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.831+0000 7f277100f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56458b9b5810 con 0x56458ba4f210 2024-05-15T14:05:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.831+0000 7f27677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458ba4e8f0 0x56458ba4ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.831+0000 7f27677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458ba4e8f0 0x56458ba4ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48342/0 (socket says 172.21.15.33:48342) 2024-05-15T14:05:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.831+0000 7f27677fe640 1 -- 172.21.15.33:0/98734433 learned_addr learned my addr 172.21.15.33:0/98734433 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.831+0000 7f2767fff640 1 --2- 172.21.15.33:0/98734433 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56458ba4f210 0x56458ba59590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.831+0000 7f27677fe640 1 --2- 172.21.15.33:0/98734433 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56458ba59ad0 0x56458ba5bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.831+0000 7f27677fe640 1 -- 172.21.15.33:0/98734433 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56458ba4f210 msgr2=0x56458ba59590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.831+0000 7f27677fe640 1 --2- 172.21.15.33:0/98734433 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56458ba4f210 0x56458ba59590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.831+0000 7f27677fe640 1 -- 172.21.15.33:0/98734433 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56458ba59ad0 msgr2=0x56458ba5bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.831+0000 7f27677fe640 1 --2- 172.21.15.33:0/98734433 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56458ba59ad0 0x56458ba5bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.831+0000 7f27677fe640 1 -- 172.21.15.33:0/98734433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56458b6c6210 con 0x56458ba4e8f0 2024-05-15T14:05:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.831+0000 7f27657fa640 1 --2- 172.21.15.33:0/3421370875 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f27540b2950 0x7f27540b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.831+0000 7f27677fe640 1 --2- 172.21.15.33:0/98734433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458ba4e8f0 0x56458ba4ecd0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2750009710 tx=0x7f2750009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d07b75db71235bb server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.831+0000 7f2767fff640 1 --2- 172.21.15.33:0/3421370875 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f27540b2950 0x7f27540b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.831+0000 7f275e7fc640 1 -- 172.21.15.33:0/98734433 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2750004030 con 0x56458ba4e8f0 2024-05-15T14:05:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.831+0000 7f27657fa640 1 -- 172.21.15.33:0/3421370875 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f27600e2ce0 con 0x7f276810d090 2024-05-15T14:05:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.831+0000 7f275e7fc640 1 -- 172.21.15.33:0/98734433 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f275000f980 con 0x56458ba4e8f0 2024-05-15T14:05:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.831+0000 7f275e7fc640 1 -- 172.21.15.33:0/98734433 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2750003a10 con 0x56458ba4e8f0 2024-05-15T14:05:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.832+0000 7f2767fff640 1 --2- 172.21.15.33:0/3421370875 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f27540b2950 0x7f27540b4e10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2758031440 tx=0x7f2758005b90 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.832+0000 7f277100f740 1 -- 172.21.15.33:0/98734433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458ba4e8f0 msgr2=0x56458ba4ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.832+0000 7f277100f740 1 --2- 172.21.15.33:0/98734433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458ba4e8f0 0x56458ba4ecd0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2750009710 tx=0x7f2750009c00 comp rx=0 tx=0).stop 2024-05-15T14:05:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.832+0000 7f277100f740 1 -- 172.21.15.33:0/98734433 shutdown_connections 2024-05-15T14:05:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.832+0000 7f277100f740 1 --2- 172.21.15.33:0/98734433 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56458ba59ad0 0x56458ba5bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.832+0000 7f277100f740 1 --2- 172.21.15.33:0/98734433 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56458ba4f210 0x56458ba59590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.832+0000 7f277100f740 1 --2- 172.21.15.33:0/98734433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458ba4e8f0 0x56458ba4ecd0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.832+0000 7f277100f740 1 -- 172.21.15.33:0/98734433 >> 172.21.15.33:0/98734433 conn(0x56458ba4ae00 msgr2=0x56458ba4b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.832+0000 7f277100f740 1 -- 172.21.15.33:0/98734433 shutdown_connections 2024-05-15T14:05:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.832+0000 7f277100f740 1 -- 172.21.15.33:0/98734433 wait complete. 2024-05-15T14:05:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.833+0000 7f277100f740 1 -- start start 2024-05-15T14:05:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.833+0000 7f277100f740 1 Processor -- start 2024-05-15T14:05:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.833+0000 7f277100f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56458ba4e8f0 0x56458ba59460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.833+0000 7f277100f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56458ba4f210 0x56458ba56750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.833+0000 7f277100f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458ba59ad0 0x56458ba56cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.833+0000 7f277100f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56458b697790 con 0x56458ba59ad0 2024-05-15T14:05:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.833+0000 7f277100f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56458b96d890 con 0x56458ba4e8f0 2024-05-15T14:05:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.833+0000 7f277100f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56458ba5c7a0 con 0x56458ba4f210 2024-05-15T14:05:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.833+0000 7f277100f740 10 client.0 _ll_register_callbacks cb 0x56458b970d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:05:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.833+0000 7f276c895640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458ba59ad0 0x56458ba56cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.833+0000 7f27677fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56458ba4f210 0x56458ba56750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.833+0000 7f2767fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56458ba4e8f0 0x56458ba59460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.833+0000 7f276c895640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458ba59ad0 0x56458ba56cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48350/0 (socket says 172.21.15.33:48350) 2024-05-15T14:05:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.833+0000 7f276c895640 1 -- 172.21.15.33:0/1823246408 learned_addr learned my addr 172.21.15.33:0/1823246408 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.833+0000 7f276c895640 1 -- 172.21.15.33:0/1823246408 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56458ba4f210 msgr2=0x56458ba56750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.833+0000 7f276c895640 1 --2- 172.21.15.33:0/1823246408 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56458ba4f210 0x56458ba56750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.833+0000 7f276c895640 1 -- 172.21.15.33:0/1823246408 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56458ba4e8f0 msgr2=0x56458ba59460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.833+0000 7f276c895640 1 --2- 172.21.15.33:0/1823246408 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56458ba4e8f0 0x56458ba59460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.834+0000 7f276c895640 1 -- 172.21.15.33:0/1823246408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56458b792620 con 0x56458ba59ad0 2024-05-15T14:05:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.834+0000 7f276c895640 1 --2- 172.21.15.33:0/1823246408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458ba59ad0 0x56458ba56cd0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f27600e04a0 tx=0x7f27600a9ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45dc0475c7e2b8b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.834+0000 7f27477fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:05:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.834+0000 7f27477fe640 1 -- 172.21.15.33:0/1823246408 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27600c3020 con 0x56458ba59ad0 2024-05-15T14:05:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.834+0000 7f27477fe640 1 -- 172.21.15.33:0/1823246408 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f27600b2bb0 con 0x56458ba59ad0 2024-05-15T14:05:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.834+0000 7f277100f740 1 -- 172.21.15.33:0/1823246408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f275000f980 con 0x56458ba59ad0 2024-05-15T14:05:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.834+0000 7f27477fe640 1 -- 172.21.15.33:0/1823246408 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f27600a9fd0 con 0x56458ba59ad0 2024-05-15T14:05:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.834+0000 7f277100f740 10 client.15903 fetch_fsmap learned FSMap version 27 2024-05-15T14:05:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.834+0000 7f277100f740 1 -- 172.21.15.33:0/1823246408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56458ba68240 con 0x56458ba59ad0 2024-05-15T14:05:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.834+0000 7f27477fe640 1 -- 172.21.15.33:0/1823246408 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f275000f980 con 0x56458ba59ad0 2024-05-15T14:05:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.834+0000 7f27477fe640 1 -- 172.21.15.33:0/1823246408 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f27600a5ad0 con 0x56458ba59ad0 2024-05-15T14:05:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.834+0000 7f277100f740 10 client.15903 fetch_fsmap finished waiting for FSMap version 27 2024-05-15T14:05:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.834+0000 7f277100f740 10 client.15903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:05:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.834+0000 7f277100f740 20 client.15903 populate_metadata read hostname 'smithi033' 2024-05-15T14:05:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.834+0000 7f277100f740 1 --2- 172.21.15.33:0/1823246408 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56458ba697f0 0x56458ba6bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.834+0000 7f277100f740 4 client.15903 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:05:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.834+0000 7f277100f740 1 -- 172.21.15.33:0/1823246408 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56458b9c3b80 con 0x56458ba697f0 2024-05-15T14:05:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.835+0000 7f2767fff640 1 --2- 172.21.15.33:0/1823246408 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56458ba697f0 0x56458ba6bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.835+0000 7f27477fe640 10 client.15903 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:05:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.835+0000 7f2767fff640 1 --2- 172.21.15.33:0/1823246408 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56458ba697f0 0x56458ba6bc00 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ab9e39cd9378ca3 server_cookie=8db6d383051789c6 in_seq=0 out_seq=0 2024-05-15T14:05:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.836+0000 7f27477fe640 1 -- 172.21.15.33:0/1823246408 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56458b9c3b80 con 0x56458ba697f0 2024-05-15T14:05:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.836+0000 7f27477fe640 10 client.15903 handle_command_reply: tid=0 2024-05-15T14:05:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.836+0000 7f277100f740 1 client.15903 shutdown 2024-05-15T14:05:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.836+0000 7f277100f740 1 -- 172.21.15.33:0/1823246408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458ba59ad0 msgr2=0x56458ba56cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.836+0000 7f277100f740 1 --2- 172.21.15.33:0/1823246408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458ba59ad0 0x56458ba56cd0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f27600e04a0 tx=0x7f27600a9ae0 comp rx=0 tx=0).stop 2024-05-15T14:05:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.836+0000 7f277100f740 1 -- 172.21.15.33:0/1823246408 shutdown_connections 2024-05-15T14:05:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.836+0000 7f277100f740 1 --2- 172.21.15.33:0/1823246408 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56458ba697f0 0x56458ba6bc00 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.836+0000 7f277100f740 1 --2- 172.21.15.33:0/1823246408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458ba59ad0 0x56458ba56cd0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.836+0000 7f277100f740 1 --2- 172.21.15.33:0/1823246408 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56458ba4f210 0x56458ba56750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.836+0000 7f27477fe640 0 client.15903 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:05:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.836+0000 7f277100f740 1 --2- 172.21.15.33:0/1823246408 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56458ba4e8f0 0x56458ba59460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.836+0000 7f277100f740 1 -- 172.21.15.33:0/1823246408 >> 172.21.15.33:0/1823246408 conn(0x56458ba4ae00 msgr2=0x56458ba4a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.836+0000 7f277100f740 1 -- 172.21.15.33:0/1823246408 shutdown_connections 2024-05-15T14:05:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.836+0000 7f277100f740 1 -- 172.21.15.33:0/1823246408 wait complete. 2024-05-15T14:05:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.836+0000 7f277100f740 20 client.15903 trim_cache size 0 max 16384 2024-05-15T14:05:00.859 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:00 smithi081 ceph-mon[36434]: pgmap v143: 97 pgs: 97 active+clean; 70 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 236 op/s 2024-05-15T14:05:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.851+0000 7f277100f740 1 Processor -- start 2024-05-15T14:05:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.851+0000 7f277100f740 1 -- start start 2024-05-15T14:05:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.852+0000 7f277100f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56458ba4e8f0 0x56458ba509b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.852+0000 7f277100f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458ba4f210 0x56458ba65670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.852+0000 7f277100f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56458ba59ad0 0x56458ba58f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.852+0000 7f277100f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f27681b0f50 con 0x56458ba4f210 2024-05-15T14:05:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.852+0000 7f277100f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f27681ae7c0 con 0x56458ba59ad0 2024-05-15T14:05:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.852+0000 7f277100f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f27681ac610 con 0x56458ba4e8f0 2024-05-15T14:05:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.852+0000 7f27677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458ba4f210 0x56458ba65670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.852+0000 7f27677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458ba4f210 0x56458ba65670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48358/0 (socket says 172.21.15.33:48358) 2024-05-15T14:05:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.852+0000 7f27677fe640 1 -- 172.21.15.33:0/51851152 learned_addr learned my addr 172.21.15.33:0/51851152 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.852+0000 7f276c895640 1 --2- 172.21.15.33:0/51851152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56458ba59ad0 0x56458ba58f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.852+0000 7f2767fff640 1 --2- 172.21.15.33:0/51851152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56458ba4e8f0 0x56458ba509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.852+0000 7f27677fe640 1 -- 172.21.15.33:0/51851152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56458ba4e8f0 msgr2=0x56458ba509b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.852+0000 7f27677fe640 1 --2- 172.21.15.33:0/51851152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56458ba4e8f0 0x56458ba509b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.852+0000 7f27677fe640 1 -- 172.21.15.33:0/51851152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56458ba59ad0 msgr2=0x56458ba58f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.852+0000 7f27677fe640 1 --2- 172.21.15.33:0/51851152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56458ba59ad0 0x56458ba58f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.852+0000 7f27677fe640 1 -- 172.21.15.33:0/51851152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56458ba4b9d0 con 0x56458ba4f210 2024-05-15T14:05:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.852+0000 7f27677fe640 1 --2- 172.21.15.33:0/51851152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458ba4f210 0x56458ba65670 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f275000f790 tx=0x7f2750005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8fed4ad40b695c server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.852+0000 7f275dffb640 1 -- 172.21.15.33:0/51851152 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275000f980 con 0x56458ba4f210 2024-05-15T14:05:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.852+0000 7f275dffb640 1 -- 172.21.15.33:0/51851152 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27500050c0 con 0x56458ba4f210 2024-05-15T14:05:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.852+0000 7f275dffb640 1 -- 172.21.15.33:0/51851152 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2750003bf0 con 0x56458ba4f210 2024-05-15T14:05:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.853+0000 7f277100f740 1 -- 172.21.15.33:0/51851152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458ba4f210 msgr2=0x56458ba65670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.853+0000 7f277100f740 1 --2- 172.21.15.33:0/51851152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458ba4f210 0x56458ba65670 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f275000f790 tx=0x7f2750005d90 comp rx=0 tx=0).stop 2024-05-15T14:05:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.853+0000 7f277100f740 1 -- 172.21.15.33:0/51851152 shutdown_connections 2024-05-15T14:05:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.853+0000 7f277100f740 1 --2- 172.21.15.33:0/51851152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56458ba59ad0 0x56458ba58f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.853+0000 7f277100f740 1 --2- 172.21.15.33:0/51851152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458ba4f210 0x56458ba65670 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.853+0000 7f277100f740 1 --2- 172.21.15.33:0/51851152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56458ba4e8f0 0x56458ba509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.853+0000 7f277100f740 1 -- 172.21.15.33:0/51851152 >> 172.21.15.33:0/51851152 conn(0x56458ba4ae00 msgr2=0x56458ba502a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.853+0000 7f277100f740 1 -- 172.21.15.33:0/51851152 shutdown_connections 2024-05-15T14:05:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.853+0000 7f277100f740 1 -- 172.21.15.33:0/51851152 wait complete. 2024-05-15T14:05:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.853+0000 7f277100f740 1 -- start start 2024-05-15T14:05:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.854+0000 7f277100f740 1 Processor -- start 2024-05-15T14:05:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.854+0000 7f277100f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56458ba4e8f0 0x56458ba61dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.854+0000 7f277100f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56458ba4f210 0x56458ba5d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.854+0000 7f277100f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458ba59ad0 0x56458ba5d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.854+0000 7f277100f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f27681acc80 con 0x56458ba59ad0 2024-05-15T14:05:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.854+0000 7f277100f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f27681ae0f0 con 0x56458ba4f210 2024-05-15T14:05:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.854+0000 7f277100f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f27681ad350 con 0x56458ba4e8f0 2024-05-15T14:05:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.854+0000 7f277100f740 10 client.0 _ll_register_callbacks cb 0x7f27600a9990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:05:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.854+0000 7f276c895640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458ba59ad0 0x56458ba5d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.854+0000 7f276c895640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458ba59ad0 0x56458ba5d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48360/0 (socket says 172.21.15.33:48360) 2024-05-15T14:05:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.854+0000 7f276c895640 1 -- 172.21.15.33:0/1019864126 learned_addr learned my addr 172.21.15.33:0/1019864126 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.854+0000 7f27677fe640 1 --2- 172.21.15.33:0/1019864126 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56458ba4f210 0x56458ba5d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.854+0000 7f2767fff640 1 --2- 172.21.15.33:0/1019864126 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56458ba4e8f0 0x56458ba61dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.854+0000 7f276c895640 1 -- 172.21.15.33:0/1019864126 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56458ba4e8f0 msgr2=0x56458ba61dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.854+0000 7f276c895640 1 --2- 172.21.15.33:0/1019864126 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56458ba4e8f0 0x56458ba61dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.854+0000 7f276c895640 1 -- 172.21.15.33:0/1019864126 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56458ba4f210 msgr2=0x56458ba5d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.854+0000 7f276c895640 1 --2- 172.21.15.33:0/1019864126 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56458ba4f210 0x56458ba5d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.854+0000 7f276c895640 1 -- 172.21.15.33:0/1019864126 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56458ba5dd20 con 0x56458ba59ad0 2024-05-15T14:05:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.854+0000 7f276c895640 1 --2- 172.21.15.33:0/1019864126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458ba59ad0 0x56458ba5d6b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f27600a6050 tx=0x7f27600a64e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d929bd5924d51a server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.855+0000 7f27477fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:05:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.855+0000 7f27477fe640 1 -- 172.21.15.33:0/1019864126 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27600c3070 con 0x56458ba59ad0 2024-05-15T14:05:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.855+0000 7f277100f740 1 -- 172.21.15.33:0/1019864126 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27500050c0 con 0x56458ba59ad0 2024-05-15T14:05:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.855+0000 7f27477fe640 1 -- 172.21.15.33:0/1019864126 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f27600a8ff0 con 0x56458ba59ad0 2024-05-15T14:05:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.855+0000 7f27477fe640 1 -- 172.21.15.33:0/1019864126 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f27600ae380 con 0x56458ba59ad0 2024-05-15T14:05:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.855+0000 7f277100f740 10 client.15909 fetch_fsmap learned FSMap version 27 2024-05-15T14:05:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.855+0000 7f277100f740 1 -- 172.21.15.33:0/1019864126 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56458bad3f60 con 0x56458ba59ad0 2024-05-15T14:05:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.855+0000 7f27477fe640 1 -- 172.21.15.33:0/1019864126 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27500050c0 con 0x56458ba59ad0 2024-05-15T14:05:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.855+0000 7f27477fe640 1 -- 172.21.15.33:0/1019864126 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f27600a9230 con 0x56458ba59ad0 2024-05-15T14:05:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.855+0000 7f277100f740 10 client.15909 fetch_fsmap finished waiting for FSMap version 27 2024-05-15T14:05:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.855+0000 7f277100f740 10 client.15909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:05:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.855+0000 7f277100f740 20 client.15909 populate_metadata read hostname 'smithi033' 2024-05-15T14:05:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.855+0000 7f277100f740 1 --2- 172.21.15.33:0/1019864126 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56458bad4760 0x56458bad6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.855+0000 7f277100f740 4 client.15909 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:05:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.855+0000 7f277100f740 1 -- 172.21.15.33:0/1019864126 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56458b9bd360 con 0x56458bad4760 2024-05-15T14:05:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.855+0000 7f2767fff640 1 --2- 172.21.15.33:0/1019864126 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56458bad4760 0x56458bad6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.856+0000 7f2767fff640 1 --2- 172.21.15.33:0/1019864126 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56458bad4760 0x56458bad6b80 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c127ecd143bcbe08 server_cookie=8651a4641abb063a in_seq=0 out_seq=0 2024-05-15T14:05:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.856+0000 7f27477fe640 10 client.15909 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:05:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.926+0000 7f27477fe640 1 -- 172.21.15.33:0/1019864126 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56458b9bd360 con 0x56458bad4760 2024-05-15T14:05:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.926+0000 7f27477fe640 10 client.15909 handle_command_reply: tid=0 2024-05-15T14:05:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.926+0000 7f277100f740 1 client.15909 shutdown 2024-05-15T14:05:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.927+0000 7f277100f740 1 -- 172.21.15.33:0/1019864126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458ba59ad0 msgr2=0x56458ba5d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.927+0000 7f277100f740 1 --2- 172.21.15.33:0/1019864126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458ba59ad0 0x56458ba5d6b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f27600a6050 tx=0x7f27600a64e0 comp rx=0 tx=0).stop 2024-05-15T14:05:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.927+0000 7f277100f740 1 -- 172.21.15.33:0/1019864126 shutdown_connections 2024-05-15T14:05:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.927+0000 7f277100f740 1 --2- 172.21.15.33:0/1019864126 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56458bad4760 0x56458bad6b80 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.927+0000 7f277100f740 1 --2- 172.21.15.33:0/1019864126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458ba59ad0 0x56458ba5d6b0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.927+0000 7f277100f740 1 --2- 172.21.15.33:0/1019864126 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56458ba4f210 0x56458ba5d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.927+0000 7f277100f740 1 --2- 172.21.15.33:0/1019864126 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56458ba4e8f0 0x56458ba61dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.927+0000 7f277100f740 1 -- 172.21.15.33:0/1019864126 >> 172.21.15.33:0/1019864126 conn(0x56458ba4ae00 msgr2=0x56458ba50b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.927+0000 7f27477fe640 0 client.15909 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:05:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.927+0000 7f277100f740 1 -- 172.21.15.33:0/1019864126 shutdown_connections 2024-05-15T14:05:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.927+0000 7f277100f740 1 -- 172.21.15.33:0/1019864126 wait complete. 2024-05-15T14:05:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.927+0000 7f277100f740 20 client.15909 trim_cache size 0 max 16384 2024-05-15T14:05:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:05:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:05:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:05:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.928+0000 7f276e2f8640 1 -- 172.21.15.33:0/3421370875 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f27540b2950 msgr2=0x7f27540b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.928+0000 7f276e2f8640 1 --2- 172.21.15.33:0/3421370875 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f27540b2950 0x7f27540b4e10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2758031440 tx=0x7f2758005b90 comp rx=0 tx=0).stop 2024-05-15T14:05:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.928+0000 7f276e2f8640 1 -- 172.21.15.33:0/3421370875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f276810d090 msgr2=0x7f276819db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.928+0000 7f276e2f8640 1 --2- 172.21.15.33:0/3421370875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f276810d090 0x7f276819db00 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f276000eb40 tx=0x7f276000c2d0 comp rx=0 tx=0).stop 2024-05-15T14:05:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.928+0000 7f276e2f8640 1 -- 172.21.15.33:0/3421370875 shutdown_connections 2024-05-15T14:05:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.928+0000 7f276e2f8640 1 --2- 172.21.15.33:0/3421370875 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f27540b2950 0x7f27540b4e10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.928+0000 7f276e2f8640 1 --2- 172.21.15.33:0/3421370875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f276810d090 0x7f276819db00 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.928+0000 7f276e2f8640 1 --2- 172.21.15.33:0/3421370875 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27681045e0 0x7f27681997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.928+0000 7f276e2f8640 1 --2- 172.21.15.33:0/3421370875 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2768103cc0 0x7f2768199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.928+0000 7f276e2f8640 1 -- 172.21.15.33:0/3421370875 >> 172.21.15.33:0/3421370875 conn(0x7f2768078a70 msgr2=0x7f27680feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.928+0000 7f276e2f8640 1 -- 172.21.15.33:0/3421370875 shutdown_connections 2024-05-15T14:05:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:00.928+0000 7f276e2f8640 1 -- 172.21.15.33:0/3421370875 wait complete. 2024-05-15T14:05:00.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:00 smithi033 ceph-mon[31650]: pgmap v143: 97 pgs: 97 active+clean; 70 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 236 op/s 2024-05-15T14:05:01.054 INFO:tasks.workunit.client.0.smithi033.stdout: 1 251076 5.56 MB/sec execute 36 sec latency 46.925 ms 2024-05-15T14:05:02.055 INFO:tasks.workunit.client.0.smithi033.stdout: 1 252314 5.63 MB/sec execute 37 sec latency 23.141 ms 2024-05-15T14:05:02.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:02 smithi052 ceph-mon[36190]: pgmap v144: 97 pgs: 97 active+clean; 70 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 236 op/s 2024-05-15T14:05:02.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:02 smithi081 ceph-mon[36434]: pgmap v144: 97 pgs: 97 active+clean; 70 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 236 op/s 2024-05-15T14:05:02.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:02 smithi033 ceph-mon[31650]: pgmap v144: 97 pgs: 97 active+clean; 70 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 236 op/s 2024-05-15T14:05:03.055 INFO:tasks.workunit.client.0.smithi033.stdout: 1 254475 5.70 MB/sec execute 38 sec latency 14.952 ms 2024-05-15T14:05:04.055 INFO:tasks.workunit.client.0.smithi033.stdout: 1 256158 5.84 MB/sec execute 39 sec latency 12.855 ms 2024-05-15T14:05:04.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:04 smithi052 ceph-mon[36190]: pgmap v145: 97 pgs: 97 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 402 op/s 2024-05-15T14:05:04.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:04 smithi081 ceph-mon[36434]: pgmap v145: 97 pgs: 97 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 402 op/s 2024-05-15T14:05:04.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:04 smithi033 ceph-mon[31650]: pgmap v145: 97 pgs: 97 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 402 op/s 2024-05-15T14:05:05.055 INFO:tasks.workunit.client.0.smithi033.stdout: 1 259186 6.16 MB/sec execute 40 sec latency 13.825 ms 2024-05-15T14:05:06.055 INFO:tasks.workunit.client.0.smithi033.stdout: 1 260886 6.15 MB/sec execute 41 sec latency 16.403 ms 2024-05-15T14:05:06.751 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:06 smithi033 ceph-mon[31650]: pgmap v146: 97 pgs: 97 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 264 op/s 2024-05-15T14:05:06.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:06 smithi052 ceph-mon[36190]: pgmap v146: 97 pgs: 97 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 264 op/s 2024-05-15T14:05:06.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:06 smithi081 ceph-mon[36434]: pgmap v146: 97 pgs: 97 active+clean; 67 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 264 op/s 2024-05-15T14:05:07.055 INFO:tasks.workunit.client.0.smithi033.stdout: 1 263073 6.33 MB/sec execute 42 sec latency 15.311 ms 2024-05-15T14:05:08.055 INFO:tasks.workunit.client.0.smithi033.stdout: 1 265873 6.52 MB/sec execute 43 sec latency 13.243 ms 2024-05-15T14:05:08.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:08 smithi052 ceph-mon[36190]: pgmap v147: 97 pgs: 97 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 365 op/s 2024-05-15T14:05:08.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:08 smithi081 ceph-mon[36434]: pgmap v147: 97 pgs: 97 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 365 op/s 2024-05-15T14:05:08.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:08 smithi033 ceph-mon[31650]: pgmap v147: 97 pgs: 97 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 365 op/s 2024-05-15T14:05:09.055 INFO:tasks.workunit.client.0.smithi033.stdout: 1 267485 6.59 MB/sec execute 44 sec latency 15.548 ms 2024-05-15T14:05:10.055 INFO:tasks.workunit.client.0.smithi033.stdout: 1 269998 6.75 MB/sec execute 45 sec latency 18.148 ms 2024-05-15T14:05:10.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:10 smithi052 ceph-mon[36190]: pgmap v148: 97 pgs: 97 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 267 op/s 2024-05-15T14:05:10.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:10 smithi052 ceph-mon[36190]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:05:10.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:10 smithi081 ceph-mon[36434]: pgmap v148: 97 pgs: 97 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 267 op/s 2024-05-15T14:05:10.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:10 smithi081 ceph-mon[36434]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:05:10.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:10 smithi033 ceph-mon[31650]: pgmap v148: 97 pgs: 97 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 267 op/s 2024-05-15T14:05:10.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:10 smithi033 ceph-mon[31650]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:05:11.055 INFO:tasks.workunit.client.0.smithi033.stdout: 1 272360 6.85 MB/sec execute 46 sec latency 11.421 ms 2024-05-15T14:05:12.055 INFO:tasks.workunit.client.0.smithi033.stdout: 1 274026 6.89 MB/sec execute 47 sec latency 14.816 ms 2024-05-15T14:05:12.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:12 smithi052 ceph-mon[36190]: pgmap v149: 97 pgs: 97 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 267 op/s 2024-05-15T14:05:12.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:12 smithi081 ceph-mon[36434]: pgmap v149: 97 pgs: 97 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 267 op/s 2024-05-15T14:05:12.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:12 smithi033 ceph-mon[31650]: pgmap v149: 97 pgs: 97 active+clean; 75 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 267 op/s 2024-05-15T14:05:13.055 INFO:tasks.workunit.client.0.smithi033.stdout: 1 276913 7.12 MB/sec execute 48 sec latency 11.470 ms 2024-05-15T14:05:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.026+0000 7f7834112640 1 -- 172.21.15.33:0/3348928433 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f782c1045e0 msgr2=0x7f782c10cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.026+0000 7f7834112640 1 --2- 172.21.15.33:0/3348928433 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f782c1045e0 0x7f782c10cac0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f781c009900 tx=0x7f781c02a480 comp rx=0 tx=0).stop 2024-05-15T14:05:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.026+0000 7f7834112640 1 -- 172.21.15.33:0/3348928433 shutdown_connections 2024-05-15T14:05:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.026+0000 7f7834112640 1 --2- 172.21.15.33:0/3348928433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f782c10d090 0x7f782c10f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.026+0000 7f7834112640 1 --2- 172.21.15.33:0/3348928433 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f782c1045e0 0x7f782c10cac0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.026+0000 7f7834112640 1 --2- 172.21.15.33:0/3348928433 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f782c103cc0 0x7f782c1040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.026+0000 7f7834112640 1 -- 172.21.15.33:0/3348928433 >> 172.21.15.33:0/3348928433 conn(0x7f782c078a70 msgr2=0x7f782c078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.026+0000 7f7834112640 1 -- 172.21.15.33:0/3348928433 shutdown_connections 2024-05-15T14:05:14.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.026+0000 7f7834112640 1 -- 172.21.15.33:0/3348928433 wait complete. 2024-05-15T14:05:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.027+0000 7f7834112640 1 Processor -- start 2024-05-15T14:05:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.027+0000 7f7834112640 1 -- start start 2024-05-15T14:05:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.027+0000 7f7834112640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f782c103cc0 0x7f782c199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.027+0000 7f7834112640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f782c1045e0 0x7f782c1997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.027+0000 7f7834112640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f782c10d090 0x7f782c19db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.027+0000 7f7834112640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f782c112cd0 con 0x7f782c10d090 2024-05-15T14:05:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.027+0000 7f7834112640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f782c112b50 con 0x7f782c1045e0 2024-05-15T14:05:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.027+0000 7f7834112640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f782c112e50 con 0x7f782c103cc0 2024-05-15T14:05:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.027+0000 7f78326af640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f782c10d090 0x7f782c19db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.027+0000 7f7831eae640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f782c103cc0 0x7f782c199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.027+0000 7f78326af640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f782c10d090 0x7f782c19db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59046/0 (socket says 172.21.15.33:59046) 2024-05-15T14:05:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.027+0000 7f7831eae640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f782c103cc0 0x7f782c199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:50138/0 (socket says 172.21.15.33:50138) 2024-05-15T14:05:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.027+0000 7f78326af640 1 -- 172.21.15.33:0/2343851311 learned_addr learned my addr 172.21.15.33:0/2343851311 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.027+0000 7f78316ad640 1 --2- 172.21.15.33:0/2343851311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f782c1045e0 0x7f782c1997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.027+0000 7f78326af640 1 -- 172.21.15.33:0/2343851311 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f782c103cc0 msgr2=0x7f782c199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.027+0000 7f78326af640 1 --2- 172.21.15.33:0/2343851311 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f782c103cc0 0x7f782c199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.028+0000 7f78326af640 1 -- 172.21.15.33:0/2343851311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f782c1045e0 msgr2=0x7f782c1997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.028+0000 7f78326af640 1 --2- 172.21.15.33:0/2343851311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f782c1045e0 0x7f782c1997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.028+0000 7f78326af640 1 -- 172.21.15.33:0/2343851311 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f782c19e260 con 0x7f782c10d090 2024-05-15T14:05:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.028+0000 7f7831eae640 1 --2- 172.21.15.33:0/2343851311 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f782c103cc0 0x7f782c199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:05:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.028+0000 7f78326af640 1 --2- 172.21.15.33:0/2343851311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f782c10d090 0x7f782c19db30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7824009c90 tx=0x7f782400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.028+0000 7f7822ffd640 1 -- 172.21.15.33:0/2343851311 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7824003b70 con 0x7f782c10d090 2024-05-15T14:05:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.028+0000 7f7822ffd640 1 -- 172.21.15.33:0/2343851311 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7824003d10 con 0x7f782c10d090 2024-05-15T14:05:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.028+0000 7f7834112640 1 -- 172.21.15.33:0/2343851311 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f782c19e550 con 0x7f782c10d090 2024-05-15T14:05:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.028+0000 7f7822ffd640 1 -- 172.21.15.33:0/2343851311 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7824010850 con 0x7f782c10d090 2024-05-15T14:05:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.028+0000 7f7834112640 1 -- 172.21.15.33:0/2343851311 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f782c1a5de0 con 0x7f782c10d090 2024-05-15T14:05:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.030+0000 7f7822ffd640 1 -- 172.21.15.33:0/2343851311 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7824004020 con 0x7f782c10d090 2024-05-15T14:05:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.031+0000 7f7836e20740 1 Processor -- start 2024-05-15T14:05:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.031+0000 7f7836e20740 1 -- start start 2024-05-15T14:05:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.031+0000 7f7836e20740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b20d228f0 0x558b20d22cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.031+0000 7f7836e20740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b20d23210 0x558b20d2d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.031+0000 7f7836e20740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b20d2dad0 0x558b20d2fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.031+0000 7f7836e20740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558b20c95c30 con 0x558b20d2dad0 2024-05-15T14:05:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.031+0000 7f7836e20740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558b20c857d0 con 0x558b20d23210 2024-05-15T14:05:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.031+0000 7f7836e20740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x558b20c89810 con 0x558b20d228f0 2024-05-15T14:05:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.031+0000 7f78316ad640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b20d228f0 0x558b20d22cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.031+0000 7f7831eae640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b20d23210 0x558b20d2d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.031+0000 7f7831eae640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b20d23210 0x558b20d2d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:41076/0 (socket says 172.21.15.33:41076) 2024-05-15T14:05:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.031+0000 7f78316ad640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b20d228f0 0x558b20d22cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:50146/0 (socket says 172.21.15.33:50146) 2024-05-15T14:05:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.031+0000 7f7831eae640 1 -- 172.21.15.33:0/958577758 learned_addr learned my addr 172.21.15.33:0/958577758 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.031+0000 7f78316ad640 1 --2- 172.21.15.33:0/958577758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b20d2dad0 0x558b20d2fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.032+0000 7f7822ffd640 1 --2- 172.21.15.33:0/2343851311 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f78180b2950 0x7f78180b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.032+0000 7f7831eae640 1 -- 172.21.15.33:0/958577758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b20d228f0 msgr2=0x558b20d22cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.032+0000 7f7831eae640 1 --2- 172.21.15.33:0/958577758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b20d228f0 0x558b20d22cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.032+0000 7f7831eae640 1 -- 172.21.15.33:0/958577758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b20d2dad0 msgr2=0x558b20d2fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.032+0000 7f7831eae640 1 --2- 172.21.15.33:0/958577758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b20d2dad0 0x558b20d2fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.032+0000 7f7831eae640 1 -- 172.21.15.33:0/958577758 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b2099a210 con 0x558b20d23210 2024-05-15T14:05:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.032+0000 7f78316ad640 1 --2- 172.21.15.33:0/958577758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b20d2dad0 0x558b20d2fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:05:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.032+0000 7f78326af640 1 --2- 172.21.15.33:0/2343851311 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f78180b2950 0x7f78180b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.032+0000 7f7831eae640 1 --2- 172.21.15.33:0/958577758 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b20d23210 0x558b20d2d590 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7828011540 tx=0x7f7828011a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f053ade44a39359 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.032+0000 7f7822ffd640 1 -- 172.21.15.33:0/2343851311 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f78240e2bb0 con 0x7f782c10d090 2024-05-15T14:05:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.032+0000 7f7817fff640 1 -- 172.21.15.33:0/958577758 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782800f640 con 0x558b20d23210 2024-05-15T14:05:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.032+0000 7f7817fff640 1 -- 172.21.15.33:0/958577758 <== mon.2 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f782800fc90 con 0x558b20d23210 2024-05-15T14:05:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.032+0000 7f7817fff640 1 -- 172.21.15.33:0/958577758 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7828019c60 con 0x558b20d23210 2024-05-15T14:05:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.033+0000 7f7836e20740 1 -- 172.21.15.33:0/958577758 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b20d23210 msgr2=0x558b20d2d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.033+0000 7f78326af640 1 --2- 172.21.15.33:0/2343851311 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f78180b2950 0x7f78180b4e10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f78240a9b40 tx=0x7f7824076770 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.033+0000 7f7836e20740 1 --2- 172.21.15.33:0/958577758 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b20d23210 0x558b20d2d590 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7828011540 tx=0x7f7828011a00 comp rx=0 tx=0).stop 2024-05-15T14:05:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.033+0000 7f7836e20740 1 -- 172.21.15.33:0/958577758 shutdown_connections 2024-05-15T14:05:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.033+0000 7f7836e20740 1 --2- 172.21.15.33:0/958577758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b20d2dad0 0x558b20d2fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.033+0000 7f7836e20740 1 --2- 172.21.15.33:0/958577758 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b20d23210 0x558b20d2d590 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.033+0000 7f7836e20740 1 --2- 172.21.15.33:0/958577758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b20d228f0 0x558b20d22cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.033+0000 7f7836e20740 1 -- 172.21.15.33:0/958577758 >> 172.21.15.33:0/958577758 conn(0x558b20d1ee00 msgr2=0x558b20d1f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.033+0000 7f7836e20740 1 -- 172.21.15.33:0/958577758 shutdown_connections 2024-05-15T14:05:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.033+0000 7f7836e20740 1 -- 172.21.15.33:0/958577758 wait complete. 2024-05-15T14:05:14.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.033+0000 7f7836e20740 1 -- start start 2024-05-15T14:05:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.033+0000 7f7836e20740 1 Processor -- start 2024-05-15T14:05:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.034+0000 7f7836e20740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b20d228f0 0x558b20d2d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.034+0000 7f7836e20740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b20d23210 0x558b20d2a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.034+0000 7f7836e20740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b20d2dad0 0x558b20d2ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.034+0000 7f7836e20740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558b2096b790 con 0x558b20d23210 2024-05-15T14:05:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.034+0000 7f7836e20740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558b20c41890 con 0x558b20d2dad0 2024-05-15T14:05:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.034+0000 7f7836e20740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x558b20d307a0 con 0x558b20d228f0 2024-05-15T14:05:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.034+0000 7f7836e20740 10 client.0 _ll_register_callbacks cb 0x558b20c44d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:05:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.034+0000 7f7831eae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b20d23210 0x558b20d2a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.034+0000 7f78316ad640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b20d228f0 0x558b20d2d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.034+0000 7f78316ad640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b20d228f0 0x558b20d2d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:50150/0 (socket says 172.21.15.33:50150) 2024-05-15T14:05:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.034+0000 7f78316ad640 1 -- 172.21.15.33:0/2895887632 learned_addr learned my addr 172.21.15.33:0/2895887632 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:14.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.034+0000 7f78316ad640 1 --2- 172.21.15.33:0/2895887632 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b20d2dad0 0x558b20d2ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.034+0000 7f78316ad640 1 -- 172.21.15.33:0/2895887632 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b20d2dad0 msgr2=0x558b20d2ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.034+0000 7f78316ad640 1 --2- 172.21.15.33:0/2895887632 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b20d2dad0 0x558b20d2ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.034+0000 7f78316ad640 1 -- 172.21.15.33:0/2895887632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b20d23210 msgr2=0x558b20d2a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.034+0000 7f78316ad640 1 --2- 172.21.15.33:0/2895887632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b20d23210 0x558b20d2a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.034+0000 7f78316ad640 1 -- 172.21.15.33:0/2895887632 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b20a66620 con 0x558b20d228f0 2024-05-15T14:05:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.034+0000 7f78316ad640 1 --2- 172.21.15.33:0/2895887632 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b20d228f0 0x558b20d2d470 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f781c02a9a0 tx=0x7f781c02a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc8d29907d1667fd server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.034+0000 7f78157fa640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-05-15T14:05:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.034+0000 7f78157fa640 1 -- 172.21.15.33:0/2895887632 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781c035040 con 0x558b20d228f0 2024-05-15T14:05:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.035+0000 7f78157fa640 1 -- 172.21.15.33:0/2895887632 <== mon.1 v2:172.21.15.81:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f781c005070 con 0x558b20d228f0 2024-05-15T14:05:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.035+0000 7f7836e20740 1 -- 172.21.15.33:0/2895887632 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f782800fc90 con 0x558b20d228f0 2024-05-15T14:05:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.035+0000 7f78157fa640 1 -- 172.21.15.33:0/2895887632 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f781c052050 con 0x558b20d228f0 2024-05-15T14:05:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.035+0000 7f7836e20740 10 client.25576 fetch_fsmap learned FSMap version 28 2024-05-15T14:05:14.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.035+0000 7f7836e20740 1 -- 172.21.15.33:0/2895887632 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558b20d3c200 con 0x558b20d228f0 2024-05-15T14:05:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.035+0000 7f78157fa640 1 -- 172.21.15.33:0/2895887632 <== mon.1 v2:172.21.15.81:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f782800fc90 con 0x558b20d228f0 2024-05-15T14:05:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.035+0000 7f78157fa640 1 -- 172.21.15.33:0/2895887632 <== mon.1 v2:172.21.15.81:3300/0 5 ==== fsmap(e 28) ==== 8342+0+0 (secure 0 0 0) 0x7f781c0035f0 con 0x558b20d228f0 2024-05-15T14:05:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.035+0000 7f7836e20740 10 client.25576 fetch_fsmap finished waiting for FSMap version 28 2024-05-15T14:05:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.035+0000 7f7836e20740 10 client.25576 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:05:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.035+0000 7f7836e20740 20 client.25576 populate_metadata read hostname 'smithi033' 2024-05-15T14:05:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.035+0000 7f7836e20740 1 --2- 172.21.15.33:0/2895887632 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558b20d3d710 0x558b20d3fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.035+0000 7f7836e20740 4 client.25576 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:05:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.035+0000 7f7836e20740 1 -- 172.21.15.33:0/2895887632 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b20c97b80 con 0x558b20d3d710 2024-05-15T14:05:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.035+0000 7f7831eae640 1 --2- 172.21.15.33:0/2895887632 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558b20d3d710 0x558b20d3fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.036+0000 7f78157fa640 10 client.25576 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:05:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.036+0000 7f7831eae640 1 --2- 172.21.15.33:0/2895887632 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558b20d3d710 0x558b20d3fb00 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dc0fb17ba46f1a5 server_cookie=f1245f1cd1331edf in_seq=0 out_seq=0 2024-05-15T14:05:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.037+0000 7f78157fa640 1 -- 172.21.15.33:0/2895887632 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558b20c97b80 con 0x558b20d3d710 2024-05-15T14:05:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.037+0000 7f78157fa640 10 client.25576 handle_command_reply: tid=0 2024-05-15T14:05:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.037+0000 7f7836e20740 1 client.25576 shutdown 2024-05-15T14:05:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.037+0000 7f7836e20740 1 -- 172.21.15.33:0/2895887632 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b20d228f0 msgr2=0x558b20d2d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.037+0000 7f7836e20740 1 --2- 172.21.15.33:0/2895887632 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b20d228f0 0x558b20d2d470 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f781c02a9a0 tx=0x7f781c02a480 comp rx=0 tx=0).stop 2024-05-15T14:05:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.037+0000 7f7836e20740 1 -- 172.21.15.33:0/2895887632 shutdown_connections 2024-05-15T14:05:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.037+0000 7f7836e20740 1 --2- 172.21.15.33:0/2895887632 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558b20d3d710 0x558b20d3fb00 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.037+0000 7f7836e20740 1 --2- 172.21.15.33:0/2895887632 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b20d2dad0 0x558b20d2ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.037+0000 7f78157fa640 0 client.25576 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:05:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.037+0000 7f7836e20740 1 --2- 172.21.15.33:0/2895887632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b20d23210 0x558b20d2a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.037+0000 7f7836e20740 1 --2- 172.21.15.33:0/2895887632 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b20d228f0 0x558b20d2d470 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.037+0000 7f7836e20740 1 -- 172.21.15.33:0/2895887632 >> 172.21.15.33:0/2895887632 conn(0x558b20d1ee00 msgr2=0x558b20d1e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.037+0000 7f7836e20740 1 -- 172.21.15.33:0/2895887632 shutdown_connections 2024-05-15T14:05:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.037+0000 7f7836e20740 1 -- 172.21.15.33:0/2895887632 wait complete. 2024-05-15T14:05:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.037+0000 7f7836e20740 20 client.25576 trim_cache size 0 max 16384 2024-05-15T14:05:14.056 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.053+0000 7f7836e20740 1 Processor -- start 2024-05-15T14:05:14.056 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.053+0000 7f7836e20740 1 -- start start 2024-05-15T14:05:14.056 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.053+0000 7f7836e20740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b20d228f0 0x558b20d24d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:14.056 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.053+0000 7f7836e20740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b20d23210 0x558b20d2cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.053+0000 7f7836e20740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b20d2dad0 0x558b20d453d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.053+0000 7f7836e20740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f782c1b1000 con 0x558b20d2dad0 2024-05-15T14:05:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.053+0000 7f7836e20740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f782c1ae8a0 con 0x558b20d23210 2024-05-15T14:05:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.053+0000 7f7836e20740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f782c1ac6f0 con 0x558b20d228f0 2024-05-15T14:05:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.053+0000 7f78316ad640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b20d2dad0 0x558b20d453d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.053+0000 7f78316ad640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b20d2dad0 0x558b20d453d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59066/0 (socket says 172.21.15.33:59066) 2024-05-15T14:05:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.053+0000 7f78316ad640 1 -- 172.21.15.33:0/1373310004 learned_addr learned my addr 172.21.15.33:0/1373310004 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.053+0000 7f7831eae640 1 --2- 172.21.15.33:0/1373310004 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b20d23210 0x558b20d2cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.053+0000 7f78316ad640 1 --2- 172.21.15.33:0/1373310004 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b20d228f0 0x558b20d24d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.053+0000 7f78316ad640 1 -- 172.21.15.33:0/1373310004 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b20d228f0 msgr2=0x558b20d24d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.053+0000 7f78316ad640 1 --2- 172.21.15.33:0/1373310004 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b20d228f0 0x558b20d24d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.053+0000 7f78316ad640 1 -- 172.21.15.33:0/1373310004 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b20d23210 msgr2=0x558b20d2cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.053+0000 7f78316ad640 1 --2- 172.21.15.33:0/1373310004 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b20d23210 0x558b20d2cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.054+0000 7f78316ad640 1 -- 172.21.15.33:0/1373310004 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b20d2b070 con 0x558b20d2dad0 2024-05-15T14:05:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.054+0000 7f7831eae640 1 --2- 172.21.15.33:0/1373310004 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b20d23210 0x558b20d2cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:05:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.054+0000 7f78316ad640 1 --2- 172.21.15.33:0/1373310004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b20d2dad0 0x558b20d453d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f781c0317c0 tx=0x7f781c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b4eb1fd49d499da server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.054+0000 7f78177fe640 1 -- 172.21.15.33:0/1373310004 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781c035040 con 0x558b20d2dad0 2024-05-15T14:05:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.054+0000 7f78177fe640 1 -- 172.21.15.33:0/1373310004 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f781c0035f0 con 0x558b20d2dad0 2024-05-15T14:05:14.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.054+0000 7f78177fe640 1 -- 172.21.15.33:0/1373310004 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781c003eb0 con 0x558b20d2dad0 2024-05-15T14:05:14.059 INFO:tasks.workunit.client.0.smithi033.stdout: 1 278457 7.09 MB/sec execute 49 sec latency 19.463 ms 2024-05-15T14:05:14.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.054+0000 7f7836e20740 1 -- 172.21.15.33:0/1373310004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b20d2dad0 msgr2=0x558b20d453d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:14.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.054+0000 7f7836e20740 1 --2- 172.21.15.33:0/1373310004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b20d2dad0 0x558b20d453d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f781c0317c0 tx=0x7f781c031400 comp rx=0 tx=0).stop 2024-05-15T14:05:14.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.055+0000 7f7836e20740 1 -- 172.21.15.33:0/1373310004 shutdown_connections 2024-05-15T14:05:14.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.055+0000 7f7836e20740 1 --2- 172.21.15.33:0/1373310004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b20d2dad0 0x558b20d453d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:14.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.055+0000 7f7836e20740 1 --2- 172.21.15.33:0/1373310004 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b20d23210 0x558b20d2cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:14.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.055+0000 7f7836e20740 1 --2- 172.21.15.33:0/1373310004 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b20d228f0 0x558b20d24d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:14.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.055+0000 7f7836e20740 1 -- 172.21.15.33:0/1373310004 >> 172.21.15.33:0/1373310004 conn(0x558b20d1ee00 msgr2=0x558b20d25c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:14.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.055+0000 7f7836e20740 1 -- 172.21.15.33:0/1373310004 shutdown_connections 2024-05-15T14:05:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.055+0000 7f7836e20740 1 -- 172.21.15.33:0/1373310004 wait complete. 2024-05-15T14:05:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.055+0000 7f7836e20740 1 -- start start 2024-05-15T14:05:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.055+0000 7f7836e20740 1 Processor -- start 2024-05-15T14:05:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.055+0000 7f7836e20740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b20d228f0 0x558b20d35fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.055+0000 7f7836e20740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b20d23210 0x558b20d31310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.055+0000 7f7836e20740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b20d2dad0 0x558b20d31870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.055+0000 7f7836e20740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f782c1acd60 con 0x558b20d228f0 2024-05-15T14:05:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.055+0000 7f7836e20740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f782c1ae1d0 con 0x558b20d2dad0 2024-05-15T14:05:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.055+0000 7f7836e20740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f782c1ad430 con 0x558b20d23210 2024-05-15T14:05:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.055+0000 7f7836e20740 10 client.0 _ll_register_callbacks cb 0x7f781c0316b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:05:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.055+0000 7f78316ad640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b20d228f0 0x558b20d35fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.055+0000 7f7831eae640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b20d23210 0x558b20d31310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.056+0000 7f7831eae640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b20d23210 0x558b20d31310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:50168/0 (socket says 172.21.15.33:50168) 2024-05-15T14:05:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.056+0000 7f78316ad640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b20d228f0 0x558b20d35fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59072/0 (socket says 172.21.15.33:59072) 2024-05-15T14:05:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.056+0000 7f7831eae640 1 -- 172.21.15.33:0/3296111946 learned_addr learned my addr 172.21.15.33:0/3296111946 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.056+0000 7f78316ad640 1 --2- 172.21.15.33:0/3296111946 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b20d2dad0 0x558b20d31870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.056+0000 7f7831eae640 1 -- 172.21.15.33:0/3296111946 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b20d2dad0 msgr2=0x558b20d31870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.056+0000 7f7831eae640 1 --2- 172.21.15.33:0/3296111946 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b20d2dad0 0x558b20d31870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.056+0000 7f7831eae640 1 -- 172.21.15.33:0/3296111946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b20d228f0 msgr2=0x558b20d35fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.056+0000 7f7831eae640 1 --2- 172.21.15.33:0/3296111946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b20d228f0 0x558b20d35fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.056+0000 7f7831eae640 1 -- 172.21.15.33:0/3296111946 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b20d320f0 con 0x558b20d23210 2024-05-15T14:05:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.056+0000 7f78316ad640 1 --2- 172.21.15.33:0/3296111946 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b20d2dad0 0x558b20d31870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:05:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.056+0000 7f7831eae640 1 --2- 172.21.15.33:0/3296111946 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b20d23210 0x558b20d31310 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7828005fd0 tx=0x7f7828005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d32ed51a029950a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.056+0000 7f78157fa640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-05-15T14:05:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.056+0000 7f78157fa640 1 -- 172.21.15.33:0/3296111946 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78280196d0 con 0x558b20d23210 2024-05-15T14:05:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.056+0000 7f78157fa640 1 -- 172.21.15.33:0/3296111946 <== mon.1 v2:172.21.15.81:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f7828019b40 con 0x558b20d23210 2024-05-15T14:05:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.056+0000 7f7836e20740 1 -- 172.21.15.33:0/3296111946 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f781c0035f0 con 0x558b20d23210 2024-05-15T14:05:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.056+0000 7f78157fa640 1 -- 172.21.15.33:0/3296111946 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7828019d40 con 0x558b20d23210 2024-05-15T14:05:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.056+0000 7f7836e20740 10 client.25582 fetch_fsmap learned FSMap version 28 2024-05-15T14:05:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.056+0000 7f7836e20740 1 -- 172.21.15.33:0/3296111946 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558b20da8000 con 0x558b20d23210 2024-05-15T14:05:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.057+0000 7f78157fa640 1 -- 172.21.15.33:0/3296111946 <== mon.1 v2:172.21.15.81:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f781c0035f0 con 0x558b20d23210 2024-05-15T14:05:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.057+0000 7f78157fa640 1 -- 172.21.15.33:0/3296111946 <== mon.1 v2:172.21.15.81:3300/0 5 ==== fsmap(e 28) ==== 8342+0+0 (secure 0 0 0) 0x7f782800f950 con 0x558b20d23210 2024-05-15T14:05:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.057+0000 7f7836e20740 10 client.25582 fetch_fsmap finished waiting for FSMap version 28 2024-05-15T14:05:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.057+0000 7f7836e20740 10 client.25582 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:05:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.057+0000 7f7836e20740 20 client.25582 populate_metadata read hostname 'smithi033' 2024-05-15T14:05:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.057+0000 7f7836e20740 1 --2- 172.21.15.33:0/3296111946 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558b20da85f0 0x558b20daaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.057+0000 7f7836e20740 4 client.25582 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:05:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.057+0000 7f7836e20740 1 -- 172.21.15.33:0/3296111946 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558b20c91360 con 0x558b20da85f0 2024-05-15T14:05:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.057+0000 7f78316ad640 1 --2- 172.21.15.33:0/3296111946 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558b20da85f0 0x558b20daaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.057+0000 7f78157fa640 10 client.25582 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:05:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.057+0000 7f78316ad640 1 --2- 172.21.15.33:0/3296111946 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558b20da85f0 0x558b20daaa10 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=967ce310e84dbbd2 server_cookie=7f9ea137905c0d57 in_seq=0 out_seq=0 2024-05-15T14:05:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.116+0000 7f78157fa640 1 -- 172.21.15.33:0/3296111946 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558b20c91360 con 0x558b20da85f0 2024-05-15T14:05:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.116+0000 7f78157fa640 10 client.25582 handle_command_reply: tid=0 2024-05-15T14:05:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.116+0000 7f7836e20740 1 client.25582 shutdown 2024-05-15T14:05:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.116+0000 7f7836e20740 1 -- 172.21.15.33:0/3296111946 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b20d23210 msgr2=0x558b20d31310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.116+0000 7f7836e20740 1 --2- 172.21.15.33:0/3296111946 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b20d23210 0x558b20d31310 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7828005fd0 tx=0x7f7828005850 comp rx=0 tx=0).stop 2024-05-15T14:05:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.116+0000 7f7836e20740 1 -- 172.21.15.33:0/3296111946 shutdown_connections 2024-05-15T14:05:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.116+0000 7f7836e20740 1 --2- 172.21.15.33:0/3296111946 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558b20da85f0 0x558b20daaa10 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.116+0000 7f7836e20740 1 --2- 172.21.15.33:0/3296111946 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558b20d2dad0 0x558b20d31870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.116+0000 7f78157fa640 0 client.25582 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:05:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.116+0000 7f7836e20740 1 --2- 172.21.15.33:0/3296111946 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558b20d23210 0x558b20d31310 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.116+0000 7f7836e20740 1 --2- 172.21.15.33:0/3296111946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b20d228f0 0x558b20d35fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.116+0000 7f7836e20740 1 -- 172.21.15.33:0/3296111946 >> 172.21.15.33:0/3296111946 conn(0x558b20d1ee00 msgr2=0x558b20d1d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.116+0000 7f7836e20740 1 -- 172.21.15.33:0/3296111946 shutdown_connections 2024-05-15T14:05:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.116+0000 7f7836e20740 1 -- 172.21.15.33:0/3296111946 wait complete. 2024-05-15T14:05:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.116+0000 7f7836e20740 20 client.25582 trim_cache size 0 max 16384 2024-05-15T14:05:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:05:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:05:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:05:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.117+0000 7f7834112640 1 -- 172.21.15.33:0/2343851311 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f78180b2950 msgr2=0x7f78180b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.117+0000 7f7834112640 1 --2- 172.21.15.33:0/2343851311 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f78180b2950 0x7f78180b4e10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f78240a9b40 tx=0x7f7824076770 comp rx=0 tx=0).stop 2024-05-15T14:05:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.117+0000 7f7834112640 1 -- 172.21.15.33:0/2343851311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f782c10d090 msgr2=0x7f782c19db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.117+0000 7f7834112640 1 --2- 172.21.15.33:0/2343851311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f782c10d090 0x7f782c19db30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7824009c90 tx=0x7f782400c6a0 comp rx=0 tx=0).stop 2024-05-15T14:05:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.117+0000 7f7834112640 1 -- 172.21.15.33:0/2343851311 shutdown_connections 2024-05-15T14:05:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.117+0000 7f7834112640 1 --2- 172.21.15.33:0/2343851311 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f78180b2950 0x7f78180b4e10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.117+0000 7f7834112640 1 --2- 172.21.15.33:0/2343851311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f782c10d090 0x7f782c19db30 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.117+0000 7f7834112640 1 --2- 172.21.15.33:0/2343851311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f782c1045e0 0x7f782c1997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.117+0000 7f7834112640 1 --2- 172.21.15.33:0/2343851311 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f782c103cc0 0x7f782c199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.117+0000 7f7834112640 1 -- 172.21.15.33:0/2343851311 >> 172.21.15.33:0/2343851311 conn(0x7f782c078a70 msgr2=0x7f782c0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.117+0000 7f7834112640 1 -- 172.21.15.33:0/2343851311 shutdown_connections 2024-05-15T14:05:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:14.117+0000 7f7834112640 1 -- 172.21.15.33:0/2343851311 wait complete. 2024-05-15T14:05:14.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:14 smithi052 ceph-mon[36190]: pgmap v150: 97 pgs: 97 active+clean; 82 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 369 op/s 2024-05-15T14:05:14.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:14 smithi081 ceph-mon[36434]: pgmap v150: 97 pgs: 97 active+clean; 82 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 369 op/s 2024-05-15T14:05:14.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:14 smithi033 ceph-mon[31650]: pgmap v150: 97 pgs: 97 active+clean; 82 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 369 op/s 2024-05-15T14:05:15.055 INFO:tasks.workunit.client.0.smithi033.stdout: 1 280668 7.23 MB/sec execute 50 sec latency 55.140 ms 2024-05-15T14:05:15.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:15 smithi052 ceph-mon[36190]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:05:15.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:15 smithi081 ceph-mon[36434]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:05:15.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:15 smithi033 ceph-mon[31650]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:05:16.055 INFO:tasks.workunit.client.0.smithi033.stdout: 1 282695 7.23 MB/sec execute 51 sec latency 45.533 ms 2024-05-15T14:05:16.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:16 smithi052 ceph-mon[36190]: pgmap v151: 97 pgs: 97 active+clean; 82 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 202 op/s 2024-05-15T14:05:16.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:16 smithi081 ceph-mon[36434]: pgmap v151: 97 pgs: 97 active+clean; 82 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 202 op/s 2024-05-15T14:05:16.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:16 smithi033 ceph-mon[31650]: pgmap v151: 97 pgs: 97 active+clean; 82 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 202 op/s 2024-05-15T14:05:17.055 INFO:tasks.workunit.client.0.smithi033.stdout: 1 284393 7.32 MB/sec execute 52 sec latency 14.598 ms 2024-05-15T14:05:18.055 INFO:tasks.workunit.client.0.smithi033.stdout: 1 286771 7.42 MB/sec execute 53 sec latency 64.654 ms 2024-05-15T14:05:18.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:18 smithi052 ceph-mon[36190]: pgmap v152: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 336 op/s 2024-05-15T14:05:18.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:18 smithi081 ceph-mon[36434]: pgmap v152: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 336 op/s 2024-05-15T14:05:18.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:18 smithi033 ceph-mon[31650]: pgmap v152: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 336 op/s 2024-05-15T14:05:19.055 INFO:tasks.workunit.client.0.smithi033.stdout: 1 288245 7.41 MB/sec execute 54 sec latency 13.962 ms 2024-05-15T14:05:20.056 INFO:tasks.workunit.client.0.smithi033.stdout: 1 290994 7.59 MB/sec execute 55 sec latency 12.308 ms 2024-05-15T14:05:20.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:20 smithi052 ceph-mon[36190]: pgmap v153: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 235 op/s 2024-05-15T14:05:20.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:20 smithi081 ceph-mon[36434]: pgmap v153: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 235 op/s 2024-05-15T14:05:20.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:20 smithi033 ceph-mon[31650]: pgmap v153: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 235 op/s 2024-05-15T14:05:21.056 INFO:tasks.workunit.client.0.smithi033.stdout: 1 292323 7.57 MB/sec execute 56 sec latency 36.681 ms 2024-05-15T14:05:22.056 INFO:tasks.workunit.client.0.smithi033.stdout: 1 294822 7.68 MB/sec execute 57 sec latency 10.378 ms 2024-05-15T14:05:22.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:22 smithi052 ceph-mon[36190]: pgmap v154: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 235 op/s 2024-05-15T14:05:22.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:22 smithi081 ceph-mon[36434]: pgmap v154: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 235 op/s 2024-05-15T14:05:22.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:22 smithi033 ceph-mon[31650]: pgmap v154: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 235 op/s 2024-05-15T14:05:23.056 INFO:tasks.workunit.client.0.smithi033.stdout: 1 296935 7.68 MB/sec execute 58 sec latency 56.438 ms 2024-05-15T14:05:24.056 INFO:tasks.workunit.client.0.smithi033.stdout: 1 298715 7.75 MB/sec execute 59 sec latency 15.594 ms 2024-05-15T14:05:24.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:24 smithi052 ceph-mon[36190]: pgmap v155: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 349 op/s 2024-05-15T14:05:24.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:24 smithi081 ceph-mon[36434]: pgmap v155: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 349 op/s 2024-05-15T14:05:24.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:24 smithi033 ceph-mon[31650]: pgmap v155: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 349 op/s 2024-05-15T14:05:25.056 INFO:tasks.workunit.client.0.smithi033.stdout: 1 301627 7.91 MB/sec execute 60 sec latency 15.034 ms 2024-05-15T14:05:26.056 INFO:tasks.workunit.client.0.smithi033.stdout: 1 302925 7.89 MB/sec execute 61 sec latency 59.221 ms 2024-05-15T14:05:26.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:26 smithi052 ceph-mon[36190]: pgmap v156: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 248 op/s 2024-05-15T14:05:26.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:26 smithi081 ceph-mon[36434]: pgmap v156: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 248 op/s 2024-05-15T14:05:26.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:26 smithi033 ceph-mon[31650]: pgmap v156: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 248 op/s 2024-05-15T14:05:27.056 INFO:tasks.workunit.client.0.smithi033.stdout: 1 305495 7.99 MB/sec execute 62 sec latency 61.231 ms 2024-05-15T14:05:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.210+0000 7fe8c26e1640 1 -- 172.21.15.33:0/7588230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8bc104600 msgr2=0x7fe8bc10cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.210+0000 7fe8c26e1640 1 --2- 172.21.15.33:0/7588230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8bc104600 0x7fe8bc10cae0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0009900 tx=0x7fe8b002a480 comp rx=0 tx=0).stop 2024-05-15T14:05:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.210+0000 7fe8c26e1640 1 -- 172.21.15.33:0/7588230 shutdown_connections 2024-05-15T14:05:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.210+0000 7fe8c26e1640 1 --2- 172.21.15.33:0/7588230 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8bc10d0b0 0x7fe8bc10f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.210+0000 7fe8c26e1640 1 --2- 172.21.15.33:0/7588230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8bc104600 0x7fe8bc10cae0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.210+0000 7fe8c26e1640 1 --2- 172.21.15.33:0/7588230 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8bc103ce0 0x7fe8bc1040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.210+0000 7fe8c26e1640 1 -- 172.21.15.33:0/7588230 >> 172.21.15.33:0/7588230 conn(0x7fe8bc078aa0 msgr2=0x7fe8bc078eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.210+0000 7fe8c26e1640 1 -- 172.21.15.33:0/7588230 shutdown_connections 2024-05-15T14:05:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.210+0000 7fe8c26e1640 1 -- 172.21.15.33:0/7588230 wait complete. 2024-05-15T14:05:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.211+0000 7fe8c26e1640 1 Processor -- start 2024-05-15T14:05:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.211+0000 7fe8c26e1640 1 -- start start 2024-05-15T14:05:27.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.211+0000 7fe8c26e1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8bc103ce0 0x7fe8bc199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:27.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.211+0000 7fe8c26e1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8bc104600 0x7fe8bc1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:27.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.211+0000 7fe8c26e1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8bc10d0b0 0x7fe8bc19db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:27.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.211+0000 7fe8c26e1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc112cf0 con 0x7fe8bc103ce0 2024-05-15T14:05:27.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.211+0000 7fe8c26e1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc112b70 con 0x7fe8bc10d0b0 2024-05-15T14:05:27.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.211+0000 7fe8c26e1640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc112e70 con 0x7fe8bc104600 2024-05-15T14:05:27.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.212+0000 7fe8bbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8bc103ce0 0x7fe8bc199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:27.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.212+0000 7fe8bbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8bc103ce0 0x7fe8bc199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40346/0 (socket says 172.21.15.33:40346) 2024-05-15T14:05:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.212+0000 7fe8bbfff640 1 -- 172.21.15.33:0/3141402163 learned_addr learned my addr 172.21.15.33:0/3141402163 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.212+0000 7fe8c0c7e640 1 --2- 172.21.15.33:0/3141402163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8bc10d0b0 0x7fe8bc19db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.212+0000 7fe8bb7fe640 1 --2- 172.21.15.33:0/3141402163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8bc104600 0x7fe8bc1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.212+0000 7fe8bbfff640 1 -- 172.21.15.33:0/3141402163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8bc104600 msgr2=0x7fe8bc1997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.212+0000 7fe8bbfff640 1 --2- 172.21.15.33:0/3141402163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8bc104600 0x7fe8bc1997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.212+0000 7fe8bbfff640 1 -- 172.21.15.33:0/3141402163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8bc10d0b0 msgr2=0x7fe8bc19db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.212+0000 7fe8bbfff640 1 --2- 172.21.15.33:0/3141402163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8bc10d0b0 0x7fe8bc19db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.212+0000 7fe8bbfff640 1 -- 172.21.15.33:0/3141402163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8bc19e280 con 0x7fe8bc103ce0 2024-05-15T14:05:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.212+0000 7fe8c0c7e640 1 --2- 172.21.15.33:0/3141402163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8bc10d0b0 0x7fe8bc19db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:05:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.212+0000 7fe8bbfff640 1 --2- 172.21.15.33:0/3141402163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8bc103ce0 0x7fe8bc199290 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00e9f0 tx=0x7fe8ac00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.212+0000 7fe8b97fa640 1 -- 172.21.15.33:0/3141402163 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac00cd80 con 0x7fe8bc103ce0 2024-05-15T14:05:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.212+0000 7fe8c26e1640 1 -- 172.21.15.33:0/3141402163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8bc19e570 con 0x7fe8bc103ce0 2024-05-15T14:05:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.212+0000 7fe8b97fa640 1 -- 172.21.15.33:0/3141402163 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8ac0054d0 con 0x7fe8bc103ce0 2024-05-15T14:05:27.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.212+0000 7fe8b97fa640 1 -- 172.21.15.33:0/3141402163 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac010690 con 0x7fe8bc103ce0 2024-05-15T14:05:27.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.212+0000 7fe8c26e1640 1 -- 172.21.15.33:0/3141402163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8bc1a5e00 con 0x7fe8bc103ce0 2024-05-15T14:05:27.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.215+0000 7fe8c5402740 1 Processor -- start 2024-05-15T14:05:27.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.215+0000 7fe8b97fa640 1 -- 172.21.15.33:0/3141402163 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe8ac004e90 con 0x7fe8bc103ce0 2024-05-15T14:05:27.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.215+0000 7fe8c5402740 1 -- start start 2024-05-15T14:05:27.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.215+0000 7fe8c5402740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d376b08f0 0x556d376b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:27.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.215+0000 7fe8c5402740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d376b1210 0x556d376bb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:27.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.215+0000 7fe8c5402740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556d376bbad0 0x556d376bdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:27.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.215+0000 7fe8c5402740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556d37623c30 con 0x556d376b08f0 2024-05-15T14:05:27.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.215+0000 7fe8c5402740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556d376137d0 con 0x556d376b1210 2024-05-15T14:05:27.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.215+0000 7fe8c5402740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x556d37617810 con 0x556d376bbad0 2024-05-15T14:05:27.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.215+0000 7fe8c0c7e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d376b08f0 0x556d376b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.215+0000 7fe8c0c7e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d376b08f0 0x556d376b0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40354/0 (socket says 172.21.15.33:40354) 2024-05-15T14:05:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.215+0000 7fe8c0c7e640 1 -- 172.21.15.33:0/3203936786 learned_addr learned my addr 172.21.15.33:0/3203936786 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.215+0000 7fe8bb7fe640 1 --2- 172.21.15.33:0/3203936786 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556d376bbad0 0x556d376bdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.215+0000 7fe8bbfff640 1 --2- 172.21.15.33:0/3203936786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d376b1210 0x556d376bb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.216+0000 7fe8bb7fe640 1 -- 172.21.15.33:0/3203936786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d376b1210 msgr2=0x556d376bb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.216+0000 7fe8bb7fe640 1 --2- 172.21.15.33:0/3203936786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d376b1210 0x556d376bb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.216+0000 7fe8bb7fe640 1 -- 172.21.15.33:0/3203936786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d376b08f0 msgr2=0x556d376b0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.216+0000 7fe8bb7fe640 1 --2- 172.21.15.33:0/3203936786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d376b08f0 0x556d376b0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.216+0000 7fe8bb7fe640 1 -- 172.21.15.33:0/3203936786 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d37328210 con 0x556d376bbad0 2024-05-15T14:05:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.216+0000 7fe8c0c7e640 1 --2- 172.21.15.33:0/3203936786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d376b08f0 0x556d376b0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:05:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.216+0000 7fe8b97fa640 1 --2- 172.21.15.33:0/3141402163 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fe8a40b2950 0x7fe8a40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.216+0000 7fe8bb7fe640 1 --2- 172.21.15.33:0/3203936786 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556d376bbad0 0x556d376bdec0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0009230 tx=0x7fe8b0032b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=946240652b5caff1 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.216+0000 7fe8c0c7e640 1 --2- 172.21.15.33:0/3141402163 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fe8a40b2950 0x7fe8a40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.216+0000 7fe8a27fc640 1 -- 172.21.15.33:0/3203936786 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0040070 con 0x556d376bbad0 2024-05-15T14:05:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.216+0000 7fe8b97fa640 1 -- 172.21.15.33:0/3141402163 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7fe8ac0e2a00 con 0x7fe8bc103ce0 2024-05-15T14:05:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.216+0000 7fe8a27fc640 1 -- 172.21.15.33:0/3203936786 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8b0004030 con 0x556d376bbad0 2024-05-15T14:05:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.216+0000 7fe8a27fc640 1 -- 172.21.15.33:0/3203936786 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0035040 con 0x556d376bbad0 2024-05-15T14:05:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.217+0000 7fe8c5402740 1 -- 172.21.15.33:0/3203936786 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556d376bbad0 msgr2=0x556d376bdec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.217+0000 7fe8c0c7e640 1 --2- 172.21.15.33:0/3141402163 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fe8a40b2950 0x7fe8a40b4e10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4009fd0 tx=0x7fe8b4009500 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.217+0000 7fe8c5402740 1 --2- 172.21.15.33:0/3203936786 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556d376bbad0 0x556d376bdec0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0009230 tx=0x7fe8b0032b20 comp rx=0 tx=0).stop 2024-05-15T14:05:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.217+0000 7fe8c5402740 1 -- 172.21.15.33:0/3203936786 shutdown_connections 2024-05-15T14:05:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.217+0000 7fe8c5402740 1 --2- 172.21.15.33:0/3203936786 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556d376bbad0 0x556d376bdec0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.217+0000 7fe8c5402740 1 --2- 172.21.15.33:0/3203936786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d376b1210 0x556d376bb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.217+0000 7fe8c5402740 1 --2- 172.21.15.33:0/3203936786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d376b08f0 0x556d376b0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.217+0000 7fe8c5402740 1 -- 172.21.15.33:0/3203936786 >> 172.21.15.33:0/3203936786 conn(0x556d376ace00 msgr2=0x556d376ad1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.217+0000 7fe8c5402740 1 -- 172.21.15.33:0/3203936786 shutdown_connections 2024-05-15T14:05:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.217+0000 7fe8c5402740 1 -- 172.21.15.33:0/3203936786 wait complete. 2024-05-15T14:05:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.217+0000 7fe8c5402740 1 -- start start 2024-05-15T14:05:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.218+0000 7fe8c5402740 1 Processor -- start 2024-05-15T14:05:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.218+0000 7fe8c5402740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d376b08f0 0x556d376bb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.218+0000 7fe8c5402740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556d376b1210 0x556d376b8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.218+0000 7fe8c5402740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d376bbad0 0x556d376b8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.218+0000 7fe8c5402740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556d372f9790 con 0x556d376b08f0 2024-05-15T14:05:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.218+0000 7fe8c5402740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556d375cf890 con 0x556d376bbad0 2024-05-15T14:05:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.218+0000 7fe8c5402740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x556d376be740 con 0x556d376b1210 2024-05-15T14:05:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.218+0000 7fe8c5402740 10 client.0 _ll_register_callbacks cb 0x556d375d2d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:05:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.218+0000 7fe8bbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d376b08f0 0x556d376bb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.218+0000 7fe8bbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d376b08f0 0x556d376bb460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40368/0 (socket says 172.21.15.33:40368) 2024-05-15T14:05:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.218+0000 7fe8bbfff640 1 -- 172.21.15.33:0/2601103615 learned_addr learned my addr 172.21.15.33:0/2601103615 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.218+0000 7fe8bb7fe640 1 --2- 172.21.15.33:0/2601103615 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556d376b1210 0x556d376b8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.218+0000 7fe8c0c7e640 1 --2- 172.21.15.33:0/2601103615 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d376bbad0 0x556d376b8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.218+0000 7fe8bbfff640 1 -- 172.21.15.33:0/2601103615 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556d376b1210 msgr2=0x556d376b8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.218+0000 7fe8bbfff640 1 --2- 172.21.15.33:0/2601103615 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556d376b1210 0x556d376b8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.218+0000 7fe8bbfff640 1 -- 172.21.15.33:0/2601103615 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d376bbad0 msgr2=0x556d376b8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.218+0000 7fe8bbfff640 1 --2- 172.21.15.33:0/2601103615 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d376bbad0 0x556d376b8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.218+0000 7fe8bbfff640 1 -- 172.21.15.33:0/2601103615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d373f4620 con 0x556d376b08f0 2024-05-15T14:05:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.218+0000 7fe8bb7fe640 1 --2- 172.21.15.33:0/2601103615 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556d376b1210 0x556d376b8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:05:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.218+0000 7fe8bbfff640 1 --2- 172.21.15.33:0/2601103615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d376b08f0 0x556d376bb460 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac0e01c0 tx=0x7fe8ac0e6140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba30f966e1ff6c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.218+0000 7fe89b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:05:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.219+0000 7fe89b7fe640 1 -- 172.21.15.33:0/2601103615 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac0e9020 con 0x556d376b08f0 2024-05-15T14:05:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.219+0000 7fe8c5402740 1 -- 172.21.15.33:0/2601103615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8b0004030 con 0x556d376b08f0 2024-05-15T14:05:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.219+0000 7fe89b7fe640 1 -- 172.21.15.33:0/2601103615 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7fe8ac0b28d0 con 0x556d376b08f0 2024-05-15T14:05:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.219+0000 7fe89b7fe640 1 -- 172.21.15.33:0/2601103615 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe8ac0ec040 con 0x556d376b08f0 2024-05-15T14:05:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.219+0000 7fe8c5402740 10 client.15939 fetch_fsmap learned FSMap version 29 2024-05-15T14:05:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.219+0000 7fe8c5402740 1 -- 172.21.15.33:0/2601103615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556d376ca230 con 0x556d376b08f0 2024-05-15T14:05:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.219+0000 7fe89b7fe640 1 -- 172.21.15.33:0/2601103615 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8b0004030 con 0x556d376b08f0 2024-05-15T14:05:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.219+0000 7fe89b7fe640 1 -- 172.21.15.33:0/2601103615 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fe8ac0a5660 con 0x556d376b08f0 2024-05-15T14:05:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.219+0000 7fe8c5402740 10 client.15939 fetch_fsmap finished waiting for FSMap version 29 2024-05-15T14:05:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.219+0000 7fe8c5402740 10 client.15939 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:05:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.219+0000 7fe8c5402740 20 client.15939 populate_metadata read hostname 'smithi033' 2024-05-15T14:05:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.219+0000 7fe8c5402740 1 --2- 172.21.15.33:0/2601103615 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556d376cb6a0 0x556d376cda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.219+0000 7fe8c5402740 4 client.15939 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:05:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.219+0000 7fe8c5402740 1 -- 172.21.15.33:0/2601103615 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d37625b80 con 0x556d376cb6a0 2024-05-15T14:05:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.219+0000 7fe8bb7fe640 1 --2- 172.21.15.33:0/2601103615 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556d376cb6a0 0x556d376cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.220+0000 7fe89b7fe640 10 client.15939 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:05:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.220+0000 7fe8bb7fe640 1 --2- 172.21.15.33:0/2601103615 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556d376cb6a0 0x556d376cda90 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caf59acbd93883a5 server_cookie=530992a066cddfab in_seq=0 out_seq=0 2024-05-15T14:05:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.220+0000 7fe89b7fe640 1 -- 172.21.15.33:0/2601103615 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556d37625b80 con 0x556d376cb6a0 2024-05-15T14:05:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.221+0000 7fe89b7fe640 10 client.15939 handle_command_reply: tid=0 2024-05-15T14:05:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.221+0000 7fe8c5402740 1 client.15939 shutdown 2024-05-15T14:05:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.221+0000 7fe8c5402740 1 -- 172.21.15.33:0/2601103615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d376b08f0 msgr2=0x556d376bb460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.221+0000 7fe8c5402740 1 --2- 172.21.15.33:0/2601103615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d376b08f0 0x556d376bb460 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac0e01c0 tx=0x7fe8ac0e6140 comp rx=0 tx=0).stop 2024-05-15T14:05:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.221+0000 7fe8c5402740 1 -- 172.21.15.33:0/2601103615 shutdown_connections 2024-05-15T14:05:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.221+0000 7fe8c5402740 1 --2- 172.21.15.33:0/2601103615 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556d376cb6a0 0x556d376cda90 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.221+0000 7fe8c5402740 1 --2- 172.21.15.33:0/2601103615 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d376bbad0 0x556d376b8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.221+0000 7fe8c5402740 1 --2- 172.21.15.33:0/2601103615 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556d376b1210 0x556d376b8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.221+0000 7fe89b7fe640 0 client.15939 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:05:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.221+0000 7fe8c5402740 1 --2- 172.21.15.33:0/2601103615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d376b08f0 0x556d376bb460 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.221+0000 7fe8c5402740 1 -- 172.21.15.33:0/2601103615 >> 172.21.15.33:0/2601103615 conn(0x556d376ace00 msgr2=0x556d376ac100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.221+0000 7fe8c5402740 1 -- 172.21.15.33:0/2601103615 shutdown_connections 2024-05-15T14:05:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.221+0000 7fe8c5402740 1 -- 172.21.15.33:0/2601103615 wait complete. 2024-05-15T14:05:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.221+0000 7fe8c5402740 20 client.15939 trim_cache size 0 max 16384 2024-05-15T14:05:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.237+0000 7fe8c5402740 1 Processor -- start 2024-05-15T14:05:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.237+0000 7fe8c5402740 1 -- start start 2024-05-15T14:05:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.237+0000 7fe8c5402740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d376b08f0 0x556d376c7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.237+0000 7fe8c5402740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d376b1210 0x556d376baf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.237+0000 7fe8c5402740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556d376bbad0 0x556d376d3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.238+0000 7fe8c5402740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc1b0f90 con 0x556d376b08f0 2024-05-15T14:05:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.238+0000 7fe8c5402740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc1ae860 con 0x556d376b1210 2024-05-15T14:05:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.238+0000 7fe8c5402740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc1ac6b0 con 0x556d376bbad0 2024-05-15T14:05:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.238+0000 7fe8bbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d376b08f0 0x556d376c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.238+0000 7fe8bbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d376b08f0 0x556d376c7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40372/0 (socket says 172.21.15.33:40372) 2024-05-15T14:05:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.238+0000 7fe8bbfff640 1 -- 172.21.15.33:0/2259838657 learned_addr learned my addr 172.21.15.33:0/2259838657 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.238+0000 7fe8c0c7e640 1 --2- 172.21.15.33:0/2259838657 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556d376bbad0 0x556d376d3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.238+0000 7fe8bb7fe640 1 --2- 172.21.15.33:0/2259838657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d376b1210 0x556d376baf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.238+0000 7fe8bbfff640 1 -- 172.21.15.33:0/2259838657 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556d376bbad0 msgr2=0x556d376d3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.238+0000 7fe8bbfff640 1 --2- 172.21.15.33:0/2259838657 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556d376bbad0 0x556d376d3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.238+0000 7fe8bbfff640 1 -- 172.21.15.33:0/2259838657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d376b1210 msgr2=0x556d376baf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.238+0000 7fe8bbfff640 1 --2- 172.21.15.33:0/2259838657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d376b1210 0x556d376baf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.238+0000 7fe8bbfff640 1 -- 172.21.15.33:0/2259838657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d376ce0c0 con 0x556d376b08f0 2024-05-15T14:05:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.238+0000 7fe8bb7fe640 1 --2- 172.21.15.33:0/2259838657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d376b1210 0x556d376baf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:05:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.238+0000 7fe8bbfff640 1 --2- 172.21.15.33:0/2259838657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d376b08f0 0x556d376c7610 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac0a99f0 tx=0x7fe8ac005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426b0af1798c0909 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.238+0000 7fe8a1ffb640 1 -- 172.21.15.33:0/2259838657 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac0e9070 con 0x556d376b08f0 2024-05-15T14:05:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.238+0000 7fe8a1ffb640 1 -- 172.21.15.33:0/2259838657 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8ac0c9020 con 0x556d376b08f0 2024-05-15T14:05:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.238+0000 7fe8a1ffb640 1 -- 172.21.15.33:0/2259838657 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac0aa3f0 con 0x556d376b08f0 2024-05-15T14:05:27.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.239+0000 7fe8c5402740 1 -- 172.21.15.33:0/2259838657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d376b08f0 msgr2=0x556d376c7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:27.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.239+0000 7fe8c5402740 1 --2- 172.21.15.33:0/2259838657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d376b08f0 0x556d376c7610 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac0a99f0 tx=0x7fe8ac005e30 comp rx=0 tx=0).stop 2024-05-15T14:05:27.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.239+0000 7fe8c5402740 1 -- 172.21.15.33:0/2259838657 shutdown_connections 2024-05-15T14:05:27.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.239+0000 7fe8c5402740 1 --2- 172.21.15.33:0/2259838657 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556d376bbad0 0x556d376d3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:27.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.239+0000 7fe8c5402740 1 --2- 172.21.15.33:0/2259838657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d376b1210 0x556d376baf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:27.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.239+0000 7fe8c5402740 1 --2- 172.21.15.33:0/2259838657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d376b08f0 0x556d376c7610 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:27.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.239+0000 7fe8c5402740 1 -- 172.21.15.33:0/2259838657 >> 172.21.15.33:0/2259838657 conn(0x556d376ace00 msgr2=0x556d376b3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:27.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.239+0000 7fe8c5402740 1 -- 172.21.15.33:0/2259838657 shutdown_connections 2024-05-15T14:05:27.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.239+0000 7fe8c5402740 1 -- 172.21.15.33:0/2259838657 wait complete. 2024-05-15T14:05:27.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.239+0000 7fe8c5402740 1 -- start start 2024-05-15T14:05:27.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.240+0000 7fe8c5402740 1 Processor -- start 2024-05-15T14:05:27.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.240+0000 7fe8c5402740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556d376b08f0 0x556d376c3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:27.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.240+0000 7fe8c5402740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d376b1210 0x556d376bf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:27.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.240+0000 7fe8c5402740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d376bbad0 0x556d376bf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:27.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.240+0000 7fe8c5402740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc1acd20 con 0x556d376b1210 2024-05-15T14:05:27.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.240+0000 7fe8c5402740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc1ae190 con 0x556d376bbad0 2024-05-15T14:05:27.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.240+0000 7fe8c5402740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe8bc1ad3f0 con 0x556d376b08f0 2024-05-15T14:05:27.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.240+0000 7fe8c5402740 10 client.0 _ll_register_callbacks cb 0x7fe8ac00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:05:27.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.240+0000 7fe8bb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d376b1210 0x556d376bf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:27.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.240+0000 7fe8bbfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556d376b08f0 0x556d376c3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:27.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.240+0000 7fe8bb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d376b1210 0x556d376bf0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40380/0 (socket says 172.21.15.33:40380) 2024-05-15T14:05:27.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.240+0000 7fe8bb7fe640 1 -- 172.21.15.33:0/2619108939 learned_addr learned my addr 172.21.15.33:0/2619108939 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:27.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.240+0000 7fe8c0c7e640 1 --2- 172.21.15.33:0/2619108939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d376bbad0 0x556d376bf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:27.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.240+0000 7fe8bb7fe640 1 -- 172.21.15.33:0/2619108939 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556d376b08f0 msgr2=0x556d376c3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:27.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.240+0000 7fe8bb7fe640 1 --2- 172.21.15.33:0/2619108939 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556d376b08f0 0x556d376c3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:27.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.240+0000 7fe8bb7fe640 1 -- 172.21.15.33:0/2619108939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d376bbad0 msgr2=0x556d376bf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:27.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.240+0000 7fe8bb7fe640 1 --2- 172.21.15.33:0/2619108939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d376bbad0 0x556d376bf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:27.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.240+0000 7fe8bb7fe640 1 -- 172.21.15.33:0/2619108939 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d376bfc20 con 0x556d376b1210 2024-05-15T14:05:27.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.240+0000 7fe8bb7fe640 1 --2- 172.21.15.33:0/2619108939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d376b1210 0x556d376bf0a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0009230 tx=0x7fe8b0032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fede239ad872735f server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:27.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.240+0000 7fe89b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:05:27.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.241+0000 7fe89b7fe640 1 -- 172.21.15.33:0/2619108939 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0040040 con 0x556d376b1210 2024-05-15T14:05:27.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.241+0000 7fe89b7fe640 1 -- 172.21.15.33:0/2619108939 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7fe8b00314a0 con 0x556d376b1210 2024-05-15T14:05:27.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.241+0000 7fe8c5402740 1 -- 172.21.15.33:0/2619108939 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8ac0c9020 con 0x556d376b1210 2024-05-15T14:05:27.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.241+0000 7fe89b7fe640 1 -- 172.21.15.33:0/2619108939 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe8b0035040 con 0x556d376b1210 2024-05-15T14:05:27.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.241+0000 7fe8c5402740 10 client.15945 fetch_fsmap learned FSMap version 29 2024-05-15T14:05:27.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.241+0000 7fe8c5402740 1 -- 172.21.15.33:0/2619108939 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556d37735d40 con 0x556d376b1210 2024-05-15T14:05:27.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.241+0000 7fe89b7fe640 1 -- 172.21.15.33:0/2619108939 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8ac0c9020 con 0x556d376b1210 2024-05-15T14:05:27.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.241+0000 7fe89b7fe640 1 -- 172.21.15.33:0/2619108939 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fe8b0004750 con 0x556d376b1210 2024-05-15T14:05:27.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.241+0000 7fe8c5402740 10 client.15945 fetch_fsmap finished waiting for FSMap version 29 2024-05-15T14:05:27.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.241+0000 7fe8c5402740 10 client.15945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:05:27.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.241+0000 7fe8c5402740 20 client.15945 populate_metadata read hostname 'smithi033' 2024-05-15T14:05:27.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.241+0000 7fe8c5402740 1 --2- 172.21.15.33:0/2619108939 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556d37736160 0x556d37738550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:27.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.241+0000 7fe8c5402740 4 client.15945 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:05:27.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.241+0000 7fe8c5402740 1 -- 172.21.15.33:0/2619108939 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556d3761f360 con 0x556d37736160 2024-05-15T14:05:27.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.241+0000 7fe8bbfff640 1 --2- 172.21.15.33:0/2619108939 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556d37736160 0x556d37738550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:27.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.242+0000 7fe8bbfff640 1 --2- 172.21.15.33:0/2619108939 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556d37736160 0x556d37738550 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd33e2243dbbe1e9 server_cookie=928a4f85a2aeaef2 in_seq=0 out_seq=0 2024-05-15T14:05:27.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.242+0000 7fe89b7fe640 10 client.15945 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:05:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.311+0000 7fe89b7fe640 1 -- 172.21.15.33:0/2619108939 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556d3761f360 con 0x556d37736160 2024-05-15T14:05:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.311+0000 7fe89b7fe640 10 client.15945 handle_command_reply: tid=0 2024-05-15T14:05:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.311+0000 7fe8c5402740 1 client.15945 shutdown 2024-05-15T14:05:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.312+0000 7fe8c5402740 1 -- 172.21.15.33:0/2619108939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d376b1210 msgr2=0x556d376bf0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.312+0000 7fe8c5402740 1 --2- 172.21.15.33:0/2619108939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d376b1210 0x556d376bf0a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0009230 tx=0x7fe8b0032b90 comp rx=0 tx=0).stop 2024-05-15T14:05:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.312+0000 7fe8c5402740 1 -- 172.21.15.33:0/2619108939 shutdown_connections 2024-05-15T14:05:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.312+0000 7fe8c5402740 1 --2- 172.21.15.33:0/2619108939 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556d37736160 0x556d37738550 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.312+0000 7fe8c5402740 1 --2- 172.21.15.33:0/2619108939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556d376bbad0 0x556d376bf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.312+0000 7fe89b7fe640 0 client.15945 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:05:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.312+0000 7fe8c5402740 1 --2- 172.21.15.33:0/2619108939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d376b1210 0x556d376bf0a0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.312+0000 7fe8c5402740 1 --2- 172.21.15.33:0/2619108939 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556d376b08f0 0x556d376c3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.312+0000 7fe8c5402740 1 -- 172.21.15.33:0/2619108939 >> 172.21.15.33:0/2619108939 conn(0x556d376ace00 msgr2=0x556d376c6600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.312+0000 7fe8c5402740 1 -- 172.21.15.33:0/2619108939 shutdown_connections 2024-05-15T14:05:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.312+0000 7fe8c5402740 1 -- 172.21.15.33:0/2619108939 wait complete. 2024-05-15T14:05:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.312+0000 7fe8c5402740 20 client.15945 trim_cache size 0 max 16384 2024-05-15T14:05:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:05:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:05:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:05:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.312+0000 7fe8c26e1640 1 -- 172.21.15.33:0/3141402163 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fe8a40b2950 msgr2=0x7fe8a40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.312+0000 7fe8c26e1640 1 --2- 172.21.15.33:0/3141402163 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fe8a40b2950 0x7fe8a40b4e10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4009fd0 tx=0x7fe8b4009500 comp rx=0 tx=0).stop 2024-05-15T14:05:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.312+0000 7fe8c26e1640 1 -- 172.21.15.33:0/3141402163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8bc103ce0 msgr2=0x7fe8bc199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.312+0000 7fe8c26e1640 1 --2- 172.21.15.33:0/3141402163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8bc103ce0 0x7fe8bc199290 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00e9f0 tx=0x7fe8ac00edf0 comp rx=0 tx=0).stop 2024-05-15T14:05:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.313+0000 7fe8c26e1640 1 -- 172.21.15.33:0/3141402163 shutdown_connections 2024-05-15T14:05:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.313+0000 7fe8c26e1640 1 --2- 172.21.15.33:0/3141402163 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fe8a40b2950 0x7fe8a40b4e10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.313+0000 7fe8c26e1640 1 --2- 172.21.15.33:0/3141402163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe8bc10d0b0 0x7fe8bc19db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.313+0000 7fe8c26e1640 1 --2- 172.21.15.33:0/3141402163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe8bc104600 0x7fe8bc1997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.313+0000 7fe8c26e1640 1 --2- 172.21.15.33:0/3141402163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8bc103ce0 0x7fe8bc199290 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.313+0000 7fe8c26e1640 1 -- 172.21.15.33:0/3141402163 >> 172.21.15.33:0/3141402163 conn(0x7fe8bc078aa0 msgr2=0x7fe8bc0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.313+0000 7fe8c26e1640 1 -- 172.21.15.33:0/3141402163 shutdown_connections 2024-05-15T14:05:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:27.313+0000 7fe8c26e1640 1 -- 172.21.15.33:0/3141402163 wait complete. 2024-05-15T14:05:28.056 INFO:tasks.workunit.client.0.smithi033.stdout: 1 307483 7.97 MB/sec execute 63 sec latency 70.394 ms 2024-05-15T14:05:28.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:28 smithi052 ceph-mon[36190]: pgmap v157: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 373 op/s 2024-05-15T14:05:28.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:28 smithi081 ceph-mon[36434]: pgmap v157: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 373 op/s 2024-05-15T14:05:28.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:28 smithi033 ceph-mon[31650]: pgmap v157: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 373 op/s 2024-05-15T14:05:29.056 INFO:tasks.workunit.client.0.smithi033.stdout: 1 309232 8.04 MB/sec execute 64 sec latency 34.974 ms 2024-05-15T14:05:30.056 INFO:tasks.workunit.client.0.smithi033.stdout: 1 311971 8.14 MB/sec execute 65 sec latency 17.153 ms 2024-05-15T14:05:30.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:30 smithi052 ceph-mon[36190]: pgmap v158: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 239 op/s 2024-05-15T14:05:30.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:30 smithi081 ceph-mon[36434]: pgmap v158: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 239 op/s 2024-05-15T14:05:30.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:30 smithi033 ceph-mon[31650]: pgmap v158: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 239 op/s 2024-05-15T14:05:31.056 INFO:tasks.workunit.client.0.smithi033.stdout: 1 313565 8.16 MB/sec execute 66 sec latency 18.414 ms 2024-05-15T14:05:32.056 INFO:tasks.workunit.client.0.smithi033.stdout: 1 316111 8.25 MB/sec execute 67 sec latency 11.782 ms 2024-05-15T14:05:32.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:32 smithi052 ceph-mon[36190]: pgmap v159: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 239 op/s 2024-05-15T14:05:32.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:32 smithi081 ceph-mon[36434]: pgmap v159: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 239 op/s 2024-05-15T14:05:32.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:32 smithi033 ceph-mon[31650]: pgmap v159: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 239 op/s 2024-05-15T14:05:33.056 INFO:tasks.workunit.client.0.smithi033.stdout: 1 318129 8.23 MB/sec execute 68 sec latency 28.906 ms 2024-05-15T14:05:33.975 INFO:tasks.quiescer.fs.[cephfs]:Total duration (36.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-15T14:05:33.975 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.35 2024-05-15T14:05:33.975 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T14:05:34.056 INFO:tasks.workunit.client.0.smithi033.stdout: 1 319851 8.28 MB/sec execute 69 sec latency 19.942 ms 2024-05-15T14:05:34.278 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:05:34.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:34 smithi033 ceph-mon[31650]: pgmap v160: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 406 op/s 2024-05-15T14:05:34.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:34 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:05:34.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:34 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:05:34.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:34 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:05:34.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:34 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:05:34.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:34 smithi052 ceph-mon[36190]: pgmap v160: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 406 op/s 2024-05-15T14:05:34.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:34 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:05:34.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:34 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:05:34.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:34 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:05:34.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:34 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:05:34.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:34 smithi081 ceph-mon[36434]: pgmap v160: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 406 op/s 2024-05-15T14:05:34.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:34 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:05:34.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:34 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:05:34.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:34 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:05:34.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:34 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:05:35.058 INFO:tasks.workunit.client.0.smithi033.stdout: 1 322441 8.36 MB/sec execute 70 sec latency 15.240 ms 2024-05-15T14:05:35.763 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:35 smithi033 ceph-mon[31650]: pgmap v161: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 292 op/s 2024-05-15T14:05:35.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:35 smithi052 ceph-mon[36190]: pgmap v161: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 292 op/s 2024-05-15T14:05:35.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:35 smithi081 ceph-mon[36434]: pgmap v161: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 292 op/s 2024-05-15T14:05:36.056 INFO:tasks.workunit.client.0.smithi033.stdout: 1 323853 8.35 MB/sec execute 71 sec latency 16.881 ms 2024-05-15T14:05:36.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.146+0000 7f471ce9d700 1 -- 172.21.15.33:0/3523421521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47181035e0 msgr2=0x7f4718103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:36.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.146+0000 7f471ce9d700 1 --2- 172.21.15.33:0/3523421521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47181035e0 0x7f4718103a60 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f470c0099c0 tx=0x7f470c009cd0 comp rx=0 tx=0).stop 2024-05-15T14:05:36.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.146+0000 7f471ce9d700 1 -- 172.21.15.33:0/3523421521 shutdown_connections 2024-05-15T14:05:36.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.146+0000 7f471ce9d700 1 --2- 172.21.15.33:0/3523421521 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4718103fa0 0x7f4718110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:36.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.146+0000 7f471ce9d700 1 --2- 172.21.15.33:0/3523421521 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4718109680 0x7f4718109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:36.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.146+0000 7f471ce9d700 1 --2- 172.21.15.33:0/3523421521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47181035e0 0x7f4718103a60 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:36.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.146+0000 7f471ce9d700 1 -- 172.21.15.33:0/3523421521 >> 172.21.15.33:0/3523421521 conn(0x7f471807a6f0 msgr2=0x7f471807ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:36.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.146+0000 7f471ce9d700 1 -- 172.21.15.33:0/3523421521 shutdown_connections 2024-05-15T14:05:36.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.146+0000 7f471ce9d700 1 -- 172.21.15.33:0/3523421521 wait complete. 2024-05-15T14:05:36.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.147+0000 7f471ce9d700 1 Processor -- start 2024-05-15T14:05:36.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.147+0000 7f471ce9d700 1 -- start start 2024-05-15T14:05:36.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.147+0000 7f471ce9d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47181035e0 0x7f4718075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:36.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.147+0000 7f471ce9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4718103fa0 0x7f4718075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:36.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.147+0000 7f471ce9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4718109680 0x7f47180762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:36.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.147+0000 7f471ce9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f47181137e0 con 0x7f4718109680 2024-05-15T14:05:36.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.147+0000 7f471ce9d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4718113660 con 0x7f4718103fa0 2024-05-15T14:05:36.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.147+0000 7f471ce9d700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4718113960 con 0x7f47181035e0 2024-05-15T14:05:36.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.147+0000 7f471659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47181035e0 0x7f4718075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:36.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.147+0000 7f471659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47181035e0 0x7f4718075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:55030/0 (socket says 172.21.15.33:55030) 2024-05-15T14:05:36.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.147+0000 7f471659c700 1 -- 172.21.15.33:0/2734226137 learned_addr learned my addr 172.21.15.33:0/2734226137 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:36.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.147+0000 7f4715d9b700 1 --2- 172.21.15.33:0/2734226137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4718103fa0 0x7f4718075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:36.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.147+0000 7f4716d9d700 1 --2- 172.21.15.33:0/2734226137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4718109680 0x7f47180762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:36.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.147+0000 7f471659c700 1 -- 172.21.15.33:0/2734226137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4718103fa0 msgr2=0x7f4718075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:36.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.147+0000 7f471659c700 1 --2- 172.21.15.33:0/2734226137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4718103fa0 0x7f4718075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:36.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.147+0000 7f471659c700 1 -- 172.21.15.33:0/2734226137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4718109680 msgr2=0x7f47180762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:36.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.148+0000 7f471659c700 1 --2- 172.21.15.33:0/2734226137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4718109680 0x7f47180762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:36.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.148+0000 7f471659c700 1 -- 172.21.15.33:0/2734226137 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47181a80f0 con 0x7f47181035e0 2024-05-15T14:05:36.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.148+0000 7f4715d9b700 1 --2- 172.21.15.33:0/2734226137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4718103fa0 0x7f4718075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:05:36.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.148+0000 7f4716d9d700 1 --2- 172.21.15.33:0/2734226137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4718109680 0x7f47180762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:05:36.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.148+0000 7f471659c700 1 --2- 172.21.15.33:0/2734226137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47181035e0 0x7f4718075840 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f470800e9a0 tx=0x7f470800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:36.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.148+0000 7f47077fe700 1 -- 172.21.15.33:0/2734226137 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470800ca10 con 0x7f47181035e0 2024-05-15T14:05:36.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.148+0000 7f47077fe700 1 -- 172.21.15.33:0/2734226137 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f470800cbb0 con 0x7f47181035e0 2024-05-15T14:05:36.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.148+0000 7f47077fe700 1 -- 172.21.15.33:0/2734226137 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4708018680 con 0x7f47181035e0 2024-05-15T14:05:36.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.148+0000 7f471ce9d700 1 -- 172.21.15.33:0/2734226137 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47181a83e0 con 0x7f47181035e0 2024-05-15T14:05:36.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.148+0000 7f471ce9d700 1 -- 172.21.15.33:0/2734226137 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47181a88f0 con 0x7f47181035e0 2024-05-15T14:05:36.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.149+0000 7f471ce9d700 1 -- 172.21.15.33:0/2734226137 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4718104cf0 con 0x7f47181035e0 2024-05-15T14:05:36.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.150+0000 7f47077fe700 1 -- 172.21.15.33:0/2734226137 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4708010430 con 0x7f47181035e0 2024-05-15T14:05:36.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.151+0000 7f47077fe700 1 --2- 172.21.15.33:0/2734226137 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f47000b29f0 0x7f47000b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:36.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.151+0000 7f4715d9b700 1 --2- 172.21.15.33:0/2734226137 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f47000b29f0 0x7f47000b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:36.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.151+0000 7f47077fe700 1 -- 172.21.15.33:0/2734226137 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f47080e2ae0 con 0x7f47181035e0 2024-05-15T14:05:36.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.152+0000 7f4715d9b700 1 --2- 172.21.15.33:0/2734226137 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f47000b29f0 0x7f47000b4eb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4718077280 tx=0x7f470c005fb0 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:36.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.156+0000 7f47077fe700 1 -- 172.21.15.33:0/2734226137 <== mon.1 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f47080aa470 con 0x7f47181035e0 2024-05-15T14:05:36.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.446+0000 7f471ce9d700 1 -- 172.21.15.33:0/2734226137 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f471802e560 con 0x7f47181035e0 2024-05-15T14:05:36.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.447+0000 7f47077fe700 1 -- 172.21.15.33:0/2734226137 <== mon.1 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11855 (secure 0 0 0) 0x7f47080a9d70 con 0x7f47181035e0 2024-05-15T14:05:36.449 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T14:05:36.449 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2030117633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2030117633},{"type":"v1","addr":"172.21.15.33:6839","nonce":2030117633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1918979061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1918979061},{"type":"v1","addr":"172.21.15.33:6841","nonce":1918979061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T13:57:19.056417+0000","modified":"2024-05-15T14:05:13.682039+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24463,"mds_2":24461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/1629697841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1629697841},{"type":"v1","addr":"172.21.15.52:6837","nonce":1629697841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.52:6839/2696044471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2696044471},{"type":"v1","addr":"172.21.15.52:6839","nonce":2696044471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.81:6839/1554395940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":1554395940},{"type":"v1","addr":"172.21.15.81:6839","nonce":1554395940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25393":{"gid":25393,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/727032511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":727032511},{"type":"v1","addr":"172.21.15.81:6835","nonce":727032511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24478,24461,24463]},"id":1}]} 2024-05-15T14:05:36.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.449+0000 7f471ce9d700 1 -- 172.21.15.33:0/2734226137 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f47000b29f0 msgr2=0x7f47000b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:36.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.449+0000 7f471ce9d700 1 --2- 172.21.15.33:0/2734226137 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f47000b29f0 0x7f47000b4eb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4718077280 tx=0x7f470c005fb0 comp rx=0 tx=0).stop 2024-05-15T14:05:36.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.449+0000 7f471ce9d700 1 -- 172.21.15.33:0/2734226137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47181035e0 msgr2=0x7f4718075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:36.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.449+0000 7f471ce9d700 1 --2- 172.21.15.33:0/2734226137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47181035e0 0x7f4718075840 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f470800e9a0 tx=0x7f470800ecb0 comp rx=0 tx=0).stop 2024-05-15T14:05:36.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.449+0000 7f471ce9d700 1 -- 172.21.15.33:0/2734226137 shutdown_connections 2024-05-15T14:05:36.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.449+0000 7f471ce9d700 1 --2- 172.21.15.33:0/2734226137 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f47000b29f0 0x7f47000b4eb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:36.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.449+0000 7f471ce9d700 1 --2- 172.21.15.33:0/2734226137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4718109680 0x7f47180762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:36.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.449+0000 7f471ce9d700 1 --2- 172.21.15.33:0/2734226137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47181035e0 0x7f4718075840 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:36.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.449+0000 7f471ce9d700 1 --2- 172.21.15.33:0/2734226137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4718103fa0 0x7f4718075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:36.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.449+0000 7f471ce9d700 1 -- 172.21.15.33:0/2734226137 >> 172.21.15.33:0/2734226137 conn(0x7f471807a6f0 msgr2=0x7f4718102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:36.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.449+0000 7f471ce9d700 1 -- 172.21.15.33:0/2734226137 shutdown_connections 2024-05-15T14:05:36.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:36.449+0000 7f471ce9d700 1 -- 172.21.15.33:0/2734226137 wait complete. 2024-05-15T14:05:36.452 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 29 2024-05-15T14:05:36.766 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:36 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/2734226137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:05:36.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:36 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/2734226137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:05:36.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:36 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/2734226137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:05:37.057 INFO:tasks.workunit.client.0.smithi033.stdout: 1 326600 8.47 MB/sec execute 72 sec latency 15.692 ms 2024-05-15T14:05:37.362 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24478 quiesce db / --timeout 90 --expiration 138.35 --await-for 45.0' 2024-05-15T14:05:37.362 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24478 quiesce db / --timeout 90 --expiration 138.35 --await-for 45.0 2024-05-15T14:05:37.647 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:05:37.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:37 smithi033 ceph-mon[31650]: pgmap v162: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 408 op/s 2024-05-15T14:05:37.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:37 smithi052 ceph-mon[36190]: pgmap v162: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 408 op/s 2024-05-15T14:05:37.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:37 smithi081 ceph-mon[36434]: pgmap v162: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 408 op/s 2024-05-15T14:05:38.057 INFO:tasks.workunit.client.0.smithi033.stdout: 1 328184 8.43 MB/sec execute 73 sec latency 15.010 ms 2024-05-15T14:05:39.057 INFO:tasks.workunit.client.0.smithi033.stdout: 1 330285 8.50 MB/sec execute 74 sec latency 60.260 ms 2024-05-15T14:05:39.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.840+0000 7f24436cb700 1 -- 172.21.15.33:0/1891410022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f243c06bb10 msgr2=0x7f243c06bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:39.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.840+0000 7f24436cb700 1 --2- 172.21.15.33:0/1891410022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f243c06bb10 0x7f243c06bf90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f242c009990 tx=0x7f242c009ca0 comp rx=0 tx=0).stop 2024-05-15T14:05:39.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.840+0000 7f24436cb700 1 -- 172.21.15.33:0/1891410022 shutdown_connections 2024-05-15T14:05:39.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.840+0000 7f24436cb700 1 --2- 172.21.15.33:0/1891410022 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f243c06c4d0 0x7f243c1103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:39.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.840+0000 7f24436cb700 1 --2- 172.21.15.33:0/1891410022 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f243c1095d0 0x7f243c1099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:39.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.840+0000 7f24436cb700 1 --2- 172.21.15.33:0/1891410022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f243c06bb10 0x7f243c06bf90 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:39.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.840+0000 7f24436cb700 1 -- 172.21.15.33:0/1891410022 >> 172.21.15.33:0/1891410022 conn(0x7f243c079480 msgr2=0x7f243c079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:39.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.840+0000 7f24436cb700 1 -- 172.21.15.33:0/1891410022 shutdown_connections 2024-05-15T14:05:39.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.840+0000 7f24436cb700 1 -- 172.21.15.33:0/1891410022 wait complete. 2024-05-15T14:05:39.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.841+0000 7f24436cb700 1 Processor -- start 2024-05-15T14:05:39.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.841+0000 7f24436cb700 1 -- start start 2024-05-15T14:05:39.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.841+0000 7f24436cb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f243c06bb10 0x7f243c19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:39.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.841+0000 7f24436cb700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f243c06c4d0 0x7f243c19fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:39.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.841+0000 7f24436cb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f243c1095d0 0x7f243c1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:39.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.841+0000 7f24436cb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f243c113700 con 0x7f243c1095d0 2024-05-15T14:05:39.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.841+0000 7f24436cb700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f243c113580 con 0x7f243c06bb10 2024-05-15T14:05:39.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.841+0000 7f24436cb700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f243c113880 con 0x7f243c06c4d0 2024-05-15T14:05:39.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.841+0000 7f2440c66700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f243c06c4d0 0x7f243c19fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:39.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.841+0000 7f2441467700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f243c06bb10 0x7f243c19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:39.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.841+0000 7f2440c66700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f243c06c4d0 0x7f243c19fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:55048/0 (socket says 172.21.15.33:55048) 2024-05-15T14:05:39.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.841+0000 7f2440c66700 1 -- 172.21.15.33:0/1738192422 learned_addr learned my addr 172.21.15.33:0/1738192422 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:39.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.841+0000 7f2441467700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f243c06bb10 0x7f243c19f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:53708/0 (socket says 172.21.15.33:53708) 2024-05-15T14:05:39.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.841+0000 7f2441c68700 1 --2- 172.21.15.33:0/1738192422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f243c1095d0 0x7f243c1999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:39.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.842+0000 7f2440c66700 1 -- 172.21.15.33:0/1738192422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f243c06bb10 msgr2=0x7f243c19f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:39.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.842+0000 7f2440c66700 1 --2- 172.21.15.33:0/1738192422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f243c06bb10 0x7f243c19f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:39.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.842+0000 7f2440c66700 1 -- 172.21.15.33:0/1738192422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f243c1095d0 msgr2=0x7f243c1999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:39.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.842+0000 7f2440c66700 1 --2- 172.21.15.33:0/1738192422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f243c1095d0 0x7f243c1999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:39.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.842+0000 7f2440c66700 1 -- 172.21.15.33:0/1738192422 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f243c19a140 con 0x7f243c06c4d0 2024-05-15T14:05:39.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.842+0000 7f2441467700 1 --2- 172.21.15.33:0/1738192422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f243c06bb10 0x7f243c19f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:05:39.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.842+0000 7f2440c66700 1 --2- 172.21.15.33:0/1738192422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f243c06c4d0 0x7f243c19fd60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f242c000c00 tx=0x7f242c00bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:39.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.842+0000 7f24327fc700 1 -- 172.21.15.33:0/1738192422 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c027070 con 0x7f243c06c4d0 2024-05-15T14:05:39.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.842+0000 7f24436cb700 1 -- 172.21.15.33:0/1738192422 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f243c19a3d0 con 0x7f243c06c4d0 2024-05-15T14:05:39.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.842+0000 7f24327fc700 1 -- 172.21.15.33:0/1738192422 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f242c022470 con 0x7f243c06c4d0 2024-05-15T14:05:39.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.842+0000 7f24327fc700 1 -- 172.21.15.33:0/1738192422 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c017650 con 0x7f243c06c4d0 2024-05-15T14:05:39.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.842+0000 7f24436cb700 1 -- 172.21.15.33:0/1738192422 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f243c1a66f0 con 0x7f243c06c4d0 2024-05-15T14:05:39.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.845+0000 7f24327fc700 1 -- 172.21.15.33:0/1738192422 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f242c02a070 con 0x7f243c06c4d0 2024-05-15T14:05:39.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.846+0000 7f24327fc700 1 --2- 172.21.15.33:0/1738192422 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f24280b2b20 0x7f24280b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:39.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.846+0000 7f2441467700 1 --2- 172.21.15.33:0/1738192422 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f24280b2b20 0x7f24280b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:39.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.846+0000 7f24327fc700 1 -- 172.21.15.33:0/1738192422 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f242c0e31f0 con 0x7f243c06c4d0 2024-05-15T14:05:39.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.846+0000 7f2441467700 1 --2- 172.21.15.33:0/1738192422 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f24280b2b20 0x7f24280b4fe0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f243c19ab20 tx=0x7f2438009260 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:39.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.847+0000 7f244df61b80 1 Processor -- start 2024-05-15T14:05:39.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.847+0000 7f244df61b80 1 -- start start 2024-05-15T14:05:39.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.847+0000 7f244df61b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56389c1f84c0 0x56389c1f88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:39.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.847+0000 7f244df61b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56389c1f8de0 0x56389c1f3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:39.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.847+0000 7f244df61b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56389c1f4360 0x56389c1f47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:39.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.847+0000 7f244df61b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56389c151f10 con 0x56389c1f8de0 2024-05-15T14:05:39.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.847+0000 7f244df61b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56389c141ab0 con 0x56389c1f4360 2024-05-15T14:05:39.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.847+0000 7f244df61b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56389c145af0 con 0x56389c1f84c0 2024-05-15T14:05:39.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.847+0000 7f2440c66700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56389c1f8de0 0x56389c1f3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:39.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.847+0000 7f2440c66700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56389c1f8de0 0x56389c1f3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37228/0 (socket says 172.21.15.33:37228) 2024-05-15T14:05:39.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.847+0000 7f2441467700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56389c1f84c0 0x56389c1f88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:39.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.847+0000 7f2441c68700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56389c1f4360 0x56389c1f47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:39.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.847+0000 7f2440c66700 1 -- 172.21.15.33:0/2203614938 learned_addr learned my addr 172.21.15.33:0/2203614938 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:39.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.847+0000 7f2441467700 1 -- 172.21.15.33:0/2203614938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56389c1f4360 msgr2=0x56389c1f47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:39.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.847+0000 7f2441467700 1 --2- 172.21.15.33:0/2203614938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56389c1f4360 0x56389c1f47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:39.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.847+0000 7f2441467700 1 -- 172.21.15.33:0/2203614938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56389c1f8de0 msgr2=0x56389c1f3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:39.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.847+0000 7f2441467700 1 --2- 172.21.15.33:0/2203614938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56389c1f8de0 0x56389c1f3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:39.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.847+0000 7f2441467700 1 -- 172.21.15.33:0/2203614938 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56389c0eb150 con 0x56389c1f84c0 2024-05-15T14:05:39.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.847+0000 7f2441c68700 1 --2- 172.21.15.33:0/2203614938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56389c1f4360 0x56389c1f47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:05:39.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.847+0000 7f2440c66700 1 --2- 172.21.15.33:0/2203614938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56389c1f8de0 0x56389c1f3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:05:39.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.848+0000 7f2441467700 1 --2- 172.21.15.33:0/2203614938 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56389c1f84c0 0x56389c1f88a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2438004300 tx=0x7f2438014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15d1455fe4fe85c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:39.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.848+0000 7f24277fe700 1 -- 172.21.15.33:0/2203614938 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2438003d10 con 0x56389c1f84c0 2024-05-15T14:05:39.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.848+0000 7f24277fe700 1 -- 172.21.15.33:0/2203614938 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f243801e8d0 con 0x56389c1f84c0 2024-05-15T14:05:39.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.848+0000 7f24277fe700 1 -- 172.21.15.33:0/2203614938 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2438026780 con 0x56389c1f84c0 2024-05-15T14:05:39.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.849+0000 7f244df61b80 1 -- 172.21.15.33:0/2203614938 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56389c1f84c0 msgr2=0x56389c1f88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:39.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.849+0000 7f244df61b80 1 --2- 172.21.15.33:0/2203614938 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56389c1f84c0 0x56389c1f88a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2438004300 tx=0x7f2438014000 comp rx=0 tx=0).stop 2024-05-15T14:05:39.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.849+0000 7f244df61b80 1 -- 172.21.15.33:0/2203614938 shutdown_connections 2024-05-15T14:05:39.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.849+0000 7f244df61b80 1 --2- 172.21.15.33:0/2203614938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56389c1f4360 0x56389c1f47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:39.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.849+0000 7f244df61b80 1 --2- 172.21.15.33:0/2203614938 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56389c1f84c0 0x56389c1f88a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:39.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.849+0000 7f244df61b80 1 --2- 172.21.15.33:0/2203614938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56389c1f8de0 0x56389c1f3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:39.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.849+0000 7f244df61b80 1 -- 172.21.15.33:0/2203614938 >> 172.21.15.33:0/2203614938 conn(0x56389c120f50 msgr2=0x56389c1ee540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:39.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.849+0000 7f244df61b80 1 -- 172.21.15.33:0/2203614938 shutdown_connections 2024-05-15T14:05:39.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.849+0000 7f244df61b80 1 -- 172.21.15.33:0/2203614938 wait complete. 2024-05-15T14:05:39.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.849+0000 7f244df61b80 1 -- start start 2024-05-15T14:05:39.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.849+0000 7f244df61b80 1 Processor -- start 2024-05-15T14:05:39.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.850+0000 7f244df61b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56389c1f4360 0x56389c1ffc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:39.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.850+0000 7f244df61b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56389c1f84c0 0x56389c1fcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:39.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.850+0000 7f244df61b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56389c1f8de0 0x56389c1fd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:39.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.850+0000 7f244df61b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56389c048140 con 0x56389c1f4360 2024-05-15T14:05:39.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.850+0000 7f244df61b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56389c0a9f40 con 0x56389c1f84c0 2024-05-15T14:05:39.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.850+0000 7f244df61b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56389c204d30 con 0x56389c1f8de0 2024-05-15T14:05:39.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.850+0000 7f244df61b80 10 client.0 _ll_register_callbacks cb 0x56389c0dcaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:05:39.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.850+0000 7f2441467700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56389c1f4360 0x56389c1ffc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:39.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.850+0000 7f2441467700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56389c1f4360 0x56389c1ffc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37240/0 (socket says 172.21.15.33:37240) 2024-05-15T14:05:39.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.850+0000 7f2441467700 1 -- 172.21.15.33:0/2049691981 learned_addr learned my addr 172.21.15.33:0/2049691981 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:39.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.850+0000 7f2441c68700 1 --2- 172.21.15.33:0/2049691981 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56389c1f8de0 0x56389c1fd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:39.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.850+0000 7f2440c66700 1 --2- 172.21.15.33:0/2049691981 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56389c1f84c0 0x56389c1fcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:39.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.850+0000 7f2441467700 1 -- 172.21.15.33:0/2049691981 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56389c1f8de0 msgr2=0x56389c1fd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:39.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.850+0000 7f2441467700 1 --2- 172.21.15.33:0/2049691981 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56389c1f8de0 0x56389c1fd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:39.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.850+0000 7f2441467700 1 -- 172.21.15.33:0/2049691981 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56389c1f84c0 msgr2=0x56389c1fcfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:39.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.850+0000 7f2441467700 1 --2- 172.21.15.33:0/2049691981 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56389c1f84c0 0x56389c1fcfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:39.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.850+0000 7f2441467700 1 -- 172.21.15.33:0/2049691981 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56389c091a70 con 0x56389c1f4360 2024-05-15T14:05:39.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.851+0000 7f2441467700 1 --2- 172.21.15.33:0/2049691981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56389c1f4360 0x56389c1ffc10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f243801e6c0 tx=0x7f243801e6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a355ef1c3055f9e server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:39.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.851+0000 7f2424ff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:05:39.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.851+0000 7f2424ff9700 1 -- 172.21.15.33:0/2049691981 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243801ec90 con 0x56389c1f4360 2024-05-15T14:05:39.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.851+0000 7f2424ff9700 1 -- 172.21.15.33:0/2049691981 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f2438005040 con 0x56389c1f4360 2024-05-15T14:05:39.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.851+0000 7f244df61b80 1 -- 172.21.15.33:0/2049691981 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f243801e8d0 con 0x56389c1f4360 2024-05-15T14:05:39.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.851+0000 7f2424ff9700 1 -- 172.21.15.33:0/2049691981 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2438005240 con 0x56389c1f4360 2024-05-15T14:05:39.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.851+0000 7f244df61b80 10 client.15963 fetch_fsmap learned FSMap version 29 2024-05-15T14:05:39.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.851+0000 7f244df61b80 1 -- 172.21.15.33:0/2049691981 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56389c0fb900 con 0x56389c1f4360 2024-05-15T14:05:39.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.851+0000 7f2424ff9700 1 -- 172.21.15.33:0/2049691981 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f243801e8d0 con 0x56389c1f4360 2024-05-15T14:05:39.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.851+0000 7f2424ff9700 1 -- 172.21.15.33:0/2049691981 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f24380334d0 con 0x56389c1f4360 2024-05-15T14:05:39.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.851+0000 7f244df61b80 10 client.15963 fetch_fsmap finished waiting for FSMap version 29 2024-05-15T14:05:39.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.851+0000 7f244df61b80 10 client.15963 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:05:39.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.852+0000 7f244df61b80 20 client.15963 populate_metadata read hostname 'smithi033' 2024-05-15T14:05:39.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.852+0000 7f244df61b80 1 --2- 172.21.15.33:0/2049691981 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56389c211790 0x56389c213b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:39.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.852+0000 7f244df61b80 4 client.15963 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:05:39.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.852+0000 7f244df61b80 1 -- 172.21.15.33:0/2049691981 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56389c153e60 con 0x56389c211790 2024-05-15T14:05:39.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.852+0000 7f2440c66700 1 --2- 172.21.15.33:0/2049691981 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56389c211790 0x56389c213b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:39.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.852+0000 7f2440c66700 1 --2- 172.21.15.33:0/2049691981 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56389c211790 0x56389c213b80 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7168a0aed279bc29 server_cookie=ce567a61e5a2ea78 in_seq=0 out_seq=0 2024-05-15T14:05:39.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.852+0000 7f2424ff9700 10 client.15963 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:05:39.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.853+0000 7f2424ff9700 1 -- 172.21.15.33:0/2049691981 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56389c153e60 con 0x56389c211790 2024-05-15T14:05:39.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.853+0000 7f2424ff9700 10 client.15963 handle_command_reply: tid=0 2024-05-15T14:05:39.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.853+0000 7f244df61b80 1 client.15963 shutdown 2024-05-15T14:05:39.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.853+0000 7f244df61b80 1 -- 172.21.15.33:0/2049691981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56389c1f4360 msgr2=0x56389c1ffc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:39.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.853+0000 7f244df61b80 1 --2- 172.21.15.33:0/2049691981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56389c1f4360 0x56389c1ffc10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f243801e6c0 tx=0x7f243801e6f0 comp rx=0 tx=0).stop 2024-05-15T14:05:39.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.853+0000 7f244df61b80 1 -- 172.21.15.33:0/2049691981 shutdown_connections 2024-05-15T14:05:39.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.853+0000 7f244df61b80 1 --2- 172.21.15.33:0/2049691981 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56389c1f8de0 0x56389c1fd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:39.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.853+0000 7f244df61b80 1 --2- 172.21.15.33:0/2049691981 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56389c211790 0x56389c213b80 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:39.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.853+0000 7f244df61b80 1 --2- 172.21.15.33:0/2049691981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56389c1f4360 0x56389c1ffc10 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:39.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.853+0000 7f244df61b80 1 --2- 172.21.15.33:0/2049691981 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56389c1f84c0 0x56389c1fcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:39.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.853+0000 7f244df61b80 1 -- 172.21.15.33:0/2049691981 >> 172.21.15.33:0/2049691981 conn(0x56389c120f50 msgr2=0x56389c1e9060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:39.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.853+0000 7f2424ff9700 0 client.15963 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:05:39.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.853+0000 7f244df61b80 1 -- 172.21.15.33:0/2049691981 shutdown_connections 2024-05-15T14:05:39.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.853+0000 7f244df61b80 1 -- 172.21.15.33:0/2049691981 wait complete. 2024-05-15T14:05:39.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.854+0000 7f244df61b80 20 client.15963 trim_cache size 0 max 16384 2024-05-15T14:05:39.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.876+0000 7f244df61b80 1 Processor -- start 2024-05-15T14:05:39.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.876+0000 7f244df61b80 1 -- start start 2024-05-15T14:05:39.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.877+0000 7f244df61b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56389c1f4360 0x56389c1e9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:39.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.877+0000 7f244df61b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56389c1f84c0 0x56389c1fa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:39.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.877+0000 7f244df61b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56389c1f8de0 0x56389c1fa7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:39.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.877+0000 7f244df61b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f243c1ae370 con 0x56389c1f8de0 2024-05-15T14:05:39.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.877+0000 7f244df61b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f243c1abc00 con 0x56389c1f4360 2024-05-15T14:05:39.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.877+0000 7f244df61b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f243c1a9a50 con 0x56389c1f84c0 2024-05-15T14:05:39.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.877+0000 7f2441c68700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56389c1f8de0 0x56389c1fa7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:39.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.877+0000 7f2440c66700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56389c1f84c0 0x56389c1fa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:39.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.877+0000 7f2441467700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56389c1f4360 0x56389c1e9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:39.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.877+0000 7f2440c66700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56389c1f84c0 0x56389c1fa2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:55080/0 (socket says 172.21.15.33:55080) 2024-05-15T14:05:39.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.877+0000 7f2440c66700 1 -- 172.21.15.33:0/317887070 learned_addr learned my addr 172.21.15.33:0/317887070 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:39.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.877+0000 7f2440c66700 1 -- 172.21.15.33:0/317887070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56389c1f4360 msgr2=0x56389c1e9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:39.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.877+0000 7f2440c66700 1 --2- 172.21.15.33:0/317887070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56389c1f4360 0x56389c1e9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:39.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.877+0000 7f2440c66700 1 -- 172.21.15.33:0/317887070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56389c1f8de0 msgr2=0x56389c1fa7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:39.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.877+0000 7f2440c66700 1 --2- 172.21.15.33:0/317887070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56389c1f8de0 0x56389c1fa7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:39.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.877+0000 7f2440c66700 1 -- 172.21.15.33:0/317887070 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56389c1e96b0 con 0x56389c1f84c0 2024-05-15T14:05:39.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.877+0000 7f2441467700 1 --2- 172.21.15.33:0/317887070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56389c1f4360 0x56389c1e9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:05:39.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.877+0000 7f2441c68700 1 --2- 172.21.15.33:0/317887070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56389c1f8de0 0x56389c1fa7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:05:39.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.877+0000 7f2440c66700 1 --2- 172.21.15.33:0/317887070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56389c1f84c0 0x56389c1fa2a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f242c0aa270 tx=0x7f242c01a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=203a679db3e83062 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:39.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.878+0000 7f2426ffd700 1 -- 172.21.15.33:0/317887070 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c0b3340 con 0x56389c1f84c0 2024-05-15T14:05:39.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.878+0000 7f2426ffd700 1 -- 172.21.15.33:0/317887070 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f242c0920c0 con 0x56389c1f84c0 2024-05-15T14:05:39.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.878+0000 7f2426ffd700 1 -- 172.21.15.33:0/317887070 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c0aaa30 con 0x56389c1f84c0 2024-05-15T14:05:39.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.878+0000 7f244df61b80 1 -- 172.21.15.33:0/317887070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56389c1f84c0 msgr2=0x56389c1fa2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:39.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.878+0000 7f244df61b80 1 --2- 172.21.15.33:0/317887070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56389c1f84c0 0x56389c1fa2a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f242c0aa270 tx=0x7f242c01a040 comp rx=0 tx=0).stop 2024-05-15T14:05:39.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.878+0000 7f244df61b80 1 -- 172.21.15.33:0/317887070 shutdown_connections 2024-05-15T14:05:39.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.878+0000 7f244df61b80 1 --2- 172.21.15.33:0/317887070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56389c1f8de0 0x56389c1fa7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:39.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.878+0000 7f244df61b80 1 --2- 172.21.15.33:0/317887070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56389c1f4360 0x56389c1e9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:39.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.878+0000 7f244df61b80 1 --2- 172.21.15.33:0/317887070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56389c1f84c0 0x56389c1fa2a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:39.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.878+0000 7f244df61b80 1 -- 172.21.15.33:0/317887070 >> 172.21.15.33:0/317887070 conn(0x56389c120f50 msgr2=0x56389c1f7a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:39.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.878+0000 7f244df61b80 1 -- 172.21.15.33:0/317887070 shutdown_connections 2024-05-15T14:05:39.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.878+0000 7f244df61b80 1 -- 172.21.15.33:0/317887070 wait complete. 2024-05-15T14:05:39.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.879+0000 7f244df61b80 1 -- start start 2024-05-15T14:05:39.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.879+0000 7f244df61b80 1 Processor -- start 2024-05-15T14:05:39.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.879+0000 7f244df61b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56389c1f4360 0x56389c215560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:39.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.879+0000 7f244df61b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56389c1f84c0 0x56389c215aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:39.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.879+0000 7f244df61b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56389c1f8de0 0x56389c215fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:39.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.879+0000 7f244df61b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f243c1aa0c0 con 0x56389c1f4360 2024-05-15T14:05:39.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.879+0000 7f244df61b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f243c1ab530 con 0x56389c1f84c0 2024-05-15T14:05:39.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.879+0000 7f244df61b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f243c1aa790 con 0x56389c1f8de0 2024-05-15T14:05:39.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.879+0000 7f244df61b80 10 client.0 _ll_register_callbacks cb 0x56389c0dcaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:05:39.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.879+0000 7f2441467700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56389c1f4360 0x56389c215560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:39.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.879+0000 7f2441467700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56389c1f4360 0x56389c215560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37268/0 (socket says 172.21.15.33:37268) 2024-05-15T14:05:39.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.879+0000 7f2441467700 1 -- 172.21.15.33:0/415279884 learned_addr learned my addr 172.21.15.33:0/415279884 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:39.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.879+0000 7f2441c68700 1 --2- 172.21.15.33:0/415279884 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56389c1f8de0 0x56389c215fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:39.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.879+0000 7f2440c66700 1 --2- 172.21.15.33:0/415279884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56389c1f84c0 0x56389c215aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:39.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.880+0000 7f2441467700 1 -- 172.21.15.33:0/415279884 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56389c1f8de0 msgr2=0x56389c215fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:39.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.880+0000 7f2441467700 1 --2- 172.21.15.33:0/415279884 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56389c1f8de0 0x56389c215fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:39.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.880+0000 7f2441467700 1 -- 172.21.15.33:0/415279884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56389c1f84c0 msgr2=0x56389c215aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:39.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.880+0000 7f2441467700 1 --2- 172.21.15.33:0/415279884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56389c1f84c0 0x56389c215aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:39.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.880+0000 7f2441467700 1 -- 172.21.15.33:0/415279884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56389c27be70 con 0x56389c1f4360 2024-05-15T14:05:39.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.880+0000 7f2441c68700 1 --2- 172.21.15.33:0/415279884 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56389c1f8de0 0x56389c215fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:05:39.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.880+0000 7f2441467700 1 --2- 172.21.15.33:0/415279884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56389c1f4360 0x56389c215560 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2438003960 tx=0x7f2438033ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8320adbda05c8353 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:39.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.880+0000 7f2424ff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:05:39.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.880+0000 7f2424ff9700 1 -- 172.21.15.33:0/415279884 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24380053e0 con 0x56389c1f4360 2024-05-15T14:05:39.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.880+0000 7f2424ff9700 1 -- 172.21.15.33:0/415279884 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f2438005580 con 0x56389c1f4360 2024-05-15T14:05:39.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.880+0000 7f244df61b80 1 -- 172.21.15.33:0/415279884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f242c0920c0 con 0x56389c1f4360 2024-05-15T14:05:39.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.880+0000 7f2424ff9700 1 -- 172.21.15.33:0/415279884 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f24380330b0 con 0x56389c1f4360 2024-05-15T14:05:39.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.880+0000 7f244df61b80 10 client.15969 fetch_fsmap learned FSMap version 29 2024-05-15T14:05:39.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.880+0000 7f244df61b80 1 -- 172.21.15.33:0/415279884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56389c27d150 con 0x56389c1f4360 2024-05-15T14:05:39.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.881+0000 7f2424ff9700 1 -- 172.21.15.33:0/415279884 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f242c0920c0 con 0x56389c1f4360 2024-05-15T14:05:39.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.881+0000 7f2424ff9700 1 -- 172.21.15.33:0/415279884 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f2438004ba0 con 0x56389c1f4360 2024-05-15T14:05:39.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.881+0000 7f244df61b80 10 client.15969 fetch_fsmap finished waiting for FSMap version 29 2024-05-15T14:05:39.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.881+0000 7f244df61b80 10 client.15969 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:05:39.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.881+0000 7f244df61b80 20 client.15969 populate_metadata read hostname 'smithi033' 2024-05-15T14:05:39.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.881+0000 7f244df61b80 1 --2- 172.21.15.33:0/415279884 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56389c27d7f0 0x56389c27fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:39.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.881+0000 7f244df61b80 4 client.15969 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.35, "await_for": 45.0}] 2024-05-15T14:05:39.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.881+0000 7f244df61b80 1 -- 172.21.15.33:0/415279884 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.35, "await_for": 45.0}) -- 0x56389c14d640 con 0x56389c27d7f0 2024-05-15T14:05:39.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.881+0000 7f2440c66700 1 --2- 172.21.15.33:0/415279884 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56389c27d7f0 0x56389c27fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:39.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.881+0000 7f2440c66700 1 --2- 172.21.15.33:0/415279884 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56389c27d7f0 0x56389c27fc10 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4a36d4bd82e4273 server_cookie=d10edf3b0a8b1efc in_seq=0 out_seq=0 2024-05-15T14:05:39.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.881+0000 7f2424ff9700 10 client.15969 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:05:39.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.944+0000 7f2424ff9700 1 -- 172.21.15.33:0/415279884 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56389c14d640 con 0x56389c27d7f0 2024-05-15T14:05:39.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.944+0000 7f2424ff9700 10 client.15969 handle_command_reply: tid=0 2024-05-15T14:05:39.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.944+0000 7f244df61b80 1 client.15969 shutdown 2024-05-15T14:05:39.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.944+0000 7f244df61b80 1 -- 172.21.15.33:0/415279884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56389c1f4360 msgr2=0x56389c215560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:39.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.944+0000 7f244df61b80 1 --2- 172.21.15.33:0/415279884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56389c1f4360 0x56389c215560 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2438003960 tx=0x7f2438033ac0 comp rx=0 tx=0).stop 2024-05-15T14:05:39.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.944+0000 7f244df61b80 1 -- 172.21.15.33:0/415279884 shutdown_connections 2024-05-15T14:05:39.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.944+0000 7f244df61b80 1 --2- 172.21.15.33:0/415279884 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56389c1f8de0 0x56389c215fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:39.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.944+0000 7f244df61b80 1 --2- 172.21.15.33:0/415279884 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56389c27d7f0 0x56389c27fc10 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:39.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.944+0000 7f244df61b80 1 --2- 172.21.15.33:0/415279884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56389c1f4360 0x56389c215560 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:39.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.944+0000 7f244df61b80 1 --2- 172.21.15.33:0/415279884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56389c1f84c0 0x56389c215aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:39.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.944+0000 7f2424ff9700 0 client.15969 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:05:39.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.944+0000 7f244df61b80 1 -- 172.21.15.33:0/415279884 >> 172.21.15.33:0/415279884 conn(0x56389c120f50 msgr2=0x56389c20cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:39.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.944+0000 7f244df61b80 1 -- 172.21.15.33:0/415279884 shutdown_connections 2024-05-15T14:05:39.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.944+0000 7f244df61b80 1 -- 172.21.15.33:0/415279884 wait complete. 2024-05-15T14:05:39.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.944+0000 7f244df61b80 20 client.15969 trim_cache size 0 max 16384 2024-05-15T14:05:39.947 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T14:05:39.947 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 29, 2024-05-15T14:05:39.947 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24478, 2024-05-15T14:05:39.947 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 18, 2024-05-15T14:05:39.948 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-05-15T14:05:39.948 INFO:teuthology.orchestra.run.smithi033.stdout: "417f377d": { 2024-05-15T14:05:39.948 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 18, 2024-05-15T14:05:39.948 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-05-15T14:05:39.948 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:05:39.948 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-05-15T14:05:39.948 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:05:39.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:05:39.948 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 90.0, 2024-05-15T14:05:39.948 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 138.3, 2024-05-15T14:05:39.948 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-05-15T14:05:39.948 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-05-15T14:05:39.948 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-05-15T14:05:39.948 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:05:39.949 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-05-15T14:05:39.949 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:05:39.949 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:05:39.949 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:05:39.949 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:05:39.949 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:05:39.949 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:05:39.949 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T14:05:39.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.945+0000 7f24436cb700 1 -- 172.21.15.33:0/1738192422 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f24280b2b20 msgr2=0x7f24280b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:39.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.945+0000 7f24436cb700 1 --2- 172.21.15.33:0/1738192422 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f24280b2b20 0x7f24280b4fe0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f243c19ab20 tx=0x7f2438009260 comp rx=0 tx=0).stop 2024-05-15T14:05:39.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.945+0000 7f24436cb700 1 -- 172.21.15.33:0/1738192422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f243c06c4d0 msgr2=0x7f243c19fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:39.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.945+0000 7f24436cb700 1 --2- 172.21.15.33:0/1738192422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f243c06c4d0 0x7f243c19fd60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f242c000c00 tx=0x7f242c00bfd0 comp rx=0 tx=0).stop 2024-05-15T14:05:39.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.945+0000 7f24436cb700 1 -- 172.21.15.33:0/1738192422 shutdown_connections 2024-05-15T14:05:39.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.945+0000 7f24436cb700 1 --2- 172.21.15.33:0/1738192422 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f24280b2b20 0x7f24280b4fe0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:39.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.945+0000 7f24436cb700 1 --2- 172.21.15.33:0/1738192422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f243c1095d0 0x7f243c1999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:39.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.945+0000 7f24436cb700 1 --2- 172.21.15.33:0/1738192422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f243c06bb10 0x7f243c19f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:39.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.945+0000 7f24436cb700 1 --2- 172.21.15.33:0/1738192422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f243c06c4d0 0x7f243c19fd60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:39.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.945+0000 7f24436cb700 1 -- 172.21.15.33:0/1738192422 >> 172.21.15.33:0/1738192422 conn(0x7f243c079480 msgr2=0x7f243c101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:39.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.945+0000 7f24436cb700 1 -- 172.21.15.33:0/1738192422 shutdown_connections 2024-05-15T14:05:39.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:39.945+0000 7f24436cb700 1 -- 172.21.15.33:0/1738192422 wait complete. 2024-05-15T14:05:40.057 INFO:tasks.workunit.client.0.smithi033.stdout: 1 331660 8.46 MB/sec execute 75 sec latency 173.750 ms 2024-05-15T14:05:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.405+0000 7f2689c83640 1 -- 172.21.15.33:0/3928765945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2684101950 msgr2=0x7f2684101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.405+0000 7f2689c83640 1 --2- 172.21.15.33:0/3928765945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2684101950 0x7f2684101dd0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2678009930 tx=0x7f267802a420 comp rx=0 tx=0).stop 2024-05-15T14:05:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.405+0000 7f2689c83640 1 -- 172.21.15.33:0/3928765945 shutdown_connections 2024-05-15T14:05:40.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.405+0000 7f2689c83640 1 --2- 172.21.15.33:0/3928765945 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2684102310 0x7f268410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:40.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.405+0000 7f2689c83640 1 --2- 172.21.15.33:0/3928765945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2684101950 0x7f2684101dd0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:40.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.405+0000 7f2689c83640 1 --2- 172.21.15.33:0/3928765945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2684107960 0x7f2684107d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:40.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.405+0000 7f2689c83640 1 -- 172.21.15.33:0/3928765945 >> 172.21.15.33:0/3928765945 conn(0x7f2684078ad0 msgr2=0x7f2684078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:40.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.405+0000 7f2689c83640 1 -- 172.21.15.33:0/3928765945 shutdown_connections 2024-05-15T14:05:40.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.405+0000 7f2689c83640 1 -- 172.21.15.33:0/3928765945 wait complete. 2024-05-15T14:05:40.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.406+0000 7f2689c83640 1 Processor -- start 2024-05-15T14:05:40.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.406+0000 7f2689c83640 1 -- start start 2024-05-15T14:05:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.406+0000 7f2689c83640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2684101950 0x7f2684103fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.406+0000 7f2689c83640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2684102310 0x7f2684104500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.406+0000 7f2689c83640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2684107960 0x7f2684104a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.406+0000 7f2689c83640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2684111c20 con 0x7f2684102310 2024-05-15T14:05:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.406+0000 7f2689c83640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2684111aa0 con 0x7f2684107960 2024-05-15T14:05:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.406+0000 7f2689c83640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2684111da0 con 0x7f2684101950 2024-05-15T14:05:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.406+0000 7f2682ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2684102310 0x7f2684104500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.407+0000 7f2682ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2684102310 0x7f2684104500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37284/0 (socket says 172.21.15.33:37284) 2024-05-15T14:05:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.407+0000 7f2682ffd640 1 -- 172.21.15.33:0/3603859609 learned_addr learned my addr 172.21.15.33:0/3603859609 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:40.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.407+0000 7f26837fe640 1 --2- 172.21.15.33:0/3603859609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2684101950 0x7f2684103fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.407+0000 7f2683fff640 1 --2- 172.21.15.33:0/3603859609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2684107960 0x7f2684104a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.407+0000 7f2682ffd640 1 -- 172.21.15.33:0/3603859609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2684101950 msgr2=0x7f2684103fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.407+0000 7f2682ffd640 1 --2- 172.21.15.33:0/3603859609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2684101950 0x7f2684103fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.407+0000 7f2682ffd640 1 -- 172.21.15.33:0/3603859609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2684107960 msgr2=0x7f2684104a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.407+0000 7f2682ffd640 1 --2- 172.21.15.33:0/3603859609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2684107960 0x7f2684104a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.407+0000 7f2682ffd640 1 -- 172.21.15.33:0/3603859609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26841a7790 con 0x7f2684102310 2024-05-15T14:05:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.407+0000 7f2683fff640 1 --2- 172.21.15.33:0/3603859609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2684107960 0x7f2684104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:05:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.407+0000 7f2682ffd640 1 --2- 172.21.15.33:0/3603859609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2684102310 0x7f2684104500 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2678005940 tx=0x7f26780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.407+0000 7f2680ff9640 1 -- 172.21.15.33:0/3603859609 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267803f030 con 0x7f2684102310 2024-05-15T14:05:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.407+0000 7f2680ff9640 1 -- 172.21.15.33:0/3603859609 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2678003cb0 con 0x7f2684102310 2024-05-15T14:05:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.407+0000 7f2689c83640 1 -- 172.21.15.33:0/3603859609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26841a7a20 con 0x7f2684102310 2024-05-15T14:05:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.407+0000 7f2680ff9640 1 -- 172.21.15.33:0/3603859609 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2678035070 con 0x7f2684102310 2024-05-15T14:05:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.407+0000 7f2689c83640 1 -- 172.21.15.33:0/3603859609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26841a7f10 con 0x7f2684102310 2024-05-15T14:05:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.410+0000 7f268c9a7740 1 Processor -- start 2024-05-15T14:05:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.410+0000 7f2680ff9640 1 -- 172.21.15.33:0/3603859609 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2678042030 con 0x7f2684102310 2024-05-15T14:05:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.410+0000 7f268c9a7740 1 -- start start 2024-05-15T14:05:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.410+0000 7f268c9a7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b621a38f0 0x556b621a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.410+0000 7f268c9a7740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b621a4210 0x556b621ae590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.410+0000 7f268c9a7740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556b621aead0 0x556b621b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.410+0000 7f268c9a7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556b62116c30 con 0x556b621a38f0 2024-05-15T14:05:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.410+0000 7f268c9a7740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556b621067d0 con 0x556b621a4210 2024-05-15T14:05:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.410+0000 7f268c9a7740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x556b6210a810 con 0x556b621aead0 2024-05-15T14:05:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.410+0000 7f2683fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b621a38f0 0x556b621a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.410+0000 7f2683fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b621a38f0 0x556b621a3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37296/0 (socket says 172.21.15.33:37296) 2024-05-15T14:05:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.410+0000 7f2683fff640 1 -- 172.21.15.33:0/1893047564 learned_addr learned my addr 172.21.15.33:0/1893047564 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.410+0000 7f2682ffd640 1 --2- 172.21.15.33:0/1893047564 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556b621aead0 0x556b621b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.410+0000 7f26837fe640 1 --2- 172.21.15.33:0/1893047564 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b621a4210 0x556b621ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.410+0000 7f2683fff640 1 -- 172.21.15.33:0/1893047564 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556b621aead0 msgr2=0x556b621b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.410+0000 7f2683fff640 1 --2- 172.21.15.33:0/1893047564 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556b621aead0 0x556b621b0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.410+0000 7f2683fff640 1 -- 172.21.15.33:0/1893047564 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b621a4210 msgr2=0x556b621ae590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.410+0000 7f2683fff640 1 --2- 172.21.15.33:0/1893047564 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b621a4210 0x556b621ae590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.411+0000 7f2683fff640 1 -- 172.21.15.33:0/1893047564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b61e1b210 con 0x556b621a38f0 2024-05-15T14:05:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.411+0000 7f2680ff9640 1 --2- 172.21.15.33:0/3603859609 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f266c0b2950 0x7f266c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.411+0000 7f2683fff640 1 --2- 172.21.15.33:0/1893047564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b621a38f0 0x556b621a3cd0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f267c009f90 tx=0x7f267c00f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87fc6b65f9ee9484 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.411+0000 7f26837fe640 1 --2- 172.21.15.33:0/3603859609 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f266c0b2950 0x7f266c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.411+0000 7f2680ff9640 1 -- 172.21.15.33:0/3603859609 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f2678102ed0 con 0x7f2684102310 2024-05-15T14:05:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.411+0000 7f2669ffb640 1 -- 172.21.15.33:0/1893047564 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c003b70 con 0x556b621a38f0 2024-05-15T14:05:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.411+0000 7f2669ffb640 1 -- 172.21.15.33:0/1893047564 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f267c003d10 con 0x556b621a38f0 2024-05-15T14:05:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.411+0000 7f2669ffb640 1 -- 172.21.15.33:0/1893047564 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c0118f0 con 0x556b621a38f0 2024-05-15T14:05:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.412+0000 7f268c9a7740 1 -- 172.21.15.33:0/1893047564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b621a38f0 msgr2=0x556b621a3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.412+0000 7f26837fe640 1 --2- 172.21.15.33:0/3603859609 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f266c0b2950 0x7f266c0b4e10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2674005fd0 tx=0x7f267400b5d0 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.412+0000 7f268c9a7740 1 --2- 172.21.15.33:0/1893047564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b621a38f0 0x556b621a3cd0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f267c009f90 tx=0x7f267c00f7f0 comp rx=0 tx=0).stop 2024-05-15T14:05:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.412+0000 7f268c9a7740 1 -- 172.21.15.33:0/1893047564 shutdown_connections 2024-05-15T14:05:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.412+0000 7f268c9a7740 1 --2- 172.21.15.33:0/1893047564 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556b621aead0 0x556b621b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.412+0000 7f268c9a7740 1 --2- 172.21.15.33:0/1893047564 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b621a4210 0x556b621ae590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.412+0000 7f268c9a7740 1 --2- 172.21.15.33:0/1893047564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b621a38f0 0x556b621a3cd0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.412+0000 7f268c9a7740 1 -- 172.21.15.33:0/1893047564 >> 172.21.15.33:0/1893047564 conn(0x556b6219fe00 msgr2=0x556b621a01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.412+0000 7f268c9a7740 1 -- 172.21.15.33:0/1893047564 shutdown_connections 2024-05-15T14:05:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.412+0000 7f268c9a7740 1 -- 172.21.15.33:0/1893047564 wait complete. 2024-05-15T14:05:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.412+0000 7f268c9a7740 1 -- start start 2024-05-15T14:05:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.412+0000 7f268c9a7740 1 Processor -- start 2024-05-15T14:05:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.413+0000 7f268c9a7740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556b621a38f0 0x556b621ae3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.413+0000 7f268c9a7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b621a4210 0x556b621ab6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.413+0000 7f268c9a7740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b621aead0 0x556b621abbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.413+0000 7f268c9a7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556b61dec790 con 0x556b621a4210 2024-05-15T14:05:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.413+0000 7f268c9a7740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556b620c2890 con 0x556b621aead0 2024-05-15T14:05:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.413+0000 7f268c9a7740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x556b621b17a0 con 0x556b621a38f0 2024-05-15T14:05:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.413+0000 7f268c9a7740 10 client.0 _ll_register_callbacks cb 0x556b620c5d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:05:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.413+0000 7f2682ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b621a4210 0x556b621ab6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.413+0000 7f2682ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b621a4210 0x556b621ab6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37304/0 (socket says 172.21.15.33:37304) 2024-05-15T14:05:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.413+0000 7f2682ffd640 1 -- 172.21.15.33:0/33355029 learned_addr learned my addr 172.21.15.33:0/33355029 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.413+0000 7f2683fff640 1 --2- 172.21.15.33:0/33355029 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b621aead0 0x556b621abbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.413+0000 7f26837fe640 1 --2- 172.21.15.33:0/33355029 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556b621a38f0 0x556b621ae3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.413+0000 7f2682ffd640 1 -- 172.21.15.33:0/33355029 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556b621a38f0 msgr2=0x556b621ae3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.413+0000 7f2682ffd640 1 --2- 172.21.15.33:0/33355029 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556b621a38f0 0x556b621ae3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.413+0000 7f2682ffd640 1 -- 172.21.15.33:0/33355029 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b621aead0 msgr2=0x556b621abbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.413+0000 7f2682ffd640 1 --2- 172.21.15.33:0/33355029 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b621aead0 0x556b621abbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.413+0000 7f2682ffd640 1 -- 172.21.15.33:0/33355029 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b61ee7620 con 0x556b621a4210 2024-05-15T14:05:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.413+0000 7f2682ffd640 1 --2- 172.21.15.33:0/33355029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b621a4210 0x556b621ab6b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2678003fd0 tx=0x7f26780092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4590f38548b6814 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.413+0000 7f2662ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:05:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.413+0000 7f2662ffd640 1 -- 172.21.15.33:0/33355029 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2678049070 con 0x556b621a4210 2024-05-15T14:05:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.413+0000 7f2662ffd640 1 -- 172.21.15.33:0/33355029 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f26780ce1b0 con 0x556b621a4210 2024-05-15T14:05:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.413+0000 7f268c9a7740 1 -- 172.21.15.33:0/33355029 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f267c003d10 con 0x556b621a4210 2024-05-15T14:05:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.414+0000 7f2662ffd640 1 -- 172.21.15.33:0/33355029 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f26780c6080 con 0x556b621a4210 2024-05-15T14:05:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.414+0000 7f268c9a7740 10 client.15981 fetch_fsmap learned FSMap version 29 2024-05-15T14:05:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.414+0000 7f268c9a7740 1 -- 172.21.15.33:0/33355029 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556b621bd160 con 0x556b621a4210 2024-05-15T14:05:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.414+0000 7f2662ffd640 1 -- 172.21.15.33:0/33355029 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f267c003d10 con 0x556b621a4210 2024-05-15T14:05:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.414+0000 7f2662ffd640 1 -- 172.21.15.33:0/33355029 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f2678089be0 con 0x556b621a4210 2024-05-15T14:05:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.414+0000 7f268c9a7740 10 client.15981 fetch_fsmap finished waiting for FSMap version 29 2024-05-15T14:05:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.414+0000 7f268c9a7740 10 client.15981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:05:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.414+0000 7f268c9a7740 20 client.15981 populate_metadata read hostname 'smithi033' 2024-05-15T14:05:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.414+0000 7f268c9a7740 1 --2- 172.21.15.33:0/33355029 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556b621be660 0x556b621c0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.414+0000 7f268c9a7740 4 client.15981 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:05:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.414+0000 7f268c9a7740 1 -- 172.21.15.33:0/33355029 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b62118b80 con 0x556b621be660 2024-05-15T14:05:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.414+0000 7f26837fe640 1 --2- 172.21.15.33:0/33355029 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556b621be660 0x556b621c0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:40.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.415+0000 7f2662ffd640 10 client.15981 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:05:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.415+0000 7f26837fe640 1 --2- 172.21.15.33:0/33355029 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556b621be660 0x556b621c0a50 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e41fad5a26f7b8e server_cookie=e851623bccce9822 in_seq=0 out_seq=0 2024-05-15T14:05:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.415+0000 7f2662ffd640 1 -- 172.21.15.33:0/33355029 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556b62118b80 con 0x556b621be660 2024-05-15T14:05:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.415+0000 7f2662ffd640 10 client.15981 handle_command_reply: tid=0 2024-05-15T14:05:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.415+0000 7f268c9a7740 1 client.15981 shutdown 2024-05-15T14:05:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.416+0000 7f268c9a7740 1 -- 172.21.15.33:0/33355029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b621a4210 msgr2=0x556b621ab6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.416+0000 7f268c9a7740 1 --2- 172.21.15.33:0/33355029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b621a4210 0x556b621ab6b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2678003fd0 tx=0x7f26780092f0 comp rx=0 tx=0).stop 2024-05-15T14:05:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.416+0000 7f268c9a7740 1 -- 172.21.15.33:0/33355029 shutdown_connections 2024-05-15T14:05:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.416+0000 7f268c9a7740 1 --2- 172.21.15.33:0/33355029 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556b621be660 0x556b621c0a50 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.416+0000 7f268c9a7740 1 --2- 172.21.15.33:0/33355029 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b621aead0 0x556b621abbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.416+0000 7f268c9a7740 1 --2- 172.21.15.33:0/33355029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b621a4210 0x556b621ab6b0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.416+0000 7f268c9a7740 1 --2- 172.21.15.33:0/33355029 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556b621a38f0 0x556b621ae3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.416+0000 7f2662ffd640 0 client.15981 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:05:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.416+0000 7f268c9a7740 1 -- 172.21.15.33:0/33355029 >> 172.21.15.33:0/33355029 conn(0x556b6219fe00 msgr2=0x556b6219f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:40.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.416+0000 7f268c9a7740 1 -- 172.21.15.33:0/33355029 shutdown_connections 2024-05-15T14:05:40.424 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.416+0000 7f268c9a7740 1 -- 172.21.15.33:0/33355029 wait complete. 2024-05-15T14:05:40.424 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.416+0000 7f268c9a7740 20 client.15981 trim_cache size 0 max 16384 2024-05-15T14:05:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.431+0000 7f268c9a7740 1 Processor -- start 2024-05-15T14:05:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.431+0000 7f268c9a7740 1 -- start start 2024-05-15T14:05:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.431+0000 7f268c9a7740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556b621a38f0 0x556b621a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.431+0000 7f268c9a7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b621a4210 0x556b621adef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.431+0000 7f268c9a7740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b621aead0 0x556b621c6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.431+0000 7f268c9a7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f26841b3090 con 0x556b621a4210 2024-05-15T14:05:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.431+0000 7f268c9a7740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f26841b0960 con 0x556b621aead0 2024-05-15T14:05:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.431+0000 7f268c9a7740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f26841ae7b0 con 0x556b621a38f0 2024-05-15T14:05:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.431+0000 7f2682ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b621a4210 0x556b621adef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.431+0000 7f2682ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b621a4210 0x556b621adef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37318/0 (socket says 172.21.15.33:37318) 2024-05-15T14:05:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.431+0000 7f2682ffd640 1 -- 172.21.15.33:0/3985504532 learned_addr learned my addr 172.21.15.33:0/3985504532 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.431+0000 7f26837fe640 1 --2- 172.21.15.33:0/3985504532 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556b621a38f0 0x556b621a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.431+0000 7f2682ffd640 1 -- 172.21.15.33:0/3985504532 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556b621a38f0 msgr2=0x556b621a5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.431+0000 7f2682ffd640 1 --2- 172.21.15.33:0/3985504532 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556b621a38f0 0x556b621a5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.431+0000 7f2682ffd640 1 -- 172.21.15.33:0/3985504532 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b621aead0 msgr2=0x556b621c6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.431+0000 7f2682ffd640 1 --2- 172.21.15.33:0/3985504532 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b621aead0 0x556b621c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.431+0000 7f2682ffd640 1 -- 172.21.15.33:0/3985504532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b6219fc30 con 0x556b621a4210 2024-05-15T14:05:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.432+0000 7f2682ffd640 1 --2- 172.21.15.33:0/3985504532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b621a4210 0x556b621adef0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f26780c6240 tx=0x7f2678003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eef9d8bd6e25668b server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.432+0000 7f26697fa640 1 -- 172.21.15.33:0/3985504532 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2678049070 con 0x556b621a4210 2024-05-15T14:05:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.432+0000 7f26697fa640 1 -- 172.21.15.33:0/3985504532 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26780c9cd0 con 0x556b621a4210 2024-05-15T14:05:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.432+0000 7f26697fa640 1 -- 172.21.15.33:0/3985504532 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26780044e0 con 0x556b621a4210 2024-05-15T14:05:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.432+0000 7f268c9a7740 1 -- 172.21.15.33:0/3985504532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b621a4210 msgr2=0x556b621adef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.432+0000 7f268c9a7740 1 --2- 172.21.15.33:0/3985504532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b621a4210 0x556b621adef0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f26780c6240 tx=0x7f2678003f40 comp rx=0 tx=0).stop 2024-05-15T14:05:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.432+0000 7f268c9a7740 1 -- 172.21.15.33:0/3985504532 shutdown_connections 2024-05-15T14:05:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.432+0000 7f268c9a7740 1 --2- 172.21.15.33:0/3985504532 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b621aead0 0x556b621c6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.432+0000 7f268c9a7740 1 --2- 172.21.15.33:0/3985504532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b621a4210 0x556b621adef0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.432+0000 7f268c9a7740 1 --2- 172.21.15.33:0/3985504532 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556b621a38f0 0x556b621a5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.432+0000 7f268c9a7740 1 -- 172.21.15.33:0/3985504532 >> 172.21.15.33:0/3985504532 conn(0x556b6219fe00 msgr2=0x556b621a6be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.432+0000 7f268c9a7740 1 -- 172.21.15.33:0/3985504532 shutdown_connections 2024-05-15T14:05:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.432+0000 7f268c9a7740 1 -- 172.21.15.33:0/3985504532 wait complete. 2024-05-15T14:05:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.433+0000 7f268c9a7740 1 -- start start 2024-05-15T14:05:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.433+0000 7f268c9a7740 1 Processor -- start 2024-05-15T14:05:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.433+0000 7f268c9a7740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b621a38f0 0x556b621b69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.433+0000 7f268c9a7740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556b621a4210 0x556b621b6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.433+0000 7f268c9a7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b621aead0 0x556b621b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.433+0000 7f268c9a7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f26841aee20 con 0x556b621aead0 2024-05-15T14:05:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.433+0000 7f268c9a7740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f26841b0290 con 0x556b621a38f0 2024-05-15T14:05:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.433+0000 7f268c9a7740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f26841af4f0 con 0x556b621a4210 2024-05-15T14:05:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.433+0000 7f268c9a7740 10 client.0 _ll_register_callbacks cb 0x7f2678005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:05:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.433+0000 7f2683fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b621aead0 0x556b621b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.433+0000 7f2683fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b621aead0 0x556b621b39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37326/0 (socket says 172.21.15.33:37326) 2024-05-15T14:05:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.433+0000 7f2683fff640 1 -- 172.21.15.33:0/1748295502 learned_addr learned my addr 172.21.15.33:0/1748295502 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.433+0000 7f2682ffd640 1 --2- 172.21.15.33:0/1748295502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556b621a4210 0x556b621b6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.434+0000 7f26837fe640 1 --2- 172.21.15.33:0/1748295502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b621a38f0 0x556b621b69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.434+0000 7f2683fff640 1 -- 172.21.15.33:0/1748295502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556b621a4210 msgr2=0x556b621b6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.434+0000 7f2683fff640 1 --2- 172.21.15.33:0/1748295502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556b621a4210 0x556b621b6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.434+0000 7f2683fff640 1 -- 172.21.15.33:0/1748295502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b621a38f0 msgr2=0x556b621b69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.434+0000 7f2683fff640 1 --2- 172.21.15.33:0/1748295502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b621a38f0 0x556b621b69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.434+0000 7f2683fff640 1 -- 172.21.15.33:0/1748295502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b621b3f00 con 0x556b621aead0 2024-05-15T14:05:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.434+0000 7f2683fff640 1 --2- 172.21.15.33:0/1748295502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b621aead0 0x556b621b39c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f267c009540 tx=0x7f267c009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14ceeaa9e62066f server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.434+0000 7f2662ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:05:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.434+0000 7f2662ffd640 1 -- 172.21.15.33:0/1748295502 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c02a060 con 0x556b621aead0 2024-05-15T14:05:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.434+0000 7f268c9a7740 1 -- 172.21.15.33:0/1748295502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26780c9cd0 con 0x556b621aead0 2024-05-15T14:05:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.434+0000 7f2662ffd640 1 -- 172.21.15.33:0/1748295502 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f267c011580 con 0x556b621aead0 2024-05-15T14:05:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.434+0000 7f2662ffd640 1 -- 172.21.15.33:0/1748295502 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f267c011780 con 0x556b621aead0 2024-05-15T14:05:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.434+0000 7f268c9a7740 10 client.15987 fetch_fsmap learned FSMap version 29 2024-05-15T14:05:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.434+0000 7f268c9a7740 1 -- 172.21.15.33:0/1748295502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556b622296b0 con 0x556b621aead0 2024-05-15T14:05:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.434+0000 7f2662ffd640 1 -- 172.21.15.33:0/1748295502 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26780c9cd0 con 0x556b621aead0 2024-05-15T14:05:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.435+0000 7f2662ffd640 1 -- 172.21.15.33:0/1748295502 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f267c005600 con 0x556b621aead0 2024-05-15T14:05:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.435+0000 7f268c9a7740 10 client.15987 fetch_fsmap finished waiting for FSMap version 29 2024-05-15T14:05:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.435+0000 7f268c9a7740 10 client.15987 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:05:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.435+0000 7f268c9a7740 20 client.15987 populate_metadata read hostname 'smithi033' 2024-05-15T14:05:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.435+0000 7f268c9a7740 1 --2- 172.21.15.33:0/1748295502 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556b62229af0 0x556b6222bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.435+0000 7f268c9a7740 4 client.15987 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:05:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.435+0000 7f268c9a7740 1 -- 172.21.15.33:0/1748295502 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556b62112360 con 0x556b62229af0 2024-05-15T14:05:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.435+0000 7f26837fe640 1 --2- 172.21.15.33:0/1748295502 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556b62229af0 0x556b6222bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.435+0000 7f2662ffd640 10 client.15987 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:05:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.435+0000 7f26837fe640 1 --2- 172.21.15.33:0/1748295502 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556b62229af0 0x556b6222bf10 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5e7aa414119d547 server_cookie=34c6d46c002d64a7 in_seq=0 out_seq=0 2024-05-15T14:05:40.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.498+0000 7f2662ffd640 1 -- 172.21.15.33:0/1748295502 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556b62112360 con 0x556b62229af0 2024-05-15T14:05:40.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.498+0000 7f2662ffd640 10 client.15987 handle_command_reply: tid=0 2024-05-15T14:05:40.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.498+0000 7f268c9a7740 1 client.15987 shutdown 2024-05-15T14:05:40.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.499+0000 7f268c9a7740 1 -- 172.21.15.33:0/1748295502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b621aead0 msgr2=0x556b621b39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:40.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.499+0000 7f268c9a7740 1 --2- 172.21.15.33:0/1748295502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b621aead0 0x556b621b39c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f267c009540 tx=0x7f267c009d30 comp rx=0 tx=0).stop 2024-05-15T14:05:40.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.499+0000 7f268c9a7740 1 -- 172.21.15.33:0/1748295502 shutdown_connections 2024-05-15T14:05:40.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.499+0000 7f268c9a7740 1 --2- 172.21.15.33:0/1748295502 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556b62229af0 0x556b6222bf10 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:40.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.499+0000 7f268c9a7740 1 --2- 172.21.15.33:0/1748295502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b621aead0 0x556b621b39c0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:40.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.499+0000 7f268c9a7740 1 --2- 172.21.15.33:0/1748295502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556b621a4210 0x556b621b6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:40.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.499+0000 7f268c9a7740 1 --2- 172.21.15.33:0/1748295502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b621a38f0 0x556b621b69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.499+0000 7f268c9a7740 1 -- 172.21.15.33:0/1748295502 >> 172.21.15.33:0/1748295502 conn(0x556b6219fe00 msgr2=0x556b6219e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.499+0000 7f2662ffd640 0 client.15987 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:05:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.499+0000 7f268c9a7740 1 -- 172.21.15.33:0/1748295502 shutdown_connections 2024-05-15T14:05:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.499+0000 7f268c9a7740 1 -- 172.21.15.33:0/1748295502 wait complete. 2024-05-15T14:05:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.499+0000 7f268c9a7740 20 client.15987 trim_cache size 0 max 16384 2024-05-15T14:05:40.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:05:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:05:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:05:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.500+0000 7f2689c83640 1 -- 172.21.15.33:0/3603859609 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f266c0b2950 msgr2=0x7f266c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.500+0000 7f2689c83640 1 --2- 172.21.15.33:0/3603859609 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f266c0b2950 0x7f266c0b4e10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2674005fd0 tx=0x7f267400b5d0 comp rx=0 tx=0).stop 2024-05-15T14:05:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.500+0000 7f2689c83640 1 -- 172.21.15.33:0/3603859609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2684102310 msgr2=0x7f2684104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.500+0000 7f2689c83640 1 --2- 172.21.15.33:0/3603859609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2684102310 0x7f2684104500 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2678005940 tx=0x7f26780042f0 comp rx=0 tx=0).stop 2024-05-15T14:05:40.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.500+0000 7f2689c83640 1 -- 172.21.15.33:0/3603859609 shutdown_connections 2024-05-15T14:05:40.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.500+0000 7f2689c83640 1 --2- 172.21.15.33:0/3603859609 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f266c0b2950 0x7f266c0b4e10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:40.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.500+0000 7f2689c83640 1 --2- 172.21.15.33:0/3603859609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2684107960 0x7f2684104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:40.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.500+0000 7f2689c83640 1 --2- 172.21.15.33:0/3603859609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2684102310 0x7f2684104500 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:40.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.500+0000 7f2689c83640 1 --2- 172.21.15.33:0/3603859609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2684101950 0x7f2684103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:40.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.500+0000 7f2689c83640 1 -- 172.21.15.33:0/3603859609 >> 172.21.15.33:0/3603859609 conn(0x7f2684078ad0 msgr2=0x7f268406b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:40.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.500+0000 7f2689c83640 1 -- 172.21.15.33:0/3603859609 shutdown_connections 2024-05-15T14:05:40.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:40.500+0000 7f2689c83640 1 -- 172.21.15.33:0/3603859609 wait complete. 2024-05-15T14:05:40.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:40 smithi052 ceph-mon[36190]: pgmap v163: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 282 op/s 2024-05-15T14:05:40.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:40 smithi081 ceph-mon[36434]: pgmap v163: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 282 op/s 2024-05-15T14:05:40.659 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:40 smithi033 ceph-mon[31650]: pgmap v163: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 282 op/s 2024-05-15T14:05:40.686 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '417f377d', quiesce took 6.7 seconds. Will release after: 11.650000000000002 2024-05-15T14:05:41.057 INFO:tasks.workunit.client.0.smithi033.stdout: 1 331660 8.35 MB/sec execute 76 sec latency 1173.794 ms 2024-05-15T14:05:42.057 INFO:tasks.workunit.client.0.smithi033.stdout: 1 331660 8.24 MB/sec execute 77 sec latency 2173.850 ms 2024-05-15T14:05:42.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:42 smithi052 ceph-mon[36190]: pgmap v164: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 282 op/s 2024-05-15T14:05:42.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:42 smithi081 ceph-mon[36434]: pgmap v164: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 282 op/s 2024-05-15T14:05:42.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:42 smithi033 ceph-mon[31650]: pgmap v164: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 282 op/s 2024-05-15T14:05:43.058 INFO:tasks.workunit.client.0.smithi033.stdout: 1 331660 8.13 MB/sec execute 78 sec latency 3173.935 ms 2024-05-15T14:05:44.058 INFO:tasks.workunit.client.0.smithi033.stdout: 1 331660 8.03 MB/sec execute 79 sec latency 4174.487 ms 2024-05-15T14:05:44.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:44 smithi052 ceph-mon[36190]: pgmap v165: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 408 op/s 2024-05-15T14:05:44.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:44 smithi081 ceph-mon[36434]: pgmap v165: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 408 op/s 2024-05-15T14:05:44.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:44 smithi033 ceph-mon[31650]: pgmap v165: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 408 op/s 2024-05-15T14:05:45.058 INFO:tasks.workunit.client.0.smithi033.stdout: 1 331660 7.93 MB/sec execute 80 sec latency 5174.566 ms 2024-05-15T14:05:46.058 INFO:tasks.workunit.client.0.smithi033.stdout: 1 331660 7.83 MB/sec execute 81 sec latency 6174.691 ms 2024-05-15T14:05:46.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:46 smithi052 ceph-mon[36190]: pgmap v166: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 241 op/s 2024-05-15T14:05:46.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:46 smithi081 ceph-mon[36434]: pgmap v166: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 241 op/s 2024-05-15T14:05:46.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:46 smithi033 ceph-mon[31650]: pgmap v166: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 241 op/s 2024-05-15T14:05:47.058 INFO:tasks.workunit.client.0.smithi033.stdout: 1 331660 7.74 MB/sec execute 82 sec latency 7174.778 ms 2024-05-15T14:05:48.058 INFO:tasks.workunit.client.0.smithi033.stdout: 1 331660 7.64 MB/sec execute 83 sec latency 8174.875 ms 2024-05-15T14:05:48.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:48 smithi052 ceph-mon[36190]: pgmap v167: 97 pgs: 97 active+clean; 85 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 273 op/s 2024-05-15T14:05:48.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:48 smithi081 ceph-mon[36434]: pgmap v167: 97 pgs: 97 active+clean; 85 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 273 op/s 2024-05-15T14:05:48.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:48 smithi033 ceph-mon[31650]: pgmap v167: 97 pgs: 97 active+clean; 85 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 273 op/s 2024-05-15T14:05:49.058 INFO:tasks.workunit.client.0.smithi033.stdout: 1 331660 7.55 MB/sec execute 84 sec latency 9174.941 ms 2024-05-15T14:05:50.058 INFO:tasks.workunit.client.0.smithi033.stdout: 1 331660 7.46 MB/sec execute 85 sec latency 10174.990 ms 2024-05-15T14:05:50.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:50 smithi052 ceph-mon[36190]: pgmap v168: 97 pgs: 97 active+clean; 85 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 2.0 MiB/s wr, 156 op/s 2024-05-15T14:05:50.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:50 smithi081 ceph-mon[36434]: pgmap v168: 97 pgs: 97 active+clean; 85 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 2.0 MiB/s wr, 156 op/s 2024-05-15T14:05:50.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:50 smithi033 ceph-mon[31650]: pgmap v168: 97 pgs: 97 active+clean; 85 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 2.0 MiB/s wr, 156 op/s 2024-05-15T14:05:51.058 INFO:tasks.workunit.client.0.smithi033.stdout: 1 331660 7.38 MB/sec execute 86 sec latency 11175.101 ms 2024-05-15T14:05:52.058 INFO:tasks.workunit.client.0.smithi033.stdout: 1 331660 7.29 MB/sec execute 87 sec latency 12175.206 ms 2024-05-15T14:05:52.336 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T14:05:52.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:52 smithi052 ceph-mon[36190]: pgmap v169: 97 pgs: 97 active+clean; 85 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 2.0 MiB/s wr, 156 op/s 2024-05-15T14:05:52.601 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:05:52.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:52 smithi081 ceph-mon[36434]: pgmap v169: 97 pgs: 97 active+clean; 85 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 2.0 MiB/s wr, 156 op/s 2024-05-15T14:05:52.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:52 smithi033 ceph-mon[31650]: pgmap v169: 97 pgs: 97 active+clean; 85 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 2.0 MiB/s wr, 156 op/s 2024-05-15T14:05:53.058 INFO:tasks.workunit.client.0.smithi033.stdout: 1 331660 7.21 MB/sec execute 88 sec latency 13175.279 ms 2024-05-15T14:05:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.592+0000 7ff4d450a640 1 -- 172.21.15.33:0/1555710288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4cc107970 msgr2=0x7ff4cc107d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.592+0000 7ff4d450a640 1 --2- 172.21.15.33:0/1555710288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4cc107970 0x7ff4cc107d50 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009cd0 tx=0x7ff4c802a8e0 comp rx=0 tx=0).stop 2024-05-15T14:05:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.592+0000 7ff4d450a640 1 -- 172.21.15.33:0/1555710288 shutdown_connections 2024-05-15T14:05:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.592+0000 7ff4d450a640 1 --2- 172.21.15.33:0/1555710288 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4cc102370 0x7ff4cc10e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.592+0000 7ff4d450a640 1 --2- 172.21.15.33:0/1555710288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff4cc1019b0 0x7ff4cc101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.592+0000 7ff4d450a640 1 --2- 172.21.15.33:0/1555710288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4cc107970 0x7ff4cc107d50 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.592+0000 7ff4d450a640 1 -- 172.21.15.33:0/1555710288 >> 172.21.15.33:0/1555710288 conn(0x7ff4cc078aa0 msgr2=0x7ff4cc078eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.592+0000 7ff4d450a640 1 -- 172.21.15.33:0/1555710288 shutdown_connections 2024-05-15T14:05:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.592+0000 7ff4d450a640 1 -- 172.21.15.33:0/1555710288 wait complete. 2024-05-15T14:05:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.593+0000 7ff4d450a640 1 Processor -- start 2024-05-15T14:05:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.593+0000 7ff4d450a640 1 -- start start 2024-05-15T14:05:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.593+0000 7ff4d450a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4cc1019b0 0x7ff4cc19a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.593+0000 7ff4d450a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4cc102370 0x7ff4cc19a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.593+0000 7ff4d450a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff4cc107970 0x7ff4cc1a50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.593+0000 7ff4d450a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff4cc111c30 con 0x7ff4cc102370 2024-05-15T14:05:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.593+0000 7ff4d450a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff4cc111ab0 con 0x7ff4cc1019b0 2024-05-15T14:05:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.593+0000 7ff4d450a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ff4cc111db0 con 0x7ff4cc107970 2024-05-15T14:05:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.593+0000 7ff4d1aa5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4cc102370 0x7ff4cc19a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.593+0000 7ff4d1aa5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4cc102370 0x7ff4cc19a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43508/0 (socket says 172.21.15.33:43508) 2024-05-15T14:05:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.593+0000 7ff4d1aa5640 1 -- 172.21.15.33:0/772968692 learned_addr learned my addr 172.21.15.33:0/772968692 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.593+0000 7ff4d2aa7640 1 --2- 172.21.15.33:0/772968692 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff4cc107970 0x7ff4cc1a50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.593+0000 7ff4d22a6640 1 --2- 172.21.15.33:0/772968692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4cc1019b0 0x7ff4cc19a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.593+0000 7ff4d1aa5640 1 -- 172.21.15.33:0/772968692 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff4cc107970 msgr2=0x7ff4cc1a50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.593+0000 7ff4d1aa5640 1 --2- 172.21.15.33:0/772968692 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff4cc107970 0x7ff4cc1a50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.593+0000 7ff4d1aa5640 1 -- 172.21.15.33:0/772968692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4cc1019b0 msgr2=0x7ff4cc19a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.593+0000 7ff4d1aa5640 1 --2- 172.21.15.33:0/772968692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4cc1019b0 0x7ff4cc19a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.594+0000 7ff4d1aa5640 1 -- 172.21.15.33:0/772968692 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4cc1a55f0 con 0x7ff4cc102370 2024-05-15T14:05:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.594+0000 7ff4d1aa5640 1 --2- 172.21.15.33:0/772968692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4cc102370 0x7ff4cc19a570 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00d770 tx=0x7ff4bc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.594+0000 7ff4c37fe640 1 -- 172.21.15.33:0/772968692 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc017070 con 0x7ff4cc102370 2024-05-15T14:05:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.594+0000 7ff4c37fe640 1 -- 172.21.15.33:0/772968692 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4bc0054d0 con 0x7ff4cc102370 2024-05-15T14:05:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.594+0000 7ff4c37fe640 1 -- 172.21.15.33:0/772968692 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc004c30 con 0x7ff4cc102370 2024-05-15T14:05:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.594+0000 7ff4d450a640 1 -- 172.21.15.33:0/772968692 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4cc1a5820 con 0x7ff4cc102370 2024-05-15T14:05:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.594+0000 7ff4d450a640 1 -- 172.21.15.33:0/772968692 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4cc1a5cf0 con 0x7ff4cc102370 2024-05-15T14:05:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.596+0000 7ff4d7225740 1 Processor -- start 2024-05-15T14:05:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.596+0000 7ff4c37fe640 1 -- 172.21.15.33:0/772968692 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff4bc004e10 con 0x7ff4cc102370 2024-05-15T14:05:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.597+0000 7ff4d7225740 1 -- start start 2024-05-15T14:05:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.597+0000 7ff4d7225740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab1bcfb7d0 0x55ab1bcfbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.597+0000 7ff4d7225740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab1bcfc0f0 0x55ab1bd0a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.597+0000 7ff4d7225740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab1bd0aee0 0x55ab1bd0d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.597+0000 7ff4d7225740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ab1bc6ec30 con 0x55ab1bcfc0f0 2024-05-15T14:05:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.597+0000 7ff4d7225740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ab1bc5e7d0 con 0x55ab1bcfb7d0 2024-05-15T14:05:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.597+0000 7ff4d7225740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55ab1bc62810 con 0x55ab1bd0aee0 2024-05-15T14:05:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.597+0000 7ff4d22a6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab1bcfc0f0 0x55ab1bd0a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.597+0000 7ff4d22a6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab1bcfc0f0 0x55ab1bd0a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43522/0 (socket says 172.21.15.33:43522) 2024-05-15T14:05:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.597+0000 7ff4d22a6640 1 -- 172.21.15.33:0/3348256294 learned_addr learned my addr 172.21.15.33:0/3348256294 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.597+0000 7ff4d1aa5640 1 --2- 172.21.15.33:0/3348256294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab1bd0aee0 0x55ab1bd0d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.597+0000 7ff4d2aa7640 1 --2- 172.21.15.33:0/3348256294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab1bcfb7d0 0x55ab1bcfbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.597+0000 7ff4d22a6640 1 -- 172.21.15.33:0/3348256294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab1bd0aee0 msgr2=0x55ab1bd0d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.597+0000 7ff4d22a6640 1 --2- 172.21.15.33:0/3348256294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab1bd0aee0 0x55ab1bd0d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.597+0000 7ff4d22a6640 1 -- 172.21.15.33:0/3348256294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab1bcfb7d0 msgr2=0x55ab1bcfbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.597+0000 7ff4d22a6640 1 --2- 172.21.15.33:0/3348256294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab1bcfb7d0 0x55ab1bcfbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.597+0000 7ff4d22a6640 1 -- 172.21.15.33:0/3348256294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab1b973210 con 0x55ab1bcfc0f0 2024-05-15T14:05:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.597+0000 7ff4d2aa7640 1 --2- 172.21.15.33:0/3348256294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab1bcfb7d0 0x55ab1bcfbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:05:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.598+0000 7ff4c37fe640 1 --2- 172.21.15.33:0/772968692 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ff4b80b2950 0x7ff4b80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.598+0000 7ff4d22a6640 1 --2- 172.21.15.33:0/3348256294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab1bcfc0f0 0x55ab1bd0a9a0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009df0 tx=0x7ff4c8009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96875bbbc7141e4b server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.598+0000 7ff4d2aa7640 1 --2- 172.21.15.33:0/772968692 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ff4b80b2950 0x7ff4b80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.598+0000 7ff4c37fe640 1 -- 172.21.15.33:0/772968692 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7ff4bc0e2180 con 0x7ff4cc102370 2024-05-15T14:05:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.598+0000 7ff4c0bfe640 1 -- 172.21.15.33:0/3348256294 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c8031a50 con 0x55ab1bcfc0f0 2024-05-15T14:05:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.598+0000 7ff4c0bfe640 1 -- 172.21.15.33:0/3348256294 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4c8036040 con 0x55ab1bcfc0f0 2024-05-15T14:05:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.598+0000 7ff4c0bfe640 1 -- 172.21.15.33:0/3348256294 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c8004540 con 0x55ab1bcfc0f0 2024-05-15T14:05:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.599+0000 7ff4d7225740 1 -- 172.21.15.33:0/3348256294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab1bcfc0f0 msgr2=0x55ab1bd0a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.599+0000 7ff4d2aa7640 1 --2- 172.21.15.33:0/772968692 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ff4b80b2950 0x7ff4b80b4e10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4005fd0 tx=0x7ff4c4005f60 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.599+0000 7ff4d7225740 1 --2- 172.21.15.33:0/3348256294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab1bcfc0f0 0x55ab1bd0a9a0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009df0 tx=0x7ff4c8009a20 comp rx=0 tx=0).stop 2024-05-15T14:05:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.599+0000 7ff4d7225740 1 -- 172.21.15.33:0/3348256294 shutdown_connections 2024-05-15T14:05:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.599+0000 7ff4d7225740 1 --2- 172.21.15.33:0/3348256294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab1bd0aee0 0x55ab1bd0d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.599+0000 7ff4d7225740 1 --2- 172.21.15.33:0/3348256294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab1bcfc0f0 0x55ab1bd0a9a0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.599+0000 7ff4d7225740 1 --2- 172.21.15.33:0/3348256294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab1bcfb7d0 0x55ab1bcfbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.599+0000 7ff4d7225740 1 -- 172.21.15.33:0/3348256294 >> 172.21.15.33:0/3348256294 conn(0x55ab1bcf7e00 msgr2=0x55ab1bcf81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.599+0000 7ff4d7225740 1 -- 172.21.15.33:0/3348256294 shutdown_connections 2024-05-15T14:05:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.599+0000 7ff4d7225740 1 -- 172.21.15.33:0/3348256294 wait complete. 2024-05-15T14:05:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.599+0000 7ff4d7225740 1 -- start start 2024-05-15T14:05:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.599+0000 7ff4d7225740 1 Processor -- start 2024-05-15T14:05:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.600+0000 7ff4d7225740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab1bcfb7d0 0x55ab1bd0a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.600+0000 7ff4d7225740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab1bcfc0f0 0x55ab1bd07b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.600+0000 7ff4d7225740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab1bd0aee0 0x55ab1bd08120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.600+0000 7ff4d7225740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ab1b944790 con 0x55ab1bd0aee0 2024-05-15T14:05:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.600+0000 7ff4d7225740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ab1bc1a890 con 0x55ab1bcfb7d0 2024-05-15T14:05:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.600+0000 7ff4d7225740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55ab1bd0db50 con 0x55ab1bcfc0f0 2024-05-15T14:05:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.600+0000 7ff4d7225740 10 client.0 _ll_register_callbacks cb 0x55ab1bc1dd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:05:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.600+0000 7ff4d2aa7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab1bd0aee0 0x55ab1bd08120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.600+0000 7ff4d2aa7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab1bd0aee0 0x55ab1bd08120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43528/0 (socket says 172.21.15.33:43528) 2024-05-15T14:05:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.600+0000 7ff4d2aa7640 1 -- 172.21.15.33:0/1615238497 learned_addr learned my addr 172.21.15.33:0/1615238497 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.600+0000 7ff4d22a6640 1 --2- 172.21.15.33:0/1615238497 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab1bcfb7d0 0x55ab1bd0a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.600+0000 7ff4d1aa5640 1 --2- 172.21.15.33:0/1615238497 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab1bcfc0f0 0x55ab1bd07b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.600+0000 7ff4d2aa7640 1 -- 172.21.15.33:0/1615238497 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab1bcfc0f0 msgr2=0x55ab1bd07b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.600+0000 7ff4d2aa7640 1 --2- 172.21.15.33:0/1615238497 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab1bcfc0f0 0x55ab1bd07b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.600+0000 7ff4d2aa7640 1 -- 172.21.15.33:0/1615238497 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab1bcfb7d0 msgr2=0x55ab1bd0a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.600+0000 7ff4d2aa7640 1 --2- 172.21.15.33:0/1615238497 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab1bcfb7d0 0x55ab1bd0a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.600+0000 7ff4d2aa7640 1 -- 172.21.15.33:0/1615238497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab1ba3f620 con 0x55ab1bd0aee0 2024-05-15T14:05:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.600+0000 7ff4d1aa5640 1 --2- 172.21.15.33:0/1615238497 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab1bcfc0f0 0x55ab1bd07b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:05:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.600+0000 7ff4d22a6640 1 --2- 172.21.15.33:0/1615238497 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab1bcfb7d0 0x55ab1bd0a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:05:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.600+0000 7ff4d2aa7640 1 --2- 172.21.15.33:0/1615238497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab1bd0aee0 0x55ab1bd08120 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4012010 tx=0x7ff4c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c84345c452bce8 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.600+0000 7ff4b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:05:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.600+0000 7ff4b5ffb640 1 -- 172.21.15.33:0/1615238497 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c400fd60 con 0x55ab1bd0aee0 2024-05-15T14:05:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.600+0000 7ff4b5ffb640 1 -- 172.21.15.33:0/1615238497 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7ff4c401f420 con 0x55ab1bd0aee0 2024-05-15T14:05:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.600+0000 7ff4d7225740 1 -- 172.21.15.33:0/1615238497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4c8036040 con 0x55ab1bd0aee0 2024-05-15T14:05:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.601+0000 7ff4b5ffb640 1 -- 172.21.15.33:0/1615238497 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff4c401f620 con 0x55ab1bd0aee0 2024-05-15T14:05:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.601+0000 7ff4d7225740 10 client.15999 fetch_fsmap learned FSMap version 29 2024-05-15T14:05:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.601+0000 7ff4d7225740 1 -- 172.21.15.33:0/1615238497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ab1bd19600 con 0x55ab1bd0aee0 2024-05-15T14:05:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.601+0000 7ff4b5ffb640 1 -- 172.21.15.33:0/1615238497 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4c8036040 con 0x55ab1bd0aee0 2024-05-15T14:05:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.601+0000 7ff4b5ffb640 1 -- 172.21.15.33:0/1615238497 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7ff4c4030800 con 0x55ab1bd0aee0 2024-05-15T14:05:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.601+0000 7ff4d7225740 10 client.15999 fetch_fsmap finished waiting for FSMap version 29 2024-05-15T14:05:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.601+0000 7ff4d7225740 10 client.15999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:05:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.601+0000 7ff4d7225740 20 client.15999 populate_metadata read hostname 'smithi033' 2024-05-15T14:05:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.601+0000 7ff4d7225740 1 --2- 172.21.15.33:0/1615238497 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55ab1bd1aac0 0x55ab1bd1ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.601+0000 7ff4d7225740 4 client.15999 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:05:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.601+0000 7ff4d7225740 1 -- 172.21.15.33:0/1615238497 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab1bc70b80 con 0x55ab1bd1aac0 2024-05-15T14:05:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.601+0000 7ff4d22a6640 1 --2- 172.21.15.33:0/1615238497 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55ab1bd1aac0 0x55ab1bd1ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.602+0000 7ff4b5ffb640 10 client.15999 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:05:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.602+0000 7ff4d22a6640 1 --2- 172.21.15.33:0/1615238497 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55ab1bd1aac0 0x55ab1bd1ceb0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a20374e919d5b9ba server_cookie=b50f273a1776cbad in_seq=0 out_seq=0 2024-05-15T14:05:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.602+0000 7ff4b5ffb640 1 -- 172.21.15.33:0/1615238497 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ab1bc70b80 con 0x55ab1bd1aac0 2024-05-15T14:05:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.602+0000 7ff4b5ffb640 10 client.15999 handle_command_reply: tid=0 2024-05-15T14:05:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.602+0000 7ff4d7225740 1 client.15999 shutdown 2024-05-15T14:05:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.602+0000 7ff4d7225740 1 -- 172.21.15.33:0/1615238497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab1bd0aee0 msgr2=0x55ab1bd08120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.602+0000 7ff4d7225740 1 --2- 172.21.15.33:0/1615238497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab1bd0aee0 0x55ab1bd08120 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4012010 tx=0x7ff4c40046c0 comp rx=0 tx=0).stop 2024-05-15T14:05:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.602+0000 7ff4d7225740 1 -- 172.21.15.33:0/1615238497 shutdown_connections 2024-05-15T14:05:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.602+0000 7ff4d7225740 1 --2- 172.21.15.33:0/1615238497 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55ab1bd1aac0 0x55ab1bd1ceb0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.602+0000 7ff4d7225740 1 --2- 172.21.15.33:0/1615238497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab1bd0aee0 0x55ab1bd08120 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.602+0000 7ff4d7225740 1 --2- 172.21.15.33:0/1615238497 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab1bcfc0f0 0x55ab1bd07b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.602+0000 7ff4b5ffb640 0 client.15999 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:05:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.602+0000 7ff4d7225740 1 --2- 172.21.15.33:0/1615238497 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab1bcfb7d0 0x55ab1bd0a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.602+0000 7ff4d7225740 1 -- 172.21.15.33:0/1615238497 >> 172.21.15.33:0/1615238497 conn(0x55ab1bcf7e00 msgr2=0x55ab1bcfdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.603+0000 7ff4d7225740 1 -- 172.21.15.33:0/1615238497 shutdown_connections 2024-05-15T14:05:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.603+0000 7ff4d7225740 1 -- 172.21.15.33:0/1615238497 wait complete. 2024-05-15T14:05:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.603+0000 7ff4d7225740 20 client.15999 trim_cache size 0 max 16384 2024-05-15T14:05:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.618+0000 7ff4d7225740 1 Processor -- start 2024-05-15T14:05:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.618+0000 7ff4d7225740 1 -- start start 2024-05-15T14:05:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.618+0000 7ff4d7225740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab1bcfb7d0 0x55ab1bd16a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.618+0000 7ff4d7225740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab1bcfc0f0 0x55ab1bd0a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.618+0000 7ff4d7225740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab1bd0aee0 0x55ab1bd22770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.618+0000 7ff4d7225740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff4cc1aed10 con 0x55ab1bd0aee0 2024-05-15T14:05:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.618+0000 7ff4d7225740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff4cc1ac5e0 con 0x55ab1bcfc0f0 2024-05-15T14:05:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.618+0000 7ff4d7225740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ff4cc1aa430 con 0x55ab1bcfb7d0 2024-05-15T14:05:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.618+0000 7ff4d2aa7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab1bd0aee0 0x55ab1bd22770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.618+0000 7ff4d2aa7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab1bd0aee0 0x55ab1bd22770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43544/0 (socket says 172.21.15.33:43544) 2024-05-15T14:05:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.618+0000 7ff4d2aa7640 1 -- 172.21.15.33:0/2415139617 learned_addr learned my addr 172.21.15.33:0/2415139617 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.618+0000 7ff4d22a6640 1 --2- 172.21.15.33:0/2415139617 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab1bcfb7d0 0x55ab1bd16a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.618+0000 7ff4d1aa5640 1 --2- 172.21.15.33:0/2415139617 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab1bcfc0f0 0x55ab1bd0a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.618+0000 7ff4d2aa7640 1 -- 172.21.15.33:0/2415139617 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab1bcfb7d0 msgr2=0x55ab1bd16a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.618+0000 7ff4d2aa7640 1 --2- 172.21.15.33:0/2415139617 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab1bcfb7d0 0x55ab1bd16a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.618+0000 7ff4d2aa7640 1 -- 172.21.15.33:0/2415139617 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab1bcfc0f0 msgr2=0x55ab1bd0a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.618+0000 7ff4d2aa7640 1 --2- 172.21.15.33:0/2415139617 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab1bcfc0f0 0x55ab1bd0a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.618+0000 7ff4d2aa7640 1 -- 172.21.15.33:0/2415139617 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab1bcf7880 con 0x55ab1bd0aee0 2024-05-15T14:05:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.619+0000 7ff4d2aa7640 1 --2- 172.21.15.33:0/2415139617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab1bd0aee0 0x55ab1bd22770 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4011b90 tx=0x7ff4c400f7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a86f00fefd9b80ac server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.619+0000 7ff4b7fff640 1 -- 172.21.15.33:0/2415139617 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c400fb60 con 0x55ab1bd0aee0 2024-05-15T14:05:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.619+0000 7ff4b7fff640 1 -- 172.21.15.33:0/2415139617 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4c40308d0 con 0x55ab1bd0aee0 2024-05-15T14:05:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.619+0000 7ff4b7fff640 1 -- 172.21.15.33:0/2415139617 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4005610 con 0x55ab1bd0aee0 2024-05-15T14:05:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.619+0000 7ff4d7225740 1 -- 172.21.15.33:0/2415139617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab1bd0aee0 msgr2=0x55ab1bd22770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.619+0000 7ff4d7225740 1 --2- 172.21.15.33:0/2415139617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab1bd0aee0 0x55ab1bd22770 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4011b90 tx=0x7ff4c400f7a0 comp rx=0 tx=0).stop 2024-05-15T14:05:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.619+0000 7ff4d7225740 1 -- 172.21.15.33:0/2415139617 shutdown_connections 2024-05-15T14:05:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.619+0000 7ff4d7225740 1 --2- 172.21.15.33:0/2415139617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab1bd0aee0 0x55ab1bd22770 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.619+0000 7ff4d7225740 1 --2- 172.21.15.33:0/2415139617 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab1bcfc0f0 0x55ab1bd0a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.619+0000 7ff4d7225740 1 --2- 172.21.15.33:0/2415139617 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab1bcfb7d0 0x55ab1bd16a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.619+0000 7ff4d7225740 1 -- 172.21.15.33:0/2415139617 >> 172.21.15.33:0/2415139617 conn(0x55ab1bcf7e00 msgr2=0x55ab1bcf2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.619+0000 7ff4d7225740 1 -- 172.21.15.33:0/2415139617 shutdown_connections 2024-05-15T14:05:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.619+0000 7ff4d7225740 1 -- 172.21.15.33:0/2415139617 wait complete. 2024-05-15T14:05:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.620+0000 7ff4d7225740 1 -- start start 2024-05-15T14:05:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.620+0000 7ff4d7225740 1 Processor -- start 2024-05-15T14:05:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.620+0000 7ff4d7225740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab1bcfb7d0 0x55ab1bd13270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.620+0000 7ff4d7225740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab1bcfc0f0 0x55ab1bd0e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.620+0000 7ff4d7225740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab1bd0aee0 0x55ab1bd0eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.620+0000 7ff4d7225740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff4cc1aaaa0 con 0x55ab1bcfb7d0 2024-05-15T14:05:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.620+0000 7ff4d7225740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff4cc1abf10 con 0x55ab1bd0aee0 2024-05-15T14:05:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.620+0000 7ff4d7225740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ff4cc1ab170 con 0x55ab1bcfc0f0 2024-05-15T14:05:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.620+0000 7ff4d7225740 10 client.0 _ll_register_callbacks cb 0x7ff4c4012680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:05:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.620+0000 7ff4d22a6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab1bcfb7d0 0x55ab1bd13270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.620+0000 7ff4d22a6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab1bcfb7d0 0x55ab1bd13270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43548/0 (socket says 172.21.15.33:43548) 2024-05-15T14:05:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.620+0000 7ff4d22a6640 1 -- 172.21.15.33:0/391825965 learned_addr learned my addr 172.21.15.33:0/391825965 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.620+0000 7ff4d1aa5640 1 --2- 172.21.15.33:0/391825965 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab1bcfc0f0 0x55ab1bd0e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.620+0000 7ff4d2aa7640 1 --2- 172.21.15.33:0/391825965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab1bd0aee0 0x55ab1bd0eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.620+0000 7ff4d22a6640 1 -- 172.21.15.33:0/391825965 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab1bcfc0f0 msgr2=0x55ab1bd0e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.620+0000 7ff4d22a6640 1 --2- 172.21.15.33:0/391825965 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab1bcfc0f0 0x55ab1bd0e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.620+0000 7ff4d22a6640 1 -- 172.21.15.33:0/391825965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab1bd0aee0 msgr2=0x55ab1bd0eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.620+0000 7ff4d22a6640 1 --2- 172.21.15.33:0/391825965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab1bd0aee0 0x55ab1bd0eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.620+0000 7ff4d22a6640 1 -- 172.21.15.33:0/391825965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab1bd0f320 con 0x55ab1bcfb7d0 2024-05-15T14:05:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.620+0000 7ff4d1aa5640 1 --2- 172.21.15.33:0/391825965 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab1bcfc0f0 0x55ab1bd0e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:05:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.620+0000 7ff4d2aa7640 1 --2- 172.21.15.33:0/391825965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab1bd0aee0 0x55ab1bd0eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:05:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.621+0000 7ff4d22a6640 1 --2- 172.21.15.33:0/391825965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab1bcfb7d0 0x55ab1bd13270 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8005cc0 tx=0x7ff4c80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12643df89cb067d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.621+0000 7ff4b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:05:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.621+0000 7ff4b5ffb640 1 -- 172.21.15.33:0/391825965 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c8054020 con 0x55ab1bcfb7d0 2024-05-15T14:05:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.621+0000 7ff4b5ffb640 1 -- 172.21.15.33:0/391825965 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7ff4c8031ae0 con 0x55ab1bcfb7d0 2024-05-15T14:05:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.621+0000 7ff4d7225740 1 -- 172.21.15.33:0/391825965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4c40308d0 con 0x55ab1bcfb7d0 2024-05-15T14:05:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.621+0000 7ff4b5ffb640 1 -- 172.21.15.33:0/391825965 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff4c8004b40 con 0x55ab1bcfb7d0 2024-05-15T14:05:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.621+0000 7ff4d7225740 10 client.16005 fetch_fsmap learned FSMap version 29 2024-05-15T14:05:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.621+0000 7ff4d7225740 1 -- 172.21.15.33:0/391825965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ab1bd85270 con 0x55ab1bcfb7d0 2024-05-15T14:05:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.621+0000 7ff4b5ffb640 1 -- 172.21.15.33:0/391825965 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4c40308d0 con 0x55ab1bcfb7d0 2024-05-15T14:05:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.621+0000 7ff4b5ffb640 1 -- 172.21.15.33:0/391825965 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7ff4c8030070 con 0x55ab1bcfb7d0 2024-05-15T14:05:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.621+0000 7ff4d7225740 10 client.16005 fetch_fsmap finished waiting for FSMap version 29 2024-05-15T14:05:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.621+0000 7ff4d7225740 10 client.16005 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:05:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.621+0000 7ff4d7225740 20 client.16005 populate_metadata read hostname 'smithi033' 2024-05-15T14:05:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.621+0000 7ff4d7225740 1 --2- 172.21.15.33:0/391825965 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55ab1bd85610 0x55ab1bd87a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.621+0000 7ff4d7225740 4 client.16005 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:05:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.621+0000 7ff4d7225740 1 -- 172.21.15.33:0/391825965 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ab1bc6a360 con 0x55ab1bd85610 2024-05-15T14:05:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.622+0000 7ff4d1aa5640 1 --2- 172.21.15.33:0/391825965 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55ab1bd85610 0x55ab1bd87a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.622+0000 7ff4d1aa5640 1 --2- 172.21.15.33:0/391825965 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55ab1bd85610 0x55ab1bd87a00 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddd2b9a973eedb85 server_cookie=fe57191f170c38a2 in_seq=0 out_seq=0 2024-05-15T14:05:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.622+0000 7ff4b5ffb640 10 client.16005 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:05:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.660+0000 7ff4b5ffb640 1 -- 172.21.15.33:0/391825965 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ab1bc6a360 con 0x55ab1bd85610 2024-05-15T14:05:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.660+0000 7ff4b5ffb640 10 client.16005 handle_command_reply: tid=0 2024-05-15T14:05:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.660+0000 7ff4d7225740 1 client.16005 shutdown 2024-05-15T14:05:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.660+0000 7ff4d7225740 1 -- 172.21.15.33:0/391825965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab1bcfb7d0 msgr2=0x55ab1bd13270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.660+0000 7ff4d7225740 1 --2- 172.21.15.33:0/391825965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab1bcfb7d0 0x55ab1bd13270 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8005cc0 tx=0x7ff4c80095c0 comp rx=0 tx=0).stop 2024-05-15T14:05:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.660+0000 7ff4d7225740 1 -- 172.21.15.33:0/391825965 shutdown_connections 2024-05-15T14:05:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.660+0000 7ff4d7225740 1 --2- 172.21.15.33:0/391825965 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55ab1bd85610 0x55ab1bd87a00 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.660+0000 7ff4d7225740 1 --2- 172.21.15.33:0/391825965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab1bd0aee0 0x55ab1bd0eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.660+0000 7ff4d7225740 1 --2- 172.21.15.33:0/391825965 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ab1bcfc0f0 0x55ab1bd0e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.660+0000 7ff4b5ffb640 0 client.16005 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:05:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.660+0000 7ff4d7225740 1 --2- 172.21.15.33:0/391825965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab1bcfb7d0 0x55ab1bd13270 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.660+0000 7ff4d7225740 1 -- 172.21.15.33:0/391825965 >> 172.21.15.33:0/391825965 conn(0x55ab1bcf7e00 msgr2=0x55ab1bd02940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.660+0000 7ff4d7225740 1 -- 172.21.15.33:0/391825965 shutdown_connections 2024-05-15T14:05:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.660+0000 7ff4d7225740 1 -- 172.21.15.33:0/391825965 wait complete. 2024-05-15T14:05:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.660+0000 7ff4d7225740 20 client.16005 trim_cache size 0 max 16384 2024-05-15T14:05:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:05:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:05:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:05:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.661+0000 7ff4d450a640 1 -- 172.21.15.33:0/772968692 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ff4b80b2950 msgr2=0x7ff4b80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.661+0000 7ff4d450a640 1 --2- 172.21.15.33:0/772968692 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ff4b80b2950 0x7ff4b80b4e10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4005fd0 tx=0x7ff4c4005f60 comp rx=0 tx=0).stop 2024-05-15T14:05:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.661+0000 7ff4d450a640 1 -- 172.21.15.33:0/772968692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4cc102370 msgr2=0x7ff4cc19a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.661+0000 7ff4d450a640 1 --2- 172.21.15.33:0/772968692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4cc102370 0x7ff4cc19a570 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00d770 tx=0x7ff4bc00dc30 comp rx=0 tx=0).stop 2024-05-15T14:05:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.661+0000 7ff4d450a640 1 -- 172.21.15.33:0/772968692 shutdown_connections 2024-05-15T14:05:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.661+0000 7ff4d450a640 1 --2- 172.21.15.33:0/772968692 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ff4b80b2950 0x7ff4b80b4e10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.661+0000 7ff4d450a640 1 --2- 172.21.15.33:0/772968692 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff4cc107970 0x7ff4cc1a50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.661+0000 7ff4d450a640 1 --2- 172.21.15.33:0/772968692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4cc102370 0x7ff4cc19a570 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.661+0000 7ff4d450a640 1 --2- 172.21.15.33:0/772968692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4cc1019b0 0x7ff4cc19a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.661+0000 7ff4d450a640 1 -- 172.21.15.33:0/772968692 >> 172.21.15.33:0/772968692 conn(0x7ff4cc078aa0 msgr2=0x7ff4cc1042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.661+0000 7ff4d450a640 1 -- 172.21.15.33:0/772968692 shutdown_connections 2024-05-15T14:05:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:05:53.661+0000 7ff4d450a640 1 -- 172.21.15.33:0/772968692 wait complete. 2024-05-15T14:05:54.059 INFO:tasks.workunit.client.0.smithi033.stdout: 1 331660 7.13 MB/sec execute 89 sec latency 14175.384 ms 2024-05-15T14:05:54.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.342+0000 7feb786e6700 1 -- 172.21.15.33:0/2053110250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb701037f0 msgr2=0x7feb70103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:54.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.342+0000 7feb786e6700 1 --2- 172.21.15.33:0/2053110250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb701037f0 0x7feb70103bd0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7feb6c0099c0 tx=0x7feb6c009cd0 comp rx=0 tx=0).stop 2024-05-15T14:05:54.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.342+0000 7feb786e6700 1 -- 172.21.15.33:0/2053110250 shutdown_connections 2024-05-15T14:05:54.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.342+0000 7feb786e6700 1 --2- 172.21.15.33:0/2053110250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb701088f0 0x7feb70110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:54.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.342+0000 7feb786e6700 1 --2- 172.21.15.33:0/2053110250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb701037f0 0x7feb70103bd0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:54.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.342+0000 7feb786e6700 1 --2- 172.21.15.33:0/2053110250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb701041a0 0x7feb701081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:54.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.342+0000 7feb786e6700 1 -- 172.21.15.33:0/2053110250 >> 172.21.15.33:0/2053110250 conn(0x7feb7007a6a0 msgr2=0x7feb7007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:54.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.342+0000 7feb786e6700 1 -- 172.21.15.33:0/2053110250 shutdown_connections 2024-05-15T14:05:54.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.342+0000 7feb786e6700 1 -- 172.21.15.33:0/2053110250 wait complete. 2024-05-15T14:05:54.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.342+0000 7feb786e6700 1 Processor -- start 2024-05-15T14:05:54.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.342+0000 7feb786e6700 1 -- start start 2024-05-15T14:05:54.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.343+0000 7feb786e6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb701037f0 0x7feb7019f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:54.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.343+0000 7feb786e6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb701041a0 0x7feb7019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:54.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.343+0000 7feb786e6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb701088f0 0x7feb70199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:54.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.343+0000 7feb786e6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feb70113760 con 0x7feb701088f0 2024-05-15T14:05:54.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.343+0000 7feb786e6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feb701135e0 con 0x7feb701041a0 2024-05-15T14:05:54.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.343+0000 7feb786e6700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7feb701138e0 con 0x7feb701037f0 2024-05-15T14:05:54.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.343+0000 7feb76c83700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb701088f0 0x7feb70199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:54.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.343+0000 7feb76c83700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb701088f0 0x7feb70199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43568/0 (socket says 172.21.15.33:43568) 2024-05-15T14:05:54.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.343+0000 7feb76c83700 1 -- 172.21.15.33:0/514938425 learned_addr learned my addr 172.21.15.33:0/514938425 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:54.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.343+0000 7feb76482700 1 --2- 172.21.15.33:0/514938425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb701037f0 0x7feb7019f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:54.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.343+0000 7feb75c81700 1 --2- 172.21.15.33:0/514938425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb701041a0 0x7feb7019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:54.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.343+0000 7feb76c83700 1 -- 172.21.15.33:0/514938425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb701037f0 msgr2=0x7feb7019f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:54.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.343+0000 7feb76c83700 1 --2- 172.21.15.33:0/514938425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb701037f0 0x7feb7019f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:54.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.343+0000 7feb76c83700 1 -- 172.21.15.33:0/514938425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb701041a0 msgr2=0x7feb7019fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:54.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.343+0000 7feb76c83700 1 --2- 172.21.15.33:0/514938425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb701041a0 0x7feb7019fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:54.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.343+0000 7feb76c83700 1 -- 172.21.15.33:0/514938425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb7019a180 con 0x7feb701088f0 2024-05-15T14:05:54.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.343+0000 7feb76482700 1 --2- 172.21.15.33:0/514938425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb701037f0 0x7feb7019f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:05:54.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.343+0000 7feb76c83700 1 --2- 172.21.15.33:0/514938425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb701088f0 0x7feb70199a20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7feb6800b970 tx=0x7feb6800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:54.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.344+0000 7feb637fe700 1 -- 172.21.15.33:0/514938425 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6800d860 con 0x7feb701088f0 2024-05-15T14:05:54.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.344+0000 7feb637fe700 1 -- 172.21.15.33:0/514938425 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb6800e430 con 0x7feb701088f0 2024-05-15T14:05:54.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.344+0000 7feb786e6700 1 -- 172.21.15.33:0/514938425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb7019a470 con 0x7feb701088f0 2024-05-15T14:05:54.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.344+0000 7feb637fe700 1 -- 172.21.15.33:0/514938425 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb68017660 con 0x7feb701088f0 2024-05-15T14:05:54.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.344+0000 7feb786e6700 1 -- 172.21.15.33:0/514938425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb701a6730 con 0x7feb701088f0 2024-05-15T14:05:54.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.345+0000 7feb786e6700 1 -- 172.21.15.33:0/514938425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb7010bb30 con 0x7feb701088f0 2024-05-15T14:05:54.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.346+0000 7feb637fe700 1 -- 172.21.15.33:0/514938425 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7feb6800da00 con 0x7feb701088f0 2024-05-15T14:05:54.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.347+0000 7feb637fe700 1 --2- 172.21.15.33:0/514938425 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7feb5c0b2a70 0x7feb5c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:54.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.347+0000 7feb76482700 1 --2- 172.21.15.33:0/514938425 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7feb5c0b2a70 0x7feb5c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:54.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.347+0000 7feb637fe700 1 -- 172.21.15.33:0/514938425 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7feb680e25a0 con 0x7feb701088f0 2024-05-15T14:05:54.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.347+0000 7feb76482700 1 --2- 172.21.15.33:0/514938425 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7feb5c0b2a70 0x7feb5c0b4f30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7feb7019ab60 tx=0x7feb6c000bc0 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:54.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.350+0000 7feb637fe700 1 -- 172.21.15.33:0/514938425 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7feb68013030 con 0x7feb701088f0 2024-05-15T14:05:54.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:54 smithi052 ceph-mon[36190]: pgmap v170: 97 pgs: 97 active+clean; 84 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 2.8 MiB/s wr, 184 op/s 2024-05-15T14:05:54.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:54 smithi081 ceph-mon[36434]: pgmap v170: 97 pgs: 97 active+clean; 84 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 2.8 MiB/s wr, 184 op/s 2024-05-15T14:05:54.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.616+0000 7feb786e6700 1 -- 172.21.15.33:0/514938425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb7019af50 con 0x7feb701088f0 2024-05-15T14:05:54.619 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:54 smithi033 ceph-mon[31650]: pgmap v170: 97 pgs: 97 active+clean; 84 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 2.8 MiB/s wr, 184 op/s 2024-05-15T14:05:54.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.617+0000 7feb637fe700 1 -- 172.21.15.33:0/514938425 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11855 (secure 0 0 0) 0x7feb680aa010 con 0x7feb701088f0 2024-05-15T14:05:54.620 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T14:05:54.620 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2030117633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2030117633},{"type":"v1","addr":"172.21.15.33:6839","nonce":2030117633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1918979061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1918979061},{"type":"v1","addr":"172.21.15.33:6841","nonce":1918979061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T13:57:19.056417+0000","modified":"2024-05-15T14:05:13.682039+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24463,"mds_2":24461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/1629697841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1629697841},{"type":"v1","addr":"172.21.15.52:6837","nonce":1629697841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.52:6839/2696044471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2696044471},{"type":"v1","addr":"172.21.15.52:6839","nonce":2696044471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.81:6839/1554395940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":1554395940},{"type":"v1","addr":"172.21.15.81:6839","nonce":1554395940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25393":{"gid":25393,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/727032511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":727032511},{"type":"v1","addr":"172.21.15.81:6835","nonce":727032511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24478,24461,24463]},"id":1}]} 2024-05-15T14:05:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.619+0000 7feb786e6700 1 -- 172.21.15.33:0/514938425 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7feb5c0b2a70 msgr2=0x7feb5c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.619+0000 7feb786e6700 1 --2- 172.21.15.33:0/514938425 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7feb5c0b2a70 0x7feb5c0b4f30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7feb7019ab60 tx=0x7feb6c000bc0 comp rx=0 tx=0).stop 2024-05-15T14:05:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.619+0000 7feb786e6700 1 -- 172.21.15.33:0/514938425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb701088f0 msgr2=0x7feb70199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.619+0000 7feb786e6700 1 --2- 172.21.15.33:0/514938425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb701088f0 0x7feb70199a20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7feb6800b970 tx=0x7feb6800bc80 comp rx=0 tx=0).stop 2024-05-15T14:05:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.619+0000 7feb786e6700 1 -- 172.21.15.33:0/514938425 shutdown_connections 2024-05-15T14:05:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.620+0000 7feb786e6700 1 --2- 172.21.15.33:0/514938425 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7feb5c0b2a70 0x7feb5c0b4f30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.620+0000 7feb786e6700 1 --2- 172.21.15.33:0/514938425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb701088f0 0x7feb70199a20 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.620+0000 7feb786e6700 1 --2- 172.21.15.33:0/514938425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feb701037f0 0x7feb7019f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.620+0000 7feb786e6700 1 --2- 172.21.15.33:0/514938425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb701041a0 0x7feb7019fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.620+0000 7feb786e6700 1 -- 172.21.15.33:0/514938425 >> 172.21.15.33:0/514938425 conn(0x7feb7007a6a0 msgr2=0x7feb70109a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.620+0000 7feb786e6700 1 -- 172.21.15.33:0/514938425 shutdown_connections 2024-05-15T14:05:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:54.620+0000 7feb786e6700 1 -- 172.21.15.33:0/514938425 wait complete. 2024-05-15T14:05:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 29 2024-05-15T14:05:55.059 INFO:tasks.workunit.client.0.smithi033.stdout: 1 331660 7.05 MB/sec execute 90 sec latency 15175.478 ms 2024-05-15T14:05:55.221 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24478 quiesce db --set-id 417f377d --release --await' 2024-05-15T14:05:55.222 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24478 quiesce db --set-id 417f377d --release --await 2024-05-15T14:05:55.475 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:05:55.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:55 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/514938425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:05:55.597 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:55 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/514938425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:05:55.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:55 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/514938425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:05:56.059 INFO:tasks.workunit.client.0.smithi033.stdout: 1 331660 6.97 MB/sec execute 91 sec latency 16175.591 ms 2024-05-15T14:05:56.548 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:56 smithi033 ceph-mon[31650]: pgmap v171: 97 pgs: 97 active+clean; 84 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 1.5 MiB/s wr, 58 op/s 2024-05-15T14:05:56.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:56 smithi052 ceph-mon[36190]: pgmap v171: 97 pgs: 97 active+clean; 84 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 1.5 MiB/s wr, 58 op/s 2024-05-15T14:05:56.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:56 smithi081 ceph-mon[36434]: pgmap v171: 97 pgs: 97 active+clean; 84 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 1.5 MiB/s wr, 58 op/s 2024-05-15T14:05:57.059 INFO:tasks.workunit.client.0.smithi033.stdout: 1 331660 6.89 MB/sec execute 92 sec latency 17175.655 ms 2024-05-15T14:05:57.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.063+0000 7fcbb395b700 1 -- 172.21.15.33:0/1123919772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbac109a00 msgr2=0x7fcbac110430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:57.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.063+0000 7fcbb395b700 1 --2- 172.21.15.33:0/1123919772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbac109a00 0x7fcbac110430 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fcba400b0d0 tx=0x7fcba400b3e0 comp rx=0 tx=0).stop 2024-05-15T14:05:57.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.063+0000 7fcbb395b700 1 -- 172.21.15.33:0/1123919772 shutdown_connections 2024-05-15T14:05:57.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.063+0000 7fcbb395b700 1 --2- 172.21.15.33:0/1123919772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbac109a00 0x7fcbac110430 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:57.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.063+0000 7fcbb395b700 1 --2- 172.21.15.33:0/1123919772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbac101920 0x7fcbac101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:57.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.063+0000 7fcbb395b700 1 --2- 172.21.15.33:0/1123919772 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbac1022d0 0x7fcbac109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:57.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.063+0000 7fcbb395b700 1 -- 172.21.15.33:0/1123919772 >> 172.21.15.33:0/1123919772 conn(0x7fcbac078640 msgr2=0x7fcbac0ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:57.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.063+0000 7fcbb395b700 1 -- 172.21.15.33:0/1123919772 shutdown_connections 2024-05-15T14:05:57.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.063+0000 7fcbb395b700 1 -- 172.21.15.33:0/1123919772 wait complete. 2024-05-15T14:05:57.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.064+0000 7fcbb395b700 1 Processor -- start 2024-05-15T14:05:57.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.064+0000 7fcbb395b700 1 -- start start 2024-05-15T14:05:57.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.064+0000 7fcbb395b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbac101920 0x7fcbac19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:57.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.064+0000 7fcbb395b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbac1022d0 0x7fcbac19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:57.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.064+0000 7fcbb395b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbac109a00 0x7fcbac199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:57.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.064+0000 7fcbb395b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac113710 con 0x7fcbac109a00 2024-05-15T14:05:57.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.064+0000 7fcbb395b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac113590 con 0x7fcbac1022d0 2024-05-15T14:05:57.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.064+0000 7fcbb395b700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac113890 con 0x7fcbac101920 2024-05-15T14:05:57.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.064+0000 7fcbb1ef8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbac109a00 0x7fcbac199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:57.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.064+0000 7fcbb1ef8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbac109a00 0x7fcbac199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43584/0 (socket says 172.21.15.33:43584) 2024-05-15T14:05:57.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.064+0000 7fcbb1ef8700 1 -- 172.21.15.33:0/4149405791 learned_addr learned my addr 172.21.15.33:0/4149405791 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:57.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.064+0000 7fcbb16f7700 1 --2- 172.21.15.33:0/4149405791 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbac101920 0x7fcbac19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:57.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.064+0000 7fcbb0ef6700 1 --2- 172.21.15.33:0/4149405791 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbac1022d0 0x7fcbac19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:57.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.064+0000 7fcbb1ef8700 1 -- 172.21.15.33:0/4149405791 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbac101920 msgr2=0x7fcbac19f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:57.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.064+0000 7fcbb1ef8700 1 --2- 172.21.15.33:0/4149405791 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbac101920 0x7fcbac19f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:57.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.064+0000 7fcbb1ef8700 1 -- 172.21.15.33:0/4149405791 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbac1022d0 msgr2=0x7fcbac19fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:57.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.064+0000 7fcbb1ef8700 1 --2- 172.21.15.33:0/4149405791 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbac1022d0 0x7fcbac19fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:57.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.065+0000 7fcbb1ef8700 1 -- 172.21.15.33:0/4149405791 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbac19a0d0 con 0x7fcbac109a00 2024-05-15T14:05:57.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.065+0000 7fcbb0ef6700 1 --2- 172.21.15.33:0/4149405791 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbac1022d0 0x7fcbac19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:05:57.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.065+0000 7fcbb1ef8700 1 --2- 172.21.15.33:0/4149405791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbac109a00 0x7fcbac199970 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fcba40095c0 tx=0x7fcba4007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:57.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.065+0000 7fcb9e7fc700 1 -- 172.21.15.33:0/4149405791 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba401f070 con 0x7fcbac109a00 2024-05-15T14:05:57.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.065+0000 7fcbb395b700 1 -- 172.21.15.33:0/4149405791 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbac19a360 con 0x7fcbac109a00 2024-05-15T14:05:57.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.065+0000 7fcb9e7fc700 1 -- 172.21.15.33:0/4149405791 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcba4003c00 con 0x7fcbac109a00 2024-05-15T14:05:57.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.065+0000 7fcb9e7fc700 1 -- 172.21.15.33:0/4149405791 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba401b6a0 con 0x7fcbac109a00 2024-05-15T14:05:57.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.065+0000 7fcbb395b700 1 -- 172.21.15.33:0/4149405791 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbac1a6630 con 0x7fcbac109a00 2024-05-15T14:05:57.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.067+0000 7fcb9e7fc700 1 -- 172.21.15.33:0/4149405791 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fcba4022070 con 0x7fcbac109a00 2024-05-15T14:05:57.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.068+0000 7fcb9e7fc700 1 --2- 172.21.15.33:0/4149405791 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fcb980b29f0 0x7fcb980b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:57.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.068+0000 7fcbb16f7700 1 --2- 172.21.15.33:0/4149405791 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fcb980b29f0 0x7fcb980b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:57.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.068+0000 7fcb9e7fc700 1 -- 172.21.15.33:0/4149405791 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7fcba400f020 con 0x7fcbac109a00 2024-05-15T14:05:57.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.069+0000 7fcbbe1f1b80 1 Processor -- start 2024-05-15T14:05:57.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.069+0000 7fcbbe1f1b80 1 -- start start 2024-05-15T14:05:57.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.069+0000 7fcbb16f7700 1 --2- 172.21.15.33:0/4149405791 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fcb980b29f0 0x7fcb980b4eb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcbac108d20 tx=0x7fcba800a380 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:57.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.069+0000 7fcbbe1f1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e97beb2420 0x55e97beb2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:57.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.069+0000 7fcbbe1f1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e97beb2d40 0x55e97beadca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:57.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.069+0000 7fcbbe1f1b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e97beae2c0 0x55e97beae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:57.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.069+0000 7fcbbe1f1b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e97be0be70 con 0x55e97beae2c0 2024-05-15T14:05:57.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.069+0000 7fcbbe1f1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e97bdfba10 con 0x55e97beb2d40 2024-05-15T14:05:57.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.069+0000 7fcbbe1f1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55e97bdffa50 con 0x55e97beb2420 2024-05-15T14:05:57.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.069+0000 7fcbb1ef8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e97beae2c0 0x55e97beae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:57.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.069+0000 7fcbb1ef8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e97beae2c0 0x55e97beae720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43600/0 (socket says 172.21.15.33:43600) 2024-05-15T14:05:57.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.069+0000 7fcbb1ef8700 1 -- 172.21.15.33:0/4081887983 learned_addr learned my addr 172.21.15.33:0/4081887983 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:57.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.069+0000 7fcbb0ef6700 1 --2- 172.21.15.33:0/4081887983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e97beb2d40 0x55e97beadca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:57.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.069+0000 7fcbb16f7700 1 --2- 172.21.15.33:0/4081887983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e97beb2420 0x55e97beb2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:57.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.070+0000 7fcbb1ef8700 1 -- 172.21.15.33:0/4081887983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e97beb2420 msgr2=0x55e97beb2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:57.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.070+0000 7fcbb1ef8700 1 --2- 172.21.15.33:0/4081887983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e97beb2420 0x55e97beb2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:57.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.070+0000 7fcbb1ef8700 1 -- 172.21.15.33:0/4081887983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e97beb2d40 msgr2=0x55e97beadca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:57.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.070+0000 7fcbb1ef8700 1 --2- 172.21.15.33:0/4081887983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e97beb2d40 0x55e97beadca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:57.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.070+0000 7fcbb1ef8700 1 -- 172.21.15.33:0/4081887983 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e97bda50b0 con 0x55e97beae2c0 2024-05-15T14:05:57.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.070+0000 7fcbb1ef8700 1 --2- 172.21.15.33:0/4081887983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e97beae2c0 0x55e97beae720 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fcba40aa920 tx=0x7fcba400b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f1df00f70810d34 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:57.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.070+0000 7fcb977fe700 1 -- 172.21.15.33:0/4081887983 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba40a63b0 con 0x55e97beae2c0 2024-05-15T14:05:57.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.070+0000 7fcb977fe700 1 -- 172.21.15.33:0/4081887983 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcba40b3340 con 0x55e97beae2c0 2024-05-15T14:05:57.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.070+0000 7fcb977fe700 1 -- 172.21.15.33:0/4081887983 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba40050d0 con 0x55e97beae2c0 2024-05-15T14:05:57.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.071+0000 7fcbbe1f1b80 1 -- 172.21.15.33:0/4081887983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e97beae2c0 msgr2=0x55e97beae720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:57.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.071+0000 7fcbbe1f1b80 1 --2- 172.21.15.33:0/4081887983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e97beae2c0 0x55e97beae720 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fcba40aa920 tx=0x7fcba400b3e0 comp rx=0 tx=0).stop 2024-05-15T14:05:57.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.071+0000 7fcbbe1f1b80 1 -- 172.21.15.33:0/4081887983 shutdown_connections 2024-05-15T14:05:57.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.071+0000 7fcbbe1f1b80 1 --2- 172.21.15.33:0/4081887983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e97beae2c0 0x55e97beae720 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:57.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.071+0000 7fcbbe1f1b80 1 --2- 172.21.15.33:0/4081887983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e97beb2420 0x55e97beb2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:57.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.071+0000 7fcbbe1f1b80 1 --2- 172.21.15.33:0/4081887983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e97beb2d40 0x55e97beadca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:57.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.071+0000 7fcbbe1f1b80 1 -- 172.21.15.33:0/4081887983 >> 172.21.15.33:0/4081887983 conn(0x55e97bddaeb0 msgr2=0x55e97bea84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:57.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.071+0000 7fcbbe1f1b80 1 -- 172.21.15.33:0/4081887983 shutdown_connections 2024-05-15T14:05:57.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.071+0000 7fcbbe1f1b80 1 -- 172.21.15.33:0/4081887983 wait complete. 2024-05-15T14:05:57.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.071+0000 7fcbbe1f1b80 1 -- start start 2024-05-15T14:05:57.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.072+0000 7fcbbe1f1b80 1 Processor -- start 2024-05-15T14:05:57.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.072+0000 7fcbbe1f1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e97beae2c0 0x55e97beb9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:57.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.072+0000 7fcbbe1f1b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e97beb2420 0x55e97beb6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:57.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.072+0000 7fcbbe1f1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e97beb2d40 0x55e97beb7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:57.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.072+0000 7fcbbe1f1b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e97bd020a0 con 0x55e97beb2420 2024-05-15T14:05:57.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.072+0000 7fcbbe1f1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e97bd63ea0 con 0x55e97beae2c0 2024-05-15T14:05:57.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.072+0000 7fcbbe1f1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55e97be0be70 con 0x55e97beb2d40 2024-05-15T14:05:57.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.072+0000 7fcbb0ef6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e97beb2420 0x55e97beb6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:57.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.072+0000 7fcbb0ef6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e97beb2420 0x55e97beb6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43608/0 (socket says 172.21.15.33:43608) 2024-05-15T14:05:57.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.072+0000 7fcbbe1f1b80 10 client.0 _ll_register_callbacks cb 0x55e97bd96a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:05:57.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.072+0000 7fcbb0ef6700 1 -- 172.21.15.33:0/482474035 learned_addr learned my addr 172.21.15.33:0/482474035 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:57.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.072+0000 7fcbb16f7700 1 --2- 172.21.15.33:0/482474035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e97beae2c0 0x55e97beb9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:57.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.072+0000 7fcbb1ef8700 1 --2- 172.21.15.33:0/482474035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e97beb2d40 0x55e97beb7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:57.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.072+0000 7fcbb0ef6700 1 -- 172.21.15.33:0/482474035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e97beb2d40 msgr2=0x55e97beb7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:57.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.072+0000 7fcbb0ef6700 1 --2- 172.21.15.33:0/482474035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e97beb2d40 0x55e97beb7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:57.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.072+0000 7fcbb0ef6700 1 -- 172.21.15.33:0/482474035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e97beae2c0 msgr2=0x55e97beb9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:57.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.072+0000 7fcbb0ef6700 1 --2- 172.21.15.33:0/482474035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e97beae2c0 0x55e97beb9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:57.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.072+0000 7fcbb0ef6700 1 -- 172.21.15.33:0/482474035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e97bd4b9d0 con 0x55e97beb2420 2024-05-15T14:05:57.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.073+0000 7fcbb0ef6700 1 --2- 172.21.15.33:0/482474035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e97beb2420 0x55e97beb6ec0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009f60 tx=0x7fcba0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1945ddf30ec5dcd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:57.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.073+0000 7fcb94ff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:05:57.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.073+0000 7fcb94ff9700 1 -- 172.21.15.33:0/482474035 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba0012650 con 0x55e97beb2420 2024-05-15T14:05:57.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.073+0000 7fcb94ff9700 1 -- 172.21.15.33:0/482474035 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7fcba00127f0 con 0x55e97beb2420 2024-05-15T14:05:57.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.073+0000 7fcbbe1f1b80 1 -- 172.21.15.33:0/482474035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcba40b3340 con 0x55e97beb2420 2024-05-15T14:05:57.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.073+0000 7fcb94ff9700 1 -- 172.21.15.33:0/482474035 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcba00129f0 con 0x55e97beb2420 2024-05-15T14:05:57.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.073+0000 7fcbbe1f1b80 10 client.16023 fetch_fsmap learned FSMap version 29 2024-05-15T14:05:57.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.073+0000 7fcbbe1f1b80 1 -- 172.21.15.33:0/482474035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e97bdb5860 con 0x55e97beb2420 2024-05-15T14:05:57.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.073+0000 7fcb94ff9700 1 -- 172.21.15.33:0/482474035 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcba40b3340 con 0x55e97beb2420 2024-05-15T14:05:57.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.073+0000 7fcb94ff9700 1 -- 172.21.15.33:0/482474035 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fcba000f040 con 0x55e97beb2420 2024-05-15T14:05:57.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.073+0000 7fcbbe1f1b80 10 client.16023 fetch_fsmap finished waiting for FSMap version 29 2024-05-15T14:05:57.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.074+0000 7fcbbe1f1b80 10 client.16023 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:05:57.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.074+0000 7fcbbe1f1b80 20 client.16023 populate_metadata read hostname 'smithi033' 2024-05-15T14:05:57.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.074+0000 7fcbbe1f1b80 1 --2- 172.21.15.33:0/482474035 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e97bec70e0 0x55e97bec9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:57.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.074+0000 7fcbbe1f1b80 4 client.16023 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:05:57.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.074+0000 7fcbbe1f1b80 1 -- 172.21.15.33:0/482474035 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e97be0ddc0 con 0x55e97bec70e0 2024-05-15T14:05:57.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.074+0000 7fcbb16f7700 1 --2- 172.21.15.33:0/482474035 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e97bec70e0 0x55e97bec9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:57.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.074+0000 7fcbb16f7700 1 --2- 172.21.15.33:0/482474035 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e97bec70e0 0x55e97bec9530 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3aee181cdf52905 server_cookie=b65a772f012648d3 in_seq=0 out_seq=0 2024-05-15T14:05:57.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.074+0000 7fcb94ff9700 10 client.16023 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:05:57.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.075+0000 7fcb94ff9700 1 -- 172.21.15.33:0/482474035 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e97be0ddc0 con 0x55e97bec70e0 2024-05-15T14:05:57.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.075+0000 7fcb94ff9700 10 client.16023 handle_command_reply: tid=0 2024-05-15T14:05:57.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.075+0000 7fcbbe1f1b80 1 client.16023 shutdown 2024-05-15T14:05:57.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.075+0000 7fcbbe1f1b80 1 -- 172.21.15.33:0/482474035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e97beb2420 msgr2=0x55e97beb6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:57.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.075+0000 7fcbbe1f1b80 1 --2- 172.21.15.33:0/482474035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e97beb2420 0x55e97beb6ec0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009f60 tx=0x7fcba0009f90 comp rx=0 tx=0).stop 2024-05-15T14:05:57.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.075+0000 7fcbbe1f1b80 1 -- 172.21.15.33:0/482474035 shutdown_connections 2024-05-15T14:05:57.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.075+0000 7fcbbe1f1b80 1 --2- 172.21.15.33:0/482474035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e97beb2d40 0x55e97beb7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:57.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.075+0000 7fcbbe1f1b80 1 --2- 172.21.15.33:0/482474035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e97beae2c0 0x55e97beb9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:57.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.075+0000 7fcbbe1f1b80 1 --2- 172.21.15.33:0/482474035 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e97bec70e0 0x55e97bec9530 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:57.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.075+0000 7fcbbe1f1b80 1 --2- 172.21.15.33:0/482474035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e97beb2420 0x55e97beb6ec0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:57.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.075+0000 7fcbbe1f1b80 1 -- 172.21.15.33:0/482474035 >> 172.21.15.33:0/482474035 conn(0x55e97bddaeb0 msgr2=0x55e97bea2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:57.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.075+0000 7fcb94ff9700 0 client.16023 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:05:57.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.076+0000 7fcbbe1f1b80 1 -- 172.21.15.33:0/482474035 shutdown_connections 2024-05-15T14:05:57.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.076+0000 7fcbbe1f1b80 1 -- 172.21.15.33:0/482474035 wait complete. 2024-05-15T14:05:57.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.076+0000 7fcbbe1f1b80 20 client.16023 trim_cache size 0 max 16384 2024-05-15T14:05:57.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.096+0000 7fcbbe1f1b80 1 Processor -- start 2024-05-15T14:05:57.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.096+0000 7fcbbe1f1b80 1 -- start start 2024-05-15T14:05:57.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.096+0000 7fcbbe1f1b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e97beae2c0 0x55e97beb5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:57.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.096+0000 7fcbbe1f1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e97beb2420 0x55e97beb1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:57.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.096+0000 7fcbbe1f1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e97beb2d40 0x55e97bea30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:57.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.096+0000 7fcbbe1f1b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac1b0570 con 0x55e97beae2c0 2024-05-15T14:05:57.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.096+0000 7fcbbe1f1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac1ade00 con 0x55e97beb2d40 2024-05-15T14:05:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.096+0000 7fcbbe1f1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac1abc50 con 0x55e97beb2420 2024-05-15T14:05:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.096+0000 7fcbb16f7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e97beae2c0 0x55e97beb5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.096+0000 7fcbb16f7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e97beae2c0 0x55e97beb5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43624/0 (socket says 172.21.15.33:43624) 2024-05-15T14:05:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.096+0000 7fcbb16f7700 1 -- 172.21.15.33:0/3617474365 learned_addr learned my addr 172.21.15.33:0/3617474365 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.096+0000 7fcbb0ef6700 1 --2- 172.21.15.33:0/3617474365 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e97beb2420 0x55e97beb1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.096+0000 7fcbb1ef8700 1 --2- 172.21.15.33:0/3617474365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e97beb2d40 0x55e97bea30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.096+0000 7fcbb16f7700 1 -- 172.21.15.33:0/3617474365 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e97beb2420 msgr2=0x55e97beb1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.096+0000 7fcbb16f7700 1 --2- 172.21.15.33:0/3617474365 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e97beb2420 0x55e97beb1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.096+0000 7fcbb16f7700 1 -- 172.21.15.33:0/3617474365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e97beb2d40 msgr2=0x55e97bea30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.096+0000 7fcbb16f7700 1 --2- 172.21.15.33:0/3617474365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e97beb2d40 0x55e97bea30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.096+0000 7fcbb16f7700 1 -- 172.21.15.33:0/3617474365 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e97bec9bb0 con 0x55e97beae2c0 2024-05-15T14:05:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.096+0000 7fcbb1ef8700 1 --2- 172.21.15.33:0/3617474365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e97beb2d40 0x55e97bea30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:05:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.097+0000 7fcbb16f7700 1 --2- 172.21.15.33:0/3617474365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e97beae2c0 0x55e97beb5e80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fcba8004300 tx=0x7fcba8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a3e4950b5ab50e server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:57.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.097+0000 7fcb96ffd700 1 -- 172.21.15.33:0/3617474365 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba802c070 con 0x55e97beae2c0 2024-05-15T14:05:57.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.097+0000 7fcb96ffd700 1 -- 172.21.15.33:0/3617474365 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcba8014070 con 0x55e97beae2c0 2024-05-15T14:05:57.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.097+0000 7fcb96ffd700 1 -- 172.21.15.33:0/3617474365 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba8016070 con 0x55e97beae2c0 2024-05-15T14:05:57.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.098+0000 7fcbbe1f1b80 1 -- 172.21.15.33:0/3617474365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e97beae2c0 msgr2=0x55e97beb5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:57.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.098+0000 7fcbbe1f1b80 1 --2- 172.21.15.33:0/3617474365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e97beae2c0 0x55e97beb5e80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fcba8004300 tx=0x7fcba8016000 comp rx=0 tx=0).stop 2024-05-15T14:05:57.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.098+0000 7fcbbe1f1b80 1 -- 172.21.15.33:0/3617474365 shutdown_connections 2024-05-15T14:05:57.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.098+0000 7fcbbe1f1b80 1 --2- 172.21.15.33:0/3617474365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e97beb2d40 0x55e97bea30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:57.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.098+0000 7fcbbe1f1b80 1 --2- 172.21.15.33:0/3617474365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e97beae2c0 0x55e97beb5e80 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:57.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.098+0000 7fcbbe1f1b80 1 --2- 172.21.15.33:0/3617474365 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e97beb2420 0x55e97beb1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:57.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.098+0000 7fcbbe1f1b80 1 -- 172.21.15.33:0/3617474365 >> 172.21.15.33:0/3617474365 conn(0x55e97bddaeb0 msgr2=0x55e97beb8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:57.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.098+0000 7fcbbe1f1b80 1 -- 172.21.15.33:0/3617474365 shutdown_connections 2024-05-15T14:05:57.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.098+0000 7fcbbe1f1b80 1 -- 172.21.15.33:0/3617474365 wait complete. 2024-05-15T14:05:57.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.098+0000 7fcbbe1f1b80 1 -- start start 2024-05-15T14:05:57.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.098+0000 7fcbbe1f1b80 1 Processor -- start 2024-05-15T14:05:57.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.099+0000 7fcbbe1f1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e97beae2c0 0x55e97bead4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:57.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.099+0000 7fcbbe1f1b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e97beb2420 0x55e97beab890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:57.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.099+0000 7fcbbe1f1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e97beb2d40 0x55e97beabec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:57.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.099+0000 7fcbbe1f1b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac1ac2c0 con 0x55e97beb2420 2024-05-15T14:05:57.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.099+0000 7fcbbe1f1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac1ad730 con 0x55e97beb2d40 2024-05-15T14:05:57.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.099+0000 7fcbbe1f1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac1ac990 con 0x55e97beae2c0 2024-05-15T14:05:57.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.099+0000 7fcbbe1f1b80 10 client.0 _ll_register_callbacks cb 0x55e97bd96a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:05:57.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.099+0000 7fcbb0ef6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e97beb2420 0x55e97beab890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:57.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.099+0000 7fcbb0ef6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e97beb2420 0x55e97beab890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43634/0 (socket says 172.21.15.33:43634) 2024-05-15T14:05:57.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.099+0000 7fcbb16f7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e97beae2c0 0x55e97bead4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:57.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.099+0000 7fcbb0ef6700 1 -- 172.21.15.33:0/259889413 learned_addr learned my addr 172.21.15.33:0/259889413 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:57.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.099+0000 7fcbb1ef8700 1 --2- 172.21.15.33:0/259889413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e97beb2d40 0x55e97beabec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:57.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.099+0000 7fcbb0ef6700 1 -- 172.21.15.33:0/259889413 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e97beae2c0 msgr2=0x55e97bead4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:57.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.099+0000 7fcbb0ef6700 1 --2- 172.21.15.33:0/259889413 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e97beae2c0 0x55e97bead4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:57.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.099+0000 7fcbb0ef6700 1 -- 172.21.15.33:0/259889413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e97beb2d40 msgr2=0x55e97beabec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:05:57.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.099+0000 7fcbb0ef6700 1 --2- 172.21.15.33:0/259889413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e97beb2d40 0x55e97beabec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:57.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.099+0000 7fcbb0ef6700 1 -- 172.21.15.33:0/259889413 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e97beac6c0 con 0x55e97beb2420 2024-05-15T14:05:57.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.099+0000 7fcbb16f7700 1 --2- 172.21.15.33:0/259889413 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e97beae2c0 0x55e97bead4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:05:57.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.100+0000 7fcbb0ef6700 1 --2- 172.21.15.33:0/259889413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e97beb2420 0x55e97beab890 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009850 tx=0x7fcba0012700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f56291a6df0a5d1d server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:57.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.100+0000 7fcb94ff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:05:57.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.100+0000 7fcb94ff9700 1 -- 172.21.15.33:0/259889413 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba0005450 con 0x55e97beb2420 2024-05-15T14:05:57.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.100+0000 7fcb94ff9700 1 -- 172.21.15.33:0/259889413 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7fcba00055f0 con 0x55e97beb2420 2024-05-15T14:05:57.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.100+0000 7fcbbe1f1b80 1 -- 172.21.15.33:0/259889413 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcba8014070 con 0x55e97beb2420 2024-05-15T14:05:57.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.100+0000 7fcb94ff9700 1 -- 172.21.15.33:0/259889413 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcba0039050 con 0x55e97beb2420 2024-05-15T14:05:57.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.100+0000 7fcbbe1f1b80 10 client.16029 fetch_fsmap learned FSMap version 29 2024-05-15T14:05:57.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.100+0000 7fcbbe1f1b80 1 -- 172.21.15.33:0/259889413 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e97bebf8c0 con 0x55e97beb2420 2024-05-15T14:05:57.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.100+0000 7fcb94ff9700 1 -- 172.21.15.33:0/259889413 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcba8014070 con 0x55e97beb2420 2024-05-15T14:05:57.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.100+0000 7fcb94ff9700 1 -- 172.21.15.33:0/259889413 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fcba0034030 con 0x55e97beb2420 2024-05-15T14:05:57.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.100+0000 7fcbbe1f1b80 10 client.16029 fetch_fsmap finished waiting for FSMap version 29 2024-05-15T14:05:57.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.100+0000 7fcbbe1f1b80 10 client.16029 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:05:57.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.100+0000 7fcbbe1f1b80 20 client.16029 populate_metadata read hostname 'smithi033' 2024-05-15T14:05:57.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.100+0000 7fcbbe1f1b80 1 --2- 172.21.15.33:0/259889413 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e97bf3c210 0x55e97bf3e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:57.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.100+0000 7fcbbe1f1b80 4 client.16029 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "417f377d", "release": true, "await": true}] 2024-05-15T14:05:57.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.100+0000 7fcbbe1f1b80 1 -- 172.21.15.33:0/259889413 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "quiesce db", "set_id": "417f377d", "release": true, "await": true}) -- 0x55e97be075a0 con 0x55e97bf3c210 2024-05-15T14:05:57.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.101+0000 7fcbb16f7700 1 --2- 172.21.15.33:0/259889413 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e97bf3c210 0x55e97bf3e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:57.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.101+0000 7fcb94ff9700 10 client.16029 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:05:57.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.101+0000 7fcbb16f7700 1 --2- 172.21.15.33:0/259889413 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e97bf3c210 0x55e97bf3e600 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b4af822c0a600c4 server_cookie=7a03d333ba306e6f in_seq=0 out_seq=0 2024-05-15T14:05:57.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.106+0000 7fcb94ff9700 1 -- 172.21.15.33:0/259889413 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e97be075a0 con 0x55e97bf3c210 2024-05-15T14:05:57.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.106+0000 7fcb94ff9700 10 client.16029 handle_command_reply: tid=0 2024-05-15T14:05:57.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.106+0000 7fcbbe1f1b80 1 client.16029 shutdown 2024-05-15T14:05:57.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.106+0000 7fcbbe1f1b80 1 -- 172.21.15.33:0/259889413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e97beb2420 msgr2=0x55e97beab890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:57.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.106+0000 7fcbbe1f1b80 1 --2- 172.21.15.33:0/259889413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e97beb2420 0x55e97beab890 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009850 tx=0x7fcba0012700 comp rx=0 tx=0).stop 2024-05-15T14:05:57.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.107+0000 7fcbbe1f1b80 1 -- 172.21.15.33:0/259889413 shutdown_connections 2024-05-15T14:05:57.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.107+0000 7fcbbe1f1b80 1 --2- 172.21.15.33:0/259889413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e97beb2d40 0x55e97beabec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:57.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.107+0000 7fcbbe1f1b80 1 --2- 172.21.15.33:0/259889413 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e97beae2c0 0x55e97bead4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:57.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.107+0000 7fcbbe1f1b80 1 --2- 172.21.15.33:0/259889413 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e97bf3c210 0x55e97bf3e600 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:57.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.107+0000 7fcbbe1f1b80 1 --2- 172.21.15.33:0/259889413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e97beb2420 0x55e97beab890 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:57.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.107+0000 7fcb94ff9700 0 client.16029 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:05:57.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.107+0000 7fcbbe1f1b80 1 -- 172.21.15.33:0/259889413 >> 172.21.15.33:0/259889413 conn(0x55e97bddaeb0 msgr2=0x55e97beb8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:57.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.107+0000 7fcbbe1f1b80 1 -- 172.21.15.33:0/259889413 shutdown_connections 2024-05-15T14:05:57.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.107+0000 7fcbbe1f1b80 1 -- 172.21.15.33:0/259889413 wait complete. 2024-05-15T14:05:57.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.107+0000 7fcbbe1f1b80 20 client.16029 trim_cache size 0 max 16384 2024-05-15T14:05:57.111 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T14:05:57.111 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 29, 2024-05-15T14:05:57.111 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24478, 2024-05-15T14:05:57.111 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 20, 2024-05-15T14:05:57.111 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-05-15T14:05:57.111 INFO:teuthology.orchestra.run.smithi033.stdout: "417f377d": { 2024-05-15T14:05:57.111 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 20, 2024-05-15T14:05:57.111 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-05-15T14:05:57.111 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:05:57.111 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-05-15T14:05:57.111 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:05:57.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:05:57.111 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 90.0, 2024-05-15T14:05:57.112 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 138.3, 2024-05-15T14:05:57.112 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-05-15T14:05:57.112 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-05-15T14:05:57.112 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-05-15T14:05:57.112 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:05:57.112 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-05-15T14:05:57.112 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:05:57.112 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:05:57.112 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:05:57.112 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:05:57.112 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:05:57.112 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:05:57.112 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T14:05:57.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.107+0000 7fcbb395b700 1 -- 172.21.15.33:0/4149405791 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fcb980b29f0 msgr2=0x7fcb980b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:57.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.107+0000 7fcbb395b700 1 --2- 172.21.15.33:0/4149405791 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fcb980b29f0 0x7fcb980b4eb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcbac108d20 tx=0x7fcba800a380 comp rx=0 tx=0).stop 2024-05-15T14:05:57.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.107+0000 7fcbb395b700 1 -- 172.21.15.33:0/4149405791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbac109a00 msgr2=0x7fcbac199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:57.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.107+0000 7fcbb395b700 1 --2- 172.21.15.33:0/4149405791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbac109a00 0x7fcbac199970 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fcba40095c0 tx=0x7fcba4007a20 comp rx=0 tx=0).stop 2024-05-15T14:05:57.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.108+0000 7fcbb395b700 1 -- 172.21.15.33:0/4149405791 shutdown_connections 2024-05-15T14:05:57.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.108+0000 7fcbb395b700 1 --2- 172.21.15.33:0/4149405791 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fcb980b29f0 0x7fcb980b4eb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:57.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.108+0000 7fcbb395b700 1 --2- 172.21.15.33:0/4149405791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbac109a00 0x7fcbac199970 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:57.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.108+0000 7fcbb395b700 1 --2- 172.21.15.33:0/4149405791 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcbac101920 0x7fcbac19f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:57.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.108+0000 7fcbb395b700 1 --2- 172.21.15.33:0/4149405791 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcbac1022d0 0x7fcbac19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:57.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.108+0000 7fcbb395b700 1 -- 172.21.15.33:0/4149405791 >> 172.21.15.33:0/4149405791 conn(0x7fcbac078640 msgr2=0x7fcbac100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:57.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.108+0000 7fcbb395b700 1 -- 172.21.15.33:0/4149405791 shutdown_connections 2024-05-15T14:05:57.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:57.108+0000 7fcbb395b700 1 -- 172.21.15.33:0/4149405791 wait complete. 2024-05-15T14:05:57.818 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '417f377d', total seconds elapsed: 23.8 2024-05-15T14:05:57.818 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.35 2024-05-15T14:05:57.818 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T14:05:58.059 INFO:tasks.workunit.client.0.smithi033.stdout: 1 333831 6.97 MB/sec execute 93 sec latency 17229.819 ms 2024-05-15T14:05:58.071 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:05:58.444 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:05:58 smithi033 ceph-mon[31650]: pgmap v172: 97 pgs: 97 active+clean; 84 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 2.3 MiB/s wr, 98 op/s 2024-05-15T14:05:58.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:05:58 smithi052 ceph-mon[36190]: pgmap v172: 97 pgs: 97 active+clean; 84 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 2.3 MiB/s wr, 98 op/s 2024-05-15T14:05:58.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:05:58 smithi081 ceph-mon[36434]: pgmap v172: 97 pgs: 97 active+clean; 84 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 2.3 MiB/s wr, 98 op/s 2024-05-15T14:05:59.059 INFO:tasks.workunit.client.0.smithi033.stdout: 1 335797 6.97 MB/sec execute 94 sec latency 38.114 ms 2024-05-15T14:05:59.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.724+0000 7f0f6caa9700 1 -- 172.21.15.33:0/2040869650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f681037f0 msgr2=0x7f0f68103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:59.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.724+0000 7f0f6caa9700 1 --2- 172.21.15.33:0/2040869650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f681037f0 0x7f0f68103bd0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f0f580099c0 tx=0x7f0f58009cd0 comp rx=0 tx=0).stop 2024-05-15T14:05:59.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.724+0000 7f0f6caa9700 1 -- 172.21.15.33:0/2040869650 shutdown_connections 2024-05-15T14:05:59.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.724+0000 7f0f6caa9700 1 --2- 172.21.15.33:0/2040869650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f681088f0 0x7f0f68110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:59.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.724+0000 7f0f6caa9700 1 --2- 172.21.15.33:0/2040869650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f681037f0 0x7f0f68103bd0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:59.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.724+0000 7f0f6caa9700 1 --2- 172.21.15.33:0/2040869650 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f681041a0 0x7f0f681081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:59.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.724+0000 7f0f6caa9700 1 -- 172.21.15.33:0/2040869650 >> 172.21.15.33:0/2040869650 conn(0x7f0f6807a6a0 msgr2=0x7f0f6807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:05:59.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.724+0000 7f0f6caa9700 1 -- 172.21.15.33:0/2040869650 shutdown_connections 2024-05-15T14:05:59.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.724+0000 7f0f6caa9700 1 -- 172.21.15.33:0/2040869650 wait complete. 2024-05-15T14:05:59.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.725+0000 7f0f6caa9700 1 Processor -- start 2024-05-15T14:05:59.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.725+0000 7f0f6caa9700 1 -- start start 2024-05-15T14:05:59.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.725+0000 7f0f6caa9700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f681037f0 0x7f0f68075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:59.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.725+0000 7f0f6caa9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f681041a0 0x7f0f68075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:59.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.725+0000 7f0f6caa9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f681088f0 0x7f0f680762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:59.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.725+0000 7f0f6caa9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0f681136c0 con 0x7f0f681041a0 2024-05-15T14:05:59.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.725+0000 7f0f6caa9700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0f68113540 con 0x7f0f681088f0 2024-05-15T14:05:59.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.725+0000 7f0f6caa9700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0f68113840 con 0x7f0f681037f0 2024-05-15T14:05:59.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.725+0000 7f0f6659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f681037f0 0x7f0f68075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:59.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.725+0000 7f0f65d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f681041a0 0x7f0f68075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:59.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.726+0000 7f0f65d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f681041a0 0x7f0f68075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43652/0 (socket says 172.21.15.33:43652) 2024-05-15T14:05:59.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.726+0000 7f0f66d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f681088f0 0x7f0f680762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:59.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.726+0000 7f0f65d9b700 1 -- 172.21.15.33:0/337321241 learned_addr learned my addr 172.21.15.33:0/337321241 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:05:59.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.726+0000 7f0f6659c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f681037f0 0x7f0f68075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:37906/0 (socket says 172.21.15.33:37906) 2024-05-15T14:05:59.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.726+0000 7f0f65d9b700 1 -- 172.21.15.33:0/337321241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f681037f0 msgr2=0x7f0f68075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:59.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.726+0000 7f0f65d9b700 1 --2- 172.21.15.33:0/337321241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f681037f0 0x7f0f68075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:59.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.726+0000 7f0f65d9b700 1 -- 172.21.15.33:0/337321241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f681088f0 msgr2=0x7f0f680762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:05:59.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.726+0000 7f0f65d9b700 1 --2- 172.21.15.33:0/337321241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f681088f0 0x7f0f680762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:05:59.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.726+0000 7f0f65d9b700 1 -- 172.21.15.33:0/337321241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f681a80c0 con 0x7f0f681041a0 2024-05-15T14:05:59.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.726+0000 7f0f66d9d700 1 --2- 172.21.15.33:0/337321241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f681088f0 0x7f0f680762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:05:59.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.726+0000 7f0f65d9b700 1 --2- 172.21.15.33:0/337321241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f681041a0 0x7f0f68075d80 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c00e9a0 tx=0x7f0f5c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:59.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.726+0000 7f0f577fe700 1 -- 172.21.15.33:0/337321241 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c00ca10 con 0x7f0f681041a0 2024-05-15T14:05:59.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.726+0000 7f0f6caa9700 1 -- 172.21.15.33:0/337321241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f681a83b0 con 0x7f0f681041a0 2024-05-15T14:05:59.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.726+0000 7f0f577fe700 1 -- 172.21.15.33:0/337321241 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f5c00cbb0 con 0x7f0f681041a0 2024-05-15T14:05:59.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.726+0000 7f0f6caa9700 1 -- 172.21.15.33:0/337321241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f681a8940 con 0x7f0f681041a0 2024-05-15T14:05:59.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.726+0000 7f0f577fe700 1 -- 172.21.15.33:0/337321241 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c018680 con 0x7f0f681041a0 2024-05-15T14:05:59.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.728+0000 7f0f6caa9700 1 -- 172.21.15.33:0/337321241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f48005330 con 0x7f0f681041a0 2024-05-15T14:05:59.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.729+0000 7f0f577fe700 1 -- 172.21.15.33:0/337321241 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0f5c018820 con 0x7f0f681041a0 2024-05-15T14:05:59.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.730+0000 7f0f577fe700 1 --2- 172.21.15.33:0/337321241 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0f500b2a70 0x7f0f500b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:05:59.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.730+0000 7f0f6659c700 1 --2- 172.21.15.33:0/337321241 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0f500b2a70 0x7f0f500b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:05:59.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.730+0000 7f0f577fe700 1 -- 172.21.15.33:0/337321241 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f0f5c0e25c0 con 0x7f0f681041a0 2024-05-15T14:05:59.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.730+0000 7f0f6659c700 1 --2- 172.21.15.33:0/337321241 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0f500b2a70 0x7f0f500b4f30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0f58003760 tx=0x7f0f58005fb0 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:05:59.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:05:59.734+0000 7f0f577fe700 1 -- 172.21.15.33:0/337321241 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0f5c0a9fb0 con 0x7f0f681041a0 2024-05-15T14:06:00.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:00.008+0000 7f0f6caa9700 1 -- 172.21.15.33:0/337321241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f480058f0 con 0x7f0f681041a0 2024-05-15T14:06:00.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:00.009+0000 7f0f577fe700 1 -- 172.21.15.33:0/337321241 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11855 (secure 0 0 0) 0x7f0f5c0a98b0 con 0x7f0f681041a0 2024-05-15T14:06:00.011 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T14:06:00.011 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2030117633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2030117633},{"type":"v1","addr":"172.21.15.33:6839","nonce":2030117633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1918979061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1918979061},{"type":"v1","addr":"172.21.15.33:6841","nonce":1918979061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T13:57:19.056417+0000","modified":"2024-05-15T14:05:13.682039+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24463,"mds_2":24461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/1629697841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1629697841},{"type":"v1","addr":"172.21.15.52:6837","nonce":1629697841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.52:6839/2696044471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2696044471},{"type":"v1","addr":"172.21.15.52:6839","nonce":2696044471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.81:6839/1554395940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":1554395940},{"type":"v1","addr":"172.21.15.81:6839","nonce":1554395940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25393":{"gid":25393,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/727032511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":727032511},{"type":"v1","addr":"172.21.15.81:6835","nonce":727032511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24478,24461,24463]},"id":1}]} 2024-05-15T14:06:00.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:00.011+0000 7f0f6caa9700 1 -- 172.21.15.33:0/337321241 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0f500b2a70 msgr2=0x7f0f500b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:00.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:00.011+0000 7f0f6caa9700 1 --2- 172.21.15.33:0/337321241 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0f500b2a70 0x7f0f500b4f30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0f58003760 tx=0x7f0f58005fb0 comp rx=0 tx=0).stop 2024-05-15T14:06:00.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:00.011+0000 7f0f6caa9700 1 -- 172.21.15.33:0/337321241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f681041a0 msgr2=0x7f0f68075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:00.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:00.011+0000 7f0f6caa9700 1 --2- 172.21.15.33:0/337321241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f681041a0 0x7f0f68075d80 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c00e9a0 tx=0x7f0f5c00ecb0 comp rx=0 tx=0).stop 2024-05-15T14:06:00.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:00.011+0000 7f0f6caa9700 1 -- 172.21.15.33:0/337321241 shutdown_connections 2024-05-15T14:06:00.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:00.011+0000 7f0f6caa9700 1 --2- 172.21.15.33:0/337321241 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0f500b2a70 0x7f0f500b4f30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:00.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:00.011+0000 7f0f6caa9700 1 --2- 172.21.15.33:0/337321241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f681088f0 0x7f0f680762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:00.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:00.011+0000 7f0f6caa9700 1 --2- 172.21.15.33:0/337321241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f681037f0 0x7f0f68075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:00.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:00.011+0000 7f0f6caa9700 1 --2- 172.21.15.33:0/337321241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f681041a0 0x7f0f68075d80 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:00.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:00.011+0000 7f0f6caa9700 1 -- 172.21.15.33:0/337321241 >> 172.21.15.33:0/337321241 conn(0x7f0f6807a6a0 msgr2=0x7f0f681017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:00.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:00.011+0000 7f0f6caa9700 1 -- 172.21.15.33:0/337321241 shutdown_connections 2024-05-15T14:06:00.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:00.011+0000 7f0f6caa9700 1 -- 172.21.15.33:0/337321241 wait complete. 2024-05-15T14:06:00.015 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 29 2024-05-15T14:06:00.059 INFO:tasks.workunit.client.0.smithi033.stdout: 1 337698 7.03 MB/sec execute 95 sec latency 19.203 ms 2024-05-15T14:06:00.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:00 smithi052 ceph-mon[36190]: pgmap v173: 97 pgs: 97 active+clean; 84 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 1.6 MiB/s wr, 67 op/s 2024-05-15T14:06:00.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:00 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/337321241' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:06:00.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:00 smithi081 ceph-mon[36434]: pgmap v173: 97 pgs: 97 active+clean; 84 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 1.6 MiB/s wr, 67 op/s 2024-05-15T14:06:00.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:00 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/337321241' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:06:00.610 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:00 smithi033 ceph-mon[31650]: pgmap v173: 97 pgs: 97 active+clean; 84 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 1.6 MiB/s wr, 67 op/s 2024-05-15T14:06:00.610 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:00 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/337321241' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:06:00.611 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24478 quiesce db / --timeout 90 --expiration 138.35 --await-for 45.0' 2024-05-15T14:06:00.611 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24478 quiesce db / --timeout 90 --expiration 138.35 --await-for 45.0 2024-05-15T14:06:00.876 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:06:01.059 INFO:tasks.workunit.client.0.smithi033.stdout: 1 339445 7.03 MB/sec execute 96 sec latency 24.785 ms 2024-05-15T14:06:02.059 INFO:tasks.workunit.client.0.smithi033.stdout: 1 341195 7.08 MB/sec execute 97 sec latency 17.416 ms 2024-05-15T14:06:02.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.583+0000 7ff80a543700 1 -- 172.21.15.33:0/2940975544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff804107a40 msgr2=0x7ff80410f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:02.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.583+0000 7ff80a543700 1 --2- 172.21.15.33:0/2940975544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff804107a40 0x7ff80410f350 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00b0a0 tx=0x7ff7fc00b3b0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.583+0000 7ff80a543700 1 -- 172.21.15.33:0/2940975544 shutdown_connections 2024-05-15T14:06:02.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.583+0000 7ff80a543700 1 --2- 172.21.15.33:0/2940975544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff804107a40 0x7ff80410f350 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.583+0000 7ff80a543700 1 --2- 172.21.15.33:0/2940975544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff804102a20 0x7ff804102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.583+0000 7ff80a543700 1 --2- 172.21.15.33:0/2940975544 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff804107080 0x7ff804107500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.583+0000 7ff80a543700 1 -- 172.21.15.33:0/2940975544 >> 172.21.15.33:0/2940975544 conn(0x7ff8040791a0 msgr2=0x7ff8040795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:02.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.583+0000 7ff80a543700 1 -- 172.21.15.33:0/2940975544 shutdown_connections 2024-05-15T14:06:02.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.583+0000 7ff80a543700 1 -- 172.21.15.33:0/2940975544 wait complete. 2024-05-15T14:06:02.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.583+0000 7ff80a543700 1 Processor -- start 2024-05-15T14:06:02.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.583+0000 7ff80a543700 1 -- start start 2024-05-15T14:06:02.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.584+0000 7ff80a543700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff804102a20 0x7ff804104120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:02.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.584+0000 7ff80a543700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff804107080 0x7ff804104660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:02.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.584+0000 7ff80a543700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff804107a40 0x7ff804104ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:02.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.584+0000 7ff80a543700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff804112610 con 0x7ff804102a20 2024-05-15T14:06:02.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.584+0000 7ff80a543700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff804112490 con 0x7ff804107a40 2024-05-15T14:06:02.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.584+0000 7ff80a543700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ff804112790 con 0x7ff804107080 2024-05-15T14:06:02.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.584+0000 7ff803fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff804102a20 0x7ff804104120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:02.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.584+0000 7ff803fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff804102a20 0x7ff804104120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43674/0 (socket says 172.21.15.33:43674) 2024-05-15T14:06:02.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.584+0000 7ff803fff700 1 -- 172.21.15.33:0/386921964 learned_addr learned my addr 172.21.15.33:0/386921964 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:02.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.584+0000 7ff7fbfff700 1 --2- 172.21.15.33:0/386921964 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff804107080 0x7ff804104660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:02.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.584+0000 7ff808ae0700 1 --2- 172.21.15.33:0/386921964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff804107a40 0x7ff804104ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:02.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.584+0000 7ff803fff700 1 -- 172.21.15.33:0/386921964 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff804107080 msgr2=0x7ff804104660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:02.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.584+0000 7ff803fff700 1 --2- 172.21.15.33:0/386921964 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff804107080 0x7ff804104660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.584+0000 7ff803fff700 1 -- 172.21.15.33:0/386921964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff804107a40 msgr2=0x7ff804104ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:02.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.584+0000 7ff803fff700 1 --2- 172.21.15.33:0/386921964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff804107a40 0x7ff804104ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.584+0000 7ff803fff700 1 -- 172.21.15.33:0/386921964 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8041a8030 con 0x7ff804102a20 2024-05-15T14:06:02.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.584+0000 7ff7fbfff700 1 --2- 172.21.15.33:0/386921964 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff804107080 0x7ff804104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:06:02.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.584+0000 7ff803fff700 1 --2- 172.21.15.33:0/386921964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff804102a20 0x7ff804104120 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff7f400e9c0 tx=0x7ff7f400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:02.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.585+0000 7ff801ffb700 1 -- 172.21.15.33:0/386921964 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f400cbd0 con 0x7ff804102a20 2024-05-15T14:06:02.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.585+0000 7ff801ffb700 1 -- 172.21.15.33:0/386921964 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7f400ee60 con 0x7ff804102a20 2024-05-15T14:06:02.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.585+0000 7ff80a543700 1 -- 172.21.15.33:0/386921964 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8041a8320 con 0x7ff804102a20 2024-05-15T14:06:02.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.585+0000 7ff801ffb700 1 -- 172.21.15.33:0/386921964 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f40055c0 con 0x7ff804102a20 2024-05-15T14:06:02.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.585+0000 7ff80a543700 1 -- 172.21.15.33:0/386921964 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8041a8800 con 0x7ff804102a20 2024-05-15T14:06:02.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.588+0000 7ff801ffb700 1 -- 172.21.15.33:0/386921964 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff7f4004750 con 0x7ff804102a20 2024-05-15T14:06:02.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.589+0000 7ff801ffb700 1 --2- 172.21.15.33:0/386921964 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ff7f00b2b20 0x7ff7f00b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:02.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.589+0000 7ff7fbfff700 1 --2- 172.21.15.33:0/386921964 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ff7f00b2b20 0x7ff7f00b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:02.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.589+0000 7ff801ffb700 1 -- 172.21.15.33:0/386921964 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7ff7f40e2790 con 0x7ff804102a20 2024-05-15T14:06:02.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.589+0000 7ff7fbfff700 1 --2- 172.21.15.33:0/386921964 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ff7f00b2b20 0x7ff7f00b4fe0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff80406ccc0 tx=0x7ff7ec009260 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:02.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.590+0000 7ff814dd9b80 1 Processor -- start 2024-05-15T14:06:02.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.590+0000 7ff814dd9b80 1 -- start start 2024-05-15T14:06:02.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.590+0000 7ff814dd9b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a34cd81e40 0x55a34cd82220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:02.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.590+0000 7ff814dd9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a34cd82760 0x55a34cd7c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:02.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.590+0000 7ff814dd9b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a34cd7cb70 0x55a34cd7cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:02.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.590+0000 7ff814dd9b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a34ccdae30 con 0x55a34cd7cb70 2024-05-15T14:06:02.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.590+0000 7ff814dd9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a34ccca9d0 con 0x55a34cd82760 2024-05-15T14:06:02.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.590+0000 7ff814dd9b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55a34cccea10 con 0x55a34cd81e40 2024-05-15T14:06:02.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.590+0000 7ff808ae0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a34cd7cb70 0x55a34cd7cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:02.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.590+0000 7ff7fbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a34cd82760 0x55a34cd7c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:02.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.590+0000 7ff803fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a34cd81e40 0x55a34cd82220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:02.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.590+0000 7ff808ae0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a34cd7cb70 0x55a34cd7cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43690/0 (socket says 172.21.15.33:43690) 2024-05-15T14:06:02.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.590+0000 7ff803fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a34cd81e40 0x55a34cd82220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:37930/0 (socket says 172.21.15.33:37930) 2024-05-15T14:06:02.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.590+0000 7ff7fbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a34cd82760 0x55a34cd7c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:37910/0 (socket says 172.21.15.33:37910) 2024-05-15T14:06:02.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.590+0000 7ff808ae0700 1 -- 172.21.15.33:0/4113109808 learned_addr learned my addr 172.21.15.33:0/4113109808 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:02.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.590+0000 7ff803fff700 1 -- 172.21.15.33:0/4113109808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a34cd82760 msgr2=0x55a34cd7c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:02.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.590+0000 7ff803fff700 1 --2- 172.21.15.33:0/4113109808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a34cd82760 0x55a34cd7c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.590+0000 7ff803fff700 1 -- 172.21.15.33:0/4113109808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a34cd7cb70 msgr2=0x55a34cd7cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:02.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.590+0000 7ff803fff700 1 --2- 172.21.15.33:0/4113109808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a34cd7cb70 0x55a34cd7cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.590+0000 7ff803fff700 1 -- 172.21.15.33:0/4113109808 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a34cc1aa70 con 0x55a34cd81e40 2024-05-15T14:06:02.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.591+0000 7ff803fff700 1 --2- 172.21.15.33:0/4113109808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a34cd81e40 0x55a34cd82220 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff7f40dfe90 tx=0x7ff7f40a94a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=691a2597c2cd1858 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:02.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.591+0000 7ff7fa7fc700 1 -- 172.21.15.33:0/4113109808 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f40aa120 con 0x55a34cd81e40 2024-05-15T14:06:02.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.591+0000 7ff7fa7fc700 1 -- 172.21.15.33:0/4113109808 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7f40a9a20 con 0x55a34cd81e40 2024-05-15T14:06:02.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.591+0000 7ff7fa7fc700 1 -- 172.21.15.33:0/4113109808 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f40cc370 con 0x55a34cd81e40 2024-05-15T14:06:02.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.591+0000 7ff814dd9b80 1 -- 172.21.15.33:0/4113109808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a34cd81e40 msgr2=0x55a34cd82220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:02.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.591+0000 7ff814dd9b80 1 --2- 172.21.15.33:0/4113109808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a34cd81e40 0x55a34cd82220 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff7f40dfe90 tx=0x7ff7f40a94a0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.592+0000 7ff814dd9b80 1 -- 172.21.15.33:0/4113109808 shutdown_connections 2024-05-15T14:06:02.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.592+0000 7ff814dd9b80 1 --2- 172.21.15.33:0/4113109808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a34cd7cb70 0x55a34cd7cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.592+0000 7ff814dd9b80 1 --2- 172.21.15.33:0/4113109808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a34cd81e40 0x55a34cd82220 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.592+0000 7ff814dd9b80 1 --2- 172.21.15.33:0/4113109808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a34cd82760 0x55a34cd7c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.592+0000 7ff814dd9b80 1 -- 172.21.15.33:0/4113109808 >> 172.21.15.33:0/4113109808 conn(0x55a34cca73f0 msgr2=0x55a34cd76d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:02.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.592+0000 7ff814dd9b80 1 -- 172.21.15.33:0/4113109808 shutdown_connections 2024-05-15T14:06:02.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.592+0000 7ff814dd9b80 1 -- 172.21.15.33:0/4113109808 wait complete. 2024-05-15T14:06:02.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.592+0000 7ff814dd9b80 1 -- start start 2024-05-15T14:06:02.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.592+0000 7ff814dd9b80 1 Processor -- start 2024-05-15T14:06:02.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.593+0000 7ff814dd9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a34cd7cb70 0x55a34cd884d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:02.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.593+0000 7ff814dd9b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a34cd81e40 0x55a34cd85870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:02.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.593+0000 7ff814dd9b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a34cd82760 0x55a34cd85db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:02.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.593+0000 7ff814dd9b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a34cbd1140 con 0x55a34cd82760 2024-05-15T14:06:02.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.593+0000 7ff814dd9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a34cc6c960 con 0x55a34cd7cb70 2024-05-15T14:06:02.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.593+0000 7ff814dd9b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55a34cd891a0 con 0x55a34cd81e40 2024-05-15T14:06:02.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.593+0000 7ff814dd9b80 10 client.0 _ll_register_callbacks cb 0x55a34cc61720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:06:02.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.593+0000 7ff7fbfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a34cd81e40 0x55a34cd85870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:02.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.593+0000 7ff7fbfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a34cd81e40 0x55a34cd85870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:37940/0 (socket says 172.21.15.33:37940) 2024-05-15T14:06:02.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.593+0000 7ff7fbfff700 1 -- 172.21.15.33:0/876504052 learned_addr learned my addr 172.21.15.33:0/876504052 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:02.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.593+0000 7ff808ae0700 1 --2- 172.21.15.33:0/876504052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a34cd82760 0x55a34cd85db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:02.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.593+0000 7ff803fff700 1 --2- 172.21.15.33:0/876504052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a34cd7cb70 0x55a34cd884d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:02.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.593+0000 7ff808ae0700 1 -- 172.21.15.33:0/876504052 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a34cd81e40 msgr2=0x55a34cd85870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:02.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.593+0000 7ff808ae0700 1 --2- 172.21.15.33:0/876504052 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a34cd81e40 0x55a34cd85870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.593+0000 7ff808ae0700 1 -- 172.21.15.33:0/876504052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a34cd7cb70 msgr2=0x55a34cd884d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:02.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.593+0000 7ff808ae0700 1 --2- 172.21.15.33:0/876504052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a34cd7cb70 0x55a34cd884d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.593+0000 7ff808ae0700 1 -- 172.21.15.33:0/876504052 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a34cc69830 con 0x55a34cd82760 2024-05-15T14:06:02.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.593+0000 7ff803fff700 1 --2- 172.21.15.33:0/876504052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a34cd7cb70 0x55a34cd884d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:06:02.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.593+0000 7ff7fbfff700 1 --2- 172.21.15.33:0/876504052 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a34cd81e40 0x55a34cd85870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:06:02.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.593+0000 7ff808ae0700 1 --2- 172.21.15.33:0/876504052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a34cd82760 0x55a34cd85db0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc009fd0 tx=0x7ff7fc00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e303937e73db3d server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:02.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.593+0000 7ff7e37fe700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:06:02.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.594+0000 7ff7e37fe700 1 -- 172.21.15.33:0/876504052 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc0076e0 con 0x55a34cd82760 2024-05-15T14:06:02.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.594+0000 7ff7e37fe700 1 -- 172.21.15.33:0/876504052 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7ff7fc007880 con 0x55a34cd82760 2024-05-15T14:06:02.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.594+0000 7ff814dd9b80 1 -- 172.21.15.33:0/876504052 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7f40a9a20 con 0x55a34cd82760 2024-05-15T14:06:02.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.594+0000 7ff7e37fe700 1 -- 172.21.15.33:0/876504052 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff7fc007a80 con 0x55a34cd82760 2024-05-15T14:06:02.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.594+0000 7ff814dd9b80 10 client.16047 fetch_fsmap learned FSMap version 29 2024-05-15T14:06:02.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.594+0000 7ff814dd9b80 1 -- 172.21.15.33:0/876504052 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a34cd93980 con 0x55a34cd82760 2024-05-15T14:06:02.602 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:02 smithi033 ceph-mon[31650]: pgmap v174: 97 pgs: 97 active+clean; 84 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 1.6 MiB/s wr, 67 op/s 2024-05-15T14:06:02.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.594+0000 7ff7e37fe700 1 -- 172.21.15.33:0/876504052 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7f40a9a20 con 0x55a34cd82760 2024-05-15T14:06:02.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.595+0000 7ff7e37fe700 1 -- 172.21.15.33:0/876504052 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7ff7fc011070 con 0x55a34cd82760 2024-05-15T14:06:02.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.595+0000 7ff814dd9b80 10 client.16047 fetch_fsmap finished waiting for FSMap version 29 2024-05-15T14:06:02.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.595+0000 7ff814dd9b80 10 client.16047 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:06:02.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.595+0000 7ff814dd9b80 20 client.16047 populate_metadata read hostname 'smithi033' 2024-05-15T14:06:02.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.595+0000 7ff814dd9b80 1 --2- 172.21.15.33:0/876504052 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55a34cd96450 0x55a34cd98870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:02.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.595+0000 7ff814dd9b80 4 client.16047 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:06:02.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.595+0000 7ff814dd9b80 1 -- 172.21.15.33:0/876504052 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a34cd48500 con 0x55a34cd96450 2024-05-15T14:06:02.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.595+0000 7ff803fff700 1 --2- 172.21.15.33:0/876504052 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55a34cd96450 0x55a34cd98870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:02.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.596+0000 7ff7e37fe700 10 client.16047 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:06:02.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.596+0000 7ff803fff700 1 --2- 172.21.15.33:0/876504052 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55a34cd96450 0x55a34cd98870 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2a0e4d9d7a02760 server_cookie=74e1bd794d39bf0 in_seq=0 out_seq=0 2024-05-15T14:06:02.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.596+0000 7ff7e37fe700 1 -- 172.21.15.33:0/876504052 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a34cd48500 con 0x55a34cd96450 2024-05-15T14:06:02.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.596+0000 7ff7e37fe700 10 client.16047 handle_command_reply: tid=0 2024-05-15T14:06:02.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.596+0000 7ff814dd9b80 1 client.16047 shutdown 2024-05-15T14:06:02.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.597+0000 7ff814dd9b80 1 -- 172.21.15.33:0/876504052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a34cd82760 msgr2=0x55a34cd85db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:02.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.597+0000 7ff814dd9b80 1 --2- 172.21.15.33:0/876504052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a34cd82760 0x55a34cd85db0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc009fd0 tx=0x7ff7fc00bd50 comp rx=0 tx=0).stop 2024-05-15T14:06:02.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.597+0000 7ff814dd9b80 1 -- 172.21.15.33:0/876504052 shutdown_connections 2024-05-15T14:06:02.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.597+0000 7ff814dd9b80 1 --2- 172.21.15.33:0/876504052 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55a34cd96450 0x55a34cd98870 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.597+0000 7ff814dd9b80 1 --2- 172.21.15.33:0/876504052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a34cd82760 0x55a34cd85db0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.597+0000 7ff814dd9b80 1 --2- 172.21.15.33:0/876504052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a34cd7cb70 0x55a34cd884d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.597+0000 7ff814dd9b80 1 --2- 172.21.15.33:0/876504052 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a34cd81e40 0x55a34cd85870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.597+0000 7ff814dd9b80 1 -- 172.21.15.33:0/876504052 >> 172.21.15.33:0/876504052 conn(0x55a34cca73f0 msgr2=0x55a34cd71870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:02.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.597+0000 7ff7e37fe700 0 client.16047 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:06:02.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.597+0000 7ff814dd9b80 1 -- 172.21.15.33:0/876504052 shutdown_connections 2024-05-15T14:06:02.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.597+0000 7ff814dd9b80 1 -- 172.21.15.33:0/876504052 wait complete. 2024-05-15T14:06:02.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.597+0000 7ff814dd9b80 20 client.16047 trim_cache size 0 max 16384 2024-05-15T14:06:02.606 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:02 smithi081 ceph-mon[36434]: pgmap v174: 97 pgs: 97 active+clean; 84 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 1.6 MiB/s wr, 67 op/s 2024-05-15T14:06:02.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.618+0000 7ff814dd9b80 1 Processor -- start 2024-05-15T14:06:02.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.618+0000 7ff814dd9b80 1 -- start start 2024-05-15T14:06:02.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.618+0000 7ff814dd9b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a34cd7cb70 0x55a34cd847d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:02.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.619+0000 7ff814dd9b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a34cd81e40 0x55a34cd920e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:02.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.619+0000 7ff814dd9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a34cd82760 0x55a34cd71980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:02.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.619+0000 7ff814dd9b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e400fe40 con 0x55a34cd7cb70 2024-05-15T14:06:02.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.619+0000 7ff814dd9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e400d6d0 con 0x55a34cd82760 2024-05-15T14:06:02.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.619+0000 7ff814dd9b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e400b520 con 0x55a34cd81e40 2024-05-15T14:06:02.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.619+0000 7ff803fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a34cd7cb70 0x55a34cd847d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:02.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.619+0000 7ff808ae0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a34cd82760 0x55a34cd71980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:02.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.619+0000 7ff803fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a34cd7cb70 0x55a34cd847d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43722/0 (socket says 172.21.15.33:43722) 2024-05-15T14:06:02.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.619+0000 7ff803fff700 1 -- 172.21.15.33:0/3230186300 learned_addr learned my addr 172.21.15.33:0/3230186300 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:02.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.619+0000 7ff7fbfff700 1 --2- 172.21.15.33:0/3230186300 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a34cd81e40 0x55a34cd920e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:02.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.619+0000 7ff808ae0700 1 -- 172.21.15.33:0/3230186300 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a34cd81e40 msgr2=0x55a34cd920e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:02.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.619+0000 7ff808ae0700 1 --2- 172.21.15.33:0/3230186300 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a34cd81e40 0x55a34cd920e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.619+0000 7ff808ae0700 1 -- 172.21.15.33:0/3230186300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a34cd7cb70 msgr2=0x55a34cd847d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:02.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.619+0000 7ff808ae0700 1 --2- 172.21.15.33:0/3230186300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a34cd7cb70 0x55a34cd847d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.619+0000 7ff808ae0700 1 -- 172.21.15.33:0/3230186300 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a34cd88840 con 0x55a34cd82760 2024-05-15T14:06:02.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.619+0000 7ff803fff700 1 --2- 172.21.15.33:0/3230186300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a34cd7cb70 0x55a34cd847d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:06:02.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.619+0000 7ff808ae0700 1 --2- 172.21.15.33:0/3230186300 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a34cd82760 0x55a34cd71980 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc002230 tx=0x7ff7fc007db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4a9a86e0a3ec6e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:02.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.620+0000 7ff7f9ffb700 1 -- 172.21.15.33:0/3230186300 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc005090 con 0x55a34cd82760 2024-05-15T14:06:02.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.620+0000 7ff7f9ffb700 1 -- 172.21.15.33:0/3230186300 <== mon.2 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7fc021070 con 0x55a34cd82760 2024-05-15T14:06:02.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.620+0000 7ff7f9ffb700 1 -- 172.21.15.33:0/3230186300 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc005490 con 0x55a34cd82760 2024-05-15T14:06:02.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.620+0000 7ff814dd9b80 1 -- 172.21.15.33:0/3230186300 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a34cd82760 msgr2=0x55a34cd71980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:02.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.620+0000 7ff814dd9b80 1 --2- 172.21.15.33:0/3230186300 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a34cd82760 0x55a34cd71980 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc002230 tx=0x7ff7fc007db0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.620+0000 7ff814dd9b80 1 -- 172.21.15.33:0/3230186300 shutdown_connections 2024-05-15T14:06:02.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.620+0000 7ff814dd9b80 1 --2- 172.21.15.33:0/3230186300 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a34cd82760 0x55a34cd71980 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.620+0000 7ff814dd9b80 1 --2- 172.21.15.33:0/3230186300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a34cd7cb70 0x55a34cd847d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.620+0000 7ff814dd9b80 1 --2- 172.21.15.33:0/3230186300 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a34cd81e40 0x55a34cd920e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.621+0000 7ff814dd9b80 1 -- 172.21.15.33:0/3230186300 >> 172.21.15.33:0/3230186300 conn(0x55a34cca73f0 msgr2=0x55a34cd82b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:02.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.621+0000 7ff814dd9b80 1 -- 172.21.15.33:0/3230186300 shutdown_connections 2024-05-15T14:06:02.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.621+0000 7ff814dd9b80 1 -- 172.21.15.33:0/3230186300 wait complete. 2024-05-15T14:06:02.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.621+0000 7ff814dd9b80 1 -- start start 2024-05-15T14:06:02.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.621+0000 7ff814dd9b80 1 Processor -- start 2024-05-15T14:06:02.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.621+0000 7ff814dd9b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a34cd7cb70 0x55a34cd8eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:02.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.621+0000 7ff814dd9b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a34cd81e40 0x55a34cd9d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:02.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.621+0000 7ff814dd9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a34cd82760 0x55a34cd9dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:02.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.621+0000 7ff814dd9b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e400bb90 con 0x55a34cd81e40 2024-05-15T14:06:02.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.621+0000 7ff814dd9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e400d000 con 0x55a34cd82760 2024-05-15T14:06:02.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.621+0000 7ff814dd9b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e400c260 con 0x55a34cd7cb70 2024-05-15T14:06:02.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.621+0000 7ff814dd9b80 10 client.0 _ll_register_callbacks cb 0x55a34cc61720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:06:02.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.622+0000 7ff7fbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a34cd81e40 0x55a34cd9d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:02.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.622+0000 7ff803fff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a34cd7cb70 0x55a34cd8eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:02.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.622+0000 7ff7fbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a34cd81e40 0x55a34cd9d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43734/0 (socket says 172.21.15.33:43734) 2024-05-15T14:06:02.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.622+0000 7ff7fbfff700 1 -- 172.21.15.33:0/2317396223 learned_addr learned my addr 172.21.15.33:0/2317396223 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:02.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.622+0000 7ff808ae0700 1 --2- 172.21.15.33:0/2317396223 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a34cd82760 0x55a34cd9dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:02.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.622+0000 7ff7fbfff700 1 -- 172.21.15.33:0/2317396223 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a34cd7cb70 msgr2=0x55a34cd8eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:02.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.622+0000 7ff7fbfff700 1 --2- 172.21.15.33:0/2317396223 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a34cd7cb70 0x55a34cd8eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.622+0000 7ff7fbfff700 1 -- 172.21.15.33:0/2317396223 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a34cd82760 msgr2=0x55a34cd9dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:02.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.622+0000 7ff7fbfff700 1 --2- 172.21.15.33:0/2317396223 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a34cd82760 0x55a34cd9dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.622+0000 7ff7fbfff700 1 -- 172.21.15.33:0/2317396223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a34cdffd10 con 0x55a34cd81e40 2024-05-15T14:06:02.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.622+0000 7ff803fff700 1 --2- 172.21.15.33:0/2317396223 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a34cd7cb70 0x55a34cd8eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:06:02.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.622+0000 7ff808ae0700 1 --2- 172.21.15.33:0/2317396223 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a34cd82760 0x55a34cd9dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:06:02.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.622+0000 7ff7fbfff700 1 --2- 172.21.15.33:0/2317396223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a34cd81e40 0x55a34cd9d870 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x55a34cd776b0 tx=0x7ff7ec0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd0d41af0eaf644 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:02.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.622+0000 7ff7e37fe700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:06:02.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.622+0000 7ff7e37fe700 1 -- 172.21.15.33:0/2317396223 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec02f060 con 0x55a34cd81e40 2024-05-15T14:06:02.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.622+0000 7ff7e37fe700 1 -- 172.21.15.33:0/2317396223 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7ff7ec004030 con 0x55a34cd81e40 2024-05-15T14:06:02.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.622+0000 7ff814dd9b80 1 -- 172.21.15.33:0/2317396223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7fc021070 con 0x55a34cd81e40 2024-05-15T14:06:02.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.623+0000 7ff7e37fe700 1 -- 172.21.15.33:0/2317396223 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff7ec004230 con 0x55a34cd81e40 2024-05-15T14:06:02.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.623+0000 7ff814dd9b80 10 client.16053 fetch_fsmap learned FSMap version 29 2024-05-15T14:06:02.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.623+0000 7ff814dd9b80 1 -- 172.21.15.33:0/2317396223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a34ce00f60 con 0x55a34cd81e40 2024-05-15T14:06:02.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.623+0000 7ff7e37fe700 1 -- 172.21.15.33:0/2317396223 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7fc021070 con 0x55a34cd81e40 2024-05-15T14:06:02.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.623+0000 7ff7e37fe700 1 -- 172.21.15.33:0/2317396223 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7ff7ec02a620 con 0x55a34cd81e40 2024-05-15T14:06:02.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.623+0000 7ff814dd9b80 10 client.16053 fetch_fsmap finished waiting for FSMap version 29 2024-05-15T14:06:02.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.623+0000 7ff814dd9b80 10 client.16053 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:06:02.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.623+0000 7ff814dd9b80 20 client.16053 populate_metadata read hostname 'smithi033' 2024-05-15T14:06:02.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.623+0000 7ff814dd9b80 1 --2- 172.21.15.33:0/2317396223 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55a34ce01710 0x55a34ce03b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:02.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.623+0000 7ff814dd9b80 4 client.16053 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.35, "await_for": 45.0}] 2024-05-15T14:06:02.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.623+0000 7ff814dd9b80 1 -- 172.21.15.33:0/2317396223 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.35, "await_for": 45.0}) -- 0x55a34ccdcd80 con 0x55a34ce01710 2024-05-15T14:06:02.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.623+0000 7ff803fff700 1 --2- 172.21.15.33:0/2317396223 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55a34ce01710 0x55a34ce03b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:02.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.624+0000 7ff803fff700 1 --2- 172.21.15.33:0/2317396223 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55a34ce01710 0x55a34ce03b30 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fda23b97557fb19 server_cookie=98d7ca7e9b01ac63 in_seq=0 out_seq=0 2024-05-15T14:06:02.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.624+0000 7ff7e37fe700 10 client.16053 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:06:02.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.698+0000 7ff7e37fe700 1 -- 172.21.15.33:0/2317396223 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a34ccdcd80 con 0x55a34ce01710 2024-05-15T14:06:02.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.698+0000 7ff7e37fe700 10 client.16053 handle_command_reply: tid=0 2024-05-15T14:06:02.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.698+0000 7ff814dd9b80 1 client.16053 shutdown 2024-05-15T14:06:02.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.698+0000 7ff814dd9b80 1 -- 172.21.15.33:0/2317396223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a34cd81e40 msgr2=0x55a34cd9d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:02.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.698+0000 7ff814dd9b80 1 --2- 172.21.15.33:0/2317396223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a34cd81e40 0x55a34cd9d870 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x55a34cd776b0 tx=0x7ff7ec0047c0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.698+0000 7ff814dd9b80 1 -- 172.21.15.33:0/2317396223 shutdown_connections 2024-05-15T14:06:02.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.698+0000 7ff814dd9b80 1 --2- 172.21.15.33:0/2317396223 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a34cd82760 0x55a34cd9dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.698+0000 7ff814dd9b80 1 --2- 172.21.15.33:0/2317396223 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a34cd7cb70 0x55a34cd8eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.698+0000 7ff814dd9b80 1 --2- 172.21.15.33:0/2317396223 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55a34ce01710 0x55a34ce03b30 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.698+0000 7ff814dd9b80 1 --2- 172.21.15.33:0/2317396223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a34cd81e40 0x55a34cd9d870 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.698+0000 7ff814dd9b80 1 -- 172.21.15.33:0/2317396223 >> 172.21.15.33:0/2317396223 conn(0x55a34cca73f0 msgr2=0x55a34cd833d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:02.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.698+0000 7ff7e37fe700 0 client.16053 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:06:02.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.698+0000 7ff814dd9b80 1 -- 172.21.15.33:0/2317396223 shutdown_connections 2024-05-15T14:06:02.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.698+0000 7ff814dd9b80 1 -- 172.21.15.33:0/2317396223 wait complete. 2024-05-15T14:06:02.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.698+0000 7ff814dd9b80 20 client.16053 trim_cache size 0 max 16384 2024-05-15T14:06:02.703 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T14:06:02.703 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 29, 2024-05-15T14:06:02.703 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24478, 2024-05-15T14:06:02.703 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 22, 2024-05-15T14:06:02.704 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-05-15T14:06:02.704 INFO:teuthology.orchestra.run.smithi033.stdout: "b5682d61": { 2024-05-15T14:06:02.704 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 22, 2024-05-15T14:06:02.704 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-05-15T14:06:02.704 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:06:02.704 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-05-15T14:06:02.704 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:06:02.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:06:02.704 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 90.0, 2024-05-15T14:06:02.704 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 138.3, 2024-05-15T14:06:02.704 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-05-15T14:06:02.704 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-05-15T14:06:02.704 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-05-15T14:06:02.704 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:06:02.704 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-05-15T14:06:02.705 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:06:02.705 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:06:02.705 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:06:02.705 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:06:02.705 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:06:02.705 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:06:02.705 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T14:06:02.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.699+0000 7ff80a543700 1 -- 172.21.15.33:0/386921964 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ff7f00b2b20 msgr2=0x7ff7f00b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:02.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.699+0000 7ff80a543700 1 --2- 172.21.15.33:0/386921964 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ff7f00b2b20 0x7ff7f00b4fe0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff80406ccc0 tx=0x7ff7ec009260 comp rx=0 tx=0).stop 2024-05-15T14:06:02.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.699+0000 7ff80a543700 1 -- 172.21.15.33:0/386921964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff804102a20 msgr2=0x7ff804104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:02.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.699+0000 7ff80a543700 1 --2- 172.21.15.33:0/386921964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff804102a20 0x7ff804104120 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff7f400e9c0 tx=0x7ff7f400ecd0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.699+0000 7ff80a543700 1 -- 172.21.15.33:0/386921964 shutdown_connections 2024-05-15T14:06:02.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.699+0000 7ff80a543700 1 --2- 172.21.15.33:0/386921964 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ff7f00b2b20 0x7ff7f00b4fe0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.699+0000 7ff80a543700 1 --2- 172.21.15.33:0/386921964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff804107a40 0x7ff804104ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.699+0000 7ff80a543700 1 --2- 172.21.15.33:0/386921964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff804102a20 0x7ff804104120 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.699+0000 7ff80a543700 1 --2- 172.21.15.33:0/386921964 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff804107080 0x7ff804104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:02.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.699+0000 7ff80a543700 1 -- 172.21.15.33:0/386921964 >> 172.21.15.33:0/386921964 conn(0x7ff8040791a0 msgr2=0x7ff8041062c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:02.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.699+0000 7ff80a543700 1 -- 172.21.15.33:0/386921964 shutdown_connections 2024-05-15T14:06:02.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:02.699+0000 7ff80a543700 1 -- 172.21.15.33:0/386921964 wait complete. 2024-05-15T14:06:02.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:02 smithi052 ceph-mon[36190]: pgmap v174: 97 pgs: 97 active+clean; 84 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 1.6 MiB/s wr, 67 op/s 2024-05-15T14:06:03.059 INFO:tasks.workunit.client.0.smithi033.stdout: 1 342407 7.06 MB/sec execute 98 sec latency 344.602 ms 2024-05-15T14:06:03.371 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b5682d61', quiesce took 5.6 seconds. Will release after: 12.750000000000002 2024-05-15T14:06:04.059 INFO:tasks.workunit.client.0.smithi033.stdout: 1 342407 6.99 MB/sec execute 99 sec latency 1344.666 ms 2024-05-15T14:06:04.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:04 smithi081 ceph-mon[36434]: pgmap v175: 97 pgs: 97 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 223 op/s 2024-05-15T14:06:04.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:04 smithi033 ceph-mon[31650]: pgmap v175: 97 pgs: 97 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 223 op/s 2024-05-15T14:06:04.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:04 smithi052 ceph-mon[36190]: pgmap v175: 97 pgs: 97 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 223 op/s 2024-05-15T14:06:05.059 INFO:tasks.workunit.client.0.smithi033.stdout: 1 342407 6.92 MB/sec execute 100 sec latency 2344.735 ms 2024-05-15T14:06:06.059 INFO:tasks.workunit.client.0.smithi033.stdout: 1 342407 6.85 MB/sec execute 101 sec latency 3344.836 ms 2024-05-15T14:06:06.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:06 smithi081 ceph-mon[36434]: pgmap v176: 97 pgs: 97 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 196 op/s 2024-05-15T14:06:06.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:06 smithi033 ceph-mon[31650]: pgmap v176: 97 pgs: 97 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 196 op/s 2024-05-15T14:06:06.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.753+0000 7f4a7d789640 1 -- 172.21.15.33:0/1673618122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a781045e0 msgr2=0x7f4a7810cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:06.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.753+0000 7f4a7d789640 1 --2- 172.21.15.33:0/1673618122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a781045e0 0x7f4a7810cac0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f4a60009900 tx=0x7f4a6002a480 comp rx=0 tx=0).stop 2024-05-15T14:06:06.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.753+0000 7f4a7d789640 1 -- 172.21.15.33:0/1673618122 shutdown_connections 2024-05-15T14:06:06.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.753+0000 7f4a7d789640 1 --2- 172.21.15.33:0/1673618122 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a7810d090 0x7f4a7810f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:06.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.753+0000 7f4a7d789640 1 --2- 172.21.15.33:0/1673618122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a781045e0 0x7f4a7810cac0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:06.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.753+0000 7f4a7d789640 1 --2- 172.21.15.33:0/1673618122 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4a78103cc0 0x7f4a781040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:06.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.753+0000 7f4a7d789640 1 -- 172.21.15.33:0/1673618122 >> 172.21.15.33:0/1673618122 conn(0x7f4a78078a70 msgr2=0x7f4a78078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:06.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.753+0000 7f4a7d789640 1 -- 172.21.15.33:0/1673618122 shutdown_connections 2024-05-15T14:06:06.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.753+0000 7f4a7d789640 1 -- 172.21.15.33:0/1673618122 wait complete. 2024-05-15T14:06:06.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.754+0000 7f4a7d789640 1 Processor -- start 2024-05-15T14:06:06.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.754+0000 7f4a7d789640 1 -- start start 2024-05-15T14:06:06.759 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.754+0000 7f4a7d789640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4a78103cc0 0x7f4a78199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:06.759 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.754+0000 7f4a7d789640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a781045e0 0x7f4a781997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:06.759 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.754+0000 7f4a7d789640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a7810d090 0x7f4a7819db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:06.759 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.754+0000 7f4a7d789640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4a78112cd0 con 0x7f4a7810d090 2024-05-15T14:06:06.759 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.754+0000 7f4a7d789640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4a78112b50 con 0x7f4a781045e0 2024-05-15T14:06:06.759 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.754+0000 7f4a7d789640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4a78112e50 con 0x7f4a78103cc0 2024-05-15T14:06:06.759 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.755+0000 7f4a777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a7810d090 0x7f4a7819db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:06.759 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.755+0000 7f4a777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a7810d090 0x7f4a7819db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46526/0 (socket says 172.21.15.33:46526) 2024-05-15T14:06:06.759 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.755+0000 7f4a777fe640 1 -- 172.21.15.33:0/3756385286 learned_addr learned my addr 172.21.15.33:0/3756385286 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.755+0000 7f4a767fc640 1 --2- 172.21.15.33:0/3756385286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a781045e0 0x7f4a781997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.755+0000 7f4a76ffd640 1 --2- 172.21.15.33:0/3756385286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4a78103cc0 0x7f4a78199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.755+0000 7f4a777fe640 1 -- 172.21.15.33:0/3756385286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4a78103cc0 msgr2=0x7f4a78199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.755+0000 7f4a777fe640 1 --2- 172.21.15.33:0/3756385286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4a78103cc0 0x7f4a78199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.755+0000 7f4a777fe640 1 -- 172.21.15.33:0/3756385286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a781045e0 msgr2=0x7f4a781997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.755+0000 7f4a777fe640 1 --2- 172.21.15.33:0/3756385286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a781045e0 0x7f4a781997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.755+0000 7f4a777fe640 1 -- 172.21.15.33:0/3756385286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a7819e260 con 0x7f4a7810d090 2024-05-15T14:06:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.755+0000 7f4a76ffd640 1 --2- 172.21.15.33:0/3756385286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4a78103cc0 0x7f4a78199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:06:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.755+0000 7f4a777fe640 1 --2- 172.21.15.33:0/3756385286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a7810d090 0x7f4a7819db30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000eb40 tx=0x7f4a7000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.755+0000 7f4a6ffff640 1 -- 172.21.15.33:0/3756385286 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70004b20 con 0x7f4a7810d090 2024-05-15T14:06:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.755+0000 7f4a7d789640 1 -- 172.21.15.33:0/3756385286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a7819e550 con 0x7f4a7810d090 2024-05-15T14:06:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.755+0000 7f4a6ffff640 1 -- 172.21.15.33:0/3756385286 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a70004cc0 con 0x7f4a7810d090 2024-05-15T14:06:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.755+0000 7f4a6ffff640 1 -- 172.21.15.33:0/3756385286 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70010720 con 0x7f4a7810d090 2024-05-15T14:06:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.755+0000 7f4a7d789640 1 -- 172.21.15.33:0/3756385286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a781a5de0 con 0x7f4a7810d090 2024-05-15T14:06:06.761 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.758+0000 7f4a8057a740 1 Processor -- start 2024-05-15T14:06:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.759+0000 7f4a6ffff640 1 -- 172.21.15.33:0/3756385286 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4a700108c0 con 0x7f4a7810d090 2024-05-15T14:06:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.759+0000 7f4a8057a740 1 -- start start 2024-05-15T14:06:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.759+0000 7f4a8057a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55849432c8f0 0x55849432ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.759+0000 7f4a8057a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55849432d210 0x558494337590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.759+0000 7f4a8057a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558494337ad0 0x558494339ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.760+0000 7f4a8057a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55849429fc30 con 0x55849432c8f0 2024-05-15T14:06:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.760+0000 7f4a8057a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55849428f7d0 con 0x55849432d210 2024-05-15T14:06:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.760+0000 7f4a8057a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x558494293810 con 0x558494337ad0 2024-05-15T14:06:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.760+0000 7f4a767fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55849432c8f0 0x55849432ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.760+0000 7f4a767fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55849432c8f0 0x55849432ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46542/0 (socket says 172.21.15.33:46542) 2024-05-15T14:06:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.760+0000 7f4a767fc640 1 -- 172.21.15.33:0/377844473 learned_addr learned my addr 172.21.15.33:0/377844473 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.760+0000 7f4a76ffd640 1 --2- 172.21.15.33:0/377844473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55849432d210 0x558494337590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.760+0000 7f4a767fc640 1 -- 172.21.15.33:0/377844473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558494337ad0 msgr2=0x558494339ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.760+0000 7f4a767fc640 1 --2- 172.21.15.33:0/377844473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558494337ad0 0x558494339ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.760+0000 7f4a767fc640 1 -- 172.21.15.33:0/377844473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55849432d210 msgr2=0x558494337590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.760+0000 7f4a767fc640 1 --2- 172.21.15.33:0/377844473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55849432d210 0x558494337590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.760+0000 7f4a767fc640 1 -- 172.21.15.33:0/377844473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558493fa4210 con 0x55849432c8f0 2024-05-15T14:06:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.760+0000 7f4a6ffff640 1 --2- 172.21.15.33:0/3756385286 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f4a640b2950 0x7f4a640b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.760+0000 7f4a767fc640 1 --2- 172.21.15.33:0/377844473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55849432c8f0 0x55849432ccd0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4a6002a9a0 tx=0x7f4a60032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b51e52f2e1f73c server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.760+0000 7f4a76ffd640 1 --2- 172.21.15.33:0/3756385286 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f4a640b2950 0x7f4a640b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.760+0000 7f4a6d7fa640 1 -- 172.21.15.33:0/377844473 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a60035050 con 0x55849432c8f0 2024-05-15T14:06:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.760+0000 7f4a6ffff640 1 -- 172.21.15.33:0/3756385286 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f4a700e2e30 con 0x7f4a7810d090 2024-05-15T14:06:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.761+0000 7f4a6d7fa640 1 -- 172.21.15.33:0/377844473 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a60004030 con 0x55849432c8f0 2024-05-15T14:06:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.761+0000 7f4a6d7fa640 1 -- 172.21.15.33:0/377844473 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a600054a0 con 0x55849432c8f0 2024-05-15T14:06:06.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.761+0000 7f4a8057a740 1 -- 172.21.15.33:0/377844473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55849432c8f0 msgr2=0x55849432ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:06.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.761+0000 7f4a76ffd640 1 --2- 172.21.15.33:0/3756385286 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f4a640b2950 0x7f4a640b4e10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55849432de90 tx=0x7f4a6800b570 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:06.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.761+0000 7f4a8057a740 1 --2- 172.21.15.33:0/377844473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55849432c8f0 0x55849432ccd0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4a6002a9a0 tx=0x7f4a60032c20 comp rx=0 tx=0).stop 2024-05-15T14:06:06.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.761+0000 7f4a8057a740 1 -- 172.21.15.33:0/377844473 shutdown_connections 2024-05-15T14:06:06.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.761+0000 7f4a8057a740 1 --2- 172.21.15.33:0/377844473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558494337ad0 0x558494339ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:06.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.761+0000 7f4a8057a740 1 --2- 172.21.15.33:0/377844473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55849432d210 0x558494337590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:06.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.761+0000 7f4a8057a740 1 --2- 172.21.15.33:0/377844473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55849432c8f0 0x55849432ccd0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:06.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.761+0000 7f4a8057a740 1 -- 172.21.15.33:0/377844473 >> 172.21.15.33:0/377844473 conn(0x558494328e00 msgr2=0x5584943291e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:06.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.761+0000 7f4a8057a740 1 -- 172.21.15.33:0/377844473 shutdown_connections 2024-05-15T14:06:06.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.761+0000 7f4a8057a740 1 -- 172.21.15.33:0/377844473 wait complete. 2024-05-15T14:06:06.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.762+0000 7f4a8057a740 1 -- start start 2024-05-15T14:06:06.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.762+0000 7f4a8057a740 1 Processor -- start 2024-05-15T14:06:06.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.762+0000 7f4a8057a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55849432c8f0 0x558494337420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:06.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.762+0000 7f4a8057a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55849432d210 0x558494334710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:06.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.762+0000 7f4a8057a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558494337ad0 0x558494334c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:06.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.762+0000 7f4a8057a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558493f75790 con 0x55849432c8f0 2024-05-15T14:06:06.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.762+0000 7f4a8057a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55849424b890 con 0x558494337ad0 2024-05-15T14:06:06.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.762+0000 7f4a8057a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55849433eb50 con 0x55849432d210 2024-05-15T14:06:06.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.762+0000 7f4a8057a740 10 client.0 _ll_register_callbacks cb 0x55849424ed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:06:06.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.762+0000 7f4a76ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55849432c8f0 0x558494337420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:06.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.762+0000 7f4a76ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55849432c8f0 0x558494337420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46548/0 (socket says 172.21.15.33:46548) 2024-05-15T14:06:06.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.762+0000 7f4a76ffd640 1 -- 172.21.15.33:0/2587125759 learned_addr learned my addr 172.21.15.33:0/2587125759 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:06.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.762+0000 7f4a767fc640 1 --2- 172.21.15.33:0/2587125759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55849432d210 0x558494334710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:06.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.762+0000 7f4a777fe640 1 --2- 172.21.15.33:0/2587125759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558494337ad0 0x558494334c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:06.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.763+0000 7f4a76ffd640 1 -- 172.21.15.33:0/2587125759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55849432d210 msgr2=0x558494334710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:06.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.763+0000 7f4a76ffd640 1 --2- 172.21.15.33:0/2587125759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55849432d210 0x558494334710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:06.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.763+0000 7f4a76ffd640 1 -- 172.21.15.33:0/2587125759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558494337ad0 msgr2=0x558494334c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:06.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.763+0000 7f4a76ffd640 1 --2- 172.21.15.33:0/2587125759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558494337ad0 0x558494334c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:06.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.763+0000 7f4a76ffd640 1 -- 172.21.15.33:0/2587125759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558494070620 con 0x55849432c8f0 2024-05-15T14:06:06.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.763+0000 7f4a76ffd640 1 --2- 172.21.15.33:0/2587125759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55849432c8f0 0x558494337420 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4a680108b0 tx=0x7f4a680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a333b38ea1d68749 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:06.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.763+0000 7f4a5e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:06:06.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.763+0000 7f4a5e7fc640 1 -- 172.21.15.33:0/2587125759 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6800fa40 con 0x55849432c8f0 2024-05-15T14:06:06.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.763+0000 7f4a5e7fc640 1 -- 172.21.15.33:0/2587125759 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f4a6800fbe0 con 0x55849432c8f0 2024-05-15T14:06:06.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.763+0000 7f4a8057a740 1 -- 172.21.15.33:0/2587125759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a60004030 con 0x55849432c8f0 2024-05-15T14:06:06.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.763+0000 7f4a5e7fc640 1 -- 172.21.15.33:0/2587125759 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4a6800fe20 con 0x55849432c8f0 2024-05-15T14:06:06.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.763+0000 7f4a8057a740 10 client.16065 fetch_fsmap learned FSMap version 29 2024-05-15T14:06:06.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.763+0000 7f4a8057a740 1 -- 172.21.15.33:0/2587125759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55849434a690 con 0x55849432c8f0 2024-05-15T14:06:06.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.763+0000 7f4a5e7fc640 1 -- 172.21.15.33:0/2587125759 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a60004030 con 0x55849432c8f0 2024-05-15T14:06:06.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.764+0000 7f4a5e7fc640 1 -- 172.21.15.33:0/2587125759 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f4a6801f580 con 0x55849432c8f0 2024-05-15T14:06:06.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.764+0000 7f4a8057a740 10 client.16065 fetch_fsmap finished waiting for FSMap version 29 2024-05-15T14:06:06.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.764+0000 7f4a8057a740 10 client.16065 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:06:06.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.764+0000 7f4a8057a740 20 client.16065 populate_metadata read hostname 'smithi033' 2024-05-15T14:06:06.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.764+0000 7f4a8057a740 1 --2- 172.21.15.33:0/2587125759 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55849434bb50 0x55849434df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:06.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.764+0000 7f4a8057a740 4 client.16065 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:06:06.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.764+0000 7f4a8057a740 1 -- 172.21.15.33:0/2587125759 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584942a1b80 con 0x55849434bb50 2024-05-15T14:06:06.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.764+0000 7f4a767fc640 1 --2- 172.21.15.33:0/2587125759 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55849434bb50 0x55849434df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.764+0000 7f4a5e7fc640 10 client.16065 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:06:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.764+0000 7f4a767fc640 1 --2- 172.21.15.33:0/2587125759 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55849434bb50 0x55849434df40 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91600c16abf562aa server_cookie=1652afcac3de592 in_seq=0 out_seq=0 2024-05-15T14:06:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.765+0000 7f4a5e7fc640 1 -- 172.21.15.33:0/2587125759 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5584942a1b80 con 0x55849434bb50 2024-05-15T14:06:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.765+0000 7f4a5e7fc640 10 client.16065 handle_command_reply: tid=0 2024-05-15T14:06:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.765+0000 7f4a8057a740 1 client.16065 shutdown 2024-05-15T14:06:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.765+0000 7f4a8057a740 1 -- 172.21.15.33:0/2587125759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55849432c8f0 msgr2=0x558494337420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.765+0000 7f4a8057a740 1 --2- 172.21.15.33:0/2587125759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55849432c8f0 0x558494337420 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4a680108b0 tx=0x7f4a680046c0 comp rx=0 tx=0).stop 2024-05-15T14:06:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.765+0000 7f4a8057a740 1 -- 172.21.15.33:0/2587125759 shutdown_connections 2024-05-15T14:06:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.765+0000 7f4a8057a740 1 --2- 172.21.15.33:0/2587125759 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55849434bb50 0x55849434df40 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.765+0000 7f4a8057a740 1 --2- 172.21.15.33:0/2587125759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558494337ad0 0x558494334c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.765+0000 7f4a8057a740 1 --2- 172.21.15.33:0/2587125759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55849432d210 0x558494334710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.765+0000 7f4a8057a740 1 --2- 172.21.15.33:0/2587125759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55849432c8f0 0x558494337420 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.765+0000 7f4a5e7fc640 0 client.16065 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:06:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.765+0000 7f4a8057a740 1 -- 172.21.15.33:0/2587125759 >> 172.21.15.33:0/2587125759 conn(0x558494328e00 msgr2=0x558494328100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.765+0000 7f4a8057a740 1 -- 172.21.15.33:0/2587125759 shutdown_connections 2024-05-15T14:06:06.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.766+0000 7f4a8057a740 1 -- 172.21.15.33:0/2587125759 wait complete. 2024-05-15T14:06:06.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.766+0000 7f4a8057a740 20 client.16065 trim_cache size 0 max 16384 2024-05-15T14:06:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.781+0000 7f4a8057a740 1 Processor -- start 2024-05-15T14:06:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.781+0000 7f4a8057a740 1 -- start start 2024-05-15T14:06:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.781+0000 7f4a8057a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55849432c8f0 0x558494347a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.781+0000 7f4a8057a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55849432d210 0x558494353830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.781+0000 7f4a8057a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558494337ad0 0x558494353d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.781+0000 7f4a8057a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4a781aec10 con 0x55849432c8f0 2024-05-15T14:06:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.781+0000 7f4a8057a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4a781ac4e0 con 0x55849432d210 2024-05-15T14:06:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.781+0000 7f4a8057a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4a781aa330 con 0x558494337ad0 2024-05-15T14:06:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.781+0000 7f4a76ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55849432c8f0 0x558494347a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.781+0000 7f4a76ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55849432c8f0 0x558494347a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46556/0 (socket says 172.21.15.33:46556) 2024-05-15T14:06:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.781+0000 7f4a76ffd640 1 -- 172.21.15.33:0/2527704742 learned_addr learned my addr 172.21.15.33:0/2527704742 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.781+0000 7f4a777fe640 1 --2- 172.21.15.33:0/2527704742 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558494337ad0 0x558494353d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.781+0000 7f4a767fc640 1 --2- 172.21.15.33:0/2527704742 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55849432d210 0x558494353830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.782+0000 7f4a76ffd640 1 -- 172.21.15.33:0/2527704742 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558494337ad0 msgr2=0x558494353d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.782+0000 7f4a76ffd640 1 --2- 172.21.15.33:0/2527704742 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558494337ad0 0x558494353d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.782+0000 7f4a76ffd640 1 -- 172.21.15.33:0/2527704742 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55849432d210 msgr2=0x558494353830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.782+0000 7f4a76ffd640 1 --2- 172.21.15.33:0/2527704742 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55849432d210 0x558494353830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.782+0000 7f4a76ffd640 1 -- 172.21.15.33:0/2527704742 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558494324600 con 0x55849432c8f0 2024-05-15T14:06:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.782+0000 7f4a777fe640 1 --2- 172.21.15.33:0/2527704742 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558494337ad0 0x558494353d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:06:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.782+0000 7f4a76ffd640 1 --2- 172.21.15.33:0/2527704742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55849432c8f0 0x558494347a50 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4a68004770 tx=0x7f4a68005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=867395ba830a0cde server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.782+0000 7f4a6cff9640 1 -- 172.21.15.33:0/2527704742 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6803e070 con 0x55849432c8f0 2024-05-15T14:06:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.782+0000 7f4a6cff9640 1 -- 172.21.15.33:0/2527704742 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a6801fdc0 con 0x55849432c8f0 2024-05-15T14:06:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.782+0000 7f4a6cff9640 1 -- 172.21.15.33:0/2527704742 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a680397a0 con 0x55849432c8f0 2024-05-15T14:06:06.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.783+0000 7f4a8057a740 1 -- 172.21.15.33:0/2527704742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55849432c8f0 msgr2=0x558494347a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:06.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.783+0000 7f4a8057a740 1 --2- 172.21.15.33:0/2527704742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55849432c8f0 0x558494347a50 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4a68004770 tx=0x7f4a68005bf0 comp rx=0 tx=0).stop 2024-05-15T14:06:06.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.783+0000 7f4a8057a740 1 -- 172.21.15.33:0/2527704742 shutdown_connections 2024-05-15T14:06:06.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.783+0000 7f4a8057a740 1 --2- 172.21.15.33:0/2527704742 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558494337ad0 0x558494353d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:06.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.783+0000 7f4a8057a740 1 --2- 172.21.15.33:0/2527704742 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55849432d210 0x558494353830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:06.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.783+0000 7f4a8057a740 1 --2- 172.21.15.33:0/2527704742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55849432c8f0 0x558494347a50 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.783+0000 7f4a8057a740 1 -- 172.21.15.33:0/2527704742 >> 172.21.15.33:0/2527704742 conn(0x558494328e00 msgr2=0x558494332fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.783+0000 7f4a8057a740 1 -- 172.21.15.33:0/2527704742 shutdown_connections 2024-05-15T14:06:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.783+0000 7f4a8057a740 1 -- 172.21.15.33:0/2527704742 wait complete. 2024-05-15T14:06:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.783+0000 7f4a8057a740 1 -- start start 2024-05-15T14:06:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.783+0000 7f4a8057a740 1 Processor -- start 2024-05-15T14:06:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.783+0000 7f4a8057a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55849432c8f0 0x558494341900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.783+0000 7f4a8057a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55849432d210 0x558494341e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.783+0000 7f4a8057a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558494337ad0 0x558494342380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.783+0000 7f4a8057a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4a781aa9a0 con 0x558494337ad0 2024-05-15T14:06:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.783+0000 7f4a8057a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4a781abe10 con 0x55849432c8f0 2024-05-15T14:06:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.783+0000 7f4a8057a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4a781ab070 con 0x55849432d210 2024-05-15T14:06:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.784+0000 7f4a8057a740 10 client.0 _ll_register_callbacks cb 0x7f4a68010460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:06:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.784+0000 7f4a777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558494337ad0 0x558494342380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.784+0000 7f4a777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558494337ad0 0x558494342380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46570/0 (socket says 172.21.15.33:46570) 2024-05-15T14:06:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.784+0000 7f4a777fe640 1 -- 172.21.15.33:0/3282515383 learned_addr learned my addr 172.21.15.33:0/3282515383 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.784+0000 7f4a76ffd640 1 --2- 172.21.15.33:0/3282515383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55849432c8f0 0x558494341900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.784+0000 7f4a777fe640 1 -- 172.21.15.33:0/3282515383 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55849432d210 msgr2=0x558494341e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.784+0000 7f4a777fe640 1 --2- 172.21.15.33:0/3282515383 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55849432d210 0x558494341e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.784+0000 7f4a777fe640 1 -- 172.21.15.33:0/3282515383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55849432c8f0 msgr2=0x558494341900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.784+0000 7f4a777fe640 1 --2- 172.21.15.33:0/3282515383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55849432c8f0 0x558494341900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.784+0000 7f4a777fe640 1 -- 172.21.15.33:0/3282515383 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584943b53f0 con 0x558494337ad0 2024-05-15T14:06:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.784+0000 7f4a76ffd640 1 --2- 172.21.15.33:0/3282515383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55849432c8f0 0x558494341900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:06:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.784+0000 7f4a777fe640 1 --2- 172.21.15.33:0/3282515383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558494337ad0 0x558494342380 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4a700aa0b0 tx=0x7f4a700a9890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e119acac34cba6c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.784+0000 7f4a5e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:06:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.784+0000 7f4a5e7fc640 1 -- 172.21.15.33:0/3282515383 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a700c3020 con 0x558494337ad0 2024-05-15T14:06:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.784+0000 7f4a5e7fc640 1 -- 172.21.15.33:0/3282515383 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f4a700b2af0 con 0x558494337ad0 2024-05-15T14:06:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.784+0000 7f4a8057a740 1 -- 172.21.15.33:0/3282515383 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a6801fdc0 con 0x558494337ad0 2024-05-15T14:06:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.784+0000 7f4a5e7fc640 1 -- 172.21.15.33:0/3282515383 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4a700aa120 con 0x558494337ad0 2024-05-15T14:06:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.784+0000 7f4a8057a740 10 client.16071 fetch_fsmap learned FSMap version 29 2024-05-15T14:06:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.784+0000 7f4a8057a740 1 -- 172.21.15.33:0/3282515383 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5584943b6610 con 0x558494337ad0 2024-05-15T14:06:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.784+0000 7f4a5e7fc640 1 -- 172.21.15.33:0/3282515383 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a6801fdc0 con 0x558494337ad0 2024-05-15T14:06:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.785+0000 7f4a5e7fc640 1 -- 172.21.15.33:0/3282515383 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f4a700e0ce0 con 0x558494337ad0 2024-05-15T14:06:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.785+0000 7f4a8057a740 10 client.16071 fetch_fsmap finished waiting for FSMap version 29 2024-05-15T14:06:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.785+0000 7f4a8057a740 10 client.16071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:06:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.785+0000 7f4a8057a740 20 client.16071 populate_metadata read hostname 'smithi033' 2024-05-15T14:06:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.785+0000 7f4a8057a740 1 --2- 172.21.15.33:0/3282515383 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5584943b6d80 0x5584943b91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.785+0000 7f4a8057a740 4 client.16071 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:06:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.785+0000 7f4a8057a740 1 -- 172.21.15.33:0/3282515383 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55849429b360 con 0x5584943b6d80 2024-05-15T14:06:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.785+0000 7f4a76ffd640 1 --2- 172.21.15.33:0/3282515383 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5584943b6d80 0x5584943b91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.785+0000 7f4a76ffd640 1 --2- 172.21.15.33:0/3282515383 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5584943b6d80 0x5584943b91c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90ff352801fb35d6 server_cookie=f645f9037eaa9443 in_seq=0 out_seq=0 2024-05-15T14:06:06.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.785+0000 7f4a5e7fc640 10 client.16071 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:06:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.832+0000 7f4a5e7fc640 1 -- 172.21.15.33:0/3282515383 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55849429b360 con 0x5584943b6d80 2024-05-15T14:06:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.832+0000 7f4a5e7fc640 10 client.16071 handle_command_reply: tid=0 2024-05-15T14:06:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.832+0000 7f4a8057a740 1 client.16071 shutdown 2024-05-15T14:06:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.833+0000 7f4a8057a740 1 -- 172.21.15.33:0/3282515383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558494337ad0 msgr2=0x558494342380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.833+0000 7f4a8057a740 1 --2- 172.21.15.33:0/3282515383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558494337ad0 0x558494342380 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f4a700aa0b0 tx=0x7f4a700a9890 comp rx=0 tx=0).stop 2024-05-15T14:06:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.833+0000 7f4a8057a740 1 -- 172.21.15.33:0/3282515383 shutdown_connections 2024-05-15T14:06:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.833+0000 7f4a8057a740 1 --2- 172.21.15.33:0/3282515383 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5584943b6d80 0x5584943b91c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.833+0000 7f4a8057a740 1 --2- 172.21.15.33:0/3282515383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558494337ad0 0x558494342380 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.833+0000 7f4a8057a740 1 --2- 172.21.15.33:0/3282515383 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55849432d210 0x558494341e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.833+0000 7f4a5e7fc640 0 client.16071 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:06:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.833+0000 7f4a8057a740 1 --2- 172.21.15.33:0/3282515383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55849432c8f0 0x558494341900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.833+0000 7f4a8057a740 1 -- 172.21.15.33:0/3282515383 >> 172.21.15.33:0/3282515383 conn(0x558494328e00 msgr2=0x558494346a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.833+0000 7f4a8057a740 1 -- 172.21.15.33:0/3282515383 shutdown_connections 2024-05-15T14:06:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.833+0000 7f4a8057a740 1 -- 172.21.15.33:0/3282515383 wait complete. 2024-05-15T14:06:06.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.833+0000 7f4a8057a740 20 client.16071 trim_cache size 0 max 16384 2024-05-15T14:06:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:06:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:06:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:06:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.833+0000 7f4a7d789640 1 -- 172.21.15.33:0/3756385286 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f4a640b2950 msgr2=0x7f4a640b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.833+0000 7f4a7d789640 1 --2- 172.21.15.33:0/3756385286 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f4a640b2950 0x7f4a640b4e10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55849432de90 tx=0x7f4a6800b570 comp rx=0 tx=0).stop 2024-05-15T14:06:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.833+0000 7f4a7d789640 1 -- 172.21.15.33:0/3756385286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a7810d090 msgr2=0x7f4a7819db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.833+0000 7f4a7d789640 1 --2- 172.21.15.33:0/3756385286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a7810d090 0x7f4a7819db30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000eb40 tx=0x7f4a7000c2d0 comp rx=0 tx=0).stop 2024-05-15T14:06:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.833+0000 7f4a7d789640 1 -- 172.21.15.33:0/3756385286 shutdown_connections 2024-05-15T14:06:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.833+0000 7f4a7d789640 1 --2- 172.21.15.33:0/3756385286 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f4a640b2950 0x7f4a640b4e10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.833+0000 7f4a7d789640 1 --2- 172.21.15.33:0/3756385286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a7810d090 0x7f4a7819db30 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.833+0000 7f4a7d789640 1 --2- 172.21.15.33:0/3756385286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a781045e0 0x7f4a781997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.834+0000 7f4a7d789640 1 --2- 172.21.15.33:0/3756385286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4a78103cc0 0x7f4a78199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.834+0000 7f4a7d789640 1 -- 172.21.15.33:0/3756385286 >> 172.21.15.33:0/3756385286 conn(0x7f4a78078a70 msgr2=0x7f4a780feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.834+0000 7f4a7d789640 1 -- 172.21.15.33:0/3756385286 shutdown_connections 2024-05-15T14:06:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:06.834+0000 7f4a7d789640 1 -- 172.21.15.33:0/3756385286 wait complete. 2024-05-15T14:06:06.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:06 smithi052 ceph-mon[36190]: pgmap v176: 97 pgs: 97 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 196 op/s 2024-05-15T14:06:07.059 INFO:tasks.workunit.client.0.smithi033.stdout: 1 342407 6.78 MB/sec execute 102 sec latency 4344.903 ms 2024-05-15T14:06:08.059 INFO:tasks.workunit.client.0.smithi033.stdout: 1 342407 6.72 MB/sec execute 103 sec latency 5344.985 ms 2024-05-15T14:06:08.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:08 smithi081 ceph-mon[36434]: pgmap v177: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 255 op/s 2024-05-15T14:06:08.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:08 smithi033 ceph-mon[31650]: pgmap v177: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 255 op/s 2024-05-15T14:06:08.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:08 smithi052 ceph-mon[36190]: pgmap v177: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 255 op/s 2024-05-15T14:06:09.060 INFO:tasks.workunit.client.0.smithi033.stdout: 1 342407 6.65 MB/sec execute 104 sec latency 6345.036 ms 2024-05-15T14:06:10.060 INFO:tasks.workunit.client.0.smithi033.stdout: 1 342407 6.59 MB/sec execute 105 sec latency 7345.137 ms 2024-05-15T14:06:10.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:10 smithi033 ceph-mon[31650]: pgmap v178: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 214 op/s 2024-05-15T14:06:10.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:10 smithi052 ceph-mon[36190]: pgmap v178: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 214 op/s 2024-05-15T14:06:10.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:10 smithi081 ceph-mon[36434]: pgmap v178: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 214 op/s 2024-05-15T14:06:11.060 INFO:tasks.workunit.client.0.smithi033.stdout: 1 342407 6.53 MB/sec execute 106 sec latency 8345.266 ms 2024-05-15T14:06:12.060 INFO:tasks.workunit.client.0.smithi033.stdout: 1 342407 6.47 MB/sec execute 107 sec latency 9345.358 ms 2024-05-15T14:06:12.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:12 smithi033 ceph-mon[31650]: pgmap v179: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 214 op/s 2024-05-15T14:06:12.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:12 smithi052 ceph-mon[36190]: pgmap v179: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 214 op/s 2024-05-15T14:06:12.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:12 smithi081 ceph-mon[36434]: pgmap v179: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 214 op/s 2024-05-15T14:06:13.060 INFO:tasks.workunit.client.0.smithi033.stdout: 1 342407 6.41 MB/sec execute 108 sec latency 10345.462 ms 2024-05-15T14:06:14.060 INFO:tasks.workunit.client.0.smithi033.stdout: 1 342407 6.35 MB/sec execute 109 sec latency 11345.569 ms 2024-05-15T14:06:14.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:14 smithi033 ceph-mon[31650]: pgmap v180: 97 pgs: 97 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 242 op/s 2024-05-15T14:06:14.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:14 smithi052 ceph-mon[36190]: pgmap v180: 97 pgs: 97 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 242 op/s 2024-05-15T14:06:14.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:14 smithi081 ceph-mon[36434]: pgmap v180: 97 pgs: 97 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 242 op/s 2024-05-15T14:06:15.060 INFO:tasks.workunit.client.0.smithi033.stdout: 1 342407 6.29 MB/sec execute 110 sec latency 12345.675 ms 2024-05-15T14:06:16.060 INFO:tasks.workunit.client.0.smithi033.stdout: 1 342407 6.23 MB/sec execute 111 sec latency 13345.785 ms 2024-05-15T14:06:16.122 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T14:06:16.391 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:06:16.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:16 smithi033 ceph-mon[31650]: pgmap v181: 97 pgs: 97 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 1.5 MiB/s wr, 85 op/s 2024-05-15T14:06:16.643 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:16 smithi052 ceph-mon[36190]: pgmap v181: 97 pgs: 97 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 1.5 MiB/s wr, 85 op/s 2024-05-15T14:06:16.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:16 smithi081 ceph-mon[36434]: pgmap v181: 97 pgs: 97 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 1.5 MiB/s wr, 85 op/s 2024-05-15T14:06:17.060 INFO:tasks.workunit.client.0.smithi033.stdout: 1 342407 6.18 MB/sec execute 112 sec latency 14345.873 ms 2024-05-15T14:06:17.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.973+0000 7f86067e5700 1 -- 172.21.15.33:0/2499352805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8600109630 msgr2=0x7f8600109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:17.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.973+0000 7f86067e5700 1 --2- 172.21.15.33:0/2499352805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8600109630 0x7f8600109a10 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f85f00099c0 tx=0x7f85f0009cd0 comp rx=0 tx=0).stop 2024-05-15T14:06:17.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.974+0000 7f86067e5700 1 -- 172.21.15.33:0/2499352805 shutdown_connections 2024-05-15T14:06:17.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.974+0000 7f86067e5700 1 --2- 172.21.15.33:0/2499352805 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f860006c6d0 0x7f8600110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:17.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.974+0000 7f86067e5700 1 --2- 172.21.15.33:0/2499352805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8600109630 0x7f8600109a10 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:17.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.974+0000 7f86067e5700 1 --2- 172.21.15.33:0/2499352805 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f860006bd10 0x7f860006c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:17.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.974+0000 7f86067e5700 1 -- 172.21.15.33:0/2499352805 >> 172.21.15.33:0/2499352805 conn(0x7f86000794e0 msgr2=0x7f86000798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:17.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.974+0000 7f86067e5700 1 -- 172.21.15.33:0/2499352805 shutdown_connections 2024-05-15T14:06:17.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.974+0000 7f86067e5700 1 -- 172.21.15.33:0/2499352805 wait complete. 2024-05-15T14:06:17.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.974+0000 7f86067e5700 1 Processor -- start 2024-05-15T14:06:17.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.974+0000 7f86067e5700 1 -- start start 2024-05-15T14:06:17.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.975+0000 7f86067e5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f860006bd10 0x7f860019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:17.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.975+0000 7f86067e5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f860006c6d0 0x7f860019fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:17.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.975+0000 7f86067e5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8600109630 0x7f86001999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:17.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.975+0000 7f86067e5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8600113740 con 0x7f860006bd10 2024-05-15T14:06:17.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.975+0000 7f86067e5700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f86001135c0 con 0x7f860006c6d0 2024-05-15T14:06:17.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.975+0000 7f86067e5700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f86001138c0 con 0x7f8600109630 2024-05-15T14:06:17.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.975+0000 7f85fffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f860006bd10 0x7f860019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:17.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.975+0000 7f85fffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f860006bd10 0x7f860019f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34564/0 (socket says 172.21.15.33:34564) 2024-05-15T14:06:17.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.975+0000 7f85fffff700 1 -- 172.21.15.33:0/1981291064 learned_addr learned my addr 172.21.15.33:0/1981291064 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:17.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.975+0000 7f85ff7fe700 1 --2- 172.21.15.33:0/1981291064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f860006c6d0 0x7f860019fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:17.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.975+0000 7f8604d82700 1 --2- 172.21.15.33:0/1981291064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8600109630 0x7f86001999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:17.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.975+0000 7f85fffff700 1 -- 172.21.15.33:0/1981291064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8600109630 msgr2=0x7f86001999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:17.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.975+0000 7f85fffff700 1 --2- 172.21.15.33:0/1981291064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8600109630 0x7f86001999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:17.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.975+0000 7f85fffff700 1 -- 172.21.15.33:0/1981291064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f860006c6d0 msgr2=0x7f860019fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:17.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.975+0000 7f85fffff700 1 --2- 172.21.15.33:0/1981291064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f860006c6d0 0x7f860019fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:17.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.975+0000 7f85fffff700 1 -- 172.21.15.33:0/1981291064 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f860019a100 con 0x7f860006bd10 2024-05-15T14:06:17.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.975+0000 7f85fffff700 1 --2- 172.21.15.33:0/1981291064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f860006bd10 0x7f860019f7e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f85f000b580 tx=0x7f85f00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:17.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.976+0000 7f85fd7fa700 1 -- 172.21.15.33:0/1981291064 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f0027070 con 0x7f860006bd10 2024-05-15T14:06:17.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.976+0000 7f86067e5700 1 -- 172.21.15.33:0/1981291064 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f860019a390 con 0x7f860006bd10 2024-05-15T14:06:17.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.976+0000 7f85fd7fa700 1 -- 172.21.15.33:0/1981291064 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85f0003f20 con 0x7f860006bd10 2024-05-15T14:06:17.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.976+0000 7f85fd7fa700 1 -- 172.21.15.33:0/1981291064 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f0022d70 con 0x7f860006bd10 2024-05-15T14:06:17.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.976+0000 7f86067e5700 1 -- 172.21.15.33:0/1981291064 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86001a66b0 con 0x7f860006bd10 2024-05-15T14:06:17.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.977+0000 7f86067e5700 1 -- 172.21.15.33:0/1981291064 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f860010bac0 con 0x7f860006bd10 2024-05-15T14:06:17.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.978+0000 7f85fd7fa700 1 -- 172.21.15.33:0/1981291064 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f85f0022500 con 0x7f860006bd10 2024-05-15T14:06:17.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.979+0000 7f85fd7fa700 1 --2- 172.21.15.33:0/1981291064 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f85e80b29f0 0x7f85e80b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:17.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.979+0000 7f85ff7fe700 1 --2- 172.21.15.33:0/1981291064 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f85e80b29f0 0x7f85e80b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:17.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.979+0000 7f85fd7fa700 1 -- 172.21.15.33:0/1981291064 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f85f00e32b0 con 0x7f860006bd10 2024-05-15T14:06:17.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.980+0000 7f85ff7fe700 1 --2- 172.21.15.33:0/1981291064 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f85e80b29f0 0x7f85e80b4eb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8600079a70 tx=0x7f85f400b410 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:17.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:17.984+0000 7f85fd7fa700 1 -- 172.21.15.33:0/1981291064 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f85f0013070 con 0x7f860006bd10 2024-05-15T14:06:18.061 INFO:tasks.workunit.client.0.smithi033.stdout: 1 342407 6.12 MB/sec execute 113 sec latency 15345.935 ms 2024-05-15T14:06:18.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:18.242+0000 7f86067e5700 1 -- 172.21.15.33:0/1981291064 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8600104ca0 con 0x7f860006bd10 2024-05-15T14:06:18.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:18.243+0000 7f85fd7fa700 1 -- 172.21.15.33:0/1981291064 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11855 (secure 0 0 0) 0x7f85f00aad30 con 0x7f860006bd10 2024-05-15T14:06:18.244 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T14:06:18.244 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2030117633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2030117633},{"type":"v1","addr":"172.21.15.33:6839","nonce":2030117633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1918979061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1918979061},{"type":"v1","addr":"172.21.15.33:6841","nonce":1918979061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T13:57:19.056417+0000","modified":"2024-05-15T14:05:13.682039+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24463,"mds_2":24461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/1629697841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1629697841},{"type":"v1","addr":"172.21.15.52:6837","nonce":1629697841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.52:6839/2696044471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2696044471},{"type":"v1","addr":"172.21.15.52:6839","nonce":2696044471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.81:6839/1554395940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":1554395940},{"type":"v1","addr":"172.21.15.81:6839","nonce":1554395940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25393":{"gid":25393,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/727032511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":727032511},{"type":"v1","addr":"172.21.15.81:6835","nonce":727032511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24478,24461,24463]},"id":1}]} 2024-05-15T14:06:18.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:18.245+0000 7f86067e5700 1 -- 172.21.15.33:0/1981291064 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f85e80b29f0 msgr2=0x7f85e80b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:18.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:18.245+0000 7f86067e5700 1 --2- 172.21.15.33:0/1981291064 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f85e80b29f0 0x7f85e80b4eb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8600079a70 tx=0x7f85f400b410 comp rx=0 tx=0).stop 2024-05-15T14:06:18.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:18.245+0000 7f86067e5700 1 -- 172.21.15.33:0/1981291064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f860006bd10 msgr2=0x7f860019f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:18.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:18.245+0000 7f86067e5700 1 --2- 172.21.15.33:0/1981291064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f860006bd10 0x7f860019f7e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f85f000b580 tx=0x7f85f00047b0 comp rx=0 tx=0).stop 2024-05-15T14:06:18.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:18.245+0000 7f86067e5700 1 -- 172.21.15.33:0/1981291064 shutdown_connections 2024-05-15T14:06:18.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:18.245+0000 7f86067e5700 1 --2- 172.21.15.33:0/1981291064 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f85e80b29f0 0x7f85e80b4eb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:18.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:18.245+0000 7f86067e5700 1 --2- 172.21.15.33:0/1981291064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8600109630 0x7f86001999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:18.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:18.245+0000 7f86067e5700 1 --2- 172.21.15.33:0/1981291064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f860006bd10 0x7f860019f7e0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:18.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:18.245+0000 7f86067e5700 1 --2- 172.21.15.33:0/1981291064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f860006c6d0 0x7f860019fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:18.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:18.245+0000 7f86067e5700 1 -- 172.21.15.33:0/1981291064 >> 172.21.15.33:0/1981291064 conn(0x7f86000794e0 msgr2=0x7f8600100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:18.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:18.245+0000 7f86067e5700 1 -- 172.21.15.33:0/1981291064 shutdown_connections 2024-05-15T14:06:18.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:18.245+0000 7f86067e5700 1 -- 172.21.15.33:0/1981291064 wait complete. 2024-05-15T14:06:18.248 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 29 2024-05-15T14:06:18.546 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:18 smithi033 ceph-mon[31650]: pgmap v182: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.6 MiB/s wr, 101 op/s 2024-05-15T14:06:18.546 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:18 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/1981291064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:06:18.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:18 smithi052 ceph-mon[36190]: pgmap v182: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.6 MiB/s wr, 101 op/s 2024-05-15T14:06:18.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:18 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/1981291064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:06:18.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:18 smithi081 ceph-mon[36434]: pgmap v182: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.6 MiB/s wr, 101 op/s 2024-05-15T14:06:18.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:18 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/1981291064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:06:19.026 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24478 quiesce db --set-id b5682d61 --release --await' 2024-05-15T14:06:19.027 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24478 quiesce db --set-id b5682d61 --release --await 2024-05-15T14:06:19.060 INFO:tasks.workunit.client.0.smithi033.stdout: 1 342407 6.07 MB/sec execute 114 sec latency 16345.971 ms 2024-05-15T14:06:19.281 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:06:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.929+0000 7f0b27a08640 1 -- 172.21.15.33:0/2277432377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b20107ab0 msgr2=0x7f0b20107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.929+0000 7f0b27a08640 1 --2- 172.21.15.33:0/2277432377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b20107ab0 0x7f0b20107e90 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0b14009930 tx=0x7f0b1402a420 comp rx=0 tx=0).stop 2024-05-15T14:06:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.929+0000 7f0b27a08640 1 -- 172.21.15.33:0/2277432377 shutdown_connections 2024-05-15T14:06:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.929+0000 7f0b27a08640 1 --2- 172.21.15.33:0/2277432377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b20101cd0 0x7f0b2010f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.929+0000 7f0b27a08640 1 --2- 172.21.15.33:0/2277432377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b20101310 0x7f0b20101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.929+0000 7f0b27a08640 1 --2- 172.21.15.33:0/2277432377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b20107ab0 0x7f0b20107e90 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.929+0000 7f0b27a08640 1 -- 172.21.15.33:0/2277432377 >> 172.21.15.33:0/2277432377 conn(0x7f0b20076c90 msgr2=0x7f0b200770a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.929+0000 7f0b27a08640 1 -- 172.21.15.33:0/2277432377 shutdown_connections 2024-05-15T14:06:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.929+0000 7f0b27a08640 1 -- 172.21.15.33:0/2277432377 wait complete. 2024-05-15T14:06:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.929+0000 7f0b27a08640 1 Processor -- start 2024-05-15T14:06:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.930+0000 7f0b27a08640 1 -- start start 2024-05-15T14:06:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.930+0000 7f0b27a08640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b20101310 0x7f0b20199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.930+0000 7f0b27a08640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b20101cd0 0x7f0b20199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.930+0000 7f0b27a08640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b20107ab0 0x7f0b2019daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.930+0000 7f0b27a08640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0b20112d90 con 0x7f0b20107ab0 2024-05-15T14:06:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.930+0000 7f0b27a08640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0b20112c10 con 0x7f0b20101cd0 2024-05-15T14:06:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.930+0000 7f0b27a08640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0b20112f10 con 0x7f0b20101310 2024-05-15T14:06:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.930+0000 7f0b25fa5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b20107ab0 0x7f0b2019daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.930+0000 7f0b25fa5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b20107ab0 0x7f0b2019daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34584/0 (socket says 172.21.15.33:34584) 2024-05-15T14:06:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.930+0000 7f0b25fa5640 1 -- 172.21.15.33:0/1575543075 learned_addr learned my addr 172.21.15.33:0/1575543075 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.930+0000 7f0b24fa3640 1 --2- 172.21.15.33:0/1575543075 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b20101cd0 0x7f0b20199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.930+0000 7f0b257a4640 1 --2- 172.21.15.33:0/1575543075 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b20101310 0x7f0b20199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.930+0000 7f0b25fa5640 1 -- 172.21.15.33:0/1575543075 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b20101310 msgr2=0x7f0b20199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.930+0000 7f0b25fa5640 1 --2- 172.21.15.33:0/1575543075 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b20101310 0x7f0b20199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.930+0000 7f0b25fa5640 1 -- 172.21.15.33:0/1575543075 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b20101cd0 msgr2=0x7f0b20199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.930+0000 7f0b25fa5640 1 --2- 172.21.15.33:0/1575543075 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b20101cd0 0x7f0b20199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.930+0000 7f0b25fa5640 1 -- 172.21.15.33:0/1575543075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b2019e1d0 con 0x7f0b20107ab0 2024-05-15T14:06:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.930+0000 7f0b257a4640 1 --2- 172.21.15.33:0/1575543075 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b20101310 0x7f0b20199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:06:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.930+0000 7f0b25fa5640 1 --2- 172.21.15.33:0/1575543075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b20107ab0 0x7f0b2019daa0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800cad0 tx=0x7f0b1800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.931+0000 7f0b127fc640 1 -- 172.21.15.33:0/1575543075 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1800ad60 con 0x7f0b20107ab0 2024-05-15T14:06:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.931+0000 7f0b27a08640 1 -- 172.21.15.33:0/1575543075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b2019e4c0 con 0x7f0b20107ab0 2024-05-15T14:06:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.931+0000 7f0b127fc640 1 -- 172.21.15.33:0/1575543075 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b18005020 con 0x7f0b20107ab0 2024-05-15T14:06:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.931+0000 7f0b127fc640 1 -- 172.21.15.33:0/1575543075 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1800f6b0 con 0x7f0b20107ab0 2024-05-15T14:06:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.931+0000 7f0b27a08640 1 -- 172.21.15.33:0/1575543075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b201a5d50 con 0x7f0b20107ab0 2024-05-15T14:06:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.933+0000 7f0b127fc640 1 -- 172.21.15.33:0/1575543075 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0b180051d0 con 0x7f0b20107ab0 2024-05-15T14:06:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.933+0000 7f0b2a7fd740 1 Processor -- start 2024-05-15T14:06:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.933+0000 7f0b2a7fd740 1 -- start start 2024-05-15T14:06:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.934+0000 7f0b2a7fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639631a38f0 0x5639631a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.934+0000 7f0b2a7fd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639631a4210 0x5639631ae590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.934+0000 7f0b2a7fd740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639631aead0 0x5639631b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.934+0000 7f0b2a7fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563963116c30 con 0x5639631a38f0 2024-05-15T14:06:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.934+0000 7f0b2a7fd740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5639631067d0 con 0x5639631a4210 2024-05-15T14:06:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.934+0000 7f0b2a7fd740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56396310a810 con 0x5639631aead0 2024-05-15T14:06:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.934+0000 7f0b24fa3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639631a38f0 0x5639631a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.934+0000 7f0b24fa3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639631a38f0 0x5639631a3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34586/0 (socket says 172.21.15.33:34586) 2024-05-15T14:06:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.934+0000 7f0b24fa3640 1 -- 172.21.15.33:0/450605326 learned_addr learned my addr 172.21.15.33:0/450605326 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.934+0000 7f0b257a4640 1 --2- 172.21.15.33:0/450605326 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639631a4210 0x5639631ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.934+0000 7f0b24fa3640 1 --2- 172.21.15.33:0/450605326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639631aead0 0x5639631b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.934+0000 7f0b24fa3640 1 -- 172.21.15.33:0/450605326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639631aead0 msgr2=0x5639631b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.934+0000 7f0b24fa3640 1 --2- 172.21.15.33:0/450605326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639631aead0 0x5639631b0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.934+0000 7f0b24fa3640 1 -- 172.21.15.33:0/450605326 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639631a4210 msgr2=0x5639631ae590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.934+0000 7f0b24fa3640 1 --2- 172.21.15.33:0/450605326 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639631a4210 0x5639631ae590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.934+0000 7f0b24fa3640 1 -- 172.21.15.33:0/450605326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563962e1b210 con 0x5639631a38f0 2024-05-15T14:06:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.934+0000 7f0b257a4640 1 --2- 172.21.15.33:0/450605326 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639631a4210 0x5639631ae590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:06:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.934+0000 7f0b127fc640 1 --2- 172.21.15.33:0/1575543075 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0b0c0b2950 0x7f0b0c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.934+0000 7f0b24fa3640 1 --2- 172.21.15.33:0/450605326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639631a38f0 0x5639631a3cd0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c00aee0 tx=0x7f0b1c00a800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42e4a0a45489324 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.934+0000 7f0b257a4640 1 --2- 172.21.15.33:0/1575543075 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0b0c0b2950 0x7f0b0c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.935+0000 7f0b0b7fe640 1 -- 172.21.15.33:0/450605326 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c00f650 con 0x5639631a38f0 2024-05-15T14:06:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.935+0000 7f0b127fc640 1 -- 172.21.15.33:0/1575543075 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f0b180e25e0 con 0x7f0b20107ab0 2024-05-15T14:06:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.935+0000 7f0b0b7fe640 1 -- 172.21.15.33:0/450605326 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b1c017030 con 0x5639631a38f0 2024-05-15T14:06:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.935+0000 7f0b0b7fe640 1 -- 172.21.15.33:0/450605326 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c01b3d0 con 0x5639631a38f0 2024-05-15T14:06:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.935+0000 7f0b2a7fd740 1 -- 172.21.15.33:0/450605326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639631a38f0 msgr2=0x5639631a3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.935+0000 7f0b257a4640 1 --2- 172.21.15.33:0/1575543075 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0b0c0b2950 0x7f0b0c0b4e10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0b14000c00 tx=0x7f0b140058b0 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.935+0000 7f0b2a7fd740 1 --2- 172.21.15.33:0/450605326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639631a38f0 0x5639631a3cd0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c00aee0 tx=0x7f0b1c00a800 comp rx=0 tx=0).stop 2024-05-15T14:06:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.935+0000 7f0b2a7fd740 1 -- 172.21.15.33:0/450605326 shutdown_connections 2024-05-15T14:06:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.935+0000 7f0b2a7fd740 1 --2- 172.21.15.33:0/450605326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639631aead0 0x5639631b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.935+0000 7f0b2a7fd740 1 --2- 172.21.15.33:0/450605326 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639631a4210 0x5639631ae590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.935+0000 7f0b2a7fd740 1 --2- 172.21.15.33:0/450605326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639631a38f0 0x5639631a3cd0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.935+0000 7f0b2a7fd740 1 -- 172.21.15.33:0/450605326 >> 172.21.15.33:0/450605326 conn(0x56396319fe00 msgr2=0x5639631a01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.936+0000 7f0b2a7fd740 1 -- 172.21.15.33:0/450605326 shutdown_connections 2024-05-15T14:06:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.936+0000 7f0b2a7fd740 1 -- 172.21.15.33:0/450605326 wait complete. 2024-05-15T14:06:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.936+0000 7f0b2a7fd740 1 -- start start 2024-05-15T14:06:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.936+0000 7f0b2a7fd740 1 Processor -- start 2024-05-15T14:06:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.936+0000 7f0b2a7fd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639631a38f0 0x5639631ae470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.936+0000 7f0b2a7fd740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639631a4210 0x5639631ab760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.936+0000 7f0b2a7fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639631aead0 0x5639631abd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.936+0000 7f0b2a7fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563962dec790 con 0x5639631aead0 2024-05-15T14:06:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.936+0000 7f0b2a7fd740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5639630c2890 con 0x5639631a38f0 2024-05-15T14:06:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.936+0000 7f0b2a7fd740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5639631b17a0 con 0x5639631a4210 2024-05-15T14:06:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.936+0000 7f0b2a7fd740 10 client.0 _ll_register_callbacks cb 0x5639630c5d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:06:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.936+0000 7f0b25fa5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639631aead0 0x5639631abd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.936+0000 7f0b25fa5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639631aead0 0x5639631abd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34602/0 (socket says 172.21.15.33:34602) 2024-05-15T14:06:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.937+0000 7f0b25fa5640 1 -- 172.21.15.33:0/51147358 learned_addr learned my addr 172.21.15.33:0/51147358 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.937+0000 7f0b24fa3640 1 --2- 172.21.15.33:0/51147358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639631a4210 0x5639631ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.937+0000 7f0b257a4640 1 --2- 172.21.15.33:0/51147358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639631a38f0 0x5639631ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.937+0000 7f0b25fa5640 1 -- 172.21.15.33:0/51147358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639631a4210 msgr2=0x5639631ab760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.937+0000 7f0b25fa5640 1 --2- 172.21.15.33:0/51147358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639631a4210 0x5639631ab760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.937+0000 7f0b25fa5640 1 -- 172.21.15.33:0/51147358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639631a38f0 msgr2=0x5639631ae470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.937+0000 7f0b25fa5640 1 --2- 172.21.15.33:0/51147358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639631a38f0 0x5639631ae470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.937+0000 7f0b25fa5640 1 -- 172.21.15.33:0/51147358 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563962ee7620 con 0x5639631aead0 2024-05-15T14:06:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.937+0000 7f0b257a4640 1 --2- 172.21.15.33:0/51147358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639631a38f0 0x5639631ae470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:06:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.937+0000 7f0b25fa5640 1 --2- 172.21.15.33:0/51147358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639631aead0 0x5639631abd00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0b180a9860 tx=0x7f0b180a9520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8386c3affb19f355 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.937+0000 7f0b08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:06:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.937+0000 7f0b08ff9640 1 -- 172.21.15.33:0/51147358 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1801d070 con 0x5639631aead0 2024-05-15T14:06:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.937+0000 7f0b08ff9640 1 -- 172.21.15.33:0/51147358 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f0b180a5b60 con 0x5639631aead0 2024-05-15T14:06:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.937+0000 7f0b2a7fd740 1 -- 172.21.15.33:0/51147358 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b1c017030 con 0x5639631aead0 2024-05-15T14:06:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.937+0000 7f0b08ff9640 1 -- 172.21.15.33:0/51147358 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0b180b2730 con 0x5639631aead0 2024-05-15T14:06:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.937+0000 7f0b2a7fd740 10 client.16089 fetch_fsmap learned FSMap version 29 2024-05-15T14:06:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.937+0000 7f0b2a7fd740 1 -- 172.21.15.33:0/51147358 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5639631bd250 con 0x5639631aead0 2024-05-15T14:06:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.937+0000 7f0b08ff9640 1 -- 172.21.15.33:0/51147358 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b1c017030 con 0x5639631aead0 2024-05-15T14:06:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.938+0000 7f0b08ff9640 1 -- 172.21.15.33:0/51147358 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f0b1806a020 con 0x5639631aead0 2024-05-15T14:06:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.938+0000 7f0b2a7fd740 10 client.16089 fetch_fsmap finished waiting for FSMap version 29 2024-05-15T14:06:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.938+0000 7f0b2a7fd740 10 client.16089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:06:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.938+0000 7f0b2a7fd740 20 client.16089 populate_metadata read hostname 'smithi033' 2024-05-15T14:06:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.938+0000 7f0b2a7fd740 1 --2- 172.21.15.33:0/51147358 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5639631be710 0x5639631c0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.938+0000 7f0b2a7fd740 4 client.16089 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:06:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.938+0000 7f0b2a7fd740 1 -- 172.21.15.33:0/51147358 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563963118b80 con 0x5639631be710 2024-05-15T14:06:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.938+0000 7f0b257a4640 1 --2- 172.21.15.33:0/51147358 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5639631be710 0x5639631c0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.938+0000 7f0b257a4640 1 --2- 172.21.15.33:0/51147358 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5639631be710 0x5639631c0b00 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eef82d781307a2d server_cookie=642703ffea549554 in_seq=0 out_seq=0 2024-05-15T14:06:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.938+0000 7f0b08ff9640 10 client.16089 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:06:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.939+0000 7f0b08ff9640 1 -- 172.21.15.33:0/51147358 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563963118b80 con 0x5639631be710 2024-05-15T14:06:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.939+0000 7f0b08ff9640 10 client.16089 handle_command_reply: tid=0 2024-05-15T14:06:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.939+0000 7f0b2a7fd740 1 client.16089 shutdown 2024-05-15T14:06:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.939+0000 7f0b2a7fd740 1 -- 172.21.15.33:0/51147358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639631aead0 msgr2=0x5639631abd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.939+0000 7f0b2a7fd740 1 --2- 172.21.15.33:0/51147358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639631aead0 0x5639631abd00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0b180a9860 tx=0x7f0b180a9520 comp rx=0 tx=0).stop 2024-05-15T14:06:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.939+0000 7f0b2a7fd740 1 -- 172.21.15.33:0/51147358 shutdown_connections 2024-05-15T14:06:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.939+0000 7f0b2a7fd740 1 --2- 172.21.15.33:0/51147358 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5639631be710 0x5639631c0b00 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.939+0000 7f0b2a7fd740 1 --2- 172.21.15.33:0/51147358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639631aead0 0x5639631abd00 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.939+0000 7f0b2a7fd740 1 --2- 172.21.15.33:0/51147358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639631a4210 0x5639631ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.939+0000 7f0b2a7fd740 1 --2- 172.21.15.33:0/51147358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639631a38f0 0x5639631ae470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.939+0000 7f0b08ff9640 0 client.16089 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:06:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.939+0000 7f0b2a7fd740 1 -- 172.21.15.33:0/51147358 >> 172.21.15.33:0/51147358 conn(0x56396319fe00 msgr2=0x56396319f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.939+0000 7f0b2a7fd740 1 -- 172.21.15.33:0/51147358 shutdown_connections 2024-05-15T14:06:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.939+0000 7f0b2a7fd740 1 -- 172.21.15.33:0/51147358 wait complete. 2024-05-15T14:06:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.940+0000 7f0b2a7fd740 20 client.16089 trim_cache size 0 max 16384 2024-05-15T14:06:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.955+0000 7f0b2a7fd740 1 Processor -- start 2024-05-15T14:06:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.955+0000 7f0b2a7fd740 1 -- start start 2024-05-15T14:06:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.955+0000 7f0b2a7fd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639631a38f0 0x5639631ba860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.955+0000 7f0b2a7fd740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639631a4210 0x56396319ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.955+0000 7f0b2a7fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639631aead0 0x56396319f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.955+0000 7f0b2a7fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201b0f30 con 0x5639631aead0 2024-05-15T14:06:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.955+0000 7f0b2a7fd740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201ae800 con 0x5639631a38f0 2024-05-15T14:06:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.955+0000 7f0b2a7fd740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201ac650 con 0x5639631a4210 2024-05-15T14:06:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.955+0000 7f0b25fa5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639631aead0 0x56396319f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.955+0000 7f0b25fa5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639631aead0 0x56396319f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34612/0 (socket says 172.21.15.33:34612) 2024-05-15T14:06:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.955+0000 7f0b25fa5640 1 -- 172.21.15.33:0/2358249553 learned_addr learned my addr 172.21.15.33:0/2358249553 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.955+0000 7f0b257a4640 1 --2- 172.21.15.33:0/2358249553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639631a38f0 0x5639631ba860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.955+0000 7f0b24fa3640 1 --2- 172.21.15.33:0/2358249553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639631a4210 0x56396319ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.955+0000 7f0b25fa5640 1 -- 172.21.15.33:0/2358249553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639631a4210 msgr2=0x56396319ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.955+0000 7f0b25fa5640 1 --2- 172.21.15.33:0/2358249553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639631a4210 0x56396319ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.955+0000 7f0b25fa5640 1 -- 172.21.15.33:0/2358249553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639631a38f0 msgr2=0x5639631ba860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.955+0000 7f0b25fa5640 1 --2- 172.21.15.33:0/2358249553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639631a38f0 0x5639631ba860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.955+0000 7f0b25fa5640 1 -- 172.21.15.33:0/2358249553 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639631a4040 con 0x5639631aead0 2024-05-15T14:06:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.955+0000 7f0b24fa3640 1 --2- 172.21.15.33:0/2358249553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639631a4210 0x56396319ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:06:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.955+0000 7f0b257a4640 1 --2- 172.21.15.33:0/2358249553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639631a38f0 0x5639631ba860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:06:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.955+0000 7f0b25fa5640 1 --2- 172.21.15.33:0/2358249553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639631aead0 0x56396319f4c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0b180a95d0 tx=0x7f0b1806f1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3272e0735e10b2ef server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.956+0000 7f0b0affd640 1 -- 172.21.15.33:0/2358249553 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1801d070 con 0x5639631aead0 2024-05-15T14:06:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.956+0000 7f0b0affd640 1 -- 172.21.15.33:0/2358249553 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b180a5b60 con 0x5639631aead0 2024-05-15T14:06:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.956+0000 7f0b0affd640 1 -- 172.21.15.33:0/2358249553 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b180a9040 con 0x5639631aead0 2024-05-15T14:06:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.956+0000 7f0b2a7fd740 1 -- 172.21.15.33:0/2358249553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639631aead0 msgr2=0x56396319f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.956+0000 7f0b2a7fd740 1 --2- 172.21.15.33:0/2358249553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639631aead0 0x56396319f4c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0b180a95d0 tx=0x7f0b1806f1f0 comp rx=0 tx=0).stop 2024-05-15T14:06:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.956+0000 7f0b2a7fd740 1 -- 172.21.15.33:0/2358249553 shutdown_connections 2024-05-15T14:06:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.956+0000 7f0b2a7fd740 1 --2- 172.21.15.33:0/2358249553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639631aead0 0x56396319f4c0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.956+0000 7f0b2a7fd740 1 --2- 172.21.15.33:0/2358249553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639631a4210 0x56396319ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.956+0000 7f0b2a7fd740 1 --2- 172.21.15.33:0/2358249553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639631a38f0 0x5639631ba860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.956+0000 7f0b2a7fd740 1 -- 172.21.15.33:0/2358249553 >> 172.21.15.33:0/2358249553 conn(0x56396319fe00 msgr2=0x56396319e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.956+0000 7f0b2a7fd740 1 -- 172.21.15.33:0/2358249553 shutdown_connections 2024-05-15T14:06:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.956+0000 7f0b2a7fd740 1 -- 172.21.15.33:0/2358249553 wait complete. 2024-05-15T14:06:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.957+0000 7f0b2a7fd740 1 -- start start 2024-05-15T14:06:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.957+0000 7f0b2a7fd740 1 Processor -- start 2024-05-15T14:06:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.957+0000 7f0b2a7fd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639631a38f0 0x5639631b4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.957+0000 7f0b2a7fd740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639631a4210 0x5639631b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.957+0000 7f0b2a7fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639631aead0 0x5639631b5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.957+0000 7f0b2a7fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201accc0 con 0x5639631aead0 2024-05-15T14:06:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.957+0000 7f0b2a7fd740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201ae130 con 0x5639631a38f0 2024-05-15T14:06:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.957+0000 7f0b2a7fd740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201ad390 con 0x5639631a4210 2024-05-15T14:06:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.957+0000 7f0b2a7fd740 10 client.0 _ll_register_callbacks cb 0x7f0b180a5060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:06:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.957+0000 7f0b25fa5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639631aead0 0x5639631b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.957+0000 7f0b25fa5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639631aead0 0x5639631b5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34626/0 (socket says 172.21.15.33:34626) 2024-05-15T14:06:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.957+0000 7f0b25fa5640 1 -- 172.21.15.33:0/94579746 learned_addr learned my addr 172.21.15.33:0/94579746 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.957+0000 7f0b257a4640 1 --2- 172.21.15.33:0/94579746 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639631a38f0 0x5639631b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.957+0000 7f0b24fa3640 1 --2- 172.21.15.33:0/94579746 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639631a4210 0x5639631b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.957+0000 7f0b25fa5640 1 -- 172.21.15.33:0/94579746 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639631a4210 msgr2=0x5639631b4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.957+0000 7f0b25fa5640 1 --2- 172.21.15.33:0/94579746 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639631a4210 0x5639631b4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.957+0000 7f0b25fa5640 1 -- 172.21.15.33:0/94579746 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639631a38f0 msgr2=0x5639631b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.957+0000 7f0b25fa5640 1 --2- 172.21.15.33:0/94579746 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639631a38f0 0x5639631b4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.957+0000 7f0b25fa5640 1 -- 172.21.15.33:0/94579746 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563963228090 con 0x5639631aead0 2024-05-15T14:06:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.958+0000 7f0b25fa5640 1 --2- 172.21.15.33:0/94579746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639631aead0 0x5639631b5180 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0b180a5950 tx=0x7f0b180a9540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e554c05428ee51a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.958+0000 7f0b08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:06:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.958+0000 7f0b08ff9640 1 -- 172.21.15.33:0/94579746 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1801d050 con 0x5639631aead0 2024-05-15T14:06:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.958+0000 7f0b08ff9640 1 -- 172.21.15.33:0/94579746 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f0b180e22e0 con 0x5639631aead0 2024-05-15T14:06:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.958+0000 7f0b2a7fd740 1 -- 172.21.15.33:0/94579746 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b180a5b60 con 0x5639631aead0 2024-05-15T14:06:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.958+0000 7f0b08ff9640 1 -- 172.21.15.33:0/94579746 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0b180a9040 con 0x5639631aead0 2024-05-15T14:06:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.958+0000 7f0b2a7fd740 10 client.16095 fetch_fsmap learned FSMap version 29 2024-05-15T14:06:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.958+0000 7f0b2a7fd740 1 -- 172.21.15.33:0/94579746 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5639632292f0 con 0x5639631aead0 2024-05-15T14:06:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.958+0000 7f0b08ff9640 1 -- 172.21.15.33:0/94579746 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b180a5b60 con 0x5639631aead0 2024-05-15T14:06:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.958+0000 7f0b08ff9640 1 -- 172.21.15.33:0/94579746 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f0b1806a070 con 0x5639631aead0 2024-05-15T14:06:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.958+0000 7f0b2a7fd740 10 client.16095 fetch_fsmap finished waiting for FSMap version 29 2024-05-15T14:06:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.958+0000 7f0b2a7fd740 10 client.16095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:06:19.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.958+0000 7f0b2a7fd740 20 client.16095 populate_metadata read hostname 'smithi033' 2024-05-15T14:06:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.958+0000 7f0b2a7fd740 1 --2- 172.21.15.33:0/94579746 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x563963229720 0x56396322bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.958+0000 7f0b2a7fd740 4 client.16095 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:06:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.958+0000 7f0b2a7fd740 1 -- 172.21.15.33:0/94579746 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563963112360 con 0x563963229720 2024-05-15T14:06:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.959+0000 7f0b257a4640 1 --2- 172.21.15.33:0/94579746 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x563963229720 0x56396322bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.959+0000 7f0b08ff9640 10 client.16095 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:06:19.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.959+0000 7f0b257a4640 1 --2- 172.21.15.33:0/94579746 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x563963229720 0x56396322bb10 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8603572c72cbaae server_cookie=c4deb7b3e0654ed0 in_seq=0 out_seq=0 2024-05-15T14:06:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.984+0000 7f0b08ff9640 1 -- 172.21.15.33:0/94579746 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563963112360 con 0x563963229720 2024-05-15T14:06:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.984+0000 7f0b08ff9640 10 client.16095 handle_command_reply: tid=0 2024-05-15T14:06:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.984+0000 7f0b2a7fd740 1 client.16095 shutdown 2024-05-15T14:06:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.984+0000 7f0b2a7fd740 1 -- 172.21.15.33:0/94579746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639631aead0 msgr2=0x5639631b5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.984+0000 7f0b2a7fd740 1 --2- 172.21.15.33:0/94579746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639631aead0 0x5639631b5180 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0b180a5950 tx=0x7f0b180a9540 comp rx=0 tx=0).stop 2024-05-15T14:06:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.984+0000 7f0b2a7fd740 1 -- 172.21.15.33:0/94579746 shutdown_connections 2024-05-15T14:06:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.984+0000 7f0b2a7fd740 1 --2- 172.21.15.33:0/94579746 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x563963229720 0x56396322bb10 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:19.986 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.984+0000 7f0b2a7fd740 1 --2- 172.21.15.33:0/94579746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639631aead0 0x5639631b5180 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.984+0000 7f0b2a7fd740 1 --2- 172.21.15.33:0/94579746 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5639631a4210 0x5639631b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.984+0000 7f0b08ff9640 0 client.16095 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:06:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.984+0000 7f0b2a7fd740 1 --2- 172.21.15.33:0/94579746 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639631a38f0 0x5639631b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.984+0000 7f0b2a7fd740 1 -- 172.21.15.33:0/94579746 >> 172.21.15.33:0/94579746 conn(0x56396319fe00 msgr2=0x5639631b9660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.984+0000 7f0b2a7fd740 1 -- 172.21.15.33:0/94579746 shutdown_connections 2024-05-15T14:06:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.984+0000 7f0b2a7fd740 1 -- 172.21.15.33:0/94579746 wait complete. 2024-05-15T14:06:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.984+0000 7f0b2a7fd740 20 client.16095 trim_cache size 0 max 16384 2024-05-15T14:06:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:06:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:06:19.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:06:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.985+0000 7f0b27a08640 1 -- 172.21.15.33:0/1575543075 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0b0c0b2950 msgr2=0x7f0b0c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.985+0000 7f0b27a08640 1 --2- 172.21.15.33:0/1575543075 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0b0c0b2950 0x7f0b0c0b4e10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0b14000c00 tx=0x7f0b140058b0 comp rx=0 tx=0).stop 2024-05-15T14:06:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.985+0000 7f0b27a08640 1 -- 172.21.15.33:0/1575543075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b20107ab0 msgr2=0x7f0b2019daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.985+0000 7f0b27a08640 1 --2- 172.21.15.33:0/1575543075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b20107ab0 0x7f0b2019daa0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800cad0 tx=0x7f0b1800ced0 comp rx=0 tx=0).stop 2024-05-15T14:06:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.985+0000 7f0b27a08640 1 -- 172.21.15.33:0/1575543075 shutdown_connections 2024-05-15T14:06:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.985+0000 7f0b27a08640 1 --2- 172.21.15.33:0/1575543075 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0b0c0b2950 0x7f0b0c0b4e10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.985+0000 7f0b27a08640 1 --2- 172.21.15.33:0/1575543075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b20107ab0 0x7f0b2019daa0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.985+0000 7f0b27a08640 1 --2- 172.21.15.33:0/1575543075 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b20101cd0 0x7f0b20199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.985+0000 7f0b27a08640 1 --2- 172.21.15.33:0/1575543075 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b20101310 0x7f0b20199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.985+0000 7f0b27a08640 1 -- 172.21.15.33:0/1575543075 >> 172.21.15.33:0/1575543075 conn(0x7f0b20076c90 msgr2=0x7f0b200ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.985+0000 7f0b27a08640 1 -- 172.21.15.33:0/1575543075 shutdown_connections 2024-05-15T14:06:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:19.985+0000 7f0b27a08640 1 -- 172.21.15.33:0/1575543075 wait complete. 2024-05-15T14:06:20.061 INFO:tasks.workunit.client.0.smithi033.stdout: 1 342407 6.02 MB/sec execute 115 sec latency 17346.036 ms 2024-05-15T14:06:20.628 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:20 smithi033 ceph-mon[31650]: pgmap v183: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 812 KiB/s wr, 42 op/s 2024-05-15T14:06:20.658 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:20 smithi081 ceph-mon[36434]: pgmap v183: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 812 KiB/s wr, 42 op/s 2024-05-15T14:06:20.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:20 smithi052 ceph-mon[36190]: pgmap v183: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 812 KiB/s wr, 42 op/s 2024-05-15T14:06:20.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.848+0000 7f5169083700 1 -- 172.21.15.33:0/3360115500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51641088f0 msgr2=0x7f5164110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:20.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.848+0000 7f5169083700 1 --2- 172.21.15.33:0/3360115500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51641088f0 0x7f5164110470 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f515c00b0d0 tx=0x7f515c00b3e0 comp rx=0 tx=0).stop 2024-05-15T14:06:20.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.848+0000 7f5169083700 1 -- 172.21.15.33:0/3360115500 shutdown_connections 2024-05-15T14:06:20.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.848+0000 7f5169083700 1 --2- 172.21.15.33:0/3360115500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51641088f0 0x7f5164110470 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:20.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.848+0000 7f5169083700 1 --2- 172.21.15.33:0/3360115500 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f51641037f0 0x7f5164103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:20.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.848+0000 7f5169083700 1 --2- 172.21.15.33:0/3360115500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51641041a0 0x7f51641081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:20.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.848+0000 7f5169083700 1 -- 172.21.15.33:0/3360115500 >> 172.21.15.33:0/3360115500 conn(0x7f516407a6a0 msgr2=0x7f516407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:20.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.848+0000 7f5169083700 1 -- 172.21.15.33:0/3360115500 shutdown_connections 2024-05-15T14:06:20.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.848+0000 7f5169083700 1 -- 172.21.15.33:0/3360115500 wait complete. 2024-05-15T14:06:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.849+0000 7f5169083700 1 Processor -- start 2024-05-15T14:06:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.849+0000 7f5169083700 1 -- start start 2024-05-15T14:06:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.849+0000 7f5169083700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51641037f0 0x7f5164075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.849+0000 7f5169083700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51641041a0 0x7f5164075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.849+0000 7f5169083700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f51641088f0 0x7f51640762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.849+0000 7f5169083700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5164113740 con 0x7f51641041a0 2024-05-15T14:06:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.849+0000 7f5169083700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f51641135c0 con 0x7f51641037f0 2024-05-15T14:06:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.849+0000 7f5169083700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f51641138c0 con 0x7f51641088f0 2024-05-15T14:06:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.849+0000 7f516259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51641041a0 0x7f5164075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.849+0000 7f516259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51641041a0 0x7f5164075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34642/0 (socket says 172.21.15.33:34642) 2024-05-15T14:06:20.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.849+0000 7f516259c700 1 -- 172.21.15.33:0/3652414240 learned_addr learned my addr 172.21.15.33:0/3652414240 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:20.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.849+0000 7f5162d9d700 1 --2- 172.21.15.33:0/3652414240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51641037f0 0x7f5164075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:20.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.849+0000 7f516359e700 1 --2- 172.21.15.33:0/3652414240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f51641088f0 0x7f51640762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:20.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.850+0000 7f516259c700 1 -- 172.21.15.33:0/3652414240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f51641088f0 msgr2=0x7f51640762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:20.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.850+0000 7f516259c700 1 --2- 172.21.15.33:0/3652414240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f51641088f0 0x7f51640762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:20.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.850+0000 7f516259c700 1 -- 172.21.15.33:0/3652414240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51641037f0 msgr2=0x7f5164075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:20.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.850+0000 7f516259c700 1 --2- 172.21.15.33:0/3652414240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51641037f0 0x7f5164075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:20.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.850+0000 7f516259c700 1 -- 172.21.15.33:0/3652414240 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51641a80d0 con 0x7f51641041a0 2024-05-15T14:06:20.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.850+0000 7f516259c700 1 --2- 172.21.15.33:0/3652414240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51641041a0 0x7f5164075d80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f515800c920 tx=0x7f515800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:20.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.850+0000 7f514bfff700 1 -- 172.21.15.33:0/3652414240 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5158004a30 con 0x7f51641041a0 2024-05-15T14:06:20.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.850+0000 7f5169083700 1 -- 172.21.15.33:0/3652414240 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51641a83c0 con 0x7f51641041a0 2024-05-15T14:06:20.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.850+0000 7f514bfff700 1 -- 172.21.15.33:0/3652414240 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51580054d0 con 0x7f51641041a0 2024-05-15T14:06:20.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.850+0000 7f514bfff700 1 -- 172.21.15.33:0/3652414240 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5158007bd0 con 0x7f51641041a0 2024-05-15T14:06:20.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.850+0000 7f5169083700 1 -- 172.21.15.33:0/3652414240 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51641a8900 con 0x7f51641041a0 2024-05-15T14:06:20.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.853+0000 7f514bfff700 1 -- 172.21.15.33:0/3652414240 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5158004bd0 con 0x7f51641041a0 2024-05-15T14:06:20.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.853+0000 7f514bfff700 1 --2- 172.21.15.33:0/3652414240 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f514c0b29f0 0x7f514c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:20.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.854+0000 7f5162d9d700 1 --2- 172.21.15.33:0/3652414240 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f514c0b29f0 0x7f514c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:20.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.854+0000 7f514bfff700 1 -- 172.21.15.33:0/3652414240 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f51580e20b0 con 0x7f51641041a0 2024-05-15T14:06:20.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.854+0000 7f5173919b80 1 Processor -- start 2024-05-15T14:06:20.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.854+0000 7f5173919b80 1 -- start start 2024-05-15T14:06:20.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.854+0000 7f5173919b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563603153da0 0x563603154180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:20.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.854+0000 7f5173919b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636031546c0 0x56360314e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:20.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.854+0000 7f5173919b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56360314ead0 0x56360314ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:20.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.854+0000 7f5173919b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5636030acd90 con 0x56360314ead0 2024-05-15T14:06:20.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.854+0000 7f5173919b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56360309c930 con 0x563603153da0 2024-05-15T14:06:20.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.854+0000 7f5173919b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5636030a0970 con 0x5636031546c0 2024-05-15T14:06:20.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.854+0000 7f5162d9d700 1 --2- 172.21.15.33:0/3652414240 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f514c0b29f0 0x7f514c0b4eb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5154009a10 tx=0x7f5154009190 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:20.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.854+0000 7f516359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56360314ead0 0x56360314ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:20.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.854+0000 7f516359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56360314ead0 0x56360314ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34646/0 (socket says 172.21.15.33:34646) 2024-05-15T14:06:20.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.854+0000 7f516359e700 1 -- 172.21.15.33:0/1833093928 learned_addr learned my addr 172.21.15.33:0/1833093928 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:20.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.854+0000 7f5162d9d700 1 --2- 172.21.15.33:0/1833093928 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563603153da0 0x563603154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:20.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.854+0000 7f516259c700 1 --2- 172.21.15.33:0/1833093928 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636031546c0 0x56360314e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:20.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.854+0000 7f516359e700 1 -- 172.21.15.33:0/1833093928 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636031546c0 msgr2=0x56360314e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:20.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.854+0000 7f516359e700 1 --2- 172.21.15.33:0/1833093928 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636031546c0 0x56360314e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:20.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.854+0000 7f516359e700 1 -- 172.21.15.33:0/1833093928 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563603153da0 msgr2=0x563603154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:20.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.854+0000 7f516359e700 1 --2- 172.21.15.33:0/1833093928 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563603153da0 0x563603154180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:20.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.854+0000 7f516359e700 1 -- 172.21.15.33:0/1833093928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563602fec9d0 con 0x56360314ead0 2024-05-15T14:06:20.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.855+0000 7f516359e700 1 --2- 172.21.15.33:0/1833093928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56360314ead0 0x56360314ef50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f515c009180 tx=0x7f515c0091b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c17229e3d672dd2d server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:20.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.855+0000 7f51497fa700 1 -- 172.21.15.33:0/1833093928 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515c003c00 con 0x56360314ead0 2024-05-15T14:06:20.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.855+0000 7f51497fa700 1 -- 172.21.15.33:0/1833093928 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f515c003da0 con 0x56360314ead0 2024-05-15T14:06:20.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.855+0000 7f51497fa700 1 -- 172.21.15.33:0/1833093928 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515c0053a0 con 0x56360314ead0 2024-05-15T14:06:20.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.856+0000 7f5173919b80 1 -- 172.21.15.33:0/1833093928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56360314ead0 msgr2=0x56360314ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:20.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.856+0000 7f5173919b80 1 --2- 172.21.15.33:0/1833093928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56360314ead0 0x56360314ef50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f515c009180 tx=0x7f515c0091b0 comp rx=0 tx=0).stop 2024-05-15T14:06:20.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.856+0000 7f5173919b80 1 -- 172.21.15.33:0/1833093928 shutdown_connections 2024-05-15T14:06:20.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.856+0000 7f5173919b80 1 --2- 172.21.15.33:0/1833093928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56360314ead0 0x56360314ef50 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:20.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.856+0000 7f5173919b80 1 --2- 172.21.15.33:0/1833093928 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563603153da0 0x563603154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:20.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.856+0000 7f5173919b80 1 --2- 172.21.15.33:0/1833093928 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636031546c0 0x56360314e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:20.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.856+0000 7f5173919b80 1 -- 172.21.15.33:0/1833093928 >> 172.21.15.33:0/1833093928 conn(0x563603079350 msgr2=0x563603148ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:20.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.856+0000 7f5173919b80 1 -- 172.21.15.33:0/1833093928 shutdown_connections 2024-05-15T14:06:20.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.856+0000 7f5173919b80 1 -- 172.21.15.33:0/1833093928 wait complete. 2024-05-15T14:06:20.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.856+0000 7f5173919b80 1 -- start start 2024-05-15T14:06:20.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.856+0000 7f5173919b80 1 Processor -- start 2024-05-15T14:06:20.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.856+0000 7f5173919b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56360314ead0 0x56360315a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:20.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.857+0000 7f5173919b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563603153da0 0x563603157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:20.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.857+0000 7f5173919b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636031546c0 0x563603157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:20.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.857+0000 7f5173919b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563602fa30a0 con 0x5636031546c0 2024-05-15T14:06:20.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.857+0000 7f5173919b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56360303e8c0 con 0x56360314ead0 2024-05-15T14:06:20.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.857+0000 7f5173919b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56360315b0a0 con 0x563603153da0 2024-05-15T14:06:20.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.857+0000 7f5173919b80 10 client.0 _ll_register_callbacks cb 0x563603033680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:06:20.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.857+0000 7f516359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636031546c0 0x563603157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:20.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.857+0000 7f516359e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636031546c0 0x563603157cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34660/0 (socket says 172.21.15.33:34660) 2024-05-15T14:06:20.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.857+0000 7f516359e700 1 -- 172.21.15.33:0/3175785810 learned_addr learned my addr 172.21.15.33:0/3175785810 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:20.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.857+0000 7f516259c700 1 --2- 172.21.15.33:0/3175785810 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563603153da0 0x563603157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:20.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.857+0000 7f5162d9d700 1 --2- 172.21.15.33:0/3175785810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56360314ead0 0x56360315a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.857+0000 7f516359e700 1 -- 172.21.15.33:0/3175785810 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563603153da0 msgr2=0x563603157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.857+0000 7f516359e700 1 --2- 172.21.15.33:0/3175785810 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563603153da0 0x563603157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.857+0000 7f516359e700 1 -- 172.21.15.33:0/3175785810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56360314ead0 msgr2=0x56360315a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.857+0000 7f516359e700 1 --2- 172.21.15.33:0/3175785810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56360314ead0 0x56360315a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.857+0000 7f516359e700 1 -- 172.21.15.33:0/3175785810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56360303b790 con 0x5636031546c0 2024-05-15T14:06:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.857+0000 7f516359e700 1 --2- 172.21.15.33:0/3175785810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636031546c0 0x563603157cb0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f515c009150 tx=0x7f515c007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2caf0fcc7fb0e7bf server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.857+0000 7f5146ff5700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:06:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.857+0000 7f5146ff5700 1 -- 172.21.15.33:0/3175785810 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515c0052b0 con 0x5636031546c0 2024-05-15T14:06:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.857+0000 7f5146ff5700 1 -- 172.21.15.33:0/3175785810 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f515c005450 con 0x5636031546c0 2024-05-15T14:06:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.857+0000 7f5173919b80 1 -- 172.21.15.33:0/3175785810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f515c003da0 con 0x5636031546c0 2024-05-15T14:06:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.858+0000 7f5146ff5700 1 -- 172.21.15.33:0/3175785810 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f515c006000 con 0x5636031546c0 2024-05-15T14:06:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.858+0000 7f5173919b80 10 client.16107 fetch_fsmap learned FSMap version 29 2024-05-15T14:06:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.858+0000 7f5173919b80 1 -- 172.21.15.33:0/3175785810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5636031657c0 con 0x5636031546c0 2024-05-15T14:06:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.858+0000 7f5146ff5700 1 -- 172.21.15.33:0/3175785810 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f515c003da0 con 0x5636031546c0 2024-05-15T14:06:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.858+0000 7f5146ff5700 1 -- 172.21.15.33:0/3175785810 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f515c011070 con 0x5636031546c0 2024-05-15T14:06:20.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.858+0000 7f5173919b80 10 client.16107 fetch_fsmap finished waiting for FSMap version 29 2024-05-15T14:06:20.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.858+0000 7f5173919b80 10 client.16107 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:06:20.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.858+0000 7f5173919b80 20 client.16107 populate_metadata read hostname 'smithi033' 2024-05-15T14:06:20.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.858+0000 7f5173919b80 1 --2- 172.21.15.33:0/3175785810 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x563603167c90 0x56360316a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:20.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.858+0000 7f5173919b80 4 client.16107 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:06:20.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.858+0000 7f5173919b80 1 -- 172.21.15.33:0/3175785810 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56360311a460 con 0x563603167c90 2024-05-15T14:06:20.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.858+0000 7f5162d9d700 1 --2- 172.21.15.33:0/3175785810 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x563603167c90 0x56360316a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:20.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.859+0000 7f5162d9d700 1 --2- 172.21.15.33:0/3175785810 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x563603167c90 0x56360316a080 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd97923a5da80aab server_cookie=1ddebb729c4e6889 in_seq=0 out_seq=0 2024-05-15T14:06:20.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.859+0000 7f5146ff5700 10 client.16107 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:06:20.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.860+0000 7f5146ff5700 1 -- 172.21.15.33:0/3175785810 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56360311a460 con 0x563603167c90 2024-05-15T14:06:20.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.860+0000 7f5146ff5700 10 client.16107 handle_command_reply: tid=0 2024-05-15T14:06:20.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.860+0000 7f5173919b80 1 client.16107 shutdown 2024-05-15T14:06:20.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.860+0000 7f5173919b80 1 -- 172.21.15.33:0/3175785810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636031546c0 msgr2=0x563603157cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:20.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.860+0000 7f5173919b80 1 --2- 172.21.15.33:0/3175785810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636031546c0 0x563603157cb0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f515c009150 tx=0x7f515c007750 comp rx=0 tx=0).stop 2024-05-15T14:06:20.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.860+0000 7f5173919b80 1 -- 172.21.15.33:0/3175785810 shutdown_connections 2024-05-15T14:06:20.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.860+0000 7f5173919b80 1 --2- 172.21.15.33:0/3175785810 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x563603167c90 0x56360316a080 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:20.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.860+0000 7f5173919b80 1 --2- 172.21.15.33:0/3175785810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636031546c0 0x563603157cb0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:20.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.860+0000 7f5173919b80 1 --2- 172.21.15.33:0/3175785810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56360314ead0 0x56360315a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:20.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.860+0000 7f5146ff5700 0 client.16107 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:06:20.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.860+0000 7f5173919b80 1 --2- 172.21.15.33:0/3175785810 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563603153da0 0x563603157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:20.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.860+0000 7f5173919b80 1 -- 172.21.15.33:0/3175785810 >> 172.21.15.33:0/3175785810 conn(0x563603079350 msgr2=0x5636031437d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:20.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.860+0000 7f5173919b80 1 -- 172.21.15.33:0/3175785810 shutdown_connections 2024-05-15T14:06:20.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.860+0000 7f5173919b80 1 -- 172.21.15.33:0/3175785810 wait complete. 2024-05-15T14:06:20.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.860+0000 7f5173919b80 20 client.16107 trim_cache size 0 max 16384 2024-05-15T14:06:20.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.880+0000 7f5173919b80 1 Processor -- start 2024-05-15T14:06:20.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.880+0000 7f5173919b80 1 -- start start 2024-05-15T14:06:20.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.880+0000 7f5173919b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56360314ead0 0x56360314e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:20.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.880+0000 7f5173919b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563603153da0 0x5636031438e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:20.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.880+0000 7f5173919b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5636031546c0 0x5636031594c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:20.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.880+0000 7f5173919b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f51641b27d0 con 0x56360314ead0 2024-05-15T14:06:20.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.880+0000 7f5173919b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f51641b0060 con 0x5636031546c0 2024-05-15T14:06:20.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.880+0000 7f5173919b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f51641adeb0 con 0x563603153da0 2024-05-15T14:06:20.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.880+0000 7f5162d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56360314ead0 0x56360314e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:20.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.880+0000 7f5162d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56360314ead0 0x56360314e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34670/0 (socket says 172.21.15.33:34670) 2024-05-15T14:06:20.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.880+0000 7f5162d9d700 1 -- 172.21.15.33:0/4124561491 learned_addr learned my addr 172.21.15.33:0/4124561491 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:20.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.881+0000 7f516259c700 1 --2- 172.21.15.33:0/4124561491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563603153da0 0x5636031438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:20.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.881+0000 7f5162d9d700 1 -- 172.21.15.33:0/4124561491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563603153da0 msgr2=0x5636031438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:20.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.881+0000 7f516359e700 1 --2- 172.21.15.33:0/4124561491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5636031546c0 0x5636031594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:20.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.881+0000 7f5162d9d700 1 --2- 172.21.15.33:0/4124561491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563603153da0 0x5636031438e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:20.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.881+0000 7f5162d9d700 1 -- 172.21.15.33:0/4124561491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5636031546c0 msgr2=0x5636031594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:20.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.881+0000 7f5162d9d700 1 --2- 172.21.15.33:0/4124561491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5636031546c0 0x5636031594c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:20.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.881+0000 7f5162d9d700 1 -- 172.21.15.33:0/4124561491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563603158640 con 0x56360314ead0 2024-05-15T14:06:20.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.881+0000 7f516259c700 1 --2- 172.21.15.33:0/4124561491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563603153da0 0x5636031438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:06:20.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.881+0000 7f516359e700 1 --2- 172.21.15.33:0/4124561491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5636031546c0 0x5636031594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:06:20.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.881+0000 7f5162d9d700 1 --2- 172.21.15.33:0/4124561491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56360314ead0 0x56360314e4b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x56360315b0d0 tx=0x7f5154013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efeef6b4a35575ec server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:20.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.881+0000 7f5148ff9700 1 -- 172.21.15.33:0/4124561491 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515401b040 con 0x56360314ead0 2024-05-15T14:06:20.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.881+0000 7f5148ff9700 1 -- 172.21.15.33:0/4124561491 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5154029830 con 0x56360314ead0 2024-05-15T14:06:20.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.881+0000 7f5148ff9700 1 -- 172.21.15.33:0/4124561491 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5154015d90 con 0x56360314ead0 2024-05-15T14:06:20.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.882+0000 7f5173919b80 1 -- 172.21.15.33:0/4124561491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56360314ead0 msgr2=0x56360314e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:20.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.882+0000 7f5173919b80 1 --2- 172.21.15.33:0/4124561491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56360314ead0 0x56360314e4b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x56360315b0d0 tx=0x7f5154013000 comp rx=0 tx=0).stop 2024-05-15T14:06:20.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.882+0000 7f5173919b80 1 -- 172.21.15.33:0/4124561491 shutdown_connections 2024-05-15T14:06:20.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.882+0000 7f5173919b80 1 --2- 172.21.15.33:0/4124561491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5636031546c0 0x5636031594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:20.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.882+0000 7f5173919b80 1 --2- 172.21.15.33:0/4124561491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56360314ead0 0x56360314e4b0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:20.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.882+0000 7f5173919b80 1 --2- 172.21.15.33:0/4124561491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563603153da0 0x5636031438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:20.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.882+0000 7f5173919b80 1 -- 172.21.15.33:0/4124561491 >> 172.21.15.33:0/4124561491 conn(0x563603079350 msgr2=0x563603158db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:20.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.882+0000 7f5173919b80 1 -- 172.21.15.33:0/4124561491 shutdown_connections 2024-05-15T14:06:20.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.882+0000 7f5173919b80 1 -- 172.21.15.33:0/4124561491 wait complete. 2024-05-15T14:06:20.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.882+0000 7f5173919b80 1 -- start start 2024-05-15T14:06:20.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.882+0000 7f5173919b80 1 Processor -- start 2024-05-15T14:06:20.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.883+0000 7f5173919b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56360314ead0 0x56360316d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:20.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.883+0000 7f5173919b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563603153da0 0x56360316b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:20.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.883+0000 7f5173919b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636031546c0 0x56360316bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:20.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.883+0000 7f5173919b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f51641ae520 con 0x56360314ead0 2024-05-15T14:06:20.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.883+0000 7f5173919b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f51641af990 con 0x563603153da0 2024-05-15T14:06:20.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.883+0000 7f5173919b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f51641aebf0 con 0x5636031546c0 2024-05-15T14:06:20.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.883+0000 7f5173919b80 10 client.0 _ll_register_callbacks cb 0x56360314aeb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:06:20.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.883+0000 7f5162d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56360314ead0 0x56360316d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:20.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.883+0000 7f5162d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56360314ead0 0x56360316d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34682/0 (socket says 172.21.15.33:34682) 2024-05-15T14:06:20.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.883+0000 7f516259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563603153da0 0x56360316b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:20.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.883+0000 7f5162d9d700 1 -- 172.21.15.33:0/2690835189 learned_addr learned my addr 172.21.15.33:0/2690835189 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:20.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.883+0000 7f516359e700 1 --2- 172.21.15.33:0/2690835189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636031546c0 0x56360316bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:20.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.883+0000 7f5162d9d700 1 -- 172.21.15.33:0/2690835189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636031546c0 msgr2=0x56360316bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:20.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.883+0000 7f5162d9d700 1 --2- 172.21.15.33:0/2690835189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636031546c0 0x56360316bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:20.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.883+0000 7f5162d9d700 1 -- 172.21.15.33:0/2690835189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563603153da0 msgr2=0x56360316b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:20.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.883+0000 7f5162d9d700 1 --2- 172.21.15.33:0/2690835189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563603153da0 0x56360316b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:20.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.883+0000 7f5162d9d700 1 -- 172.21.15.33:0/2690835189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56360316c730 con 0x56360314ead0 2024-05-15T14:06:20.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.883+0000 7f516359e700 1 --2- 172.21.15.33:0/2690835189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636031546c0 0x56360316bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:06:20.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.883+0000 7f516259c700 1 --2- 172.21.15.33:0/2690835189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563603153da0 0x56360316b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:06:20.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.883+0000 7f5162d9d700 1 --2- 172.21.15.33:0/2690835189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56360314ead0 0x56360316d620 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f515400e770 tx=0x7f515400e7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b04496580cdcd8 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:20.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.883+0000 7f5146ff5700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:06:20.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.884+0000 7f5146ff5700 1 -- 172.21.15.33:0/2690835189 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5154029380 con 0x56360314ead0 2024-05-15T14:06:20.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.884+0000 7f5146ff5700 1 -- 172.21.15.33:0/2690835189 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f5154029520 con 0x56360314ead0 2024-05-15T14:06:20.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.884+0000 7f5173919b80 1 -- 172.21.15.33:0/2690835189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5154029830 con 0x56360314ead0 2024-05-15T14:06:20.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.884+0000 7f5146ff5700 1 -- 172.21.15.33:0/2690835189 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5154015990 con 0x56360314ead0 2024-05-15T14:06:20.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.884+0000 7f5173919b80 10 client.16113 fetch_fsmap learned FSMap version 29 2024-05-15T14:06:20.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.884+0000 7f5173919b80 1 -- 172.21.15.33:0/2690835189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5636031d31d0 con 0x56360314ead0 2024-05-15T14:06:20.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.884+0000 7f5146ff5700 1 -- 172.21.15.33:0/2690835189 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5154029830 con 0x56360314ead0 2024-05-15T14:06:20.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.884+0000 7f5146ff5700 1 -- 172.21.15.33:0/2690835189 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f5154029ce0 con 0x56360314ead0 2024-05-15T14:06:20.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.884+0000 7f5173919b80 10 client.16113 fetch_fsmap finished waiting for FSMap version 29 2024-05-15T14:06:20.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.884+0000 7f5173919b80 10 client.16113 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:06:20.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.884+0000 7f5173919b80 20 client.16113 populate_metadata read hostname 'smithi033' 2024-05-15T14:06:20.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.884+0000 7f5173919b80 1 --2- 172.21.15.33:0/2690835189 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5636031d37c0 0x5636031d5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:20.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.884+0000 7f5173919b80 4 client.16113 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b5682d61", "release": true, "await": true}] 2024-05-15T14:06:20.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.884+0000 7f5173919b80 1 -- 172.21.15.33:0/2690835189 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b5682d61", "release": true, "await": true}) -- 0x5636030aece0 con 0x5636031d37c0 2024-05-15T14:06:20.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.885+0000 7f516259c700 1 --2- 172.21.15.33:0/2690835189 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5636031d37c0 0x5636031d5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:20.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.885+0000 7f516259c700 1 --2- 172.21.15.33:0/2690835189 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5636031d37c0 0x5636031d5be0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=809076ee96a0bbdc server_cookie=1de0e84339de0e50 in_seq=0 out_seq=0 2024-05-15T14:06:20.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.885+0000 7f5146ff5700 10 client.16113 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:06:20.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.890+0000 7f5146ff5700 1 -- 172.21.15.33:0/2690835189 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5636030aece0 con 0x5636031d37c0 2024-05-15T14:06:20.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.890+0000 7f5146ff5700 10 client.16113 handle_command_reply: tid=0 2024-05-15T14:06:20.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.890+0000 7f5173919b80 1 client.16113 shutdown 2024-05-15T14:06:20.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.890+0000 7f5173919b80 1 -- 172.21.15.33:0/2690835189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56360314ead0 msgr2=0x56360316d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:20.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.890+0000 7f5173919b80 1 --2- 172.21.15.33:0/2690835189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56360314ead0 0x56360316d620 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f515400e770 tx=0x7f515400e7a0 comp rx=0 tx=0).stop 2024-05-15T14:06:20.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.890+0000 7f5173919b80 1 -- 172.21.15.33:0/2690835189 shutdown_connections 2024-05-15T14:06:20.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.890+0000 7f5173919b80 1 --2- 172.21.15.33:0/2690835189 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636031546c0 0x56360316bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:20.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.890+0000 7f5173919b80 1 --2- 172.21.15.33:0/2690835189 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5636031d37c0 0x5636031d5be0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:20.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.890+0000 7f5173919b80 1 --2- 172.21.15.33:0/2690835189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56360314ead0 0x56360316d620 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:20.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.890+0000 7f5173919b80 1 --2- 172.21.15.33:0/2690835189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563603153da0 0x56360316b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:20.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.890+0000 7f5173919b80 1 -- 172.21.15.33:0/2690835189 >> 172.21.15.33:0/2690835189 conn(0x563603079350 msgr2=0x563603163bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:20.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.890+0000 7f5146ff5700 0 client.16113 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:06:20.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.890+0000 7f5173919b80 1 -- 172.21.15.33:0/2690835189 shutdown_connections 2024-05-15T14:06:20.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.891+0000 7f5173919b80 1 -- 172.21.15.33:0/2690835189 wait complete. 2024-05-15T14:06:20.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.891+0000 7f5173919b80 20 client.16113 trim_cache size 0 max 16384 2024-05-15T14:06:20.894 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T14:06:20.895 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 29, 2024-05-15T14:06:20.895 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24478, 2024-05-15T14:06:20.895 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 24, 2024-05-15T14:06:20.895 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-05-15T14:06:20.895 INFO:teuthology.orchestra.run.smithi033.stdout: "b5682d61": { 2024-05-15T14:06:20.895 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 24, 2024-05-15T14:06:20.895 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-05-15T14:06:20.895 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:06:20.895 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-05-15T14:06:20.895 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:06:20.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:06:20.895 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 90.0, 2024-05-15T14:06:20.895 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 138.3, 2024-05-15T14:06:20.895 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-05-15T14:06:20.896 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-05-15T14:06:20.896 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-05-15T14:06:20.896 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:06:20.896 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-05-15T14:06:20.896 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:06:20.896 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:06:20.896 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:06:20.896 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:06:20.896 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:06:20.896 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:06:20.896 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T14:06:20.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.891+0000 7f5169083700 1 -- 172.21.15.33:0/3652414240 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f514c0b29f0 msgr2=0x7f514c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:20.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.891+0000 7f5169083700 1 --2- 172.21.15.33:0/3652414240 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f514c0b29f0 0x7f514c0b4eb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5154009a10 tx=0x7f5154009190 comp rx=0 tx=0).stop 2024-05-15T14:06:20.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.891+0000 7f5169083700 1 -- 172.21.15.33:0/3652414240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51641041a0 msgr2=0x7f5164075d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:20.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.891+0000 7f5169083700 1 --2- 172.21.15.33:0/3652414240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51641041a0 0x7f5164075d80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f515800c920 tx=0x7f515800cc30 comp rx=0 tx=0).stop 2024-05-15T14:06:20.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.892+0000 7f5169083700 1 -- 172.21.15.33:0/3652414240 shutdown_connections 2024-05-15T14:06:20.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.892+0000 7f5169083700 1 --2- 172.21.15.33:0/3652414240 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f514c0b29f0 0x7f514c0b4eb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:20.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.892+0000 7f5169083700 1 --2- 172.21.15.33:0/3652414240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f51641088f0 0x7f51640762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:20.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.892+0000 7f5169083700 1 --2- 172.21.15.33:0/3652414240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f51641037f0 0x7f5164075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:20.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.892+0000 7f5169083700 1 --2- 172.21.15.33:0/3652414240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51641041a0 0x7f5164075d80 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:20.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.892+0000 7f5169083700 1 -- 172.21.15.33:0/3652414240 >> 172.21.15.33:0/3652414240 conn(0x7f516407a6a0 msgr2=0x7f51641017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:20.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.892+0000 7f5169083700 1 -- 172.21.15.33:0/3652414240 shutdown_connections 2024-05-15T14:06:20.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:06:20.892+0000 7f5169083700 1 -- 172.21.15.33:0/3652414240 wait complete. 2024-05-15T14:06:21.061 INFO:tasks.workunit.client.0.smithi033.stdout: 1 342974 5.98 MB/sec execute 116 sec latency 18176.601 ms 2024-05-15T14:06:21.508 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b5682d61', total seconds elapsed: 23.7 2024-05-15T14:06:22.061 INFO:tasks.workunit.client.0.smithi033.stdout: 1 344894 6.03 MB/sec execute 117 sec latency 13.835 ms 2024-05-15T14:06:22.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:22 smithi033 ceph-mon[31650]: pgmap v184: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 812 KiB/s wr, 42 op/s 2024-05-15T14:06:22.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:22 smithi052 ceph-mon[36190]: pgmap v184: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 812 KiB/s wr, 42 op/s 2024-05-15T14:06:22.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:22 smithi081 ceph-mon[36434]: pgmap v184: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 812 KiB/s wr, 42 op/s 2024-05-15T14:06:23.061 INFO:tasks.workunit.client.0.smithi033.stdout: 1 347813 6.14 MB/sec execute 118 sec latency 12.346 ms 2024-05-15T14:06:24.061 INFO:tasks.workunit.client.0.smithi033.stdout: 1 349314 6.13 MB/sec execute 119 sec latency 12.787 ms 2024-05-15T14:06:24.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:24 smithi033 ceph-mon[31650]: pgmap v185: 97 pgs: 97 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 1.0 MiB/s wr, 79 op/s 2024-05-15T14:06:24.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:24 smithi052 ceph-mon[36190]: pgmap v185: 97 pgs: 97 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 1.0 MiB/s wr, 79 op/s 2024-05-15T14:06:24.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:24 smithi081 ceph-mon[36434]: pgmap v185: 97 pgs: 97 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 1.0 MiB/s wr, 79 op/s 2024-05-15T14:06:25.061 INFO:tasks.workunit.client.0.smithi033.stdout: 1 351597 6.20 MB/sec execute 120 sec latency 10.872 ms 2024-05-15T14:06:26.061 INFO:tasks.workunit.client.0.smithi033.stdout: 1 353940 6.24 MB/sec execute 121 sec latency 15.569 ms 2024-05-15T14:06:26.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:26 smithi033 ceph-mon[31650]: pgmap v186: 97 pgs: 97 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 338 KiB/s wr, 52 op/s 2024-05-15T14:06:26.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:26 smithi052 ceph-mon[36190]: pgmap v186: 97 pgs: 97 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 338 KiB/s wr, 52 op/s 2024-05-15T14:06:26.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:26 smithi081 ceph-mon[36434]: pgmap v186: 97 pgs: 97 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 338 KiB/s wr, 52 op/s 2024-05-15T14:06:27.061 INFO:tasks.workunit.client.0.smithi033.stdout: 1 355634 6.26 MB/sec execute 122 sec latency 18.919 ms 2024-05-15T14:06:28.061 INFO:tasks.workunit.client.0.smithi033.stdout: 1 357874 6.31 MB/sec execute 123 sec latency 14.653 ms 2024-05-15T14:06:28.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:28 smithi033 ceph-mon[31650]: pgmap v187: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.1 MiB/s wr, 149 op/s 2024-05-15T14:06:28.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:28 smithi052 ceph-mon[36190]: pgmap v187: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.1 MiB/s wr, 149 op/s 2024-05-15T14:06:28.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:28 smithi081 ceph-mon[36434]: pgmap v187: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.1 MiB/s wr, 149 op/s 2024-05-15T14:06:29.061 INFO:tasks.workunit.client.0.smithi033.stdout: 1 359357 6.32 MB/sec execute 124 sec latency 16.592 ms 2024-05-15T14:06:30.061 INFO:tasks.workunit.client.0.smithi033.stdout: 1 361932 6.40 MB/sec execute 125 sec latency 32.338 ms 2024-05-15T14:06:30.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:30 smithi033 ceph-mon[31650]: pgmap v188: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.0 MiB/s wr, 134 op/s 2024-05-15T14:06:30.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:30 smithi052 ceph-mon[36190]: pgmap v188: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.0 MiB/s wr, 134 op/s 2024-05-15T14:06:30.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:30 smithi081 ceph-mon[36434]: pgmap v188: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.0 MiB/s wr, 134 op/s 2024-05-15T14:06:31.061 INFO:tasks.workunit.client.0.smithi033.stdout: 1 363350 6.40 MB/sec execute 126 sec latency 20.439 ms 2024-05-15T14:06:32.061 INFO:tasks.workunit.client.0.smithi033.stdout: 1 365712 6.46 MB/sec execute 127 sec latency 16.416 ms 2024-05-15T14:06:32.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:32 smithi033 ceph-mon[31650]: pgmap v189: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.0 MiB/s wr, 134 op/s 2024-05-15T14:06:32.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:32 smithi052 ceph-mon[36190]: pgmap v189: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.0 MiB/s wr, 134 op/s 2024-05-15T14:06:32.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:32 smithi081 ceph-mon[36434]: pgmap v189: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.0 MiB/s wr, 134 op/s 2024-05-15T14:06:33.061 INFO:tasks.workunit.client.0.smithi033.stdout: 1 367150 6.45 MB/sec execute 128 sec latency 14.544 ms 2024-05-15T14:06:33.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.076+0000 7f52bbf32640 1 -- 172.21.15.33:0/2642726947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52b4107960 msgr2=0x7f52b4107d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:33.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.076+0000 7f52bbf32640 1 --2- 172.21.15.33:0/2642726947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52b4107960 0x7f52b4107d40 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009930 tx=0x7f52b002a420 comp rx=0 tx=0).stop 2024-05-15T14:06:33.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.076+0000 7f52bbf32640 1 -- 172.21.15.33:0/2642726947 shutdown_connections 2024-05-15T14:06:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.076+0000 7f52bbf32640 1 --2- 172.21.15.33:0/2642726947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52b4102310 0x7f52b410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.076+0000 7f52bbf32640 1 --2- 172.21.15.33:0/2642726947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52b4101950 0x7f52b4101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.076+0000 7f52bbf32640 1 --2- 172.21.15.33:0/2642726947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52b4107960 0x7f52b4107d40 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.076+0000 7f52bbf32640 1 -- 172.21.15.33:0/2642726947 >> 172.21.15.33:0/2642726947 conn(0x7f52b4078ad0 msgr2=0x7f52b4078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.076+0000 7f52bbf32640 1 -- 172.21.15.33:0/2642726947 shutdown_connections 2024-05-15T14:06:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.076+0000 7f52bbf32640 1 -- 172.21.15.33:0/2642726947 wait complete. 2024-05-15T14:06:33.078 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.076+0000 7f52bbf32640 1 Processor -- start 2024-05-15T14:06:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.076+0000 7f52bbf32640 1 -- start start 2024-05-15T14:06:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.077+0000 7f52bbf32640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52b4101950 0x7f52b4199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.077+0000 7f52bbf32640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52b4102310 0x7f52b4199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.077+0000 7f52bbf32640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52b4107960 0x7f52b419db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.077+0000 7f52bbf32640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f52b4111c20 con 0x7f52b4101950 2024-05-15T14:06:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.077+0000 7f52bbf32640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f52b4111aa0 con 0x7f52b4102310 2024-05-15T14:06:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.077+0000 7f52bbf32640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f52b4111da0 con 0x7f52b4107960 2024-05-15T14:06:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.077+0000 7f52b9cce640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52b4101950 0x7f52b4199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.077+0000 7f52b9cce640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52b4101950 0x7f52b4199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41154/0 (socket says 172.21.15.33:41154) 2024-05-15T14:06:33.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.077+0000 7f52b9cce640 1 -- 172.21.15.33:0/1514788329 learned_addr learned my addr 172.21.15.33:0/1514788329 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:33.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.077+0000 7f52ba4cf640 1 --2- 172.21.15.33:0/1514788329 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52b4107960 0x7f52b419db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:33.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.077+0000 7f52b94cd640 1 --2- 172.21.15.33:0/1514788329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52b4102310 0x7f52b4199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:33.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.077+0000 7f52ba4cf640 1 -- 172.21.15.33:0/1514788329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52b4102310 msgr2=0x7f52b4199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:33.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.077+0000 7f52ba4cf640 1 --2- 172.21.15.33:0/1514788329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52b4102310 0x7f52b4199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:33.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.077+0000 7f52ba4cf640 1 -- 172.21.15.33:0/1514788329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52b4101950 msgr2=0x7f52b4199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:33.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.077+0000 7f52ba4cf640 1 --2- 172.21.15.33:0/1514788329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52b4101950 0x7f52b4199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:33.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.077+0000 7f52ba4cf640 1 -- 172.21.15.33:0/1514788329 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b419e230 con 0x7f52b4107960 2024-05-15T14:06:33.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.077+0000 7f52b9cce640 1 --2- 172.21.15.33:0/1514788329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52b4101950 0x7f52b4199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:06:33.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.077+0000 7f52ba4cf640 1 --2- 172.21.15.33:0/1514788329 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52b4107960 0x7f52b419db00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f52ac009fd0 tx=0x7f52ac00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:33.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.078+0000 7f52aaffd640 1 -- 172.21.15.33:0/1514788329 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac00ce20 con 0x7f52b4107960 2024-05-15T14:06:33.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.078+0000 7f52bbf32640 1 -- 172.21.15.33:0/1514788329 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b419e520 con 0x7f52b4107960 2024-05-15T14:06:33.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.078+0000 7f52aaffd640 1 -- 172.21.15.33:0/1514788329 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52ac005020 con 0x7f52b4107960 2024-05-15T14:06:33.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.078+0000 7f52aaffd640 1 -- 172.21.15.33:0/1514788329 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac010690 con 0x7f52b4107960 2024-05-15T14:06:33.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.078+0000 7f52bbf32640 1 -- 172.21.15.33:0/1514788329 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b41a5db0 con 0x7f52b4107960 2024-05-15T14:06:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.080+0000 7f52bec4e740 1 Processor -- start 2024-05-15T14:06:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.081+0000 7f52aaffd640 1 -- 172.21.15.33:0/1514788329 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f52ac0054d0 con 0x7f52b4107960 2024-05-15T14:06:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.081+0000 7f52bec4e740 1 -- start start 2024-05-15T14:06:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.081+0000 7f52bec4e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e06338c8f0 0x55e06338ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.081+0000 7f52bec4e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e06338d210 0x55e063397590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.081+0000 7f52bec4e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e063397ad0 0x55e063399ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.081+0000 7f52bec4e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e0632ffc30 con 0x55e06338d210 2024-05-15T14:06:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.081+0000 7f52bec4e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e0632ef7d0 con 0x55e06338c8f0 2024-05-15T14:06:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.081+0000 7f52bec4e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55e0632f3810 con 0x55e063397ad0 2024-05-15T14:06:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.081+0000 7f52b9cce640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e06338d210 0x55e063397590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.081+0000 7f52b94cd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e063397ad0 0x55e063399ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.081+0000 7f52b94cd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e063397ad0 0x55e063399ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:53462/0 (socket says 172.21.15.33:53462) 2024-05-15T14:06:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.081+0000 7f52b94cd640 1 -- 172.21.15.33:0/4227175096 learned_addr learned my addr 172.21.15.33:0/4227175096 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.081+0000 7f52b94cd640 1 --2- 172.21.15.33:0/4227175096 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e06338c8f0 0x55e06338ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.081+0000 7f52b9cce640 1 -- 172.21.15.33:0/4227175096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e063397ad0 msgr2=0x55e063399ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.081+0000 7f52b9cce640 1 --2- 172.21.15.33:0/4227175096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e063397ad0 0x55e063399ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.081+0000 7f52b9cce640 1 -- 172.21.15.33:0/4227175096 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e06338c8f0 msgr2=0x55e06338ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.081+0000 7f52b9cce640 1 --2- 172.21.15.33:0/4227175096 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e06338c8f0 0x55e06338ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.082+0000 7f52b9cce640 1 -- 172.21.15.33:0/4227175096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e063004210 con 0x55e06338d210 2024-05-15T14:06:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.082+0000 7f52aaffd640 1 --2- 172.21.15.33:0/1514788329 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f52a00b2950 0x7f52a00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.082+0000 7f52b94cd640 1 --2- 172.21.15.33:0/1514788329 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f52a00b2950 0x7f52a00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.082+0000 7f52b9cce640 1 --2- 172.21.15.33:0/4227175096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e06338d210 0x55e063397590 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f52b0004770 tx=0x7f52b0032ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe585af43ebc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.082+0000 7f52aaffd640 1 -- 172.21.15.33:0/1514788329 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f52ac0e2c40 con 0x7f52b4107960 2024-05-15T14:06:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.082+0000 7f529ffff640 1 -- 172.21.15.33:0/4227175096 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0035040 con 0x55e06338d210 2024-05-15T14:06:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.082+0000 7f529ffff640 1 -- 172.21.15.33:0/4227175096 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52b00317a0 con 0x55e06338d210 2024-05-15T14:06:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.082+0000 7f529ffff640 1 -- 172.21.15.33:0/4227175096 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b00047a0 con 0x55e06338d210 2024-05-15T14:06:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.083+0000 7f52bec4e740 1 -- 172.21.15.33:0/4227175096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e06338d210 msgr2=0x55e063397590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.083+0000 7f52bec4e740 1 --2- 172.21.15.33:0/4227175096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e06338d210 0x55e063397590 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f52b0004770 tx=0x7f52b0032ed0 comp rx=0 tx=0).stop 2024-05-15T14:06:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.083+0000 7f52b94cd640 1 --2- 172.21.15.33:0/1514788329 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f52a00b2950 0x7f52a00b4e10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x55e06338a220 tx=0x7f52a4005f90 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.083+0000 7f52bec4e740 1 -- 172.21.15.33:0/4227175096 shutdown_connections 2024-05-15T14:06:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.083+0000 7f52bec4e740 1 --2- 172.21.15.33:0/4227175096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e063397ad0 0x55e063399ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.083+0000 7f52bec4e740 1 --2- 172.21.15.33:0/4227175096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e06338d210 0x55e063397590 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.083+0000 7f52bec4e740 1 --2- 172.21.15.33:0/4227175096 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e06338c8f0 0x55e06338ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.083+0000 7f52bec4e740 1 -- 172.21.15.33:0/4227175096 >> 172.21.15.33:0/4227175096 conn(0x55e063388e00 msgr2=0x55e0633891e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.083+0000 7f52bec4e740 1 -- 172.21.15.33:0/4227175096 shutdown_connections 2024-05-15T14:06:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.083+0000 7f52bec4e740 1 -- 172.21.15.33:0/4227175096 wait complete. 2024-05-15T14:06:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.083+0000 7f52bec4e740 1 -- start start 2024-05-15T14:06:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.083+0000 7f52bec4e740 1 Processor -- start 2024-05-15T14:06:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.084+0000 7f52bec4e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e06338c8f0 0x55e063397410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.084+0000 7f52bec4e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e06338d210 0x55e063394700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.084+0000 7f52bec4e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e063397ad0 0x55e063394c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.084+0000 7f52bec4e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e062fd5790 con 0x55e06338d210 2024-05-15T14:06:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.084+0000 7f52bec4e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e0632ab890 con 0x55e06338c8f0 2024-05-15T14:06:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.084+0000 7f52bec4e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55e06339a740 con 0x55e063397ad0 2024-05-15T14:06:33.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.084+0000 7f52bec4e740 10 client.0 _ll_register_callbacks cb 0x55e0632aed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:06:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.084+0000 7f52b94cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e06338d210 0x55e063394700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.084+0000 7f52ba4cf640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e063397ad0 0x55e063394c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.084+0000 7f52b94cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e06338d210 0x55e063394700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41166/0 (socket says 172.21.15.33:41166) 2024-05-15T14:06:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.084+0000 7f52b94cd640 1 -- 172.21.15.33:0/640412613 learned_addr learned my addr 172.21.15.33:0/640412613 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.084+0000 7f52b9cce640 1 --2- 172.21.15.33:0/640412613 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e06338c8f0 0x55e063397410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.084+0000 7f52b94cd640 1 -- 172.21.15.33:0/640412613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e063397ad0 msgr2=0x55e063394c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.084+0000 7f52b94cd640 1 --2- 172.21.15.33:0/640412613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e063397ad0 0x55e063394c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.084+0000 7f52b94cd640 1 -- 172.21.15.33:0/640412613 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e06338c8f0 msgr2=0x55e063397410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.084+0000 7f52b94cd640 1 --2- 172.21.15.33:0/640412613 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e06338c8f0 0x55e063397410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.084+0000 7f52b94cd640 1 -- 172.21.15.33:0/640412613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0630d0620 con 0x55e06338d210 2024-05-15T14:06:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.084+0000 7f52ba4cf640 1 --2- 172.21.15.33:0/640412613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e063397ad0 0x55e063394c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:06:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.084+0000 7f52b9cce640 1 --2- 172.21.15.33:0/640412613 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e06338c8f0 0x55e063397410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:06:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.084+0000 7f52b94cd640 1 --2- 172.21.15.33:0/640412613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e06338d210 0x55e063394700 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f52a4003b40 tx=0x7f52a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d1a93aea341ad81 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.084+0000 7f529d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:06:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.085+0000 7f529d7fa640 1 -- 172.21.15.33:0/640412613 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a4005020 con 0x55e06338d210 2024-05-15T14:06:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.085+0000 7f529d7fa640 1 -- 172.21.15.33:0/640412613 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f52a401a050 con 0x55e06338d210 2024-05-15T14:06:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.085+0000 7f52bec4e740 1 -- 172.21.15.33:0/640412613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52b00317a0 con 0x55e06338d210 2024-05-15T14:06:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.085+0000 7f529d7fa640 1 -- 172.21.15.33:0/640412613 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f52a40051c0 con 0x55e06338d210 2024-05-15T14:06:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.085+0000 7f52bec4e740 10 client.16125 fetch_fsmap learned FSMap version 29 2024-05-15T14:06:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.085+0000 7f52bec4e740 1 -- 172.21.15.33:0/640412613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e0633a6180 con 0x55e06338d210 2024-05-15T14:06:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.085+0000 7f529d7fa640 1 -- 172.21.15.33:0/640412613 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52b00317a0 con 0x55e06338d210 2024-05-15T14:06:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.085+0000 7f529d7fa640 1 -- 172.21.15.33:0/640412613 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f52a4020620 con 0x55e06338d210 2024-05-15T14:06:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.085+0000 7f52bec4e740 10 client.16125 fetch_fsmap finished waiting for FSMap version 29 2024-05-15T14:06:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.085+0000 7f52bec4e740 10 client.16125 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:06:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.085+0000 7f52bec4e740 20 client.16125 populate_metadata read hostname 'smithi033' 2024-05-15T14:06:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.085+0000 7f52bec4e740 1 --2- 172.21.15.33:0/640412613 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e0633a76b0 0x55e0633a9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.085+0000 7f52bec4e740 4 client.16125 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:06:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.085+0000 7f52bec4e740 1 -- 172.21.15.33:0/640412613 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e063301b80 con 0x55e0633a76b0 2024-05-15T14:06:33.093 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.085+0000 7f52b9cce640 1 --2- 172.21.15.33:0/640412613 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e0633a76b0 0x55e0633a9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.086+0000 7f52b9cce640 1 --2- 172.21.15.33:0/640412613 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e0633a76b0 0x55e0633a9aa0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98298bcd3e8ef987 server_cookie=ee21fbb839a6f9e in_seq=0 out_seq=0 2024-05-15T14:06:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.086+0000 7f529d7fa640 10 client.16125 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:06:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.086+0000 7f529d7fa640 1 -- 172.21.15.33:0/640412613 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e063301b80 con 0x55e0633a76b0 2024-05-15T14:06:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.086+0000 7f529d7fa640 10 client.16125 handle_command_reply: tid=0 2024-05-15T14:06:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.087+0000 7f52bec4e740 1 client.16125 shutdown 2024-05-15T14:06:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.087+0000 7f52bec4e740 1 -- 172.21.15.33:0/640412613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e06338d210 msgr2=0x55e063394700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.087+0000 7f52bec4e740 1 --2- 172.21.15.33:0/640412613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e06338d210 0x55e063394700 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f52a4003b40 tx=0x7f52a4004730 comp rx=0 tx=0).stop 2024-05-15T14:06:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.087+0000 7f52bec4e740 1 -- 172.21.15.33:0/640412613 shutdown_connections 2024-05-15T14:06:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.087+0000 7f52bec4e740 1 --2- 172.21.15.33:0/640412613 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e0633a76b0 0x55e0633a9aa0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.087+0000 7f52bec4e740 1 --2- 172.21.15.33:0/640412613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e063397ad0 0x55e063394c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.087+0000 7f52bec4e740 1 --2- 172.21.15.33:0/640412613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e06338d210 0x55e063394700 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.087+0000 7f52bec4e740 1 --2- 172.21.15.33:0/640412613 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e06338c8f0 0x55e063397410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.087+0000 7f529d7fa640 0 client.16125 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:06:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.087+0000 7f52bec4e740 1 -- 172.21.15.33:0/640412613 >> 172.21.15.33:0/640412613 conn(0x55e063388e00 msgr2=0x55e063388100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.087+0000 7f52bec4e740 1 -- 172.21.15.33:0/640412613 shutdown_connections 2024-05-15T14:06:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.087+0000 7f52bec4e740 1 -- 172.21.15.33:0/640412613 wait complete. 2024-05-15T14:06:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.087+0000 7f52bec4e740 20 client.16125 trim_cache size 0 max 16384 2024-05-15T14:06:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.103+0000 7f52bec4e740 1 Processor -- start 2024-05-15T14:06:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.103+0000 7f52bec4e740 1 -- start start 2024-05-15T14:06:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.104+0000 7f52bec4e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e06338c8f0 0x55e0633a3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.104+0000 7f52bec4e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e06338d210 0x55e063387f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.104+0000 7f52bec4e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e063397ad0 0x55e0633884c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.104+0000 7f52bec4e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f52b41b0f90 con 0x55e063397ad0 2024-05-15T14:06:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.104+0000 7f52bec4e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f52b41ae860 con 0x55e06338c8f0 2024-05-15T14:06:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.104+0000 7f52bec4e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f52b41ac6b0 con 0x55e06338d210 2024-05-15T14:06:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.104+0000 7f52ba4cf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e063397ad0 0x55e0633884c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.104+0000 7f52ba4cf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e063397ad0 0x55e0633884c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41182/0 (socket says 172.21.15.33:41182) 2024-05-15T14:06:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.104+0000 7f52ba4cf640 1 -- 172.21.15.33:0/4258847118 learned_addr learned my addr 172.21.15.33:0/4258847118 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.104+0000 7f52b94cd640 1 --2- 172.21.15.33:0/4258847118 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e06338d210 0x55e063387f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.104+0000 7f52b9cce640 1 --2- 172.21.15.33:0/4258847118 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e06338c8f0 0x55e0633a3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.104+0000 7f52ba4cf640 1 -- 172.21.15.33:0/4258847118 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e06338d210 msgr2=0x55e063387f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.104+0000 7f52ba4cf640 1 --2- 172.21.15.33:0/4258847118 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e06338d210 0x55e063387f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.104+0000 7f52ba4cf640 1 -- 172.21.15.33:0/4258847118 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e06338c8f0 msgr2=0x55e0633a3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.104+0000 7f52ba4cf640 1 --2- 172.21.15.33:0/4258847118 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e06338c8f0 0x55e0633a3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.104+0000 7f52ba4cf640 1 -- 172.21.15.33:0/4258847118 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0633902b0 con 0x55e063397ad0 2024-05-15T14:06:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.104+0000 7f52b94cd640 1 --2- 172.21.15.33:0/4258847118 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e06338d210 0x55e063387f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:06:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.104+0000 7f52ba4cf640 1 --2- 172.21.15.33:0/4258847118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e063397ad0 0x55e0633884c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f52ac0e0400 tx=0x7f52ac0a9b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9748e04e05de7cc server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.104+0000 7f529f7fe640 1 -- 172.21.15.33:0/4258847118 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac0c4020 con 0x55e063397ad0 2024-05-15T14:06:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.104+0000 7f529f7fe640 1 -- 172.21.15.33:0/4258847118 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52ac0ae2e0 con 0x55e063397ad0 2024-05-15T14:06:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.104+0000 7f529f7fe640 1 -- 172.21.15.33:0/4258847118 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac0b2b40 con 0x55e063397ad0 2024-05-15T14:06:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.105+0000 7f52bec4e740 1 -- 172.21.15.33:0/4258847118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e063397ad0 msgr2=0x55e0633884c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.105+0000 7f52bec4e740 1 --2- 172.21.15.33:0/4258847118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e063397ad0 0x55e0633884c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f52ac0e0400 tx=0x7f52ac0a9b50 comp rx=0 tx=0).stop 2024-05-15T14:06:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.105+0000 7f52bec4e740 1 -- 172.21.15.33:0/4258847118 shutdown_connections 2024-05-15T14:06:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.105+0000 7f52bec4e740 1 --2- 172.21.15.33:0/4258847118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e063397ad0 0x55e0633884c0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.105+0000 7f52bec4e740 1 --2- 172.21.15.33:0/4258847118 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e06338d210 0x55e063387f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.105+0000 7f52bec4e740 1 --2- 172.21.15.33:0/4258847118 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e06338c8f0 0x55e0633a3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.105+0000 7f52bec4e740 1 -- 172.21.15.33:0/4258847118 >> 172.21.15.33:0/4258847118 conn(0x55e063388e00 msgr2=0x55e063387870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.105+0000 7f52bec4e740 1 -- 172.21.15.33:0/4258847118 shutdown_connections 2024-05-15T14:06:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.105+0000 7f52bec4e740 1 -- 172.21.15.33:0/4258847118 wait complete. 2024-05-15T14:06:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.105+0000 7f52bec4e740 1 -- start start 2024-05-15T14:06:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.105+0000 7f52bec4e740 1 Processor -- start 2024-05-15T14:06:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.106+0000 7f52bec4e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e06338c8f0 0x55e06339ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.106+0000 7f52bec4e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e06338d210 0x55e06339d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.106+0000 7f52bec4e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e063397ad0 0x55e06340ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.106+0000 7f52bec4e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f52b41acd20 con 0x55e06338d210 2024-05-15T14:06:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.106+0000 7f52bec4e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f52b41ae190 con 0x55e063397ad0 2024-05-15T14:06:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.106+0000 7f52bec4e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f52b41ad3f0 con 0x55e06338c8f0 2024-05-15T14:06:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.106+0000 7f52bec4e740 10 client.0 _ll_register_callbacks cb 0x7f52a4003900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:06:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.106+0000 7f52b94cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e06338d210 0x55e06339d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.106+0000 7f52b9cce640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e06338c8f0 0x55e06339ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.106+0000 7f52b9cce640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e06338c8f0 0x55e06339ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:53474/0 (socket says 172.21.15.33:53474) 2024-05-15T14:06:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.106+0000 7f52ba4cf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e063397ad0 0x55e06340ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.106+0000 7f52b9cce640 1 -- 172.21.15.33:0/3070400078 learned_addr learned my addr 172.21.15.33:0/3070400078 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.106+0000 7f52b9cce640 1 -- 172.21.15.33:0/3070400078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e063397ad0 msgr2=0x55e06340ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.106+0000 7f52b9cce640 1 --2- 172.21.15.33:0/3070400078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e063397ad0 0x55e06340ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.106+0000 7f52b9cce640 1 -- 172.21.15.33:0/3070400078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e06338d210 msgr2=0x55e06339d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.106+0000 7f52b9cce640 1 --2- 172.21.15.33:0/3070400078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e06338d210 0x55e06339d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.106+0000 7f52b9cce640 1 -- 172.21.15.33:0/3070400078 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0634104d0 con 0x55e06338c8f0 2024-05-15T14:06:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.106+0000 7f52b94cd640 1 --2- 172.21.15.33:0/3070400078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e06338d210 0x55e06339d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:06:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.106+0000 7f52b9cce640 1 --2- 172.21.15.33:0/3070400078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e06338c8f0 0x55e06339ccd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009a50 tx=0x7f52b0005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=884af7ae426bf9a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.106+0000 7f529d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-05-15T14:06:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.107+0000 7f529d7fa640 1 -- 172.21.15.33:0/3070400078 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0035040 con 0x55e06338c8f0 2024-05-15T14:06:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.107+0000 7f52bec4e740 1 -- 172.21.15.33:0/3070400078 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52ac0ae2e0 con 0x55e06338c8f0 2024-05-15T14:06:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.107+0000 7f529d7fa640 1 -- 172.21.15.33:0/3070400078 <== mon.1 v2:172.21.15.81:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f52b0031c50 con 0x55e06338c8f0 2024-05-15T14:06:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.107+0000 7f529d7fa640 1 -- 172.21.15.33:0/3070400078 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f52b0031e50 con 0x55e06338c8f0 2024-05-15T14:06:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.107+0000 7f52bec4e740 10 client.25753 fetch_fsmap learned FSMap version 29 2024-05-15T14:06:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.107+0000 7f52bec4e740 1 -- 172.21.15.33:0/3070400078 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e0634117e0 con 0x55e06338c8f0 2024-05-15T14:06:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.107+0000 7f529d7fa640 1 -- 172.21.15.33:0/3070400078 <== mon.1 v2:172.21.15.81:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52ac0ae2e0 con 0x55e06338c8f0 2024-05-15T14:06:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.107+0000 7f529d7fa640 1 -- 172.21.15.33:0/3070400078 <== mon.1 v2:172.21.15.81:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f52b0051440 con 0x55e06338c8f0 2024-05-15T14:06:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.107+0000 7f52bec4e740 10 client.25753 fetch_fsmap finished waiting for FSMap version 29 2024-05-15T14:06:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.107+0000 7f52bec4e740 10 client.25753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:06:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.107+0000 7f52bec4e740 20 client.25753 populate_metadata read hostname 'smithi033' 2024-05-15T14:06:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.107+0000 7f52bec4e740 1 --2- 172.21.15.33:0/3070400078 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e063411a60 0x55e063413e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.107+0000 7f52bec4e740 4 client.25753 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:06:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.107+0000 7f52bec4e740 1 -- 172.21.15.33:0/3070400078 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e0632fb360 con 0x55e063411a60 2024-05-15T14:06:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.108+0000 7f52b94cd640 1 --2- 172.21.15.33:0/3070400078 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e063411a60 0x55e063413e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.108+0000 7f52b94cd640 1 --2- 172.21.15.33:0/3070400078 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e063411a60 0x55e063413e50 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b24bebbee397a64a server_cookie=5a2298b1c226a9c in_seq=0 out_seq=0 2024-05-15T14:06:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.108+0000 7f529d7fa640 10 client.25753 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:06:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.170+0000 7f529d7fa640 1 -- 172.21.15.33:0/3070400078 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e0632fb360 con 0x55e063411a60 2024-05-15T14:06:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.170+0000 7f529d7fa640 10 client.25753 handle_command_reply: tid=0 2024-05-15T14:06:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.170+0000 7f52bec4e740 1 client.25753 shutdown 2024-05-15T14:06:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.171+0000 7f52bec4e740 1 -- 172.21.15.33:0/3070400078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e06338c8f0 msgr2=0x55e06339ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.171+0000 7f52bec4e740 1 --2- 172.21.15.33:0/3070400078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e06338c8f0 0x55e06339ccd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009a50 tx=0x7f52b0005d50 comp rx=0 tx=0).stop 2024-05-15T14:06:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.171+0000 7f52bec4e740 1 -- 172.21.15.33:0/3070400078 shutdown_connections 2024-05-15T14:06:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.171+0000 7f52bec4e740 1 --2- 172.21.15.33:0/3070400078 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e063411a60 0x55e063413e50 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.171+0000 7f52bec4e740 1 --2- 172.21.15.33:0/3070400078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e063397ad0 0x55e06340ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.171+0000 7f52bec4e740 1 --2- 172.21.15.33:0/3070400078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e06338d210 0x55e06339d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.171+0000 7f529d7fa640 0 client.25753 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:06:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.171+0000 7f52bec4e740 1 --2- 172.21.15.33:0/3070400078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e06338c8f0 0x55e06339ccd0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.171+0000 7f52bec4e740 1 -- 172.21.15.33:0/3070400078 >> 172.21.15.33:0/3070400078 conn(0x55e063388e00 msgr2=0x55e063388100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.171+0000 7f52bec4e740 1 -- 172.21.15.33:0/3070400078 shutdown_connections 2024-05-15T14:06:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.171+0000 7f52bec4e740 1 -- 172.21.15.33:0/3070400078 wait complete. 2024-05-15T14:06:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.171+0000 7f52bec4e740 20 client.25753 trim_cache size 0 max 16384 2024-05-15T14:06:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:06:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:06:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:06:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.172+0000 7f52bbf32640 1 -- 172.21.15.33:0/1514788329 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f52a00b2950 msgr2=0x7f52a00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.172+0000 7f52bbf32640 1 --2- 172.21.15.33:0/1514788329 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f52a00b2950 0x7f52a00b4e10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x55e06338a220 tx=0x7f52a4005f90 comp rx=0 tx=0).stop 2024-05-15T14:06:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.172+0000 7f52bbf32640 1 -- 172.21.15.33:0/1514788329 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52b4107960 msgr2=0x7f52b419db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.172+0000 7f52bbf32640 1 --2- 172.21.15.33:0/1514788329 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52b4107960 0x7f52b419db00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f52ac009fd0 tx=0x7f52ac00ef10 comp rx=0 tx=0).stop 2024-05-15T14:06:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.172+0000 7f52bbf32640 1 -- 172.21.15.33:0/1514788329 shutdown_connections 2024-05-15T14:06:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.172+0000 7f52bbf32640 1 --2- 172.21.15.33:0/1514788329 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f52a00b2950 0x7f52a00b4e10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.172+0000 7f52bbf32640 1 --2- 172.21.15.33:0/1514788329 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52b4107960 0x7f52b419db00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.172+0000 7f52bbf32640 1 --2- 172.21.15.33:0/1514788329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52b4102310 0x7f52b4199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.172+0000 7f52bbf32640 1 --2- 172.21.15.33:0/1514788329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52b4101950 0x7f52b4199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.172+0000 7f52bbf32640 1 -- 172.21.15.33:0/1514788329 >> 172.21.15.33:0/1514788329 conn(0x7f52b4078ad0 msgr2=0x7f52b406b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.172+0000 7f52bbf32640 1 -- 172.21.15.33:0/1514788329 shutdown_connections 2024-05-15T14:06:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:33.172+0000 7f52bbf32640 1 -- 172.21.15.33:0/1514788329 wait complete. 2024-05-15T14:06:34.061 INFO:tasks.workunit.client.0.smithi033.stdout: 1 369289 6.51 MB/sec execute 129 sec latency 19.668 ms 2024-05-15T14:06:34.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:34 smithi033 ceph-mon[31650]: pgmap v190: 97 pgs: 97 active+clean; 101 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 236 op/s 2024-05-15T14:06:34.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:34 smithi033 ceph-mon[31650]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:06:34.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:34 smithi052 ceph-mon[36190]: pgmap v190: 97 pgs: 97 active+clean; 101 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 236 op/s 2024-05-15T14:06:34.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:34 smithi052 ceph-mon[36190]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:06:34.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:34 smithi081 ceph-mon[36434]: pgmap v190: 97 pgs: 97 active+clean; 101 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 236 op/s 2024-05-15T14:06:34.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:34 smithi081 ceph-mon[36434]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:06:35.061 INFO:tasks.workunit.client.0.smithi033.stdout: 1 371212 6.51 MB/sec execute 130 sec latency 44.178 ms 2024-05-15T14:06:35.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:35 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:06:35.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:35 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:06:35.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:35 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:06:35.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:35 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:06:35.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:35 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:06:35.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:35 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:06:35.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:35 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:06:35.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:35 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:06:35.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:35 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:06:35.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:35 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:06:35.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:35 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:06:35.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:35 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:06:36.062 INFO:tasks.workunit.client.0.smithi033.stdout: 1 373093 6.55 MB/sec execute 131 sec latency 15.775 ms 2024-05-15T14:06:36.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:36 smithi033 ceph-mon[31650]: pgmap v191: 97 pgs: 97 active+clean; 101 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 200 op/s 2024-05-15T14:06:36.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:36 smithi052 ceph-mon[36190]: pgmap v191: 97 pgs: 97 active+clean; 101 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 200 op/s 2024-05-15T14:06:36.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:36 smithi081 ceph-mon[36434]: pgmap v191: 97 pgs: 97 active+clean; 101 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 200 op/s 2024-05-15T14:06:37.062 INFO:tasks.workunit.client.0.smithi033.stdout: 1 375794 6.61 MB/sec execute 132 sec latency 19.035 ms 2024-05-15T14:06:38.062 INFO:tasks.workunit.client.0.smithi033.stdout: 1 377206 6.61 MB/sec execute 133 sec latency 12.086 ms 2024-05-15T14:06:38.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:38 smithi033 ceph-mon[31650]: pgmap v192: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 345 op/s 2024-05-15T14:06:38.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:38 smithi052 ceph-mon[36190]: pgmap v192: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 345 op/s 2024-05-15T14:06:38.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:38 smithi081 ceph-mon[36434]: pgmap v192: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 345 op/s 2024-05-15T14:06:39.062 INFO:tasks.workunit.client.0.smithi033.stdout: 1 379363 6.66 MB/sec execute 134 sec latency 15.383 ms 2024-05-15T14:06:40.062 INFO:tasks.workunit.client.0.smithi033.stdout: 1 380821 6.66 MB/sec execute 135 sec latency 19.897 ms 2024-05-15T14:06:40.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:40 smithi033 ceph-mon[31650]: pgmap v193: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 248 op/s 2024-05-15T14:06:40.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:40 smithi052 ceph-mon[36190]: pgmap v193: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 248 op/s 2024-05-15T14:06:40.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:40 smithi081 ceph-mon[36434]: pgmap v193: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 248 op/s 2024-05-15T14:06:41.062 INFO:tasks.workunit.client.0.smithi033.stdout: 1 383194 6.72 MB/sec execute 136 sec latency 15.347 ms 2024-05-15T14:06:42.062 INFO:tasks.workunit.client.0.smithi033.stdout: 1 384514 6.72 MB/sec execute 137 sec latency 19.387 ms 2024-05-15T14:06:42.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:42 smithi033 ceph-mon[31650]: pgmap v194: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 248 op/s 2024-05-15T14:06:42.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:42 smithi052 ceph-mon[36190]: pgmap v194: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 248 op/s 2024-05-15T14:06:42.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:42 smithi081 ceph-mon[36434]: pgmap v194: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 248 op/s 2024-05-15T14:06:43.062 INFO:tasks.workunit.client.0.smithi033.stdout: 1 386918 6.77 MB/sec execute 138 sec latency 20.731 ms 2024-05-15T14:06:44.062 INFO:tasks.workunit.client.0.smithi033.stdout: 1 388210 6.76 MB/sec execute 139 sec latency 20.259 ms 2024-05-15T14:06:44.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:44 smithi033 ceph-mon[31650]: pgmap v195: 97 pgs: 97 active+clean; 103 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 331 op/s 2024-05-15T14:06:44.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:44 smithi052 ceph-mon[36190]: pgmap v195: 97 pgs: 97 active+clean; 103 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 331 op/s 2024-05-15T14:06:44.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:44 smithi081 ceph-mon[36434]: pgmap v195: 97 pgs: 97 active+clean; 103 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 331 op/s 2024-05-15T14:06:45.062 INFO:tasks.workunit.client.0.smithi033.stdout: 1 390467 6.81 MB/sec execute 140 sec latency 60.154 ms 2024-05-15T14:06:46.062 INFO:tasks.workunit.client.0.smithi033.stdout: 1 391778 6.80 MB/sec execute 141 sec latency 20.441 ms 2024-05-15T14:06:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.265+0000 7f1ee5233640 1 -- 172.21.15.33:0/2844968331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee0101930 msgr2=0x7f1ee0101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.265+0000 7f1ee5233640 1 --2- 172.21.15.33:0/2844968331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee0101930 0x7f1ee0101db0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8009930 tx=0x7f1ec802a420 comp rx=0 tx=0).stop 2024-05-15T14:06:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.265+0000 7f1ee5233640 1 -- 172.21.15.33:0/2844968331 shutdown_connections 2024-05-15T14:06:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.265+0000 7f1ee5233640 1 --2- 172.21.15.33:0/2844968331 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ee01022f0 0x7f1ee010e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.265+0000 7f1ee5233640 1 --2- 172.21.15.33:0/2844968331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee0101930 0x7f1ee0101db0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.265+0000 7f1ee5233640 1 --2- 172.21.15.33:0/2844968331 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ee0107940 0x7f1ee0107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.265+0000 7f1ee5233640 1 -- 172.21.15.33:0/2844968331 >> 172.21.15.33:0/2844968331 conn(0x7f1ee0078a70 msgr2=0x7f1ee0078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.265+0000 7f1ee5233640 1 -- 172.21.15.33:0/2844968331 shutdown_connections 2024-05-15T14:06:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.265+0000 7f1ee5233640 1 -- 172.21.15.33:0/2844968331 wait complete. 2024-05-15T14:06:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.266+0000 7f1ee5233640 1 Processor -- start 2024-05-15T14:06:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.266+0000 7f1ee5233640 1 -- start start 2024-05-15T14:06:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.266+0000 7f1ee5233640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ee0101930 0x7f1ee01992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.266+0000 7f1ee5233640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ee01022f0 0x7f1ee0199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.266+0000 7f1ee5233640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee0107940 0x7f1ee019db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.266+0000 7f1ee5233640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee0111bd0 con 0x7f1ee0107940 2024-05-15T14:06:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.266+0000 7f1ee5233640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee0111a50 con 0x7f1ee0101930 2024-05-15T14:06:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.266+0000 7f1ee5233640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee0111d50 con 0x7f1ee01022f0 2024-05-15T14:06:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.266+0000 7f1ed7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ee01022f0 0x7f1ee0199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.266+0000 7f1edf59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee0107940 0x7f1ee019db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.266+0000 7f1ed7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ee01022f0 0x7f1ee0199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:54200/0 (socket says 172.21.15.33:54200) 2024-05-15T14:06:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.266+0000 7f1edf59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee0107940 0x7f1ee019db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50710/0 (socket says 172.21.15.33:50710) 2024-05-15T14:06:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.266+0000 7f1ed7fff640 1 -- 172.21.15.33:0/1865541162 learned_addr learned my addr 172.21.15.33:0/1865541162 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.266+0000 7f1eded9d640 1 --2- 172.21.15.33:0/1865541162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ee0101930 0x7f1ee01992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.266+0000 7f1ed7fff640 1 -- 172.21.15.33:0/1865541162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ee0101930 msgr2=0x7f1ee01992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.266+0000 7f1ed7fff640 1 --2- 172.21.15.33:0/1865541162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ee0101930 0x7f1ee01992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.266+0000 7f1ed7fff640 1 -- 172.21.15.33:0/1865541162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee0107940 msgr2=0x7f1ee019db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.266+0000 7f1ed7fff640 1 --2- 172.21.15.33:0/1865541162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee0107940 0x7f1ee019db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.266+0000 7f1ed7fff640 1 -- 172.21.15.33:0/1865541162 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ee019e2b0 con 0x7f1ee01022f0 2024-05-15T14:06:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.267+0000 7f1ed7fff640 1 --2- 172.21.15.33:0/1865541162 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ee01022f0 0x7f1ee0199800 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8005940 tx=0x7f1ec80042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.267+0000 7f1edcd99640 1 -- 172.21.15.33:0/1865541162 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec803f030 con 0x7f1ee01022f0 2024-05-15T14:06:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.267+0000 7f1ee5233640 1 -- 172.21.15.33:0/1865541162 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ee019e540 con 0x7f1ee01022f0 2024-05-15T14:06:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.267+0000 7f1edcd99640 1 -- 172.21.15.33:0/1865541162 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ec8003cb0 con 0x7f1ee01022f0 2024-05-15T14:06:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.267+0000 7f1edcd99640 1 -- 172.21.15.33:0/1865541162 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec8035070 con 0x7f1ee01022f0 2024-05-15T14:06:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.267+0000 7f1ee5233640 1 -- 172.21.15.33:0/1865541162 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ee01a5e30 con 0x7f1ee01022f0 2024-05-15T14:06:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.269+0000 7f1ee802a740 1 Processor -- start 2024-05-15T14:06:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.270+0000 7f1ee802a740 1 -- start start 2024-05-15T14:06:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.270+0000 7f1edcd99640 1 -- 172.21.15.33:0/1865541162 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1ec8042030 con 0x7f1ee01022f0 2024-05-15T14:06:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.270+0000 7f1ee802a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a4b958f0 0x5587a4b95cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.270+0000 7f1ee802a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5587a4b96210 0x5587a4ba0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.270+0000 7f1ee802a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587a4ba0ad0 0x5587a4ba2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.270+0000 7f1ee802a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5587a4b08c30 con 0x5587a4b958f0 2024-05-15T14:06:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.270+0000 7f1ee802a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5587a4af87d0 con 0x5587a4b96210 2024-05-15T14:06:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.270+0000 7f1ee802a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5587a4afc810 con 0x5587a4ba0ad0 2024-05-15T14:06:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.270+0000 7f1edf59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a4b958f0 0x5587a4b95cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.270+0000 7f1edf59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a4b958f0 0x5587a4b95cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50712/0 (socket says 172.21.15.33:50712) 2024-05-15T14:06:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.270+0000 7f1edf59e640 1 -- 172.21.15.33:0/3924495515 learned_addr learned my addr 172.21.15.33:0/3924495515 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.270+0000 7f1ed7fff640 1 --2- 172.21.15.33:0/3924495515 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587a4ba0ad0 0x5587a4ba2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.270+0000 7f1eded9d640 1 --2- 172.21.15.33:0/3924495515 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5587a4b96210 0x5587a4ba0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.270+0000 7f1edf59e640 1 -- 172.21.15.33:0/3924495515 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587a4ba0ad0 msgr2=0x5587a4ba2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.270+0000 7f1edf59e640 1 --2- 172.21.15.33:0/3924495515 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587a4ba0ad0 0x5587a4ba2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.270+0000 7f1edf59e640 1 -- 172.21.15.33:0/3924495515 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5587a4b96210 msgr2=0x5587a4ba0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.270+0000 7f1edf59e640 1 --2- 172.21.15.33:0/3924495515 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5587a4b96210 0x5587a4ba0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.270+0000 7f1edf59e640 1 -- 172.21.15.33:0/3924495515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587a480d210 con 0x5587a4b958f0 2024-05-15T14:06:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.271+0000 7f1edf59e640 1 --2- 172.21.15.33:0/3924495515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a4b958f0 0x5587a4b95cd0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1ed80039f0 tx=0x7f1ed8003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c3177ec56ad0b36 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.271+0000 7f1edcd99640 1 --2- 172.21.15.33:0/1865541162 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f1ec00b2950 0x7f1ec00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.271+0000 7f1ed57fa640 1 -- 172.21.15.33:0/3924495515 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed801a470 con 0x5587a4b958f0 2024-05-15T14:06:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.271+0000 7f1ed57fa640 1 -- 172.21.15.33:0/3924495515 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ed8015070 con 0x5587a4b958f0 2024-05-15T14:06:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.271+0000 7f1ed57fa640 1 -- 172.21.15.33:0/3924495515 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed8010800 con 0x5587a4b958f0 2024-05-15T14:06:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.271+0000 7f1eded9d640 1 --2- 172.21.15.33:0/1865541162 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f1ec00b2950 0x7f1ec00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.271+0000 7f1edcd99640 1 -- 172.21.15.33:0/1865541162 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f1ec8102df0 con 0x7f1ee01022f0 2024-05-15T14:06:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.272+0000 7f1ee802a740 1 -- 172.21.15.33:0/3924495515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a4b958f0 msgr2=0x5587a4b95cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.272+0000 7f1ee802a740 1 --2- 172.21.15.33:0/3924495515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a4b958f0 0x5587a4b95cd0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1ed80039f0 tx=0x7f1ed8003eb0 comp rx=0 tx=0).stop 2024-05-15T14:06:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.272+0000 7f1eded9d640 1 --2- 172.21.15.33:0/1865541162 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f1ec00b2950 0x7f1ec00b4e10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x5587a4b96e90 tx=0x7f1ed0005f90 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.272+0000 7f1ee802a740 1 -- 172.21.15.33:0/3924495515 shutdown_connections 2024-05-15T14:06:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.272+0000 7f1ee802a740 1 --2- 172.21.15.33:0/3924495515 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587a4ba0ad0 0x5587a4ba2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.272+0000 7f1ee802a740 1 --2- 172.21.15.33:0/3924495515 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5587a4b96210 0x5587a4ba0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.272+0000 7f1ee802a740 1 --2- 172.21.15.33:0/3924495515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a4b958f0 0x5587a4b95cd0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.272+0000 7f1ee802a740 1 -- 172.21.15.33:0/3924495515 >> 172.21.15.33:0/3924495515 conn(0x5587a4b91e00 msgr2=0x5587a4b921e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.272+0000 7f1ee802a740 1 -- 172.21.15.33:0/3924495515 shutdown_connections 2024-05-15T14:06:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.272+0000 7f1ee802a740 1 -- 172.21.15.33:0/3924495515 wait complete. 2024-05-15T14:06:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.272+0000 7f1ee802a740 1 -- start start 2024-05-15T14:06:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.272+0000 7f1ee802a740 1 Processor -- start 2024-05-15T14:06:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.272+0000 7f1ee802a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a4b958f0 0x5587a4ba0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.272+0000 7f1ee802a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587a4b96210 0x5587a4b9d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.272+0000 7f1ee802a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5587a4ba0ad0 0x5587a4b9dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.272+0000 7f1ee802a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5587a47de790 con 0x5587a4b958f0 2024-05-15T14:06:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.272+0000 7f1ee802a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5587a4ab4890 con 0x5587a4ba0ad0 2024-05-15T14:06:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.272+0000 7f1ee802a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5587a4ba37a0 con 0x5587a4b96210 2024-05-15T14:06:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.273+0000 7f1ee802a740 10 client.0 _ll_register_callbacks cb 0x5587a4ab7d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:06:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.273+0000 7f1eded9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a4b958f0 0x5587a4ba0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.273+0000 7f1ed7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587a4b96210 0x5587a4b9d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.273+0000 7f1eded9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a4b958f0 0x5587a4ba0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50722/0 (socket says 172.21.15.33:50722) 2024-05-15T14:06:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.273+0000 7f1eded9d640 1 -- 172.21.15.33:0/1616796011 learned_addr learned my addr 172.21.15.33:0/1616796011 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.273+0000 7f1edf59e640 1 --2- 172.21.15.33:0/1616796011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5587a4ba0ad0 0x5587a4b9dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.273+0000 7f1ed7fff640 1 -- 172.21.15.33:0/1616796011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5587a4ba0ad0 msgr2=0x5587a4b9dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.273+0000 7f1ed7fff640 1 --2- 172.21.15.33:0/1616796011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5587a4ba0ad0 0x5587a4b9dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.273+0000 7f1ed7fff640 1 -- 172.21.15.33:0/1616796011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a4b958f0 msgr2=0x5587a4ba0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.273+0000 7f1ed7fff640 1 --2- 172.21.15.33:0/1616796011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a4b958f0 0x5587a4ba0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.273+0000 7f1ed7fff640 1 -- 172.21.15.33:0/1616796011 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587a48d9620 con 0x5587a4b96210 2024-05-15T14:06:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.273+0000 7f1eded9d640 1 --2- 172.21.15.33:0/1616796011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a4b958f0 0x5587a4ba0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:06:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.273+0000 7f1ed7fff640 1 --2- 172.21.15.33:0/1616796011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587a4b96210 0x5587a4b9d760 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8100550 tx=0x7f1ec80092f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c889715eccc17f01 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.273+0000 7f1ece7fc640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-05-15T14:06:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.273+0000 7f1ece7fc640 1 -- 172.21.15.33:0/1616796011 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec8049070 con 0x5587a4b96210 2024-05-15T14:06:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.273+0000 7f1ece7fc640 1 -- 172.21.15.33:0/1616796011 <== mon.1 v2:172.21.15.81:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f1ec80ca340 con 0x5587a4b96210 2024-05-15T14:06:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.273+0000 7f1ee802a740 1 -- 172.21.15.33:0/1616796011 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ed8015070 con 0x5587a4b96210 2024-05-15T14:06:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.273+0000 7f1ece7fc640 1 -- 172.21.15.33:0/1616796011 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1ec80c5fa0 con 0x5587a4b96210 2024-05-15T14:06:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.273+0000 7f1ee802a740 10 client.25765 fetch_fsmap learned FSMap version 30 2024-05-15T14:06:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.273+0000 7f1ee802a740 1 -- 172.21.15.33:0/1616796011 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5587a4baf200 con 0x5587a4b96210 2024-05-15T14:06:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.274+0000 7f1ece7fc640 1 -- 172.21.15.33:0/1616796011 <== mon.1 v2:172.21.15.81:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ed8015070 con 0x5587a4b96210 2024-05-15T14:06:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.274+0000 7f1ece7fc640 1 -- 172.21.15.33:0/1616796011 <== mon.1 v2:172.21.15.81:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7f1ec80d2ab0 con 0x5587a4b96210 2024-05-15T14:06:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.274+0000 7f1ee802a740 10 client.25765 fetch_fsmap finished waiting for FSMap version 30 2024-05-15T14:06:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.274+0000 7f1ee802a740 10 client.25765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:06:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.274+0000 7f1ee802a740 20 client.25765 populate_metadata read hostname 'smithi033' 2024-05-15T14:06:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.274+0000 7f1ee802a740 1 --2- 172.21.15.33:0/1616796011 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5587a4bb06b0 0x5587a4bb2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.274+0000 7f1ee802a740 4 client.25765 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:06:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.274+0000 7f1ee802a740 1 -- 172.21.15.33:0/1616796011 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587a4b0ab80 con 0x5587a4bb06b0 2024-05-15T14:06:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.274+0000 7f1eded9d640 1 --2- 172.21.15.33:0/1616796011 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5587a4bb06b0 0x5587a4bb2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.274+0000 7f1ece7fc640 10 client.25765 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:06:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.274+0000 7f1eded9d640 1 --2- 172.21.15.33:0/1616796011 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5587a4bb06b0 0x5587a4bb2b00 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88f7d732eac5a5c8 server_cookie=cbfa79042256e5a in_seq=0 out_seq=0 2024-05-15T14:06:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.275+0000 7f1ece7fc640 1 -- 172.21.15.33:0/1616796011 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5587a4b0ab80 con 0x5587a4bb06b0 2024-05-15T14:06:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.275+0000 7f1ece7fc640 10 client.25765 handle_command_reply: tid=0 2024-05-15T14:06:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.275+0000 7f1ee802a740 1 client.25765 shutdown 2024-05-15T14:06:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.275+0000 7f1ee802a740 1 -- 172.21.15.33:0/1616796011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587a4b96210 msgr2=0x5587a4b9d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.275+0000 7f1ee802a740 1 --2- 172.21.15.33:0/1616796011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587a4b96210 0x5587a4b9d760 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8100550 tx=0x7f1ec80092f0 comp rx=0 tx=0).stop 2024-05-15T14:06:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.275+0000 7f1ee802a740 1 -- 172.21.15.33:0/1616796011 shutdown_connections 2024-05-15T14:06:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.275+0000 7f1ee802a740 1 --2- 172.21.15.33:0/1616796011 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5587a4bb06b0 0x5587a4bb2b00 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.275+0000 7f1ee802a740 1 --2- 172.21.15.33:0/1616796011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5587a4ba0ad0 0x5587a4b9dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.275+0000 7f1ee802a740 1 --2- 172.21.15.33:0/1616796011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587a4b96210 0x5587a4b9d760 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.275+0000 7f1ece7fc640 0 client.25765 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:06:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.275+0000 7f1ee802a740 1 --2- 172.21.15.33:0/1616796011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a4b958f0 0x5587a4ba0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.275+0000 7f1ee802a740 1 -- 172.21.15.33:0/1616796011 >> 172.21.15.33:0/1616796011 conn(0x5587a4b91e00 msgr2=0x5587a4b91100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.275+0000 7f1ee802a740 1 -- 172.21.15.33:0/1616796011 shutdown_connections 2024-05-15T14:06:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.275+0000 7f1ee802a740 1 -- 172.21.15.33:0/1616796011 wait complete. 2024-05-15T14:06:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.275+0000 7f1ee802a740 20 client.25765 trim_cache size 0 max 16384 2024-05-15T14:06:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.291+0000 7f1ee802a740 1 Processor -- start 2024-05-15T14:06:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.291+0000 7f1ee802a740 1 -- start start 2024-05-15T14:06:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.291+0000 7f1ee802a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5587a4b91e00 0x5587a4b9ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.291+0000 7f1ee802a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a4b958f0 0x5587a4b98ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.291+0000 7f1ee802a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587a4b96210 0x5587a4bb8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.291+0000 7f1ee802a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee01b0ea0 con 0x5587a4b958f0 2024-05-15T14:06:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.291+0000 7f1ee802a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee01ae740 con 0x5587a4b91e00 2024-05-15T14:06:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.291+0000 7f1ee802a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee01ac590 con 0x5587a4b96210 2024-05-15T14:06:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.291+0000 7f1ed7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a4b958f0 0x5587a4b98ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.291+0000 7f1ed7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a4b958f0 0x5587a4b98ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50730/0 (socket says 172.21.15.33:50730) 2024-05-15T14:06:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.291+0000 7f1ed7fff640 1 -- 172.21.15.33:0/653025817 learned_addr learned my addr 172.21.15.33:0/653025817 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.291+0000 7f1edf59e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587a4b96210 0x5587a4bb8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.291+0000 7f1ed7fff640 1 -- 172.21.15.33:0/653025817 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587a4b96210 msgr2=0x5587a4bb8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.291+0000 7f1ed7fff640 1 --2- 172.21.15.33:0/653025817 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587a4b96210 0x5587a4bb8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.291+0000 7f1eded9d640 1 --2- 172.21.15.33:0/653025817 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5587a4b91e00 0x5587a4b9ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.291+0000 7f1ed7fff640 1 -- 172.21.15.33:0/653025817 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5587a4b91e00 msgr2=0x5587a4b9ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.291+0000 7f1ed7fff640 1 --2- 172.21.15.33:0/653025817 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5587a4b91e00 0x5587a4b9ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.291+0000 7f1ed7fff640 1 -- 172.21.15.33:0/653025817 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587a4b9e070 con 0x5587a4b958f0 2024-05-15T14:06:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.292+0000 7f1ed7fff640 1 --2- 172.21.15.33:0/653025817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a4b958f0 0x5587a4b98ab0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8005ec0 tx=0x7f1ec80d2f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac2d1eb95bc68bec server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.292+0000 7f1ed4ff9640 1 -- 172.21.15.33:0/653025817 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec8049070 con 0x5587a4b958f0 2024-05-15T14:06:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.292+0000 7f1ed4ff9640 1 -- 172.21.15.33:0/653025817 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ec80ca7e0 con 0x5587a4b958f0 2024-05-15T14:06:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.292+0000 7f1ed4ff9640 1 -- 172.21.15.33:0/653025817 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec80044e0 con 0x5587a4b958f0 2024-05-15T14:06:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.292+0000 7f1ee802a740 1 -- 172.21.15.33:0/653025817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a4b958f0 msgr2=0x5587a4b98ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.292+0000 7f1ee802a740 1 --2- 172.21.15.33:0/653025817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a4b958f0 0x5587a4b98ab0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8005ec0 tx=0x7f1ec80d2f90 comp rx=0 tx=0).stop 2024-05-15T14:06:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.292+0000 7f1ee802a740 1 -- 172.21.15.33:0/653025817 shutdown_connections 2024-05-15T14:06:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.292+0000 7f1ee802a740 1 --2- 172.21.15.33:0/653025817 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587a4b96210 0x5587a4bb8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.292+0000 7f1ee802a740 1 --2- 172.21.15.33:0/653025817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a4b958f0 0x5587a4b98ab0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.292+0000 7f1ee802a740 1 --2- 172.21.15.33:0/653025817 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5587a4b91e00 0x5587a4b9ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.292+0000 7f1ee802a740 1 -- 172.21.15.33:0/653025817 >> 172.21.15.33:0/653025817 conn(0x5587a4b8d400 msgr2=0x5587a4b983a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.293+0000 7f1ee802a740 1 -- 172.21.15.33:0/653025817 shutdown_connections 2024-05-15T14:06:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.293+0000 7f1ee802a740 1 -- 172.21.15.33:0/653025817 wait complete. 2024-05-15T14:06:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.293+0000 7f1ee802a740 1 -- start start 2024-05-15T14:06:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.293+0000 7f1ee802a740 1 Processor -- start 2024-05-15T14:06:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.293+0000 7f1ee802a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5587a4b91e00 0x5587a4ba4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.293+0000 7f1ee802a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a4b958f0 0x5587a4ba4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.293+0000 7f1ee802a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587a4b96210 0x5587a4ba5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.293+0000 7f1ee802a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee01acc00 con 0x5587a4b958f0 2024-05-15T14:06:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.293+0000 7f1ee802a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee01ae070 con 0x5587a4b91e00 2024-05-15T14:06:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.293+0000 7f1ee802a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee01ad2d0 con 0x5587a4b96210 2024-05-15T14:06:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.293+0000 7f1ee802a740 10 client.0 _ll_register_callbacks cb 0x7f1ec8005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:06:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.293+0000 7f1ed7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a4b958f0 0x5587a4ba4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.293+0000 7f1ed7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a4b958f0 0x5587a4ba4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50734/0 (socket says 172.21.15.33:50734) 2024-05-15T14:06:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.293+0000 7f1ed7fff640 1 -- 172.21.15.33:0/3412660416 learned_addr learned my addr 172.21.15.33:0/3412660416 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.293+0000 7f1edf59e640 1 --2- 172.21.15.33:0/3412660416 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587a4b96210 0x5587a4ba5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.293+0000 7f1eded9d640 1 --2- 172.21.15.33:0/3412660416 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5587a4b91e00 0x5587a4ba4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.294+0000 7f1ed7fff640 1 -- 172.21.15.33:0/3412660416 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587a4b96210 msgr2=0x5587a4ba5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.294+0000 7f1ed7fff640 1 --2- 172.21.15.33:0/3412660416 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587a4b96210 0x5587a4ba5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.294+0000 7f1ed7fff640 1 -- 172.21.15.33:0/3412660416 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5587a4b91e00 msgr2=0x5587a4ba4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.294+0000 7f1ed7fff640 1 --2- 172.21.15.33:0/3412660416 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5587a4b91e00 0x5587a4ba4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.294+0000 7f1ed7fff640 1 -- 172.21.15.33:0/3412660416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587a4c1e620 con 0x5587a4b958f0 2024-05-15T14:06:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.294+0000 7f1ed7fff640 1 --2- 172.21.15.33:0/3412660416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a4b958f0 0x5587a4ba4d40 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8003fd0 tx=0x7f1ec80c65f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7917a2ca1925fc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.294+0000 7f1ece7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:06:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.294+0000 7f1ece7fc640 1 -- 172.21.15.33:0/3412660416 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec8049070 con 0x5587a4b958f0 2024-05-15T14:06:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.294+0000 7f1ece7fc640 1 -- 172.21.15.33:0/3412660416 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7f1ec8003800 con 0x5587a4b958f0 2024-05-15T14:06:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.294+0000 7f1ee802a740 1 -- 172.21.15.33:0/3412660416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ec80ca7e0 con 0x5587a4b958f0 2024-05-15T14:06:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.294+0000 7f1ece7fc640 1 -- 172.21.15.33:0/3412660416 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1ec80c9850 con 0x5587a4b958f0 2024-05-15T14:06:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.294+0000 7f1ee802a740 10 client.16149 fetch_fsmap learned FSMap version 30 2024-05-15T14:06:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.294+0000 7f1ee802a740 1 -- 172.21.15.33:0/3412660416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5587a4c1e620 con 0x5587a4b958f0 2024-05-15T14:06:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.294+0000 7f1ece7fc640 1 -- 172.21.15.33:0/3412660416 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ec80ca7e0 con 0x5587a4b958f0 2024-05-15T14:06:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.294+0000 7f1ece7fc640 1 -- 172.21.15.33:0/3412660416 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7f1ec803bb20 con 0x5587a4b958f0 2024-05-15T14:06:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.294+0000 7f1ee802a740 10 client.16149 fetch_fsmap finished waiting for FSMap version 30 2024-05-15T14:06:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.294+0000 7f1ee802a740 10 client.16149 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:06:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.294+0000 7f1ee802a740 20 client.16149 populate_metadata read hostname 'smithi033' 2024-05-15T14:06:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.295+0000 7f1ee802a740 1 --2- 172.21.15.33:0/3412660416 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5587a4ba0ad0 0x5587a4c21b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.295+0000 7f1ee802a740 4 client.16149 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:06:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.295+0000 7f1ee802a740 1 -- 172.21.15.33:0/3412660416 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5587a4b04360 con 0x5587a4ba0ad0 2024-05-15T14:06:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.295+0000 7f1eded9d640 1 --2- 172.21.15.33:0/3412660416 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5587a4ba0ad0 0x5587a4c21b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.295+0000 7f1ece7fc640 10 client.16149 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:06:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.295+0000 7f1eded9d640 1 --2- 172.21.15.33:0/3412660416 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5587a4ba0ad0 0x5587a4c21b50 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d1a9cbd27e7bdca server_cookie=14f7a860cac0a633 in_seq=0 out_seq=0 2024-05-15T14:06:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.362+0000 7f1ece7fc640 1 -- 172.21.15.33:0/3412660416 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5587a4b04360 con 0x5587a4ba0ad0 2024-05-15T14:06:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.362+0000 7f1ece7fc640 10 client.16149 handle_command_reply: tid=0 2024-05-15T14:06:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.362+0000 7f1ee802a740 1 client.16149 shutdown 2024-05-15T14:06:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.362+0000 7f1ee802a740 1 -- 172.21.15.33:0/3412660416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a4b958f0 msgr2=0x5587a4ba4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.362+0000 7f1ee802a740 1 --2- 172.21.15.33:0/3412660416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a4b958f0 0x5587a4ba4d40 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8003fd0 tx=0x7f1ec80c65f0 comp rx=0 tx=0).stop 2024-05-15T14:06:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.362+0000 7f1ee802a740 1 -- 172.21.15.33:0/3412660416 shutdown_connections 2024-05-15T14:06:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.362+0000 7f1ee802a740 1 --2- 172.21.15.33:0/3412660416 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5587a4ba0ad0 0x5587a4c21b50 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.362+0000 7f1ee802a740 1 --2- 172.21.15.33:0/3412660416 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5587a4b96210 0x5587a4ba5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.362+0000 7f1ee802a740 1 --2- 172.21.15.33:0/3412660416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587a4b958f0 0x5587a4ba4d40 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.362+0000 7f1ee802a740 1 --2- 172.21.15.33:0/3412660416 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5587a4b91e00 0x5587a4ba4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.362+0000 7f1ee802a740 1 -- 172.21.15.33:0/3412660416 >> 172.21.15.33:0/3412660416 conn(0x5587a4b8d400 msgr2=0x5587a4b98c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.362+0000 7f1ece7fc640 0 client.16149 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:06:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.362+0000 7f1ee802a740 1 -- 172.21.15.33:0/3412660416 shutdown_connections 2024-05-15T14:06:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.362+0000 7f1ee802a740 1 -- 172.21.15.33:0/3412660416 wait complete. 2024-05-15T14:06:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.362+0000 7f1ee802a740 20 client.16149 trim_cache size 0 max 16384 2024-05-15T14:06:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:06:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:06:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:06:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.363+0000 7f1ee5233640 1 -- 172.21.15.33:0/1865541162 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f1ec00b2950 msgr2=0x7f1ec00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.363+0000 7f1ee5233640 1 --2- 172.21.15.33:0/1865541162 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f1ec00b2950 0x7f1ec00b4e10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x5587a4b96e90 tx=0x7f1ed0005f90 comp rx=0 tx=0).stop 2024-05-15T14:06:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.363+0000 7f1ee5233640 1 -- 172.21.15.33:0/1865541162 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ee01022f0 msgr2=0x7f1ee0199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.363+0000 7f1ee5233640 1 --2- 172.21.15.33:0/1865541162 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ee01022f0 0x7f1ee0199800 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8005940 tx=0x7f1ec80042f0 comp rx=0 tx=0).stop 2024-05-15T14:06:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.363+0000 7f1ee5233640 1 -- 172.21.15.33:0/1865541162 shutdown_connections 2024-05-15T14:06:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.363+0000 7f1ee5233640 1 --2- 172.21.15.33:0/1865541162 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f1ec00b2950 0x7f1ec00b4e10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.363+0000 7f1ee5233640 1 --2- 172.21.15.33:0/1865541162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee0107940 0x7f1ee019db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.363+0000 7f1ee5233640 1 --2- 172.21.15.33:0/1865541162 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ee01022f0 0x7f1ee0199800 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.363+0000 7f1ee5233640 1 --2- 172.21.15.33:0/1865541162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ee0101930 0x7f1ee01992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.363+0000 7f1ee5233640 1 -- 172.21.15.33:0/1865541162 >> 172.21.15.33:0/1865541162 conn(0x7f1ee0078a70 msgr2=0x7f1ee006a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.363+0000 7f1ee5233640 1 -- 172.21.15.33:0/1865541162 shutdown_connections 2024-05-15T14:06:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:46.363+0000 7f1ee5233640 1 -- 172.21.15.33:0/1865541162 wait complete. 2024-05-15T14:06:46.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:46 smithi033 ceph-mon[31650]: pgmap v196: 97 pgs: 97 active+clean; 103 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 229 op/s 2024-05-15T14:06:46.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:46 smithi052 ceph-mon[36190]: pgmap v196: 97 pgs: 97 active+clean; 103 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 229 op/s 2024-05-15T14:06:46.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:46 smithi081 ceph-mon[36434]: pgmap v196: 97 pgs: 97 active+clean; 103 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 229 op/s 2024-05-15T14:06:47.062 INFO:tasks.workunit.client.0.smithi033.stdout: 1 393967 6.85 MB/sec execute 142 sec latency 31.288 ms 2024-05-15T14:06:48.062 INFO:tasks.workunit.client.0.smithi033.stdout: 1 395407 6.84 MB/sec execute 143 sec latency 21.085 ms 2024-05-15T14:06:48.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:48 smithi033 ceph-mon[31650]: pgmap v197: 97 pgs: 97 active+clean; 101 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 353 op/s 2024-05-15T14:06:48.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:48 smithi052 ceph-mon[36190]: pgmap v197: 97 pgs: 97 active+clean; 101 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 353 op/s 2024-05-15T14:06:48.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:48 smithi081 ceph-mon[36434]: pgmap v197: 97 pgs: 97 active+clean; 101 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 353 op/s 2024-05-15T14:06:49.062 INFO:tasks.workunit.client.0.smithi033.stdout: 1 397732 6.89 MB/sec execute 144 sec latency 13.834 ms 2024-05-15T14:06:50.062 INFO:tasks.workunit.client.0.smithi033.stdout: 1 399170 6.88 MB/sec execute 145 sec latency 18.912 ms 2024-05-15T14:06:50.659 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:50 smithi081 ceph-mon[36434]: pgmap v198: 97 pgs: 97 active+clean; 101 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 207 op/s 2024-05-15T14:06:50.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:50 smithi033 ceph-mon[31650]: pgmap v198: 97 pgs: 97 active+clean; 101 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 207 op/s 2024-05-15T14:06:50.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:50 smithi052 ceph-mon[36190]: pgmap v198: 97 pgs: 97 active+clean; 101 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 207 op/s 2024-05-15T14:06:51.062 INFO:tasks.workunit.client.0.smithi033.stdout: 1 401283 6.93 MB/sec execute 146 sec latency 13.878 ms 2024-05-15T14:06:52.062 INFO:tasks.workunit.client.0.smithi033.stdout: 1 402733 6.92 MB/sec execute 147 sec latency 23.319 ms 2024-05-15T14:06:52.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:52 smithi033 ceph-mon[31650]: pgmap v199: 97 pgs: 97 active+clean; 101 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 207 op/s 2024-05-15T14:06:52.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:52 smithi052 ceph-mon[36190]: pgmap v199: 97 pgs: 97 active+clean; 101 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 207 op/s 2024-05-15T14:06:52.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:52 smithi081 ceph-mon[36434]: pgmap v199: 97 pgs: 97 active+clean; 101 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 207 op/s 2024-05-15T14:06:53.062 INFO:tasks.workunit.client.0.smithi033.stdout: 1 404836 6.96 MB/sec execute 148 sec latency 18.818 ms 2024-05-15T14:06:54.062 INFO:tasks.workunit.client.0.smithi033.stdout: 1 406879 6.97 MB/sec execute 149 sec latency 19.312 ms 2024-05-15T14:06:54.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:54 smithi033 ceph-mon[31650]: pgmap v200: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 295 op/s 2024-05-15T14:06:54.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:54 smithi052 ceph-mon[36190]: pgmap v200: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 295 op/s 2024-05-15T14:06:54.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:54 smithi081 ceph-mon[36434]: pgmap v200: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 295 op/s 2024-05-15T14:06:55.062 INFO:tasks.workunit.client.0.smithi033.stdout: 1 408527 7.00 MB/sec execute 150 sec latency 35.523 ms 2024-05-15T14:06:56.062 INFO:tasks.workunit.client.0.smithi033.stdout: 1 411114 7.04 MB/sec execute 151 sec latency 19.888 ms 2024-05-15T14:06:56.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:56 smithi033 ceph-mon[31650]: pgmap v201: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 211 op/s 2024-05-15T14:06:56.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:56 smithi052 ceph-mon[36190]: pgmap v201: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 211 op/s 2024-05-15T14:06:56.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:56 smithi081 ceph-mon[36434]: pgmap v201: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 211 op/s 2024-05-15T14:06:57.063 INFO:tasks.workunit.client.0.smithi033.stdout: 1 412612 7.05 MB/sec execute 152 sec latency 21.830 ms 2024-05-15T14:06:58.063 INFO:tasks.workunit.client.0.smithi033.stdout: 1 415002 7.10 MB/sec execute 153 sec latency 14.074 ms 2024-05-15T14:06:58.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:06:58 smithi033 ceph-mon[31650]: pgmap v202: 97 pgs: 97 active+clean; 115 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 325 op/s 2024-05-15T14:06:58.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:06:58 smithi052 ceph-mon[36190]: pgmap v202: 97 pgs: 97 active+clean; 115 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 325 op/s 2024-05-15T14:06:58.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:06:58 smithi081 ceph-mon[36434]: pgmap v202: 97 pgs: 97 active+clean; 115 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 325 op/s 2024-05-15T14:06:59.063 INFO:tasks.workunit.client.0.smithi033.stdout: 1 416429 7.10 MB/sec execute 154 sec latency 17.252 ms 2024-05-15T14:06:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.454+0000 7fe17d445640 1 -- 172.21.15.33:0/694516056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe178101930 msgr2=0x7fe178101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.454+0000 7fe17d445640 1 --2- 172.21.15.33:0/694516056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe178101930 0x7fe178101db0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fe160009930 tx=0x7fe16002a420 comp rx=0 tx=0).stop 2024-05-15T14:06:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.454+0000 7fe17d445640 1 -- 172.21.15.33:0/694516056 shutdown_connections 2024-05-15T14:06:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.454+0000 7fe17d445640 1 --2- 172.21.15.33:0/694516056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1781022f0 0x7fe17810e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.454+0000 7fe17d445640 1 --2- 172.21.15.33:0/694516056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe178101930 0x7fe178101db0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.454+0000 7fe17d445640 1 --2- 172.21.15.33:0/694516056 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe178107940 0x7fe178107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.454+0000 7fe17d445640 1 -- 172.21.15.33:0/694516056 >> 172.21.15.33:0/694516056 conn(0x7fe178078a70 msgr2=0x7fe178078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:59.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.454+0000 7fe17d445640 1 -- 172.21.15.33:0/694516056 shutdown_connections 2024-05-15T14:06:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.454+0000 7fe17d445640 1 -- 172.21.15.33:0/694516056 wait complete. 2024-05-15T14:06:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.455+0000 7fe17d445640 1 Processor -- start 2024-05-15T14:06:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.455+0000 7fe17d445640 1 -- start start 2024-05-15T14:06:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.455+0000 7fe17d445640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe178101930 0x7fe1781992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.455+0000 7fe17d445640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1781022f0 0x7fe178199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.455+0000 7fe17d445640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe178107940 0x7fe17819daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.455+0000 7fe17d445640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe178111c00 con 0x7fe178107940 2024-05-15T14:06:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.455+0000 7fe17d445640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe178111a80 con 0x7fe1781022f0 2024-05-15T14:06:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.455+0000 7fe17d445640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe178111d80 con 0x7fe178101930 2024-05-15T14:06:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.455+0000 7fe176ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe178101930 0x7fe1781992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.455+0000 7fe1777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe178107940 0x7fe17819daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.455+0000 7fe176ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe178101930 0x7fe1781992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:60286/0 (socket says 172.21.15.33:60286) 2024-05-15T14:06:59.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.455+0000 7fe176ffd640 1 -- 172.21.15.33:0/3190037844 learned_addr learned my addr 172.21.15.33:0/3190037844 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.455+0000 7fe16ffff640 1 --2- 172.21.15.33:0/3190037844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1781022f0 0x7fe178199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.455+0000 7fe176ffd640 1 -- 172.21.15.33:0/3190037844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1781022f0 msgr2=0x7fe178199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.455+0000 7fe176ffd640 1 --2- 172.21.15.33:0/3190037844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1781022f0 0x7fe178199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.455+0000 7fe176ffd640 1 -- 172.21.15.33:0/3190037844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe178107940 msgr2=0x7fe17819daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.455+0000 7fe176ffd640 1 --2- 172.21.15.33:0/3190037844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe178107940 0x7fe17819daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.455+0000 7fe176ffd640 1 -- 172.21.15.33:0/3190037844 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe17819e220 con 0x7fe178101930 2024-05-15T14:06:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.456+0000 7fe176ffd640 1 --2- 172.21.15.33:0/3190037844 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe178101930 0x7fe1781992d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe16800e9f0 tx=0x7fe16800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.456+0000 7fe174ff9640 1 -- 172.21.15.33:0/3190037844 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16800ccc0 con 0x7fe178101930 2024-05-15T14:06:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.456+0000 7fe174ff9640 1 -- 172.21.15.33:0/3190037844 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1680108e0 con 0x7fe178101930 2024-05-15T14:06:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.456+0000 7fe17d445640 1 -- 172.21.15.33:0/3190037844 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe17819e510 con 0x7fe178101930 2024-05-15T14:06:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.456+0000 7fe174ff9640 1 -- 172.21.15.33:0/3190037844 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe168018880 con 0x7fe178101930 2024-05-15T14:06:59.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.456+0000 7fe17d445640 1 -- 172.21.15.33:0/3190037844 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1781a5da0 con 0x7fe178101930 2024-05-15T14:06:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.458+0000 7fe174ff9640 1 -- 172.21.15.33:0/3190037844 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe168010430 con 0x7fe178101930 2024-05-15T14:06:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.458+0000 7fe180239740 1 Processor -- start 2024-05-15T14:06:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.458+0000 7fe180239740 1 -- start start 2024-05-15T14:06:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.459+0000 7fe180239740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561dabdc17f0 0x561dabdc1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.459+0000 7fe180239740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561dabdc2110 0x561dabdd09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.459+0000 7fe180239740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561dabdd0ee0 0x561dabdd32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.459+0000 7fe180239740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561dabd34c30 con 0x561dabdc17f0 2024-05-15T14:06:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.459+0000 7fe180239740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561dabd247d0 con 0x561dabdd0ee0 2024-05-15T14:06:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.459+0000 7fe180239740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x561dabd28810 con 0x561dabdc2110 2024-05-15T14:06:59.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.459+0000 7fe1777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561dabdc17f0 0x561dabdc1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.459+0000 7fe176ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561dabdc2110 0x561dabdd09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.459+0000 7fe1777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561dabdc17f0 0x561dabdc1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55352/0 (socket says 172.21.15.33:55352) 2024-05-15T14:06:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.459+0000 7fe1777fe640 1 -- 172.21.15.33:0/33809307 learned_addr learned my addr 172.21.15.33:0/33809307 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.459+0000 7fe16ffff640 1 --2- 172.21.15.33:0/33809307 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561dabdd0ee0 0x561dabdd32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.460+0000 7fe1777fe640 1 -- 172.21.15.33:0/33809307 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561dabdc2110 msgr2=0x561dabdd09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.460+0000 7fe1777fe640 1 --2- 172.21.15.33:0/33809307 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561dabdc2110 0x561dabdd09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.460+0000 7fe1777fe640 1 -- 172.21.15.33:0/33809307 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561dabdd0ee0 msgr2=0x561dabdd32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.460+0000 7fe1777fe640 1 --2- 172.21.15.33:0/33809307 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561dabdd0ee0 0x561dabdd32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.460+0000 7fe1777fe640 1 -- 172.21.15.33:0/33809307 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561daba39210 con 0x561dabdc17f0 2024-05-15T14:06:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.460+0000 7fe176ffd640 1 --2- 172.21.15.33:0/33809307 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561dabdc2110 0x561dabdd09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:06:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.460+0000 7fe16ffff640 1 --2- 172.21.15.33:0/33809307 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561dabdd0ee0 0x561dabdd32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:06:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.460+0000 7fe1777fe640 1 --2- 172.21.15.33:0/33809307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561dabdc17f0 0x561dabdc1bd0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe170011440 tx=0x7fe170011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5de2cf7a33e5d5f server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.460+0000 7fe174ff9640 1 --2- 172.21.15.33:0/3190037844 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fe1640b29d0 0x7fe1640b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.460+0000 7fe16d7fa640 1 -- 172.21.15.33:0/33809307 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170003c30 con 0x561dabdc17f0 2024-05-15T14:06:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.460+0000 7fe16d7fa640 1 -- 172.21.15.33:0/33809307 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe170003dd0 con 0x561dabdc17f0 2024-05-15T14:06:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.460+0000 7fe16d7fa640 1 -- 172.21.15.33:0/33809307 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17000fae0 con 0x561dabdc17f0 2024-05-15T14:06:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.460+0000 7fe176ffd640 1 --2- 172.21.15.33:0/3190037844 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fe1640b29d0 0x7fe1640b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.460+0000 7fe174ff9640 1 -- 172.21.15.33:0/3190037844 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7fe1680e28d0 con 0x7fe178101930 2024-05-15T14:06:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.461+0000 7fe180239740 1 -- 172.21.15.33:0/33809307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561dabdc17f0 msgr2=0x561dabdc1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.461+0000 7fe176ffd640 1 --2- 172.21.15.33:0/3190037844 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fe1640b29d0 0x7fe1640b4e90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe1680e00e0 tx=0x7fe1680e8550 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.461+0000 7fe180239740 1 --2- 172.21.15.33:0/33809307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561dabdc17f0 0x561dabdc1bd0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe170011440 tx=0x7fe170011900 comp rx=0 tx=0).stop 2024-05-15T14:06:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.461+0000 7fe180239740 1 -- 172.21.15.33:0/33809307 shutdown_connections 2024-05-15T14:06:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.461+0000 7fe180239740 1 --2- 172.21.15.33:0/33809307 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561dabdd0ee0 0x561dabdd32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.461+0000 7fe180239740 1 --2- 172.21.15.33:0/33809307 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561dabdc2110 0x561dabdd09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.461+0000 7fe180239740 1 --2- 172.21.15.33:0/33809307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561dabdc17f0 0x561dabdc1bd0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.461+0000 7fe180239740 1 -- 172.21.15.33:0/33809307 >> 172.21.15.33:0/33809307 conn(0x561dabdbde00 msgr2=0x561dabdbe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.461+0000 7fe180239740 1 -- 172.21.15.33:0/33809307 shutdown_connections 2024-05-15T14:06:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.461+0000 7fe180239740 1 -- 172.21.15.33:0/33809307 wait complete. 2024-05-15T14:06:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.461+0000 7fe180239740 1 -- start start 2024-05-15T14:06:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.461+0000 7fe180239740 1 Processor -- start 2024-05-15T14:06:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.462+0000 7fe180239740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561dabdc17f0 0x561dabdd0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.462+0000 7fe180239740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561dabdc2110 0x561dabdcdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.462+0000 7fe180239740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561dabdd0ee0 0x561dabdce090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.462+0000 7fe180239740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561daba0a790 con 0x561dabdc17f0 2024-05-15T14:06:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.462+0000 7fe180239740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561dabce0890 con 0x561dabdc2110 2024-05-15T14:06:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.462+0000 7fe180239740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x561dabdd3bb0 con 0x561dabdd0ee0 2024-05-15T14:06:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.462+0000 7fe1777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561dabdc17f0 0x561dabdd0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.462+0000 7fe180239740 10 client.0 _ll_register_callbacks cb 0x561dabce3d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:06:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.462+0000 7fe1777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561dabdc17f0 0x561dabdd0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55362/0 (socket says 172.21.15.33:55362) 2024-05-15T14:06:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.462+0000 7fe16ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561dabdc2110 0x561dabdcdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:59.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.462+0000 7fe1777fe640 1 -- 172.21.15.33:0/3239431621 learned_addr learned my addr 172.21.15.33:0/3239431621 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.462+0000 7fe1777fe640 1 --2- 172.21.15.33:0/3239431621 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561dabdd0ee0 0x561dabdce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.462+0000 7fe16ffff640 1 -- 172.21.15.33:0/3239431621 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561dabdd0ee0 msgr2=0x561dabdce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.462+0000 7fe16ffff640 1 --2- 172.21.15.33:0/3239431621 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561dabdd0ee0 0x561dabdce090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.462+0000 7fe16ffff640 1 -- 172.21.15.33:0/3239431621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561dabdc17f0 msgr2=0x561dabdd0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.462+0000 7fe16ffff640 1 --2- 172.21.15.33:0/3239431621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561dabdc17f0 0x561dabdd0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.462+0000 7fe16ffff640 1 -- 172.21.15.33:0/3239431621 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dabb05620 con 0x561dabdc2110 2024-05-15T14:06:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.462+0000 7fe16ffff640 1 --2- 172.21.15.33:0/3239431621 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561dabdc2110 0x561dabdcdb30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe160031440 tx=0x7fe160031e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8e134c92f20302c server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.462+0000 7fe15e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-05-15T14:06:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.462+0000 7fe15e7fc640 1 -- 172.21.15.33:0/3239431621 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe160035040 con 0x561dabdc2110 2024-05-15T14:06:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.463+0000 7fe15e7fc640 1 -- 172.21.15.33:0/3239431621 <== mon.2 v2:172.21.15.52:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7fe1600317e0 con 0x561dabdc2110 2024-05-15T14:06:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.463+0000 7fe180239740 1 -- 172.21.15.33:0/3239431621 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe170003dd0 con 0x561dabdc2110 2024-05-15T14:06:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.463+0000 7fe15e7fc640 1 -- 172.21.15.33:0/3239431621 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe160004040 con 0x561dabdc2110 2024-05-15T14:06:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.463+0000 7fe180239740 10 client.25688 fetch_fsmap learned FSMap version 30 2024-05-15T14:06:59.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.463+0000 7fe180239740 1 -- 172.21.15.33:0/3239431621 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561dabddf650 con 0x561dabdc2110 2024-05-15T14:06:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.463+0000 7fe15e7fc640 1 -- 172.21.15.33:0/3239431621 <== mon.2 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe170003dd0 con 0x561dabdc2110 2024-05-15T14:06:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.463+0000 7fe15e7fc640 1 -- 172.21.15.33:0/3239431621 <== mon.2 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7fe1600047a0 con 0x561dabdc2110 2024-05-15T14:06:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.463+0000 7fe180239740 10 client.25688 fetch_fsmap finished waiting for FSMap version 30 2024-05-15T14:06:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.463+0000 7fe180239740 10 client.25688 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:06:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.463+0000 7fe180239740 20 client.25688 populate_metadata read hostname 'smithi033' 2024-05-15T14:06:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.463+0000 7fe180239740 1 --2- 172.21.15.33:0/3239431621 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x561dabde0b10 0x561dabde2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.463+0000 7fe180239740 4 client.25688 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:06:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.463+0000 7fe180239740 1 -- 172.21.15.33:0/3239431621 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561dabd36b80 con 0x561dabde0b10 2024-05-15T14:06:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.463+0000 7fe1777fe640 1 --2- 172.21.15.33:0/3239431621 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x561dabde0b10 0x561dabde2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.464+0000 7fe15e7fc640 10 client.25688 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:06:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.464+0000 7fe1777fe640 1 --2- 172.21.15.33:0/3239431621 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x561dabde0b10 0x561dabde2f00 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c17cb3b8be1f93b server_cookie=e4402c34512f1f4b in_seq=0 out_seq=0 2024-05-15T14:06:59.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.464+0000 7fe15e7fc640 1 -- 172.21.15.33:0/3239431621 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561dabd36b80 con 0x561dabde0b10 2024-05-15T14:06:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.464+0000 7fe15e7fc640 10 client.25688 handle_command_reply: tid=0 2024-05-15T14:06:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.464+0000 7fe180239740 1 client.25688 shutdown 2024-05-15T14:06:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.465+0000 7fe180239740 1 -- 172.21.15.33:0/3239431621 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561dabdc2110 msgr2=0x561dabdcdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.465+0000 7fe180239740 1 --2- 172.21.15.33:0/3239431621 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561dabdc2110 0x561dabdcdb30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe160031440 tx=0x7fe160031e80 comp rx=0 tx=0).stop 2024-05-15T14:06:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.465+0000 7fe180239740 1 -- 172.21.15.33:0/3239431621 shutdown_connections 2024-05-15T14:06:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.465+0000 7fe180239740 1 --2- 172.21.15.33:0/3239431621 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x561dabde0b10 0x561dabde2f00 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.465+0000 7fe180239740 1 --2- 172.21.15.33:0/3239431621 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561dabdd0ee0 0x561dabdce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.465+0000 7fe180239740 1 --2- 172.21.15.33:0/3239431621 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561dabdc2110 0x561dabdcdb30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.465+0000 7fe15e7fc640 0 client.25688 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:06:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.465+0000 7fe180239740 1 --2- 172.21.15.33:0/3239431621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561dabdc17f0 0x561dabdd0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.465+0000 7fe180239740 1 -- 172.21.15.33:0/3239431621 >> 172.21.15.33:0/3239431621 conn(0x561dabdbde00 msgr2=0x561dabdc3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.465+0000 7fe180239740 1 -- 172.21.15.33:0/3239431621 shutdown_connections 2024-05-15T14:06:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.465+0000 7fe180239740 1 -- 172.21.15.33:0/3239431621 wait complete. 2024-05-15T14:06:59.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.465+0000 7fe180239740 20 client.25688 trim_cache size 0 max 16384 2024-05-15T14:06:59.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.480+0000 7fe180239740 1 Processor -- start 2024-05-15T14:06:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.480+0000 7fe180239740 1 -- start start 2024-05-15T14:06:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.480+0000 7fe180239740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561dabdc17f0 0x561dabddca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.480+0000 7fe180239740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561dabdc2110 0x561dabdd0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.480+0000 7fe180239740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561dabdd0ee0 0x561dabde8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.480+0000 7fe180239740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe1781aed40 con 0x561dabdc17f0 2024-05-15T14:06:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.480+0000 7fe180239740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe1781ac5e0 con 0x561dabdd0ee0 2024-05-15T14:06:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.480+0000 7fe180239740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe1781aa430 con 0x561dabdc2110 2024-05-15T14:06:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.480+0000 7fe1777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561dabdc17f0 0x561dabddca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.480+0000 7fe1777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561dabdc17f0 0x561dabddca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55364/0 (socket says 172.21.15.33:55364) 2024-05-15T14:06:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.480+0000 7fe16ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561dabdc2110 0x561dabdd0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.480+0000 7fe1777fe640 1 -- 172.21.15.33:0/2964394879 learned_addr learned my addr 172.21.15.33:0/2964394879 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.481+0000 7fe1777fe640 1 -- 172.21.15.33:0/2964394879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561dabdc2110 msgr2=0x561dabdd0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:59.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.481+0000 7fe1777fe640 1 --2- 172.21.15.33:0/2964394879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561dabdc2110 0x561dabdd0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.481+0000 7fe1777fe640 1 -- 172.21.15.33:0/2964394879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561dabdd0ee0 msgr2=0x561dabde8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.481+0000 7fe1777fe640 1 --2- 172.21.15.33:0/2964394879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561dabdd0ee0 0x561dabde8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.481+0000 7fe1777fe640 1 -- 172.21.15.33:0/2964394879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dabde3530 con 0x561dabdc17f0 2024-05-15T14:06:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.481+0000 7fe16ffff640 1 --2- 172.21.15.33:0/2964394879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561dabdc2110 0x561dabdd0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:06:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.481+0000 7fe1777fe640 1 --2- 172.21.15.33:0/2964394879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561dabdc17f0 0x561dabddca80 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe170000f80 tx=0x7fe170009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc496e16c0930a19 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.481+0000 7fe16cff9640 1 -- 172.21.15.33:0/2964394879 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170003cf0 con 0x561dabdc17f0 2024-05-15T14:06:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.481+0000 7fe16cff9640 1 -- 172.21.15.33:0/2964394879 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe17002f050 con 0x561dabdc17f0 2024-05-15T14:06:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.481+0000 7fe16cff9640 1 -- 172.21.15.33:0/2964394879 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17002f710 con 0x561dabdc17f0 2024-05-15T14:06:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.482+0000 7fe180239740 1 -- 172.21.15.33:0/2964394879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561dabdc17f0 msgr2=0x561dabddca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.482+0000 7fe180239740 1 --2- 172.21.15.33:0/2964394879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561dabdc17f0 0x561dabddca80 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe170000f80 tx=0x7fe170009900 comp rx=0 tx=0).stop 2024-05-15T14:06:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.482+0000 7fe180239740 1 -- 172.21.15.33:0/2964394879 shutdown_connections 2024-05-15T14:06:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.482+0000 7fe180239740 1 --2- 172.21.15.33:0/2964394879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561dabdd0ee0 0x561dabde8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.482+0000 7fe180239740 1 --2- 172.21.15.33:0/2964394879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561dabdc2110 0x561dabdd0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.482+0000 7fe180239740 1 --2- 172.21.15.33:0/2964394879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561dabdc17f0 0x561dabddca80 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.482+0000 7fe180239740 1 -- 172.21.15.33:0/2964394879 >> 172.21.15.33:0/2964394879 conn(0x561dabdbde00 msgr2=0x561dabdc3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.482+0000 7fe180239740 1 -- 172.21.15.33:0/2964394879 shutdown_connections 2024-05-15T14:06:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.482+0000 7fe180239740 1 -- 172.21.15.33:0/2964394879 wait complete. 2024-05-15T14:06:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.482+0000 7fe180239740 1 -- start start 2024-05-15T14:06:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.482+0000 7fe180239740 1 Processor -- start 2024-05-15T14:06:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.483+0000 7fe180239740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561dabdc17f0 0x561dabdd94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.483+0000 7fe180239740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561dabdc2110 0x561dabdd4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.483+0000 7fe180239740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561dabdd0ee0 0x561dabdd4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.483+0000 7fe180239740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe1781aaaa0 con 0x561dabdc17f0 2024-05-15T14:06:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.483+0000 7fe180239740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe1781abf10 con 0x561dabdd0ee0 2024-05-15T14:06:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.483+0000 7fe180239740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe1781ab170 con 0x561dabdc2110 2024-05-15T14:06:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.483+0000 7fe180239740 10 client.0 _ll_register_callbacks cb 0x7fe160031b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:06:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.483+0000 7fe1777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561dabdc17f0 0x561dabdd94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.483+0000 7fe1777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561dabdc17f0 0x561dabdd94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55376/0 (socket says 172.21.15.33:55376) 2024-05-15T14:06:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.483+0000 7fe1777fe640 1 -- 172.21.15.33:0/1321258569 learned_addr learned my addr 172.21.15.33:0/1321258569 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:06:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.483+0000 7fe16ffff640 1 --2- 172.21.15.33:0/1321258569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561dabdc2110 0x561dabdd4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.483+0000 7fe1777fe640 1 --2- 172.21.15.33:0/1321258569 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561dabdd0ee0 0x561dabdd4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.483+0000 7fe1777fe640 1 -- 172.21.15.33:0/1321258569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561dabdc2110 msgr2=0x561dabdd4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.483+0000 7fe1777fe640 1 --2- 172.21.15.33:0/1321258569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561dabdc2110 0x561dabdd4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.483+0000 7fe1777fe640 1 -- 172.21.15.33:0/1321258569 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561dabdd0ee0 msgr2=0x561dabdd4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:06:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.483+0000 7fe1777fe640 1 --2- 172.21.15.33:0/1321258569 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561dabdd0ee0 0x561dabdd4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.483+0000 7fe1777fe640 1 -- 172.21.15.33:0/1321258569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dabe4a310 con 0x561dabdc17f0 2024-05-15T14:06:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.483+0000 7fe1777fe640 1 --2- 172.21.15.33:0/1321258569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561dabdc17f0 0x561dabdd94d0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe170005f10 tx=0x7fe170009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7605b9e98bcbfb4d server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:06:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.483+0000 7fe15e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:06:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.483+0000 7fe15e7fc640 1 -- 172.21.15.33:0/1321258569 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17000f7f0 con 0x561dabdc17f0 2024-05-15T14:06:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.483+0000 7fe15e7fc640 1 -- 172.21.15.33:0/1321258569 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8498+0+0 (secure 0 0 0) 0x7fe17000f990 con 0x561dabdc17f0 2024-05-15T14:06:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.483+0000 7fe180239740 1 -- 172.21.15.33:0/1321258569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe17002f050 con 0x561dabdc17f0 2024-05-15T14:06:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.484+0000 7fe15e7fc640 1 -- 172.21.15.33:0/1321258569 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe170047050 con 0x561dabdc17f0 2024-05-15T14:06:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.484+0000 7fe180239740 10 client.16167 fetch_fsmap learned FSMap version 30 2024-05-15T14:06:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.484+0000 7fe180239740 1 -- 172.21.15.33:0/1321258569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561dabe4b4c0 con 0x561dabdc17f0 2024-05-15T14:06:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.484+0000 7fe15e7fc640 1 -- 172.21.15.33:0/1321258569 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe17002f050 con 0x561dabdc17f0 2024-05-15T14:06:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.484+0000 7fe15e7fc640 1 -- 172.21.15.33:0/1321258569 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7fe17002fce0 con 0x561dabdc17f0 2024-05-15T14:06:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.484+0000 7fe180239740 10 client.16167 fetch_fsmap finished waiting for FSMap version 30 2024-05-15T14:06:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.484+0000 7fe180239740 10 client.16167 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:06:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.484+0000 7fe180239740 20 client.16167 populate_metadata read hostname 'smithi033' 2024-05-15T14:06:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.484+0000 7fe180239740 1 --2- 172.21.15.33:0/1321258569 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x561dabe4b980 0x561dabe4dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:06:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.484+0000 7fe180239740 4 client.16167 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:06:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.484+0000 7fe180239740 1 -- 172.21.15.33:0/1321258569 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561dabd30360 con 0x561dabe4b980 2024-05-15T14:06:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.484+0000 7fe16ffff640 1 --2- 172.21.15.33:0/1321258569 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x561dabe4b980 0x561dabe4dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:06:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.485+0000 7fe15e7fc640 10 client.16167 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:06:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.485+0000 7fe16ffff640 1 --2- 172.21.15.33:0/1321258569 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x561dabe4b980 0x561dabe4dd70 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=454e6885a827d4c7 server_cookie=71ae47df0b29a050 in_seq=0 out_seq=0 2024-05-15T14:06:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.544+0000 7fe15e7fc640 1 -- 172.21.15.33:0/1321258569 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561dabd30360 con 0x561dabe4b980 2024-05-15T14:06:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.544+0000 7fe15e7fc640 10 client.16167 handle_command_reply: tid=0 2024-05-15T14:06:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.544+0000 7fe180239740 1 client.16167 shutdown 2024-05-15T14:06:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.544+0000 7fe180239740 1 -- 172.21.15.33:0/1321258569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561dabdc17f0 msgr2=0x561dabdd94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.544+0000 7fe180239740 1 --2- 172.21.15.33:0/1321258569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561dabdc17f0 0x561dabdd94d0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe170005f10 tx=0x7fe170009a70 comp rx=0 tx=0).stop 2024-05-15T14:06:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.544+0000 7fe180239740 1 -- 172.21.15.33:0/1321258569 shutdown_connections 2024-05-15T14:06:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.544+0000 7fe180239740 1 --2- 172.21.15.33:0/1321258569 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x561dabe4b980 0x561dabe4dd70 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.544+0000 7fe180239740 1 --2- 172.21.15.33:0/1321258569 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561dabdd0ee0 0x561dabdd4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.544+0000 7fe180239740 1 --2- 172.21.15.33:0/1321258569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561dabdc2110 0x561dabdd4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.544+0000 7fe180239740 1 --2- 172.21.15.33:0/1321258569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561dabdc17f0 0x561dabdd94d0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.544+0000 7fe180239740 1 -- 172.21.15.33:0/1321258569 >> 172.21.15.33:0/1321258569 conn(0x561dabdbde00 msgr2=0x561dabdc3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.544+0000 7fe15e7fc640 0 client.16167 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:06:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.545+0000 7fe180239740 1 -- 172.21.15.33:0/1321258569 shutdown_connections 2024-05-15T14:06:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.545+0000 7fe180239740 1 -- 172.21.15.33:0/1321258569 wait complete. 2024-05-15T14:06:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.545+0000 7fe180239740 20 client.16167 trim_cache size 0 max 16384 2024-05-15T14:06:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:06:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:06:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:06:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.545+0000 7fe17d445640 1 -- 172.21.15.33:0/3190037844 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fe1640b29d0 msgr2=0x7fe1640b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.545+0000 7fe17d445640 1 --2- 172.21.15.33:0/3190037844 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fe1640b29d0 0x7fe1640b4e90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe1680e00e0 tx=0x7fe1680e8550 comp rx=0 tx=0).stop 2024-05-15T14:06:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.545+0000 7fe17d445640 1 -- 172.21.15.33:0/3190037844 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe178101930 msgr2=0x7fe1781992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:06:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.545+0000 7fe17d445640 1 --2- 172.21.15.33:0/3190037844 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe178101930 0x7fe1781992d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe16800e9f0 tx=0x7fe16800edf0 comp rx=0 tx=0).stop 2024-05-15T14:06:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.545+0000 7fe17d445640 1 -- 172.21.15.33:0/3190037844 shutdown_connections 2024-05-15T14:06:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.545+0000 7fe17d445640 1 --2- 172.21.15.33:0/3190037844 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fe1640b29d0 0x7fe1640b4e90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.545+0000 7fe17d445640 1 --2- 172.21.15.33:0/3190037844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe178107940 0x7fe17819daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.545+0000 7fe17d445640 1 --2- 172.21.15.33:0/3190037844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe1781022f0 0x7fe178199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.545+0000 7fe17d445640 1 --2- 172.21.15.33:0/3190037844 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe178101930 0x7fe1781992d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:06:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.545+0000 7fe17d445640 1 -- 172.21.15.33:0/3190037844 >> 172.21.15.33:0/3190037844 conn(0x7fe178078a70 msgr2=0x7fe17806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:06:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.546+0000 7fe17d445640 1 -- 172.21.15.33:0/3190037844 shutdown_connections 2024-05-15T14:06:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:06:59.546+0000 7fe17d445640 1 -- 172.21.15.33:0/3190037844 wait complete. 2024-05-15T14:07:00.063 INFO:tasks.workunit.client.0.smithi033.stdout: 1 418227 7.10 MB/sec execute 155 sec latency 53.271 ms 2024-05-15T14:07:00.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:00 smithi033 ceph-mon[31650]: pgmap v203: 97 pgs: 97 active+clean; 115 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 201 op/s 2024-05-15T14:07:00.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:00 smithi052 ceph-mon[36190]: pgmap v203: 97 pgs: 97 active+clean; 115 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 201 op/s 2024-05-15T14:07:00.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:00 smithi081 ceph-mon[36434]: pgmap v203: 97 pgs: 97 active+clean; 115 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 201 op/s 2024-05-15T14:07:01.063 INFO:tasks.workunit.client.0.smithi033.stdout: 1 419092 7.10 MB/sec execute 156 sec latency 24.450 ms 2024-05-15T14:07:02.063 INFO:tasks.workunit.client.0.smithi033.stdout: 1 421356 7.12 MB/sec execute 157 sec latency 13.109 ms 2024-05-15T14:07:02.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:02 smithi033 ceph-mon[31650]: pgmap v204: 97 pgs: 97 active+clean; 115 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 201 op/s 2024-05-15T14:07:02.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:02 smithi052 ceph-mon[36190]: pgmap v204: 97 pgs: 97 active+clean; 115 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 201 op/s 2024-05-15T14:07:02.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:02 smithi081 ceph-mon[36434]: pgmap v204: 97 pgs: 97 active+clean; 115 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 201 op/s 2024-05-15T14:07:03.063 INFO:tasks.workunit.client.0.smithi033.stdout: 1 423005 7.13 MB/sec execute 158 sec latency 19.934 ms 2024-05-15T14:07:04.063 INFO:tasks.workunit.client.0.smithi033.stdout: 1 425030 7.16 MB/sec execute 159 sec latency 18.108 ms 2024-05-15T14:07:04.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:04 smithi033 ceph-mon[31650]: pgmap v205: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 381 op/s 2024-05-15T14:07:04.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:04 smithi052 ceph-mon[36190]: pgmap v205: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 381 op/s 2024-05-15T14:07:04.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:04 smithi081 ceph-mon[36434]: pgmap v205: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 381 op/s 2024-05-15T14:07:05.063 INFO:tasks.workunit.client.0.smithi033.stdout: 1 426178 7.16 MB/sec execute 160 sec latency 24.790 ms 2024-05-15T14:07:05.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:05 smithi033 ceph-mon[31650]: osdmap e85: 12 total, 12 up, 12 in 2024-05-15T14:07:05.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:05 smithi052 ceph-mon[36190]: osdmap e85: 12 total, 12 up, 12 in 2024-05-15T14:07:05.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:05 smithi081 ceph-mon[36434]: osdmap e85: 12 total, 12 up, 12 in 2024-05-15T14:07:06.063 INFO:tasks.workunit.client.0.smithi033.stdout: 1 428172 7.16 MB/sec execute 161 sec latency 41.168 ms 2024-05-15T14:07:06.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:06 smithi033 ceph-mon[31650]: pgmap v207: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 352 op/s 2024-05-15T14:07:06.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:06 smithi052 ceph-mon[36190]: pgmap v207: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 352 op/s 2024-05-15T14:07:06.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:06 smithi081 ceph-mon[36434]: pgmap v207: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 352 op/s 2024-05-15T14:07:07.063 INFO:tasks.workunit.client.0.smithi033.stdout: 1 429962 7.19 MB/sec execute 162 sec latency 15.712 ms 2024-05-15T14:07:08.063 INFO:tasks.workunit.client.0.smithi033.stdout: 1 431779 7.19 MB/sec execute 163 sec latency 18.979 ms 2024-05-15T14:07:08.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:08 smithi052 ceph-mon[36190]: pgmap v208: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 343 op/s 2024-05-15T14:07:08.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:08 smithi081 ceph-mon[36434]: pgmap v208: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 343 op/s 2024-05-15T14:07:08.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:08 smithi033 ceph-mon[31650]: pgmap v208: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 343 op/s 2024-05-15T14:07:09.063 INFO:tasks.workunit.client.0.smithi033.stdout: 1 433555 7.22 MB/sec execute 164 sec latency 16.558 ms 2024-05-15T14:07:10.063 INFO:tasks.workunit.client.0.smithi033.stdout: 1 435744 7.25 MB/sec execute 165 sec latency 15.284 ms 2024-05-15T14:07:10.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:10 smithi052 ceph-mon[36190]: pgmap v209: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 343 op/s 2024-05-15T14:07:10.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:10 smithi081 ceph-mon[36434]: pgmap v209: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 343 op/s 2024-05-15T14:07:10.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:10 smithi033 ceph-mon[31650]: pgmap v209: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 343 op/s 2024-05-15T14:07:11.063 INFO:tasks.workunit.client.0.smithi033.stdout: 1 437181 7.25 MB/sec execute 166 sec latency 37.206 ms 2024-05-15T14:07:11.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:11 smithi052 ceph-mon[36190]: osdmap e86: 12 total, 12 up, 12 in 2024-05-15T14:07:11.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:11 smithi081 ceph-mon[36434]: osdmap e86: 12 total, 12 up, 12 in 2024-05-15T14:07:11.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:11 smithi033 ceph-mon[31650]: osdmap e86: 12 total, 12 up, 12 in 2024-05-15T14:07:12.063 INFO:tasks.workunit.client.0.smithi033.stdout: 1 440015 7.31 MB/sec execute 167 sec latency 15.060 ms 2024-05-15T14:07:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.640+0000 7fa9bb61d640 1 -- 172.21.15.33:0/4209673844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9b41045e0 msgr2=0x7fa9b410cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.640+0000 7fa9bb61d640 1 --2- 172.21.15.33:0/4209673844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9b41045e0 0x7fa9b410cac0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4009900 tx=0x7fa9a402a480 comp rx=0 tx=0).stop 2024-05-15T14:07:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.640+0000 7fa9bb61d640 1 -- 172.21.15.33:0/4209673844 shutdown_connections 2024-05-15T14:07:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.640+0000 7fa9bb61d640 1 --2- 172.21.15.33:0/4209673844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9b410d090 0x7fa9b410f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.640+0000 7fa9bb61d640 1 --2- 172.21.15.33:0/4209673844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9b41045e0 0x7fa9b410cac0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.640+0000 7fa9bb61d640 1 --2- 172.21.15.33:0/4209673844 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9b4103cc0 0x7fa9b41040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.640+0000 7fa9bb61d640 1 -- 172.21.15.33:0/4209673844 >> 172.21.15.33:0/4209673844 conn(0x7fa9b4078a70 msgr2=0x7fa9b4078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:07:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.640+0000 7fa9bb61d640 1 -- 172.21.15.33:0/4209673844 shutdown_connections 2024-05-15T14:07:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.640+0000 7fa9bb61d640 1 -- 172.21.15.33:0/4209673844 wait complete. 2024-05-15T14:07:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.640+0000 7fa9bb61d640 1 Processor -- start 2024-05-15T14:07:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.641+0000 7fa9bb61d640 1 -- start start 2024-05-15T14:07:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.641+0000 7fa9bb61d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9b4103cc0 0x7fa9b419b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.641+0000 7fa9bb61d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9b41045e0 0x7fa9b419bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.641+0000 7fa9bb61d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9b410d090 0x7fa9b419fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.641+0000 7fa9bb61d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b4112d20 con 0x7fa9b41045e0 2024-05-15T14:07:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.641+0000 7fa9bb61d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b4112ba0 con 0x7fa9b410d090 2024-05-15T14:07:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.641+0000 7fa9bb61d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b4112ea0 con 0x7fa9b4103cc0 2024-05-15T14:07:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.641+0000 7fa9b8bb8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9b41045e0 0x7fa9b419bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.641+0000 7fa9b93b9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9b4103cc0 0x7fa9b419b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.641+0000 7fa9b8bb8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9b41045e0 0x7fa9b419bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58782/0 (socket says 172.21.15.33:58782) 2024-05-15T14:07:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.641+0000 7fa9b8bb8640 1 -- 172.21.15.33:0/4171243874 learned_addr learned my addr 172.21.15.33:0/4171243874 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:07:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.641+0000 7fa9b9bba640 1 --2- 172.21.15.33:0/4171243874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9b410d090 0x7fa9b419fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.641+0000 7fa9b93b9640 1 -- 172.21.15.33:0/4171243874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9b410d090 msgr2=0x7fa9b419fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.641+0000 7fa9b93b9640 1 --2- 172.21.15.33:0/4171243874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9b410d090 0x7fa9b419fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.641+0000 7fa9b93b9640 1 -- 172.21.15.33:0/4171243874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9b41045e0 msgr2=0x7fa9b419bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.641+0000 7fa9b93b9640 1 --2- 172.21.15.33:0/4171243874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9b41045e0 0x7fa9b419bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.641+0000 7fa9b93b9640 1 -- 172.21.15.33:0/4171243874 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b41a0570 con 0x7fa9b4103cc0 2024-05-15T14:07:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.641+0000 7fa9b8bb8640 1 --2- 172.21.15.33:0/4171243874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9b41045e0 0x7fa9b419bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:07:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.641+0000 7fa9b93b9640 1 --2- 172.21.15.33:0/4171243874 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9b4103cc0 0x7fa9b419b580 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa9b000e9f0 tx=0x7fa9b000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:07:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.642+0000 7fa9aa7fc640 1 -- 172.21.15.33:0/4171243874 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b000ccc0 con 0x7fa9b4103cc0 2024-05-15T14:07:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.642+0000 7fa9aa7fc640 1 -- 172.21.15.33:0/4171243874 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9b0005020 con 0x7fa9b4103cc0 2024-05-15T14:07:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.642+0000 7fa9bb61d640 1 -- 172.21.15.33:0/4171243874 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9b41a0860 con 0x7fa9b4103cc0 2024-05-15T14:07:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.642+0000 7fa9aa7fc640 1 -- 172.21.15.33:0/4171243874 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b000ce60 con 0x7fa9b4103cc0 2024-05-15T14:07:12.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.642+0000 7fa9bb61d640 1 -- 172.21.15.33:0/4171243874 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9b41a80f0 con 0x7fa9b4103cc0 2024-05-15T14:07:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.644+0000 7fa9aa7fc640 1 -- 172.21.15.33:0/4171243874 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa9b00051d0 con 0x7fa9b4103cc0 2024-05-15T14:07:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.644+0000 7fa9be40c740 1 Processor -- start 2024-05-15T14:07:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.645+0000 7fa9be40c740 1 -- start start 2024-05-15T14:07:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.645+0000 7fa9be40c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c1136c8f0 0x560c1136ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.645+0000 7fa9be40c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c1136d210 0x560c11377590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.645+0000 7fa9be40c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c11377ad0 0x560c11379ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.645+0000 7fa9be40c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560c112dfc30 con 0x560c11377ad0 2024-05-15T14:07:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.645+0000 7fa9be40c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x560c112cf7d0 con 0x560c1136c8f0 2024-05-15T14:07:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.645+0000 7fa9be40c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x560c112d3810 con 0x560c1136d210 2024-05-15T14:07:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.645+0000 7fa9b9bba640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c1136c8f0 0x560c1136ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.645+0000 7fa9b9bba640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c1136c8f0 0x560c1136ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:58486/0 (socket says 172.21.15.33:58486) 2024-05-15T14:07:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.645+0000 7fa9b9bba640 1 -- 172.21.15.33:0/4106105119 learned_addr learned my addr 172.21.15.33:0/4106105119 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:07:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.645+0000 7fa9b8bb8640 1 --2- 172.21.15.33:0/4106105119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c11377ad0 0x560c11379ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.645+0000 7fa9b93b9640 1 --2- 172.21.15.33:0/4106105119 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c1136d210 0x560c11377590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.645+0000 7fa9aa7fc640 1 --2- 172.21.15.33:0/4171243874 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fa9a00b29d0 0x7fa9a00b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.645+0000 7fa9b9bba640 1 -- 172.21.15.33:0/4106105119 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c1136d210 msgr2=0x560c11377590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.645+0000 7fa9b9bba640 1 --2- 172.21.15.33:0/4106105119 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c1136d210 0x560c11377590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.645+0000 7fa9b9bba640 1 -- 172.21.15.33:0/4106105119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c11377ad0 msgr2=0x560c11379ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.645+0000 7fa9b9bba640 1 --2- 172.21.15.33:0/4106105119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c11377ad0 0x560c11379ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.645+0000 7fa9b9bba640 1 -- 172.21.15.33:0/4106105119 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c10fe4210 con 0x560c1136c8f0 2024-05-15T14:07:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.645+0000 7fa9aa7fc640 1 -- 172.21.15.33:0/4171243874 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8542+0+0 (secure 0 0 0) 0x7fa9b00e24e0 con 0x7fa9b4103cc0 2024-05-15T14:07:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.646+0000 7fa9b9bba640 1 --2- 172.21.15.33:0/4106105119 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c1136c8f0 0x560c1136ccd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac009f40 tx=0x7fa9ac0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd7d2b3d5352ce6e server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:07:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.646+0000 7fa9b9bba640 1 --2- 172.21.15.33:0/4171243874 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fa9a00b29d0 0x7fa9a00b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.646+0000 7fa99f7fe640 1 -- 172.21.15.33:0/4106105119 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac004250 con 0x560c1136c8f0 2024-05-15T14:07:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.646+0000 7fa99f7fe640 1 -- 172.21.15.33:0/4106105119 <== mon.2 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9ac00fe50 con 0x560c1136c8f0 2024-05-15T14:07:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.646+0000 7fa99f7fe640 1 -- 172.21.15.33:0/4106105119 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac01bab0 con 0x560c1136c8f0 2024-05-15T14:07:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.647+0000 7fa9be40c740 1 -- 172.21.15.33:0/4106105119 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c1136c8f0 msgr2=0x560c1136ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.647+0000 7fa9b9bba640 1 --2- 172.21.15.33:0/4171243874 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fa9a00b29d0 0x7fa9a00b4e90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac020000 tx=0x7fa9ac021000 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:07:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.647+0000 7fa9be40c740 1 --2- 172.21.15.33:0/4106105119 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c1136c8f0 0x560c1136ccd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac009f40 tx=0x7fa9ac0118d0 comp rx=0 tx=0).stop 2024-05-15T14:07:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.647+0000 7fa9be40c740 1 -- 172.21.15.33:0/4106105119 shutdown_connections 2024-05-15T14:07:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.647+0000 7fa9be40c740 1 --2- 172.21.15.33:0/4106105119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c11377ad0 0x560c11379ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.647+0000 7fa9be40c740 1 --2- 172.21.15.33:0/4106105119 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c1136d210 0x560c11377590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.647+0000 7fa9be40c740 1 --2- 172.21.15.33:0/4106105119 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c1136c8f0 0x560c1136ccd0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.647+0000 7fa9be40c740 1 -- 172.21.15.33:0/4106105119 >> 172.21.15.33:0/4106105119 conn(0x560c11368e00 msgr2=0x560c113691e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:07:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.647+0000 7fa9be40c740 1 -- 172.21.15.33:0/4106105119 shutdown_connections 2024-05-15T14:07:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.647+0000 7fa9be40c740 1 -- 172.21.15.33:0/4106105119 wait complete. 2024-05-15T14:07:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.647+0000 7fa9be40c740 1 -- start start 2024-05-15T14:07:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.647+0000 7fa9be40c740 1 Processor -- start 2024-05-15T14:07:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.648+0000 7fa9be40c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c1136c8f0 0x560c11377440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.648+0000 7fa9be40c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c1136d210 0x560c11374730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.648+0000 7fa9be40c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c11377ad0 0x560c11374c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.648+0000 7fa9be40c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560c10fb5790 con 0x560c11377ad0 2024-05-15T14:07:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.648+0000 7fa9be40c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x560c1128b890 con 0x560c1136c8f0 2024-05-15T14:07:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.648+0000 7fa9be40c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x560c1137a7a0 con 0x560c1136d210 2024-05-15T14:07:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.648+0000 7fa9be40c740 10 client.0 _ll_register_callbacks cb 0x560c1128ed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:07:12.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.648+0000 7fa9b9bba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c11377ad0 0x560c11374c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.648+0000 7fa9b93b9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c1136c8f0 0x560c11377440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.648+0000 7fa9b8bb8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c1136d210 0x560c11374730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.648+0000 7fa9b93b9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c1136c8f0 0x560c11377440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:58494/0 (socket says 172.21.15.33:58494) 2024-05-15T14:07:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.648+0000 7fa9b93b9640 1 -- 172.21.15.33:0/256480409 learned_addr learned my addr 172.21.15.33:0/256480409 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:07:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.648+0000 7fa9b93b9640 1 -- 172.21.15.33:0/256480409 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c1136d210 msgr2=0x560c11374730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.648+0000 7fa9b93b9640 1 --2- 172.21.15.33:0/256480409 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c1136d210 0x560c11374730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.648+0000 7fa9b93b9640 1 -- 172.21.15.33:0/256480409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c11377ad0 msgr2=0x560c11374c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.648+0000 7fa9b93b9640 1 --2- 172.21.15.33:0/256480409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c11377ad0 0x560c11374c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.648+0000 7fa9b93b9640 1 -- 172.21.15.33:0/256480409 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c110b0620 con 0x560c1136c8f0 2024-05-15T14:07:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.648+0000 7fa9b93b9640 1 --2- 172.21.15.33:0/256480409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c1136c8f0 0x560c11377440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa9b000c6a0 tx=0x7fa9b00a9640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5071ac956c2dbcd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:07:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.648+0000 7fa99cff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-05-15T14:07:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.648+0000 7fa99cff9640 1 -- 172.21.15.33:0/256480409 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b001c070 con 0x560c1136c8f0 2024-05-15T14:07:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.649+0000 7fa99cff9640 1 -- 172.21.15.33:0/256480409 <== mon.2 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8542+0+0 (secure 0 0 0) 0x7fa9b00dfea0 con 0x560c1136c8f0 2024-05-15T14:07:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.649+0000 7fa9be40c740 1 -- 172.21.15.33:0/256480409 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9ac00fe50 con 0x560c1136c8f0 2024-05-15T14:07:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.649+0000 7fa99cff9640 1 -- 172.21.15.33:0/256480409 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa9b00e00a0 con 0x560c1136c8f0 2024-05-15T14:07:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.649+0000 7fa9be40c740 10 client.25703 fetch_fsmap learned FSMap version 30 2024-05-15T14:07:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.649+0000 7fa9be40c740 1 -- 172.21.15.33:0/256480409 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560c11386220 con 0x560c1136c8f0 2024-05-15T14:07:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.649+0000 7fa99cff9640 1 -- 172.21.15.33:0/256480409 <== mon.2 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9ac00fe50 con 0x560c1136c8f0 2024-05-15T14:07:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.649+0000 7fa99cff9640 1 -- 172.21.15.33:0/256480409 <== mon.2 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7fa9b00b2180 con 0x560c1136c8f0 2024-05-15T14:07:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.649+0000 7fa9be40c740 10 client.25703 fetch_fsmap finished waiting for FSMap version 30 2024-05-15T14:07:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.649+0000 7fa9be40c740 10 client.25703 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:07:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.649+0000 7fa9be40c740 20 client.25703 populate_metadata read hostname 'smithi033' 2024-05-15T14:07:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.649+0000 7fa9be40c740 1 --2- 172.21.15.33:0/256480409 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x560c11387690 0x560c11389a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.649+0000 7fa9be40c740 4 client.25703 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:07:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.649+0000 7fa9be40c740 1 -- 172.21.15.33:0/256480409 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c112e1b80 con 0x560c11387690 2024-05-15T14:07:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.649+0000 7fa9b8bb8640 1 --2- 172.21.15.33:0/256480409 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x560c11387690 0x560c11389a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.650+0000 7fa99cff9640 10 client.25703 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:07:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.650+0000 7fa9b8bb8640 1 --2- 172.21.15.33:0/256480409 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x560c11387690 0x560c11389a80 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5063083bea0fd28 server_cookie=f49cf0de04a30738 in_seq=0 out_seq=0 2024-05-15T14:07:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.650+0000 7fa99cff9640 1 -- 172.21.15.33:0/256480409 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560c112e1b80 con 0x560c11387690 2024-05-15T14:07:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.650+0000 7fa99cff9640 10 client.25703 handle_command_reply: tid=0 2024-05-15T14:07:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.651+0000 7fa9be40c740 1 client.25703 shutdown 2024-05-15T14:07:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.651+0000 7fa9be40c740 1 -- 172.21.15.33:0/256480409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c1136c8f0 msgr2=0x560c11377440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.651+0000 7fa9be40c740 1 --2- 172.21.15.33:0/256480409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c1136c8f0 0x560c11377440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa9b000c6a0 tx=0x7fa9b00a9640 comp rx=0 tx=0).stop 2024-05-15T14:07:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.651+0000 7fa9be40c740 1 -- 172.21.15.33:0/256480409 shutdown_connections 2024-05-15T14:07:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.651+0000 7fa9be40c740 1 --2- 172.21.15.33:0/256480409 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x560c11387690 0x560c11389a80 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.651+0000 7fa9be40c740 1 --2- 172.21.15.33:0/256480409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c11377ad0 0x560c11374c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.651+0000 7fa99cff9640 0 client.25703 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:07:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.651+0000 7fa9be40c740 1 --2- 172.21.15.33:0/256480409 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c1136d210 0x560c11374730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.651+0000 7fa9be40c740 1 --2- 172.21.15.33:0/256480409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c1136c8f0 0x560c11377440 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.651+0000 7fa9be40c740 1 -- 172.21.15.33:0/256480409 >> 172.21.15.33:0/256480409 conn(0x560c11368e00 msgr2=0x560c11368100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:07:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.651+0000 7fa9be40c740 1 -- 172.21.15.33:0/256480409 shutdown_connections 2024-05-15T14:07:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.651+0000 7fa9be40c740 1 -- 172.21.15.33:0/256480409 wait complete. 2024-05-15T14:07:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.651+0000 7fa9be40c740 20 client.25703 trim_cache size 0 max 16384 2024-05-15T14:07:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.669+0000 7fa9be40c740 1 Processor -- start 2024-05-15T14:07:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.670+0000 7fa9be40c740 1 -- start start 2024-05-15T14:07:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.670+0000 7fa9be40c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c1136c8f0 0x560c11376f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.670+0000 7fa9be40c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c1136d210 0x560c11377440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.670+0000 7fa9be40c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c11377ad0 0x560c1138f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.670+0000 7fa9be40c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b41b32a0 con 0x560c11377ad0 2024-05-15T14:07:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.670+0000 7fa9be40c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b41b0b40 con 0x560c1136d210 2024-05-15T14:07:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.670+0000 7fa9be40c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b41ae9b0 con 0x560c1136c8f0 2024-05-15T14:07:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.670+0000 7fa9b93b9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c1136c8f0 0x560c11376f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.670+0000 7fa9b93b9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c1136c8f0 0x560c11376f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:48344/0 (socket says 172.21.15.33:48344) 2024-05-15T14:07:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.670+0000 7fa9b93b9640 1 -- 172.21.15.33:0/646676680 learned_addr learned my addr 172.21.15.33:0/646676680 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:07:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.670+0000 7fa9b9bba640 1 --2- 172.21.15.33:0/646676680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c11377ad0 0x560c1138f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.670+0000 7fa9b8bb8640 1 --2- 172.21.15.33:0/646676680 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c1136d210 0x560c11377440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.670+0000 7fa9b93b9640 1 -- 172.21.15.33:0/646676680 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c1136d210 msgr2=0x560c11377440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.670+0000 7fa9b93b9640 1 --2- 172.21.15.33:0/646676680 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c1136d210 0x560c11377440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.670+0000 7fa9b93b9640 1 -- 172.21.15.33:0/646676680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c11377ad0 msgr2=0x560c1138f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.670+0000 7fa9b93b9640 1 --2- 172.21.15.33:0/646676680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c11377ad0 0x560c1138f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.670+0000 7fa9b93b9640 1 -- 172.21.15.33:0/646676680 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c11368c30 con 0x560c1136c8f0 2024-05-15T14:07:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.671+0000 7fa9b93b9640 1 --2- 172.21.15.33:0/646676680 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c1136c8f0 0x560c11376f00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa9b00a94b0 tx=0x7fa9b0009f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54f205ede6ffde87 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:07:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.671+0000 7fa99effd640 1 -- 172.21.15.33:0/646676680 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b00a9eb0 con 0x560c1136c8f0 2024-05-15T14:07:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.671+0000 7fa99effd640 1 -- 172.21.15.33:0/646676680 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9b00a97b0 con 0x560c1136c8f0 2024-05-15T14:07:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.671+0000 7fa99effd640 1 -- 172.21.15.33:0/646676680 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b00dfea0 con 0x560c1136c8f0 2024-05-15T14:07:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.671+0000 7fa9be40c740 1 -- 172.21.15.33:0/646676680 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c1136c8f0 msgr2=0x560c11376f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.672+0000 7fa9be40c740 1 --2- 172.21.15.33:0/646676680 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c1136c8f0 0x560c11376f00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa9b00a94b0 tx=0x7fa9b0009f00 comp rx=0 tx=0).stop 2024-05-15T14:07:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.672+0000 7fa9be40c740 1 -- 172.21.15.33:0/646676680 shutdown_connections 2024-05-15T14:07:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.672+0000 7fa9be40c740 1 --2- 172.21.15.33:0/646676680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c11377ad0 0x560c1138f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.672+0000 7fa9be40c740 1 --2- 172.21.15.33:0/646676680 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c1136d210 0x560c11377440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.672+0000 7fa9be40c740 1 --2- 172.21.15.33:0/646676680 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c1136c8f0 0x560c11376f00 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.672+0000 7fa9be40c740 1 -- 172.21.15.33:0/646676680 >> 172.21.15.33:0/646676680 conn(0x560c11368e00 msgr2=0x560c113767f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:07:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.672+0000 7fa9be40c740 1 -- 172.21.15.33:0/646676680 shutdown_connections 2024-05-15T14:07:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.672+0000 7fa9be40c740 1 -- 172.21.15.33:0/646676680 wait complete. 2024-05-15T14:07:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.672+0000 7fa9be40c740 1 -- start start 2024-05-15T14:07:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.672+0000 7fa9be40c740 1 Processor -- start 2024-05-15T14:07:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.672+0000 7fa9be40c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c1136c8f0 0x560c1137d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.672+0000 7fa9be40c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c1136d210 0x560c1137de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.672+0000 7fa9be40c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c11377ad0 0x560c113f0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.672+0000 7fa9be40c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b41af000 con 0x560c11377ad0 2024-05-15T14:07:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.672+0000 7fa9be40c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b41b0470 con 0x560c1136d210 2024-05-15T14:07:12.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.673+0000 7fa9be40c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b41af6d0 con 0x560c1136c8f0 2024-05-15T14:07:12.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.673+0000 7fa9be40c740 10 client.0 _ll_register_callbacks cb 0x7fa9b00a5670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:07:12.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.673+0000 7fa9b8bb8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c1136d210 0x560c1137de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:12.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.673+0000 7fa9b8bb8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c1136d210 0x560c1137de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:58518/0 (socket says 172.21.15.33:58518) 2024-05-15T14:07:12.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.673+0000 7fa9b9bba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c11377ad0 0x560c113f0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:12.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.673+0000 7fa9b8bb8640 1 -- 172.21.15.33:0/1221789123 learned_addr learned my addr 172.21.15.33:0/1221789123 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:07:12.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.673+0000 7fa9b93b9640 1 --2- 172.21.15.33:0/1221789123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c1136c8f0 0x560c1137d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:12.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.673+0000 7fa9b8bb8640 1 -- 172.21.15.33:0/1221789123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c1136c8f0 msgr2=0x560c1137d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:12.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.673+0000 7fa9b8bb8640 1 --2- 172.21.15.33:0/1221789123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c1136c8f0 0x560c1137d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:12.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.673+0000 7fa9b8bb8640 1 -- 172.21.15.33:0/1221789123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c11377ad0 msgr2=0x560c113f0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:12.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.673+0000 7fa9b8bb8640 1 --2- 172.21.15.33:0/1221789123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c11377ad0 0x560c113f0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:12.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.673+0000 7fa9b8bb8640 1 -- 172.21.15.33:0/1221789123 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c1137e6d0 con 0x560c1136d210 2024-05-15T14:07:12.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.673+0000 7fa9b8bb8640 1 --2- 172.21.15.33:0/1221789123 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c1136d210 0x560c1137de30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4004650 tx=0x7fa9a4003bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb95327411f60760 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:07:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.673+0000 7fa99cff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-05-15T14:07:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.673+0000 7fa99cff9640 1 -- 172.21.15.33:0/1221789123 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a404a070 con 0x560c1136d210 2024-05-15T14:07:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.673+0000 7fa99cff9640 1 -- 172.21.15.33:0/1221789123 <== mon.2 v2:172.21.15.52:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8542+0+0 (secure 0 0 0) 0x7fa9a4005220 con 0x560c1136d210 2024-05-15T14:07:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.673+0000 7fa9be40c740 1 -- 172.21.15.33:0/1221789123 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9b00a97b0 con 0x560c1136d210 2024-05-15T14:07:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.674+0000 7fa99cff9640 1 -- 172.21.15.33:0/1221789123 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa9a4035040 con 0x560c1136d210 2024-05-15T14:07:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.674+0000 7fa9be40c740 10 client.25709 fetch_fsmap learned FSMap version 30 2024-05-15T14:07:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.674+0000 7fa9be40c740 1 -- 172.21.15.33:0/1221789123 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560c113f2600 con 0x560c1136d210 2024-05-15T14:07:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.674+0000 7fa99cff9640 1 -- 172.21.15.33:0/1221789123 <== mon.2 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9b00a97b0 con 0x560c1136d210 2024-05-15T14:07:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.674+0000 7fa99cff9640 1 -- 172.21.15.33:0/1221789123 <== mon.2 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7fa9a4046310 con 0x560c1136d210 2024-05-15T14:07:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.674+0000 7fa9be40c740 10 client.25709 fetch_fsmap finished waiting for FSMap version 30 2024-05-15T14:07:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.674+0000 7fa9be40c740 10 client.25709 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:07:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.674+0000 7fa9be40c740 20 client.25709 populate_metadata read hostname 'smithi033' 2024-05-15T14:07:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.674+0000 7fa9be40c740 1 --2- 172.21.15.33:0/1221789123 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x560c113f2db0 0x560c113f51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.674+0000 7fa9be40c740 4 client.25709 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:07:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.674+0000 7fa9be40c740 1 -- 172.21.15.33:0/1221789123 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560c112db360 con 0x560c113f2db0 2024-05-15T14:07:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.674+0000 7fa9b93b9640 1 --2- 172.21.15.33:0/1221789123 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x560c113f2db0 0x560c113f51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.675+0000 7fa99cff9640 10 client.25709 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:07:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.675+0000 7fa9b93b9640 1 --2- 172.21.15.33:0/1221789123 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x560c113f2db0 0x560c113f51d0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3b1efb67f1d7a5c server_cookie=6564234f8d08a79 in_seq=0 out_seq=0 2024-05-15T14:07:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.737+0000 7fa99cff9640 1 -- 172.21.15.33:0/1221789123 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560c112db360 con 0x560c113f2db0 2024-05-15T14:07:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.737+0000 7fa99cff9640 10 client.25709 handle_command_reply: tid=0 2024-05-15T14:07:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.737+0000 7fa9be40c740 1 client.25709 shutdown 2024-05-15T14:07:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.737+0000 7fa9be40c740 1 -- 172.21.15.33:0/1221789123 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c1136d210 msgr2=0x560c1137de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.737+0000 7fa9be40c740 1 --2- 172.21.15.33:0/1221789123 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c1136d210 0x560c1137de30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4004650 tx=0x7fa9a4003bd0 comp rx=0 tx=0).stop 2024-05-15T14:07:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.737+0000 7fa9be40c740 1 -- 172.21.15.33:0/1221789123 shutdown_connections 2024-05-15T14:07:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.737+0000 7fa9be40c740 1 --2- 172.21.15.33:0/1221789123 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x560c113f2db0 0x560c113f51d0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.737+0000 7fa9be40c740 1 --2- 172.21.15.33:0/1221789123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c11377ad0 0x560c113f0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.737+0000 7fa9be40c740 1 --2- 172.21.15.33:0/1221789123 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560c1136d210 0x560c1137de30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.737+0000 7fa99cff9640 0 client.25709 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:07:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.737+0000 7fa9be40c740 1 --2- 172.21.15.33:0/1221789123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560c1136c8f0 0x560c1137d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.737+0000 7fa9be40c740 1 -- 172.21.15.33:0/1221789123 >> 172.21.15.33:0/1221789123 conn(0x560c11368e00 msgr2=0x560c11376f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:07:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.737+0000 7fa9be40c740 1 -- 172.21.15.33:0/1221789123 shutdown_connections 2024-05-15T14:07:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.737+0000 7fa9be40c740 1 -- 172.21.15.33:0/1221789123 wait complete. 2024-05-15T14:07:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.737+0000 7fa9be40c740 20 client.25709 trim_cache size 0 max 16384 2024-05-15T14:07:12.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:07:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:07:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:07:12.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.738+0000 7fa9bb61d640 1 -- 172.21.15.33:0/4171243874 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fa9a00b29d0 msgr2=0x7fa9a00b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.738+0000 7fa9bb61d640 1 --2- 172.21.15.33:0/4171243874 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fa9a00b29d0 0x7fa9a00b4e90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac020000 tx=0x7fa9ac021000 comp rx=0 tx=0).stop 2024-05-15T14:07:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.738+0000 7fa9bb61d640 1 -- 172.21.15.33:0/4171243874 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9b4103cc0 msgr2=0x7fa9b419b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.738+0000 7fa9bb61d640 1 --2- 172.21.15.33:0/4171243874 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9b4103cc0 0x7fa9b419b580 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa9b000e9f0 tx=0x7fa9b000edf0 comp rx=0 tx=0).stop 2024-05-15T14:07:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.738+0000 7fa9bb61d640 1 -- 172.21.15.33:0/4171243874 shutdown_connections 2024-05-15T14:07:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.738+0000 7fa9bb61d640 1 --2- 172.21.15.33:0/4171243874 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fa9a00b29d0 0x7fa9a00b4e90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.738+0000 7fa9bb61d640 1 --2- 172.21.15.33:0/4171243874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9b410d090 0x7fa9b419fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.738+0000 7fa9bb61d640 1 --2- 172.21.15.33:0/4171243874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9b41045e0 0x7fa9b419bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.738+0000 7fa9bb61d640 1 --2- 172.21.15.33:0/4171243874 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa9b4103cc0 0x7fa9b419b580 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.738+0000 7fa9bb61d640 1 -- 172.21.15.33:0/4171243874 >> 172.21.15.33:0/4171243874 conn(0x7fa9b4078a70 msgr2=0x7fa9b41090b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:07:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.738+0000 7fa9bb61d640 1 -- 172.21.15.33:0/4171243874 shutdown_connections 2024-05-15T14:07:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:12.738+0000 7fa9bb61d640 1 -- 172.21.15.33:0/4171243874 wait complete. 2024-05-15T14:07:12.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:12 smithi052 ceph-mon[36190]: pgmap v211: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 160 op/s 2024-05-15T14:07:12.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:12 smithi081 ceph-mon[36434]: pgmap v211: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 160 op/s 2024-05-15T14:07:12.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:12 smithi033 ceph-mon[31650]: pgmap v211: 97 pgs: 97 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 160 op/s 2024-05-15T14:07:13.063 INFO:tasks.workunit.client.0.smithi033.stdout: 1 441240 7.30 MB/sec execute 168 sec latency 55.353 ms 2024-05-15T14:07:14.063 INFO:tasks.workunit.client.0.smithi033.stdout: 1 443484 7.33 MB/sec execute 169 sec latency 19.937 ms 2024-05-15T14:07:14.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:14 smithi052 ceph-mon[36190]: pgmap v212: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 293 op/s 2024-05-15T14:07:14.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:14 smithi081 ceph-mon[36434]: pgmap v212: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 293 op/s 2024-05-15T14:07:14.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:14 smithi033 ceph-mon[31650]: pgmap v212: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 293 op/s 2024-05-15T14:07:15.064 INFO:tasks.workunit.client.0.smithi033.stdout: 1 444625 7.31 MB/sec execute 170 sec latency 51.655 ms 2024-05-15T14:07:16.064 INFO:tasks.workunit.client.0.smithi033.stdout: 1 447093 7.36 MB/sec execute 171 sec latency 45.678 ms 2024-05-15T14:07:16.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:16 smithi052 ceph-mon[36190]: pgmap v213: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 259 op/s 2024-05-15T14:07:16.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:16 smithi081 ceph-mon[36434]: pgmap v213: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 259 op/s 2024-05-15T14:07:16.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:16 smithi033 ceph-mon[31650]: pgmap v213: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 259 op/s 2024-05-15T14:07:17.063 INFO:tasks.workunit.client.0.smithi033.stdout: 1 448452 7.36 MB/sec execute 172 sec latency 20.945 ms 2024-05-15T14:07:18.064 INFO:tasks.workunit.client.0.smithi033.stdout: 1 450708 7.39 MB/sec execute 173 sec latency 12.825 ms 2024-05-15T14:07:18.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:18 smithi052 ceph-mon[36190]: pgmap v214: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 312 op/s 2024-05-15T14:07:18.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:18 smithi081 ceph-mon[36434]: pgmap v214: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 312 op/s 2024-05-15T14:07:18.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:18 smithi033 ceph-mon[31650]: pgmap v214: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 312 op/s 2024-05-15T14:07:19.064 INFO:tasks.workunit.client.0.smithi033.stdout: 1 452025 7.38 MB/sec execute 174 sec latency 16.339 ms 2024-05-15T14:07:20.064 INFO:tasks.workunit.client.0.smithi033.stdout: 1 454148 7.41 MB/sec execute 175 sec latency 147.347 ms 2024-05-15T14:07:20.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:20 smithi052 ceph-mon[36190]: pgmap v215: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 312 op/s 2024-05-15T14:07:20.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:20 smithi081 ceph-mon[36434]: pgmap v215: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 312 op/s 2024-05-15T14:07:20.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:20 smithi033 ceph-mon[31650]: pgmap v215: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 312 op/s 2024-05-15T14:07:21.064 INFO:tasks.workunit.client.0.smithi033.stdout: 1 455462 7.41 MB/sec execute 176 sec latency 44.797 ms 2024-05-15T14:07:22.064 INFO:tasks.workunit.client.0.smithi033.stdout: 1 457906 7.44 MB/sec execute 177 sec latency 13.194 ms 2024-05-15T14:07:22.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:22 smithi052 ceph-mon[36190]: pgmap v216: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 289 op/s 2024-05-15T14:07:22.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:22 smithi081 ceph-mon[36434]: pgmap v216: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 289 op/s 2024-05-15T14:07:22.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:22 smithi033 ceph-mon[31650]: pgmap v216: 97 pgs: 97 active+clean; 122 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 289 op/s 2024-05-15T14:07:23.064 INFO:tasks.workunit.client.0.smithi033.stdout: 1 458685 7.47 MB/sec execute 178 sec latency 184.058 ms 2024-05-15T14:07:24.064 INFO:tasks.workunit.client.0.smithi033.stdout: 1 458685 7.43 MB/sec execute 179 sec latency 1184.152 ms 2024-05-15T14:07:24.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:24 smithi052 ceph-mon[36190]: pgmap v217: 97 pgs: 97 active+clean; 128 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 369 op/s 2024-05-15T14:07:24.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:24 smithi081 ceph-mon[36434]: pgmap v217: 97 pgs: 97 active+clean; 128 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 369 op/s 2024-05-15T14:07:24.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:24 smithi033 ceph-mon[31650]: pgmap v217: 97 pgs: 97 active+clean; 128 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 369 op/s 2024-05-15T14:07:25.064 INFO:tasks.workunit.client.0.smithi033.stdout: 1 458685 7.39 MB/sec execute 180 sec latency 2184.228 ms 2024-05-15T14:07:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.839+0000 7fc8e0df0640 1 -- 172.21.15.33:0/3171774287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8dc101930 msgr2=0x7fc8dc101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.839+0000 7fc8e0df0640 1 --2- 172.21.15.33:0/3171774287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8dc101930 0x7fc8dc101db0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009930 tx=0x7fc8d002a420 comp rx=0 tx=0).stop 2024-05-15T14:07:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.839+0000 7fc8e0df0640 1 -- 172.21.15.33:0/3171774287 shutdown_connections 2024-05-15T14:07:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.839+0000 7fc8e0df0640 1 --2- 172.21.15.33:0/3171774287 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8dc1022f0 0x7fc8dc10e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.839+0000 7fc8e0df0640 1 --2- 172.21.15.33:0/3171774287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8dc101930 0x7fc8dc101db0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.839+0000 7fc8e0df0640 1 --2- 172.21.15.33:0/3171774287 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8dc107940 0x7fc8dc107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.839+0000 7fc8e0df0640 1 -- 172.21.15.33:0/3171774287 >> 172.21.15.33:0/3171774287 conn(0x7fc8dc078a70 msgr2=0x7fc8dc078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:07:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.839+0000 7fc8e0df0640 1 -- 172.21.15.33:0/3171774287 shutdown_connections 2024-05-15T14:07:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.839+0000 7fc8e0df0640 1 -- 172.21.15.33:0/3171774287 wait complete. 2024-05-15T14:07:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.840+0000 7fc8e0df0640 1 Processor -- start 2024-05-15T14:07:25.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.840+0000 7fc8e0df0640 1 -- start start 2024-05-15T14:07:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.840+0000 7fc8e0df0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8dc101930 0x7fc8dc1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.840+0000 7fc8e0df0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8dc1022f0 0x7fc8dc1997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.840+0000 7fc8e0df0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8dc107940 0x7fc8dc19db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.840+0000 7fc8e0df0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc111c00 con 0x7fc8dc107940 2024-05-15T14:07:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.840+0000 7fc8e0df0640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc111a80 con 0x7fc8dc101930 2024-05-15T14:07:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.840+0000 7fc8e0df0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc111d80 con 0x7fc8dc1022f0 2024-05-15T14:07:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.840+0000 7fc8dad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8dc107940 0x7fc8dc19db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.840+0000 7fc8d9d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8dc1022f0 0x7fc8dc1997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.840+0000 7fc8dad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8dc107940 0x7fc8dc19db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49116/0 (socket says 172.21.15.33:49116) 2024-05-15T14:07:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.840+0000 7fc8dad9d640 1 -- 172.21.15.33:0/3322900549 learned_addr learned my addr 172.21.15.33:0/3322900549 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:07:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.840+0000 7fc8da59c640 1 --2- 172.21.15.33:0/3322900549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8dc101930 0x7fc8dc1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.840+0000 7fc8dad9d640 1 -- 172.21.15.33:0/3322900549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8dc1022f0 msgr2=0x7fc8dc1997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.840+0000 7fc8dad9d640 1 --2- 172.21.15.33:0/3322900549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8dc1022f0 0x7fc8dc1997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.840+0000 7fc8dad9d640 1 -- 172.21.15.33:0/3322900549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8dc101930 msgr2=0x7fc8dc1992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:25.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.840+0000 7fc8dad9d640 1 --2- 172.21.15.33:0/3322900549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8dc101930 0x7fc8dc1992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.840+0000 7fc8dad9d640 1 -- 172.21.15.33:0/3322900549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8dc19e240 con 0x7fc8dc107940 2024-05-15T14:07:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.840+0000 7fc8dad9d640 1 --2- 172.21.15.33:0/3322900549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8dc107940 0x7fc8dc19db10 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc8d400d750 tx=0x7fc8d400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:07:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.841+0000 7fc8cb7fe640 1 -- 172.21.15.33:0/3322900549 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4017070 con 0x7fc8dc107940 2024-05-15T14:07:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.841+0000 7fc8cb7fe640 1 -- 172.21.15.33:0/3322900549 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8d40054d0 con 0x7fc8dc107940 2024-05-15T14:07:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.841+0000 7fc8e0df0640 1 -- 172.21.15.33:0/3322900549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8dc19e530 con 0x7fc8dc107940 2024-05-15T14:07:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.841+0000 7fc8cb7fe640 1 -- 172.21.15.33:0/3322900549 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4004c60 con 0x7fc8dc107940 2024-05-15T14:07:25.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.841+0000 7fc8e0df0640 1 -- 172.21.15.33:0/3322900549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8dc1a5dc0 con 0x7fc8dc107940 2024-05-15T14:07:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.843+0000 7fc8e3be6740 1 Processor -- start 2024-05-15T14:07:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.843+0000 7fc8cb7fe640 1 -- 172.21.15.33:0/3322900549 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc8d4009bd0 con 0x7fc8dc107940 2024-05-15T14:07:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.844+0000 7fc8e3be6740 1 -- start start 2024-05-15T14:07:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.844+0000 7fc8e3be6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f53eb67d0 0x558f53eb6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.844+0000 7fc8e3be6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f53eb70f0 0x558f53ec59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.844+0000 7fc8e3be6740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f53ec5ee0 0x558f53ec82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.844+0000 7fc8e3be6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558f53e29c30 con 0x558f53eb67d0 2024-05-15T14:07:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.844+0000 7fc8e3be6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558f53e197d0 con 0x558f53eb70f0 2024-05-15T14:07:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.844+0000 7fc8e3be6740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x558f53e1d810 con 0x558f53ec5ee0 2024-05-15T14:07:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.844+0000 7fc8d9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f53eb67d0 0x558f53eb6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.844+0000 7fc8d9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f53eb67d0 0x558f53eb6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49130/0 (socket says 172.21.15.33:49130) 2024-05-15T14:07:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.844+0000 7fc8d9d9b640 1 -- 172.21.15.33:0/3951408918 learned_addr learned my addr 172.21.15.33:0/3951408918 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:07:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.844+0000 7fc8da59c640 1 --2- 172.21.15.33:0/3951408918 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f53eb70f0 0x558f53ec59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.844+0000 7fc8d9d9b640 1 --2- 172.21.15.33:0/3951408918 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f53ec5ee0 0x558f53ec82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.844+0000 7fc8d9d9b640 1 -- 172.21.15.33:0/3951408918 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f53ec5ee0 msgr2=0x558f53ec82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.844+0000 7fc8d9d9b640 1 --2- 172.21.15.33:0/3951408918 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f53ec5ee0 0x558f53ec82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.844+0000 7fc8d9d9b640 1 -- 172.21.15.33:0/3951408918 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f53eb70f0 msgr2=0x558f53ec59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.844+0000 7fc8d9d9b640 1 --2- 172.21.15.33:0/3951408918 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f53eb70f0 0x558f53ec59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.844+0000 7fc8d9d9b640 1 -- 172.21.15.33:0/3951408918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f53b2e210 con 0x558f53eb67d0 2024-05-15T14:07:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.844+0000 7fc8cb7fe640 1 --2- 172.21.15.33:0/3322900549 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc8c40b2950 0x7fc8c40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.845+0000 7fc8d9d9b640 1 --2- 172.21.15.33:0/3951408918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f53eb67d0 0x558f53eb6bb0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009930 tx=0x7fc8d0005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad989b1545b7541 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:07:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.845+0000 7fc8da59c640 1 --2- 172.21.15.33:0/3322900549 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc8c40b2950 0x7fc8c40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.845+0000 7fc8cb7fe640 1 -- 172.21.15.33:0/3322900549 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8542+0+0 (secure 0 0 0) 0x7fc8d40e1ff0 con 0x7fc8dc107940 2024-05-15T14:07:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.845+0000 7fc8c8ff9640 1 -- 172.21.15.33:0/3951408918 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d0044030 con 0x558f53eb67d0 2024-05-15T14:07:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.845+0000 7fc8c8ff9640 1 -- 172.21.15.33:0/3951408918 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8d0031660 con 0x558f53eb67d0 2024-05-15T14:07:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.845+0000 7fc8c8ff9640 1 -- 172.21.15.33:0/3951408918 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d0035040 con 0x558f53eb67d0 2024-05-15T14:07:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.846+0000 7fc8e3be6740 1 -- 172.21.15.33:0/3951408918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f53eb67d0 msgr2=0x558f53eb6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:25.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.846+0000 7fc8e3be6740 1 --2- 172.21.15.33:0/3951408918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f53eb67d0 0x558f53eb6bb0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009930 tx=0x7fc8d0005c00 comp rx=0 tx=0).stop 2024-05-15T14:07:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.846+0000 7fc8da59c640 1 --2- 172.21.15.33:0/3322900549 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc8c40b2950 0x7fc8c40b4e10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc005fd0 tx=0x7fc8cc009620 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:07:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.846+0000 7fc8e3be6740 1 -- 172.21.15.33:0/3951408918 shutdown_connections 2024-05-15T14:07:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.846+0000 7fc8e3be6740 1 --2- 172.21.15.33:0/3951408918 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f53ec5ee0 0x558f53ec82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.846+0000 7fc8e3be6740 1 --2- 172.21.15.33:0/3951408918 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f53eb70f0 0x558f53ec59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.846+0000 7fc8e3be6740 1 --2- 172.21.15.33:0/3951408918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f53eb67d0 0x558f53eb6bb0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.846+0000 7fc8e3be6740 1 -- 172.21.15.33:0/3951408918 >> 172.21.15.33:0/3951408918 conn(0x558f53eb2e00 msgr2=0x558f53eb31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:07:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.846+0000 7fc8e3be6740 1 -- 172.21.15.33:0/3951408918 shutdown_connections 2024-05-15T14:07:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.846+0000 7fc8e3be6740 1 -- 172.21.15.33:0/3951408918 wait complete. 2024-05-15T14:07:25.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.846+0000 7fc8e3be6740 1 -- start start 2024-05-15T14:07:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.846+0000 7fc8e3be6740 1 Processor -- start 2024-05-15T14:07:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.846+0000 7fc8e3be6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f53eb67d0 0x558f53ec5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.846+0000 7fc8e3be6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f53eb70f0 0x558f53ec2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.846+0000 7fc8e3be6740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f53ec5ee0 0x558f53ec3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.846+0000 7fc8e3be6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558f53aff790 con 0x558f53eb67d0 2024-05-15T14:07:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.846+0000 7fc8e3be6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558f53dd5890 con 0x558f53eb70f0 2024-05-15T14:07:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.846+0000 7fc8e3be6740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x558f53ec8b50 con 0x558f53ec5ee0 2024-05-15T14:07:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.846+0000 7fc8e3be6740 10 client.0 _ll_register_callbacks cb 0x558f53dd8d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:07:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.847+0000 7fc8da59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f53eb67d0 0x558f53ec5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.847+0000 7fc8da59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f53eb67d0 0x558f53ec5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49142/0 (socket says 172.21.15.33:49142) 2024-05-15T14:07:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.847+0000 7fc8da59c640 1 -- 172.21.15.33:0/591927669 learned_addr learned my addr 172.21.15.33:0/591927669 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:07:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.847+0000 7fc8d9d9b640 1 --2- 172.21.15.33:0/591927669 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f53eb70f0 0x558f53ec2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.847+0000 7fc8dad9d640 1 --2- 172.21.15.33:0/591927669 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f53ec5ee0 0x558f53ec3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.847+0000 7fc8da59c640 1 -- 172.21.15.33:0/591927669 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f53ec5ee0 msgr2=0x558f53ec3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.847+0000 7fc8da59c640 1 --2- 172.21.15.33:0/591927669 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f53ec5ee0 0x558f53ec3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.847+0000 7fc8da59c640 1 -- 172.21.15.33:0/591927669 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f53eb70f0 msgr2=0x558f53ec2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.847+0000 7fc8da59c640 1 --2- 172.21.15.33:0/591927669 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f53eb70f0 0x558f53ec2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:25.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.847+0000 7fc8da59c640 1 -- 172.21.15.33:0/591927669 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f53bfa620 con 0x558f53eb67d0 2024-05-15T14:07:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.847+0000 7fc8dad9d640 1 --2- 172.21.15.33:0/591927669 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f53ec5ee0 0x558f53ec3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:07:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.847+0000 7fc8da59c640 1 --2- 172.21.15.33:0/591927669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f53eb67d0 0x558f53ec5820 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc0095f0 tx=0x7fc8cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7490ba6ec13e70e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:07:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.847+0000 7fc8c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:07:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.847+0000 7fc8c1ffb640 1 -- 172.21.15.33:0/591927669 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc01f830 con 0x558f53eb67d0 2024-05-15T14:07:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.847+0000 7fc8c1ffb640 1 -- 172.21.15.33:0/591927669 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8542+0+0 (secure 0 0 0) 0x7fc8cc01f9d0 con 0x558f53eb67d0 2024-05-15T14:07:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.847+0000 7fc8e3be6740 1 -- 172.21.15.33:0/591927669 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8d0031660 con 0x558f53eb67d0 2024-05-15T14:07:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.847+0000 7fc8c1ffb640 1 -- 172.21.15.33:0/591927669 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc8cc01fc10 con 0x558f53eb67d0 2024-05-15T14:07:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.847+0000 7fc8e3be6740 10 client.16197 fetch_fsmap learned FSMap version 30 2024-05-15T14:07:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.847+0000 7fc8e3be6740 1 -- 172.21.15.33:0/591927669 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558f53ed4690 con 0x558f53eb67d0 2024-05-15T14:07:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.848+0000 7fc8c1ffb640 1 -- 172.21.15.33:0/591927669 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8d0031660 con 0x558f53eb67d0 2024-05-15T14:07:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.848+0000 7fc8c1ffb640 1 -- 172.21.15.33:0/591927669 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7fc8cc00f3c0 con 0x558f53eb67d0 2024-05-15T14:07:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.848+0000 7fc8e3be6740 10 client.16197 fetch_fsmap finished waiting for FSMap version 30 2024-05-15T14:07:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.848+0000 7fc8e3be6740 10 client.16197 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:07:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.848+0000 7fc8e3be6740 20 client.16197 populate_metadata read hostname 'smithi033' 2024-05-15T14:07:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.848+0000 7fc8e3be6740 1 --2- 172.21.15.33:0/591927669 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558f53ed5b50 0x558f53ed7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.848+0000 7fc8e3be6740 4 client.16197 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:07:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.848+0000 7fc8e3be6740 1 -- 172.21.15.33:0/591927669 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f53e2bb80 con 0x558f53ed5b50 2024-05-15T14:07:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.848+0000 7fc8d9d9b640 1 --2- 172.21.15.33:0/591927669 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558f53ed5b50 0x558f53ed7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.848+0000 7fc8d9d9b640 1 --2- 172.21.15.33:0/591927669 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558f53ed5b50 0x558f53ed7f40 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae707207f6ddce6f server_cookie=6bfa6f205d46087 in_seq=0 out_seq=0 2024-05-15T14:07:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.848+0000 7fc8c1ffb640 10 client.16197 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:07:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.849+0000 7fc8c1ffb640 1 -- 172.21.15.33:0/591927669 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558f53e2bb80 con 0x558f53ed5b50 2024-05-15T14:07:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.849+0000 7fc8c1ffb640 10 client.16197 handle_command_reply: tid=0 2024-05-15T14:07:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.849+0000 7fc8e3be6740 1 client.16197 shutdown 2024-05-15T14:07:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.849+0000 7fc8e3be6740 1 -- 172.21.15.33:0/591927669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f53eb67d0 msgr2=0x558f53ec5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.849+0000 7fc8e3be6740 1 --2- 172.21.15.33:0/591927669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f53eb67d0 0x558f53ec5820 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc0095f0 tx=0x7fc8cc004a40 comp rx=0 tx=0).stop 2024-05-15T14:07:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.849+0000 7fc8e3be6740 1 -- 172.21.15.33:0/591927669 shutdown_connections 2024-05-15T14:07:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.849+0000 7fc8e3be6740 1 --2- 172.21.15.33:0/591927669 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558f53ed5b50 0x558f53ed7f40 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.849+0000 7fc8e3be6740 1 --2- 172.21.15.33:0/591927669 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f53ec5ee0 0x558f53ec3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.849+0000 7fc8e3be6740 1 --2- 172.21.15.33:0/591927669 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f53eb70f0 0x558f53ec2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.849+0000 7fc8c1ffb640 0 client.16197 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:07:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.849+0000 7fc8e3be6740 1 --2- 172.21.15.33:0/591927669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f53eb67d0 0x558f53ec5820 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.849+0000 7fc8e3be6740 1 -- 172.21.15.33:0/591927669 >> 172.21.15.33:0/591927669 conn(0x558f53eb2e00 msgr2=0x558f53eb8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:07:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.849+0000 7fc8e3be6740 1 -- 172.21.15.33:0/591927669 shutdown_connections 2024-05-15T14:07:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.849+0000 7fc8e3be6740 1 -- 172.21.15.33:0/591927669 wait complete. 2024-05-15T14:07:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.849+0000 7fc8e3be6740 20 client.16197 trim_cache size 0 max 16384 2024-05-15T14:07:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.865+0000 7fc8e3be6740 1 Processor -- start 2024-05-15T14:07:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.865+0000 7fc8e3be6740 1 -- start start 2024-05-15T14:07:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.865+0000 7fc8e3be6740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f53eb67d0 0x558f53ebe1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.865+0000 7fc8e3be6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f53eb70f0 0x558f53ec5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.865+0000 7fc8e3be6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f53ec5ee0 0x558f53eb89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.865+0000 7fc8e3be6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc1aeda0 con 0x558f53eb70f0 2024-05-15T14:07:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.865+0000 7fc8e3be6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc1ac640 con 0x558f53ec5ee0 2024-05-15T14:07:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.865+0000 7fc8e3be6740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc1aa4d0 con 0x558f53eb67d0 2024-05-15T14:07:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.865+0000 7fc8d9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f53eb70f0 0x558f53ec5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.865+0000 7fc8d9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f53eb70f0 0x558f53ec5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49148/0 (socket says 172.21.15.33:49148) 2024-05-15T14:07:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.865+0000 7fc8d9d9b640 1 -- 172.21.15.33:0/3417588871 learned_addr learned my addr 172.21.15.33:0/3417588871 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:07:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.865+0000 7fc8da59c640 1 --2- 172.21.15.33:0/3417588871 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f53eb67d0 0x558f53ebe1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.865+0000 7fc8d9d9b640 1 -- 172.21.15.33:0/3417588871 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f53eb67d0 msgr2=0x558f53ebe1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.865+0000 7fc8d9d9b640 1 --2- 172.21.15.33:0/3417588871 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f53eb67d0 0x558f53ebe1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.865+0000 7fc8dad9d640 1 --2- 172.21.15.33:0/3417588871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f53ec5ee0 0x558f53eb89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.865+0000 7fc8d9d9b640 1 -- 172.21.15.33:0/3417588871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f53ec5ee0 msgr2=0x558f53eb89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.865+0000 7fc8d9d9b640 1 --2- 172.21.15.33:0/3417588871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f53ec5ee0 0x558f53eb89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.865+0000 7fc8d9d9b640 1 -- 172.21.15.33:0/3417588871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f53eb6f20 con 0x558f53eb70f0 2024-05-15T14:07:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.865+0000 7fc8dad9d640 1 --2- 172.21.15.33:0/3417588871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f53ec5ee0 0x558f53eb89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:07:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.866+0000 7fc8d9d9b640 1 --2- 172.21.15.33:0/3417588871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f53eb70f0 0x558f53ec5350 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009900 tx=0x7fc8d0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b7141bd03626e16 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:07:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.866+0000 7fc8c3fff640 1 -- 172.21.15.33:0/3417588871 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d0035040 con 0x558f53eb70f0 2024-05-15T14:07:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.866+0000 7fc8c3fff640 1 -- 172.21.15.33:0/3417588871 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8d004d030 con 0x558f53eb70f0 2024-05-15T14:07:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.866+0000 7fc8c3fff640 1 -- 172.21.15.33:0/3417588871 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d004f530 con 0x558f53eb70f0 2024-05-15T14:07:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.866+0000 7fc8e3be6740 1 -- 172.21.15.33:0/3417588871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f53eb70f0 msgr2=0x558f53ec5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.866+0000 7fc8e3be6740 1 --2- 172.21.15.33:0/3417588871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f53eb70f0 0x558f53ec5350 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009900 tx=0x7fc8d0005fd0 comp rx=0 tx=0).stop 2024-05-15T14:07:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.866+0000 7fc8e3be6740 1 -- 172.21.15.33:0/3417588871 shutdown_connections 2024-05-15T14:07:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.866+0000 7fc8e3be6740 1 --2- 172.21.15.33:0/3417588871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f53ec5ee0 0x558f53eb89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.866+0000 7fc8e3be6740 1 --2- 172.21.15.33:0/3417588871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f53eb70f0 0x558f53ec5350 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.866+0000 7fc8e3be6740 1 --2- 172.21.15.33:0/3417588871 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f53eb67d0 0x558f53ebe1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.866+0000 7fc8e3be6740 1 -- 172.21.15.33:0/3417588871 >> 172.21.15.33:0/3417588871 conn(0x558f53eb2e00 msgr2=0x558f53eb82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:07:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.866+0000 7fc8e3be6740 1 -- 172.21.15.33:0/3417588871 shutdown_connections 2024-05-15T14:07:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.866+0000 7fc8e3be6740 1 -- 172.21.15.33:0/3417588871 wait complete. 2024-05-15T14:07:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.867+0000 7fc8e3be6740 1 -- start start 2024-05-15T14:07:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.867+0000 7fc8e3be6740 1 Processor -- start 2024-05-15T14:07:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.867+0000 7fc8e3be6740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f53eb67d0 0x558f53ece1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.867+0000 7fc8e3be6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f53eb70f0 0x558f53ec9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.867+0000 7fc8e3be6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f53ec5ee0 0x558f53ec9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.867+0000 7fc8e3be6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc1aab20 con 0x558f53ec5ee0 2024-05-15T14:07:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.867+0000 7fc8e3be6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc1abf70 con 0x558f53eb70f0 2024-05-15T14:07:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.867+0000 7fc8e3be6740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc1ab1d0 con 0x558f53eb67d0 2024-05-15T14:07:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.867+0000 7fc8e3be6740 10 client.0 _ll_register_callbacks cb 0x7fc8cc003b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:07:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.867+0000 7fc8dad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f53ec5ee0 0x558f53ec9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.867+0000 7fc8da59c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f53eb67d0 0x558f53ece1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.867+0000 7fc8dad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f53ec5ee0 0x558f53ec9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49150/0 (socket says 172.21.15.33:49150) 2024-05-15T14:07:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.867+0000 7fc8dad9d640 1 -- 172.21.15.33:0/1530892980 learned_addr learned my addr 172.21.15.33:0/1530892980 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:07:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.867+0000 7fc8d9d9b640 1 --2- 172.21.15.33:0/1530892980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f53eb70f0 0x558f53ec9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.867+0000 7fc8dad9d640 1 -- 172.21.15.33:0/1530892980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f53eb67d0 msgr2=0x558f53ece1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.867+0000 7fc8dad9d640 1 --2- 172.21.15.33:0/1530892980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f53eb67d0 0x558f53ece1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.867+0000 7fc8dad9d640 1 -- 172.21.15.33:0/1530892980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f53eb70f0 msgr2=0x558f53ec9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.867+0000 7fc8dad9d640 1 --2- 172.21.15.33:0/1530892980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f53eb70f0 0x558f53ec9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.867+0000 7fc8dad9d640 1 -- 172.21.15.33:0/1530892980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f53eca0a0 con 0x558f53ec5ee0 2024-05-15T14:07:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.867+0000 7fc8da59c640 1 --2- 172.21.15.33:0/1530892980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f53eb67d0 0x558f53ece1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:07:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.868+0000 7fc8dad9d640 1 --2- 172.21.15.33:0/1530892980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f53ec5ee0 0x558f53ec9a60 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc8d40df790 tx=0x7fc8d40a8f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ae51fb785a4da server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:07:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.868+0000 7fc8c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:07:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.868+0000 7fc8c1ffb640 1 -- 172.21.15.33:0/1530892980 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d40c3060 con 0x558f53ec5ee0 2024-05-15T14:07:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.868+0000 7fc8c1ffb640 1 -- 172.21.15.33:0/1530892980 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8542+0+0 (secure 0 0 0) 0x7fc8d40ad380 con 0x558f53ec5ee0 2024-05-15T14:07:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.868+0000 7fc8e3be6740 1 -- 172.21.15.33:0/1530892980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8d004d030 con 0x558f53ec5ee0 2024-05-15T14:07:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.868+0000 7fc8c1ffb640 1 -- 172.21.15.33:0/1530892980 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc8d4020040 con 0x558f53ec5ee0 2024-05-15T14:07:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.868+0000 7fc8e3be6740 10 client.16203 fetch_fsmap learned FSMap version 30 2024-05-15T14:07:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.868+0000 7fc8e3be6740 1 -- 172.21.15.33:0/1530892980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558f53f40180 con 0x558f53ec5ee0 2024-05-15T14:07:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.868+0000 7fc8c1ffb640 1 -- 172.21.15.33:0/1530892980 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8d004d030 con 0x558f53ec5ee0 2024-05-15T14:07:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.868+0000 7fc8c1ffb640 1 -- 172.21.15.33:0/1530892980 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7fc8d40a4c30 con 0x558f53ec5ee0 2024-05-15T14:07:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.868+0000 7fc8e3be6740 10 client.16203 fetch_fsmap finished waiting for FSMap version 30 2024-05-15T14:07:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.868+0000 7fc8e3be6740 10 client.16203 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:07:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.868+0000 7fc8e3be6740 20 client.16203 populate_metadata read hostname 'smithi033' 2024-05-15T14:07:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.868+0000 7fc8e3be6740 1 --2- 172.21.15.33:0/1530892980 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558f53f40880 0x558f53f42ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.868+0000 7fc8e3be6740 4 client.16203 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:07:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.868+0000 7fc8e3be6740 1 -- 172.21.15.33:0/1530892980 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558f53e25360 con 0x558f53f40880 2024-05-15T14:07:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.869+0000 7fc8da59c640 1 --2- 172.21.15.33:0/1530892980 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558f53f40880 0x558f53f42ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.869+0000 7fc8da59c640 1 --2- 172.21.15.33:0/1530892980 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558f53f40880 0x558f53f42ca0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e4644f4d59b072e server_cookie=4fad105e17682d7f in_seq=0 out_seq=0 2024-05-15T14:07:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.869+0000 7fc8c1ffb640 10 client.16203 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:07:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.942+0000 7fc8c1ffb640 1 -- 172.21.15.33:0/1530892980 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558f53e25360 con 0x558f53f40880 2024-05-15T14:07:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.942+0000 7fc8c1ffb640 10 client.16203 handle_command_reply: tid=0 2024-05-15T14:07:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.942+0000 7fc8e3be6740 1 client.16203 shutdown 2024-05-15T14:07:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.943+0000 7fc8e3be6740 1 -- 172.21.15.33:0/1530892980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f53ec5ee0 msgr2=0x558f53ec9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.943+0000 7fc8e3be6740 1 --2- 172.21.15.33:0/1530892980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f53ec5ee0 0x558f53ec9a60 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc8d40df790 tx=0x7fc8d40a8f10 comp rx=0 tx=0).stop 2024-05-15T14:07:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.943+0000 7fc8e3be6740 1 -- 172.21.15.33:0/1530892980 shutdown_connections 2024-05-15T14:07:25.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.943+0000 7fc8e3be6740 1 --2- 172.21.15.33:0/1530892980 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558f53f40880 0x558f53f42ca0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.944+0000 7fc8e3be6740 1 --2- 172.21.15.33:0/1530892980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f53ec5ee0 0x558f53ec9a60 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.944+0000 7fc8c1ffb640 0 client.16203 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:07:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.944+0000 7fc8e3be6740 1 --2- 172.21.15.33:0/1530892980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f53eb70f0 0x558f53ec9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.944+0000 7fc8e3be6740 1 --2- 172.21.15.33:0/1530892980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f53eb67d0 0x558f53ece1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:25.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.944+0000 7fc8e3be6740 1 -- 172.21.15.33:0/1530892980 >> 172.21.15.33:0/1530892980 conn(0x558f53eb2e00 msgr2=0x558f53ebe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:07:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.944+0000 7fc8e3be6740 1 -- 172.21.15.33:0/1530892980 shutdown_connections 2024-05-15T14:07:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.944+0000 7fc8e3be6740 1 -- 172.21.15.33:0/1530892980 wait complete. 2024-05-15T14:07:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.944+0000 7fc8e3be6740 20 client.16203 trim_cache size 0 max 16384 2024-05-15T14:07:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:07:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:07:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:07:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.945+0000 7fc8e0df0640 1 -- 172.21.15.33:0/3322900549 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc8c40b2950 msgr2=0x7fc8c40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.945+0000 7fc8e0df0640 1 --2- 172.21.15.33:0/3322900549 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc8c40b2950 0x7fc8c40b4e10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc005fd0 tx=0x7fc8cc009620 comp rx=0 tx=0).stop 2024-05-15T14:07:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.945+0000 7fc8e0df0640 1 -- 172.21.15.33:0/3322900549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8dc107940 msgr2=0x7fc8dc19db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.945+0000 7fc8e0df0640 1 --2- 172.21.15.33:0/3322900549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8dc107940 0x7fc8dc19db10 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc8d400d750 tx=0x7fc8d400dc10 comp rx=0 tx=0).stop 2024-05-15T14:07:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.946+0000 7fc8e0df0640 1 -- 172.21.15.33:0/3322900549 shutdown_connections 2024-05-15T14:07:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.946+0000 7fc8e0df0640 1 --2- 172.21.15.33:0/3322900549 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc8c40b2950 0x7fc8c40b4e10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.946+0000 7fc8e0df0640 1 --2- 172.21.15.33:0/3322900549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8dc107940 0x7fc8dc19db10 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.946+0000 7fc8e0df0640 1 --2- 172.21.15.33:0/3322900549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8dc1022f0 0x7fc8dc1997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.946+0000 7fc8e0df0640 1 --2- 172.21.15.33:0/3322900549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8dc101930 0x7fc8dc1992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.946+0000 7fc8e0df0640 1 -- 172.21.15.33:0/3322900549 >> 172.21.15.33:0/3322900549 conn(0x7fc8dc078a70 msgr2=0x7fc8dc100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:07:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.946+0000 7fc8e0df0640 1 -- 172.21.15.33:0/3322900549 shutdown_connections 2024-05-15T14:07:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:25.946+0000 7fc8e0df0640 1 -- 172.21.15.33:0/3322900549 wait complete. 2024-05-15T14:07:26.064 INFO:tasks.workunit.client.0.smithi033.stdout: 1 458685 7.35 MB/sec execute 181 sec latency 3184.272 ms 2024-05-15T14:07:26.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:26 smithi052 ceph-mon[36190]: pgmap v218: 97 pgs: 97 active+clean; 128 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 259 op/s 2024-05-15T14:07:26.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:26 smithi081 ceph-mon[36434]: pgmap v218: 97 pgs: 97 active+clean; 128 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 259 op/s 2024-05-15T14:07:26.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:26 smithi033 ceph-mon[31650]: pgmap v218: 97 pgs: 97 active+clean; 128 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 259 op/s 2024-05-15T14:07:27.064 INFO:tasks.workunit.client.0.smithi033.stdout: 1 458685 7.31 MB/sec execute 182 sec latency 4184.345 ms 2024-05-15T14:07:28.064 INFO:tasks.workunit.client.0.smithi033.stdout: 1 458685 7.27 MB/sec execute 183 sec latency 5184.387 ms 2024-05-15T14:07:28.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:28 smithi052 ceph-mon[36190]: pgmap v219: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 336 op/s 2024-05-15T14:07:28.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:28 smithi081 ceph-mon[36434]: pgmap v219: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 336 op/s 2024-05-15T14:07:28.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:28 smithi033 ceph-mon[31650]: pgmap v219: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 336 op/s 2024-05-15T14:07:29.064 INFO:tasks.workunit.client.0.smithi033.stdout: 1 458685 7.23 MB/sec execute 184 sec latency 6184.435 ms 2024-05-15T14:07:29.666 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.3 2024-05-15T14:07:29.667 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T14:07:29.933 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:07:30.064 INFO:tasks.workunit.client.0.smithi033.stdout: 1 458685 7.19 MB/sec execute 185 sec latency 7184.480 ms 2024-05-15T14:07:30.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:30 smithi033 ceph-mon[31650]: pgmap v220: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 2.1 MiB/s wr, 185 op/s 2024-05-15T14:07:30.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:30 smithi052 ceph-mon[36190]: pgmap v220: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 2.1 MiB/s wr, 185 op/s 2024-05-15T14:07:30.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:30 smithi081 ceph-mon[36434]: pgmap v220: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 2.1 MiB/s wr, 185 op/s 2024-05-15T14:07:31.065 INFO:tasks.workunit.client.0.smithi033.stdout: 1 458685 7.15 MB/sec execute 186 sec latency 8184.578 ms 2024-05-15T14:07:31.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.717+0000 7f96d43cb700 1 -- 172.21.15.33:0/88517659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96cc06bab0 msgr2=0x7f96cc06bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:31.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.717+0000 7f96d43cb700 1 --2- 172.21.15.33:0/88517659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96cc06bab0 0x7f96cc06bf30 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f96bc009990 tx=0x7f96bc009ca0 comp rx=0 tx=0).stop 2024-05-15T14:07:31.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.717+0000 7f96d43cb700 1 -- 172.21.15.33:0/88517659 shutdown_connections 2024-05-15T14:07:31.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.717+0000 7f96d43cb700 1 --2- 172.21.15.33:0/88517659 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96cc06c470 0x7f96cc1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:31.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.717+0000 7f96d43cb700 1 --2- 172.21.15.33:0/88517659 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96cc1095a0 0x7f96cc109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:31.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.717+0000 7f96d43cb700 1 --2- 172.21.15.33:0/88517659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96cc06bab0 0x7f96cc06bf30 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:31.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.717+0000 7f96d43cb700 1 -- 172.21.15.33:0/88517659 >> 172.21.15.33:0/88517659 conn(0x7f96cc0793b0 msgr2=0x7f96cc0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:07:31.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.717+0000 7f96d43cb700 1 -- 172.21.15.33:0/88517659 shutdown_connections 2024-05-15T14:07:31.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.717+0000 7f96d43cb700 1 -- 172.21.15.33:0/88517659 wait complete. 2024-05-15T14:07:31.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.717+0000 7f96d43cb700 1 Processor -- start 2024-05-15T14:07:31.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.718+0000 7f96d43cb700 1 -- start start 2024-05-15T14:07:31.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.718+0000 7f96d43cb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96cc06bab0 0x7f96cc19be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:31.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.718+0000 7f96d43cb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96cc06c470 0x7f96cc19c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:31.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.718+0000 7f96d43cb700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96cc1095a0 0x7f96cc1a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:31.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.718+0000 7f96d43cb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc113730 con 0x7f96cc06bab0 2024-05-15T14:07:31.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.718+0000 7f96d43cb700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc1135b0 con 0x7f96cc06c470 2024-05-15T14:07:31.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.718+0000 7f96d43cb700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc1138b0 con 0x7f96cc1095a0 2024-05-15T14:07:31.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.718+0000 7f96d2167700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96cc06bab0 0x7f96cc19be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:31.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.718+0000 7f96d2167700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96cc06bab0 0x7f96cc19be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49166/0 (socket says 172.21.15.33:49166) 2024-05-15T14:07:31.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.718+0000 7f96d2167700 1 -- 172.21.15.33:0/1058794834 learned_addr learned my addr 172.21.15.33:0/1058794834 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:07:31.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.718+0000 7f96d1966700 1 --2- 172.21.15.33:0/1058794834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96cc06c470 0x7f96cc19c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:31.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.718+0000 7f96d2968700 1 --2- 172.21.15.33:0/1058794834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96cc1095a0 0x7f96cc1a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:31.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.718+0000 7f96d2167700 1 -- 172.21.15.33:0/1058794834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96cc1095a0 msgr2=0x7f96cc1a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:31.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.718+0000 7f96d2167700 1 --2- 172.21.15.33:0/1058794834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96cc1095a0 0x7f96cc1a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:31.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.718+0000 7f96d2167700 1 -- 172.21.15.33:0/1058794834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96cc06c470 msgr2=0x7f96cc19c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:31.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.718+0000 7f96d2167700 1 --2- 172.21.15.33:0/1058794834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96cc06c470 0x7f96cc19c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:31.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.718+0000 7f96d2167700 1 -- 172.21.15.33:0/1058794834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96cc1a0e70 con 0x7f96cc06bab0 2024-05-15T14:07:31.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.718+0000 7f96d2167700 1 --2- 172.21.15.33:0/1058794834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96cc06bab0 0x7f96cc19be60 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f96c800c920 tx=0x7f96c800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:07:31.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.719+0000 7f96c37fe700 1 -- 172.21.15.33:0/1058794834 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c8007b00 con 0x7f96cc06bab0 2024-05-15T14:07:31.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.719+0000 7f96c37fe700 1 -- 172.21.15.33:0/1058794834 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96c8004ce0 con 0x7f96cc06bab0 2024-05-15T14:07:31.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.719+0000 7f96d43cb700 1 -- 172.21.15.33:0/1058794834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96cc1a1160 con 0x7f96cc06bab0 2024-05-15T14:07:31.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.719+0000 7f96c37fe700 1 -- 172.21.15.33:0/1058794834 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c8005550 con 0x7f96cc06bab0 2024-05-15T14:07:31.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.719+0000 7f96d43cb700 1 -- 172.21.15.33:0/1058794834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96cc1a89e0 con 0x7f96cc06bab0 2024-05-15T14:07:31.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.720+0000 7f96d43cb700 1 -- 172.21.15.33:0/1058794834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96cc06a230 con 0x7f96cc06bab0 2024-05-15T14:07:31.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.721+0000 7f96c37fe700 1 -- 172.21.15.33:0/1058794834 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f96c8007ca0 con 0x7f96cc06bab0 2024-05-15T14:07:31.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.722+0000 7f96c37fe700 1 --2- 172.21.15.33:0/1058794834 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f96b80b2a70 0x7f96b80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:31.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.722+0000 7f96d1966700 1 --2- 172.21.15.33:0/1058794834 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f96b80b2a70 0x7f96b80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:31.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.722+0000 7f96c37fe700 1 -- 172.21.15.33:0/1058794834 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8542+0+0 (secure 0 0 0) 0x7f96c80e24a0 con 0x7f96cc06bab0 2024-05-15T14:07:31.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.722+0000 7f96d1966700 1 --2- 172.21.15.33:0/1058794834 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f96b80b2a70 0x7f96b80b4f30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f96bc00b5c0 tx=0x7f96bc005960 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:07:31.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.726+0000 7f96c37fe700 1 -- 172.21.15.33:0/1058794834 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f96c8014030 con 0x7f96cc06bab0 2024-05-15T14:07:32.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:31.999+0000 7f96d43cb700 1 -- 172.21.15.33:0/1058794834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96cc1a8c70 con 0x7f96cc06bab0 2024-05-15T14:07:32.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:32.000+0000 7f96c37fe700 1 -- 172.21.15.33:0/1058794834 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11856 (secure 0 0 0) 0x7f96c80a9e70 con 0x7f96cc06bab0 2024-05-15T14:07:32.002 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T14:07:32.002 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2030117633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2030117633},{"type":"v1","addr":"172.21.15.33:6839","nonce":2030117633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1918979061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1918979061},{"type":"v1","addr":"172.21.15.33:6841","nonce":1918979061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T13:57:19.056417+0000","modified":"2024-05-15T14:06:33.266410+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24463,"mds_2":24461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/1629697841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1629697841},{"type":"v1","addr":"172.21.15.52:6837","nonce":1629697841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.52:6839/2696044471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2696044471},{"type":"v1","addr":"172.21.15.52:6839","nonce":2696044471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.81:6839/1554395940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":1554395940},{"type":"v1","addr":"172.21.15.81:6839","nonce":1554395940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25393":{"gid":25393,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/727032511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":727032511},{"type":"v1","addr":"172.21.15.81:6835","nonce":727032511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24463,24478,24461]},"id":1}]} 2024-05-15T14:07:32.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:32.002+0000 7f96d43cb700 1 -- 172.21.15.33:0/1058794834 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f96b80b2a70 msgr2=0x7f96b80b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:32.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:32.002+0000 7f96d43cb700 1 --2- 172.21.15.33:0/1058794834 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f96b80b2a70 0x7f96b80b4f30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f96bc00b5c0 tx=0x7f96bc005960 comp rx=0 tx=0).stop 2024-05-15T14:07:32.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:32.002+0000 7f96d43cb700 1 -- 172.21.15.33:0/1058794834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96cc06bab0 msgr2=0x7f96cc19be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:32.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:32.002+0000 7f96d43cb700 1 --2- 172.21.15.33:0/1058794834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96cc06bab0 0x7f96cc19be60 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f96c800c920 tx=0x7f96c800cc30 comp rx=0 tx=0).stop 2024-05-15T14:07:32.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:32.002+0000 7f96d43cb700 1 -- 172.21.15.33:0/1058794834 shutdown_connections 2024-05-15T14:07:32.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:32.002+0000 7f96d43cb700 1 --2- 172.21.15.33:0/1058794834 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f96b80b2a70 0x7f96b80b4f30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:32.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:32.002+0000 7f96d43cb700 1 --2- 172.21.15.33:0/1058794834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96cc1095a0 0x7f96cc1a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:32.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:32.002+0000 7f96d43cb700 1 --2- 172.21.15.33:0/1058794834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96cc06bab0 0x7f96cc19be60 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:32.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:32.002+0000 7f96d43cb700 1 --2- 172.21.15.33:0/1058794834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96cc06c470 0x7f96cc19c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:32.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:32.002+0000 7f96d43cb700 1 -- 172.21.15.33:0/1058794834 >> 172.21.15.33:0/1058794834 conn(0x7f96cc0793b0 msgr2=0x7f96cc10e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:07:32.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:32.002+0000 7f96d43cb700 1 -- 172.21.15.33:0/1058794834 shutdown_connections 2024-05-15T14:07:32.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:32.003+0000 7f96d43cb700 1 -- 172.21.15.33:0/1058794834 wait complete. 2024-05-15T14:07:32.006 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-05-15T14:07:32.064 INFO:tasks.workunit.client.0.smithi033.stdout: 1 458685 7.11 MB/sec execute 187 sec latency 9184.629 ms 2024-05-15T14:07:32.569 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:32 smithi033 ceph-mon[31650]: pgmap v221: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 2.1 MiB/s wr, 185 op/s 2024-05-15T14:07:32.569 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:32 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/1058794834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:07:32.570 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24478 quiesce db / --timeout 90 --expiration 150.3 --await-for 45.0' 2024-05-15T14:07:32.571 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24478 quiesce db / --timeout 90 --expiration 150.3 --await-for 45.0 2024-05-15T14:07:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:07:32.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:32 smithi052 ceph-mon[36190]: pgmap v221: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 2.1 MiB/s wr, 185 op/s 2024-05-15T14:07:32.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:32 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/1058794834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:07:32.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:32 smithi081 ceph-mon[36434]: pgmap v221: 97 pgs: 97 active+clean; 118 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 2.1 MiB/s wr, 185 op/s 2024-05-15T14:07:32.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:32 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/1058794834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:07:33.064 INFO:tasks.workunit.client.0.smithi033.stdout: 1 458685 7.07 MB/sec execute 188 sec latency 10184.658 ms 2024-05-15T14:07:34.065 INFO:tasks.workunit.client.0.smithi033.stdout: 1 459867 7.11 MB/sec execute 189 sec latency 10457.554 ms 2024-05-15T14:07:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.585+0000 7f1a75d4b700 1 -- 172.21.15.33:0/2068603391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a70102440 msgr2=0x7f1a701028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.585+0000 7f1a75d4b700 1 --2- 172.21.15.33:0/2068603391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a70102440 0x7f1a701028c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f1a58009990 tx=0x7f1a58009ca0 comp rx=0 tx=0).stop 2024-05-15T14:07:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.585+0000 7f1a75d4b700 1 -- 172.21.15.33:0/2068603391 shutdown_connections 2024-05-15T14:07:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.585+0000 7f1a75d4b700 1 --2- 172.21.15.33:0/2068603391 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a70102e00 0x7f1a7010f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:34.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.585+0000 7f1a75d4b700 1 --2- 172.21.15.33:0/2068603391 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1a701084c0 0x7f1a701088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:34.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.585+0000 7f1a75d4b700 1 --2- 172.21.15.33:0/2068603391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a70102440 0x7f1a701028c0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:34.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.585+0000 7f1a75d4b700 1 -- 172.21.15.33:0/2068603391 >> 172.21.15.33:0/2068603391 conn(0x7f1a700793e0 msgr2=0x7f1a700797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:07:34.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.585+0000 7f1a75d4b700 1 -- 172.21.15.33:0/2068603391 shutdown_connections 2024-05-15T14:07:34.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.585+0000 7f1a75d4b700 1 -- 172.21.15.33:0/2068603391 wait complete. 2024-05-15T14:07:34.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.586+0000 7f1a75d4b700 1 Processor -- start 2024-05-15T14:07:34.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.586+0000 7f1a75d4b700 1 -- start start 2024-05-15T14:07:34.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.586+0000 7f1a75d4b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a70102440 0x7f1a7019bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:34.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.586+0000 7f1a75d4b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1a70102e00 0x7f1a7019c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:34.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.586+0000 7f1a75d4b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a701084c0 0x7f1a701a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.586+0000 7f1a75d4b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1a70113710 con 0x7f1a701084c0 2024-05-15T14:07:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.586+0000 7f1a75d4b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1a70113590 con 0x7f1a70102440 2024-05-15T14:07:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.586+0000 7f1a75d4b700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1a70113890 con 0x7f1a70102e00 2024-05-15T14:07:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.587+0000 7f1a6ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a701084c0 0x7f1a701a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.587+0000 7f1a6f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a70102440 0x7f1a7019bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.587+0000 7f1a66ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1a70102e00 0x7f1a7019c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.587+0000 7f1a6f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a70102440 0x7f1a7019bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:60240/0 (socket says 172.21.15.33:60240) 2024-05-15T14:07:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.587+0000 7f1a66ffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1a70102e00 0x7f1a7019c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:58070/0 (socket says 172.21.15.33:58070) 2024-05-15T14:07:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.587+0000 7f1a6ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a701084c0 0x7f1a701a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44048/0 (socket says 172.21.15.33:44048) 2024-05-15T14:07:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.587+0000 7f1a6f7fe700 1 -- 172.21.15.33:0/893811619 learned_addr learned my addr 172.21.15.33:0/893811619 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:07:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.587+0000 7f1a6f7fe700 1 -- 172.21.15.33:0/893811619 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1a70102e00 msgr2=0x7f1a7019c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.587+0000 7f1a6f7fe700 1 --2- 172.21.15.33:0/893811619 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1a70102e00 0x7f1a7019c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.587+0000 7f1a6f7fe700 1 -- 172.21.15.33:0/893811619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a701084c0 msgr2=0x7f1a701a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.587+0000 7f1a6f7fe700 1 --2- 172.21.15.33:0/893811619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a701084c0 0x7f1a701a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.587+0000 7f1a6f7fe700 1 -- 172.21.15.33:0/893811619 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a701a0d40 con 0x7f1a70102440 2024-05-15T14:07:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.587+0000 7f1a6ffff700 1 --2- 172.21.15.33:0/893811619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a701084c0 0x7f1a701a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:07:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.587+0000 7f1a6f7fe700 1 --2- 172.21.15.33:0/893811619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a70102440 0x7f1a7019bd30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1a6000e9c0 tx=0x7f1a6000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:07:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.587+0000 7f1a6d7fa700 1 -- 172.21.15.33:0/893811619 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6000cbd0 con 0x7f1a70102440 2024-05-15T14:07:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.587+0000 7f1a6d7fa700 1 -- 172.21.15.33:0/893811619 <== mon.2 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a6000ee60 con 0x7f1a70102440 2024-05-15T14:07:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.587+0000 7f1a6d7fa700 1 -- 172.21.15.33:0/893811619 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a600055c0 con 0x7f1a70102440 2024-05-15T14:07:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.587+0000 7f1a75d4b700 1 -- 172.21.15.33:0/893811619 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a701a1030 con 0x7f1a70102440 2024-05-15T14:07:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.587+0000 7f1a75d4b700 1 -- 172.21.15.33:0/893811619 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a701a88b0 con 0x7f1a70102440 2024-05-15T14:07:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.590+0000 7f1a6d7fa700 1 -- 172.21.15.33:0/893811619 <== mon.2 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1a60004750 con 0x7f1a70102440 2024-05-15T14:07:34.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.591+0000 7f1a6d7fa700 1 --2- 172.21.15.33:0/893811619 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f1a5c0b2aa0 0x7f1a5c0b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:34.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.591+0000 7f1a66ffd700 1 --2- 172.21.15.33:0/893811619 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f1a5c0b2aa0 0x7f1a5c0b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:34.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.591+0000 7f1a6d7fa700 1 -- 172.21.15.33:0/893811619 <== mon.2 v2:172.21.15.52:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8542+0+0 (secure 0 0 0) 0x7f1a600e28d0 con 0x7f1a70102440 2024-05-15T14:07:34.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.591+0000 7f1a805e1b80 1 Processor -- start 2024-05-15T14:07:34.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.591+0000 7f1a805e1b80 1 -- start start 2024-05-15T14:07:34.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.591+0000 7f1a66ffd700 1 --2- 172.21.15.33:0/893811619 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f1a5c0b2aa0 0x7f1a5c0b4f60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800b2e0 tx=0x7f1a58005960 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:07:34.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.592+0000 7f1a805e1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b3cb31e4b0 0x55b3cb31e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:34.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.592+0000 7f1a805e1b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3cb31edd0 0x55b3cb319d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:34.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.592+0000 7f1a805e1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3cb31a350 0x55b3cb31a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:34.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.592+0000 7f1a805e1b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b3cb277f00 con 0x55b3cb31edd0 2024-05-15T14:07:34.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.592+0000 7f1a805e1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b3cb267aa0 con 0x55b3cb31e4b0 2024-05-15T14:07:34.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.592+0000 7f1a805e1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55b3cb26bae0 con 0x55b3cb31a350 2024-05-15T14:07:34.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.592+0000 7f1a66ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3cb31edd0 0x55b3cb319d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:34.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.592+0000 7f1a6f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b3cb31e4b0 0x55b3cb31e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:34.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.592+0000 7f1a6f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b3cb31e4b0 0x55b3cb31e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:60254/0 (socket says 172.21.15.33:60254) 2024-05-15T14:07:34.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.592+0000 7f1a6f7fe700 1 -- 172.21.15.33:0/3034662476 learned_addr learned my addr 172.21.15.33:0/3034662476 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:07:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.592+0000 7f1a66ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3cb31edd0 0x55b3cb319d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44062/0 (socket says 172.21.15.33:44062) 2024-05-15T14:07:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.592+0000 7f1a6ffff700 1 --2- 172.21.15.33:0/3034662476 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3cb31a350 0x55b3cb31a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.592+0000 7f1a6f7fe700 1 -- 172.21.15.33:0/3034662476 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3cb31a350 msgr2=0x55b3cb31a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.592+0000 7f1a6f7fe700 1 --2- 172.21.15.33:0/3034662476 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3cb31a350 0x55b3cb31a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.592+0000 7f1a6f7fe700 1 -- 172.21.15.33:0/3034662476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3cb31edd0 msgr2=0x55b3cb319d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.592+0000 7f1a6f7fe700 1 --2- 172.21.15.33:0/3034662476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3cb31edd0 0x55b3cb319d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.592+0000 7f1a6f7fe700 1 -- 172.21.15.33:0/3034662476 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3cb211140 con 0x55b3cb31e4b0 2024-05-15T14:07:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.592+0000 7f1a66ffd700 1 --2- 172.21.15.33:0/3034662476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3cb31edd0 0x55b3cb319d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:07:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.592+0000 7f1a6f7fe700 1 --2- 172.21.15.33:0/3034662476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b3cb31e4b0 0x55b3cb31e890 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1a600a5bc0 tx=0x7f1a600a9580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa1b1ec48c982dfa server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:07:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.592+0000 7f1a65ffb700 1 -- 172.21.15.33:0/3034662476 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a600a9650 con 0x55b3cb31e4b0 2024-05-15T14:07:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.593+0000 7f1a65ffb700 1 -- 172.21.15.33:0/3034662476 <== mon.2 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a600aa240 con 0x55b3cb31e4b0 2024-05-15T14:07:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.593+0000 7f1a65ffb700 1 -- 172.21.15.33:0/3034662476 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a600aa000 con 0x55b3cb31e4b0 2024-05-15T14:07:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.593+0000 7f1a805e1b80 1 -- 172.21.15.33:0/3034662476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b3cb31e4b0 msgr2=0x55b3cb31e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.593+0000 7f1a805e1b80 1 --2- 172.21.15.33:0/3034662476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b3cb31e4b0 0x55b3cb31e890 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1a600a5bc0 tx=0x7f1a600a9580 comp rx=0 tx=0).stop 2024-05-15T14:07:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.593+0000 7f1a805e1b80 1 -- 172.21.15.33:0/3034662476 shutdown_connections 2024-05-15T14:07:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.593+0000 7f1a805e1b80 1 --2- 172.21.15.33:0/3034662476 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3cb31a350 0x55b3cb31a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.593+0000 7f1a805e1b80 1 --2- 172.21.15.33:0/3034662476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b3cb31e4b0 0x55b3cb31e890 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.593+0000 7f1a805e1b80 1 --2- 172.21.15.33:0/3034662476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3cb31edd0 0x55b3cb319d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.593+0000 7f1a805e1b80 1 -- 172.21.15.33:0/3034662476 >> 172.21.15.33:0/3034662476 conn(0x55b3cb246f40 msgr2=0x55b3cb314530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:07:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.593+0000 7f1a805e1b80 1 -- 172.21.15.33:0/3034662476 shutdown_connections 2024-05-15T14:07:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.593+0000 7f1a805e1b80 1 -- 172.21.15.33:0/3034662476 wait complete. 2024-05-15T14:07:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.594+0000 7f1a805e1b80 1 -- start start 2024-05-15T14:07:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.594+0000 7f1a805e1b80 1 Processor -- start 2024-05-15T14:07:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.594+0000 7f1a805e1b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3cb31a350 0x55b3cb325ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.594+0000 7f1a805e1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b3cb31e4b0 0x55b3cb323040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.594+0000 7f1a805e1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3cb31edd0 0x55b3cb323580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.594+0000 7f1a805e1b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b3cb16e130 con 0x55b3cb31a350 2024-05-15T14:07:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.594+0000 7f1a805e1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b3cb1cff30 con 0x55b3cb31e4b0 2024-05-15T14:07:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.594+0000 7f1a805e1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55b3cb326910 con 0x55b3cb31edd0 2024-05-15T14:07:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.594+0000 7f1a805e1b80 10 client.0 _ll_register_callbacks cb 0x55b3cb202ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:07:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.594+0000 7f1a6f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3cb31a350 0x55b3cb325ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.594+0000 7f1a6f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3cb31a350 0x55b3cb325ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44076/0 (socket says 172.21.15.33:44076) 2024-05-15T14:07:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.594+0000 7f1a6f7fe700 1 -- 172.21.15.33:0/853679200 learned_addr learned my addr 172.21.15.33:0/853679200 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:07:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.594+0000 7f1a6ffff700 1 --2- 172.21.15.33:0/853679200 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3cb31edd0 0x55b3cb323580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.595+0000 7f1a66ffd700 1 --2- 172.21.15.33:0/853679200 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b3cb31e4b0 0x55b3cb323040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.595+0000 7f1a6f7fe700 1 -- 172.21.15.33:0/853679200 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3cb31edd0 msgr2=0x55b3cb323580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.595+0000 7f1a6f7fe700 1 --2- 172.21.15.33:0/853679200 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3cb31edd0 0x55b3cb323580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.595+0000 7f1a6f7fe700 1 -- 172.21.15.33:0/853679200 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b3cb31e4b0 msgr2=0x55b3cb323040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.595+0000 7f1a6f7fe700 1 --2- 172.21.15.33:0/853679200 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b3cb31e4b0 0x55b3cb323040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.595+0000 7f1a6f7fe700 1 -- 172.21.15.33:0/853679200 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3cb1b7a60 con 0x55b3cb31a350 2024-05-15T14:07:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.595+0000 7f1a6f7fe700 1 --2- 172.21.15.33:0/853679200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3cb31a350 0x55b3cb325ca0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f1a6000cf60 tx=0x7f1a600aa050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958d465285d5f6f server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:07:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.595+0000 7f1a4effd700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:07:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.595+0000 7f1a4effd700 1 -- 172.21.15.33:0/853679200 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6006d020 con 0x55b3cb31a350 2024-05-15T14:07:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.595+0000 7f1a4effd700 1 -- 172.21.15.33:0/853679200 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8542+0+0 (secure 0 0 0) 0x7f1a600b2720 con 0x55b3cb31a350 2024-05-15T14:07:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.595+0000 7f1a805e1b80 1 -- 172.21.15.33:0/853679200 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a600aa240 con 0x55b3cb31a350 2024-05-15T14:07:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.595+0000 7f1a4effd700 1 -- 172.21.15.33:0/853679200 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1a600a9650 con 0x55b3cb31a350 2024-05-15T14:07:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.595+0000 7f1a805e1b80 10 client.16221 fetch_fsmap learned FSMap version 30 2024-05-15T14:07:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.595+0000 7f1a805e1b80 1 -- 172.21.15.33:0/853679200 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b3cb2218f0 con 0x55b3cb31a350 2024-05-15T14:07:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.596+0000 7f1a4effd700 1 -- 172.21.15.33:0/853679200 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a600adef0 con 0x55b3cb31a350 2024-05-15T14:07:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.596+0000 7f1a4effd700 1 -- 172.21.15.33:0/853679200 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7f1a600a5630 con 0x55b3cb31a350 2024-05-15T14:07:34.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.596+0000 7f1a805e1b80 10 client.16221 fetch_fsmap finished waiting for FSMap version 30 2024-05-15T14:07:34.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.596+0000 7f1a805e1b80 10 client.16221 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:07:34.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.596+0000 7f1a805e1b80 20 client.16221 populate_metadata read hostname 'smithi033' 2024-05-15T14:07:34.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.596+0000 7f1a805e1b80 1 --2- 172.21.15.33:0/853679200 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55b3cb333440 0x55b3cb335830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:34.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.596+0000 7f1a805e1b80 4 client.16221 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:07:34.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.596+0000 7f1a805e1b80 1 -- 172.21.15.33:0/853679200 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1a600aa240 con 0x55b3cb333440 2024-05-15T14:07:34.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.596+0000 7f1a66ffd700 1 --2- 172.21.15.33:0/853679200 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55b3cb333440 0x55b3cb335830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:34.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.596+0000 7f1a4effd700 10 client.16221 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:07:34.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.596+0000 7f1a66ffd700 1 --2- 172.21.15.33:0/853679200 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55b3cb333440 0x55b3cb335830 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f18707abe9f9059a server_cookie=c18782a7490a9046 in_seq=0 out_seq=0 2024-05-15T14:07:34.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.597+0000 7f1a4effd700 1 -- 172.21.15.33:0/853679200 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f1a600aa240 con 0x55b3cb333440 2024-05-15T14:07:34.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.597+0000 7f1a4effd700 10 client.16221 handle_command_reply: tid=0 2024-05-15T14:07:34.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.597+0000 7f1a805e1b80 1 client.16221 shutdown 2024-05-15T14:07:34.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.597+0000 7f1a805e1b80 1 -- 172.21.15.33:0/853679200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3cb31a350 msgr2=0x55b3cb325ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:34.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.597+0000 7f1a805e1b80 1 --2- 172.21.15.33:0/853679200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3cb31a350 0x55b3cb325ca0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f1a6000cf60 tx=0x7f1a600aa050 comp rx=0 tx=0).stop 2024-05-15T14:07:34.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.598+0000 7f1a805e1b80 1 -- 172.21.15.33:0/853679200 shutdown_connections 2024-05-15T14:07:34.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.598+0000 7f1a805e1b80 1 --2- 172.21.15.33:0/853679200 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3cb31edd0 0x55b3cb323580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:34.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.598+0000 7f1a805e1b80 1 --2- 172.21.15.33:0/853679200 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55b3cb333440 0x55b3cb335830 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:34.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.598+0000 7f1a805e1b80 1 --2- 172.21.15.33:0/853679200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3cb31a350 0x55b3cb325ca0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:34.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.598+0000 7f1a805e1b80 1 --2- 172.21.15.33:0/853679200 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b3cb31e4b0 0x55b3cb323040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:34.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.598+0000 7f1a805e1b80 1 -- 172.21.15.33:0/853679200 >> 172.21.15.33:0/853679200 conn(0x55b3cb246f40 msgr2=0x55b3cb30f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:07:34.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.598+0000 7f1a4effd700 0 client.16221 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:07:34.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.598+0000 7f1a805e1b80 1 -- 172.21.15.33:0/853679200 shutdown_connections 2024-05-15T14:07:34.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.598+0000 7f1a805e1b80 1 -- 172.21.15.33:0/853679200 wait complete. 2024-05-15T14:07:34.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.598+0000 7f1a805e1b80 20 client.16221 trim_cache size 0 max 16384 2024-05-15T14:07:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.620+0000 7f1a805e1b80 1 Processor -- start 2024-05-15T14:07:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.620+0000 7f1a805e1b80 1 -- start start 2024-05-15T14:07:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.621+0000 7f1a805e1b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3cb31a350 0x55b3cb322000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.621+0000 7f1a805e1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3cb31e4b0 0x55b3cb319390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.621+0000 7f1a805e1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b3cb31edd0 0x55b3cb30f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.621+0000 7f1a805e1b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1a701b2900 con 0x55b3cb31a350 2024-05-15T14:07:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.621+0000 7f1a805e1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1a701b0160 con 0x55b3cb31edd0 2024-05-15T14:07:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.621+0000 7f1a805e1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1a701adfd0 con 0x55b3cb31e4b0 2024-05-15T14:07:34.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.621+0000 7f1a6f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3cb31a350 0x55b3cb322000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:34.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.621+0000 7f1a6ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b3cb31edd0 0x55b3cb30f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:34.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.621+0000 7f1a6f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3cb31a350 0x55b3cb322000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44080/0 (socket says 172.21.15.33:44080) 2024-05-15T14:07:34.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.621+0000 7f1a6f7fe700 1 -- 172.21.15.33:0/2601875926 learned_addr learned my addr 172.21.15.33:0/2601875926 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:07:34.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.621+0000 7f1a66ffd700 1 --2- 172.21.15.33:0/2601875926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3cb31e4b0 0x55b3cb319390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:34.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.621+0000 7f1a6f7fe700 1 -- 172.21.15.33:0/2601875926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3cb31e4b0 msgr2=0x55b3cb319390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:34.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.621+0000 7f1a6f7fe700 1 --2- 172.21.15.33:0/2601875926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3cb31e4b0 0x55b3cb319390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:34.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.621+0000 7f1a6f7fe700 1 -- 172.21.15.33:0/2601875926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b3cb31edd0 msgr2=0x55b3cb30f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:34.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.621+0000 7f1a6f7fe700 1 --2- 172.21.15.33:0/2601875926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b3cb31edd0 0x55b3cb30f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:34.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.621+0000 7f1a6f7fe700 1 -- 172.21.15.33:0/2601875926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3cb30ccf0 con 0x55b3cb31a350 2024-05-15T14:07:34.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.621+0000 7f1a6ffff700 1 --2- 172.21.15.33:0/2601875926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b3cb31edd0 0x55b3cb30f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:07:34.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.621+0000 7f1a6f7fe700 1 --2- 172.21.15.33:0/2601875926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3cb31a350 0x55b3cb322000 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f1a600a97f0 tx=0x7f1a600c4300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b14ce1bd99f570 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:07:34.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.622+0000 7f1a657fa700 1 -- 172.21.15.33:0/2601875926 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a600a9490 con 0x55b3cb31a350 2024-05-15T14:07:34.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.622+0000 7f1a657fa700 1 -- 172.21.15.33:0/2601875926 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a600aa240 con 0x55b3cb31a350 2024-05-15T14:07:34.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.622+0000 7f1a657fa700 1 -- 172.21.15.33:0/2601875926 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6006f2c0 con 0x55b3cb31a350 2024-05-15T14:07:34.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.623+0000 7f1a805e1b80 1 -- 172.21.15.33:0/2601875926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3cb31a350 msgr2=0x55b3cb322000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:34.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.623+0000 7f1a805e1b80 1 --2- 172.21.15.33:0/2601875926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3cb31a350 0x55b3cb322000 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f1a600a97f0 tx=0x7f1a600c4300 comp rx=0 tx=0).stop 2024-05-15T14:07:34.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.623+0000 7f1a805e1b80 1 -- 172.21.15.33:0/2601875926 shutdown_connections 2024-05-15T14:07:34.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.623+0000 7f1a805e1b80 1 --2- 172.21.15.33:0/2601875926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b3cb31edd0 0x55b3cb30f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:34.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.623+0000 7f1a805e1b80 1 --2- 172.21.15.33:0/2601875926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3cb31a350 0x55b3cb322000 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:34.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.623+0000 7f1a805e1b80 1 --2- 172.21.15.33:0/2601875926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3cb31e4b0 0x55b3cb319390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:34.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.623+0000 7f1a805e1b80 1 -- 172.21.15.33:0/2601875926 >> 172.21.15.33:0/2601875926 conn(0x55b3cb246f40 msgr2=0x55b3cb31d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:07:34.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.623+0000 7f1a805e1b80 1 -- 172.21.15.33:0/2601875926 shutdown_connections 2024-05-15T14:07:34.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.623+0000 7f1a805e1b80 1 -- 172.21.15.33:0/2601875926 wait complete. 2024-05-15T14:07:34.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.623+0000 7f1a805e1b80 1 -- start start 2024-05-15T14:07:34.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.623+0000 7f1a805e1b80 1 Processor -- start 2024-05-15T14:07:34.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.624+0000 7f1a805e1b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3cb31a350 0x55b3cb338310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:34.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.624+0000 7f1a805e1b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3cb31e4b0 0x55b3cb338850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:34.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.624+0000 7f1a805e1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b3cb31edd0 0x55b3cb336810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:34.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.624+0000 7f1a805e1b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1a701ae620 con 0x55b3cb31e4b0 2024-05-15T14:07:34.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.624+0000 7f1a805e1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1a701afa90 con 0x55b3cb31edd0 2024-05-15T14:07:34.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.624+0000 7f1a805e1b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1a701aecf0 con 0x55b3cb31a350 2024-05-15T14:07:34.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.624+0000 7f1a805e1b80 10 client.0 _ll_register_callbacks cb 0x55b3cb202ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:07:34.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.624+0000 7f1a6ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b3cb31edd0 0x55b3cb336810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:34.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.624+0000 7f1a66ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3cb31e4b0 0x55b3cb338850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:34.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.624+0000 7f1a66ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3cb31e4b0 0x55b3cb338850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44090/0 (socket says 172.21.15.33:44090) 2024-05-15T14:07:34.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.624+0000 7f1a6f7fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3cb31a350 0x55b3cb338310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:34.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.624+0000 7f1a66ffd700 1 -- 172.21.15.33:0/1714241549 learned_addr learned my addr 172.21.15.33:0/1714241549 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:07:34.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.624+0000 7f1a6ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b3cb31edd0 0x55b3cb336810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:60276/0 (socket says 172.21.15.33:60276) 2024-05-15T14:07:34.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.624+0000 7f1a6f7fe700 1 -- 172.21.15.33:0/1714241549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b3cb31edd0 msgr2=0x55b3cb336810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:34.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.624+0000 7f1a6f7fe700 1 --2- 172.21.15.33:0/1714241549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b3cb31edd0 0x55b3cb336810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:34.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.624+0000 7f1a6f7fe700 1 -- 172.21.15.33:0/1714241549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3cb31e4b0 msgr2=0x55b3cb338850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:34.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.624+0000 7f1a6f7fe700 1 --2- 172.21.15.33:0/1714241549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3cb31e4b0 0x55b3cb338850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:34.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.625+0000 7f1a6f7fe700 1 -- 172.21.15.33:0/1714241549 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3cb336fe0 con 0x55b3cb31a350 2024-05-15T14:07:34.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.625+0000 7f1a6f7fe700 1 --2- 172.21.15.33:0/1714241549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3cb31a350 0x55b3cb338310 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1a600a6db0 tx=0x7f1a600a9910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f430fbc7a71bc5c server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:07:34.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.625+0000 7f1a4effd700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-05-15T14:07:34.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.625+0000 7f1a4effd700 1 -- 172.21.15.33:0/1714241549 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6006d070 con 0x55b3cb31a350 2024-05-15T14:07:34.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.625+0000 7f1a4effd700 1 -- 172.21.15.33:0/1714241549 <== mon.1 v2:172.21.15.81:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8542+0+0 (secure 0 0 0) 0x7f1a600a8a70 con 0x55b3cb31a350 2024-05-15T14:07:34.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.625+0000 7f1a805e1b80 1 -- 172.21.15.33:0/1714241549 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a600aa240 con 0x55b3cb31a350 2024-05-15T14:07:34.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.625+0000 7f1a4effd700 1 -- 172.21.15.33:0/1714241549 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1a600a9490 con 0x55b3cb31a350 2024-05-15T14:07:34.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.625+0000 7f1a805e1b80 10 client.25846 fetch_fsmap learned FSMap version 30 2024-05-15T14:07:34.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.625+0000 7f1a805e1b80 1 -- 172.21.15.33:0/1714241549 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b3cb336fe0 con 0x55b3cb31a350 2024-05-15T14:07:34.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.625+0000 7f1a4effd700 1 -- 172.21.15.33:0/1714241549 <== mon.1 v2:172.21.15.81:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a600aa240 con 0x55b3cb31a350 2024-05-15T14:07:34.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.626+0000 7f1a4effd700 1 -- 172.21.15.33:0/1714241549 <== mon.1 v2:172.21.15.81:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7f1a600af080 con 0x55b3cb31a350 2024-05-15T14:07:34.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.626+0000 7f1a805e1b80 10 client.25846 fetch_fsmap finished waiting for FSMap version 30 2024-05-15T14:07:34.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.626+0000 7f1a805e1b80 10 client.25846 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:07:34.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.626+0000 7f1a805e1b80 20 client.25846 populate_metadata read hostname 'smithi033' 2024-05-15T14:07:34.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.626+0000 7f1a805e1b80 1 --2- 172.21.15.33:0/1714241549 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55b3cb39e690 0x55b3cb3a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:34.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.626+0000 7f1a805e1b80 4 client.25846 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.3, "await_for": 45.0}] 2024-05-15T14:07:34.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.626+0000 7f1a805e1b80 1 -- 172.21.15.33:0/1714241549 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.3, "await_for": 45.0}) -- 0x55b3cb279e50 con 0x55b3cb39e690 2024-05-15T14:07:34.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.626+0000 7f1a66ffd700 1 --2- 172.21.15.33:0/1714241549 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55b3cb39e690 0x55b3cb3a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:34.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.626+0000 7f1a66ffd700 1 --2- 172.21.15.33:0/1714241549 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55b3cb39e690 0x55b3cb3a0ab0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=134eb2a6da2b6ebb server_cookie=7f407814c6173fd8 in_seq=0 out_seq=0 2024-05-15T14:07:34.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.626+0000 7f1a4effd700 10 client.25846 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:07:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.757+0000 7f1a4effd700 1 -- 172.21.15.33:0/1714241549 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b3cb279e50 con 0x55b3cb39e690 2024-05-15T14:07:34.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.757+0000 7f1a4effd700 10 client.25846 handle_command_reply: tid=0 2024-05-15T14:07:34.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.757+0000 7f1a805e1b80 1 client.25846 shutdown 2024-05-15T14:07:34.759 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:34 smithi033 ceph-mon[31650]: pgmap v222: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 2.7 MiB/s wr, 206 op/s 2024-05-15T14:07:34.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.757+0000 7f1a805e1b80 1 -- 172.21.15.33:0/1714241549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3cb31a350 msgr2=0x55b3cb338310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:34.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.757+0000 7f1a805e1b80 1 --2- 172.21.15.33:0/1714241549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3cb31a350 0x55b3cb338310 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1a600a6db0 tx=0x7f1a600a9910 comp rx=0 tx=0).stop 2024-05-15T14:07:34.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.757+0000 7f1a805e1b80 1 -- 172.21.15.33:0/1714241549 shutdown_connections 2024-05-15T14:07:34.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.757+0000 7f1a805e1b80 1 --2- 172.21.15.33:0/1714241549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b3cb31edd0 0x55b3cb336810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:34.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.757+0000 7f1a805e1b80 1 --2- 172.21.15.33:0/1714241549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b3cb31a350 0x55b3cb338310 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:34.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.757+0000 7f1a805e1b80 1 --2- 172.21.15.33:0/1714241549 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55b3cb39e690 0x55b3cb3a0ab0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:34.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.757+0000 7f1a805e1b80 1 --2- 172.21.15.33:0/1714241549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b3cb31e4b0 0x55b3cb338850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:34.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.757+0000 7f1a4effd700 0 client.25846 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:07:34.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.757+0000 7f1a805e1b80 1 -- 172.21.15.33:0/1714241549 >> 172.21.15.33:0/1714241549 conn(0x55b3cb246f40 msgr2=0x55b3cb31d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:07:34.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.757+0000 7f1a805e1b80 1 -- 172.21.15.33:0/1714241549 shutdown_connections 2024-05-15T14:07:34.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.758+0000 7f1a805e1b80 1 -- 172.21.15.33:0/1714241549 wait complete. 2024-05-15T14:07:34.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.758+0000 7f1a805e1b80 20 client.25846 trim_cache size 0 max 16384 2024-05-15T14:07:34.761 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T14:07:34.761 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 30, 2024-05-15T14:07:34.761 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24478, 2024-05-15T14:07:34.761 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 26, 2024-05-15T14:07:34.761 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-05-15T14:07:34.761 INFO:teuthology.orchestra.run.smithi033.stdout: "c7303461": { 2024-05-15T14:07:34.761 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 26, 2024-05-15T14:07:34.761 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-05-15T14:07:34.762 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:07:34.762 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-05-15T14:07:34.762 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:07:34.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:07:34.762 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 90.0, 2024-05-15T14:07:34.762 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 150.3, 2024-05-15T14:07:34.762 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-05-15T14:07:34.762 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-05-15T14:07:34.762 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-05-15T14:07:34.762 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:07:34.762 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-05-15T14:07:34.762 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:07:34.762 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:07:34.762 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:07:34.763 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:07:34.763 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:07:34.763 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:07:34.763 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T14:07:34.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.758+0000 7f1a75d4b700 1 -- 172.21.15.33:0/893811619 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f1a5c0b2aa0 msgr2=0x7f1a5c0b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:34.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.758+0000 7f1a75d4b700 1 --2- 172.21.15.33:0/893811619 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f1a5c0b2aa0 0x7f1a5c0b4f60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800b2e0 tx=0x7f1a58005960 comp rx=0 tx=0).stop 2024-05-15T14:07:34.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.758+0000 7f1a75d4b700 1 -- 172.21.15.33:0/893811619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a70102440 msgr2=0x7f1a7019bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:34.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.758+0000 7f1a75d4b700 1 --2- 172.21.15.33:0/893811619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a70102440 0x7f1a7019bd30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1a6000e9c0 tx=0x7f1a6000ecd0 comp rx=0 tx=0).stop 2024-05-15T14:07:34.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.758+0000 7f1a75d4b700 1 -- 172.21.15.33:0/893811619 shutdown_connections 2024-05-15T14:07:34.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.758+0000 7f1a75d4b700 1 --2- 172.21.15.33:0/893811619 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f1a5c0b2aa0 0x7f1a5c0b4f60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:34.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.758+0000 7f1a75d4b700 1 --2- 172.21.15.33:0/893811619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a701084c0 0x7f1a701a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:34.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.758+0000 7f1a75d4b700 1 --2- 172.21.15.33:0/893811619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a70102440 0x7f1a7019bd30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:34.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.758+0000 7f1a75d4b700 1 --2- 172.21.15.33:0/893811619 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1a70102e00 0x7f1a7019c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:34.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.758+0000 7f1a75d4b700 1 -- 172.21.15.33:0/893811619 >> 172.21.15.33:0/893811619 conn(0x7f1a700793e0 msgr2=0x7f1a701002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:07:34.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.758+0000 7f1a75d4b700 1 -- 172.21.15.33:0/893811619 shutdown_connections 2024-05-15T14:07:34.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:07:34.758+0000 7f1a75d4b700 1 -- 172.21.15.33:0/893811619 wait complete. 2024-05-15T14:07:34.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:34 smithi052 ceph-mon[36190]: pgmap v222: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 2.7 MiB/s wr, 206 op/s 2024-05-15T14:07:34.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:34 smithi081 ceph-mon[36434]: pgmap v222: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 2.7 MiB/s wr, 206 op/s 2024-05-15T14:07:35.065 INFO:tasks.workunit.client.0.smithi033.stdout: 1 461705 7.12 MB/sec execute 190 sec latency 309.144 ms 2024-05-15T14:07:35.405 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c7303461', quiesce took 5.7 seconds. Will release after: 24.6 2024-05-15T14:07:35.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:35 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:07:35.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:35 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:07:35.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:35 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:07:36.065 INFO:tasks.workunit.client.0.smithi033.stdout: 1 461705 7.08 MB/sec execute 191 sec latency 1309.192 ms 2024-05-15T14:07:36.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:36 smithi052 ceph-mon[36190]: pgmap v223: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 1.3 MiB/s wr, 97 op/s 2024-05-15T14:07:36.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:36 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:07:36.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:36 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:07:36.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:36 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:07:36.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:36 smithi081 ceph-mon[36434]: pgmap v223: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 1.3 MiB/s wr, 97 op/s 2024-05-15T14:07:36.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:36 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:07:36.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:36 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:07:36.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:36 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:07:36.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:36 smithi033 ceph-mon[31650]: pgmap v223: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 1.3 MiB/s wr, 97 op/s 2024-05-15T14:07:36.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:36 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:07:36.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:36 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:07:36.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:36 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:07:37.065 INFO:tasks.workunit.client.0.smithi033.stdout: 1 461705 7.05 MB/sec execute 192 sec latency 2309.292 ms 2024-05-15T14:07:38.065 INFO:tasks.workunit.client.0.smithi033.stdout: 1 461705 7.01 MB/sec execute 193 sec latency 3309.405 ms 2024-05-15T14:07:38.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:38 smithi052 ceph-mon[36190]: pgmap v224: 97 pgs: 97 active+clean; 133 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 2.8 MiB/s wr, 154 op/s 2024-05-15T14:07:38.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:38 smithi081 ceph-mon[36434]: pgmap v224: 97 pgs: 97 active+clean; 133 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 2.8 MiB/s wr, 154 op/s 2024-05-15T14:07:38.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:38 smithi033 ceph-mon[31650]: pgmap v224: 97 pgs: 97 active+clean; 133 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 2.8 MiB/s wr, 154 op/s 2024-05-15T14:07:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.052+0000 7f8403a93640 1 -- 172.21.15.33:0/946835839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83fc1033f0 msgr2=0x7f83fc10f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.052+0000 7f8403a93640 1 --2- 172.21.15.33:0/946835839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83fc1033f0 0x7f83fc10f900 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f83f4009cd0 tx=0x7f83f402a8e0 comp rx=0 tx=0).stop 2024-05-15T14:07:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.052+0000 7f8403a93640 1 -- 172.21.15.33:0/946835839 shutdown_connections 2024-05-15T14:07:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.052+0000 7f8403a93640 1 --2- 172.21.15.33:0/946835839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83fc1033f0 0x7f83fc10f900 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.052+0000 7f8403a93640 1 --2- 172.21.15.33:0/946835839 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83fc102a30 0x7f83fc102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.052+0000 7f8403a93640 1 --2- 172.21.15.33:0/946835839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83fc108a40 0x7f83fc108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.052+0000 7f8403a93640 1 -- 172.21.15.33:0/946835839 >> 172.21.15.33:0/946835839 conn(0x7f83fc0fe4e0 msgr2=0x7f83fc100900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:07:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.052+0000 7f8403a93640 1 -- 172.21.15.33:0/946835839 shutdown_connections 2024-05-15T14:07:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.052+0000 7f8403a93640 1 -- 172.21.15.33:0/946835839 wait complete. 2024-05-15T14:07:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.052+0000 7f8403a93640 1 Processor -- start 2024-05-15T14:07:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.052+0000 7f8403a93640 1 -- start start 2024-05-15T14:07:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.053+0000 7f8403a93640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83fc102a30 0x7f83fc199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.053+0000 7f8403a93640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83fc1033f0 0x7f83fc199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.053+0000 7f8403a93640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83fc108a40 0x7f83fc19da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.053+0000 7f8403a93640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc112d00 con 0x7f83fc108a40 2024-05-15T14:07:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.053+0000 7f8403a93640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc112b80 con 0x7f83fc1033f0 2024-05-15T14:07:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.053+0000 7f8403a93640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc112e80 con 0x7f83fc102a30 2024-05-15T14:07:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.053+0000 7f8402030640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83fc108a40 0x7f83fc19da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.053+0000 7f8402030640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83fc108a40 0x7f83fc19da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44106/0 (socket says 172.21.15.33:44106) 2024-05-15T14:07:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.053+0000 7f8402030640 1 -- 172.21.15.33:0/1320541603 learned_addr learned my addr 172.21.15.33:0/1320541603 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:07:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.053+0000 7f840102e640 1 --2- 172.21.15.33:0/1320541603 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83fc1033f0 0x7f83fc199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.053+0000 7f840182f640 1 --2- 172.21.15.33:0/1320541603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83fc102a30 0x7f83fc199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.053+0000 7f8402030640 1 -- 172.21.15.33:0/1320541603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83fc102a30 msgr2=0x7f83fc199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.053+0000 7f8402030640 1 --2- 172.21.15.33:0/1320541603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83fc102a30 0x7f83fc199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.053+0000 7f8402030640 1 -- 172.21.15.33:0/1320541603 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83fc1033f0 msgr2=0x7f83fc199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.053+0000 7f8402030640 1 --2- 172.21.15.33:0/1320541603 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83fc1033f0 0x7f83fc199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.053+0000 7f8402030640 1 -- 172.21.15.33:0/1320541603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83fc19e1c0 con 0x7f83fc108a40 2024-05-15T14:07:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.053+0000 7f840102e640 1 --2- 172.21.15.33:0/1320541603 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83fc1033f0 0x7f83fc199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:07:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.053+0000 7f8402030640 1 --2- 172.21.15.33:0/1320541603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83fc108a40 0x7f83fc19da90 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f83f4033040 tx=0x7f83f4003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:07:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.054+0000 7f83f2ffd640 1 -- 172.21.15.33:0/1320541603 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f4004040 con 0x7f83fc108a40 2024-05-15T14:07:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.054+0000 7f8403a93640 1 -- 172.21.15.33:0/1320541603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83fc19e450 con 0x7f83fc108a40 2024-05-15T14:07:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.054+0000 7f83f2ffd640 1 -- 172.21.15.33:0/1320541603 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83f4004690 con 0x7f83fc108a40 2024-05-15T14:07:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.054+0000 7f83f2ffd640 1 -- 172.21.15.33:0/1320541603 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f4030d80 con 0x7f83fc108a40 2024-05-15T14:07:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.054+0000 7f8403a93640 1 -- 172.21.15.33:0/1320541603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83fc1a5d40 con 0x7f83fc108a40 2024-05-15T14:07:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.057+0000 7f84067ab740 1 Processor -- start 2024-05-15T14:07:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.057+0000 7f83f2ffd640 1 -- 172.21.15.33:0/1320541603 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f83f40041e0 con 0x7f83fc108a40 2024-05-15T14:07:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.057+0000 7f84067ab740 1 -- start start 2024-05-15T14:07:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.057+0000 7f84067ab740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611061f4830 0x5611061f4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.057+0000 7f84067ab740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611061f5150 0x5611062039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.057+0000 7f84067ab740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561106203ee0 0x5611062062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.057+0000 7f84067ab740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561106167c30 con 0x5611061f4830 2024-05-15T14:07:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.057+0000 7f84067ab740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5611061577d0 con 0x5611061f5150 2024-05-15T14:07:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.057+0000 7f84067ab740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56110615b810 con 0x561106203ee0 2024-05-15T14:07:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.057+0000 7f840102e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611061f4830 0x5611061f4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.058+0000 7f840102e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611061f4830 0x5611061f4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44114/0 (socket says 172.21.15.33:44114) 2024-05-15T14:07:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.058+0000 7f840102e640 1 -- 172.21.15.33:0/3956790346 learned_addr learned my addr 172.21.15.33:0/3956790346 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:07:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.058+0000 7f840182f640 1 --2- 172.21.15.33:0/3956790346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611061f5150 0x5611062039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.058+0000 7f840102e640 1 --2- 172.21.15.33:0/3956790346 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561106203ee0 0x5611062062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.058+0000 7f840102e640 1 -- 172.21.15.33:0/3956790346 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561106203ee0 msgr2=0x5611062062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.058+0000 7f840102e640 1 --2- 172.21.15.33:0/3956790346 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561106203ee0 0x5611062062d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.058+0000 7f840102e640 1 -- 172.21.15.33:0/3956790346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611061f5150 msgr2=0x5611062039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.058+0000 7f840102e640 1 --2- 172.21.15.33:0/3956790346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611061f5150 0x5611062039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.058+0000 7f840102e640 1 -- 172.21.15.33:0/3956790346 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561105e6c210 con 0x5611061f4830 2024-05-15T14:07:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.058+0000 7f840102e640 1 --2- 172.21.15.33:0/3956790346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611061f4830 0x5611061f4c10 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f83ec013970 tx=0x7f83ec00abf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcdbb9d2079e0235 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:07:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.058+0000 7f83f2ffd640 1 --2- 172.21.15.33:0/1320541603 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f83e80b2950 0x7f83e80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.058+0000 7f840182f640 1 --2- 172.21.15.33:0/1320541603 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f83e80b2950 0x7f83e80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.058+0000 7f83e7fff640 1 -- 172.21.15.33:0/3956790346 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec019070 con 0x5611061f4830 2024-05-15T14:07:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.058+0000 7f83f2ffd640 1 -- 172.21.15.33:0/1320541603 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8542+0+0 (secure 0 0 0) 0x7f83f402e030 con 0x7f83fc108a40 2024-05-15T14:07:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.058+0000 7f83e7fff640 1 -- 172.21.15.33:0/3956790346 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83ec011d70 con 0x5611061f4830 2024-05-15T14:07:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.058+0000 7f83e7fff640 1 -- 172.21.15.33:0/3956790346 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec004cd0 con 0x5611061f4830 2024-05-15T14:07:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.059+0000 7f84067ab740 1 -- 172.21.15.33:0/3956790346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611061f4830 msgr2=0x5611061f4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.059+0000 7f84067ab740 1 --2- 172.21.15.33:0/3956790346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611061f4830 0x5611061f4c10 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f83ec013970 tx=0x7f83ec00abf0 comp rx=0 tx=0).stop 2024-05-15T14:07:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.059+0000 7f840182f640 1 --2- 172.21.15.33:0/1320541603 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f83e80b2950 0x7f83e80b4e10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f83f8009d20 tx=0x7f83f8005f90 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:07:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.059+0000 7f84067ab740 1 -- 172.21.15.33:0/3956790346 shutdown_connections 2024-05-15T14:07:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.059+0000 7f84067ab740 1 --2- 172.21.15.33:0/3956790346 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561106203ee0 0x5611062062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.059+0000 7f84067ab740 1 --2- 172.21.15.33:0/3956790346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611061f5150 0x5611062039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.059+0000 7f84067ab740 1 --2- 172.21.15.33:0/3956790346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611061f4830 0x5611061f4c10 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.059+0000 7f84067ab740 1 -- 172.21.15.33:0/3956790346 >> 172.21.15.33:0/3956790346 conn(0x5611061f0e00 msgr2=0x5611061f11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:07:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.059+0000 7f84067ab740 1 -- 172.21.15.33:0/3956790346 shutdown_connections 2024-05-15T14:07:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.059+0000 7f84067ab740 1 -- 172.21.15.33:0/3956790346 wait complete. 2024-05-15T14:07:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.060+0000 7f84067ab740 1 -- start start 2024-05-15T14:07:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.060+0000 7f84067ab740 1 Processor -- start 2024-05-15T14:07:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.060+0000 7f84067ab740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5611061f4830 0x5611062038a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.060+0000 7f84067ab740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611061f5150 0x561106200b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.060+0000 7f84067ab740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561106203ee0 0x5611062011a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.060+0000 7f84067ab740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561105e3d790 con 0x561106203ee0 2024-05-15T14:07:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.060+0000 7f84067ab740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561106113890 con 0x5611061f5150 2024-05-15T14:07:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.060+0000 7f84067ab740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x561106206b50 con 0x5611061f4830 2024-05-15T14:07:39.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.060+0000 7f84067ab740 10 client.0 _ll_register_callbacks cb 0x561106116d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:07:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.060+0000 7f8402030640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561106203ee0 0x5611062011a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.060+0000 7f8402030640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561106203ee0 0x5611062011a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44128/0 (socket says 172.21.15.33:44128) 2024-05-15T14:07:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.060+0000 7f8402030640 1 -- 172.21.15.33:0/483410852 learned_addr learned my addr 172.21.15.33:0/483410852 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:07:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.060+0000 7f840182f640 1 --2- 172.21.15.33:0/483410852 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5611061f4830 0x5611062038a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.060+0000 7f840102e640 1 --2- 172.21.15.33:0/483410852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611061f5150 0x561106200b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.061+0000 7f8402030640 1 -- 172.21.15.33:0/483410852 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5611061f4830 msgr2=0x5611062038a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.061+0000 7f8402030640 1 --2- 172.21.15.33:0/483410852 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5611061f4830 0x5611062038a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.061+0000 7f8402030640 1 -- 172.21.15.33:0/483410852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611061f5150 msgr2=0x561106200b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.061+0000 7f8402030640 1 --2- 172.21.15.33:0/483410852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611061f5150 0x561106200b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.061+0000 7f8402030640 1 -- 172.21.15.33:0/483410852 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561105f38620 con 0x561106203ee0 2024-05-15T14:07:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.061+0000 7f840102e640 1 --2- 172.21.15.33:0/483410852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611061f5150 0x561106200b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:07:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.061+0000 7f8402030640 1 --2- 172.21.15.33:0/483410852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561106203ee0 0x5611062011a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f83f40ca610 tx=0x7f83f40307d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a20c16ef07a8d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:07:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.061+0000 7f83e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:07:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.061+0000 7f83e57fa640 1 -- 172.21.15.33:0/483410852 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f4004840 con 0x561106203ee0 2024-05-15T14:07:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.061+0000 7f83e57fa640 1 -- 172.21.15.33:0/483410852 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8542+0+0 (secure 0 0 0) 0x7f83f40c6ba0 con 0x561106203ee0 2024-05-15T14:07:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.061+0000 7f84067ab740 1 -- 172.21.15.33:0/483410852 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83ec011d70 con 0x561106203ee0 2024-05-15T14:07:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.061+0000 7f83e57fa640 1 -- 172.21.15.33:0/483410852 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f83f40ca420 con 0x561106203ee0 2024-05-15T14:07:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.061+0000 7f84067ab740 10 client.16239 fetch_fsmap learned FSMap version 30 2024-05-15T14:07:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.061+0000 7f84067ab740 1 -- 172.21.15.33:0/483410852 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561106212660 con 0x561106203ee0 2024-05-15T14:07:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.061+0000 7f83e57fa640 1 -- 172.21.15.33:0/483410852 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83ec011d70 con 0x561106203ee0 2024-05-15T14:07:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.061+0000 7f83e57fa640 1 -- 172.21.15.33:0/483410852 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7f83f404d020 con 0x561106203ee0 2024-05-15T14:07:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.062+0000 7f84067ab740 10 client.16239 fetch_fsmap finished waiting for FSMap version 30 2024-05-15T14:07:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.062+0000 7f84067ab740 10 client.16239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:07:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.062+0000 7f84067ab740 20 client.16239 populate_metadata read hostname 'smithi033' 2024-05-15T14:07:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.062+0000 7f84067ab740 1 --2- 172.21.15.33:0/483410852 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x561106213b60 0x561106215f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.062+0000 7f84067ab740 4 client.16239 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:07:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.062+0000 7f84067ab740 1 -- 172.21.15.33:0/483410852 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561106169b80 con 0x561106213b60 2024-05-15T14:07:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.062+0000 7f840182f640 1 --2- 172.21.15.33:0/483410852 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x561106213b60 0x561106215f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.062+0000 7f840182f640 1 --2- 172.21.15.33:0/483410852 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x561106213b60 0x561106215f50 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b6b67c86328fdd4 server_cookie=774909fc5b3adb17 in_seq=0 out_seq=0 2024-05-15T14:07:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.062+0000 7f83e57fa640 10 client.16239 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:07:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.063+0000 7f83e57fa640 1 -- 172.21.15.33:0/483410852 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561106169b80 con 0x561106213b60 2024-05-15T14:07:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.063+0000 7f83e57fa640 10 client.16239 handle_command_reply: tid=0 2024-05-15T14:07:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.063+0000 7f84067ab740 1 client.16239 shutdown 2024-05-15T14:07:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.063+0000 7f84067ab740 1 -- 172.21.15.33:0/483410852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561106203ee0 msgr2=0x5611062011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.063+0000 7f84067ab740 1 --2- 172.21.15.33:0/483410852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561106203ee0 0x5611062011a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f83f40ca610 tx=0x7f83f40307d0 comp rx=0 tx=0).stop 2024-05-15T14:07:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.063+0000 7f84067ab740 1 -- 172.21.15.33:0/483410852 shutdown_connections 2024-05-15T14:07:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.063+0000 7f84067ab740 1 --2- 172.21.15.33:0/483410852 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x561106213b60 0x561106215f50 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.063+0000 7f84067ab740 1 --2- 172.21.15.33:0/483410852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561106203ee0 0x5611062011a0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.063+0000 7f84067ab740 1 --2- 172.21.15.33:0/483410852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611061f5150 0x561106200b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.063+0000 7f83e57fa640 0 client.16239 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:07:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.063+0000 7f84067ab740 1 --2- 172.21.15.33:0/483410852 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5611061f4830 0x5611062038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.063+0000 7f84067ab740 1 -- 172.21.15.33:0/483410852 >> 172.21.15.33:0/483410852 conn(0x5611061f0e00 msgr2=0x5611061f0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:07:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.063+0000 7f84067ab740 1 -- 172.21.15.33:0/483410852 shutdown_connections 2024-05-15T14:07:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.063+0000 7f84067ab740 1 -- 172.21.15.33:0/483410852 wait complete. 2024-05-15T14:07:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.063+0000 7f84067ab740 20 client.16239 trim_cache size 0 max 16384 2024-05-15T14:07:39.074 INFO:tasks.workunit.client.0.smithi033.stdout: 1 461705 6.97 MB/sec execute 194 sec latency 4309.464 ms 2024-05-15T14:07:39.082 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.079+0000 7f84067ab740 1 Processor -- start 2024-05-15T14:07:39.082 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.079+0000 7f84067ab740 1 -- start start 2024-05-15T14:07:39.082 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.079+0000 7f84067ab740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5611061f4830 0x5611061fc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:39.082 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.079+0000 7f84067ab740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611061f5150 0x56110620fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:39.082 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.079+0000 7f84067ab740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561106203ee0 0x5611062033d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:39.082 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.079+0000 7f84067ab740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc1aed00 con 0x561106203ee0 2024-05-15T14:07:39.082 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.079+0000 7f84067ab740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc1ac5a0 con 0x5611061f5150 2024-05-15T14:07:39.082 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.079+0000 7f84067ab740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc1aa410 con 0x5611061f4830 2024-05-15T14:07:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.079+0000 7f8402030640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561106203ee0 0x5611062033d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.079+0000 7f8402030640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561106203ee0 0x5611062033d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44134/0 (socket says 172.21.15.33:44134) 2024-05-15T14:07:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.079+0000 7f8402030640 1 -- 172.21.15.33:0/1621707356 learned_addr learned my addr 172.21.15.33:0/1621707356 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:07:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.079+0000 7f840182f640 1 --2- 172.21.15.33:0/1621707356 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5611061f4830 0x5611061fc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.079+0000 7f840102e640 1 --2- 172.21.15.33:0/1621707356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611061f5150 0x56110620fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.079+0000 7f8402030640 1 -- 172.21.15.33:0/1621707356 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5611061f4830 msgr2=0x5611061fc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.079+0000 7f8402030640 1 --2- 172.21.15.33:0/1621707356 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5611061f4830 0x5611061fc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.079+0000 7f8402030640 1 -- 172.21.15.33:0/1621707356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611061f5150 msgr2=0x56110620fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.079+0000 7f8402030640 1 --2- 172.21.15.33:0/1621707356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611061f5150 0x56110620fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.079+0000 7f8402030640 1 -- 172.21.15.33:0/1621707356 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611061f4f80 con 0x561106203ee0 2024-05-15T14:07:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.079+0000 7f8402030640 1 --2- 172.21.15.33:0/1621707356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561106203ee0 0x5611062033d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f83f4009430 tx=0x7f83f402a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f07c2c890eca197 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:07:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.080+0000 7f83e77fe640 1 -- 172.21.15.33:0/1621707356 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f40ca080 con 0x561106203ee0 2024-05-15T14:07:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.080+0000 7f83e77fe640 1 -- 172.21.15.33:0/1621707356 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83f4036030 con 0x561106203ee0 2024-05-15T14:07:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.080+0000 7f83e77fe640 1 -- 172.21.15.33:0/1621707356 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f4090180 con 0x561106203ee0 2024-05-15T14:07:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.080+0000 7f84067ab740 1 -- 172.21.15.33:0/1621707356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561106203ee0 msgr2=0x5611062033d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.080+0000 7f84067ab740 1 --2- 172.21.15.33:0/1621707356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561106203ee0 0x5611062033d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f83f4009430 tx=0x7f83f402a8e0 comp rx=0 tx=0).stop 2024-05-15T14:07:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.080+0000 7f84067ab740 1 -- 172.21.15.33:0/1621707356 shutdown_connections 2024-05-15T14:07:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.080+0000 7f84067ab740 1 --2- 172.21.15.33:0/1621707356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561106203ee0 0x5611062033d0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.080+0000 7f84067ab740 1 --2- 172.21.15.33:0/1621707356 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611061f5150 0x56110620fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.080+0000 7f84067ab740 1 --2- 172.21.15.33:0/1621707356 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5611061f4830 0x5611061fc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.080+0000 7f84067ab740 1 -- 172.21.15.33:0/1621707356 >> 172.21.15.33:0/1621707356 conn(0x5611061f0e00 msgr2=0x5611061fb940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:07:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.080+0000 7f84067ab740 1 -- 172.21.15.33:0/1621707356 shutdown_connections 2024-05-15T14:07:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.080+0000 7f84067ab740 1 -- 172.21.15.33:0/1621707356 wait complete. 2024-05-15T14:07:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.081+0000 7f84067ab740 1 -- start start 2024-05-15T14:07:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.081+0000 7f84067ab740 1 Processor -- start 2024-05-15T14:07:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.081+0000 7f84067ab740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611061f4830 0x5611062073f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.081+0000 7f84067ab740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611061f5150 0x561106207930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.081+0000 7f84067ab740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561106203ee0 0x561106276440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.081+0000 7f84067ab740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc1aaa60 con 0x5611061f5150 2024-05-15T14:07:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.081+0000 7f84067ab740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc1abed0 con 0x5611061f4830 2024-05-15T14:07:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.081+0000 7f84067ab740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc1ab130 con 0x561106203ee0 2024-05-15T14:07:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.081+0000 7f84067ab740 10 client.0 _ll_register_callbacks cb 0x7f83f40c6810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:07:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.081+0000 7f840102e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611061f5150 0x561106207930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.081+0000 7f840102e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611061f5150 0x561106207930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44148/0 (socket says 172.21.15.33:44148) 2024-05-15T14:07:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.081+0000 7f840102e640 1 -- 172.21.15.33:0/2082241990 learned_addr learned my addr 172.21.15.33:0/2082241990 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:07:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.081+0000 7f8402030640 1 --2- 172.21.15.33:0/2082241990 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561106203ee0 0x561106276440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.081+0000 7f840182f640 1 --2- 172.21.15.33:0/2082241990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611061f4830 0x5611062073f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.081+0000 7f840102e640 1 -- 172.21.15.33:0/2082241990 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561106203ee0 msgr2=0x561106276440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.081+0000 7f840102e640 1 --2- 172.21.15.33:0/2082241990 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561106203ee0 0x561106276440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.081+0000 7f840102e640 1 -- 172.21.15.33:0/2082241990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611061f4830 msgr2=0x5611062073f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.081+0000 7f840102e640 1 --2- 172.21.15.33:0/2082241990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611061f4830 0x5611062073f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.082+0000 7f840102e640 1 -- 172.21.15.33:0/2082241990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561106276a30 con 0x5611061f5150 2024-05-15T14:07:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.082+0000 7f840102e640 1 --2- 172.21.15.33:0/2082241990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611061f5150 0x561106207930 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x5611062003f0 tx=0x7f83ec005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c554e7f37d5029b server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:07:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.082+0000 7f83e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:07:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.082+0000 7f83e57fa640 1 -- 172.21.15.33:0/2082241990 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec02b060 con 0x5611061f5150 2024-05-15T14:07:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.082+0000 7f83e57fa640 1 -- 172.21.15.33:0/2082241990 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8542+0+0 (secure 0 0 0) 0x7f83ec005510 con 0x5611061f5150 2024-05-15T14:07:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.082+0000 7f84067ab740 1 -- 172.21.15.33:0/2082241990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83f4036030 con 0x5611061f5150 2024-05-15T14:07:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.082+0000 7f83e57fa640 1 -- 172.21.15.33:0/2082241990 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f83ec019040 con 0x5611061f5150 2024-05-15T14:07:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.082+0000 7f84067ab740 10 client.16245 fetch_fsmap learned FSMap version 30 2024-05-15T14:07:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.082+0000 7f84067ab740 1 -- 172.21.15.33:0/2082241990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56110627ddc0 con 0x5611061f5150 2024-05-15T14:07:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.082+0000 7f83e57fa640 1 -- 172.21.15.33:0/2082241990 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83f4036030 con 0x5611061f5150 2024-05-15T14:07:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.082+0000 7f83e57fa640 1 -- 172.21.15.33:0/2082241990 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7f83ec01d6b0 con 0x5611061f5150 2024-05-15T14:07:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.082+0000 7f84067ab740 10 client.16245 fetch_fsmap finished waiting for FSMap version 30 2024-05-15T14:07:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.082+0000 7f84067ab740 10 client.16245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:07:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.082+0000 7f84067ab740 20 client.16245 populate_metadata read hostname 'smithi033' 2024-05-15T14:07:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.082+0000 7f84067ab740 1 --2- 172.21.15.33:0/2082241990 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56110627e1d0 0x5611062805f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.082+0000 7f84067ab740 4 client.16245 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:07:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.082+0000 7f84067ab740 1 -- 172.21.15.33:0/2082241990 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561106163360 con 0x56110627e1d0 2024-05-15T14:07:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.083+0000 7f840182f640 1 --2- 172.21.15.33:0/2082241990 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56110627e1d0 0x5611062805f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.083+0000 7f83e57fa640 10 client.16245 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:07:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.083+0000 7f840182f640 1 --2- 172.21.15.33:0/2082241990 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56110627e1d0 0x5611062805f0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e9c457c30bce7ab server_cookie=cf0bc020dcb9d168 in_seq=0 out_seq=0 2024-05-15T14:07:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.126+0000 7f83e57fa640 1 -- 172.21.15.33:0/2082241990 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561106163360 con 0x56110627e1d0 2024-05-15T14:07:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.126+0000 7f83e57fa640 10 client.16245 handle_command_reply: tid=0 2024-05-15T14:07:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.126+0000 7f84067ab740 1 client.16245 shutdown 2024-05-15T14:07:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.126+0000 7f84067ab740 1 -- 172.21.15.33:0/2082241990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611061f5150 msgr2=0x561106207930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.126+0000 7f84067ab740 1 --2- 172.21.15.33:0/2082241990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611061f5150 0x561106207930 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x5611062003f0 tx=0x7f83ec005c50 comp rx=0 tx=0).stop 2024-05-15T14:07:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.126+0000 7f84067ab740 1 -- 172.21.15.33:0/2082241990 shutdown_connections 2024-05-15T14:07:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.126+0000 7f84067ab740 1 --2- 172.21.15.33:0/2082241990 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56110627e1d0 0x5611062805f0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.126+0000 7f84067ab740 1 --2- 172.21.15.33:0/2082241990 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561106203ee0 0x561106276440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.126+0000 7f84067ab740 1 --2- 172.21.15.33:0/2082241990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611061f5150 0x561106207930 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.126+0000 7f83e57fa640 0 client.16245 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:07:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.126+0000 7f84067ab740 1 --2- 172.21.15.33:0/2082241990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611061f4830 0x5611062073f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.126+0000 7f84067ab740 1 -- 172.21.15.33:0/2082241990 >> 172.21.15.33:0/2082241990 conn(0x5611061f0e00 msgr2=0x5611061fc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:07:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.126+0000 7f84067ab740 1 -- 172.21.15.33:0/2082241990 shutdown_connections 2024-05-15T14:07:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.126+0000 7f84067ab740 1 -- 172.21.15.33:0/2082241990 wait complete. 2024-05-15T14:07:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.126+0000 7f84067ab740 20 client.16245 trim_cache size 0 max 16384 2024-05-15T14:07:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:07:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:07:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:07:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.127+0000 7f8403a93640 1 -- 172.21.15.33:0/1320541603 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f83e80b2950 msgr2=0x7f83e80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.127+0000 7f8403a93640 1 --2- 172.21.15.33:0/1320541603 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f83e80b2950 0x7f83e80b4e10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f83f8009d20 tx=0x7f83f8005f90 comp rx=0 tx=0).stop 2024-05-15T14:07:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.127+0000 7f8403a93640 1 -- 172.21.15.33:0/1320541603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83fc108a40 msgr2=0x7f83fc19da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.127+0000 7f8403a93640 1 --2- 172.21.15.33:0/1320541603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83fc108a40 0x7f83fc19da90 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f83f4033040 tx=0x7f83f4003970 comp rx=0 tx=0).stop 2024-05-15T14:07:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.127+0000 7f8403a93640 1 -- 172.21.15.33:0/1320541603 shutdown_connections 2024-05-15T14:07:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.127+0000 7f8403a93640 1 --2- 172.21.15.33:0/1320541603 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f83e80b2950 0x7f83e80b4e10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.127+0000 7f8403a93640 1 --2- 172.21.15.33:0/1320541603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83fc108a40 0x7f83fc19da90 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.127+0000 7f8403a93640 1 --2- 172.21.15.33:0/1320541603 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83fc1033f0 0x7f83fc199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.127+0000 7f8403a93640 1 --2- 172.21.15.33:0/1320541603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f83fc102a30 0x7f83fc199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.127+0000 7f8403a93640 1 -- 172.21.15.33:0/1320541603 >> 172.21.15.33:0/1320541603 conn(0x7f83fc0fe4e0 msgr2=0x7f83fc0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:07:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.127+0000 7f8403a93640 1 -- 172.21.15.33:0/1320541603 shutdown_connections 2024-05-15T14:07:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:39.127+0000 7f8403a93640 1 -- 172.21.15.33:0/1320541603 wait complete. 2024-05-15T14:07:40.065 INFO:tasks.workunit.client.0.smithi033.stdout: 1 461705 6.94 MB/sec execute 195 sec latency 5309.493 ms 2024-05-15T14:07:40.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:40 smithi052 ceph-mon[36190]: pgmap v225: 97 pgs: 97 active+clean; 133 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 2.1 MiB/s wr, 77 op/s 2024-05-15T14:07:40.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:40 smithi081 ceph-mon[36434]: pgmap v225: 97 pgs: 97 active+clean; 133 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 2.1 MiB/s wr, 77 op/s 2024-05-15T14:07:40.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:40 smithi033 ceph-mon[31650]: pgmap v225: 97 pgs: 97 active+clean; 133 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 2.1 MiB/s wr, 77 op/s 2024-05-15T14:07:41.065 INFO:tasks.workunit.client.0.smithi033.stdout: 1 461705 6.90 MB/sec execute 196 sec latency 6309.586 ms 2024-05-15T14:07:42.065 INFO:tasks.workunit.client.0.smithi033.stdout: 1 461705 6.87 MB/sec execute 197 sec latency 7309.680 ms 2024-05-15T14:07:42.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:42 smithi052 ceph-mon[36190]: pgmap v226: 97 pgs: 97 active+clean; 133 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 2.1 MiB/s wr, 77 op/s 2024-05-15T14:07:42.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:42 smithi081 ceph-mon[36434]: pgmap v226: 97 pgs: 97 active+clean; 133 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 2.1 MiB/s wr, 77 op/s 2024-05-15T14:07:42.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:42 smithi033 ceph-mon[31650]: pgmap v226: 97 pgs: 97 active+clean; 133 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 2.1 MiB/s wr, 77 op/s 2024-05-15T14:07:43.065 INFO:tasks.workunit.client.0.smithi033.stdout: 1 461705 6.83 MB/sec execute 198 sec latency 8309.719 ms 2024-05-15T14:07:44.065 INFO:tasks.workunit.client.0.smithi033.stdout: 1 461705 6.80 MB/sec execute 199 sec latency 9309.805 ms 2024-05-15T14:07:44.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:44 smithi052 ceph-mon[36190]: pgmap v227: 97 pgs: 97 active+clean; 134 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 2.8 MiB/s wr, 125 op/s 2024-05-15T14:07:44.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:44 smithi081 ceph-mon[36434]: pgmap v227: 97 pgs: 97 active+clean; 134 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 2.8 MiB/s wr, 125 op/s 2024-05-15T14:07:44.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:44 smithi033 ceph-mon[31650]: pgmap v227: 97 pgs: 97 active+clean; 134 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 2.8 MiB/s wr, 125 op/s 2024-05-15T14:07:45.065 INFO:tasks.workunit.client.0.smithi033.stdout: 1 461705 6.77 MB/sec execute 200 sec latency 10309.840 ms 2024-05-15T14:07:45.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:45 smithi081 ceph-mon[36434]: pgmap v228: 97 pgs: 97 active+clean; 134 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 2.2 MiB/s wr, 103 op/s 2024-05-15T14:07:45.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:45 smithi033 ceph-mon[31650]: pgmap v228: 97 pgs: 97 active+clean; 134 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 2.2 MiB/s wr, 103 op/s 2024-05-15T14:07:46.065 INFO:tasks.workunit.client.0.smithi033.stdout: 1 461705 6.73 MB/sec execute 201 sec latency 11309.884 ms 2024-05-15T14:07:46.090 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:45 smithi052 ceph-mon[36190]: pgmap v228: 97 pgs: 97 active+clean; 134 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 2.2 MiB/s wr, 103 op/s 2024-05-15T14:07:47.066 INFO:tasks.workunit.client.0.smithi033.stdout: 1 461705 6.70 MB/sec execute 202 sec latency 12310.034 ms 2024-05-15T14:07:48.066 INFO:tasks.workunit.client.0.smithi033.stdout: 1 461705 6.67 MB/sec execute 203 sec latency 13310.132 ms 2024-05-15T14:07:48.591 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:48 smithi052 ceph-mon[36190]: pgmap v229: 97 pgs: 97 active+clean; 133 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 2.9 MiB/s wr, 131 op/s 2024-05-15T14:07:48.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:48 smithi081 ceph-mon[36434]: pgmap v229: 97 pgs: 97 active+clean; 133 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 2.9 MiB/s wr, 131 op/s 2024-05-15T14:07:48.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:48 smithi033 ceph-mon[31650]: pgmap v229: 97 pgs: 97 active+clean; 133 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 2.9 MiB/s wr, 131 op/s 2024-05-15T14:07:49.066 INFO:tasks.workunit.client.0.smithi033.stdout: 1 461705 6.63 MB/sec execute 204 sec latency 14310.250 ms 2024-05-15T14:07:50.066 INFO:tasks.workunit.client.0.smithi033.stdout: 1 461705 6.60 MB/sec execute 205 sec latency 15310.399 ms 2024-05-15T14:07:50.590 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:50 smithi052 ceph-mon[36190]: pgmap v230: 97 pgs: 97 active+clean; 133 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 1.4 MiB/s wr, 75 op/s 2024-05-15T14:07:50.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:50 smithi081 ceph-mon[36434]: pgmap v230: 97 pgs: 97 active+clean; 133 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 1.4 MiB/s wr, 75 op/s 2024-05-15T14:07:50.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:50 smithi033 ceph-mon[31650]: pgmap v230: 97 pgs: 97 active+clean; 133 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 1.4 MiB/s wr, 75 op/s 2024-05-15T14:07:51.066 INFO:tasks.workunit.client.0.smithi033.stdout: 1 461705 6.57 MB/sec execute 206 sec latency 16310.509 ms 2024-05-15T14:07:52.066 INFO:tasks.workunit.client.0.smithi033.stdout: 1 461705 6.54 MB/sec execute 207 sec latency 17310.585 ms 2024-05-15T14:07:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.229+0000 7f89718b0640 1 -- 172.21.15.33:0/564690708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f896c102310 msgr2=0x7f896c10e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.229+0000 7f89718b0640 1 --2- 172.21.15.33:0/564690708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f896c102310 0x7f896c10e820 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8964009cd0 tx=0x7f896402a8e0 comp rx=0 tx=0).stop 2024-05-15T14:07:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.229+0000 7f89718b0640 1 -- 172.21.15.33:0/564690708 shutdown_connections 2024-05-15T14:07:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.229+0000 7f89718b0640 1 --2- 172.21.15.33:0/564690708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f896c102310 0x7f896c10e820 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.229+0000 7f89718b0640 1 --2- 172.21.15.33:0/564690708 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f896c101950 0x7f896c101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.229+0000 7f89718b0640 1 --2- 172.21.15.33:0/564690708 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f896c107960 0x7f896c107d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.229+0000 7f89718b0640 1 -- 172.21.15.33:0/564690708 >> 172.21.15.33:0/564690708 conn(0x7f896c078ad0 msgr2=0x7f896c078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:07:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.229+0000 7f89718b0640 1 -- 172.21.15.33:0/564690708 shutdown_connections 2024-05-15T14:07:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.229+0000 7f89718b0640 1 -- 172.21.15.33:0/564690708 wait complete. 2024-05-15T14:07:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.229+0000 7f89718b0640 1 Processor -- start 2024-05-15T14:07:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.229+0000 7f89718b0640 1 -- start start 2024-05-15T14:07:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.230+0000 7f89718b0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f896c101950 0x7f896c199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.230+0000 7f89718b0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f896c102310 0x7f896c199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.230+0000 7f89718b0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f896c107960 0x7f896c19dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.230+0000 7f89718b0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f896c111c20 con 0x7f896c102310 2024-05-15T14:07:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.230+0000 7f89718b0640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f896c111aa0 con 0x7f896c107960 2024-05-15T14:07:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.230+0000 7f89718b0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f896c111da0 con 0x7f896c101950 2024-05-15T14:07:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.230+0000 7f8963fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f896c102310 0x7f896c199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.230+0000 7f8963fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f896c102310 0x7f896c199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36460/0 (socket says 172.21.15.33:36460) 2024-05-15T14:07:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.230+0000 7f8963fff640 1 -- 172.21.15.33:0/3967842795 learned_addr learned my addr 172.21.15.33:0/3967842795 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:07:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.230+0000 7f896affd640 1 --2- 172.21.15.33:0/3967842795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f896c101950 0x7f896c199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.230+0000 7f896b7fe640 1 --2- 172.21.15.33:0/3967842795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f896c107960 0x7f896c19dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.230+0000 7f8963fff640 1 -- 172.21.15.33:0/3967842795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f896c101950 msgr2=0x7f896c199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.230+0000 7f8963fff640 1 --2- 172.21.15.33:0/3967842795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f896c101950 0x7f896c199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.230+0000 7f8963fff640 1 -- 172.21.15.33:0/3967842795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f896c107960 msgr2=0x7f896c19dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.230+0000 7f8963fff640 1 --2- 172.21.15.33:0/3967842795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f896c107960 0x7f896c19dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.230+0000 7f8963fff640 1 -- 172.21.15.33:0/3967842795 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f896c19e200 con 0x7f896c102310 2024-05-15T14:07:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.230+0000 7f8963fff640 1 --2- 172.21.15.33:0/3967842795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f896c102310 0x7f896c199750 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f895400d950 tx=0x7f895400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:07:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.230+0000 7f89708ae640 1 -- 172.21.15.33:0/3967842795 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8954017030 con 0x7f896c102310 2024-05-15T14:07:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.230+0000 7f89708ae640 1 -- 172.21.15.33:0/3967842795 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8954004020 con 0x7f896c102310 2024-05-15T14:07:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.230+0000 7f89718b0640 1 -- 172.21.15.33:0/3967842795 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f896c19e4f0 con 0x7f896c102310 2024-05-15T14:07:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.230+0000 7f89708ae640 1 -- 172.21.15.33:0/3967842795 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89540047a0 con 0x7f896c102310 2024-05-15T14:07:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.231+0000 7f89718b0640 1 -- 172.21.15.33:0/3967842795 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f896c1a5d80 con 0x7f896c102310 2024-05-15T14:07:52.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.233+0000 7f89708ae640 1 -- 172.21.15.33:0/3967842795 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8954009a30 con 0x7f896c102310 2024-05-15T14:07:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.233+0000 7f89745cb740 1 Processor -- start 2024-05-15T14:07:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.234+0000 7f89745cb740 1 -- start start 2024-05-15T14:07:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.234+0000 7f89745cb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f11a0d8f0 0x563f11a0dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.234+0000 7f89745cb740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f11a0e210 0x563f11a18590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.234+0000 7f89745cb740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563f11a18ad0 0x563f11a1aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.234+0000 7f89745cb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563f11980c30 con 0x563f11a0d8f0 2024-05-15T14:07:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.234+0000 7f89745cb740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563f119707d0 con 0x563f11a0e210 2024-05-15T14:07:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.234+0000 7f89745cb740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x563f11974810 con 0x563f11a18ad0 2024-05-15T14:07:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.234+0000 7f896b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f11a0d8f0 0x563f11a0dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.234+0000 7f896b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f11a0d8f0 0x563f11a0dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36474/0 (socket says 172.21.15.33:36474) 2024-05-15T14:07:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.234+0000 7f896b7fe640 1 -- 172.21.15.33:0/104488685 learned_addr learned my addr 172.21.15.33:0/104488685 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:07:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.234+0000 7f8963fff640 1 --2- 172.21.15.33:0/104488685 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563f11a18ad0 0x563f11a1aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.234+0000 7f896affd640 1 --2- 172.21.15.33:0/104488685 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f11a0e210 0x563f11a18590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.234+0000 7f896b7fe640 1 -- 172.21.15.33:0/104488685 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563f11a18ad0 msgr2=0x563f11a1aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.234+0000 7f896b7fe640 1 --2- 172.21.15.33:0/104488685 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563f11a18ad0 0x563f11a1aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.234+0000 7f896b7fe640 1 -- 172.21.15.33:0/104488685 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f11a0e210 msgr2=0x563f11a18590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.234+0000 7f896b7fe640 1 --2- 172.21.15.33:0/104488685 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f11a0e210 0x563f11a18590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.234+0000 7f896b7fe640 1 -- 172.21.15.33:0/104488685 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f11685210 con 0x563f11a0d8f0 2024-05-15T14:07:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.234+0000 7f8963fff640 1 --2- 172.21.15.33:0/104488685 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563f11a18ad0 0x563f11a1aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:07:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.234+0000 7f89708ae640 1 --2- 172.21.15.33:0/3967842795 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f89580b29d0 0x7f89580b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.234+0000 7f896affd640 1 --2- 172.21.15.33:0/3967842795 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f89580b29d0 0x7f89580b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.235+0000 7f896b7fe640 1 --2- 172.21.15.33:0/104488685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f11a0d8f0 0x563f11a0dcd0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f8964035000 tx=0x7f8964009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e9422e5e4eb945 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:07:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.235+0000 7f89708ae640 1 -- 172.21.15.33:0/3967842795 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8542+0+0 (secure 0 0 0) 0x7f89540e20c0 con 0x7f896c102310 2024-05-15T14:07:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.235+0000 7f8961ffb640 1 -- 172.21.15.33:0/104488685 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8964031850 con 0x563f11a0d8f0 2024-05-15T14:07:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.235+0000 7f8961ffb640 1 -- 172.21.15.33:0/104488685 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8964036040 con 0x563f11a0d8f0 2024-05-15T14:07:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.235+0000 7f8961ffb640 1 -- 172.21.15.33:0/104488685 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89640044a0 con 0x563f11a0d8f0 2024-05-15T14:07:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.235+0000 7f89745cb740 1 -- 172.21.15.33:0/104488685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f11a0d8f0 msgr2=0x563f11a0dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.235+0000 7f896affd640 1 --2- 172.21.15.33:0/3967842795 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f89580b29d0 0x7f89580b4e90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x563f11a0ee90 tx=0x7f895c005f90 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:07:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.235+0000 7f89745cb740 1 --2- 172.21.15.33:0/104488685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f11a0d8f0 0x563f11a0dcd0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f8964035000 tx=0x7f8964009a20 comp rx=0 tx=0).stop 2024-05-15T14:07:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.236+0000 7f89745cb740 1 -- 172.21.15.33:0/104488685 shutdown_connections 2024-05-15T14:07:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.236+0000 7f89745cb740 1 --2- 172.21.15.33:0/104488685 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563f11a18ad0 0x563f11a1aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.236+0000 7f89745cb740 1 --2- 172.21.15.33:0/104488685 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f11a0e210 0x563f11a18590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.236+0000 7f89745cb740 1 --2- 172.21.15.33:0/104488685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f11a0d8f0 0x563f11a0dcd0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.236+0000 7f89745cb740 1 -- 172.21.15.33:0/104488685 >> 172.21.15.33:0/104488685 conn(0x563f11a09e00 msgr2=0x563f11a0a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:07:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.236+0000 7f89745cb740 1 -- 172.21.15.33:0/104488685 shutdown_connections 2024-05-15T14:07:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.236+0000 7f89745cb740 1 -- 172.21.15.33:0/104488685 wait complete. 2024-05-15T14:07:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.236+0000 7f89745cb740 1 -- start start 2024-05-15T14:07:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.236+0000 7f89745cb740 1 Processor -- start 2024-05-15T14:07:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.236+0000 7f89745cb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f11a0d8f0 0x563f11a18490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.236+0000 7f89745cb740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f11a0e210 0x563f11a15780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.236+0000 7f89745cb740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563f11a18ad0 0x563f11a15d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.236+0000 7f89745cb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563f11656790 con 0x563f11a0d8f0 2024-05-15T14:07:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.237+0000 7f89745cb740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563f1192c890 con 0x563f11a0e210 2024-05-15T14:07:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.237+0000 7f89745cb740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x563f11a1b740 con 0x563f11a18ad0 2024-05-15T14:07:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.237+0000 7f89745cb740 10 client.0 _ll_register_callbacks cb 0x563f1192fd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:07:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.237+0000 7f896affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f11a0d8f0 0x563f11a18490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.237+0000 7f896affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f11a0d8f0 0x563f11a18490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36476/0 (socket says 172.21.15.33:36476) 2024-05-15T14:07:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.237+0000 7f896affd640 1 -- 172.21.15.33:0/2410134383 learned_addr learned my addr 172.21.15.33:0/2410134383 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:07:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.237+0000 7f8963fff640 1 --2- 172.21.15.33:0/2410134383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f11a0e210 0x563f11a15780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.237+0000 7f896b7fe640 1 --2- 172.21.15.33:0/2410134383 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563f11a18ad0 0x563f11a15d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.237+0000 7f896affd640 1 -- 172.21.15.33:0/2410134383 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563f11a18ad0 msgr2=0x563f11a15d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.237+0000 7f896affd640 1 --2- 172.21.15.33:0/2410134383 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563f11a18ad0 0x563f11a15d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.237+0000 7f896affd640 1 -- 172.21.15.33:0/2410134383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f11a0e210 msgr2=0x563f11a15780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.237+0000 7f896affd640 1 --2- 172.21.15.33:0/2410134383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f11a0e210 0x563f11a15780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.237+0000 7f896affd640 1 -- 172.21.15.33:0/2410134383 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f11751620 con 0x563f11a0d8f0 2024-05-15T14:07:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.237+0000 7f896b7fe640 1 --2- 172.21.15.33:0/2410134383 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563f11a18ad0 0x563f11a15d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:07:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.237+0000 7f896affd640 1 --2- 172.21.15.33:0/2410134383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f11a0d8f0 0x563f11a18490 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f895c003970 tx=0x7f895c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b5611895215b6b server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:07:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.237+0000 7f894affd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:07:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.237+0000 7f894affd640 1 -- 172.21.15.33:0/2410134383 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c011d80 con 0x563f11a0d8f0 2024-05-15T14:07:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.237+0000 7f894affd640 1 -- 172.21.15.33:0/2410134383 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8542+0+0 (secure 0 0 0) 0x7f895c01e420 con 0x563f11a0d8f0 2024-05-15T14:07:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.237+0000 7f89745cb740 1 -- 172.21.15.33:0/2410134383 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8964036040 con 0x563f11a0d8f0 2024-05-15T14:07:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.238+0000 7f894affd640 1 -- 172.21.15.33:0/2410134383 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f895c01e620 con 0x563f11a0d8f0 2024-05-15T14:07:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.238+0000 7f89745cb740 10 client.16257 fetch_fsmap learned FSMap version 30 2024-05-15T14:07:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.238+0000 7f89745cb740 1 -- 172.21.15.33:0/2410134383 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563f11a271a0 con 0x563f11a0d8f0 2024-05-15T14:07:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.238+0000 7f894affd640 1 -- 172.21.15.33:0/2410134383 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8964036040 con 0x563f11a0d8f0 2024-05-15T14:07:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.238+0000 7f894affd640 1 -- 172.21.15.33:0/2410134383 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7f895c028830 con 0x563f11a0d8f0 2024-05-15T14:07:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.238+0000 7f89745cb740 10 client.16257 fetch_fsmap finished waiting for FSMap version 30 2024-05-15T14:07:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.238+0000 7f89745cb740 10 client.16257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:07:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.238+0000 7f89745cb740 20 client.16257 populate_metadata read hostname 'smithi033' 2024-05-15T14:07:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.238+0000 7f89745cb740 1 --2- 172.21.15.33:0/2410134383 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x563f11a286d0 0x563f11a2aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.238+0000 7f89745cb740 4 client.16257 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:07:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.238+0000 7f89745cb740 1 -- 172.21.15.33:0/2410134383 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f11982b80 con 0x563f11a286d0 2024-05-15T14:07:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.238+0000 7f8963fff640 1 --2- 172.21.15.33:0/2410134383 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x563f11a286d0 0x563f11a2aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.239+0000 7f8963fff640 1 --2- 172.21.15.33:0/2410134383 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x563f11a286d0 0x563f11a2aac0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d38e8d289973f0b3 server_cookie=ddab52b03ed9ac63 in_seq=0 out_seq=0 2024-05-15T14:07:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.239+0000 7f894affd640 10 client.16257 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:07:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.239+0000 7f894affd640 1 -- 172.21.15.33:0/2410134383 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563f11982b80 con 0x563f11a286d0 2024-05-15T14:07:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.239+0000 7f894affd640 10 client.16257 handle_command_reply: tid=0 2024-05-15T14:07:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.239+0000 7f89745cb740 1 client.16257 shutdown 2024-05-15T14:07:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.240+0000 7f89745cb740 1 -- 172.21.15.33:0/2410134383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f11a0d8f0 msgr2=0x563f11a18490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.240+0000 7f89745cb740 1 --2- 172.21.15.33:0/2410134383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f11a0d8f0 0x563f11a18490 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f895c003970 tx=0x7f895c0046c0 comp rx=0 tx=0).stop 2024-05-15T14:07:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.240+0000 7f89745cb740 1 -- 172.21.15.33:0/2410134383 shutdown_connections 2024-05-15T14:07:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.240+0000 7f89745cb740 1 --2- 172.21.15.33:0/2410134383 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x563f11a286d0 0x563f11a2aac0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.240+0000 7f89745cb740 1 --2- 172.21.15.33:0/2410134383 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563f11a18ad0 0x563f11a15d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.240+0000 7f89745cb740 1 --2- 172.21.15.33:0/2410134383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f11a0e210 0x563f11a15780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.240+0000 7f894affd640 0 client.16257 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:07:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.240+0000 7f89745cb740 1 --2- 172.21.15.33:0/2410134383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f11a0d8f0 0x563f11a18490 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:52.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.240+0000 7f89745cb740 1 -- 172.21.15.33:0/2410134383 >> 172.21.15.33:0/2410134383 conn(0x563f11a09e00 msgr2=0x563f11a09100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:07:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.240+0000 7f89745cb740 1 -- 172.21.15.33:0/2410134383 shutdown_connections 2024-05-15T14:07:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.240+0000 7f89745cb740 1 -- 172.21.15.33:0/2410134383 wait complete. 2024-05-15T14:07:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.240+0000 7f89745cb740 20 client.16257 trim_cache size 0 max 16384 2024-05-15T14:07:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.255+0000 7f89745cb740 1 Processor -- start 2024-05-15T14:07:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.255+0000 7f89745cb740 1 -- start start 2024-05-15T14:07:52.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.255+0000 7f89745cb740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f11a0d8f0 0x563f11a04fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.255+0000 7f89745cb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f11a0e210 0x563f11a24610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.255+0000 7f89745cb740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563f11a18ad0 0x563f11a17fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.255+0000 7f89745cb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f896c1b1050 con 0x563f11a0e210 2024-05-15T14:07:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.255+0000 7f89745cb740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f896c1ae8f0 con 0x563f11a0d8f0 2024-05-15T14:07:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.255+0000 7f89745cb740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f896c1ac760 con 0x563f11a18ad0 2024-05-15T14:07:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.256+0000 7f8963fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f11a0e210 0x563f11a24610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.256+0000 7f8963fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f11a0e210 0x563f11a24610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36492/0 (socket says 172.21.15.33:36492) 2024-05-15T14:07:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.256+0000 7f8963fff640 1 -- 172.21.15.33:0/3722075895 learned_addr learned my addr 172.21.15.33:0/3722075895 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:07:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.256+0000 7f896affd640 1 --2- 172.21.15.33:0/3722075895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f11a0d8f0 0x563f11a04fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.256+0000 7f896b7fe640 1 --2- 172.21.15.33:0/3722075895 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563f11a18ad0 0x563f11a17fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.256+0000 7f8963fff640 1 -- 172.21.15.33:0/3722075895 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563f11a18ad0 msgr2=0x563f11a17fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.256+0000 7f8963fff640 1 --2- 172.21.15.33:0/3722075895 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563f11a18ad0 0x563f11a17fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.256+0000 7f8963fff640 1 -- 172.21.15.33:0/3722075895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f11a0d8f0 msgr2=0x563f11a04fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:52.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.256+0000 7f8963fff640 1 --2- 172.21.15.33:0/3722075895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f11a0d8f0 0x563f11a04fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.256+0000 7f8963fff640 1 -- 172.21.15.33:0/3722075895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f11a0e040 con 0x563f11a0e210 2024-05-15T14:07:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.256+0000 7f896b7fe640 1 --2- 172.21.15.33:0/3722075895 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563f11a18ad0 0x563f11a17fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:07:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.256+0000 7f896affd640 1 --2- 172.21.15.33:0/3722075895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f11a0d8f0 0x563f11a04fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:07:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.256+0000 7f8963fff640 1 --2- 172.21.15.33:0/3722075895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f11a0e210 0x563f11a24610 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f89540a5410 tx=0x7f89540e23c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2fa8db152f13c87 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:07:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.256+0000 7f89617fa640 1 -- 172.21.15.33:0/3722075895 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89540a9a90 con 0x563f11a0e210 2024-05-15T14:07:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.256+0000 7f89617fa640 1 -- 172.21.15.33:0/3722075895 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89540a9390 con 0x563f11a0e210 2024-05-15T14:07:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.256+0000 7f89617fa640 1 -- 172.21.15.33:0/3722075895 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89540041d0 con 0x563f11a0e210 2024-05-15T14:07:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.257+0000 7f89745cb740 1 -- 172.21.15.33:0/3722075895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f11a0e210 msgr2=0x563f11a24610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.257+0000 7f89745cb740 1 --2- 172.21.15.33:0/3722075895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f11a0e210 0x563f11a24610 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f89540a5410 tx=0x7f89540e23c0 comp rx=0 tx=0).stop 2024-05-15T14:07:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.257+0000 7f89745cb740 1 -- 172.21.15.33:0/3722075895 shutdown_connections 2024-05-15T14:07:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.257+0000 7f89745cb740 1 --2- 172.21.15.33:0/3722075895 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563f11a18ad0 0x563f11a17fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.257+0000 7f89745cb740 1 --2- 172.21.15.33:0/3722075895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f11a0e210 0x563f11a24610 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.257+0000 7f89745cb740 1 --2- 172.21.15.33:0/3722075895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f11a0d8f0 0x563f11a04fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.257+0000 7f89745cb740 1 -- 172.21.15.33:0/3722075895 >> 172.21.15.33:0/3722075895 conn(0x563f11a09e00 msgr2=0x563f11a0fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:07:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.257+0000 7f89745cb740 1 -- 172.21.15.33:0/3722075895 shutdown_connections 2024-05-15T14:07:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.257+0000 7f89745cb740 1 -- 172.21.15.33:0/3722075895 wait complete. 2024-05-15T14:07:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.257+0000 7f89745cb740 1 -- start start 2024-05-15T14:07:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.257+0000 7f89745cb740 1 Processor -- start 2024-05-15T14:07:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.258+0000 7f89745cb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f11a0d8f0 0x563f11a20d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.258+0000 7f89745cb740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563f11a0e210 0x563f11a1c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.258+0000 7f89745cb740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f11a18ad0 0x563f11a1c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.258+0000 7f89745cb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f896c1acdb0 con 0x563f11a0d8f0 2024-05-15T14:07:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.258+0000 7f89745cb740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f896c1ae220 con 0x563f11a18ad0 2024-05-15T14:07:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.258+0000 7f89745cb740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f896c1ad480 con 0x563f11a0e210 2024-05-15T14:07:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.258+0000 7f89745cb740 10 client.0 _ll_register_callbacks cb 0x7f895c0035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:07:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.258+0000 7f896affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f11a0d8f0 0x563f11a20d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.258+0000 7f896affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f11a0d8f0 0x563f11a20d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36508/0 (socket says 172.21.15.33:36508) 2024-05-15T14:07:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.258+0000 7f896affd640 1 -- 172.21.15.33:0/3835741909 learned_addr learned my addr 172.21.15.33:0/3835741909 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:07:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.258+0000 7f896b7fe640 1 --2- 172.21.15.33:0/3835741909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f11a18ad0 0x563f11a1c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.258+0000 7f8963fff640 1 --2- 172.21.15.33:0/3835741909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563f11a0e210 0x563f11a1c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.258+0000 7f896affd640 1 -- 172.21.15.33:0/3835741909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563f11a0e210 msgr2=0x563f11a1c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.258+0000 7f896affd640 1 --2- 172.21.15.33:0/3835741909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563f11a0e210 0x563f11a1c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.258+0000 7f896affd640 1 -- 172.21.15.33:0/3835741909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f11a18ad0 msgr2=0x563f11a1c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:07:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.258+0000 7f896affd640 1 --2- 172.21.15.33:0/3835741909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f11a18ad0 0x563f11a1c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.258+0000 7f896affd640 1 -- 172.21.15.33:0/3835741909 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f11a1cce0 con 0x563f11a0d8f0 2024-05-15T14:07:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.258+0000 7f896affd640 1 --2- 172.21.15.33:0/3835741909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f11a0d8f0 0x563f11a20d90 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f895c003970 tx=0x7f895c0045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840f8eeba2a42594 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:07:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.258+0000 7f894affd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:07:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.258+0000 7f894affd640 1 -- 172.21.15.33:0/3835741909 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c03d060 con 0x563f11a0d8f0 2024-05-15T14:07:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.259+0000 7f894affd640 1 -- 172.21.15.33:0/3835741909 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8542+0+0 (secure 0 0 0) 0x7f895c01e420 con 0x563f11a0d8f0 2024-05-15T14:07:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.259+0000 7f89745cb740 1 -- 172.21.15.33:0/3835741909 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89540a9390 con 0x563f11a0d8f0 2024-05-15T14:07:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.259+0000 7f894affd640 1 -- 172.21.15.33:0/3835741909 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f895c034040 con 0x563f11a0d8f0 2024-05-15T14:07:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.259+0000 7f89745cb740 10 client.16263 fetch_fsmap learned FSMap version 30 2024-05-15T14:07:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.259+0000 7f89745cb740 1 -- 172.21.15.33:0/3835741909 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563f11a92e00 con 0x563f11a0d8f0 2024-05-15T14:07:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.259+0000 7f894affd640 1 -- 172.21.15.33:0/3835741909 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89540a9390 con 0x563f11a0d8f0 2024-05-15T14:07:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.259+0000 7f894affd640 1 -- 172.21.15.33:0/3835741909 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7f895c0287d0 con 0x563f11a0d8f0 2024-05-15T14:07:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.259+0000 7f89745cb740 10 client.16263 fetch_fsmap finished waiting for FSMap version 30 2024-05-15T14:07:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.259+0000 7f89745cb740 10 client.16263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:07:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.259+0000 7f89745cb740 20 client.16263 populate_metadata read hostname 'smithi033' 2024-05-15T14:07:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.259+0000 7f89745cb740 1 --2- 172.21.15.33:0/3835741909 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x563f11a934c0 0x563f11a958e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:07:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.259+0000 7f89745cb740 4 client.16263 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:07:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.259+0000 7f89745cb740 1 -- 172.21.15.33:0/3835741909 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563f1197c360 con 0x563f11a934c0 2024-05-15T14:07:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.260+0000 7f8963fff640 1 --2- 172.21.15.33:0/3835741909 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x563f11a934c0 0x563f11a958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:07:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.260+0000 7f8963fff640 1 --2- 172.21.15.33:0/3835741909 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x563f11a934c0 0x563f11a958e0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f25860b0470a0f3e server_cookie=5908222665d4f48d in_seq=0 out_seq=0 2024-05-15T14:07:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.260+0000 7f894affd640 10 client.16263 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:07:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.296+0000 7f894affd640 1 -- 172.21.15.33:0/3835741909 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563f1197c360 con 0x563f11a934c0 2024-05-15T14:07:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.296+0000 7f894affd640 10 client.16263 handle_command_reply: tid=0 2024-05-15T14:07:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.296+0000 7f89745cb740 1 client.16263 shutdown 2024-05-15T14:07:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.297+0000 7f89745cb740 1 -- 172.21.15.33:0/3835741909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f11a0d8f0 msgr2=0x563f11a20d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.297+0000 7f89745cb740 1 --2- 172.21.15.33:0/3835741909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f11a0d8f0 0x563f11a20d90 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f895c003970 tx=0x7f895c0045c0 comp rx=0 tx=0).stop 2024-05-15T14:07:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.297+0000 7f89745cb740 1 -- 172.21.15.33:0/3835741909 shutdown_connections 2024-05-15T14:07:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.297+0000 7f89745cb740 1 --2- 172.21.15.33:0/3835741909 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x563f11a934c0 0x563f11a958e0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.297+0000 7f89745cb740 1 --2- 172.21.15.33:0/3835741909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f11a18ad0 0x563f11a1c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.297+0000 7f89745cb740 1 --2- 172.21.15.33:0/3835741909 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563f11a0e210 0x563f11a1c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.297+0000 7f894affd640 0 client.16263 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:07:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.297+0000 7f89745cb740 1 --2- 172.21.15.33:0/3835741909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f11a0d8f0 0x563f11a20d90 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.297+0000 7f89745cb740 1 -- 172.21.15.33:0/3835741909 >> 172.21.15.33:0/3835741909 conn(0x563f11a09e00 msgr2=0x563f11a0fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:07:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.297+0000 7f89745cb740 1 -- 172.21.15.33:0/3835741909 shutdown_connections 2024-05-15T14:07:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.297+0000 7f89745cb740 1 -- 172.21.15.33:0/3835741909 wait complete. 2024-05-15T14:07:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.297+0000 7f89745cb740 20 client.16263 trim_cache size 0 max 16384 2024-05-15T14:07:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:07:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:07:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:07:52.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.298+0000 7f89718b0640 1 -- 172.21.15.33:0/3967842795 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f89580b29d0 msgr2=0x7f89580b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:52.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.298+0000 7f89718b0640 1 --2- 172.21.15.33:0/3967842795 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f89580b29d0 0x7f89580b4e90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x563f11a0ee90 tx=0x7f895c005f90 comp rx=0 tx=0).stop 2024-05-15T14:07:52.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.298+0000 7f89718b0640 1 -- 172.21.15.33:0/3967842795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f896c102310 msgr2=0x7f896c199750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:07:52.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.298+0000 7f89718b0640 1 --2- 172.21.15.33:0/3967842795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f896c102310 0x7f896c199750 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f895400d950 tx=0x7f895400de10 comp rx=0 tx=0).stop 2024-05-15T14:07:52.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.298+0000 7f89718b0640 1 -- 172.21.15.33:0/3967842795 shutdown_connections 2024-05-15T14:07:52.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.298+0000 7f89718b0640 1 --2- 172.21.15.33:0/3967842795 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f89580b29d0 0x7f89580b4e90 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:52.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.298+0000 7f89718b0640 1 --2- 172.21.15.33:0/3967842795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f896c107960 0x7f896c19dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.298+0000 7f89718b0640 1 --2- 172.21.15.33:0/3967842795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f896c102310 0x7f896c199750 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.298+0000 7f89718b0640 1 --2- 172.21.15.33:0/3967842795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f896c101950 0x7f896c199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:07:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.298+0000 7f89718b0640 1 -- 172.21.15.33:0/3967842795 >> 172.21.15.33:0/3967842795 conn(0x7f896c078ad0 msgr2=0x7f896c100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:07:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.298+0000 7f89718b0640 1 -- 172.21.15.33:0/3967842795 shutdown_connections 2024-05-15T14:07:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:07:52.298+0000 7f89718b0640 1 -- 172.21.15.33:0/3967842795 wait complete. 2024-05-15T14:07:52.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:52 smithi081 ceph-mon[36434]: pgmap v231: 97 pgs: 97 active+clean; 133 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 1.4 MiB/s wr, 75 op/s 2024-05-15T14:07:52.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:52 smithi033 ceph-mon[31650]: pgmap v231: 97 pgs: 97 active+clean; 133 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 1.4 MiB/s wr, 75 op/s 2024-05-15T14:07:52.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:52 smithi052 ceph-mon[36190]: pgmap v231: 97 pgs: 97 active+clean; 133 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 1.4 MiB/s wr, 75 op/s 2024-05-15T14:07:53.066 INFO:tasks.workunit.client.0.smithi033.stdout: 1 461705 6.51 MB/sec execute 208 sec latency 18310.633 ms 2024-05-15T14:07:54.066 INFO:tasks.workunit.client.0.smithi033.stdout: 1 461705 6.47 MB/sec execute 209 sec latency 19310.729 ms 2024-05-15T14:07:54.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:54 smithi081 ceph-mon[36434]: pgmap v232: 97 pgs: 97 active+clean; 130 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 2.0 MiB/s wr, 103 op/s 2024-05-15T14:07:54.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:54 smithi033 ceph-mon[31650]: pgmap v232: 97 pgs: 97 active+clean; 130 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 2.0 MiB/s wr, 103 op/s 2024-05-15T14:07:54.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:54 smithi052 ceph-mon[36190]: pgmap v232: 97 pgs: 97 active+clean; 130 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 2.0 MiB/s wr, 103 op/s 2024-05-15T14:07:55.066 INFO:tasks.workunit.client.0.smithi033.stdout: 1 461705 6.44 MB/sec execute 210 sec latency 20310.851 ms 2024-05-15T14:07:56.066 INFO:tasks.workunit.client.0.smithi033.stdout: 1 461705 6.41 MB/sec execute 211 sec latency 21310.945 ms 2024-05-15T14:07:56.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:56 smithi081 ceph-mon[36434]: pgmap v233: 97 pgs: 97 active+clean; 130 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 1.2 MiB/s wr, 55 op/s 2024-05-15T14:07:56.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:56 smithi033 ceph-mon[31650]: pgmap v233: 97 pgs: 97 active+clean; 130 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 1.2 MiB/s wr, 55 op/s 2024-05-15T14:07:56.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:56 smithi052 ceph-mon[36190]: pgmap v233: 97 pgs: 97 active+clean; 130 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 1.2 MiB/s wr, 55 op/s 2024-05-15T14:07:57.067 INFO:tasks.workunit.client.0.smithi033.stdout: 1 461705 6.38 MB/sec execute 212 sec latency 22311.086 ms 2024-05-15T14:07:58.067 INFO:tasks.workunit.client.0.smithi033.stdout: 1 461705 6.35 MB/sec execute 213 sec latency 23311.180 ms 2024-05-15T14:07:58.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:07:58 smithi081 ceph-mon[36434]: pgmap v234: 97 pgs: 97 active+clean; 129 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 1.8 MiB/s wr, 79 op/s 2024-05-15T14:07:58.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:07:58 smithi033 ceph-mon[31650]: pgmap v234: 97 pgs: 97 active+clean; 129 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 1.8 MiB/s wr, 79 op/s 2024-05-15T14:07:58.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:07:58 smithi052 ceph-mon[36190]: pgmap v234: 97 pgs: 97 active+clean; 129 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 1.8 MiB/s wr, 79 op/s 2024-05-15T14:07:59.067 INFO:tasks.workunit.client.0.smithi033.stdout: 1 461705 6.32 MB/sec execute 214 sec latency 24311.291 ms 2024-05-15T14:08:00.007 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T14:08:00.067 INFO:tasks.workunit.client.0.smithi033.stdout: 1 461705 6.29 MB/sec execute 215 sec latency 25311.371 ms 2024-05-15T14:08:00.286 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:08:00.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:00 smithi081 ceph-mon[36434]: pgmap v235: 97 pgs: 97 active+clean; 129 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 1.1 MiB/s wr, 51 op/s 2024-05-15T14:08:00.645 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:00 smithi033 ceph-mon[31650]: pgmap v235: 97 pgs: 97 active+clean; 129 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 1.1 MiB/s wr, 51 op/s 2024-05-15T14:08:00.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:00 smithi052 ceph-mon[36190]: pgmap v235: 97 pgs: 97 active+clean; 129 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 1.1 MiB/s wr, 51 op/s 2024-05-15T14:08:01.067 INFO:tasks.workunit.client.0.smithi033.stdout: 1 461705 6.26 MB/sec execute 216 sec latency 26311.426 ms 2024-05-15T14:08:01.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.888+0000 7f08f6194700 1 -- 172.21.15.33:0/312503256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08f006c6d0 msgr2=0x7f08f0110400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:01.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.888+0000 7f08f6194700 1 --2- 172.21.15.33:0/312503256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08f006c6d0 0x7f08f0110400 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f08e800b0d0 tx=0x7f08e800b3e0 comp rx=0 tx=0).stop 2024-05-15T14:08:01.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.888+0000 7f08f6194700 1 -- 172.21.15.33:0/312503256 shutdown_connections 2024-05-15T14:08:01.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.888+0000 7f08f6194700 1 --2- 172.21.15.33:0/312503256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08f006c6d0 0x7f08f0110400 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:01.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.888+0000 7f08f6194700 1 --2- 172.21.15.33:0/312503256 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08f0109630 0x7f08f0109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:01.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.888+0000 7f08f6194700 1 --2- 172.21.15.33:0/312503256 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08f006bd10 0x7f08f006c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:01.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.888+0000 7f08f6194700 1 -- 172.21.15.33:0/312503256 >> 172.21.15.33:0/312503256 conn(0x7f08f00794e0 msgr2=0x7f08f00798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:01.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.888+0000 7f08f6194700 1 -- 172.21.15.33:0/312503256 shutdown_connections 2024-05-15T14:08:01.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.888+0000 7f08f6194700 1 -- 172.21.15.33:0/312503256 wait complete. 2024-05-15T14:08:01.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.889+0000 7f08f6194700 1 Processor -- start 2024-05-15T14:08:01.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.889+0000 7f08f6194700 1 -- start start 2024-05-15T14:08:01.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.889+0000 7f08f6194700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08f006bd10 0x7f08f019f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:01.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.889+0000 7f08f6194700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08f006c6d0 0x7f08f019fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:01.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.889+0000 7f08f6194700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08f0109630 0x7f08f0199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:01.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.889+0000 7f08f6194700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f08f0113860 con 0x7f08f006bd10 2024-05-15T14:08:01.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.889+0000 7f08f6194700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f08f01136e0 con 0x7f08f0109630 2024-05-15T14:08:01.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.889+0000 7f08f6194700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f08f01139e0 con 0x7f08f006c6d0 2024-05-15T14:08:01.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.889+0000 7f08ef7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08f006bd10 0x7f08f019f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:01.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.889+0000 7f08ef7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08f006bd10 0x7f08f019f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54810/0 (socket says 172.21.15.33:54810) 2024-05-15T14:08:01.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.889+0000 7f08ef7fe700 1 -- 172.21.15.33:0/3562353983 learned_addr learned my addr 172.21.15.33:0/3562353983 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:08:01.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.890+0000 7f08effff700 1 --2- 172.21.15.33:0/3562353983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08f0109630 0x7f08f0199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:01.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.890+0000 7f08eeffd700 1 --2- 172.21.15.33:0/3562353983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08f006c6d0 0x7f08f019fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:01.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.890+0000 7f08ef7fe700 1 -- 172.21.15.33:0/3562353983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08f006c6d0 msgr2=0x7f08f019fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:01.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.890+0000 7f08ef7fe700 1 --2- 172.21.15.33:0/3562353983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08f006c6d0 0x7f08f019fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:01.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.890+0000 7f08ef7fe700 1 -- 172.21.15.33:0/3562353983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08f0109630 msgr2=0x7f08f0199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:01.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.890+0000 7f08ef7fe700 1 --2- 172.21.15.33:0/3562353983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08f0109630 0x7f08f0199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:01.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.890+0000 7f08ef7fe700 1 -- 172.21.15.33:0/3562353983 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08f019a240 con 0x7f08f006bd10 2024-05-15T14:08:01.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.890+0000 7f08eeffd700 1 --2- 172.21.15.33:0/3562353983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08f006c6d0 0x7f08f019fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:08:01.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.890+0000 7f08ef7fe700 1 --2- 172.21.15.33:0/3562353983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08f006bd10 0x7f08f019f920 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f08e000b530 tx=0x7f08e000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:01.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.890+0000 7f08ecff9700 1 -- 172.21.15.33:0/3562353983 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e0015020 con 0x7f08f006bd10 2024-05-15T14:08:01.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.890+0000 7f08ecff9700 1 -- 172.21.15.33:0/3562353983 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08e0011ce0 con 0x7f08f006bd10 2024-05-15T14:08:01.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.890+0000 7f08f6194700 1 -- 172.21.15.33:0/3562353983 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08f019a530 con 0x7f08f006bd10 2024-05-15T14:08:01.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.890+0000 7f08ecff9700 1 -- 172.21.15.33:0/3562353983 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e0010d20 con 0x7f08f006bd10 2024-05-15T14:08:01.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.890+0000 7f08f6194700 1 -- 172.21.15.33:0/3562353983 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08f01a67f0 con 0x7f08f006bd10 2024-05-15T14:08:01.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.891+0000 7f08f6194700 1 -- 172.21.15.33:0/3562353983 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08f010bac0 con 0x7f08f006bd10 2024-05-15T14:08:01.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.892+0000 7f08ecff9700 1 -- 172.21.15.33:0/3562353983 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f08e0018020 con 0x7f08f006bd10 2024-05-15T14:08:01.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.893+0000 7f08ecff9700 1 --2- 172.21.15.33:0/3562353983 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f08d80b29f0 0x7f08d80b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:01.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.893+0000 7f08eeffd700 1 --2- 172.21.15.33:0/3562353983 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f08d80b29f0 0x7f08d80b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:01.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.893+0000 7f08ecff9700 1 -- 172.21.15.33:0/3562353983 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8542+0+0 (secure 0 0 0) 0x7f08e00e22d0 con 0x7f08f006bd10 2024-05-15T14:08:01.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.894+0000 7f08eeffd700 1 --2- 172.21.15.33:0/3562353983 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f08d80b29f0 0x7f08d80b4eb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f08f0079a70 tx=0x7f08e4008040 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:01.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:01.897+0000 7f08ecff9700 1 -- 172.21.15.33:0/3562353983 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f08e001f020 con 0x7f08f006bd10 2024-05-15T14:08:02.067 INFO:tasks.workunit.client.0.smithi033.stdout: 1 461705 6.24 MB/sec execute 217 sec latency 27311.462 ms 2024-05-15T14:08:02.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:02.160+0000 7f08f6194700 1 -- 172.21.15.33:0/3562353983 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f08f006a230 con 0x7f08f006bd10 2024-05-15T14:08:02.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:02.161+0000 7f08ecff9700 1 -- 172.21.15.33:0/3562353983 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11856 (secure 0 0 0) 0x7f08e00a9db0 con 0x7f08f006bd10 2024-05-15T14:08:02.164 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T14:08:02.164 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2030117633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2030117633},{"type":"v1","addr":"172.21.15.33:6839","nonce":2030117633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1918979061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1918979061},{"type":"v1","addr":"172.21.15.33:6841","nonce":1918979061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T13:57:19.056417+0000","modified":"2024-05-15T14:06:33.266410+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24463,"mds_2":24461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/1629697841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1629697841},{"type":"v1","addr":"172.21.15.52:6837","nonce":1629697841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.52:6839/2696044471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2696044471},{"type":"v1","addr":"172.21.15.52:6839","nonce":2696044471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.81:6839/1554395940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":1554395940},{"type":"v1","addr":"172.21.15.81:6839","nonce":1554395940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25393":{"gid":25393,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/727032511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":727032511},{"type":"v1","addr":"172.21.15.81:6835","nonce":727032511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24463,24478,24461]},"id":1}]} 2024-05-15T14:08:02.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:02.164+0000 7f08f6194700 1 -- 172.21.15.33:0/3562353983 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f08d80b29f0 msgr2=0x7f08d80b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:02.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:02.164+0000 7f08f6194700 1 --2- 172.21.15.33:0/3562353983 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f08d80b29f0 0x7f08d80b4eb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f08f0079a70 tx=0x7f08e4008040 comp rx=0 tx=0).stop 2024-05-15T14:08:02.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:02.164+0000 7f08f6194700 1 -- 172.21.15.33:0/3562353983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08f006bd10 msgr2=0x7f08f019f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:02.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:02.164+0000 7f08f6194700 1 --2- 172.21.15.33:0/3562353983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08f006bd10 0x7f08f019f920 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f08e000b530 tx=0x7f08e000b8f0 comp rx=0 tx=0).stop 2024-05-15T14:08:02.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:02.164+0000 7f08f6194700 1 -- 172.21.15.33:0/3562353983 shutdown_connections 2024-05-15T14:08:02.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:02.164+0000 7f08f6194700 1 --2- 172.21.15.33:0/3562353983 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f08d80b29f0 0x7f08d80b4eb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:02.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:02.164+0000 7f08f6194700 1 --2- 172.21.15.33:0/3562353983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f08f0109630 0x7f08f0199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:02.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:02.164+0000 7f08f6194700 1 --2- 172.21.15.33:0/3562353983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08f006bd10 0x7f08f019f920 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:02.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:02.164+0000 7f08f6194700 1 --2- 172.21.15.33:0/3562353983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08f006c6d0 0x7f08f019fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:02.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:02.164+0000 7f08f6194700 1 -- 172.21.15.33:0/3562353983 >> 172.21.15.33:0/3562353983 conn(0x7f08f00794e0 msgr2=0x7f08f0100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:02.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:02.164+0000 7f08f6194700 1 -- 172.21.15.33:0/3562353983 shutdown_connections 2024-05-15T14:08:02.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:02.164+0000 7f08f6194700 1 -- 172.21.15.33:0/3562353983 wait complete. 2024-05-15T14:08:02.168 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-05-15T14:08:02.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:02 smithi033 ceph-mon[31650]: pgmap v236: 97 pgs: 97 active+clean; 129 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 1.1 MiB/s wr, 51 op/s 2024-05-15T14:08:02.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:02 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/3562353983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:08:02.787 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24478 quiesce db --set-id c7303461 --release --await' 2024-05-15T14:08:02.787 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24478 quiesce db --set-id c7303461 --release --await 2024-05-15T14:08:02.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:02 smithi052 ceph-mon[36190]: pgmap v236: 97 pgs: 97 active+clean; 129 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 1.1 MiB/s wr, 51 op/s 2024-05-15T14:08:02.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:02 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/3562353983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:08:02.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:02 smithi081 ceph-mon[36434]: pgmap v236: 97 pgs: 97 active+clean; 129 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 1.1 MiB/s wr, 51 op/s 2024-05-15T14:08:02.915 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:02 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/3562353983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:08:03.041 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:08:03.067 INFO:tasks.workunit.client.0.smithi033.stdout: 1 461705 6.21 MB/sec execute 218 sec latency 28311.494 ms 2024-05-15T14:08:04.067 INFO:tasks.workunit.client.0.smithi033.stdout: 1 461705 6.18 MB/sec execute 219 sec latency 29311.527 ms 2024-05-15T14:08:04.372 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:04 smithi033 ceph-mon[31650]: pgmap v237: 97 pgs: 97 active+clean; 126 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 1.2 MiB/s wr, 76 op/s 2024-05-15T14:08:04.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.610+0000 7f942f9e1700 1 -- 172.21.15.33:0/884644766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94281041a0 msgr2=0x7f94281081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:04.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.610+0000 7f942f9e1700 1 --2- 172.21.15.33:0/884644766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94281041a0 0x7f94281081c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f94180099c0 tx=0x7f9418009cd0 comp rx=0 tx=0).stop 2024-05-15T14:08:04.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.610+0000 7f942f9e1700 1 -- 172.21.15.33:0/884644766 shutdown_connections 2024-05-15T14:08:04.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.610+0000 7f942f9e1700 1 --2- 172.21.15.33:0/884644766 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94281088f0 0x7f9428110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:04.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.610+0000 7f942f9e1700 1 --2- 172.21.15.33:0/884644766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f94281037f0 0x7f9428103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:04.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.611+0000 7f942f9e1700 1 --2- 172.21.15.33:0/884644766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94281041a0 0x7f94281081c0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:04.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.611+0000 7f942f9e1700 1 -- 172.21.15.33:0/884644766 >> 172.21.15.33:0/884644766 conn(0x7f942807a6a0 msgr2=0x7f942807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:04.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.611+0000 7f942f9e1700 1 -- 172.21.15.33:0/884644766 shutdown_connections 2024-05-15T14:08:04.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.611+0000 7f942f9e1700 1 -- 172.21.15.33:0/884644766 wait complete. 2024-05-15T14:08:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.611+0000 7f942f9e1700 1 Processor -- start 2024-05-15T14:08:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.611+0000 7f942f9e1700 1 -- start start 2024-05-15T14:08:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.611+0000 7f942f9e1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94281037f0 0x7f942819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.611+0000 7f942f9e1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f94281041a0 0x7f942819fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.611+0000 7f942f9e1700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94281088f0 0x7f9428199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.611+0000 7f942f9e1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f94281136d0 con 0x7f94281037f0 2024-05-15T14:08:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.612+0000 7f942f9e1700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9428113550 con 0x7f94281041a0 2024-05-15T14:08:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.612+0000 7f942f9e1700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9428113850 con 0x7f94281088f0 2024-05-15T14:08:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.612+0000 7f942d77d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94281037f0 0x7f942819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.612+0000 7f942d77d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94281037f0 0x7f942819f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36612/0 (socket says 172.21.15.33:36612) 2024-05-15T14:08:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.612+0000 7f942d77d700 1 -- 172.21.15.33:0/1997952508 learned_addr learned my addr 172.21.15.33:0/1997952508 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:08:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.612+0000 7f942cf7c700 1 --2- 172.21.15.33:0/1997952508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f94281041a0 0x7f942819fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.612+0000 7f942df7e700 1 --2- 172.21.15.33:0/1997952508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94281088f0 0x7f9428199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.612+0000 7f942d77d700 1 -- 172.21.15.33:0/1997952508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94281088f0 msgr2=0x7f9428199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.612+0000 7f942d77d700 1 --2- 172.21.15.33:0/1997952508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94281088f0 0x7f9428199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:04.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.612+0000 7f942d77d700 1 -- 172.21.15.33:0/1997952508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f94281041a0 msgr2=0x7f942819fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:04.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.612+0000 7f942d77d700 1 --2- 172.21.15.33:0/1997952508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f94281041a0 0x7f942819fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:04.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.612+0000 7f942d77d700 1 -- 172.21.15.33:0/1997952508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f942819a0f0 con 0x7f94281037f0 2024-05-15T14:08:04.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.612+0000 7f942d77d700 1 --2- 172.21.15.33:0/1997952508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94281037f0 0x7f942819f7d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f942400c920 tx=0x7f942400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:04.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.612+0000 7f941e7fc700 1 -- 172.21.15.33:0/1997952508 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9424004a30 con 0x7f94281037f0 2024-05-15T14:08:04.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.612+0000 7f942f9e1700 1 -- 172.21.15.33:0/1997952508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f942819a3e0 con 0x7f94281037f0 2024-05-15T14:08:04.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.612+0000 7f941e7fc700 1 -- 172.21.15.33:0/1997952508 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94240054d0 con 0x7f94281037f0 2024-05-15T14:08:04.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.612+0000 7f941e7fc700 1 -- 172.21.15.33:0/1997952508 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9424007bd0 con 0x7f94281037f0 2024-05-15T14:08:04.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.613+0000 7f942f9e1700 1 -- 172.21.15.33:0/1997952508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94281a66a0 con 0x7f94281037f0 2024-05-15T14:08:04.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.615+0000 7f941e7fc700 1 -- 172.21.15.33:0/1997952508 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9424004bd0 con 0x7f94281037f0 2024-05-15T14:08:04.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.616+0000 7f941e7fc700 1 --2- 172.21.15.33:0/1997952508 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f94140b29f0 0x7f94140b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:04.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.616+0000 7f942cf7c700 1 --2- 172.21.15.33:0/1997952508 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f94140b29f0 0x7f94140b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:04.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.616+0000 7f941e7fc700 1 -- 172.21.15.33:0/1997952508 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8542+0+0 (secure 0 0 0) 0x7f94240e2140 con 0x7f94281037f0 2024-05-15T14:08:04.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.616+0000 7f943a277b80 1 Processor -- start 2024-05-15T14:08:04.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.616+0000 7f943a277b80 1 -- start start 2024-05-15T14:08:04.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.616+0000 7f942cf7c700 1 --2- 172.21.15.33:0/1997952508 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f94140b29f0 0x7f94140b4eb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9428103160 tx=0x7f9418005fb0 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:04.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.616+0000 7f943a277b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5654328eeda0 0x5654328ef180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:04.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.616+0000 7f943a277b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654328ef6c0 0x5654328e94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:04.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.616+0000 7f943a277b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654328e9ad0 0x5654328e9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:04.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.617+0000 7f943a277b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x565432847d90 con 0x5654328e9ad0 2024-05-15T14:08:04.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.617+0000 7f943a277b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x565432837930 con 0x5654328ef6c0 2024-05-15T14:08:04.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.617+0000 7f943a277b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x56543283b970 con 0x5654328eeda0 2024-05-15T14:08:04.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.617+0000 7f942df7e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654328e9ad0 0x5654328e9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:04.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.617+0000 7f942df7e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654328e9ad0 0x5654328e9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36628/0 (socket says 172.21.15.33:36628) 2024-05-15T14:08:04.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.617+0000 7f942df7e700 1 -- 172.21.15.33:0/3166593592 learned_addr learned my addr 172.21.15.33:0/3166593592 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:08:04.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.617+0000 7f942d77d700 1 --2- 172.21.15.33:0/3166593592 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5654328eeda0 0x5654328ef180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:04.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.617+0000 7f942cf7c700 1 --2- 172.21.15.33:0/3166593592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654328ef6c0 0x5654328e94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:04.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.617+0000 7f942df7e700 1 -- 172.21.15.33:0/3166593592 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5654328eeda0 msgr2=0x5654328ef180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:04.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.617+0000 7f942df7e700 1 --2- 172.21.15.33:0/3166593592 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5654328eeda0 0x5654328ef180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:04.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.617+0000 7f942df7e700 1 -- 172.21.15.33:0/3166593592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654328ef6c0 msgr2=0x5654328e94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:04.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.617+0000 7f942df7e700 1 --2- 172.21.15.33:0/3166593592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654328ef6c0 0x5654328e94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:04.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.617+0000 7f942df7e700 1 -- 172.21.15.33:0/3166593592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654327879d0 con 0x5654328e9ad0 2024-05-15T14:08:04.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.617+0000 7f942cf7c700 1 --2- 172.21.15.33:0/3166593592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654328ef6c0 0x5654328e94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:08:04.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.617+0000 7f942df7e700 1 --2- 172.21.15.33:0/3166593592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654328e9ad0 0x5654328e9f50 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9420011680 tx=0x7f9420009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fff0987599944cc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:04.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.617+0000 7f94137fe700 1 -- 172.21.15.33:0/3166593592 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420019030 con 0x5654328e9ad0 2024-05-15T14:08:04.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.617+0000 7f94137fe700 1 -- 172.21.15.33:0/3166593592 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94200092e0 con 0x5654328e9ad0 2024-05-15T14:08:04.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.617+0000 7f94137fe700 1 -- 172.21.15.33:0/3166593592 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420007500 con 0x5654328e9ad0 2024-05-15T14:08:04.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.618+0000 7f943a277b80 1 -- 172.21.15.33:0/3166593592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654328e9ad0 msgr2=0x5654328e9f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:04.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.618+0000 7f943a277b80 1 --2- 172.21.15.33:0/3166593592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654328e9ad0 0x5654328e9f50 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9420011680 tx=0x7f9420009920 comp rx=0 tx=0).stop 2024-05-15T14:08:04.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.618+0000 7f943a277b80 1 -- 172.21.15.33:0/3166593592 shutdown_connections 2024-05-15T14:08:04.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.618+0000 7f943a277b80 1 --2- 172.21.15.33:0/3166593592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654328e9ad0 0x5654328e9f50 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:04.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.618+0000 7f943a277b80 1 --2- 172.21.15.33:0/3166593592 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5654328eeda0 0x5654328ef180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:04.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.618+0000 7f943a277b80 1 --2- 172.21.15.33:0/3166593592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654328ef6c0 0x5654328e94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:04.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.618+0000 7f943a277b80 1 -- 172.21.15.33:0/3166593592 >> 172.21.15.33:0/3166593592 conn(0x565432814350 msgr2=0x5654328e3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:04.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.618+0000 7f943a277b80 1 -- 172.21.15.33:0/3166593592 shutdown_connections 2024-05-15T14:08:04.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.618+0000 7f943a277b80 1 -- 172.21.15.33:0/3166593592 wait complete. 2024-05-15T14:08:04.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.619+0000 7f943a277b80 1 -- start start 2024-05-15T14:08:04.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.619+0000 7f943a277b80 1 Processor -- start 2024-05-15T14:08:04.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.619+0000 7f943a277b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654328e9ad0 0x5654328f53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:04.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.619+0000 7f943a277b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654328eeda0 0x5654328f2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:04.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.619+0000 7f943a277b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5654328ef6c0 0x5654328f2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:04.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.619+0000 7f943a277b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56543273e0a0 con 0x5654328eeda0 2024-05-15T14:08:04.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.619+0000 7f943a277b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5654327d98c0 con 0x5654328e9ad0 2024-05-15T14:08:04.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.619+0000 7f943a277b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5654328f6100 con 0x5654328ef6c0 2024-05-15T14:08:04.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.619+0000 7f943a277b80 10 client.0 _ll_register_callbacks cb 0x5654327ce680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:08:04.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.619+0000 7f942cf7c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654328eeda0 0x5654328f2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:04.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.619+0000 7f942cf7c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654328eeda0 0x5654328f2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36638/0 (socket says 172.21.15.33:36638) 2024-05-15T14:08:04.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.619+0000 7f942cf7c700 1 -- 172.21.15.33:0/1057940477 learned_addr learned my addr 172.21.15.33:0/1057940477 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:08:04.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.619+0000 7f942df7e700 1 --2- 172.21.15.33:0/1057940477 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5654328ef6c0 0x5654328f2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:04.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.620+0000 7f942d77d700 1 --2- 172.21.15.33:0/1057940477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654328e9ad0 0x5654328f53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:04.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.620+0000 7f942cf7c700 1 -- 172.21.15.33:0/1057940477 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5654328ef6c0 msgr2=0x5654328f2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:04.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.620+0000 7f942cf7c700 1 --2- 172.21.15.33:0/1057940477 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5654328ef6c0 0x5654328f2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:04.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.620+0000 7f942cf7c700 1 -- 172.21.15.33:0/1057940477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654328e9ad0 msgr2=0x5654328f53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:04.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.620+0000 7f942cf7c700 1 --2- 172.21.15.33:0/1057940477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654328e9ad0 0x5654328f53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:04.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.620+0000 7f942cf7c700 1 -- 172.21.15.33:0/1057940477 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654327d6790 con 0x5654328eeda0 2024-05-15T14:08:04.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.620+0000 7f942cf7c700 1 --2- 172.21.15.33:0/1057940477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654328eeda0 0x5654328f2780 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f94180099c0 tx=0x7f9418009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6940fcd22e2a3ee9 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:04.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.620+0000 7f9410ff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:08:04.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.620+0000 7f9410ff9700 1 -- 172.21.15.33:0/1057940477 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9418035060 con 0x5654328eeda0 2024-05-15T14:08:04.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.620+0000 7f9410ff9700 1 -- 172.21.15.33:0/1057940477 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8542+0+0 (secure 0 0 0) 0x7f941800bdf0 con 0x5654328eeda0 2024-05-15T14:08:04.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.620+0000 7f943a277b80 1 -- 172.21.15.33:0/1057940477 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94200092e0 con 0x5654328eeda0 2024-05-15T14:08:04.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.620+0000 7f9410ff9700 1 -- 172.21.15.33:0/1057940477 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f94180056a0 con 0x5654328eeda0 2024-05-15T14:08:04.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.620+0000 7f943a277b80 10 client.16281 fetch_fsmap learned FSMap version 30 2024-05-15T14:08:04.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.620+0000 7f943a277b80 1 -- 172.21.15.33:0/1057940477 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x565432900830 con 0x5654328eeda0 2024-05-15T14:08:04.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.620+0000 7f9410ff9700 1 -- 172.21.15.33:0/1057940477 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94200092e0 con 0x5654328eeda0 2024-05-15T14:08:04.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.621+0000 7f9410ff9700 1 -- 172.21.15.33:0/1057940477 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7f941802f070 con 0x5654328eeda0 2024-05-15T14:08:04.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.621+0000 7f943a277b80 10 client.16281 fetch_fsmap finished waiting for FSMap version 30 2024-05-15T14:08:04.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.621+0000 7f943a277b80 10 client.16281 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:08:04.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.621+0000 7f943a277b80 20 client.16281 populate_metadata read hostname 'smithi033' 2024-05-15T14:08:04.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.621+0000 7f943a277b80 1 --2- 172.21.15.33:0/1057940477 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x565432902e70 0x565432905260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:04.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.621+0000 7f943a277b80 4 client.16281 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:08:04.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.621+0000 7f943a277b80 1 -- 172.21.15.33:0/1057940477 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654328b5460 con 0x565432902e70 2024-05-15T14:08:04.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.621+0000 7f942d77d700 1 --2- 172.21.15.33:0/1057940477 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x565432902e70 0x565432905260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:04.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.621+0000 7f942d77d700 1 --2- 172.21.15.33:0/1057940477 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x565432902e70 0x565432905260 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=655d87f9a99faad7 server_cookie=d1185cdbe8924f5 in_seq=0 out_seq=0 2024-05-15T14:08:04.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.621+0000 7f9410ff9700 10 client.16281 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:08:04.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.622+0000 7f9410ff9700 1 -- 172.21.15.33:0/1057940477 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5654328b5460 con 0x565432902e70 2024-05-15T14:08:04.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.622+0000 7f9410ff9700 10 client.16281 handle_command_reply: tid=0 2024-05-15T14:08:04.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.622+0000 7f943a277b80 1 client.16281 shutdown 2024-05-15T14:08:04.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.622+0000 7f943a277b80 1 -- 172.21.15.33:0/1057940477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654328eeda0 msgr2=0x5654328f2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:04.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.622+0000 7f943a277b80 1 --2- 172.21.15.33:0/1057940477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654328eeda0 0x5654328f2780 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f94180099c0 tx=0x7f9418009cd0 comp rx=0 tx=0).stop 2024-05-15T14:08:04.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.622+0000 7f943a277b80 1 -- 172.21.15.33:0/1057940477 shutdown_connections 2024-05-15T14:08:04.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.622+0000 7f943a277b80 1 --2- 172.21.15.33:0/1057940477 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5654328ef6c0 0x5654328f2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:04.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.623+0000 7f943a277b80 1 --2- 172.21.15.33:0/1057940477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654328e9ad0 0x5654328f53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:04.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.623+0000 7f943a277b80 1 --2- 172.21.15.33:0/1057940477 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x565432902e70 0x565432905260 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:04.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.623+0000 7f943a277b80 1 --2- 172.21.15.33:0/1057940477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654328eeda0 0x5654328f2780 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:04.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.623+0000 7f9410ff9700 0 client.16281 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:08:04.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.623+0000 7f943a277b80 1 -- 172.21.15.33:0/1057940477 >> 172.21.15.33:0/1057940477 conn(0x565432814350 msgr2=0x5654328de7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:04.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.623+0000 7f943a277b80 1 -- 172.21.15.33:0/1057940477 shutdown_connections 2024-05-15T14:08:04.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.623+0000 7f943a277b80 1 -- 172.21.15.33:0/1057940477 wait complete. 2024-05-15T14:08:04.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.623+0000 7f943a277b80 20 client.16281 trim_cache size 0 max 16384 2024-05-15T14:08:04.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.644+0000 7f943a277b80 1 Processor -- start 2024-05-15T14:08:04.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.644+0000 7f943a277b80 1 -- start start 2024-05-15T14:08:04.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.644+0000 7f943a277b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654328e9ad0 0x5654328ff010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:04.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.644+0000 7f943a277b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5654328eeda0 0x5654328de8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:04.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.644+0000 7f943a277b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654328ef6c0 0x5654328f46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:04.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.644+0000 7f943a277b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f94281b0620 con 0x5654328e9ad0 2024-05-15T14:08:04.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.644+0000 7f943a277b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f94281ade80 con 0x5654328ef6c0 2024-05-15T14:08:04.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.644+0000 7f943a277b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f94281abcf0 con 0x5654328eeda0 2024-05-15T14:08:04.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.644+0000 7f942d77d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654328e9ad0 0x5654328ff010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:04.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.644+0000 7f942d77d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654328e9ad0 0x5654328ff010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36648/0 (socket says 172.21.15.33:36648) 2024-05-15T14:08:04.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.644+0000 7f942d77d700 1 -- 172.21.15.33:0/3024253184 learned_addr learned my addr 172.21.15.33:0/3024253184 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:08:04.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.645+0000 7f942cf7c700 1 --2- 172.21.15.33:0/3024253184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5654328eeda0 0x5654328de8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:04.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.645+0000 7f942df7e700 1 --2- 172.21.15.33:0/3024253184 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654328ef6c0 0x5654328f46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:04.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.645+0000 7f942d77d700 1 -- 172.21.15.33:0/3024253184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5654328eeda0 msgr2=0x5654328de8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:04.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.645+0000 7f942d77d700 1 --2- 172.21.15.33:0/3024253184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5654328eeda0 0x5654328de8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:04.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.645+0000 7f942d77d700 1 -- 172.21.15.33:0/3024253184 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654328ef6c0 msgr2=0x5654328f46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:04.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.645+0000 7f942d77d700 1 --2- 172.21.15.33:0/3024253184 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654328ef6c0 0x5654328f46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:04.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.645+0000 7f942d77d700 1 -- 172.21.15.33:0/3024253184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654328ef3d0 con 0x5654328e9ad0 2024-05-15T14:08:04.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.645+0000 7f942df7e700 1 --2- 172.21.15.33:0/3024253184 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654328ef6c0 0x5654328f46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:08:04.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.645+0000 7f942d77d700 1 --2- 172.21.15.33:0/3024253184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654328e9ad0 0x5654328ff010 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f94240a6680 tx=0x7f9424000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf8b842462fba81 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:04.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.645+0000 7f9412ffd700 1 -- 172.21.15.33:0/3024253184 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942401d040 con 0x5654328e9ad0 2024-05-15T14:08:04.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.645+0000 7f9412ffd700 1 -- 172.21.15.33:0/3024253184 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94240b2270 con 0x5654328e9ad0 2024-05-15T14:08:04.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.645+0000 7f9412ffd700 1 -- 172.21.15.33:0/3024253184 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94240a9670 con 0x5654328e9ad0 2024-05-15T14:08:04.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.646+0000 7f943a277b80 1 -- 172.21.15.33:0/3024253184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654328e9ad0 msgr2=0x5654328ff010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:04.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.646+0000 7f943a277b80 1 --2- 172.21.15.33:0/3024253184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654328e9ad0 0x5654328ff010 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f94240a6680 tx=0x7f9424000bc0 comp rx=0 tx=0).stop 2024-05-15T14:08:04.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.646+0000 7f943a277b80 1 -- 172.21.15.33:0/3024253184 shutdown_connections 2024-05-15T14:08:04.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.646+0000 7f943a277b80 1 --2- 172.21.15.33:0/3024253184 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654328ef6c0 0x5654328f46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:04.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.646+0000 7f943a277b80 1 --2- 172.21.15.33:0/3024253184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654328e9ad0 0x5654328ff010 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:04.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.646+0000 7f943a277b80 1 --2- 172.21.15.33:0/3024253184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5654328eeda0 0x5654328de8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:04.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.646+0000 7f943a277b80 1 -- 172.21.15.33:0/3024253184 >> 172.21.15.33:0/3024253184 conn(0x565432814350 msgr2=0x5654328f0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:04.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.646+0000 7f943a277b80 1 -- 172.21.15.33:0/3024253184 shutdown_connections 2024-05-15T14:08:04.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.646+0000 7f943a277b80 1 -- 172.21.15.33:0/3024253184 wait complete. 2024-05-15T14:08:04.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.646+0000 7f943a277b80 1 -- start start 2024-05-15T14:08:04.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.646+0000 7f943a277b80 1 Processor -- start 2024-05-15T14:08:04.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.647+0000 7f943a277b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654328e9ad0 0x5654328f8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:04.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.647+0000 7f943a277b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654328eeda0 0x5654328f88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:04.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.647+0000 7f943a277b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5654328ef6c0 0x5654328f8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:04.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.647+0000 7f943a277b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f94281ac340 con 0x5654328eeda0 2024-05-15T14:08:04.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.647+0000 7f943a277b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f94281ad7b0 con 0x5654328e9ad0 2024-05-15T14:08:04.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.647+0000 7f943a277b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f94281aca10 con 0x5654328ef6c0 2024-05-15T14:08:04.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.647+0000 7f943a277b80 10 client.0 _ll_register_callbacks cb 0x5654327ce680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:08:04.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.647+0000 7f942cf7c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654328eeda0 0x5654328f88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:04.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.647+0000 7f942cf7c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654328eeda0 0x5654328f88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36658/0 (socket says 172.21.15.33:36658) 2024-05-15T14:08:04.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.647+0000 7f942cf7c700 1 -- 172.21.15.33:0/2065971988 learned_addr learned my addr 172.21.15.33:0/2065971988 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:08:04.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.647+0000 7f942d77d700 1 --2- 172.21.15.33:0/2065971988 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654328e9ad0 0x5654328f8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:04.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.647+0000 7f942df7e700 1 --2- 172.21.15.33:0/2065971988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5654328ef6c0 0x5654328f8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:04.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.647+0000 7f942cf7c700 1 -- 172.21.15.33:0/2065971988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5654328ef6c0 msgr2=0x5654328f8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:04.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.647+0000 7f942cf7c700 1 --2- 172.21.15.33:0/2065971988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5654328ef6c0 0x5654328f8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:04.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.647+0000 7f942cf7c700 1 -- 172.21.15.33:0/2065971988 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654328e9ad0 msgr2=0x5654328f8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:04.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.647+0000 7f942cf7c700 1 --2- 172.21.15.33:0/2065971988 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654328e9ad0 0x5654328f8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:04.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.647+0000 7f942cf7c700 1 -- 172.21.15.33:0/2065971988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56543296d7a0 con 0x5654328eeda0 2024-05-15T14:08:04.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.647+0000 7f942df7e700 1 --2- 172.21.15.33:0/2065971988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5654328ef6c0 0x5654328f8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:08:04.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.647+0000 7f942cf7c700 1 --2- 172.21.15.33:0/2065971988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654328eeda0 0x5654328f88a0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f94180098e0 tx=0x7f9418024410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be8d3b195a153ad server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:04.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.647+0000 7f9410ff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:08:04.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.647+0000 7f9410ff9700 1 -- 172.21.15.33:0/2065971988 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9418035070 con 0x5654328eeda0 2024-05-15T14:08:04.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.648+0000 7f943a277b80 1 -- 172.21.15.33:0/2065971988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94240b2270 con 0x5654328eeda0 2024-05-15T14:08:04.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.648+0000 7f9410ff9700 1 -- 172.21.15.33:0/2065971988 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8542+0+0 (secure 0 0 0) 0x7f9418030430 con 0x5654328eeda0 2024-05-15T14:08:04.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.648+0000 7f9410ff9700 1 -- 172.21.15.33:0/2065971988 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9418047050 con 0x5654328eeda0 2024-05-15T14:08:04.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.648+0000 7f943a277b80 10 client.16287 fetch_fsmap learned FSMap version 30 2024-05-15T14:08:04.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.648+0000 7f943a277b80 1 -- 172.21.15.33:0/2065971988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56543296ea30 con 0x5654328eeda0 2024-05-15T14:08:04.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.648+0000 7f9410ff9700 1 -- 172.21.15.33:0/2065971988 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94240b2270 con 0x5654328eeda0 2024-05-15T14:08:04.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.648+0000 7f9410ff9700 1 -- 172.21.15.33:0/2065971988 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7f941802f040 con 0x5654328eeda0 2024-05-15T14:08:04.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.648+0000 7f943a277b80 10 client.16287 fetch_fsmap finished waiting for FSMap version 30 2024-05-15T14:08:04.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.648+0000 7f943a277b80 10 client.16287 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:08:04.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.648+0000 7f943a277b80 20 client.16287 populate_metadata read hostname 'smithi033' 2024-05-15T14:08:04.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.648+0000 7f943a277b80 1 --2- 172.21.15.33:0/2065971988 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56543296f180 0x5654329715a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:04.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.648+0000 7f943a277b80 4 client.16287 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c7303461", "release": true, "await": true}] 2024-05-15T14:08:04.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.648+0000 7f943a277b80 1 -- 172.21.15.33:0/2065971988 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c7303461", "release": true, "await": true}) -- 0x565432849ce0 con 0x56543296f180 2024-05-15T14:08:04.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.648+0000 7f942d77d700 1 --2- 172.21.15.33:0/2065971988 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56543296f180 0x5654329715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:04.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.649+0000 7f942d77d700 1 --2- 172.21.15.33:0/2065971988 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56543296f180 0x5654329715a0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ea3d32c306b3172 server_cookie=9e2b8aae0092e39 in_seq=0 out_seq=0 2024-05-15T14:08:04.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.649+0000 7f9410ff9700 10 client.16287 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:08:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.660+0000 7f9410ff9700 1 -- 172.21.15.33:0/2065971988 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x565432849ce0 con 0x56543296f180 2024-05-15T14:08:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.660+0000 7f9410ff9700 10 client.16287 handle_command_reply: tid=0 2024-05-15T14:08:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.660+0000 7f943a277b80 1 client.16287 shutdown 2024-05-15T14:08:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.661+0000 7f943a277b80 1 -- 172.21.15.33:0/2065971988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654328eeda0 msgr2=0x5654328f88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.661+0000 7f943a277b80 1 --2- 172.21.15.33:0/2065971988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654328eeda0 0x5654328f88a0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f94180098e0 tx=0x7f9418024410 comp rx=0 tx=0).stop 2024-05-15T14:08:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.661+0000 7f943a277b80 1 -- 172.21.15.33:0/2065971988 shutdown_connections 2024-05-15T14:08:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.661+0000 7f943a277b80 1 --2- 172.21.15.33:0/2065971988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5654328ef6c0 0x5654328f8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.661+0000 7f943a277b80 1 --2- 172.21.15.33:0/2065971988 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5654328e9ad0 0x5654328f8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.661+0000 7f943a277b80 1 --2- 172.21.15.33:0/2065971988 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x56543296f180 0x5654329715a0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.661+0000 7f943a277b80 1 --2- 172.21.15.33:0/2065971988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5654328eeda0 0x5654328f88a0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.661+0000 7f9410ff9700 0 client.16287 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:08:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.661+0000 7f943a277b80 1 -- 172.21.15.33:0/2065971988 >> 172.21.15.33:0/2065971988 conn(0x565432814350 msgr2=0x5654328ee190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.661+0000 7f943a277b80 1 -- 172.21.15.33:0/2065971988 shutdown_connections 2024-05-15T14:08:04.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.661+0000 7f943a277b80 1 -- 172.21.15.33:0/2065971988 wait complete. 2024-05-15T14:08:04.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.661+0000 7f943a277b80 20 client.16287 trim_cache size 0 max 16384 2024-05-15T14:08:04.665 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T14:08:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 30, 2024-05-15T14:08:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24478, 2024-05-15T14:08:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 28, 2024-05-15T14:08:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-05-15T14:08:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: "c7303461": { 2024-05-15T14:08:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 28, 2024-05-15T14:08:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-05-15T14:08:04.665 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:08:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-05-15T14:08:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:08:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:08:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 90.0, 2024-05-15T14:08:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 150.3, 2024-05-15T14:08:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-05-15T14:08:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-05-15T14:08:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-05-15T14:08:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:08:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-05-15T14:08:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:08:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:08:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:08:04.666 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:08:04.667 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:08:04.667 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:08:04.667 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T14:08:04.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.662+0000 7f942f9e1700 1 -- 172.21.15.33:0/1997952508 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f94140b29f0 msgr2=0x7f94140b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:04.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.662+0000 7f942f9e1700 1 --2- 172.21.15.33:0/1997952508 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f94140b29f0 0x7f94140b4eb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9428103160 tx=0x7f9418005fb0 comp rx=0 tx=0).stop 2024-05-15T14:08:04.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.662+0000 7f942f9e1700 1 -- 172.21.15.33:0/1997952508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94281037f0 msgr2=0x7f942819f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:04.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.662+0000 7f942f9e1700 1 --2- 172.21.15.33:0/1997952508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94281037f0 0x7f942819f7d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f942400c920 tx=0x7f942400cc30 comp rx=0 tx=0).stop 2024-05-15T14:08:04.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.662+0000 7f942f9e1700 1 -- 172.21.15.33:0/1997952508 shutdown_connections 2024-05-15T14:08:04.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.662+0000 7f942f9e1700 1 --2- 172.21.15.33:0/1997952508 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f94140b29f0 0x7f94140b4eb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:04.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.662+0000 7f942f9e1700 1 --2- 172.21.15.33:0/1997952508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94281088f0 0x7f9428199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:04.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.662+0000 7f942f9e1700 1 --2- 172.21.15.33:0/1997952508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94281037f0 0x7f942819f7d0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:04.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.662+0000 7f942f9e1700 1 --2- 172.21.15.33:0/1997952508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f94281041a0 0x7f942819fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:04.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.662+0000 7f942f9e1700 1 -- 172.21.15.33:0/1997952508 >> 172.21.15.33:0/1997952508 conn(0x7f942807a6a0 msgr2=0x7f9428109a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:04.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.662+0000 7f942f9e1700 1 -- 172.21.15.33:0/1997952508 shutdown_connections 2024-05-15T14:08:04.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:08:04.662+0000 7f942f9e1700 1 -- 172.21.15.33:0/1997952508 wait complete. 2024-05-15T14:08:04.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:04 smithi052 ceph-mon[36190]: pgmap v237: 97 pgs: 97 active+clean; 126 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 1.2 MiB/s wr, 76 op/s 2024-05-15T14:08:04.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:04 smithi081 ceph-mon[36434]: pgmap v237: 97 pgs: 97 active+clean; 126 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 1.2 MiB/s wr, 76 op/s 2024-05-15T14:08:05.067 INFO:tasks.workunit.client.0.smithi033.stdout: 1 462095 6.17 MB/sec execute 220 sec latency 29908.923 ms 2024-05-15T14:08:05.309 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c7303461', total seconds elapsed: 35.6 2024-05-15T14:08:05.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.396+0000 7fc1e0f2b640 1 -- 172.21.15.33:0/4247283837 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1dc101930 msgr2=0x7fc1dc101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:05.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.396+0000 7fc1e0f2b640 1 --2- 172.21.15.33:0/4247283837 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1dc101930 0x7fc1dc101db0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0009890 tx=0x7fc1d002a380 comp rx=0 tx=0).stop 2024-05-15T14:08:05.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.396+0000 7fc1e0f2b640 1 -- 172.21.15.33:0/4247283837 shutdown_connections 2024-05-15T14:08:05.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.396+0000 7fc1e0f2b640 1 --2- 172.21.15.33:0/4247283837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1dc1022f0 0x7fc1dc10e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:05.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.396+0000 7fc1e0f2b640 1 --2- 172.21.15.33:0/4247283837 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1dc101930 0x7fc1dc101db0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:05.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.396+0000 7fc1e0f2b640 1 --2- 172.21.15.33:0/4247283837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1dc107940 0x7fc1dc107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:05.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.396+0000 7fc1e0f2b640 1 -- 172.21.15.33:0/4247283837 >> 172.21.15.33:0/4247283837 conn(0x7fc1dc078a70 msgr2=0x7fc1dc078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.397+0000 7fc1e0f2b640 1 -- 172.21.15.33:0/4247283837 shutdown_connections 2024-05-15T14:08:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.397+0000 7fc1e0f2b640 1 -- 172.21.15.33:0/4247283837 wait complete. 2024-05-15T14:08:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.397+0000 7fc1e0f2b640 1 Processor -- start 2024-05-15T14:08:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.397+0000 7fc1e0f2b640 1 -- start start 2024-05-15T14:08:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.397+0000 7fc1e0f2b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1dc101930 0x7fc1dc19a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:05.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.398+0000 7fc1e0f2b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1dc1022f0 0x7fc1dc19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.398+0000 7fc1e0f2b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1dc107940 0x7fc1dc1a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.398+0000 7fc1e0f2b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc111c00 con 0x7fc1dc107940 2024-05-15T14:08:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.398+0000 7fc1e0f2b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc111a80 con 0x7fc1dc1022f0 2024-05-15T14:08:05.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.398+0000 7fc1e0f2b640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc111d80 con 0x7fc1dc101930 2024-05-15T14:08:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.398+0000 7fc1dad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1dc107940 0x7fc1dc1a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.398+0000 7fc1da59c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1dc101930 0x7fc1dc19a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.398+0000 7fc1dad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1dc107940 0x7fc1dc1a5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36676/0 (socket says 172.21.15.33:36676) 2024-05-15T14:08:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.398+0000 7fc1d9d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1dc1022f0 0x7fc1dc19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.398+0000 7fc1dad9d640 1 -- 172.21.15.33:0/1076146276 learned_addr learned my addr 172.21.15.33:0/1076146276 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:08:05.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.398+0000 7fc1da59c640 1 -- 172.21.15.33:0/1076146276 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1dc1022f0 msgr2=0x7fc1dc19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.398+0000 7fc1da59c640 1 --2- 172.21.15.33:0/1076146276 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1dc1022f0 0x7fc1dc19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.398+0000 7fc1da59c640 1 -- 172.21.15.33:0/1076146276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1dc107940 msgr2=0x7fc1dc1a5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.398+0000 7fc1da59c640 1 --2- 172.21.15.33:0/1076146276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1dc107940 0x7fc1dc1a5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.398+0000 7fc1da59c640 1 -- 172.21.15.33:0/1076146276 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1dc1a5680 con 0x7fc1dc101930 2024-05-15T14:08:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.398+0000 7fc1dad9d640 1 --2- 172.21.15.33:0/1076146276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1dc107940 0x7fc1dc1a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:08:05.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.398+0000 7fc1da59c640 1 --2- 172.21.15.33:0/1076146276 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1dc101930 0x7fc1dc19a0c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc00e790 tx=0x7fc1cc00ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.398+0000 7fc1cb7fe640 1 -- 172.21.15.33:0/1076146276 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc00eea0 con 0x7fc1dc101930 2024-05-15T14:08:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.398+0000 7fc1cb7fe640 1 -- 172.21.15.33:0/1076146276 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1cc00ccc0 con 0x7fc1dc101930 2024-05-15T14:08:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.399+0000 7fc1cb7fe640 1 -- 172.21.15.33:0/1076146276 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc018680 con 0x7fc1dc101930 2024-05-15T14:08:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.399+0000 7fc1e0f2b640 1 -- 172.21.15.33:0/1076146276 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1dc1a5850 con 0x7fc1dc101930 2024-05-15T14:08:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.399+0000 7fc1e0f2b640 1 -- 172.21.15.33:0/1076146276 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1dc1a5dc0 con 0x7fc1dc101930 2024-05-15T14:08:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.403+0000 7fc1cb7fe640 1 -- 172.21.15.33:0/1076146276 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc1cc010430 con 0x7fc1dc101930 2024-05-15T14:08:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.403+0000 7fc1e3c46740 1 Processor -- start 2024-05-15T14:08:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.403+0000 7fc1e3c46740 1 -- start start 2024-05-15T14:08:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.404+0000 7fc1e3c46740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a45b88f0 0x55e4a45b8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:05.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.404+0000 7fc1e3c46740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4a45b9210 0x55e4a45c3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.404+0000 7fc1e3c46740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4a45c3ad0 0x55e4a45c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.404+0000 7fc1e3c46740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e4a452bc30 con 0x55e4a45b9210 2024-05-15T14:08:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.404+0000 7fc1e3c46740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e4a451b7d0 con 0x55e4a45b88f0 2024-05-15T14:08:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.404+0000 7fc1e3c46740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55e4a451f810 con 0x55e4a45c3ad0 2024-05-15T14:08:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.404+0000 7fc1da59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4a45b9210 0x55e4a45c3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.404+0000 7fc1dad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a45b88f0 0x55e4a45b8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.404+0000 7fc1d9d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4a45c3ad0 0x55e4a45c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:05.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.404+0000 7fc1dad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a45b88f0 0x55e4a45b8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:53074/0 (socket says 172.21.15.33:53074) 2024-05-15T14:08:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.404+0000 7fc1d9d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4a45c3ad0 0x55e4a45c5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:36626/0 (socket says 172.21.15.33:36626) 2024-05-15T14:08:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.404+0000 7fc1dad9d640 1 -- 172.21.15.33:0/3431492915 learned_addr learned my addr 172.21.15.33:0/3431492915 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:08:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.404+0000 7fc1dad9d640 1 -- 172.21.15.33:0/3431492915 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4a45c3ad0 msgr2=0x55e4a45c5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.404+0000 7fc1dad9d640 1 --2- 172.21.15.33:0/3431492915 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4a45c3ad0 0x55e4a45c5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.404+0000 7fc1dad9d640 1 -- 172.21.15.33:0/3431492915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4a45b9210 msgr2=0x55e4a45c3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.404+0000 7fc1dad9d640 1 --2- 172.21.15.33:0/3431492915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4a45b9210 0x55e4a45c3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.404+0000 7fc1dad9d640 1 -- 172.21.15.33:0/3431492915 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4a4230210 con 0x55e4a45b88f0 2024-05-15T14:08:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.404+0000 7fc1d9d9b640 1 --2- 172.21.15.33:0/3431492915 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4a45c3ad0 0x55e4a45c5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:08:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.405+0000 7fc1cb7fe640 1 --2- 172.21.15.33:0/1076146276 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc1c40b29d0 0x7fc1c40b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.405+0000 7fc1dad9d640 1 --2- 172.21.15.33:0/3431492915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a45b88f0 0x55e4a45b8cd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc1d40118b0 tx=0x7fc1d4011d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b88b782ab6edbd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.405+0000 7fc1da59c640 1 --2- 172.21.15.33:0/1076146276 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc1c40b29d0 0x7fc1c40b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.405+0000 7fc1c8ff9640 1 -- 172.21.15.33:0/3431492915 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d401e070 con 0x55e4a45b88f0 2024-05-15T14:08:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.405+0000 7fc1c8ff9640 1 -- 172.21.15.33:0/3431492915 <== mon.2 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1d400fe70 con 0x55e4a45b88f0 2024-05-15T14:08:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.405+0000 7fc1c8ff9640 1 -- 172.21.15.33:0/3431492915 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d4019d30 con 0x55e4a45b88f0 2024-05-15T14:08:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.405+0000 7fc1cb7fe640 1 -- 172.21.15.33:0/1076146276 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8542+0+0 (secure 0 0 0) 0x7fc1cc0e26c0 con 0x7fc1dc101930 2024-05-15T14:08:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.406+0000 7fc1e3c46740 1 -- 172.21.15.33:0/3431492915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a45b88f0 msgr2=0x55e4a45b8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.406+0000 7fc1da59c640 1 --2- 172.21.15.33:0/1076146276 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc1c40b29d0 0x7fc1c40b4e90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc0dfeb0 tx=0x7fc1cc0e8550 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.406+0000 7fc1e3c46740 1 --2- 172.21.15.33:0/3431492915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a45b88f0 0x55e4a45b8cd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc1d40118b0 tx=0x7fc1d4011d70 comp rx=0 tx=0).stop 2024-05-15T14:08:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.406+0000 7fc1e3c46740 1 -- 172.21.15.33:0/3431492915 shutdown_connections 2024-05-15T14:08:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.406+0000 7fc1e3c46740 1 --2- 172.21.15.33:0/3431492915 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4a45c3ad0 0x55e4a45c5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.406+0000 7fc1e3c46740 1 --2- 172.21.15.33:0/3431492915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4a45b9210 0x55e4a45c3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.406+0000 7fc1e3c46740 1 --2- 172.21.15.33:0/3431492915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a45b88f0 0x55e4a45b8cd0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.406+0000 7fc1e3c46740 1 -- 172.21.15.33:0/3431492915 >> 172.21.15.33:0/3431492915 conn(0x55e4a45b4e00 msgr2=0x55e4a45b51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:05.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.406+0000 7fc1e3c46740 1 -- 172.21.15.33:0/3431492915 shutdown_connections 2024-05-15T14:08:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.406+0000 7fc1e3c46740 1 -- 172.21.15.33:0/3431492915 wait complete. 2024-05-15T14:08:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.406+0000 7fc1e3c46740 1 -- start start 2024-05-15T14:08:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.406+0000 7fc1e3c46740 1 Processor -- start 2024-05-15T14:08:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.407+0000 7fc1e3c46740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4a45b88f0 0x55e4a45c3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.407+0000 7fc1e3c46740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a45b9210 0x55e4a45c0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.407+0000 7fc1e3c46740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4a45c3ad0 0x55e4a45c0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:05.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.407+0000 7fc1e3c46740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e4a4201790 con 0x55e4a45b88f0 2024-05-15T14:08:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.407+0000 7fc1e3c46740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e4a44d7890 con 0x55e4a45b9210 2024-05-15T14:08:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.407+0000 7fc1e3c46740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55e4a45c67a0 con 0x55e4a45c3ad0 2024-05-15T14:08:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.407+0000 7fc1e3c46740 10 client.0 _ll_register_callbacks cb 0x55e4a44dad60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:08:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.407+0000 7fc1dad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4a45b88f0 0x55e4a45c3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.407+0000 7fc1dad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4a45b88f0 0x55e4a45c3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36692/0 (socket says 172.21.15.33:36692) 2024-05-15T14:08:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.407+0000 7fc1dad9d640 1 -- 172.21.15.33:0/3395783472 learned_addr learned my addr 172.21.15.33:0/3395783472 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:08:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.407+0000 7fc1d9d9b640 1 --2- 172.21.15.33:0/3395783472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a45b9210 0x55e4a45c0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.407+0000 7fc1dad9d640 1 --2- 172.21.15.33:0/3395783472 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4a45c3ad0 0x55e4a45c0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.407+0000 7fc1dad9d640 1 -- 172.21.15.33:0/3395783472 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4a45c3ad0 msgr2=0x55e4a45c0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.407+0000 7fc1dad9d640 1 --2- 172.21.15.33:0/3395783472 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4a45c3ad0 0x55e4a45c0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.407+0000 7fc1dad9d640 1 -- 172.21.15.33:0/3395783472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a45b9210 msgr2=0x55e4a45c0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.407+0000 7fc1dad9d640 1 --2- 172.21.15.33:0/3395783472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a45b9210 0x55e4a45c0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.407+0000 7fc1dad9d640 1 -- 172.21.15.33:0/3395783472 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4a42fc620 con 0x55e4a45b88f0 2024-05-15T14:08:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.407+0000 7fc1d9d9b640 1 --2- 172.21.15.33:0/3395783472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a45b9210 0x55e4a45c0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:08:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.407+0000 7fc1dad9d640 1 --2- 172.21.15.33:0/3395783472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4a45b88f0 0x55e4a45c3350 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4012fd0 tx=0x7fc1d4019990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=887d60459c83bc23 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.407+0000 7fc1c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:08:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.408+0000 7fc1c1ffb640 1 -- 172.21.15.33:0/3395783472 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d401e030 con 0x55e4a45b88f0 2024-05-15T14:08:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.408+0000 7fc1c1ffb640 1 -- 172.21.15.33:0/3395783472 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8542+0+0 (secure 0 0 0) 0x7fc1d4004c40 con 0x55e4a45b88f0 2024-05-15T14:08:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.408+0000 7fc1e3c46740 1 -- 172.21.15.33:0/3395783472 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1d400fe70 con 0x55e4a45b88f0 2024-05-15T14:08:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.408+0000 7fc1c1ffb640 1 -- 172.21.15.33:0/3395783472 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc1d4036050 con 0x55e4a45b88f0 2024-05-15T14:08:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.408+0000 7fc1e3c46740 10 client.16299 fetch_fsmap learned FSMap version 30 2024-05-15T14:08:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.408+0000 7fc1e3c46740 1 -- 172.21.15.33:0/3395783472 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e4a45d20d0 con 0x55e4a45b88f0 2024-05-15T14:08:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.408+0000 7fc1c1ffb640 1 -- 172.21.15.33:0/3395783472 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1d400fe70 con 0x55e4a45b88f0 2024-05-15T14:08:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.408+0000 7fc1c1ffb640 1 -- 172.21.15.33:0/3395783472 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7fc1d40109b0 con 0x55e4a45b88f0 2024-05-15T14:08:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.408+0000 7fc1e3c46740 10 client.16299 fetch_fsmap finished waiting for FSMap version 30 2024-05-15T14:08:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.408+0000 7fc1e3c46740 10 client.16299 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:08:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.408+0000 7fc1e3c46740 20 client.16299 populate_metadata read hostname 'smithi033' 2024-05-15T14:08:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.408+0000 7fc1e3c46740 1 --2- 172.21.15.33:0/3395783472 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e4a45d3530 0x55e4a45d5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.408+0000 7fc1e3c46740 4 client.16299 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:08:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.408+0000 7fc1e3c46740 1 -- 172.21.15.33:0/3395783472 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4a452db80 con 0x55e4a45d3530 2024-05-15T14:08:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.408+0000 7fc1d9d9b640 1 --2- 172.21.15.33:0/3395783472 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e4a45d3530 0x55e4a45d5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.409+0000 7fc1c1ffb640 10 client.16299 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:08:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.409+0000 7fc1d9d9b640 1 --2- 172.21.15.33:0/3395783472 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e4a45d3530 0x55e4a45d5920 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d54995824eca6a7e server_cookie=f527d5d23ff2540 in_seq=0 out_seq=0 2024-05-15T14:08:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.409+0000 7fc1c1ffb640 1 -- 172.21.15.33:0/3395783472 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e4a452db80 con 0x55e4a45d3530 2024-05-15T14:08:05.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.409+0000 7fc1c1ffb640 10 client.16299 handle_command_reply: tid=0 2024-05-15T14:08:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.409+0000 7fc1e3c46740 1 client.16299 shutdown 2024-05-15T14:08:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.410+0000 7fc1e3c46740 1 -- 172.21.15.33:0/3395783472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4a45b88f0 msgr2=0x55e4a45c3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.410+0000 7fc1e3c46740 1 --2- 172.21.15.33:0/3395783472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4a45b88f0 0x55e4a45c3350 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4012fd0 tx=0x7fc1d4019990 comp rx=0 tx=0).stop 2024-05-15T14:08:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.410+0000 7fc1e3c46740 1 -- 172.21.15.33:0/3395783472 shutdown_connections 2024-05-15T14:08:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.410+0000 7fc1e3c46740 1 --2- 172.21.15.33:0/3395783472 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e4a45d3530 0x55e4a45d5920 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.410+0000 7fc1e3c46740 1 --2- 172.21.15.33:0/3395783472 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4a45c3ad0 0x55e4a45c0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.410+0000 7fc1c1ffb640 0 client.16299 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:08:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.410+0000 7fc1e3c46740 1 --2- 172.21.15.33:0/3395783472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a45b9210 0x55e4a45c0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.410+0000 7fc1e3c46740 1 --2- 172.21.15.33:0/3395783472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4a45b88f0 0x55e4a45c3350 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.410+0000 7fc1e3c46740 1 -- 172.21.15.33:0/3395783472 >> 172.21.15.33:0/3395783472 conn(0x55e4a45b4e00 msgr2=0x55e4a45b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.410+0000 7fc1e3c46740 1 -- 172.21.15.33:0/3395783472 shutdown_connections 2024-05-15T14:08:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.410+0000 7fc1e3c46740 1 -- 172.21.15.33:0/3395783472 wait complete. 2024-05-15T14:08:05.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.410+0000 7fc1e3c46740 20 client.16299 trim_cache size 0 max 16384 2024-05-15T14:08:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.426+0000 7fc1e3c46740 1 Processor -- start 2024-05-15T14:08:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.426+0000 7fc1e3c46740 1 -- start start 2024-05-15T14:08:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.427+0000 7fc1e3c46740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a45b88f0 0x55e4a45bfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.427+0000 7fc1e3c46740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4a45b9210 0x55e4a45c2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.427+0000 7fc1e3c46740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4a45c3ad0 0x55e4a45db090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.427+0000 7fc1e3c46740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc1b1130 con 0x55e4a45b9210 2024-05-15T14:08:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.427+0000 7fc1e3c46740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc1ae9d0 con 0x55e4a45b88f0 2024-05-15T14:08:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.427+0000 7fc1e3c46740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc1ac840 con 0x55e4a45c3ad0 2024-05-15T14:08:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.427+0000 7fc1d9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4a45b9210 0x55e4a45c2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.427+0000 7fc1dad9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4a45c3ad0 0x55e4a45db090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.427+0000 7fc1d9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4a45b9210 0x55e4a45c2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36696/0 (socket says 172.21.15.33:36696) 2024-05-15T14:08:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.427+0000 7fc1d9d9b640 1 -- 172.21.15.33:0/4147857503 learned_addr learned my addr 172.21.15.33:0/4147857503 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:08:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.427+0000 7fc1dad9d640 1 --2- 172.21.15.33:0/4147857503 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a45b88f0 0x55e4a45bfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.427+0000 7fc1d9d9b640 1 -- 172.21.15.33:0/4147857503 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4a45c3ad0 msgr2=0x55e4a45db090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.427+0000 7fc1d9d9b640 1 --2- 172.21.15.33:0/4147857503 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4a45c3ad0 0x55e4a45db090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:05.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.427+0000 7fc1d9d9b640 1 -- 172.21.15.33:0/4147857503 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a45b88f0 msgr2=0x55e4a45bfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.427+0000 7fc1d9d9b640 1 --2- 172.21.15.33:0/4147857503 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a45b88f0 0x55e4a45bfdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.427+0000 7fc1d9d9b640 1 -- 172.21.15.33:0/4147857503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4a45b4c30 con 0x55e4a45b9210 2024-05-15T14:08:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.427+0000 7fc1dad9d640 1 --2- 172.21.15.33:0/4147857503 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4a45c3ad0 0x55e4a45db090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:08:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.427+0000 7fc1d9d9b640 1 --2- 172.21.15.33:0/4147857503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4a45b9210 0x55e4a45c2e80 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc1d00097b0 tx=0x7fc1d00058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd44b6ae7f5b7cfc server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.427+0000 7fc1c3fff640 1 -- 172.21.15.33:0/4147857503 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0037040 con 0x55e4a45b9210 2024-05-15T14:08:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.427+0000 7fc1c3fff640 1 -- 172.21.15.33:0/4147857503 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1d0030900 con 0x55e4a45b9210 2024-05-15T14:08:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.427+0000 7fc1c3fff640 1 -- 172.21.15.33:0/4147857503 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0003d10 con 0x55e4a45b9210 2024-05-15T14:08:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.428+0000 7fc1e3c46740 1 -- 172.21.15.33:0/4147857503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4a45b9210 msgr2=0x55e4a45c2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.428+0000 7fc1e3c46740 1 --2- 172.21.15.33:0/4147857503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4a45b9210 0x55e4a45c2e80 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc1d00097b0 tx=0x7fc1d00058d0 comp rx=0 tx=0).stop 2024-05-15T14:08:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.428+0000 7fc1e3c46740 1 -- 172.21.15.33:0/4147857503 shutdown_connections 2024-05-15T14:08:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.428+0000 7fc1e3c46740 1 --2- 172.21.15.33:0/4147857503 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4a45c3ad0 0x55e4a45db090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.428+0000 7fc1e3c46740 1 --2- 172.21.15.33:0/4147857503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4a45b9210 0x55e4a45c2e80 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:05.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.428+0000 7fc1e3c46740 1 --2- 172.21.15.33:0/4147857503 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a45b88f0 0x55e4a45bfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.428+0000 7fc1e3c46740 1 -- 172.21.15.33:0/4147857503 >> 172.21.15.33:0/4147857503 conn(0x55e4a45b4e00 msgr2=0x55e4a45bab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.428+0000 7fc1e3c46740 1 -- 172.21.15.33:0/4147857503 shutdown_connections 2024-05-15T14:08:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.428+0000 7fc1e3c46740 1 -- 172.21.15.33:0/4147857503 wait complete. 2024-05-15T14:08:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.428+0000 7fc1e3c46740 1 -- start start 2024-05-15T14:08:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.429+0000 7fc1e3c46740 1 Processor -- start 2024-05-15T14:08:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.429+0000 7fc1e3c46740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4a45b88f0 0x55e4a45c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.429+0000 7fc1e3c46740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4a45b9210 0x55e4a45c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.429+0000 7fc1e3c46740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a45c3ad0 0x55e4a463bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.429+0000 7fc1e3c46740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc1ace90 con 0x55e4a45b9210 2024-05-15T14:08:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.429+0000 7fc1e3c46740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc1ae300 con 0x55e4a45c3ad0 2024-05-15T14:08:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.429+0000 7fc1e3c46740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc1ad560 con 0x55e4a45b88f0 2024-05-15T14:08:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.429+0000 7fc1e3c46740 10 client.0 _ll_register_callbacks cb 0x7fc1cc005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:08:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.429+0000 7fc1dad9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4a45b88f0 0x55e4a45c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:05.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.429+0000 7fc1d9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4a45b9210 0x55e4a45c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.429+0000 7fc1dad9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a45c3ad0 0x55e4a463bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.429+0000 7fc1dad9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4a45b88f0 0x55e4a45c6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:36664/0 (socket says 172.21.15.33:36664) 2024-05-15T14:08:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.429+0000 7fc1dad9d640 1 -- 172.21.15.33:0/3568564759 learned_addr learned my addr 172.21.15.33:0/3568564759 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:08:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.429+0000 7fc1d9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4a45b9210 0x55e4a45c72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36712/0 (socket says 172.21.15.33:36712) 2024-05-15T14:08:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.429+0000 7fc1dad9d640 1 -- 172.21.15.33:0/3568564759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a45c3ad0 msgr2=0x55e4a463bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.429+0000 7fc1dad9d640 1 --2- 172.21.15.33:0/3568564759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a45c3ad0 0x55e4a463bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.429+0000 7fc1dad9d640 1 -- 172.21.15.33:0/3568564759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4a45b9210 msgr2=0x55e4a45c72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.429+0000 7fc1dad9d640 1 --2- 172.21.15.33:0/3568564759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4a45b9210 0x55e4a45c72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.429+0000 7fc1dad9d640 1 -- 172.21.15.33:0/3568564759 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4a463c240 con 0x55e4a45b88f0 2024-05-15T14:08:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.430+0000 7fc1dad9d640 1 --2- 172.21.15.33:0/3568564759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4a45b88f0 0x55e4a45c6d80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4010f20 tx=0x7fc1d40198f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e01ccf398667af7 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:05.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.430+0000 7fc1c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-05-15T14:08:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.430+0000 7fc1c1ffb640 1 -- 172.21.15.33:0/3568564759 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d401e070 con 0x55e4a45b88f0 2024-05-15T14:08:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.430+0000 7fc1c1ffb640 1 -- 172.21.15.33:0/3568564759 <== mon.1 v2:172.21.15.81:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8542+0+0 (secure 0 0 0) 0x7fc1d4011110 con 0x55e4a45b88f0 2024-05-15T14:08:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.430+0000 7fc1e3c46740 1 -- 172.21.15.33:0/3568564759 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1d0030900 con 0x55e4a45b88f0 2024-05-15T14:08:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.430+0000 7fc1c1ffb640 1 -- 172.21.15.33:0/3568564759 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc1d4011310 con 0x55e4a45b88f0 2024-05-15T14:08:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.430+0000 7fc1e3c46740 10 client.25909 fetch_fsmap learned FSMap version 30 2024-05-15T14:08:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.430+0000 7fc1e3c46740 1 -- 172.21.15.33:0/3568564759 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e4a463d550 con 0x55e4a45b88f0 2024-05-15T14:08:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.430+0000 7fc1c1ffb640 1 -- 172.21.15.33:0/3568564759 <== mon.1 v2:172.21.15.81:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1d0030900 con 0x55e4a45b88f0 2024-05-15T14:08:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.430+0000 7fc1c1ffb640 1 -- 172.21.15.33:0/3568564759 <== mon.1 v2:172.21.15.81:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7fc1d40289d0 con 0x55e4a45b88f0 2024-05-15T14:08:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.431+0000 7fc1e3c46740 10 client.25909 fetch_fsmap finished waiting for FSMap version 30 2024-05-15T14:08:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.431+0000 7fc1e3c46740 10 client.25909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:08:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.431+0000 7fc1e3c46740 20 client.25909 populate_metadata read hostname 'smithi033' 2024-05-15T14:08:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.431+0000 7fc1e3c46740 1 --2- 172.21.15.33:0/3568564759 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e4a463db30 0x55e4a463ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.431+0000 7fc1e3c46740 4 client.25909 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:08:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.431+0000 7fc1e3c46740 1 -- 172.21.15.33:0/3568564759 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e4a4527360 con 0x55e4a463db30 2024-05-15T14:08:05.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.431+0000 7fc1d9d9b640 1 --2- 172.21.15.33:0/3568564759 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e4a463db30 0x55e4a463ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:05.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.431+0000 7fc1d9d9b640 1 --2- 172.21.15.33:0/3568564759 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e4a463db30 0x55e4a463ff50 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f86a9d3e5d4c4084 server_cookie=229170e6ead13ce2 in_seq=0 out_seq=0 2024-05-15T14:08:05.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.431+0000 7fc1c1ffb640 10 client.25909 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:08:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.453+0000 7fc1c1ffb640 1 -- 172.21.15.33:0/3568564759 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e4a4527360 con 0x55e4a463db30 2024-05-15T14:08:05.457 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.453+0000 7fc1c1ffb640 10 client.25909 handle_command_reply: tid=0 2024-05-15T14:08:05.458 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.453+0000 7fc1e3c46740 1 client.25909 shutdown 2024-05-15T14:08:05.458 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.453+0000 7fc1e3c46740 1 -- 172.21.15.33:0/3568564759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4a45b88f0 msgr2=0x55e4a45c6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:05.458 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.453+0000 7fc1e3c46740 1 --2- 172.21.15.33:0/3568564759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4a45b88f0 0x55e4a45c6d80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4010f20 tx=0x7fc1d40198f0 comp rx=0 tx=0).stop 2024-05-15T14:08:05.458 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.453+0000 7fc1e3c46740 1 -- 172.21.15.33:0/3568564759 shutdown_connections 2024-05-15T14:08:05.458 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.453+0000 7fc1e3c46740 1 --2- 172.21.15.33:0/3568564759 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e4a463db30 0x55e4a463ff50 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:05.458 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.453+0000 7fc1e3c46740 1 --2- 172.21.15.33:0/3568564759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4a45c3ad0 0x55e4a463bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:05.458 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.453+0000 7fc1c1ffb640 0 client.25909 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:08:05.458 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.453+0000 7fc1e3c46740 1 --2- 172.21.15.33:0/3568564759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4a45b9210 0x55e4a45c72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.453+0000 7fc1e3c46740 1 --2- 172.21.15.33:0/3568564759 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e4a45b88f0 0x55e4a45c6d80 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.453+0000 7fc1e3c46740 1 -- 172.21.15.33:0/3568564759 >> 172.21.15.33:0/3568564759 conn(0x55e4a45b4e00 msgr2=0x55e4a45ba2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.453+0000 7fc1e3c46740 1 -- 172.21.15.33:0/3568564759 shutdown_connections 2024-05-15T14:08:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.453+0000 7fc1e3c46740 1 -- 172.21.15.33:0/3568564759 wait complete. 2024-05-15T14:08:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.453+0000 7fc1e3c46740 20 client.25909 trim_cache size 0 max 16384 2024-05-15T14:08:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:08:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:08:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:08:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.454+0000 7fc1e0f2b640 1 -- 172.21.15.33:0/1076146276 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc1c40b29d0 msgr2=0x7fc1c40b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.454+0000 7fc1e0f2b640 1 --2- 172.21.15.33:0/1076146276 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc1c40b29d0 0x7fc1c40b4e90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc0dfeb0 tx=0x7fc1cc0e8550 comp rx=0 tx=0).stop 2024-05-15T14:08:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.454+0000 7fc1e0f2b640 1 -- 172.21.15.33:0/1076146276 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1dc101930 msgr2=0x7fc1dc19a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.454+0000 7fc1e0f2b640 1 --2- 172.21.15.33:0/1076146276 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1dc101930 0x7fc1dc19a0c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc00e790 tx=0x7fc1cc00ec50 comp rx=0 tx=0).stop 2024-05-15T14:08:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.454+0000 7fc1e0f2b640 1 -- 172.21.15.33:0/1076146276 shutdown_connections 2024-05-15T14:08:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.454+0000 7fc1e0f2b640 1 --2- 172.21.15.33:0/1076146276 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc1c40b29d0 0x7fc1c40b4e90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.454+0000 7fc1e0f2b640 1 --2- 172.21.15.33:0/1076146276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1dc107940 0x7fc1dc1a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.454+0000 7fc1e0f2b640 1 --2- 172.21.15.33:0/1076146276 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1dc1022f0 0x7fc1dc19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.454+0000 7fc1e0f2b640 1 --2- 172.21.15.33:0/1076146276 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1dc101930 0x7fc1dc19a0c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.454+0000 7fc1e0f2b640 1 -- 172.21.15.33:0/1076146276 >> 172.21.15.33:0/1076146276 conn(0x7fc1dc078a70 msgr2=0x7fc1dc1045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.454+0000 7fc1e0f2b640 1 -- 172.21.15.33:0/1076146276 shutdown_connections 2024-05-15T14:08:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:05.454+0000 7fc1e0f2b640 1 -- 172.21.15.33:0/1076146276 wait complete. 2024-05-15T14:08:06.067 INFO:tasks.workunit.client.0.smithi033.stdout: 1 463328 6.17 MB/sec execute 221 sec latency 25.113 ms 2024-05-15T14:08:06.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:06 smithi081 ceph-mon[36434]: pgmap v238: 97 pgs: 97 active+clean; 126 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 628 KiB/s wr, 49 op/s 2024-05-15T14:08:06.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:06 smithi081 ceph-mon[36434]: daemon mds.f restarted 2024-05-15T14:08:06.607 INFO:journalctl@ceph.mds.f.smithi081.stdout:May 15 14:08:06 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-f[66264]: ignoring --setuser ceph since I am not root 2024-05-15T14:08:06.607 INFO:journalctl@ceph.mds.f.smithi081.stdout:May 15 14:08:06 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-f[66264]: ignoring --setgroup ceph since I am not root 2024-05-15T14:08:06.607 INFO:journalctl@ceph.mds.f.smithi081.stdout:May 15 14:08:06 smithi081 ceph-mds[66268]: -- 172.21.15.81:0/2268484927 <== mon.1 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55853f55da40 con 0x55853f530c00 2024-05-15T14:08:06.607 INFO:journalctl@ceph.mds.f.smithi081.stdout:May 15 14:08:06 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-f[66264]: starting mds.f at 2024-05-15T14:08:06.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:06 smithi033 ceph-mon[31650]: pgmap v238: 97 pgs: 97 active+clean; 126 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 628 KiB/s wr, 49 op/s 2024-05-15T14:08:06.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:06 smithi033 ceph-mon[31650]: daemon mds.f restarted 2024-05-15T14:08:06.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:06 smithi052 ceph-mon[36190]: pgmap v238: 97 pgs: 97 active+clean; 126 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 628 KiB/s wr, 49 op/s 2024-05-15T14:08:06.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:06 smithi052 ceph-mon[36190]: daemon mds.f restarted 2024-05-15T14:08:07.067 INFO:tasks.workunit.client.0.smithi033.stdout: 1 465685 6.20 MB/sec execute 222 sec latency 60.219 ms 2024-05-15T14:08:07.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:07 smithi033 ceph-mon[31650]: osdmap e87: 12 total, 12 up, 12 in 2024-05-15T14:08:07.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:07 smithi033 ceph-mon[31650]: mds.? [v2:172.21.15.81:6834/1297849376,v1:172.21.15.81:6835/1297849376] up:boot 2024-05-15T14:08:07.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:07 smithi033 ceph-mon[31650]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 2 up:standby-replay 7 up:standby 2024-05-15T14:08:07.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:07 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-15T14:08:07.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:07 smithi033 ceph-mon[31650]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:08:07.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:07 smithi052 ceph-mon[36190]: osdmap e87: 12 total, 12 up, 12 in 2024-05-15T14:08:07.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:07 smithi052 ceph-mon[36190]: mds.? [v2:172.21.15.81:6834/1297849376,v1:172.21.15.81:6835/1297849376] up:boot 2024-05-15T14:08:07.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:07 smithi052 ceph-mon[36190]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 2 up:standby-replay 7 up:standby 2024-05-15T14:08:07.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:07 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-15T14:08:07.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:07 smithi052 ceph-mon[36190]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:08:07.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:07 smithi081 ceph-mon[36434]: osdmap e87: 12 total, 12 up, 12 in 2024-05-15T14:08:07.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:07 smithi081 ceph-mon[36434]: mds.? [v2:172.21.15.81:6834/1297849376,v1:172.21.15.81:6835/1297849376] up:boot 2024-05-15T14:08:07.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:07 smithi081 ceph-mon[36434]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 2 up:standby-replay 7 up:standby 2024-05-15T14:08:07.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:07 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-15T14:08:07.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:07 smithi081 ceph-mon[36434]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:08:08.067 INFO:tasks.workunit.client.0.smithi033.stdout: 1 466424 6.18 MB/sec execute 223 sec latency 39.812 ms 2024-05-15T14:08:08.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:08 smithi033 ceph-mon[31650]: pgmap v240: 97 pgs: 97 active+clean; 136 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 1.6 MiB/s wr, 122 op/s 2024-05-15T14:08:08.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:08 smithi033 ceph-mon[31650]: osdmap e88: 12 total, 12 up, 12 in 2024-05-15T14:08:08.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:08 smithi052 ceph-mon[36190]: pgmap v240: 97 pgs: 97 active+clean; 136 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 1.6 MiB/s wr, 122 op/s 2024-05-15T14:08:08.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:08 smithi052 ceph-mon[36190]: osdmap e88: 12 total, 12 up, 12 in 2024-05-15T14:08:08.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:08 smithi081 ceph-mon[36434]: pgmap v240: 97 pgs: 97 active+clean; 136 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 1.6 MiB/s wr, 122 op/s 2024-05-15T14:08:08.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:08 smithi081 ceph-mon[36434]: osdmap e88: 12 total, 12 up, 12 in 2024-05-15T14:08:09.067 INFO:tasks.workunit.client.0.smithi033.stdout: 1 469213 6.23 MB/sec execute 224 sec latency 13.443 ms 2024-05-15T14:08:10.067 INFO:tasks.workunit.client.0.smithi033.stdout: 1 470552 6.23 MB/sec execute 225 sec latency 19.567 ms 2024-05-15T14:08:10.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:10 smithi033 ceph-mon[31650]: pgmap v242: 97 pgs: 97 active+clean; 136 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 2.0 MiB/s wr, 152 op/s 2024-05-15T14:08:10.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:10 smithi052 ceph-mon[36190]: pgmap v242: 97 pgs: 97 active+clean; 136 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 2.0 MiB/s wr, 152 op/s 2024-05-15T14:08:10.856 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:10 smithi081 ceph-mon[36434]: pgmap v242: 97 pgs: 97 active+clean; 136 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 2.0 MiB/s wr, 152 op/s 2024-05-15T14:08:11.067 INFO:tasks.workunit.client.0.smithi033.stdout: 1 472915 6.26 MB/sec execute 226 sec latency 16.300 ms 2024-05-15T14:08:12.067 INFO:tasks.workunit.client.0.smithi033.stdout: 1 474400 6.26 MB/sec execute 227 sec latency 18.695 ms 2024-05-15T14:08:12.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:12 smithi033 ceph-mon[31650]: pgmap v243: 97 pgs: 97 active+clean; 136 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 1.9 MiB/s wr, 115 op/s 2024-05-15T14:08:12.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:12 smithi052 ceph-mon[36190]: pgmap v243: 97 pgs: 97 active+clean; 136 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 1.9 MiB/s wr, 115 op/s 2024-05-15T14:08:12.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:12 smithi081 ceph-mon[36434]: pgmap v243: 97 pgs: 97 active+clean; 136 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 1.9 MiB/s wr, 115 op/s 2024-05-15T14:08:13.067 INFO:tasks.workunit.client.0.smithi033.stdout: 1 476486 6.29 MB/sec execute 228 sec latency 15.320 ms 2024-05-15T14:08:14.067 INFO:tasks.workunit.client.0.smithi033.stdout: 1 477935 6.29 MB/sec execute 229 sec latency 22.574 ms 2024-05-15T14:08:14.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:14 smithi033 ceph-mon[31650]: pgmap v244: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 239 op/s 2024-05-15T14:08:14.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:14 smithi052 ceph-mon[36190]: pgmap v244: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 239 op/s 2024-05-15T14:08:14.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:14 smithi081 ceph-mon[36434]: pgmap v244: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 239 op/s 2024-05-15T14:08:15.068 INFO:tasks.workunit.client.0.smithi033.stdout: 1 480079 6.32 MB/sec execute 230 sec latency 16.713 ms 2024-05-15T14:08:16.068 INFO:tasks.workunit.client.0.smithi033.stdout: 1 482058 6.32 MB/sec execute 231 sec latency 20.253 ms 2024-05-15T14:08:16.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:16 smithi052 ceph-mon[36190]: pgmap v245: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 214 op/s 2024-05-15T14:08:16.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:16 smithi052 ceph-mon[36190]: osdmap e89: 12 total, 12 up, 12 in 2024-05-15T14:08:16.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:16 smithi081 ceph-mon[36434]: pgmap v245: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 214 op/s 2024-05-15T14:08:16.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:16 smithi081 ceph-mon[36434]: osdmap e89: 12 total, 12 up, 12 in 2024-05-15T14:08:16.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:16 smithi033 ceph-mon[31650]: pgmap v245: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 214 op/s 2024-05-15T14:08:16.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:16 smithi033 ceph-mon[31650]: osdmap e89: 12 total, 12 up, 12 in 2024-05-15T14:08:17.068 INFO:tasks.workunit.client.0.smithi033.stdout: 1 483789 6.35 MB/sec execute 232 sec latency 10.763 ms 2024-05-15T14:08:18.068 INFO:tasks.workunit.client.0.smithi033.stdout: 1 485826 6.35 MB/sec execute 233 sec latency 18.692 ms 2024-05-15T14:08:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.550+0000 7fd456b4e640 1 -- 172.21.15.33:0/1516151506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd450101930 msgr2=0x7fd450101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.550+0000 7fd456b4e640 1 --2- 172.21.15.33:0/1516151506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd450101930 0x7fd450101db0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd438009930 tx=0x7fd43802a420 comp rx=0 tx=0).stop 2024-05-15T14:08:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.550+0000 7fd456b4e640 1 -- 172.21.15.33:0/1516151506 shutdown_connections 2024-05-15T14:08:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.550+0000 7fd456b4e640 1 --2- 172.21.15.33:0/1516151506 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4501022f0 0x7fd45010e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.550+0000 7fd456b4e640 1 --2- 172.21.15.33:0/1516151506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd450101930 0x7fd450101db0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.550+0000 7fd456b4e640 1 --2- 172.21.15.33:0/1516151506 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd450107940 0x7fd450107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.550+0000 7fd456b4e640 1 -- 172.21.15.33:0/1516151506 >> 172.21.15.33:0/1516151506 conn(0x7fd450078a70 msgr2=0x7fd450078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.550+0000 7fd456b4e640 1 -- 172.21.15.33:0/1516151506 shutdown_connections 2024-05-15T14:08:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.550+0000 7fd456b4e640 1 -- 172.21.15.33:0/1516151506 wait complete. 2024-05-15T14:08:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.550+0000 7fd456b4e640 1 Processor -- start 2024-05-15T14:08:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.550+0000 7fd456b4e640 1 -- start start 2024-05-15T14:08:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.551+0000 7fd456b4e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd450101930 0x7fd450199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.551+0000 7fd456b4e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4501022f0 0x7fd450199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.551+0000 7fd456b4e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd450107940 0x7fd45019dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.551+0000 7fd456b4e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd450111bd0 con 0x7fd450101930 2024-05-15T14:08:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.551+0000 7fd456b4e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd450111a50 con 0x7fd450107940 2024-05-15T14:08:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.551+0000 7fd456b4e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd450111d50 con 0x7fd4501022f0 2024-05-15T14:08:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.551+0000 7fd4548ea640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd450101930 0x7fd450199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.551+0000 7fd4548ea640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd450101930 0x7fd450199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43784/0 (socket says 172.21.15.33:43784) 2024-05-15T14:08:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.551+0000 7fd4548ea640 1 -- 172.21.15.33:0/1977400553 learned_addr learned my addr 172.21.15.33:0/1977400553 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:08:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.551+0000 7fd44ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4501022f0 0x7fd450199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.551+0000 7fd4550eb640 1 --2- 172.21.15.33:0/1977400553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd450107940 0x7fd45019dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.551+0000 7fd4548ea640 1 -- 172.21.15.33:0/1977400553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4501022f0 msgr2=0x7fd450199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.551+0000 7fd4548ea640 1 --2- 172.21.15.33:0/1977400553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4501022f0 0x7fd450199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.551+0000 7fd4548ea640 1 -- 172.21.15.33:0/1977400553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd450107940 msgr2=0x7fd45019dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.551+0000 7fd4548ea640 1 --2- 172.21.15.33:0/1977400553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd450107940 0x7fd45019dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.551+0000 7fd4548ea640 1 -- 172.21.15.33:0/1977400553 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd45019e2e0 con 0x7fd450101930 2024-05-15T14:08:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.551+0000 7fd4548ea640 1 --2- 172.21.15.33:0/1977400553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd450101930 0x7fd450199340 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd448009d50 tx=0x7fd44800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.551+0000 7fd44dffb640 1 -- 172.21.15.33:0/1977400553 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd448003b70 con 0x7fd450101930 2024-05-15T14:08:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.551+0000 7fd456b4e640 1 -- 172.21.15.33:0/1977400553 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd45019e5d0 con 0x7fd450101930 2024-05-15T14:08:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.551+0000 7fd44dffb640 1 -- 172.21.15.33:0/1977400553 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd448003d10 con 0x7fd450101930 2024-05-15T14:08:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.552+0000 7fd44dffb640 1 -- 172.21.15.33:0/1977400553 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4480108b0 con 0x7fd450101930 2024-05-15T14:08:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.552+0000 7fd456b4e640 1 -- 172.21.15.33:0/1977400553 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4501a5e60 con 0x7fd450101930 2024-05-15T14:08:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.555+0000 7fd459860740 1 Processor -- start 2024-05-15T14:08:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.555+0000 7fd44dffb640 1 -- 172.21.15.33:0/1977400553 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd448004000 con 0x7fd450101930 2024-05-15T14:08:18.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.556+0000 7fd459860740 1 -- start start 2024-05-15T14:08:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.556+0000 7fd459860740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556591c498f0 0x556591c49cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.556+0000 7fd459860740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556591c4a210 0x556591c54590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.556+0000 7fd459860740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556591c54ad0 0x556591c56ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.556+0000 7fd459860740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556591bbcc30 con 0x556591c4a210 2024-05-15T14:08:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.556+0000 7fd459860740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556591bac7d0 con 0x556591c54ad0 2024-05-15T14:08:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.556+0000 7fd459860740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x556591bb0810 con 0x556591c498f0 2024-05-15T14:08:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.556+0000 7fd4550eb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556591c498f0 0x556591c49cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.556+0000 7fd4548ea640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556591c4a210 0x556591c54590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.556+0000 7fd4550eb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556591c498f0 0x556591c49cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:53440/0 (socket says 172.21.15.33:53440) 2024-05-15T14:08:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.556+0000 7fd4550eb640 1 -- 172.21.15.33:0/4200325283 learned_addr learned my addr 172.21.15.33:0/4200325283 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:08:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.556+0000 7fd44ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556591c54ad0 0x556591c56ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.556+0000 7fd44dffb640 1 --2- 172.21.15.33:0/1977400553 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fd43c0b2950 0x7fd43c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.556+0000 7fd4550eb640 1 -- 172.21.15.33:0/4200325283 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556591c54ad0 msgr2=0x556591c56ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.556+0000 7fd4550eb640 1 --2- 172.21.15.33:0/4200325283 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556591c54ad0 0x556591c56ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.556+0000 7fd4550eb640 1 -- 172.21.15.33:0/4200325283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556591c4a210 msgr2=0x556591c54590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.556+0000 7fd4550eb640 1 --2- 172.21.15.33:0/4200325283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556591c4a210 0x556591c54590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.556+0000 7fd4550eb640 1 -- 172.21.15.33:0/4200325283 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565918c1210 con 0x556591c498f0 2024-05-15T14:08:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.556+0000 7fd4548ea640 1 --2- 172.21.15.33:0/4200325283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556591c4a210 0x556591c54590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:08:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.557+0000 7fd4550eb640 1 --2- 172.21.15.33:0/1977400553 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fd43c0b2950 0x7fd43c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.557+0000 7fd44dffb640 1 -- 172.21.15.33:0/1977400553 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fd4480e2c70 con 0x7fd450101930 2024-05-15T14:08:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.557+0000 7fd4550eb640 1 --2- 172.21.15.33:0/4200325283 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556591c498f0 0x556591c49cd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd44000b7a0 tx=0x7fd44000bc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74c927363137f94b server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.557+0000 7fd446ffd640 1 -- 172.21.15.33:0/4200325283 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd440024070 con 0x556591c498f0 2024-05-15T14:08:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.557+0000 7fd4550eb640 1 --2- 172.21.15.33:0/1977400553 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fd43c0b2950 0x7fd43c0b4e10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd440021000 tx=0x7fd44001e000 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.557+0000 7fd446ffd640 1 -- 172.21.15.33:0/4200325283 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd44001e930 con 0x556591c498f0 2024-05-15T14:08:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.557+0000 7fd446ffd640 1 -- 172.21.15.33:0/4200325283 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4400116a0 con 0x556591c498f0 2024-05-15T14:08:18.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.558+0000 7fd459860740 1 -- 172.21.15.33:0/4200325283 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556591c498f0 msgr2=0x556591c49cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.558+0000 7fd459860740 1 --2- 172.21.15.33:0/4200325283 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556591c498f0 0x556591c49cd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd44000b7a0 tx=0x7fd44000bc60 comp rx=0 tx=0).stop 2024-05-15T14:08:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.558+0000 7fd459860740 1 -- 172.21.15.33:0/4200325283 shutdown_connections 2024-05-15T14:08:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.558+0000 7fd459860740 1 --2- 172.21.15.33:0/4200325283 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556591c54ad0 0x556591c56ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.558+0000 7fd459860740 1 --2- 172.21.15.33:0/4200325283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556591c4a210 0x556591c54590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.558+0000 7fd459860740 1 --2- 172.21.15.33:0/4200325283 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556591c498f0 0x556591c49cd0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.558+0000 7fd459860740 1 -- 172.21.15.33:0/4200325283 >> 172.21.15.33:0/4200325283 conn(0x556591c45e00 msgr2=0x556591c461e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.558+0000 7fd459860740 1 -- 172.21.15.33:0/4200325283 shutdown_connections 2024-05-15T14:08:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.558+0000 7fd459860740 1 -- 172.21.15.33:0/4200325283 wait complete. 2024-05-15T14:08:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.558+0000 7fd459860740 1 -- start start 2024-05-15T14:08:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.558+0000 7fd459860740 1 Processor -- start 2024-05-15T14:08:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.559+0000 7fd459860740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556591c498f0 0x556591c54430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.559+0000 7fd459860740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556591c4a210 0x556591c51720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.559+0000 7fd459860740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556591c54ad0 0x556591c51c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.559+0000 7fd459860740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556591892790 con 0x556591c54ad0 2024-05-15T14:08:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.559+0000 7fd459860740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556591b68890 con 0x556591c498f0 2024-05-15T14:08:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.559+0000 7fd459860740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x556591c577a0 con 0x556591c4a210 2024-05-15T14:08:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.559+0000 7fd459860740 10 client.0 _ll_register_callbacks cb 0x556591b6bd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:08:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.559+0000 7fd4550eb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556591c54ad0 0x556591c51c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.559+0000 7fd44ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556591c4a210 0x556591c51720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.559+0000 7fd4548ea640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556591c498f0 0x556591c54430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.559+0000 7fd44ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556591c4a210 0x556591c51720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:53454/0 (socket says 172.21.15.33:53454) 2024-05-15T14:08:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.559+0000 7fd4548ea640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556591c498f0 0x556591c54430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:43892/0 (socket says 172.21.15.33:43892) 2024-05-15T14:08:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.559+0000 7fd44ffff640 1 -- 172.21.15.33:0/3737301328 learned_addr learned my addr 172.21.15.33:0/3737301328 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:08:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.559+0000 7fd44ffff640 1 -- 172.21.15.33:0/3737301328 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556591c498f0 msgr2=0x556591c54430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.559+0000 7fd44ffff640 1 --2- 172.21.15.33:0/3737301328 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556591c498f0 0x556591c54430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.559+0000 7fd44ffff640 1 -- 172.21.15.33:0/3737301328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556591c54ad0 msgr2=0x556591c51c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.559+0000 7fd44ffff640 1 --2- 172.21.15.33:0/3737301328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556591c54ad0 0x556591c51c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.559+0000 7fd44ffff640 1 -- 172.21.15.33:0/3737301328 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55659198d620 con 0x556591c4a210 2024-05-15T14:08:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.559+0000 7fd4548ea640 1 --2- 172.21.15.33:0/3737301328 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556591c498f0 0x556591c54430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:08:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.559+0000 7fd4550eb640 1 --2- 172.21.15.33:0/3737301328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556591c54ad0 0x556591c51c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:08:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.560+0000 7fd44ffff640 1 --2- 172.21.15.33:0/3737301328 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556591c4a210 0x556591c51720 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd43802a940 tx=0x7fd438031f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee48abf41102923 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.560+0000 7fd42ffff640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-05-15T14:08:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.560+0000 7fd42ffff640 1 -- 172.21.15.33:0/3737301328 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd438035040 con 0x556591c4a210 2024-05-15T14:08:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.560+0000 7fd459860740 1 -- 172.21.15.33:0/3737301328 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd44001e930 con 0x556591c4a210 2024-05-15T14:08:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.560+0000 7fd42ffff640 1 -- 172.21.15.33:0/3737301328 <== mon.1 v2:172.21.15.81:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fd4380317e0 con 0x556591c4a210 2024-05-15T14:08:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.560+0000 7fd42ffff640 1 -- 172.21.15.33:0/3737301328 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd438004040 con 0x556591c4a210 2024-05-15T14:08:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.560+0000 7fd459860740 10 client.25927 fetch_fsmap learned FSMap version 32 2024-05-15T14:08:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.560+0000 7fd459860740 1 -- 172.21.15.33:0/3737301328 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556591c631c0 con 0x556591c4a210 2024-05-15T14:08:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.560+0000 7fd42ffff640 1 -- 172.21.15.33:0/3737301328 <== mon.1 v2:172.21.15.81:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd44001e930 con 0x556591c4a210 2024-05-15T14:08:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.560+0000 7fd42ffff640 1 -- 172.21.15.33:0/3737301328 <== mon.1 v2:172.21.15.81:3300/0 5 ==== fsmap(e 32) ==== 8342+0+0 (secure 0 0 0) 0x7fd4380048c0 con 0x556591c4a210 2024-05-15T14:08:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.560+0000 7fd459860740 10 client.25927 fetch_fsmap finished waiting for FSMap version 32 2024-05-15T14:08:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.560+0000 7fd459860740 10 client.25927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:08:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.560+0000 7fd459860740 20 client.25927 populate_metadata read hostname 'smithi033' 2024-05-15T14:08:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.560+0000 7fd459860740 1 --2- 172.21.15.33:0/3737301328 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556591c64680 0x556591c66a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.560+0000 7fd459860740 4 client.25927 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:08:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.560+0000 7fd459860740 1 -- 172.21.15.33:0/3737301328 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556591bbeb80 con 0x556591c64680 2024-05-15T14:08:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.561+0000 7fd4548ea640 1 --2- 172.21.15.33:0/3737301328 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556591c64680 0x556591c66a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.561+0000 7fd4548ea640 1 --2- 172.21.15.33:0/3737301328 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556591c64680 0x556591c66a70 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38cdf2e3ced39f79 server_cookie=a4bf6d819461a21d in_seq=0 out_seq=0 2024-05-15T14:08:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.561+0000 7fd42ffff640 10 client.25927 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:08:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.561+0000 7fd42ffff640 1 -- 172.21.15.33:0/3737301328 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556591bbeb80 con 0x556591c64680 2024-05-15T14:08:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.562+0000 7fd42ffff640 10 client.25927 handle_command_reply: tid=0 2024-05-15T14:08:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.562+0000 7fd459860740 1 client.25927 shutdown 2024-05-15T14:08:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.562+0000 7fd459860740 1 -- 172.21.15.33:0/3737301328 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556591c4a210 msgr2=0x556591c51720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.562+0000 7fd459860740 1 --2- 172.21.15.33:0/3737301328 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556591c4a210 0x556591c51720 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd43802a940 tx=0x7fd438031f10 comp rx=0 tx=0).stop 2024-05-15T14:08:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.562+0000 7fd459860740 1 -- 172.21.15.33:0/3737301328 shutdown_connections 2024-05-15T14:08:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.562+0000 7fd459860740 1 --2- 172.21.15.33:0/3737301328 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556591c64680 0x556591c66a70 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.562+0000 7fd459860740 1 --2- 172.21.15.33:0/3737301328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556591c54ad0 0x556591c51c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.562+0000 7fd42ffff640 0 client.25927 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:08:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.562+0000 7fd459860740 1 --2- 172.21.15.33:0/3737301328 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556591c4a210 0x556591c51720 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.562+0000 7fd459860740 1 --2- 172.21.15.33:0/3737301328 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556591c498f0 0x556591c54430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.562+0000 7fd459860740 1 -- 172.21.15.33:0/3737301328 >> 172.21.15.33:0/3737301328 conn(0x556591c45e00 msgr2=0x556591c45100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.562+0000 7fd459860740 1 -- 172.21.15.33:0/3737301328 shutdown_connections 2024-05-15T14:08:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.562+0000 7fd459860740 1 -- 172.21.15.33:0/3737301328 wait complete. 2024-05-15T14:08:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.562+0000 7fd459860740 20 client.25927 trim_cache size 0 max 16384 2024-05-15T14:08:18.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.577+0000 7fd459860740 1 Processor -- start 2024-05-15T14:08:18.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.577+0000 7fd459860740 1 -- start start 2024-05-15T14:08:18.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.578+0000 7fd459860740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556591c498f0 0x556591c60670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:18.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.578+0000 7fd459860740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556591c4a210 0x556591c53f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:18.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.578+0000 7fd459860740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556591c54ad0 0x556591c6c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:18.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.578+0000 7fd459860740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd4501b0ff0 con 0x556591c498f0 2024-05-15T14:08:18.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.578+0000 7fd459860740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd4501ae890 con 0x556591c54ad0 2024-05-15T14:08:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.578+0000 7fd459860740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd4501ac700 con 0x556591c4a210 2024-05-15T14:08:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.578+0000 7fd4548ea640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556591c498f0 0x556591c60670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.578+0000 7fd4548ea640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556591c498f0 0x556591c60670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43806/0 (socket says 172.21.15.33:43806) 2024-05-15T14:08:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.578+0000 7fd4548ea640 1 -- 172.21.15.33:0/1132394619 learned_addr learned my addr 172.21.15.33:0/1132394619 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:08:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.578+0000 7fd4550eb640 1 --2- 172.21.15.33:0/1132394619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556591c54ad0 0x556591c6c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.578+0000 7fd44ffff640 1 --2- 172.21.15.33:0/1132394619 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556591c4a210 0x556591c53f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.578+0000 7fd4548ea640 1 -- 172.21.15.33:0/1132394619 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556591c4a210 msgr2=0x556591c53f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.578+0000 7fd4548ea640 1 --2- 172.21.15.33:0/1132394619 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556591c4a210 0x556591c53f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.578+0000 7fd4548ea640 1 -- 172.21.15.33:0/1132394619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556591c54ad0 msgr2=0x556591c6c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.578+0000 7fd4548ea640 1 --2- 172.21.15.33:0/1132394619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556591c54ad0 0x556591c6c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.578+0000 7fd4548ea640 1 -- 172.21.15.33:0/1132394619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556591c4a040 con 0x556591c498f0 2024-05-15T14:08:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.578+0000 7fd4548ea640 1 --2- 172.21.15.33:0/1132394619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556591c498f0 0x556591c60670 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fd44801f000 tx=0x7fd4480e6720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8adcc8651eb8f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.578+0000 7fd4467fc640 1 -- 172.21.15.33:0/1132394619 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4480a9830 con 0x556591c498f0 2024-05-15T14:08:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.578+0000 7fd4467fc640 1 -- 172.21.15.33:0/1132394619 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd448009820 con 0x556591c498f0 2024-05-15T14:08:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.578+0000 7fd4467fc640 1 -- 172.21.15.33:0/1132394619 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4480a9650 con 0x556591c498f0 2024-05-15T14:08:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.579+0000 7fd459860740 1 -- 172.21.15.33:0/1132394619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556591c498f0 msgr2=0x556591c60670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.579+0000 7fd459860740 1 --2- 172.21.15.33:0/1132394619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556591c498f0 0x556591c60670 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fd44801f000 tx=0x7fd4480e6720 comp rx=0 tx=0).stop 2024-05-15T14:08:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.579+0000 7fd459860740 1 -- 172.21.15.33:0/1132394619 shutdown_connections 2024-05-15T14:08:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.579+0000 7fd459860740 1 --2- 172.21.15.33:0/1132394619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556591c54ad0 0x556591c6c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.579+0000 7fd459860740 1 --2- 172.21.15.33:0/1132394619 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556591c4a210 0x556591c53f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.579+0000 7fd459860740 1 --2- 172.21.15.33:0/1132394619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556591c498f0 0x556591c60670 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.579+0000 7fd459860740 1 -- 172.21.15.33:0/1132394619 >> 172.21.15.33:0/1132394619 conn(0x556591c45e00 msgr2=0x556591c4cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.579+0000 7fd459860740 1 -- 172.21.15.33:0/1132394619 shutdown_connections 2024-05-15T14:08:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.579+0000 7fd459860740 1 -- 172.21.15.33:0/1132394619 wait complete. 2024-05-15T14:08:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.580+0000 7fd459860740 1 -- start start 2024-05-15T14:08:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.580+0000 7fd459860740 1 Processor -- start 2024-05-15T14:08:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.580+0000 7fd459860740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556591c498f0 0x556591c5cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.580+0000 7fd459860740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556591c4a210 0x556591c58130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.580+0000 7fd459860740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556591c54ad0 0x556591c58670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.580+0000 7fd459860740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd4501acd50 con 0x556591c54ad0 2024-05-15T14:08:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.580+0000 7fd459860740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd4501ae1c0 con 0x556591c498f0 2024-05-15T14:08:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.580+0000 7fd459860740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd4501ad420 con 0x556591c4a210 2024-05-15T14:08:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.580+0000 7fd459860740 10 client.0 _ll_register_callbacks cb 0x7fd4380095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:08:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.580+0000 7fd4550eb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556591c54ad0 0x556591c58670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.580+0000 7fd4550eb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556591c54ad0 0x556591c58670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43820/0 (socket says 172.21.15.33:43820) 2024-05-15T14:08:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.580+0000 7fd4550eb640 1 -- 172.21.15.33:0/1400545126 learned_addr learned my addr 172.21.15.33:0/1400545126 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:08:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.580+0000 7fd44ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556591c4a210 0x556591c58130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.580+0000 7fd4548ea640 1 --2- 172.21.15.33:0/1400545126 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556591c498f0 0x556591c5cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.581+0000 7fd44ffff640 1 -- 172.21.15.33:0/1400545126 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556591c498f0 msgr2=0x556591c5cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.581+0000 7fd44ffff640 1 --2- 172.21.15.33:0/1400545126 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556591c498f0 0x556591c5cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.581+0000 7fd44ffff640 1 -- 172.21.15.33:0/1400545126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556591c54ad0 msgr2=0x556591c58670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.581+0000 7fd44ffff640 1 --2- 172.21.15.33:0/1400545126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556591c54ad0 0x556591c58670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.581+0000 7fd44ffff640 1 -- 172.21.15.33:0/1400545126 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556591c58ce0 con 0x556591c4a210 2024-05-15T14:08:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.581+0000 7fd4550eb640 1 --2- 172.21.15.33:0/1400545126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556591c54ad0 0x556591c58670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:08:18.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.581+0000 7fd4548ea640 1 --2- 172.21.15.33:0/1400545126 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556591c498f0 0x556591c5cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:08:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.581+0000 7fd44ffff640 1 --2- 172.21.15.33:0/1400545126 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556591c4a210 0x556591c58130 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd4380096c0 tx=0x7fd438004410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e2dd134cb4d4afd server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.581+0000 7fd42ffff640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-05-15T14:08:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.581+0000 7fd42ffff640 1 -- 172.21.15.33:0/1400545126 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd438035070 con 0x556591c4a210 2024-05-15T14:08:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.581+0000 7fd42ffff640 1 -- 172.21.15.33:0/1400545126 <== mon.1 v2:172.21.15.81:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fd438004970 con 0x556591c4a210 2024-05-15T14:08:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.581+0000 7fd459860740 1 -- 172.21.15.33:0/1400545126 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd448009820 con 0x556591c4a210 2024-05-15T14:08:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.581+0000 7fd42ffff640 1 -- 172.21.15.33:0/1400545126 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd438053040 con 0x556591c4a210 2024-05-15T14:08:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.581+0000 7fd459860740 10 client.25933 fetch_fsmap learned FSMap version 32 2024-05-15T14:08:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.581+0000 7fd459860740 1 -- 172.21.15.33:0/1400545126 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556591cced90 con 0x556591c4a210 2024-05-15T14:08:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.581+0000 7fd42ffff640 1 -- 172.21.15.33:0/1400545126 <== mon.1 v2:172.21.15.81:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd448009820 con 0x556591c4a210 2024-05-15T14:08:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.582+0000 7fd42ffff640 1 -- 172.21.15.33:0/1400545126 <== mon.1 v2:172.21.15.81:3300/0 5 ==== fsmap(e 32) ==== 8342+0+0 (secure 0 0 0) 0x7fd438056070 con 0x556591c4a210 2024-05-15T14:08:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.582+0000 7fd459860740 10 client.25933 fetch_fsmap finished waiting for FSMap version 32 2024-05-15T14:08:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.582+0000 7fd459860740 10 client.25933 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:08:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.582+0000 7fd459860740 20 client.25933 populate_metadata read hostname 'smithi033' 2024-05-15T14:08:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.582+0000 7fd459860740 1 --2- 172.21.15.33:0/1400545126 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556591ccf590 0x556591cd1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.582+0000 7fd459860740 4 client.25933 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:08:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.582+0000 7fd459860740 1 -- 172.21.15.33:0/1400545126 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556591c41630 con 0x556591ccf590 2024-05-15T14:08:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.582+0000 7fd4548ea640 1 --2- 172.21.15.33:0/1400545126 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556591ccf590 0x556591cd1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.582+0000 7fd42ffff640 10 client.25933 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:08:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.582+0000 7fd4548ea640 1 --2- 172.21.15.33:0/1400545126 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556591ccf590 0x556591cd1980 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=232d1419bc518340 server_cookie=dac4026f09eaf1f9 in_seq=0 out_seq=0 2024-05-15T14:08:18.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.672+0000 7fd42ffff640 1 -- 172.21.15.33:0/1400545126 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556591c41630 con 0x556591ccf590 2024-05-15T14:08:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.672+0000 7fd42ffff640 10 client.25933 handle_command_reply: tid=0 2024-05-15T14:08:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.672+0000 7fd459860740 1 client.25933 shutdown 2024-05-15T14:08:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.672+0000 7fd459860740 1 -- 172.21.15.33:0/1400545126 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556591c4a210 msgr2=0x556591c58130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.672+0000 7fd459860740 1 --2- 172.21.15.33:0/1400545126 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556591c4a210 0x556591c58130 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd4380096c0 tx=0x7fd438004410 comp rx=0 tx=0).stop 2024-05-15T14:08:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.672+0000 7fd459860740 1 -- 172.21.15.33:0/1400545126 shutdown_connections 2024-05-15T14:08:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.672+0000 7fd459860740 1 --2- 172.21.15.33:0/1400545126 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556591ccf590 0x556591cd1980 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.672+0000 7fd459860740 1 --2- 172.21.15.33:0/1400545126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556591c54ad0 0x556591c58670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.672+0000 7fd459860740 1 --2- 172.21.15.33:0/1400545126 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556591c4a210 0x556591c58130 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.672+0000 7fd42ffff640 0 client.25933 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:08:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.672+0000 7fd459860740 1 --2- 172.21.15.33:0/1400545126 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556591c498f0 0x556591c5cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:18.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.672+0000 7fd459860740 1 -- 172.21.15.33:0/1400545126 >> 172.21.15.33:0/1400545126 conn(0x556591c45e00 msgr2=0x556591c4c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.673+0000 7fd459860740 1 -- 172.21.15.33:0/1400545126 shutdown_connections 2024-05-15T14:08:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.673+0000 7fd459860740 1 -- 172.21.15.33:0/1400545126 wait complete. 2024-05-15T14:08:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.673+0000 7fd459860740 20 client.25933 trim_cache size 0 max 16384 2024-05-15T14:08:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:08:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:08:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:08:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.674+0000 7fd456b4e640 1 -- 172.21.15.33:0/1977400553 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fd43c0b2950 msgr2=0x7fd43c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.674+0000 7fd456b4e640 1 --2- 172.21.15.33:0/1977400553 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fd43c0b2950 0x7fd43c0b4e10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd440021000 tx=0x7fd44001e000 comp rx=0 tx=0).stop 2024-05-15T14:08:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.674+0000 7fd456b4e640 1 -- 172.21.15.33:0/1977400553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd450101930 msgr2=0x7fd450199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.674+0000 7fd456b4e640 1 --2- 172.21.15.33:0/1977400553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd450101930 0x7fd450199340 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd448009d50 tx=0x7fd44800c6a0 comp rx=0 tx=0).stop 2024-05-15T14:08:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.674+0000 7fd456b4e640 1 -- 172.21.15.33:0/1977400553 shutdown_connections 2024-05-15T14:08:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.674+0000 7fd456b4e640 1 --2- 172.21.15.33:0/1977400553 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fd43c0b2950 0x7fd43c0b4e10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.674+0000 7fd456b4e640 1 --2- 172.21.15.33:0/1977400553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd450107940 0x7fd45019dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.674+0000 7fd456b4e640 1 --2- 172.21.15.33:0/1977400553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4501022f0 0x7fd450199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.674+0000 7fd456b4e640 1 --2- 172.21.15.33:0/1977400553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd450101930 0x7fd450199340 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.674+0000 7fd456b4e640 1 -- 172.21.15.33:0/1977400553 >> 172.21.15.33:0/1977400553 conn(0x7fd450078a70 msgr2=0x7fd450100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.674+0000 7fd456b4e640 1 -- 172.21.15.33:0/1977400553 shutdown_connections 2024-05-15T14:08:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:18.674+0000 7fd456b4e640 1 -- 172.21.15.33:0/1977400553 wait complete. 2024-05-15T14:08:18.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:18 smithi033 ceph-mon[31650]: pgmap v247: 97 pgs: 97 active+clean; 152 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 203 op/s 2024-05-15T14:08:18.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:18 smithi052 ceph-mon[36190]: pgmap v247: 97 pgs: 97 active+clean; 152 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 203 op/s 2024-05-15T14:08:18.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:18 smithi081 ceph-mon[36434]: pgmap v247: 97 pgs: 97 active+clean; 152 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 203 op/s 2024-05-15T14:08:19.068 INFO:tasks.workunit.client.0.smithi033.stdout: 1 487313 6.37 MB/sec execute 234 sec latency 79.789 ms 2024-05-15T14:08:20.068 INFO:tasks.workunit.client.0.smithi033.stdout: 1 489191 6.38 MB/sec execute 235 sec latency 97.838 ms 2024-05-15T14:08:20.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:20 smithi052 ceph-mon[36190]: pgmap v248: 97 pgs: 97 active+clean; 152 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 201 op/s 2024-05-15T14:08:20.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:20 smithi052 ceph-mon[36190]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:08:20.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:20 smithi081 ceph-mon[36434]: pgmap v248: 97 pgs: 97 active+clean; 152 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 201 op/s 2024-05-15T14:08:20.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:20 smithi081 ceph-mon[36434]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:08:20.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:20 smithi033 ceph-mon[31650]: pgmap v248: 97 pgs: 97 active+clean; 152 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 201 op/s 2024-05-15T14:08:20.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:20 smithi033 ceph-mon[31650]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:08:21.068 INFO:tasks.workunit.client.0.smithi033.stdout: 1 489191 6.35 MB/sec execute 236 sec latency 1097.883 ms 2024-05-15T14:08:22.068 INFO:tasks.workunit.client.0.smithi033.stdout: 1 489191 6.32 MB/sec execute 237 sec latency 2097.946 ms 2024-05-15T14:08:22.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:22 smithi052 ceph-mon[36190]: pgmap v249: 97 pgs: 97 active+clean; 152 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 201 op/s 2024-05-15T14:08:22.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:22 smithi081 ceph-mon[36434]: pgmap v249: 97 pgs: 97 active+clean; 152 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 201 op/s 2024-05-15T14:08:22.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:22 smithi033 ceph-mon[31650]: pgmap v249: 97 pgs: 97 active+clean; 152 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 201 op/s 2024-05-15T14:08:23.068 INFO:tasks.workunit.client.0.smithi033.stdout: 1 489191 6.30 MB/sec execute 238 sec latency 3098.040 ms 2024-05-15T14:08:24.068 INFO:tasks.workunit.client.0.smithi033.stdout: 1 489216 6.27 MB/sec execute 239 sec latency 3317.022 ms 2024-05-15T14:08:24.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:24 smithi052 ceph-mon[36190]: pgmap v250: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.6 MiB/s wr, 241 op/s 2024-05-15T14:08:24.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:24 smithi081 ceph-mon[36434]: pgmap v250: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.6 MiB/s wr, 241 op/s 2024-05-15T14:08:24.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:24 smithi033 ceph-mon[31650]: pgmap v250: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.6 MiB/s wr, 241 op/s 2024-05-15T14:08:25.068 INFO:tasks.workunit.client.0.smithi033.stdout: 1 489216 6.24 MB/sec execute 240 sec latency 1338.385 ms 2024-05-15T14:08:25.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:25 smithi052 ceph-mon[36190]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:08:25.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:25 smithi081 ceph-mon[36434]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:08:25.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:25 smithi033 ceph-mon[31650]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:08:26.068 INFO:tasks.workunit.client.0.smithi033.stdout: 1 489216 6.22 MB/sec execute 241 sec latency 2338.428 ms 2024-05-15T14:08:26.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:26 smithi052 ceph-mon[36190]: pgmap v251: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.6 MiB/s wr, 241 op/s 2024-05-15T14:08:26.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:26 smithi081 ceph-mon[36434]: pgmap v251: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.6 MiB/s wr, 241 op/s 2024-05-15T14:08:26.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:26 smithi033 ceph-mon[31650]: pgmap v251: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.6 MiB/s wr, 241 op/s 2024-05-15T14:08:27.068 INFO:tasks.workunit.client.0.smithi033.stdout: 1 489216 6.19 MB/sec execute 242 sec latency 3338.529 ms 2024-05-15T14:08:28.068 INFO:tasks.workunit.client.0.smithi033.stdout: 1 489216 6.17 MB/sec execute 243 sec latency 4338.610 ms 2024-05-15T14:08:28.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:28 smithi052 ceph-mon[36190]: pgmap v252: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.4 MiB/s wr, 210 op/s 2024-05-15T14:08:28.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:28 smithi081 ceph-mon[36434]: pgmap v252: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.4 MiB/s wr, 210 op/s 2024-05-15T14:08:28.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:28 smithi033 ceph-mon[31650]: pgmap v252: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.4 MiB/s wr, 210 op/s 2024-05-15T14:08:29.069 INFO:tasks.workunit.client.0.smithi033.stdout: 1 489357 6.15 MB/sec execute 244 sec latency 4555.285 ms 2024-05-15T14:08:30.069 INFO:tasks.workunit.client.0.smithi033.stdout: 1 489357 6.12 MB/sec execute 245 sec latency 1701.939 ms 2024-05-15T14:08:30.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:30 smithi052 ceph-mon[36190]: pgmap v253: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 604 KiB/s wr, 121 op/s 2024-05-15T14:08:30.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:30 smithi081 ceph-mon[36434]: pgmap v253: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 604 KiB/s wr, 121 op/s 2024-05-15T14:08:30.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:30 smithi033 ceph-mon[31650]: pgmap v253: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 604 KiB/s wr, 121 op/s 2024-05-15T14:08:31.069 INFO:tasks.workunit.client.0.smithi033.stdout: 1 489357 6.10 MB/sec execute 246 sec latency 2701.999 ms 2024-05-15T14:08:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.777+0000 7f0322425640 1 -- 172.21.15.33:0/3987134671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f031c108a60 msgr2=0x7f031c108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.777+0000 7f0322425640 1 --2- 172.21.15.33:0/3987134671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f031c108a60 0x7f031c108e40 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f030c009930 tx=0x7f030c02a420 comp rx=0 tx=0).stop 2024-05-15T14:08:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.777+0000 7f0322425640 1 -- 172.21.15.33:0/3987134671 shutdown_connections 2024-05-15T14:08:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.777+0000 7f0322425640 1 --2- 172.21.15.33:0/3987134671 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f031c103410 0x7f031c10f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.777+0000 7f0322425640 1 --2- 172.21.15.33:0/3987134671 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f031c102a50 0x7f031c102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.777+0000 7f0322425640 1 --2- 172.21.15.33:0/3987134671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f031c108a60 0x7f031c108e40 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.777+0000 7f0322425640 1 -- 172.21.15.33:0/3987134671 >> 172.21.15.33:0/3987134671 conn(0x7f031c0fe540 msgr2=0x7f031c100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.777+0000 7f0322425640 1 -- 172.21.15.33:0/3987134671 shutdown_connections 2024-05-15T14:08:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.777+0000 7f0322425640 1 -- 172.21.15.33:0/3987134671 wait complete. 2024-05-15T14:08:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.778+0000 7f0322425640 1 Processor -- start 2024-05-15T14:08:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.778+0000 7f0322425640 1 -- start start 2024-05-15T14:08:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.778+0000 7f0322425640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f031c102a50 0x7f031c074d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.778+0000 7f0322425640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f031c103410 0x7f031c075290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.778+0000 7f0322425640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f031c108a60 0x7f031c0757d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.778+0000 7f0322425640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f031c112d70 con 0x7f031c103410 2024-05-15T14:08:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.778+0000 7f0322425640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f031c112bf0 con 0x7f031c102a50 2024-05-15T14:08:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.778+0000 7f0322425640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f031c112ef0 con 0x7f031c108a60 2024-05-15T14:08:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.779+0000 7f0313fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f031c103410 0x7f031c075290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:31.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.779+0000 7f0313fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f031c103410 0x7f031c075290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51702/0 (socket says 172.21.15.33:51702) 2024-05-15T14:08:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.779+0000 7f0313fff640 1 -- 172.21.15.33:0/1563108834 learned_addr learned my addr 172.21.15.33:0/1563108834 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:08:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.779+0000 7f03209c2640 1 --2- 172.21.15.33:0/1563108834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f031c108a60 0x7f031c0757d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.779+0000 7f031bfff640 1 --2- 172.21.15.33:0/1563108834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f031c102a50 0x7f031c074d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.779+0000 7f0313fff640 1 -- 172.21.15.33:0/1563108834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f031c108a60 msgr2=0x7f031c0757d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.779+0000 7f0313fff640 1 --2- 172.21.15.33:0/1563108834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f031c108a60 0x7f031c0757d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.779+0000 7f0313fff640 1 -- 172.21.15.33:0/1563108834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f031c102a50 msgr2=0x7f031c074d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.779+0000 7f0313fff640 1 --2- 172.21.15.33:0/1563108834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f031c102a50 0x7f031c074d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.779+0000 7f0313fff640 1 -- 172.21.15.33:0/1563108834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f031c1a77b0 con 0x7f031c103410 2024-05-15T14:08:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.779+0000 7f03209c2640 1 --2- 172.21.15.33:0/1563108834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f031c108a60 0x7f031c0757d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:08:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.779+0000 7f0313fff640 1 --2- 172.21.15.33:0/1563108834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f031c103410 0x7f031c075290 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f030400d770 tx=0x7f030400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.779+0000 7f0319ffb640 1 -- 172.21.15.33:0/1563108834 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0304017070 con 0x7f031c103410 2024-05-15T14:08:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.779+0000 7f0319ffb640 1 -- 172.21.15.33:0/1563108834 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03040054d0 con 0x7f031c103410 2024-05-15T14:08:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.779+0000 7f0322425640 1 -- 172.21.15.33:0/1563108834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f031c1a7aa0 con 0x7f031c103410 2024-05-15T14:08:31.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.779+0000 7f0319ffb640 1 -- 172.21.15.33:0/1563108834 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0304004c30 con 0x7f031c103410 2024-05-15T14:08:31.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.779+0000 7f0322425640 1 -- 172.21.15.33:0/1563108834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f031c1a7fa0 con 0x7f031c103410 2024-05-15T14:08:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.783+0000 7f0325218740 1 Processor -- start 2024-05-15T14:08:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.783+0000 7f0325218740 1 -- start start 2024-05-15T14:08:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.783+0000 7f0319ffb640 1 -- 172.21.15.33:0/1563108834 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0304004e10 con 0x7f031c103410 2024-05-15T14:08:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.783+0000 7f0325218740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ad805e8f0 0x556ad805ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.783+0000 7f0325218740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ad805f210 0x556ad8069590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.783+0000 7f0325218740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ad8069ad0 0x556ad806bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.783+0000 7f0325218740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556ad7fd1c30 con 0x556ad805e8f0 2024-05-15T14:08:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.783+0000 7f0325218740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556ad7fc17d0 con 0x556ad805f210 2024-05-15T14:08:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.783+0000 7f0325218740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x556ad7fc5810 con 0x556ad8069ad0 2024-05-15T14:08:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.783+0000 7f03209c2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ad805e8f0 0x556ad805ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.783+0000 7f03209c2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ad805e8f0 0x556ad805ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51714/0 (socket says 172.21.15.33:51714) 2024-05-15T14:08:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.783+0000 7f03209c2640 1 -- 172.21.15.33:0/1929858101 learned_addr learned my addr 172.21.15.33:0/1929858101 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:08:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.783+0000 7f03209c2640 1 -- 172.21.15.33:0/1929858101 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ad8069ad0 msgr2=0x556ad806bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.783+0000 7f03209c2640 1 --2- 172.21.15.33:0/1929858101 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ad8069ad0 0x556ad806bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.783+0000 7f031bfff640 1 --2- 172.21.15.33:0/1929858101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ad805f210 0x556ad8069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.783+0000 7f03209c2640 1 -- 172.21.15.33:0/1929858101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ad805f210 msgr2=0x556ad8069590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:31.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.783+0000 7f03209c2640 1 --2- 172.21.15.33:0/1929858101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ad805f210 0x556ad8069590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.783+0000 7f03209c2640 1 -- 172.21.15.33:0/1929858101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ad7cd6210 con 0x556ad805e8f0 2024-05-15T14:08:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.784+0000 7f03209c2640 1 --2- 172.21.15.33:0/1929858101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ad805e8f0 0x556ad805ecd0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f0314009f30 tx=0x7f0314010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f754fec7fd17e8f server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.784+0000 7f03127fc640 1 -- 172.21.15.33:0/1929858101 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0314016070 con 0x556ad805e8f0 2024-05-15T14:08:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.784+0000 7f03127fc640 1 -- 172.21.15.33:0/1929858101 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f031401bb10 con 0x556ad805e8f0 2024-05-15T14:08:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.784+0000 7f03127fc640 1 -- 172.21.15.33:0/1929858101 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0314012ba0 con 0x556ad805e8f0 2024-05-15T14:08:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.784+0000 7f0319ffb640 1 --2- 172.21.15.33:0/1563108834 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f03080b2950 0x7f03080b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.784+0000 7f031bfff640 1 --2- 172.21.15.33:0/1563108834 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f03080b2950 0x7f03080b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.784+0000 7f0319ffb640 1 -- 172.21.15.33:0/1563108834 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f03040e2260 con 0x7f031c103410 2024-05-15T14:08:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.784+0000 7f0325218740 1 -- 172.21.15.33:0/1929858101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ad805e8f0 msgr2=0x556ad805ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.784+0000 7f0325218740 1 --2- 172.21.15.33:0/1929858101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ad805e8f0 0x556ad805ecd0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f0314009f30 tx=0x7f0314010800 comp rx=0 tx=0).stop 2024-05-15T14:08:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.784+0000 7f031bfff640 1 --2- 172.21.15.33:0/1563108834 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f03080b2950 0x7f03080b4e10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f030c006010 tx=0x7f030c005a20 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.785+0000 7f0325218740 1 -- 172.21.15.33:0/1929858101 shutdown_connections 2024-05-15T14:08:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.785+0000 7f0325218740 1 --2- 172.21.15.33:0/1929858101 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ad8069ad0 0x556ad806bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:31.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.785+0000 7f0325218740 1 --2- 172.21.15.33:0/1929858101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ad805f210 0x556ad8069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.785+0000 7f0325218740 1 --2- 172.21.15.33:0/1929858101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ad805e8f0 0x556ad805ecd0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.785+0000 7f0325218740 1 -- 172.21.15.33:0/1929858101 >> 172.21.15.33:0/1929858101 conn(0x556ad805ae00 msgr2=0x556ad805b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.785+0000 7f0325218740 1 -- 172.21.15.33:0/1929858101 shutdown_connections 2024-05-15T14:08:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.785+0000 7f0325218740 1 -- 172.21.15.33:0/1929858101 wait complete. 2024-05-15T14:08:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.785+0000 7f0325218740 1 -- start start 2024-05-15T14:08:31.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.785+0000 7f0325218740 1 Processor -- start 2024-05-15T14:08:31.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.785+0000 7f0325218740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ad805e8f0 0x556ad8069470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.786+0000 7f0325218740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ad805f210 0x556ad8066760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.786+0000 7f0325218740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ad8069ad0 0x556ad8066d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.786+0000 7f0325218740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556ad7ca7790 con 0x556ad8069ad0 2024-05-15T14:08:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.786+0000 7f0325218740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556ad7f7d890 con 0x556ad805e8f0 2024-05-15T14:08:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.786+0000 7f0325218740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x556ad806c7a0 con 0x556ad805f210 2024-05-15T14:08:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.786+0000 7f0325218740 10 client.0 _ll_register_callbacks cb 0x556ad7f80d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:08:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.786+0000 7f03209c2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ad8069ad0 0x556ad8066d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.786+0000 7f03209c2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ad8069ad0 0x556ad8066d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51720/0 (socket says 172.21.15.33:51720) 2024-05-15T14:08:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.786+0000 7f03209c2640 1 -- 172.21.15.33:0/4144609113 learned_addr learned my addr 172.21.15.33:0/4144609113 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:08:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.786+0000 7f031bfff640 1 --2- 172.21.15.33:0/4144609113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ad805e8f0 0x556ad8069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.786+0000 7f0313fff640 1 --2- 172.21.15.33:0/4144609113 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ad805f210 0x556ad8066760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.786+0000 7f03209c2640 1 -- 172.21.15.33:0/4144609113 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ad805f210 msgr2=0x556ad8066760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.786+0000 7f03209c2640 1 --2- 172.21.15.33:0/4144609113 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ad805f210 0x556ad8066760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:31.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.786+0000 7f03209c2640 1 -- 172.21.15.33:0/4144609113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ad805e8f0 msgr2=0x556ad8069470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.786+0000 7f03209c2640 1 --2- 172.21.15.33:0/4144609113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ad805e8f0 0x556ad8069470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.786+0000 7f03209c2640 1 -- 172.21.15.33:0/4144609113 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ad7da2620 con 0x556ad8069ad0 2024-05-15T14:08:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.786+0000 7f03209c2640 1 --2- 172.21.15.33:0/4144609113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ad8069ad0 0x556ad8066d00 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f03140127c0 tx=0x7f0314004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c22bacc47beb1f server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.786+0000 7f02fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:08:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.786+0000 7f02fb7fe640 1 -- 172.21.15.33:0/4144609113 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0314029060 con 0x556ad8069ad0 2024-05-15T14:08:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.786+0000 7f02fb7fe640 1 -- 172.21.15.33:0/4144609113 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f031401b670 con 0x556ad8069ad0 2024-05-15T14:08:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.786+0000 7f0325218740 1 -- 172.21.15.33:0/4144609113 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f031401bb10 con 0x556ad8069ad0 2024-05-15T14:08:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.787+0000 7f02fb7fe640 1 -- 172.21.15.33:0/4144609113 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0314016070 con 0x556ad8069ad0 2024-05-15T14:08:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.787+0000 7f0325218740 10 client.16341 fetch_fsmap learned FSMap version 34 2024-05-15T14:08:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.787+0000 7f0325218740 1 -- 172.21.15.33:0/4144609113 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556ad8078200 con 0x556ad8069ad0 2024-05-15T14:08:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.787+0000 7f02fb7fe640 1 -- 172.21.15.33:0/4144609113 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f031401bb10 con 0x556ad8069ad0 2024-05-15T14:08:31.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.787+0000 7f02fb7fe640 1 -- 172.21.15.33:0/4144609113 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f031400f040 con 0x556ad8069ad0 2024-05-15T14:08:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.787+0000 7f0325218740 10 client.16341 fetch_fsmap finished waiting for FSMap version 34 2024-05-15T14:08:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.787+0000 7f0325218740 10 client.16341 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:08:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.787+0000 7f0325218740 20 client.16341 populate_metadata read hostname 'smithi033' 2024-05-15T14:08:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.787+0000 7f0325218740 1 --2- 172.21.15.33:0/4144609113 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556ad8079600 0x556ad807b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.787+0000 7f0325218740 4 client.16341 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:08:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.787+0000 7f0325218740 1 -- 172.21.15.33:0/4144609113 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ad7fd3b80 con 0x556ad8079600 2024-05-15T14:08:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.787+0000 7f031bfff640 1 --2- 172.21.15.33:0/4144609113 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556ad8079600 0x556ad807b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.788+0000 7f02fb7fe640 10 client.16341 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:08:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.788+0000 7f031bfff640 1 --2- 172.21.15.33:0/4144609113 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556ad8079600 0x556ad807b9f0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82aebfa5b29767ee server_cookie=804f1d1d92c64ef in_seq=0 out_seq=0 2024-05-15T14:08:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.789+0000 7f02fb7fe640 1 -- 172.21.15.33:0/4144609113 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556ad7fd3b80 con 0x556ad8079600 2024-05-15T14:08:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.789+0000 7f02fb7fe640 10 client.16341 handle_command_reply: tid=0 2024-05-15T14:08:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.789+0000 7f0325218740 1 client.16341 shutdown 2024-05-15T14:08:31.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.789+0000 7f0325218740 1 -- 172.21.15.33:0/4144609113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ad8069ad0 msgr2=0x556ad8066d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.789+0000 7f0325218740 1 --2- 172.21.15.33:0/4144609113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ad8069ad0 0x556ad8066d00 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f03140127c0 tx=0x7f0314004c70 comp rx=0 tx=0).stop 2024-05-15T14:08:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.789+0000 7f0325218740 1 -- 172.21.15.33:0/4144609113 shutdown_connections 2024-05-15T14:08:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.789+0000 7f0325218740 1 --2- 172.21.15.33:0/4144609113 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556ad8079600 0x556ad807b9f0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.789+0000 7f0325218740 1 --2- 172.21.15.33:0/4144609113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ad8069ad0 0x556ad8066d00 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.789+0000 7f02fb7fe640 0 client.16341 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:08:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.789+0000 7f0325218740 1 --2- 172.21.15.33:0/4144609113 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ad805f210 0x556ad8066760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.789+0000 7f0325218740 1 --2- 172.21.15.33:0/4144609113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ad805e8f0 0x556ad8069470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.789+0000 7f0325218740 1 -- 172.21.15.33:0/4144609113 >> 172.21.15.33:0/4144609113 conn(0x556ad805ae00 msgr2=0x556ad805a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.789+0000 7f0325218740 1 -- 172.21.15.33:0/4144609113 shutdown_connections 2024-05-15T14:08:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.789+0000 7f0325218740 1 -- 172.21.15.33:0/4144609113 wait complete. 2024-05-15T14:08:31.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.789+0000 7f0325218740 20 client.16341 trim_cache size 0 max 16384 2024-05-15T14:08:31.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.804+0000 7f0325218740 1 Processor -- start 2024-05-15T14:08:31.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.804+0000 7f0325218740 1 -- start start 2024-05-15T14:08:31.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.805+0000 7f0325218740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ad805e8f0 0x556ad8075670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:31.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.805+0000 7f0325218740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ad805f210 0x556ad8068fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:31.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.805+0000 7f0325218740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ad8069ad0 0x556ad8080ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:31.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.805+0000 7f0325218740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f031c1b3380 con 0x556ad805f210 2024-05-15T14:08:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.805+0000 7f0325218740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f031c1b0bc0 con 0x556ad805e8f0 2024-05-15T14:08:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.805+0000 7f0325218740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f031c1aea30 con 0x556ad8069ad0 2024-05-15T14:08:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.805+0000 7f0313fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ad805f210 0x556ad8068fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.805+0000 7f0313fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ad805f210 0x556ad8068fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51732/0 (socket says 172.21.15.33:51732) 2024-05-15T14:08:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.805+0000 7f0313fff640 1 -- 172.21.15.33:0/4008828089 learned_addr learned my addr 172.21.15.33:0/4008828089 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:08:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.805+0000 7f03209c2640 1 --2- 172.21.15.33:0/4008828089 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ad8069ad0 0x556ad8080ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.805+0000 7f0313fff640 1 -- 172.21.15.33:0/4008828089 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ad8069ad0 msgr2=0x556ad8080ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.805+0000 7f0313fff640 1 --2- 172.21.15.33:0/4008828089 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ad8069ad0 0x556ad8080ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.805+0000 7f0313fff640 1 -- 172.21.15.33:0/4008828089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ad805e8f0 msgr2=0x556ad8075670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.805+0000 7f0313fff640 1 --2- 172.21.15.33:0/4008828089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ad805e8f0 0x556ad8075670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.805+0000 7f0313fff640 1 -- 172.21.15.33:0/4008828089 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ad805fbc0 con 0x556ad805f210 2024-05-15T14:08:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.805+0000 7f03209c2640 1 --2- 172.21.15.33:0/4008828089 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ad8069ad0 0x556ad8080ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:08:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.805+0000 7f0313fff640 1 --2- 172.21.15.33:0/4008828089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ad805f210 0x556ad8068fa0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f03040a6740 tx=0x7f03040a90a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=463bbe435fa29701 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:31.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.805+0000 7f0311ffb640 1 -- 172.21.15.33:0/4008828089 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0304020040 con 0x556ad805f210 2024-05-15T14:08:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.805+0000 7f0311ffb640 1 -- 172.21.15.33:0/4008828089 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03040a9bd0 con 0x556ad805f210 2024-05-15T14:08:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.805+0000 7f0311ffb640 1 -- 172.21.15.33:0/4008828089 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03040dfbc0 con 0x556ad805f210 2024-05-15T14:08:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.806+0000 7f0325218740 1 -- 172.21.15.33:0/4008828089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ad805f210 msgr2=0x556ad8068fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.806+0000 7f0325218740 1 --2- 172.21.15.33:0/4008828089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ad805f210 0x556ad8068fa0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f03040a6740 tx=0x7f03040a90a0 comp rx=0 tx=0).stop 2024-05-15T14:08:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.806+0000 7f0325218740 1 -- 172.21.15.33:0/4008828089 shutdown_connections 2024-05-15T14:08:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.806+0000 7f0325218740 1 --2- 172.21.15.33:0/4008828089 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ad8069ad0 0x556ad8080ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.806+0000 7f0325218740 1 --2- 172.21.15.33:0/4008828089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ad805f210 0x556ad8068fa0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.806+0000 7f0325218740 1 --2- 172.21.15.33:0/4008828089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ad805e8f0 0x556ad8075670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.806+0000 7f0325218740 1 -- 172.21.15.33:0/4008828089 >> 172.21.15.33:0/4008828089 conn(0x556ad805ae00 msgr2=0x556ad8060b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.806+0000 7f0325218740 1 -- 172.21.15.33:0/4008828089 shutdown_connections 2024-05-15T14:08:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.806+0000 7f0325218740 1 -- 172.21.15.33:0/4008828089 wait complete. 2024-05-15T14:08:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.806+0000 7f0325218740 1 -- start start 2024-05-15T14:08:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.807+0000 7f0325218740 1 Processor -- start 2024-05-15T14:08:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.807+0000 7f0325218740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ad805e8f0 0x556ad8071ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.807+0000 7f0325218740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ad805f210 0x556ad806d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.807+0000 7f0325218740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ad8069ad0 0x556ad806d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.807+0000 7f0325218740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f031c1af080 con 0x556ad805e8f0 2024-05-15T14:08:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.807+0000 7f0325218740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f031c1b04f0 con 0x556ad805f210 2024-05-15T14:08:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.807+0000 7f0325218740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f031c1af750 con 0x556ad8069ad0 2024-05-15T14:08:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.807+0000 7f0325218740 10 client.0 _ll_register_callbacks cb 0x7f0314009bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:08:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.807+0000 7f031bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ad805e8f0 0x556ad8071ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.807+0000 7f031bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ad805e8f0 0x556ad8071ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51744/0 (socket says 172.21.15.33:51744) 2024-05-15T14:08:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.807+0000 7f031bfff640 1 -- 172.21.15.33:0/3749679274 learned_addr learned my addr 172.21.15.33:0/3749679274 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:08:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.807+0000 7f0313fff640 1 --2- 172.21.15.33:0/3749679274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ad805f210 0x556ad806d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.807+0000 7f03209c2640 1 --2- 172.21.15.33:0/3749679274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ad8069ad0 0x556ad806d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.807+0000 7f031bfff640 1 -- 172.21.15.33:0/3749679274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ad8069ad0 msgr2=0x556ad806d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.807+0000 7f031bfff640 1 --2- 172.21.15.33:0/3749679274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ad8069ad0 0x556ad806d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.807+0000 7f031bfff640 1 -- 172.21.15.33:0/3749679274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ad805f210 msgr2=0x556ad806d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.807+0000 7f031bfff640 1 --2- 172.21.15.33:0/3749679274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ad805f210 0x556ad806d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.807+0000 7f031bfff640 1 -- 172.21.15.33:0/3749679274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ad806e180 con 0x556ad805e8f0 2024-05-15T14:08:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.807+0000 7f03209c2640 1 --2- 172.21.15.33:0/3749679274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ad8069ad0 0x556ad806d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:08:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.808+0000 7f031bfff640 1 --2- 172.21.15.33:0/3749679274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ad805e8f0 0x556ad8071ff0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f030c000c00 tx=0x7f030c02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df035a43ae2c58e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.808+0000 7f02fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:08:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.808+0000 7f02fb7fe640 1 -- 172.21.15.33:0/3749679274 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c035050 con 0x556ad805e8f0 2024-05-15T14:08:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.808+0000 7f02fb7fe640 1 -- 172.21.15.33:0/3749679274 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f030c042040 con 0x556ad805e8f0 2024-05-15T14:08:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.808+0000 7f0325218740 1 -- 172.21.15.33:0/3749679274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03040a9bd0 con 0x556ad805e8f0 2024-05-15T14:08:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.808+0000 7f02fb7fe640 1 -- 172.21.15.33:0/3749679274 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f030c05f050 con 0x556ad805e8f0 2024-05-15T14:08:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.808+0000 7f0325218740 10 client.16347 fetch_fsmap learned FSMap version 34 2024-05-15T14:08:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.808+0000 7f0325218740 1 -- 172.21.15.33:0/3749679274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556ad80e3c50 con 0x556ad805e8f0 2024-05-15T14:08:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.808+0000 7f02fb7fe640 1 -- 172.21.15.33:0/3749679274 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03040a9bd0 con 0x556ad805e8f0 2024-05-15T14:08:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.808+0000 7f02fb7fe640 1 -- 172.21.15.33:0/3749679274 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f030c050ce0 con 0x556ad805e8f0 2024-05-15T14:08:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.808+0000 7f0325218740 10 client.16347 fetch_fsmap finished waiting for FSMap version 34 2024-05-15T14:08:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.808+0000 7f0325218740 10 client.16347 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:08:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.808+0000 7f0325218740 20 client.16347 populate_metadata read hostname 'smithi033' 2024-05-15T14:08:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.808+0000 7f0325218740 1 --2- 172.21.15.33:0/3749679274 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556ad80e4400 0x556ad80e6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.808+0000 7f0325218740 4 client.16347 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:08:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.808+0000 7f0325218740 1 -- 172.21.15.33:0/3749679274 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556ad7fcd360 con 0x556ad80e4400 2024-05-15T14:08:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.809+0000 7f0313fff640 1 --2- 172.21.15.33:0/3749679274 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556ad80e4400 0x556ad80e6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.809+0000 7f02fb7fe640 10 client.16347 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:08:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.809+0000 7f0313fff640 1 --2- 172.21.15.33:0/3749679274 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556ad80e4400 0x556ad80e6820 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb23c5f3e6a6236 server_cookie=c1d74a3f333d0a88 in_seq=0 out_seq=0 2024-05-15T14:08:31.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.843+0000 7f02fb7fe640 1 -- 172.21.15.33:0/3749679274 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556ad7fcd360 con 0x556ad80e4400 2024-05-15T14:08:31.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.843+0000 7f02fb7fe640 10 client.16347 handle_command_reply: tid=0 2024-05-15T14:08:31.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.843+0000 7f0325218740 1 client.16347 shutdown 2024-05-15T14:08:31.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.844+0000 7f0325218740 1 -- 172.21.15.33:0/3749679274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ad805e8f0 msgr2=0x556ad8071ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:31.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.844+0000 7f0325218740 1 --2- 172.21.15.33:0/3749679274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ad805e8f0 0x556ad8071ff0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f030c000c00 tx=0x7f030c02a420 comp rx=0 tx=0).stop 2024-05-15T14:08:31.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.844+0000 7f0325218740 1 -- 172.21.15.33:0/3749679274 shutdown_connections 2024-05-15T14:08:31.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.844+0000 7f0325218740 1 --2- 172.21.15.33:0/3749679274 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556ad80e4400 0x556ad80e6820 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:31.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.844+0000 7f0325218740 1 --2- 172.21.15.33:0/3749679274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556ad8069ad0 0x556ad806d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:31.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.844+0000 7f0325218740 1 --2- 172.21.15.33:0/3749679274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ad805f210 0x556ad806d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:31.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.844+0000 7f02fb7fe640 0 client.16347 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:08:31.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.844+0000 7f0325218740 1 --2- 172.21.15.33:0/3749679274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ad805e8f0 0x556ad8071ff0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:31.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.844+0000 7f0325218740 1 -- 172.21.15.33:0/3749679274 >> 172.21.15.33:0/3749679274 conn(0x556ad805ae00 msgr2=0x556ad8061350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:31.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.844+0000 7f0325218740 1 -- 172.21.15.33:0/3749679274 shutdown_connections 2024-05-15T14:08:31.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.844+0000 7f0325218740 1 -- 172.21.15.33:0/3749679274 wait complete. 2024-05-15T14:08:31.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.844+0000 7f0325218740 20 client.16347 trim_cache size 0 max 16384 2024-05-15T14:08:31.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:08:31.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:08:31.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:08:31.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.844+0000 7f0322425640 1 -- 172.21.15.33:0/1563108834 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f03080b2950 msgr2=0x7f03080b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:31.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.844+0000 7f0322425640 1 --2- 172.21.15.33:0/1563108834 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f03080b2950 0x7f03080b4e10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f030c006010 tx=0x7f030c005a20 comp rx=0 tx=0).stop 2024-05-15T14:08:31.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.844+0000 7f0322425640 1 -- 172.21.15.33:0/1563108834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f031c103410 msgr2=0x7f031c075290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:31.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.844+0000 7f0322425640 1 --2- 172.21.15.33:0/1563108834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f031c103410 0x7f031c075290 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f030400d770 tx=0x7f030400dc30 comp rx=0 tx=0).stop 2024-05-15T14:08:31.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.844+0000 7f0322425640 1 -- 172.21.15.33:0/1563108834 shutdown_connections 2024-05-15T14:08:31.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.844+0000 7f0322425640 1 --2- 172.21.15.33:0/1563108834 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f03080b2950 0x7f03080b4e10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:31.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.844+0000 7f0322425640 1 --2- 172.21.15.33:0/1563108834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f031c108a60 0x7f031c0757d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:31.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.844+0000 7f0322425640 1 --2- 172.21.15.33:0/1563108834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f031c103410 0x7f031c075290 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:31.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.844+0000 7f0322425640 1 --2- 172.21.15.33:0/1563108834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f031c102a50 0x7f031c074d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:31.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.845+0000 7f0322425640 1 -- 172.21.15.33:0/1563108834 >> 172.21.15.33:0/1563108834 conn(0x7f031c0fe540 msgr2=0x7f031c0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:31.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.845+0000 7f0322425640 1 -- 172.21.15.33:0/1563108834 shutdown_connections 2024-05-15T14:08:31.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:31.845+0000 7f0322425640 1 -- 172.21.15.33:0/1563108834 wait complete. 2024-05-15T14:08:32.069 INFO:tasks.workunit.client.0.smithi033.stdout: 1 489357 6.07 MB/sec execute 247 sec latency 3702.111 ms 2024-05-15T14:08:32.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:32 smithi052 ceph-mon[36190]: pgmap v254: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 604 KiB/s wr, 121 op/s 2024-05-15T14:08:32.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:32 smithi081 ceph-mon[36434]: pgmap v254: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 604 KiB/s wr, 121 op/s 2024-05-15T14:08:32.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:32 smithi033 ceph-mon[31650]: pgmap v254: 97 pgs: 97 active+clean; 146 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 604 KiB/s wr, 121 op/s 2024-05-15T14:08:33.069 INFO:tasks.workunit.client.0.smithi033.stdout: 1 489357 6.05 MB/sec execute 248 sec latency 4702.220 ms 2024-05-15T14:08:34.069 INFO:tasks.workunit.client.0.smithi033.stdout: 1 489520 6.03 MB/sec execute 249 sec latency 4918.900 ms 2024-05-15T14:08:34.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:34 smithi052 ceph-mon[36190]: pgmap v255: 97 pgs: 97 active+clean; 147 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 651 KiB/s wr, 140 op/s 2024-05-15T14:08:34.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:34 smithi081 ceph-mon[36434]: pgmap v255: 97 pgs: 97 active+clean; 147 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 651 KiB/s wr, 140 op/s 2024-05-15T14:08:34.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:34 smithi033 ceph-mon[31650]: pgmap v255: 97 pgs: 97 active+clean; 147 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 651 KiB/s wr, 140 op/s 2024-05-15T14:08:35.069 INFO:tasks.workunit.client.0.smithi033.stdout: 1 489520 6.01 MB/sec execute 250 sec latency 1748.213 ms 2024-05-15T14:08:36.069 INFO:tasks.workunit.client.0.smithi033.stdout: 1 489520 5.99 MB/sec execute 251 sec latency 2748.352 ms 2024-05-15T14:08:36.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:36 smithi052 ceph-mon[36190]: pgmap v256: 97 pgs: 97 active+clean; 147 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 79 KiB/s wr, 25 op/s 2024-05-15T14:08:36.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:36 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:08:36.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:36 smithi081 ceph-mon[36434]: pgmap v256: 97 pgs: 97 active+clean; 147 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 79 KiB/s wr, 25 op/s 2024-05-15T14:08:36.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:36 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:08:36.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:36 smithi033 ceph-mon[31650]: pgmap v256: 97 pgs: 97 active+clean; 147 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 79 KiB/s wr, 25 op/s 2024-05-15T14:08:36.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:36 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:08:37.069 INFO:tasks.workunit.client.0.smithi033.stdout: 1 489520 5.96 MB/sec execute 252 sec latency 3748.412 ms 2024-05-15T14:08:37.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:37 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:08:37.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:37 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:08:37.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:37 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:08:37.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:37 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:08:37.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:37 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:08:37.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:37 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:08:37.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:37 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:08:37.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:37 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:08:37.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:37 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:08:38.069 INFO:tasks.workunit.client.0.smithi033.stdout: 1 489660 5.94 MB/sec execute 253 sec latency 4182.652 ms 2024-05-15T14:08:38.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:38 smithi052 ceph-mon[36190]: pgmap v257: 97 pgs: 97 active+clean; 147 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 88 KiB/s wr, 29 op/s 2024-05-15T14:08:38.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:38 smithi081 ceph-mon[36434]: pgmap v257: 97 pgs: 97 active+clean; 147 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 88 KiB/s wr, 29 op/s 2024-05-15T14:08:38.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:38 smithi033 ceph-mon[31650]: pgmap v257: 97 pgs: 97 active+clean; 147 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 88 KiB/s wr, 29 op/s 2024-05-15T14:08:39.069 INFO:tasks.workunit.client.0.smithi033.stdout: 1 489783 5.92 MB/sec execute 254 sec latency 742.918 ms 2024-05-15T14:08:40.070 INFO:tasks.workunit.client.0.smithi033.stdout: 1 489783 5.90 MB/sec execute 255 sec latency 1677.601 ms 2024-05-15T14:08:40.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:40 smithi052 ceph-mon[36190]: pgmap v258: 97 pgs: 97 active+clean; 147 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 56 KiB/s wr, 23 op/s 2024-05-15T14:08:40.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:40 smithi081 ceph-mon[36434]: pgmap v258: 97 pgs: 97 active+clean; 147 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 56 KiB/s wr, 23 op/s 2024-05-15T14:08:40.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:40 smithi033 ceph-mon[31650]: pgmap v258: 97 pgs: 97 active+clean; 147 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 56 KiB/s wr, 23 op/s 2024-05-15T14:08:41.070 INFO:tasks.workunit.client.0.smithi033.stdout: 1 489783 5.88 MB/sec execute 256 sec latency 2677.709 ms 2024-05-15T14:08:42.070 INFO:tasks.workunit.client.0.smithi033.stdout: 1 489783 5.85 MB/sec execute 257 sec latency 3677.811 ms 2024-05-15T14:08:42.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:42 smithi052 ceph-mon[36190]: pgmap v259: 97 pgs: 97 active+clean; 147 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 56 KiB/s wr, 23 op/s 2024-05-15T14:08:42.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:42 smithi081 ceph-mon[36434]: pgmap v259: 97 pgs: 97 active+clean; 147 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 56 KiB/s wr, 23 op/s 2024-05-15T14:08:42.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:42 smithi033 ceph-mon[31650]: pgmap v259: 97 pgs: 97 active+clean; 147 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 56 KiB/s wr, 23 op/s 2024-05-15T14:08:43.070 INFO:tasks.workunit.client.0.smithi033.stdout: 1 489783 5.83 MB/sec execute 258 sec latency 4677.914 ms 2024-05-15T14:08:44.070 INFO:tasks.workunit.client.0.smithi033.stdout: 1 490880 5.83 MB/sec execute 259 sec latency 4890.612 ms 2024-05-15T14:08:44.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:44 smithi052 ceph-mon[36190]: pgmap v260: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 2.0 MiB/s wr, 99 op/s 2024-05-15T14:08:44.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:44 smithi081 ceph-mon[36434]: pgmap v260: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 2.0 MiB/s wr, 99 op/s 2024-05-15T14:08:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.935+0000 7fce0686d640 1 -- 172.21.15.33:0/2857783018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce00103cc0 msgr2=0x7fce001040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.935+0000 7fce0686d640 1 --2- 172.21.15.33:0/2857783018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce00103cc0 0x7fce001040a0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009900 tx=0x7fcdf002a480 comp rx=0 tx=0).stop 2024-05-15T14:08:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.935+0000 7fce0686d640 1 -- 172.21.15.33:0/2857783018 shutdown_connections 2024-05-15T14:08:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.935+0000 7fce0686d640 1 --2- 172.21.15.33:0/2857783018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fce0010d090 0x7fce0010f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.935+0000 7fce0686d640 1 --2- 172.21.15.33:0/2857783018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce001045e0 0x7fce0010cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.935+0000 7fce0686d640 1 --2- 172.21.15.33:0/2857783018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce00103cc0 0x7fce001040a0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.935+0000 7fce0686d640 1 -- 172.21.15.33:0/2857783018 >> 172.21.15.33:0/2857783018 conn(0x7fce00078a70 msgr2=0x7fce00078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.935+0000 7fce0686d640 1 -- 172.21.15.33:0/2857783018 shutdown_connections 2024-05-15T14:08:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.935+0000 7fce0686d640 1 -- 172.21.15.33:0/2857783018 wait complete. 2024-05-15T14:08:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.936+0000 7fce0686d640 1 Processor -- start 2024-05-15T14:08:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.936+0000 7fce0686d640 1 -- start start 2024-05-15T14:08:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.936+0000 7fce0686d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fce00103cc0 0x7fce00199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.936+0000 7fce0686d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce001045e0 0x7fce001997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.936+0000 7fce0686d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce0010d090 0x7fce0019db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.936+0000 7fce0686d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fce00112d20 con 0x7fce0010d090 2024-05-15T14:08:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.936+0000 7fce0686d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fce00112ba0 con 0x7fce00103cc0 2024-05-15T14:08:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.936+0000 7fce0686d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fce00112ea0 con 0x7fce001045e0 2024-05-15T14:08:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.936+0000 7fce04e0a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce0010d090 0x7fce0019db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.936+0000 7fce04e0a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce0010d090 0x7fce0019db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54612/0 (socket says 172.21.15.33:54612) 2024-05-15T14:08:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.936+0000 7fce04e0a640 1 -- 172.21.15.33:0/1744977513 learned_addr learned my addr 172.21.15.33:0/1744977513 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:08:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.936+0000 7fcdf7fff640 1 --2- 172.21.15.33:0/1744977513 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce001045e0 0x7fce001997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.936+0000 7fcdfffff640 1 --2- 172.21.15.33:0/1744977513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fce00103cc0 0x7fce00199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.936+0000 7fce04e0a640 1 -- 172.21.15.33:0/1744977513 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce001045e0 msgr2=0x7fce001997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.936+0000 7fce04e0a640 1 --2- 172.21.15.33:0/1744977513 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce001045e0 0x7fce001997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.936+0000 7fce04e0a640 1 -- 172.21.15.33:0/1744977513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fce00103cc0 msgr2=0x7fce00199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.936+0000 7fce04e0a640 1 --2- 172.21.15.33:0/1744977513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fce00103cc0 0x7fce00199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.936+0000 7fce04e0a640 1 -- 172.21.15.33:0/1744977513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0019e250 con 0x7fce0010d090 2024-05-15T14:08:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.936+0000 7fce04e0a640 1 --2- 172.21.15.33:0/1744977513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce0010d090 0x7fce0019db20 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fcdf800e9d0 tx=0x7fcdf800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.937+0000 7fcdfdffb640 1 -- 172.21.15.33:0/1744977513 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf800cd80 con 0x7fce0010d090 2024-05-15T14:08:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.937+0000 7fcdfdffb640 1 -- 172.21.15.33:0/1744977513 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdf8005020 con 0x7fce0010d090 2024-05-15T14:08:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.937+0000 7fce0686d640 1 -- 172.21.15.33:0/1744977513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce0019e540 con 0x7fce0010d090 2024-05-15T14:08:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.937+0000 7fcdfdffb640 1 -- 172.21.15.33:0/1744977513 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf80116d0 con 0x7fce0010d090 2024-05-15T14:08:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.937+0000 7fce0686d640 1 -- 172.21.15.33:0/1744977513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce001a5dd0 con 0x7fce0010d090 2024-05-15T14:08:44.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:44 smithi033 ceph-mon[31650]: pgmap v260: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 2.0 MiB/s wr, 99 op/s 2024-05-15T14:08:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.939+0000 7fcdfdffb640 1 -- 172.21.15.33:0/1744977513 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fcdf80051d0 con 0x7fce0010d090 2024-05-15T14:08:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.939+0000 7fce09581740 1 Processor -- start 2024-05-15T14:08:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.939+0000 7fce09581740 1 -- start start 2024-05-15T14:08:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.940+0000 7fce09581740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556dcfd638f0 0x556dcfd63cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.940+0000 7fce09581740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dcfd64210 0x556dcfd6e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.940+0000 7fce09581740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556dcfd6ead0 0x556dcfd70ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.940+0000 7fce09581740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556dcfcd6c30 con 0x556dcfd64210 2024-05-15T14:08:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.940+0000 7fce09581740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556dcfcc67d0 con 0x556dcfd638f0 2024-05-15T14:08:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.940+0000 7fce09581740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x556dcfcca810 con 0x556dcfd6ead0 2024-05-15T14:08:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.940+0000 7fcdfffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dcfd64210 0x556dcfd6e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.940+0000 7fcdfffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dcfd64210 0x556dcfd6e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54616/0 (socket says 172.21.15.33:54616) 2024-05-15T14:08:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.940+0000 7fcdfffff640 1 -- 172.21.15.33:0/723298088 learned_addr learned my addr 172.21.15.33:0/723298088 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:08:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.940+0000 7fcdf7fff640 1 --2- 172.21.15.33:0/723298088 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556dcfd638f0 0x556dcfd63cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.940+0000 7fcdf7fff640 1 --2- 172.21.15.33:0/723298088 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556dcfd6ead0 0x556dcfd70ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.940+0000 7fcdfffff640 1 -- 172.21.15.33:0/723298088 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556dcfd6ead0 msgr2=0x556dcfd70ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.940+0000 7fcdfffff640 1 --2- 172.21.15.33:0/723298088 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556dcfd6ead0 0x556dcfd70ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.940+0000 7fcdfffff640 1 -- 172.21.15.33:0/723298088 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556dcfd638f0 msgr2=0x556dcfd63cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.940+0000 7fcdfffff640 1 --2- 172.21.15.33:0/723298088 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556dcfd638f0 0x556dcfd63cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.940+0000 7fcdfffff640 1 -- 172.21.15.33:0/723298088 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dcf9db210 con 0x556dcfd64210 2024-05-15T14:08:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.940+0000 7fcdfdffb640 1 --2- 172.21.15.33:0/1744977513 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fcdec0b2950 0x7fcdec0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.940+0000 7fcdf7fff640 1 --2- 172.21.15.33:0/1744977513 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fcdec0b2950 0x7fcdec0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.940+0000 7fcdfdffb640 1 -- 172.21.15.33:0/1744977513 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fcdf80e2810 con 0x7fce0010d090 2024-05-15T14:08:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.940+0000 7fcdfffff640 1 --2- 172.21.15.33:0/723298088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dcfd64210 0x556dcfd6e590 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009900 tx=0x7fcdf00040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b4639e211ed2805 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.941+0000 7fcdf67fc640 1 -- 172.21.15.33:0/723298088 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0035030 con 0x556dcfd64210 2024-05-15T14:08:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.941+0000 7fcdf67fc640 1 -- 172.21.15.33:0/723298088 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdf00043b0 con 0x556dcfd64210 2024-05-15T14:08:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.941+0000 7fcdf67fc640 1 -- 172.21.15.33:0/723298088 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0004de0 con 0x556dcfd64210 2024-05-15T14:08:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.941+0000 7fce09581740 1 -- 172.21.15.33:0/723298088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dcfd64210 msgr2=0x556dcfd6e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.941+0000 7fcdf7fff640 1 --2- 172.21.15.33:0/1744977513 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fcdec0b2950 0x7fcdec0b4e10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x556dcfd61220 tx=0x7fcde8004730 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.941+0000 7fce09581740 1 --2- 172.21.15.33:0/723298088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dcfd64210 0x556dcfd6e590 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009900 tx=0x7fcdf00040c0 comp rx=0 tx=0).stop 2024-05-15T14:08:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.941+0000 7fce09581740 1 -- 172.21.15.33:0/723298088 shutdown_connections 2024-05-15T14:08:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.941+0000 7fce09581740 1 --2- 172.21.15.33:0/723298088 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556dcfd6ead0 0x556dcfd70ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.941+0000 7fce09581740 1 --2- 172.21.15.33:0/723298088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dcfd64210 0x556dcfd6e590 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.941+0000 7fce09581740 1 --2- 172.21.15.33:0/723298088 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556dcfd638f0 0x556dcfd63cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.941+0000 7fce09581740 1 -- 172.21.15.33:0/723298088 >> 172.21.15.33:0/723298088 conn(0x556dcfd5fe00 msgr2=0x556dcfd601e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.941+0000 7fce09581740 1 -- 172.21.15.33:0/723298088 shutdown_connections 2024-05-15T14:08:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.941+0000 7fce09581740 1 -- 172.21.15.33:0/723298088 wait complete. 2024-05-15T14:08:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.942+0000 7fce09581740 1 -- start start 2024-05-15T14:08:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.942+0000 7fce09581740 1 Processor -- start 2024-05-15T14:08:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.942+0000 7fce09581740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556dcfd638f0 0x556dcfd6e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.942+0000 7fce09581740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556dcfd64210 0x556dcfd6b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.942+0000 7fce09581740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dcfd6ead0 0x556dcfd6bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.942+0000 7fce09581740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556dcf9ac790 con 0x556dcfd6ead0 2024-05-15T14:08:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.942+0000 7fce09581740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556dcfc82890 con 0x556dcfd638f0 2024-05-15T14:08:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.942+0000 7fce09581740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x556dcfd71740 con 0x556dcfd64210 2024-05-15T14:08:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.942+0000 7fce09581740 10 client.0 _ll_register_callbacks cb 0x556dcfc85d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:08:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.942+0000 7fce04e0a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dcfd6ead0 0x556dcfd6bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.942+0000 7fce04e0a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dcfd6ead0 0x556dcfd6bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54620/0 (socket says 172.21.15.33:54620) 2024-05-15T14:08:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.942+0000 7fcdfffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556dcfd638f0 0x556dcfd6e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.942+0000 7fce04e0a640 1 -- 172.21.15.33:0/2775965275 learned_addr learned my addr 172.21.15.33:0/2775965275 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:08:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.943+0000 7fcdf7fff640 1 --2- 172.21.15.33:0/2775965275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556dcfd64210 0x556dcfd6b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.943+0000 7fce04e0a640 1 -- 172.21.15.33:0/2775965275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556dcfd64210 msgr2=0x556dcfd6b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:44.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.943+0000 7fce04e0a640 1 --2- 172.21.15.33:0/2775965275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556dcfd64210 0x556dcfd6b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.943+0000 7fce04e0a640 1 -- 172.21.15.33:0/2775965275 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556dcfd638f0 msgr2=0x556dcfd6e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.943+0000 7fce04e0a640 1 --2- 172.21.15.33:0/2775965275 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556dcfd638f0 0x556dcfd6e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.943+0000 7fce04e0a640 1 -- 172.21.15.33:0/2775965275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dcfaa7620 con 0x556dcfd6ead0 2024-05-15T14:08:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.943+0000 7fcdfffff640 1 --2- 172.21.15.33:0/2775965275 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556dcfd638f0 0x556dcfd6e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:08:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.943+0000 7fce04e0a640 1 --2- 172.21.15.33:0/2775965275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dcfd6ead0 0x556dcfd6bcd0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80a97e0 tx=0x7fcdf80a96b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dfa51adc22806b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.943+0000 7fcddf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:08:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.943+0000 7fcddf7fe640 1 -- 172.21.15.33:0/2775965275 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf801f040 con 0x556dcfd6ead0 2024-05-15T14:08:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.943+0000 7fcddf7fe640 1 -- 172.21.15.33:0/2775965275 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fcdf80a8d40 con 0x556dcfd6ead0 2024-05-15T14:08:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.943+0000 7fce09581740 1 -- 172.21.15.33:0/2775965275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdf00043b0 con 0x556dcfd6ead0 2024-05-15T14:08:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.943+0000 7fcddf7fe640 1 -- 172.21.15.33:0/2775965275 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcdf80ade90 con 0x556dcfd6ead0 2024-05-15T14:08:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.943+0000 7fce09581740 10 client.16359 fetch_fsmap learned FSMap version 34 2024-05-15T14:08:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.943+0000 7fce09581740 1 -- 172.21.15.33:0/2775965275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556dcfd7d180 con 0x556dcfd6ead0 2024-05-15T14:08:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.943+0000 7fcddf7fe640 1 -- 172.21.15.33:0/2775965275 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdf00043b0 con 0x556dcfd6ead0 2024-05-15T14:08:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.944+0000 7fcddf7fe640 1 -- 172.21.15.33:0/2775965275 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fcdf800f040 con 0x556dcfd6ead0 2024-05-15T14:08:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.944+0000 7fce09581740 10 client.16359 fetch_fsmap finished waiting for FSMap version 34 2024-05-15T14:08:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.944+0000 7fce09581740 10 client.16359 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:08:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.944+0000 7fce09581740 20 client.16359 populate_metadata read hostname 'smithi033' 2024-05-15T14:08:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.944+0000 7fce09581740 1 --2- 172.21.15.33:0/2775965275 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556dcfd7e700 0x556dcfd80b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.944+0000 7fce09581740 4 client.16359 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:08:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.944+0000 7fce09581740 1 -- 172.21.15.33:0/2775965275 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556dcfcd8b80 con 0x556dcfd7e700 2024-05-15T14:08:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.944+0000 7fcdfffff640 1 --2- 172.21.15.33:0/2775965275 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556dcfd7e700 0x556dcfd80b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.944+0000 7fcddf7fe640 10 client.16359 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:08:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.944+0000 7fcdfffff640 1 --2- 172.21.15.33:0/2775965275 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556dcfd7e700 0x556dcfd80b50 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee223ba4da6e7319 server_cookie=de7742df55b527ca in_seq=0 out_seq=0 2024-05-15T14:08:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.945+0000 7fcddf7fe640 1 -- 172.21.15.33:0/2775965275 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556dcfcd8b80 con 0x556dcfd7e700 2024-05-15T14:08:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.945+0000 7fcddf7fe640 10 client.16359 handle_command_reply: tid=0 2024-05-15T14:08:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.945+0000 7fce09581740 1 client.16359 shutdown 2024-05-15T14:08:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.945+0000 7fce09581740 1 -- 172.21.15.33:0/2775965275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dcfd6ead0 msgr2=0x556dcfd6bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.945+0000 7fce09581740 1 --2- 172.21.15.33:0/2775965275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dcfd6ead0 0x556dcfd6bcd0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80a97e0 tx=0x7fcdf80a96b0 comp rx=0 tx=0).stop 2024-05-15T14:08:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.945+0000 7fce09581740 1 -- 172.21.15.33:0/2775965275 shutdown_connections 2024-05-15T14:08:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.945+0000 7fce09581740 1 --2- 172.21.15.33:0/2775965275 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556dcfd7e700 0x556dcfd80b50 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.945+0000 7fce09581740 1 --2- 172.21.15.33:0/2775965275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dcfd6ead0 0x556dcfd6bcd0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.945+0000 7fce09581740 1 --2- 172.21.15.33:0/2775965275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556dcfd64210 0x556dcfd6b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.945+0000 7fcddf7fe640 0 client.16359 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:08:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.945+0000 7fce09581740 1 --2- 172.21.15.33:0/2775965275 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556dcfd638f0 0x556dcfd6e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.945+0000 7fce09581740 1 -- 172.21.15.33:0/2775965275 >> 172.21.15.33:0/2775965275 conn(0x556dcfd5fe00 msgr2=0x556dcfd5f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.945+0000 7fce09581740 1 -- 172.21.15.33:0/2775965275 shutdown_connections 2024-05-15T14:08:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.945+0000 7fce09581740 1 -- 172.21.15.33:0/2775965275 wait complete. 2024-05-15T14:08:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.945+0000 7fce09581740 20 client.16359 trim_cache size 0 max 16384 2024-05-15T14:08:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.961+0000 7fce09581740 1 Processor -- start 2024-05-15T14:08:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.961+0000 7fce09581740 1 -- start start 2024-05-15T14:08:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.961+0000 7fce09581740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556dcfd638f0 0x556dcfd6df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.961+0000 7fce09581740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dcfd64210 0x556dcfd66a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.961+0000 7fce09581740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556dcfd6ead0 0x556dcfd66fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.961+0000 7fce09581740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fce001b1040 con 0x556dcfd64210 2024-05-15T14:08:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.961+0000 7fce09581740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fce001ae880 con 0x556dcfd6ead0 2024-05-15T14:08:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.961+0000 7fce09581740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fce001ac6f0 con 0x556dcfd638f0 2024-05-15T14:08:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.961+0000 7fcdf7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dcfd64210 0x556dcfd66a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.961+0000 7fcdf7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dcfd64210 0x556dcfd66a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54624/0 (socket says 172.21.15.33:54624) 2024-05-15T14:08:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.961+0000 7fcdf7fff640 1 -- 172.21.15.33:0/1089608245 learned_addr learned my addr 172.21.15.33:0/1089608245 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:08:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.961+0000 7fcdfffff640 1 --2- 172.21.15.33:0/1089608245 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556dcfd638f0 0x556dcfd6df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.961+0000 7fce04e0a640 1 --2- 172.21.15.33:0/1089608245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556dcfd6ead0 0x556dcfd66fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.961+0000 7fcdf7fff640 1 -- 172.21.15.33:0/1089608245 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556dcfd638f0 msgr2=0x556dcfd6df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.961+0000 7fcdf7fff640 1 --2- 172.21.15.33:0/1089608245 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556dcfd638f0 0x556dcfd6df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.961+0000 7fcdf7fff640 1 -- 172.21.15.33:0/1089608245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556dcfd6ead0 msgr2=0x556dcfd66fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.961+0000 7fcdf7fff640 1 --2- 172.21.15.33:0/1089608245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556dcfd6ead0 0x556dcfd66fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.961+0000 7fcdf7fff640 1 -- 172.21.15.33:0/1089608245 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dcfd5b830 con 0x556dcfd64210 2024-05-15T14:08:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.961+0000 7fce04e0a640 1 --2- 172.21.15.33:0/1089608245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556dcfd6ead0 0x556dcfd66fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:08:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.962+0000 7fcdf7fff640 1 --2- 172.21.15.33:0/1089608245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dcfd64210 0x556dcfd66a60 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fcde80045a0 tx=0x7fcde8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5038d56ec5a6d7f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.962+0000 7fcdf5ffb640 1 -- 172.21.15.33:0/1089608245 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde80204e0 con 0x556dcfd64210 2024-05-15T14:08:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.962+0000 7fcdf5ffb640 1 -- 172.21.15.33:0/1089608245 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcde8020b30 con 0x556dcfd64210 2024-05-15T14:08:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.962+0000 7fcdf5ffb640 1 -- 172.21.15.33:0/1089608245 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde801e910 con 0x556dcfd64210 2024-05-15T14:08:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.962+0000 7fce09581740 1 -- 172.21.15.33:0/1089608245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dcfd64210 msgr2=0x556dcfd66a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.962+0000 7fce09581740 1 --2- 172.21.15.33:0/1089608245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dcfd64210 0x556dcfd66a60 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fcde80045a0 tx=0x7fcde8015000 comp rx=0 tx=0).stop 2024-05-15T14:08:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.962+0000 7fce09581740 1 -- 172.21.15.33:0/1089608245 shutdown_connections 2024-05-15T14:08:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.962+0000 7fce09581740 1 --2- 172.21.15.33:0/1089608245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556dcfd6ead0 0x556dcfd66fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.962+0000 7fce09581740 1 --2- 172.21.15.33:0/1089608245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dcfd64210 0x556dcfd66a60 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.962+0000 7fce09581740 1 --2- 172.21.15.33:0/1089608245 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556dcfd638f0 0x556dcfd6df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.962+0000 7fce09581740 1 -- 172.21.15.33:0/1089608245 >> 172.21.15.33:0/1089608245 conn(0x556dcfd5fe00 msgr2=0x556dcfd66350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.963+0000 7fce09581740 1 -- 172.21.15.33:0/1089608245 shutdown_connections 2024-05-15T14:08:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.963+0000 7fce09581740 1 -- 172.21.15.33:0/1089608245 wait complete. 2024-05-15T14:08:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.963+0000 7fce09581740 1 -- start start 2024-05-15T14:08:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.963+0000 7fce09581740 1 Processor -- start 2024-05-15T14:08:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.963+0000 7fce09581740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556dcfd638f0 0x556dcfd725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.963+0000 7fce09581740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556dcfd64210 0x556dcfd72b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.963+0000 7fce09581740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dcfd6ead0 0x556dcfd73040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.963+0000 7fce09581740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fce001acd40 con 0x556dcfd6ead0 2024-05-15T14:08:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.963+0000 7fce09581740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fce001ae1b0 con 0x556dcfd64210 2024-05-15T14:08:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.963+0000 7fce09581740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fce001ad410 con 0x556dcfd638f0 2024-05-15T14:08:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.963+0000 7fce09581740 10 client.0 _ll_register_callbacks cb 0x7fcdf80a5270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:08:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.963+0000 7fce04e0a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dcfd6ead0 0x556dcfd73040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.963+0000 7fcdf7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556dcfd64210 0x556dcfd72b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.964+0000 7fce04e0a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dcfd6ead0 0x556dcfd73040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54630/0 (socket says 172.21.15.33:54630) 2024-05-15T14:08:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.964+0000 7fcdf7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556dcfd64210 0x556dcfd72b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:59728/0 (socket says 172.21.15.33:59728) 2024-05-15T14:08:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.964+0000 7fce04e0a640 1 -- 172.21.15.33:0/3116246150 learned_addr learned my addr 172.21.15.33:0/3116246150 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:08:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.964+0000 7fcdfffff640 1 --2- 172.21.15.33:0/3116246150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556dcfd638f0 0x556dcfd725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.964+0000 7fce04e0a640 1 -- 172.21.15.33:0/3116246150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556dcfd638f0 msgr2=0x556dcfd725c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.964+0000 7fce04e0a640 1 --2- 172.21.15.33:0/3116246150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556dcfd638f0 0x556dcfd725c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.964+0000 7fce04e0a640 1 -- 172.21.15.33:0/3116246150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556dcfd64210 msgr2=0x556dcfd72b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.964+0000 7fce04e0a640 1 --2- 172.21.15.33:0/3116246150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556dcfd64210 0x556dcfd72b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.964+0000 7fce04e0a640 1 -- 172.21.15.33:0/3116246150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dcfde8090 con 0x556dcfd6ead0 2024-05-15T14:08:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.964+0000 7fcdf7fff640 1 --2- 172.21.15.33:0/3116246150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556dcfd64210 0x556dcfd72b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:08:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.964+0000 7fce04e0a640 1 --2- 172.21.15.33:0/3116246150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dcfd6ead0 0x556dcfd73040 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8009fd0 tx=0x7fcdf80a5ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fae80aed44c17eb server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.964+0000 7fcddf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:08:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.964+0000 7fcddf7fe640 1 -- 172.21.15.33:0/3116246150 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf801f070 con 0x556dcfd6ead0 2024-05-15T14:08:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.964+0000 7fcddf7fe640 1 -- 172.21.15.33:0/3116246150 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7fcdf80a87b0 con 0x556dcfd6ead0 2024-05-15T14:08:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.964+0000 7fce09581740 1 -- 172.21.15.33:0/3116246150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcde8020b30 con 0x556dcfd6ead0 2024-05-15T14:08:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.964+0000 7fcddf7fe640 1 -- 172.21.15.33:0/3116246150 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcdf808f050 con 0x556dcfd6ead0 2024-05-15T14:08:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.964+0000 7fce09581740 10 client.16365 fetch_fsmap learned FSMap version 34 2024-05-15T14:08:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.964+0000 7fce09581740 1 -- 172.21.15.33:0/3116246150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556dcfde8090 con 0x556dcfd6ead0 2024-05-15T14:08:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.964+0000 7fcddf7fe640 1 -- 172.21.15.33:0/3116246150 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcde8020b30 con 0x556dcfd6ead0 2024-05-15T14:08:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.965+0000 7fcddf7fe640 1 -- 172.21.15.33:0/3116246150 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fcdf8088060 con 0x556dcfd6ead0 2024-05-15T14:08:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.965+0000 7fce09581740 10 client.16365 fetch_fsmap finished waiting for FSMap version 34 2024-05-15T14:08:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.965+0000 7fce09581740 10 client.16365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:08:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.965+0000 7fce09581740 20 client.16365 populate_metadata read hostname 'smithi033' 2024-05-15T14:08:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.965+0000 7fce09581740 1 --2- 172.21.15.33:0/3116246150 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556dcfde9740 0x556dcfdebb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.965+0000 7fce09581740 4 client.16365 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:08:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.965+0000 7fce09581740 1 -- 172.21.15.33:0/3116246150 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556dcfcd2360 con 0x556dcfde9740 2024-05-15T14:08:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.965+0000 7fcdfffff640 1 --2- 172.21.15.33:0/3116246150 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556dcfde9740 0x556dcfdebb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.965+0000 7fcdfffff640 1 --2- 172.21.15.33:0/3116246150 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556dcfde9740 0x556dcfdebb60 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1b10bdd764289fc server_cookie=1d7966c724b02c0e in_seq=0 out_seq=0 2024-05-15T14:08:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.965+0000 7fcddf7fe640 10 client.16365 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:08:45.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.998+0000 7fcddf7fe640 1 -- 172.21.15.33:0/3116246150 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556dcfcd2360 con 0x556dcfde9740 2024-05-15T14:08:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.998+0000 7fcddf7fe640 10 client.16365 handle_command_reply: tid=0 2024-05-15T14:08:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.998+0000 7fce09581740 1 client.16365 shutdown 2024-05-15T14:08:45.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.999+0000 7fce09581740 1 -- 172.21.15.33:0/3116246150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dcfd6ead0 msgr2=0x556dcfd73040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:45.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.999+0000 7fce09581740 1 --2- 172.21.15.33:0/3116246150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dcfd6ead0 0x556dcfd73040 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8009fd0 tx=0x7fcdf80a5ff0 comp rx=0 tx=0).stop 2024-05-15T14:08:45.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.999+0000 7fce09581740 1 -- 172.21.15.33:0/3116246150 shutdown_connections 2024-05-15T14:08:45.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.999+0000 7fce09581740 1 --2- 172.21.15.33:0/3116246150 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556dcfde9740 0x556dcfdebb60 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:45.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.999+0000 7fce09581740 1 --2- 172.21.15.33:0/3116246150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556dcfd6ead0 0x556dcfd73040 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:45.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.999+0000 7fcddf7fe640 0 client.16365 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:08:45.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.999+0000 7fce09581740 1 --2- 172.21.15.33:0/3116246150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556dcfd64210 0x556dcfd72b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:45.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.999+0000 7fce09581740 1 --2- 172.21.15.33:0/3116246150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556dcfd638f0 0x556dcfd725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:45.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.999+0000 7fce09581740 1 -- 172.21.15.33:0/3116246150 >> 172.21.15.33:0/3116246150 conn(0x556dcfd5fe00 msgr2=0x556dcfd66be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:45.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.999+0000 7fce09581740 1 -- 172.21.15.33:0/3116246150 shutdown_connections 2024-05-15T14:08:45.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.999+0000 7fce09581740 1 -- 172.21.15.33:0/3116246150 wait complete. 2024-05-15T14:08:45.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.999+0000 7fce09581740 20 client.16365 trim_cache size 0 max 16384 2024-05-15T14:08:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:08:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:08:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:08:45.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.999+0000 7fce0686d640 1 -- 172.21.15.33:0/1744977513 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fcdec0b2950 msgr2=0x7fcdec0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:45.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.999+0000 7fce0686d640 1 --2- 172.21.15.33:0/1744977513 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fcdec0b2950 0x7fcdec0b4e10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x556dcfd61220 tx=0x7fcde8004730 comp rx=0 tx=0).stop 2024-05-15T14:08:45.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.999+0000 7fce0686d640 1 -- 172.21.15.33:0/1744977513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce0010d090 msgr2=0x7fce0019db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:45.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.999+0000 7fce0686d640 1 --2- 172.21.15.33:0/1744977513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce0010d090 0x7fce0019db20 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fcdf800e9d0 tx=0x7fcdf800ee90 comp rx=0 tx=0).stop 2024-05-15T14:08:45.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.999+0000 7fce0686d640 1 -- 172.21.15.33:0/1744977513 shutdown_connections 2024-05-15T14:08:45.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.999+0000 7fce0686d640 1 --2- 172.21.15.33:0/1744977513 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fcdec0b2950 0x7fcdec0b4e10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:45.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.999+0000 7fce0686d640 1 --2- 172.21.15.33:0/1744977513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce0010d090 0x7fce0019db20 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:45.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.999+0000 7fce0686d640 1 --2- 172.21.15.33:0/1744977513 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce001045e0 0x7fce001997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:45.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.999+0000 7fce0686d640 1 --2- 172.21.15.33:0/1744977513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fce00103cc0 0x7fce00199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:45.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:44.999+0000 7fce0686d640 1 -- 172.21.15.33:0/1744977513 >> 172.21.15.33:0/1744977513 conn(0x7fce00078a70 msgr2=0x7fce000fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:45.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:45.000+0000 7fce0686d640 1 -- 172.21.15.33:0/1744977513 shutdown_connections 2024-05-15T14:08:45.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:45.000+0000 7fce0686d640 1 -- 172.21.15.33:0/1744977513 wait complete. 2024-05-15T14:08:45.070 INFO:tasks.workunit.client.0.smithi033.stdout: 1 492644 5.84 MB/sec execute 260 sec latency 218.580 ms 2024-05-15T14:08:46.070 INFO:tasks.workunit.client.0.smithi033.stdout: 1 492644 5.81 MB/sec execute 261 sec latency 1218.696 ms 2024-05-15T14:08:46.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:46 smithi052 ceph-mon[36190]: pgmap v261: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 1.9 MiB/s wr, 80 op/s 2024-05-15T14:08:46.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:46 smithi081 ceph-mon[36434]: pgmap v261: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 1.9 MiB/s wr, 80 op/s 2024-05-15T14:08:46.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:46 smithi033 ceph-mon[31650]: pgmap v261: 97 pgs: 97 active+clean; 144 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 1.9 MiB/s wr, 80 op/s 2024-05-15T14:08:47.070 INFO:tasks.workunit.client.0.smithi033.stdout: 1 492644 5.79 MB/sec execute 262 sec latency 2218.829 ms 2024-05-15T14:08:47.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:47 smithi033 ceph-mon[31650]: pgmap v262: 97 pgs: 97 active+clean; 143 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 2.2 MiB/s wr, 129 op/s 2024-05-15T14:08:48.070 INFO:tasks.workunit.client.0.smithi033.stdout: 1 492644 5.77 MB/sec execute 263 sec latency 3218.899 ms 2024-05-15T14:08:48.090 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:47 smithi052 ceph-mon[36190]: pgmap v262: 97 pgs: 97 active+clean; 143 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 2.2 MiB/s wr, 129 op/s 2024-05-15T14:08:48.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:47 smithi081 ceph-mon[36434]: pgmap v262: 97 pgs: 97 active+clean; 143 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 2.2 MiB/s wr, 129 op/s 2024-05-15T14:08:49.070 INFO:tasks.workunit.client.0.smithi033.stdout: 1 492799 5.75 MB/sec execute 264 sec latency 3433.336 ms 2024-05-15T14:08:50.071 INFO:tasks.workunit.client.0.smithi033.stdout: 1 492799 5.73 MB/sec execute 265 sec latency 1628.645 ms 2024-05-15T14:08:50.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:50 smithi081 ceph-mon[36434]: pgmap v263: 97 pgs: 97 active+clean; 143 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 2.2 MiB/s wr, 125 op/s 2024-05-15T14:08:50.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:50 smithi033 ceph-mon[31650]: pgmap v263: 97 pgs: 97 active+clean; 143 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 2.2 MiB/s wr, 125 op/s 2024-05-15T14:08:50.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:50 smithi052 ceph-mon[36190]: pgmap v263: 97 pgs: 97 active+clean; 143 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 2.2 MiB/s wr, 125 op/s 2024-05-15T14:08:51.071 INFO:tasks.workunit.client.0.smithi033.stdout: 1 492799 5.71 MB/sec execute 266 sec latency 2628.738 ms 2024-05-15T14:08:52.071 INFO:tasks.workunit.client.0.smithi033.stdout: 1 492799 5.69 MB/sec execute 267 sec latency 3628.810 ms 2024-05-15T14:08:52.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:52 smithi081 ceph-mon[36434]: pgmap v264: 97 pgs: 97 active+clean; 143 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 2.2 MiB/s wr, 125 op/s 2024-05-15T14:08:52.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:52 smithi033 ceph-mon[31650]: pgmap v264: 97 pgs: 97 active+clean; 143 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 2.2 MiB/s wr, 125 op/s 2024-05-15T14:08:52.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:52 smithi052 ceph-mon[36190]: pgmap v264: 97 pgs: 97 active+clean; 143 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 2.2 MiB/s wr, 125 op/s 2024-05-15T14:08:53.071 INFO:tasks.workunit.client.0.smithi033.stdout: 1 492799 5.67 MB/sec execute 268 sec latency 4628.917 ms 2024-05-15T14:08:54.071 INFO:tasks.workunit.client.0.smithi033.stdout: 1 494249 5.69 MB/sec execute 269 sec latency 4847.778 ms 2024-05-15T14:08:54.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:54 smithi081 ceph-mon[36434]: pgmap v265: 97 pgs: 97 active+clean; 143 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 2.2 MiB/s wr, 134 op/s 2024-05-15T14:08:54.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:54 smithi033 ceph-mon[31650]: pgmap v265: 97 pgs: 97 active+clean; 143 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 2.2 MiB/s wr, 134 op/s 2024-05-15T14:08:54.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:54 smithi052 ceph-mon[36190]: pgmap v265: 97 pgs: 97 active+clean; 143 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 2.2 MiB/s wr, 134 op/s 2024-05-15T14:08:55.071 INFO:tasks.workunit.client.0.smithi033.stdout: 1 496281 5.69 MB/sec execute 270 sec latency 25.637 ms 2024-05-15T14:08:56.071 INFO:tasks.workunit.client.0.smithi033.stdout: 1 497993 5.72 MB/sec execute 271 sec latency 10.196 ms 2024-05-15T14:08:56.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:56 smithi033 ceph-mon[31650]: pgmap v266: 97 pgs: 97 active+clean; 143 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 317 KiB/s wr, 58 op/s 2024-05-15T14:08:56.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:56 smithi052 ceph-mon[36190]: pgmap v266: 97 pgs: 97 active+clean; 143 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 317 KiB/s wr, 58 op/s 2024-05-15T14:08:56.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:56 smithi081 ceph-mon[36434]: pgmap v266: 97 pgs: 97 active+clean; 143 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 317 KiB/s wr, 58 op/s 2024-05-15T14:08:57.071 INFO:tasks.workunit.client.0.smithi033.stdout: 1 499856 5.72 MB/sec execute 272 sec latency 23.699 ms 2024-05-15T14:08:58.071 INFO:tasks.workunit.client.0.smithi033.stdout: 1 501537 5.74 MB/sec execute 273 sec latency 18.103 ms 2024-05-15T14:08:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.093+0000 7f9daf5e5640 1 -- 172.21.15.33:0/1846318866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9da8108a60 msgr2=0x7f9da8108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.093+0000 7f9daf5e5640 1 --2- 172.21.15.33:0/1846318866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9da8108a60 0x7f9da8108e40 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f9da4009930 tx=0x7f9da402a420 comp rx=0 tx=0).stop 2024-05-15T14:08:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.093+0000 7f9daf5e5640 1 -- 172.21.15.33:0/1846318866 shutdown_connections 2024-05-15T14:08:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.093+0000 7f9daf5e5640 1 --2- 172.21.15.33:0/1846318866 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9da8103410 0x7f9da810f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.093+0000 7f9daf5e5640 1 --2- 172.21.15.33:0/1846318866 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9da8102a50 0x7f9da8102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.093+0000 7f9daf5e5640 1 --2- 172.21.15.33:0/1846318866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9da8108a60 0x7f9da8108e40 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.093+0000 7f9daf5e5640 1 -- 172.21.15.33:0/1846318866 >> 172.21.15.33:0/1846318866 conn(0x7f9da80fe540 msgr2=0x7f9da8100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.094+0000 7f9daf5e5640 1 -- 172.21.15.33:0/1846318866 shutdown_connections 2024-05-15T14:08:58.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.094+0000 7f9daf5e5640 1 -- 172.21.15.33:0/1846318866 wait complete. 2024-05-15T14:08:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.094+0000 7f9daf5e5640 1 Processor -- start 2024-05-15T14:08:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.094+0000 7f9daf5e5640 1 -- start start 2024-05-15T14:08:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.094+0000 7f9daf5e5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9da8102a50 0x7f9da8199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.094+0000 7f9daf5e5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9da8103410 0x7f9da81997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.094+0000 7f9daf5e5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9da8108a60 0x7f9da819db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.094+0000 7f9daf5e5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9da8112cf0 con 0x7f9da8108a60 2024-05-15T14:08:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.094+0000 7f9daf5e5640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9da8112b70 con 0x7f9da8102a50 2024-05-15T14:08:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.094+0000 7f9daf5e5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9da8112e70 con 0x7f9da8103410 2024-05-15T14:08:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.095+0000 7f9dadb82640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9da8108a60 0x7f9da819db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.095+0000 7f9dadb82640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9da8108a60 0x7f9da819db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59280/0 (socket says 172.21.15.33:59280) 2024-05-15T14:08:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.095+0000 7f9dadb82640 1 -- 172.21.15.33:0/2495478499 learned_addr learned my addr 172.21.15.33:0/2495478499 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:08:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.095+0000 7f9dad381640 1 --2- 172.21.15.33:0/2495478499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9da8102a50 0x7f9da8199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.095+0000 7f9dacb80640 1 --2- 172.21.15.33:0/2495478499 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9da8103410 0x7f9da81997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.095+0000 7f9dadb82640 1 -- 172.21.15.33:0/2495478499 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9da8103410 msgr2=0x7f9da81997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.095+0000 7f9dadb82640 1 --2- 172.21.15.33:0/2495478499 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9da8103410 0x7f9da81997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.095+0000 7f9dadb82640 1 -- 172.21.15.33:0/2495478499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9da8102a50 msgr2=0x7f9da8199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.095+0000 7f9dadb82640 1 --2- 172.21.15.33:0/2495478499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9da8102a50 0x7f9da8199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.095+0000 7f9dadb82640 1 -- 172.21.15.33:0/2495478499 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da819e280 con 0x7f9da8108a60 2024-05-15T14:08:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.095+0000 7f9dadb82640 1 --2- 172.21.15.33:0/2495478499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9da8108a60 0x7f9da819db50 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f9da000eb10 tx=0x7f9da000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.095+0000 7f9d9e7fc640 1 -- 172.21.15.33:0/2495478499 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da0004b20 con 0x7f9da8108a60 2024-05-15T14:08:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.095+0000 7f9daf5e5640 1 -- 172.21.15.33:0/2495478499 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da819e570 con 0x7f9da8108a60 2024-05-15T14:08:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.095+0000 7f9d9e7fc640 1 -- 172.21.15.33:0/2495478499 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9da0004cc0 con 0x7f9da8108a60 2024-05-15T14:08:58.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.095+0000 7f9d9e7fc640 1 -- 172.21.15.33:0/2495478499 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da0010670 con 0x7f9da8108a60 2024-05-15T14:08:58.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.095+0000 7f9daf5e5640 1 -- 172.21.15.33:0/2495478499 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da81a5e00 con 0x7f9da8108a60 2024-05-15T14:08:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.098+0000 7f9d9e7fc640 1 -- 172.21.15.33:0/2495478499 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9da0005020 con 0x7f9da8108a60 2024-05-15T14:08:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.098+0000 7f9db23d6740 1 Processor -- start 2024-05-15T14:08:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.098+0000 7f9db23d6740 1 -- start start 2024-05-15T14:08:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.098+0000 7f9db23d6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad0cdc8f0 0x560ad0cdccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.098+0000 7f9db23d6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad0cdd210 0x560ad0ce7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.098+0000 7f9db23d6740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad0ce7ad0 0x560ad0ce9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.098+0000 7f9db23d6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560ad0c4fc30 con 0x560ad0cdd210 2024-05-15T14:08:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.098+0000 7f9db23d6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x560ad0c3f7d0 con 0x560ad0cdc8f0 2024-05-15T14:08:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.098+0000 7f9db23d6740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x560ad0c43810 con 0x560ad0ce7ad0 2024-05-15T14:08:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.099+0000 7f9dad381640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad0cdd210 0x560ad0ce7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.099+0000 7f9dacb80640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad0cdc8f0 0x560ad0cdccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.099+0000 7f9dad381640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad0cdd210 0x560ad0ce7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59292/0 (socket says 172.21.15.33:59292) 2024-05-15T14:08:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.099+0000 7f9dad381640 1 -- 172.21.15.33:0/216685089 learned_addr learned my addr 172.21.15.33:0/216685089 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:08:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.099+0000 7f9dacb80640 1 --2- 172.21.15.33:0/216685089 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad0ce7ad0 0x560ad0ce9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.099+0000 7f9d9e7fc640 1 --2- 172.21.15.33:0/2495478499 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f9d940b2950 0x7f9d940b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.099+0000 7f9dad381640 1 -- 172.21.15.33:0/216685089 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad0ce7ad0 msgr2=0x560ad0ce9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.099+0000 7f9dad381640 1 --2- 172.21.15.33:0/216685089 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad0ce7ad0 0x560ad0ce9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.099+0000 7f9dad381640 1 -- 172.21.15.33:0/216685089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad0cdc8f0 msgr2=0x560ad0cdccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.099+0000 7f9dad381640 1 --2- 172.21.15.33:0/216685089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad0cdc8f0 0x560ad0cdccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.099+0000 7f9dad381640 1 -- 172.21.15.33:0/216685089 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ad0954210 con 0x560ad0cdd210 2024-05-15T14:08:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.099+0000 7f9dacb80640 1 --2- 172.21.15.33:0/216685089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad0cdc8f0 0x560ad0cdccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:08:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.099+0000 7f9dadb82640 1 --2- 172.21.15.33:0/2495478499 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f9d940b2950 0x7f9d940b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.099+0000 7f9d9e7fc640 1 -- 172.21.15.33:0/2495478499 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f9da00e2b30 con 0x7f9da8108a60 2024-05-15T14:08:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.099+0000 7f9dad381640 1 --2- 172.21.15.33:0/216685089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad0cdd210 0x560ad0ce7590 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f9da402a940 tx=0x7f9da4005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=724c77452b24866e server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.099+0000 7f9dadb82640 1 --2- 172.21.15.33:0/2495478499 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f9d940b2950 0x7f9d940b4e10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9da00049d0 tx=0x7f9da001d000 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.099+0000 7f9d937fe640 1 -- 172.21.15.33:0/216685089 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da4040070 con 0x560ad0cdd210 2024-05-15T14:08:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.100+0000 7f9d937fe640 1 -- 172.21.15.33:0/216685089 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9da4004030 con 0x560ad0cdd210 2024-05-15T14:08:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.100+0000 7f9d937fe640 1 -- 172.21.15.33:0/216685089 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da4035040 con 0x560ad0cdd210 2024-05-15T14:08:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.100+0000 7f9db23d6740 1 -- 172.21.15.33:0/216685089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad0cdd210 msgr2=0x560ad0ce7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.100+0000 7f9db23d6740 1 --2- 172.21.15.33:0/216685089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad0cdd210 0x560ad0ce7590 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f9da402a940 tx=0x7f9da4005be0 comp rx=0 tx=0).stop 2024-05-15T14:08:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.100+0000 7f9db23d6740 1 -- 172.21.15.33:0/216685089 shutdown_connections 2024-05-15T14:08:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.100+0000 7f9db23d6740 1 --2- 172.21.15.33:0/216685089 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad0ce7ad0 0x560ad0ce9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.100+0000 7f9db23d6740 1 --2- 172.21.15.33:0/216685089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad0cdd210 0x560ad0ce7590 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.100+0000 7f9db23d6740 1 --2- 172.21.15.33:0/216685089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad0cdc8f0 0x560ad0cdccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.100+0000 7f9db23d6740 1 -- 172.21.15.33:0/216685089 >> 172.21.15.33:0/216685089 conn(0x560ad0cd8e00 msgr2=0x560ad0cd91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.100+0000 7f9db23d6740 1 -- 172.21.15.33:0/216685089 shutdown_connections 2024-05-15T14:08:58.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.100+0000 7f9db23d6740 1 -- 172.21.15.33:0/216685089 wait complete. 2024-05-15T14:08:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.101+0000 7f9db23d6740 1 -- start start 2024-05-15T14:08:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.101+0000 7f9db23d6740 1 Processor -- start 2024-05-15T14:08:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.101+0000 7f9db23d6740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad0cdc8f0 0x560ad0ce7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.101+0000 7f9db23d6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad0cdd210 0x560ad0ce4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.101+0000 7f9db23d6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad0ce7ad0 0x560ad0ce4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.101+0000 7f9db23d6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560ad0925790 con 0x560ad0cdd210 2024-05-15T14:08:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.101+0000 7f9db23d6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x560ad0bfb890 con 0x560ad0ce7ad0 2024-05-15T14:08:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.101+0000 7f9db23d6740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x560ad0cea740 con 0x560ad0cdc8f0 2024-05-15T14:08:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.101+0000 7f9db23d6740 10 client.0 _ll_register_callbacks cb 0x560ad0bfed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:08:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.101+0000 7f9dad381640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad0cdd210 0x560ad0ce4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.101+0000 7f9dad381640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad0cdd210 0x560ad0ce4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59306/0 (socket says 172.21.15.33:59306) 2024-05-15T14:08:58.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.101+0000 7f9dad381640 1 -- 172.21.15.33:0/4198486275 learned_addr learned my addr 172.21.15.33:0/4198486275 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:08:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.101+0000 7f9dacb80640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad0cdc8f0 0x560ad0ce7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.102+0000 7f9dacb80640 1 --2- 172.21.15.33:0/4198486275 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad0ce7ad0 0x560ad0ce4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.102+0000 7f9dad381640 1 -- 172.21.15.33:0/4198486275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad0cdc8f0 msgr2=0x560ad0ce7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.102+0000 7f9dad381640 1 --2- 172.21.15.33:0/4198486275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad0cdc8f0 0x560ad0ce7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.102+0000 7f9dad381640 1 -- 172.21.15.33:0/4198486275 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad0ce7ad0 msgr2=0x560ad0ce4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.102+0000 7f9dad381640 1 --2- 172.21.15.33:0/4198486275 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad0ce7ad0 0x560ad0ce4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.102+0000 7f9dad381640 1 -- 172.21.15.33:0/4198486275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ad0a20620 con 0x560ad0cdd210 2024-05-15T14:08:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.102+0000 7f9dacb80640 1 --2- 172.21.15.33:0/4198486275 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad0ce7ad0 0x560ad0ce4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:08:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.102+0000 7f9dad381640 1 --2- 172.21.15.33:0/4198486275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad0cdd210 0x560ad0ce4710 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f9da4000c00 tx=0x7f9da4004490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2003ebe542d90355 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.102+0000 7f9d90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:08:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.102+0000 7f9d90ff9640 1 -- 172.21.15.33:0/4198486275 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da404c020 con 0x560ad0cdd210 2024-05-15T14:08:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.102+0000 7f9d90ff9640 1 -- 172.21.15.33:0/4198486275 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f9da4005100 con 0x560ad0cdd210 2024-05-15T14:08:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.102+0000 7f9db23d6740 1 -- 172.21.15.33:0/4198486275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9da4004030 con 0x560ad0cdd210 2024-05-15T14:08:58.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.102+0000 7f9d90ff9640 1 -- 172.21.15.33:0/4198486275 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9da4057050 con 0x560ad0cdd210 2024-05-15T14:08:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.102+0000 7f9db23d6740 10 client.16377 fetch_fsmap learned FSMap version 34 2024-05-15T14:08:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.102+0000 7f9db23d6740 1 -- 172.21.15.33:0/4198486275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560ad0cf6200 con 0x560ad0cdd210 2024-05-15T14:08:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.102+0000 7f9d90ff9640 1 -- 172.21.15.33:0/4198486275 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9da4004030 con 0x560ad0cdd210 2024-05-15T14:08:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.103+0000 7f9d90ff9640 1 -- 172.21.15.33:0/4198486275 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f9da40035f0 con 0x560ad0cdd210 2024-05-15T14:08:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.103+0000 7f9db23d6740 10 client.16377 fetch_fsmap finished waiting for FSMap version 34 2024-05-15T14:08:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.103+0000 7f9db23d6740 10 client.16377 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:08:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.103+0000 7f9db23d6740 20 client.16377 populate_metadata read hostname 'smithi033' 2024-05-15T14:08:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.103+0000 7f9db23d6740 1 --2- 172.21.15.33:0/4198486275 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x560ad0cf7550 0x560ad0cf9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.103+0000 7f9db23d6740 4 client.16377 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:08:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.103+0000 7f9db23d6740 1 -- 172.21.15.33:0/4198486275 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ad0c51b80 con 0x560ad0cf7550 2024-05-15T14:08:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.103+0000 7f9dacb80640 1 --2- 172.21.15.33:0/4198486275 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x560ad0cf7550 0x560ad0cf9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.103+0000 7f9d90ff9640 10 client.16377 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:08:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.103+0000 7f9dacb80640 1 --2- 172.21.15.33:0/4198486275 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x560ad0cf7550 0x560ad0cf9940 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcb68daa3746a5ac server_cookie=dc33d7f52e827fd4 in_seq=0 out_seq=0 2024-05-15T14:08:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.104+0000 7f9d90ff9640 1 -- 172.21.15.33:0/4198486275 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560ad0c51b80 con 0x560ad0cf7550 2024-05-15T14:08:58.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.104+0000 7f9d90ff9640 10 client.16377 handle_command_reply: tid=0 2024-05-15T14:08:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.104+0000 7f9db23d6740 1 client.16377 shutdown 2024-05-15T14:08:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.104+0000 7f9db23d6740 1 -- 172.21.15.33:0/4198486275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad0cdd210 msgr2=0x560ad0ce4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.104+0000 7f9db23d6740 1 --2- 172.21.15.33:0/4198486275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad0cdd210 0x560ad0ce4710 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f9da4000c00 tx=0x7f9da4004490 comp rx=0 tx=0).stop 2024-05-15T14:08:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.105+0000 7f9db23d6740 1 -- 172.21.15.33:0/4198486275 shutdown_connections 2024-05-15T14:08:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.105+0000 7f9db23d6740 1 --2- 172.21.15.33:0/4198486275 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x560ad0cf7550 0x560ad0cf9940 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.105+0000 7f9db23d6740 1 --2- 172.21.15.33:0/4198486275 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad0ce7ad0 0x560ad0ce4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.105+0000 7f9db23d6740 1 --2- 172.21.15.33:0/4198486275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad0cdd210 0x560ad0ce4710 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.105+0000 7f9db23d6740 1 --2- 172.21.15.33:0/4198486275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad0cdc8f0 0x560ad0ce7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.105+0000 7f9db23d6740 1 -- 172.21.15.33:0/4198486275 >> 172.21.15.33:0/4198486275 conn(0x560ad0cd8e00 msgr2=0x560ad0cd8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.105+0000 7f9d90ff9640 0 client.16377 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:08:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.105+0000 7f9db23d6740 1 -- 172.21.15.33:0/4198486275 shutdown_connections 2024-05-15T14:08:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.105+0000 7f9db23d6740 1 -- 172.21.15.33:0/4198486275 wait complete. 2024-05-15T14:08:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.105+0000 7f9db23d6740 20 client.16377 trim_cache size 0 max 16384 2024-05-15T14:08:58.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.121+0000 7f9db23d6740 1 Processor -- start 2024-05-15T14:08:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.121+0000 7f9db23d6740 1 -- start start 2024-05-15T14:08:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.121+0000 7f9db23d6740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad0cdc8f0 0x560ad0cf3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.121+0000 7f9db23d6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad0cdd210 0x560ad0ce6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.121+0000 7f9db23d6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad0ce7ad0 0x560ad0cfefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.121+0000 7f9db23d6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9da81b10c0 con 0x560ad0ce7ad0 2024-05-15T14:08:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.121+0000 7f9db23d6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9da81ae900 con 0x560ad0cdd210 2024-05-15T14:08:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.121+0000 7f9db23d6740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9da81ac770 con 0x560ad0cdc8f0 2024-05-15T14:08:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.121+0000 7f9dacb80640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad0ce7ad0 0x560ad0cfefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.121+0000 7f9dad381640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad0cdd210 0x560ad0ce6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.121+0000 7f9dacb80640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad0ce7ad0 0x560ad0cfefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59312/0 (socket says 172.21.15.33:59312) 2024-05-15T14:08:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.121+0000 7f9dacb80640 1 -- 172.21.15.33:0/994531294 learned_addr learned my addr 172.21.15.33:0/994531294 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:08:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.121+0000 7f9dacb80640 1 --2- 172.21.15.33:0/994531294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad0cdc8f0 0x560ad0cf3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.122+0000 7f9dacb80640 1 -- 172.21.15.33:0/994531294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad0cdc8f0 msgr2=0x560ad0cf3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:58.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.122+0000 7f9dacb80640 1 --2- 172.21.15.33:0/994531294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad0cdc8f0 0x560ad0cf3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.122+0000 7f9dacb80640 1 -- 172.21.15.33:0/994531294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad0cdd210 msgr2=0x560ad0ce6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.122+0000 7f9dacb80640 1 --2- 172.21.15.33:0/994531294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad0cdd210 0x560ad0ce6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.122+0000 7f9dacb80640 1 -- 172.21.15.33:0/994531294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ad0cea740 con 0x560ad0ce7ad0 2024-05-15T14:08:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.122+0000 7f9dad381640 1 --2- 172.21.15.33:0/994531294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad0cdd210 0x560ad0ce6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:08:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.122+0000 7f9dacb80640 1 --2- 172.21.15.33:0/994531294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad0ce7ad0 0x560ad0cfefd0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f9d98009fd0 tx=0x7f9d98005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b6e4e872017eaf server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.122+0000 7f9d92ffd640 1 -- 172.21.15.33:0/994531294 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9801a040 con 0x560ad0ce7ad0 2024-05-15T14:08:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.122+0000 7f9d92ffd640 1 -- 172.21.15.33:0/994531294 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d9802a070 con 0x560ad0ce7ad0 2024-05-15T14:08:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.122+0000 7f9d92ffd640 1 -- 172.21.15.33:0/994531294 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98005550 con 0x560ad0ce7ad0 2024-05-15T14:08:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.123+0000 7f9db23d6740 1 -- 172.21.15.33:0/994531294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad0ce7ad0 msgr2=0x560ad0cfefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.123+0000 7f9db23d6740 1 --2- 172.21.15.33:0/994531294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad0ce7ad0 0x560ad0cfefd0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f9d98009fd0 tx=0x7f9d98005f90 comp rx=0 tx=0).stop 2024-05-15T14:08:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.123+0000 7f9db23d6740 1 -- 172.21.15.33:0/994531294 shutdown_connections 2024-05-15T14:08:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.123+0000 7f9db23d6740 1 --2- 172.21.15.33:0/994531294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad0ce7ad0 0x560ad0cfefd0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:58.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.123+0000 7f9db23d6740 1 --2- 172.21.15.33:0/994531294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad0cdd210 0x560ad0ce6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:58.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.123+0000 7f9db23d6740 1 --2- 172.21.15.33:0/994531294 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad0cdc8f0 0x560ad0cf3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:58.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.123+0000 7f9db23d6740 1 -- 172.21.15.33:0/994531294 >> 172.21.15.33:0/994531294 conn(0x560ad0cd8e00 msgr2=0x560ad0cdfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:58.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.123+0000 7f9db23d6740 1 -- 172.21.15.33:0/994531294 shutdown_connections 2024-05-15T14:08:58.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.123+0000 7f9db23d6740 1 -- 172.21.15.33:0/994531294 wait complete. 2024-05-15T14:08:58.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.123+0000 7f9db23d6740 1 -- start start 2024-05-15T14:08:58.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.123+0000 7f9db23d6740 1 Processor -- start 2024-05-15T14:08:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.124+0000 7f9db23d6740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad0cdc8f0 0x560ad0cea9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.124+0000 7f9db23d6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad0cdd210 0x560ad0ceaef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.124+0000 7f9db23d6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad0ce7ad0 0x560ad0d5fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.124+0000 7f9db23d6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9da81acdc0 con 0x560ad0ce7ad0 2024-05-15T14:08:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.124+0000 7f9db23d6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9da81ae230 con 0x560ad0cdd210 2024-05-15T14:08:58.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.124+0000 7f9db23d6740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f9da81ad490 con 0x560ad0cdc8f0 2024-05-15T14:08:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.124+0000 7f9db23d6740 10 client.0 _ll_register_callbacks cb 0x7f9da4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:08:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.124+0000 7f9dacb80640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad0cdc8f0 0x560ad0cea9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.124+0000 7f9dad381640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad0cdd210 0x560ad0ceaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.124+0000 7f9dacb80640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad0ce7ad0 0x560ad0d5fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.124+0000 7f9dad381640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad0cdd210 0x560ad0ceaef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:53686/0 (socket says 172.21.15.33:53686) 2024-05-15T14:08:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.124+0000 7f9dad381640 1 -- 172.21.15.33:0/172460116 learned_addr learned my addr 172.21.15.33:0/172460116 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:08:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.124+0000 7f9dad381640 1 -- 172.21.15.33:0/172460116 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad0cdc8f0 msgr2=0x560ad0cea9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.124+0000 7f9dad381640 1 --2- 172.21.15.33:0/172460116 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad0cdc8f0 0x560ad0cea9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.124+0000 7f9dad381640 1 -- 172.21.15.33:0/172460116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad0ce7ad0 msgr2=0x560ad0d5fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:08:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.124+0000 7f9dad381640 1 --2- 172.21.15.33:0/172460116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad0ce7ad0 0x560ad0d5fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.124+0000 7f9dad381640 1 -- 172.21.15.33:0/172460116 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ad0d600b0 con 0x560ad0cdd210 2024-05-15T14:08:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.124+0000 7f9dad381640 1 --2- 172.21.15.33:0/172460116 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad0cdd210 0x560ad0ceaef0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9da4005c40 tx=0x7f9da4005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e676cf3b6a6bb5f server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:08:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.124+0000 7f9d90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-05-15T14:08:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.124+0000 7f9d90ff9640 1 -- 172.21.15.33:0/172460116 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da404c040 con 0x560ad0cdd210 2024-05-15T14:08:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.125+0000 7f9d90ff9640 1 -- 172.21.15.33:0/172460116 <== mon.2 v2:172.21.15.52:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f9da40316b0 con 0x560ad0cdd210 2024-05-15T14:08:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.125+0000 7f9db23d6740 1 -- 172.21.15.33:0/172460116 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d9802a070 con 0x560ad0cdd210 2024-05-15T14:08:58.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.125+0000 7f9d90ff9640 1 -- 172.21.15.33:0/172460116 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9da4040070 con 0x560ad0cdd210 2024-05-15T14:08:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.125+0000 7f9db23d6740 10 client.25874 fetch_fsmap learned FSMap version 34 2024-05-15T14:08:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.125+0000 7f9db23d6740 1 -- 172.21.15.33:0/172460116 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560ad0d612f0 con 0x560ad0cdd210 2024-05-15T14:08:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.125+0000 7f9d90ff9640 1 -- 172.21.15.33:0/172460116 <== mon.2 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d9802a070 con 0x560ad0cdd210 2024-05-15T14:08:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.125+0000 7f9d90ff9640 1 -- 172.21.15.33:0/172460116 <== mon.2 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f9da4004770 con 0x560ad0cdd210 2024-05-15T14:08:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.125+0000 7f9db23d6740 10 client.25874 fetch_fsmap finished waiting for FSMap version 34 2024-05-15T14:08:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.125+0000 7f9db23d6740 10 client.25874 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:08:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.125+0000 7f9db23d6740 20 client.25874 populate_metadata read hostname 'smithi033' 2024-05-15T14:08:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.125+0000 7f9db23d6740 1 --2- 172.21.15.33:0/172460116 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x560ad0d619d0 0x560ad0d63df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:08:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.125+0000 7f9db23d6740 4 client.25874 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:08:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.125+0000 7f9db23d6740 1 -- 172.21.15.33:0/172460116 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560ad0c4b360 con 0x560ad0d619d0 2024-05-15T14:08:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.125+0000 7f9dacb80640 1 --2- 172.21.15.33:0/172460116 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x560ad0d619d0 0x560ad0d63df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:08:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.126+0000 7f9d90ff9640 10 client.25874 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:08:58.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.126+0000 7f9dacb80640 1 --2- 172.21.15.33:0/172460116 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x560ad0d619d0 0x560ad0d63df0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=682b9fbc6f5b5516 server_cookie=b782bec7f36e473d in_seq=0 out_seq=0 2024-05-15T14:08:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.165+0000 7f9d90ff9640 1 -- 172.21.15.33:0/172460116 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560ad0c4b360 con 0x560ad0d619d0 2024-05-15T14:08:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.165+0000 7f9d90ff9640 10 client.25874 handle_command_reply: tid=0 2024-05-15T14:08:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.165+0000 7f9db23d6740 1 client.25874 shutdown 2024-05-15T14:08:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.165+0000 7f9db23d6740 1 -- 172.21.15.33:0/172460116 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad0cdd210 msgr2=0x560ad0ceaef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.165+0000 7f9db23d6740 1 --2- 172.21.15.33:0/172460116 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad0cdd210 0x560ad0ceaef0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9da4005c40 tx=0x7f9da4005850 comp rx=0 tx=0).stop 2024-05-15T14:08:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.165+0000 7f9db23d6740 1 -- 172.21.15.33:0/172460116 shutdown_connections 2024-05-15T14:08:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.165+0000 7f9db23d6740 1 --2- 172.21.15.33:0/172460116 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x560ad0d619d0 0x560ad0d63df0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.165+0000 7f9db23d6740 1 --2- 172.21.15.33:0/172460116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad0ce7ad0 0x560ad0d5fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.165+0000 7f9db23d6740 1 --2- 172.21.15.33:0/172460116 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad0cdd210 0x560ad0ceaef0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.165+0000 7f9d90ff9640 0 client.25874 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:08:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.165+0000 7f9db23d6740 1 --2- 172.21.15.33:0/172460116 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad0cdc8f0 0x560ad0cea9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.165+0000 7f9db23d6740 1 -- 172.21.15.33:0/172460116 >> 172.21.15.33:0/172460116 conn(0x560ad0cd8e00 msgr2=0x560ad0cdf3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.165+0000 7f9db23d6740 1 -- 172.21.15.33:0/172460116 shutdown_connections 2024-05-15T14:08:58.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.165+0000 7f9db23d6740 1 -- 172.21.15.33:0/172460116 wait complete. 2024-05-15T14:08:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.165+0000 7f9db23d6740 20 client.25874 trim_cache size 0 max 16384 2024-05-15T14:08:58.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:08:58.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:08:58.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:08:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.166+0000 7f9daf5e5640 1 -- 172.21.15.33:0/2495478499 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f9d940b2950 msgr2=0x7f9d940b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.166+0000 7f9daf5e5640 1 --2- 172.21.15.33:0/2495478499 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f9d940b2950 0x7f9d940b4e10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9da00049d0 tx=0x7f9da001d000 comp rx=0 tx=0).stop 2024-05-15T14:08:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.166+0000 7f9daf5e5640 1 -- 172.21.15.33:0/2495478499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9da8108a60 msgr2=0x7f9da819db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:08:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.166+0000 7f9daf5e5640 1 --2- 172.21.15.33:0/2495478499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9da8108a60 0x7f9da819db50 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f9da000eb10 tx=0x7f9da000c2d0 comp rx=0 tx=0).stop 2024-05-15T14:08:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.166+0000 7f9daf5e5640 1 -- 172.21.15.33:0/2495478499 shutdown_connections 2024-05-15T14:08:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.166+0000 7f9daf5e5640 1 --2- 172.21.15.33:0/2495478499 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f9d940b2950 0x7f9d940b4e10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.166+0000 7f9daf5e5640 1 --2- 172.21.15.33:0/2495478499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9da8108a60 0x7f9da819db50 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.166+0000 7f9daf5e5640 1 --2- 172.21.15.33:0/2495478499 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9da8103410 0x7f9da81997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.166+0000 7f9daf5e5640 1 --2- 172.21.15.33:0/2495478499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9da8102a50 0x7f9da8199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:08:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.166+0000 7f9daf5e5640 1 -- 172.21.15.33:0/2495478499 >> 172.21.15.33:0/2495478499 conn(0x7f9da80fe540 msgr2=0x7f9da80fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:08:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.166+0000 7f9daf5e5640 1 -- 172.21.15.33:0/2495478499 shutdown_connections 2024-05-15T14:08:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:08:58.166+0000 7f9daf5e5640 1 -- 172.21.15.33:0/2495478499 wait complete. 2024-05-15T14:08:58.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:08:58 smithi033 ceph-mon[31650]: pgmap v267: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 954 KiB/s wr, 137 op/s 2024-05-15T14:08:58.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:08:58 smithi052 ceph-mon[36190]: pgmap v267: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 954 KiB/s wr, 137 op/s 2024-05-15T14:08:58.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:08:58 smithi081 ceph-mon[36434]: pgmap v267: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 954 KiB/s wr, 137 op/s 2024-05-15T14:08:59.071 INFO:tasks.workunit.client.0.smithi033.stdout: 1 503383 5.75 MB/sec execute 274 sec latency 68.347 ms 2024-05-15T14:09:00.071 INFO:tasks.workunit.client.0.smithi033.stdout: 1 505049 5.77 MB/sec execute 275 sec latency 12.825 ms 2024-05-15T14:09:00.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:00 smithi033 ceph-mon[31650]: pgmap v268: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 688 KiB/s wr, 88 op/s 2024-05-15T14:09:00.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:00 smithi052 ceph-mon[36190]: pgmap v268: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 688 KiB/s wr, 88 op/s 2024-05-15T14:09:00.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:00 smithi081 ceph-mon[36434]: pgmap v268: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 688 KiB/s wr, 88 op/s 2024-05-15T14:09:01.071 INFO:tasks.workunit.client.0.smithi033.stdout: 1 506177 5.77 MB/sec execute 276 sec latency 22.940 ms 2024-05-15T14:09:01.582 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.0 2024-05-15T14:09:01.582 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T14:09:01.858 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:09:02.071 INFO:tasks.workunit.client.0.smithi033.stdout: 1 508303 5.80 MB/sec execute 277 sec latency 14.863 ms 2024-05-15T14:09:02.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:02 smithi033 ceph-mon[31650]: pgmap v269: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 688 KiB/s wr, 88 op/s 2024-05-15T14:09:02.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:02 smithi052 ceph-mon[36190]: pgmap v269: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 688 KiB/s wr, 88 op/s 2024-05-15T14:09:02.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:02 smithi081 ceph-mon[36434]: pgmap v269: 97 pgs: 97 active+clean; 149 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 688 KiB/s wr, 88 op/s 2024-05-15T14:09:03.071 INFO:tasks.workunit.client.0.smithi033.stdout: 1 509566 5.80 MB/sec execute 278 sec latency 24.676 ms 2024-05-15T14:09:03.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.800+0000 7f20b80f2700 1 -- 172.21.15.33:0/2575647209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20b0108510 msgr2=0x7f20b01088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:03.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.800+0000 7f20b80f2700 1 --2- 172.21.15.33:0/2575647209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20b0108510 0x7f20b01088f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f20ac009990 tx=0x7f20ac009ca0 comp rx=0 tx=0).stop 2024-05-15T14:09:03.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.800+0000 7f20b80f2700 1 -- 172.21.15.33:0/2575647209 shutdown_connections 2024-05-15T14:09:03.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.800+0000 7f20b80f2700 1 --2- 172.21.15.33:0/2575647209 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20b0102df0 0x7f20b010f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:03.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.800+0000 7f20b80f2700 1 --2- 172.21.15.33:0/2575647209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20b0108510 0x7f20b01088f0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:03.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.800+0000 7f20b80f2700 1 --2- 172.21.15.33:0/2575647209 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f20b0102430 0x7f20b01028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:03.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.800+0000 7f20b80f2700 1 -- 172.21.15.33:0/2575647209 >> 172.21.15.33:0/2575647209 conn(0x7f20b00793b0 msgr2=0x7f20b00797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:03.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.800+0000 7f20b80f2700 1 -- 172.21.15.33:0/2575647209 shutdown_connections 2024-05-15T14:09:03.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.800+0000 7f20b80f2700 1 -- 172.21.15.33:0/2575647209 wait complete. 2024-05-15T14:09:03.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.800+0000 7f20b80f2700 1 Processor -- start 2024-05-15T14:09:03.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.801+0000 7f20b80f2700 1 -- start start 2024-05-15T14:09:03.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.801+0000 7f20b80f2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f20b0102430 0x7f20b0104070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:03.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.801+0000 7f20b80f2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20b0102df0 0x7f20b01045b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:03.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.801+0000 7f20b80f2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20b0108510 0x7f20b0104af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:03.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.801+0000 7f20b80f2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f20b01125c0 con 0x7f20b0108510 2024-05-15T14:09:03.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.801+0000 7f20b80f2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f20b0112440 con 0x7f20b0102430 2024-05-15T14:09:03.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.801+0000 7f20b80f2700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f20b0112740 con 0x7f20b0102df0 2024-05-15T14:09:03.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.801+0000 7f20b668f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20b0108510 0x7f20b0104af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:03.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.801+0000 7f20b568d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20b0102df0 0x7f20b01045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:03.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.801+0000 7f20b5e8e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f20b0102430 0x7f20b0104070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:03.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.801+0000 7f20b568d700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20b0102df0 0x7f20b01045b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:56360/0 (socket says 172.21.15.33:56360) 2024-05-15T14:09:03.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.801+0000 7f20b668f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20b0108510 0x7f20b0104af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37430/0 (socket says 172.21.15.33:37430) 2024-05-15T14:09:03.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.801+0000 7f20b668f700 1 -- 172.21.15.33:0/1944599151 learned_addr learned my addr 172.21.15.33:0/1944599151 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:09:03.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.801+0000 7f20b5e8e700 1 -- 172.21.15.33:0/1944599151 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20b0102df0 msgr2=0x7f20b01045b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:03.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.801+0000 7f20b5e8e700 1 --2- 172.21.15.33:0/1944599151 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20b0102df0 0x7f20b01045b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:03.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.801+0000 7f20b5e8e700 1 -- 172.21.15.33:0/1944599151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20b0108510 msgr2=0x7f20b0104af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:03.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.801+0000 7f20b5e8e700 1 --2- 172.21.15.33:0/1944599151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20b0108510 0x7f20b0104af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:03.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.801+0000 7f20b5e8e700 1 -- 172.21.15.33:0/1944599151 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b01a80f0 con 0x7f20b0102430 2024-05-15T14:09:03.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.801+0000 7f20b668f700 1 --2- 172.21.15.33:0/1944599151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20b0108510 0x7f20b0104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:09:03.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.802+0000 7f20b5e8e700 1 --2- 172.21.15.33:0/1944599151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f20b0102430 0x7f20b0104070 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f20ac006010 tx=0x7f20ac0057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:03.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.802+0000 7f20a6ffd700 1 -- 172.21.15.33:0/1944599151 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac027070 con 0x7f20b0102430 2024-05-15T14:09:03.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.802+0000 7f20a6ffd700 1 -- 172.21.15.33:0/1944599151 <== mon.2 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20ac022470 con 0x7f20b0102430 2024-05-15T14:09:03.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.802+0000 7f20b80f2700 1 -- 172.21.15.33:0/1944599151 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b01a8380 con 0x7f20b0102430 2024-05-15T14:09:03.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.802+0000 7f20a6ffd700 1 -- 172.21.15.33:0/1944599151 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac017650 con 0x7f20b0102430 2024-05-15T14:09:03.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.802+0000 7f20b80f2700 1 -- 172.21.15.33:0/1944599151 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b01a8830 con 0x7f20b0102430 2024-05-15T14:09:03.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.803+0000 7f20b80f2700 1 -- 172.21.15.33:0/1944599151 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20b006a230 con 0x7f20b0102430 2024-05-15T14:09:03.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.805+0000 7f20a6ffd700 1 -- 172.21.15.33:0/1944599151 <== mon.2 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f20ac02a070 con 0x7f20b0102430 2024-05-15T14:09:03.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.805+0000 7f20a6ffd700 1 --2- 172.21.15.33:0/1944599151 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f209c0b2a70 0x7f209c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:03.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.806+0000 7f20b568d700 1 --2- 172.21.15.33:0/1944599151 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f209c0b2a70 0x7f209c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:03.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.806+0000 7f20a6ffd700 1 -- 172.21.15.33:0/1944599151 <== mon.2 v2:172.21.15.52:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8542+0+0 (secure 0 0 0) 0x7f20ac0e3270 con 0x7f20b0102430 2024-05-15T14:09:03.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.806+0000 7f20b568d700 1 --2- 172.21.15.33:0/1944599151 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f209c0b2a70 0x7f209c0b4f30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f20b0079920 tx=0x7f20a00073d0 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:03.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:03.809+0000 7f20a6ffd700 1 -- 172.21.15.33:0/1944599151 <== mon.2 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f20ac013070 con 0x7f20b0102430 2024-05-15T14:09:04.071 INFO:tasks.workunit.client.0.smithi033.stdout: 1 511790 5.82 MB/sec execute 279 sec latency 17.363 ms 2024-05-15T14:09:04.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:04.112+0000 7f20b80f2700 1 -- 172.21.15.33:0/1944599151 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20b01a8a90 con 0x7f20b0102430 2024-05-15T14:09:04.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:04.113+0000 7f20a6ffd700 1 -- 172.21.15.33:0/1944599151 <== mon.2 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11862 (secure 0 0 0) 0x7f20ac0aacd0 con 0x7f20b0102430 2024-05-15T14:09:04.116 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T14:09:04.116 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2030117633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2030117633},{"type":"v1","addr":"172.21.15.33:6839","nonce":2030117633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1918979061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1918979061},{"type":"v1","addr":"172.21.15.33:6841","nonce":1918979061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T13:57:19.056417+0000","modified":"2024-05-15T14:08:23.686464+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24463,"mds_2":24461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/1629697841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1629697841},{"type":"v1","addr":"172.21.15.52:6837","nonce":1629697841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.52:6839/2696044471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2696044471},{"type":"v1","addr":"172.21.15.52:6839","nonce":2696044471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.81:6839/1554395940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":1554395940},{"type":"v1","addr":"172.21.15.81:6839","nonce":1554395940}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25915":{"gid":25915,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/1297849376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1297849376},{"type":"v1","addr":"172.21.15.81:6835","nonce":1297849376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24461,24463,24478]},"id":1}]} 2024-05-15T14:09:04.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:04.115+0000 7f20b80f2700 1 -- 172.21.15.33:0/1944599151 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f209c0b2a70 msgr2=0x7f209c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:04.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:04.115+0000 7f20b80f2700 1 --2- 172.21.15.33:0/1944599151 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f209c0b2a70 0x7f209c0b4f30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f20b0079920 tx=0x7f20a00073d0 comp rx=0 tx=0).stop 2024-05-15T14:09:04.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:04.115+0000 7f20b80f2700 1 -- 172.21.15.33:0/1944599151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f20b0102430 msgr2=0x7f20b0104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:04.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:04.115+0000 7f20b80f2700 1 --2- 172.21.15.33:0/1944599151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f20b0102430 0x7f20b0104070 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f20ac006010 tx=0x7f20ac0057c0 comp rx=0 tx=0).stop 2024-05-15T14:09:04.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:04.115+0000 7f20b80f2700 1 -- 172.21.15.33:0/1944599151 shutdown_connections 2024-05-15T14:09:04.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:04.115+0000 7f20b80f2700 1 --2- 172.21.15.33:0/1944599151 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f209c0b2a70 0x7f209c0b4f30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:04.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:04.115+0000 7f20b80f2700 1 --2- 172.21.15.33:0/1944599151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20b0108510 0x7f20b0104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:04.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:04.115+0000 7f20b80f2700 1 --2- 172.21.15.33:0/1944599151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f20b0102430 0x7f20b0104070 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:04.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:04.116+0000 7f20b80f2700 1 --2- 172.21.15.33:0/1944599151 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20b0102df0 0x7f20b01045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:04.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:04.116+0000 7f20b80f2700 1 -- 172.21.15.33:0/1944599151 >> 172.21.15.33:0/1944599151 conn(0x7f20b00793b0 msgr2=0x7f20b006bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:04.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:04.116+0000 7f20b80f2700 1 -- 172.21.15.33:0/1944599151 shutdown_connections 2024-05-15T14:09:04.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:04.116+0000 7f20b80f2700 1 -- 172.21.15.33:0/1944599151 wait complete. 2024-05-15T14:09:04.119 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 34 2024-05-15T14:09:04.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:04 smithi033 ceph-mon[31650]: pgmap v270: 97 pgs: 97 active+clean; 153 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 229 op/s 2024-05-15T14:09:04.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:04 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/1944599151' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:09:04.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:04 smithi052 ceph-mon[36190]: pgmap v270: 97 pgs: 97 active+clean; 153 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 229 op/s 2024-05-15T14:09:04.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:04 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/1944599151' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:09:04.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:04 smithi081 ceph-mon[36434]: pgmap v270: 97 pgs: 97 active+clean; 153 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 229 op/s 2024-05-15T14:09:04.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:04 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/1944599151' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:09:04.923 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24478 quiesce db / --timeout 90 --expiration 138.0 --await-for 45.0' 2024-05-15T14:09:04.923 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24478 quiesce db / --timeout 90 --expiration 138.0 --await-for 45.0 2024-05-15T14:09:05.071 INFO:tasks.workunit.client.0.smithi033.stdout: 1 512361 5.81 MB/sec execute 280 sec latency 37.433 ms 2024-05-15T14:09:05.232 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:09:05.613 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:05 smithi033 ceph-mon[31650]: osdmap e90: 12 total, 12 up, 12 in 2024-05-15T14:09:05.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:05 smithi052 ceph-mon[36190]: osdmap e90: 12 total, 12 up, 12 in 2024-05-15T14:09:05.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:05 smithi081 ceph-mon[36434]: osdmap e90: 12 total, 12 up, 12 in 2024-05-15T14:09:06.071 INFO:tasks.workunit.client.0.smithi033.stdout: 1 514572 5.83 MB/sec execute 281 sec latency 27.559 ms 2024-05-15T14:09:06.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:06 smithi033 ceph-mon[31650]: pgmap v272: 97 pgs: 97 active+clean; 153 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 264 op/s 2024-05-15T14:09:06.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:06 smithi052 ceph-mon[36190]: pgmap v272: 97 pgs: 97 active+clean; 153 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 264 op/s 2024-05-15T14:09:06.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:06 smithi081 ceph-mon[36434]: pgmap v272: 97 pgs: 97 active+clean; 153 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 264 op/s 2024-05-15T14:09:07.071 INFO:tasks.workunit.client.0.smithi033.stdout: 1 515998 5.83 MB/sec execute 282 sec latency 20.509 ms 2024-05-15T14:09:07.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.247+0000 7fb7c2041700 1 -- 172.21.15.33:0/3277696660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7bc1084c0 msgr2=0x7fb7bc1088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:07.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.247+0000 7fb7c2041700 1 --2- 172.21.15.33:0/3277696660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7bc1084c0 0x7fb7bc1088a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac009990 tx=0x7fb7ac009ca0 comp rx=0 tx=0).stop 2024-05-15T14:09:07.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.247+0000 7fb7c2041700 1 -- 172.21.15.33:0/3277696660 shutdown_connections 2024-05-15T14:09:07.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.247+0000 7fb7c2041700 1 --2- 172.21.15.33:0/3277696660 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7bc102e00 0x7fb7bc10f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:07.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.247+0000 7fb7c2041700 1 --2- 172.21.15.33:0/3277696660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7bc1084c0 0x7fb7bc1088a0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:07.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.247+0000 7fb7c2041700 1 --2- 172.21.15.33:0/3277696660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7bc102440 0x7fb7bc1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:07.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.247+0000 7fb7c2041700 1 -- 172.21.15.33:0/3277696660 >> 172.21.15.33:0/3277696660 conn(0x7fb7bc0793e0 msgr2=0x7fb7bc0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:07.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.247+0000 7fb7c2041700 1 -- 172.21.15.33:0/3277696660 shutdown_connections 2024-05-15T14:09:07.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.247+0000 7fb7c2041700 1 -- 172.21.15.33:0/3277696660 wait complete. 2024-05-15T14:09:07.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.248+0000 7fb7c2041700 1 Processor -- start 2024-05-15T14:09:07.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.248+0000 7fb7c2041700 1 -- start start 2024-05-15T14:09:07.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.248+0000 7fb7c2041700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7bc102440 0x7fb7bc19f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:07.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.248+0000 7fb7c2041700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7bc102e00 0x7fb7bc19fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:07.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.248+0000 7fb7c2041700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7bc1084c0 0x7fb7bc1999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:07.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.248+0000 7fb7c2041700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc06bc70 con 0x7fb7bc102440 2024-05-15T14:09:07.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.248+0000 7fb7c2041700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc06baf0 con 0x7fb7bc1084c0 2024-05-15T14:09:07.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.248+0000 7fb7c2041700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc06bdf0 con 0x7fb7bc102e00 2024-05-15T14:09:07.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.248+0000 7fb7bb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7bc102440 0x7fb7bc19f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:07.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.248+0000 7fb7baffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7bc102e00 0x7fb7bc19fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:07.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.248+0000 7fb7baffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7bc102e00 0x7fb7bc19fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:56380/0 (socket says 172.21.15.33:56380) 2024-05-15T14:09:07.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.249+0000 7fb7baffd700 1 -- 172.21.15.33:0/2846862452 learned_addr learned my addr 172.21.15.33:0/2846862452 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:09:07.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.249+0000 7fb7bbfff700 1 --2- 172.21.15.33:0/2846862452 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7bc1084c0 0x7fb7bc1999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:07.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.248+0000 7fb7bb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7bc102440 0x7fb7bc19f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37460/0 (socket says 172.21.15.33:37460) 2024-05-15T14:09:07.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.249+0000 7fb7baffd700 1 -- 172.21.15.33:0/2846862452 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7bc1084c0 msgr2=0x7fb7bc1999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:07.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.249+0000 7fb7baffd700 1 --2- 172.21.15.33:0/2846862452 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7bc1084c0 0x7fb7bc1999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:07.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.249+0000 7fb7baffd700 1 -- 172.21.15.33:0/2846862452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7bc102440 msgr2=0x7fb7bc19f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:07.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.249+0000 7fb7baffd700 1 --2- 172.21.15.33:0/2846862452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7bc102440 0x7fb7bc19f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:07.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.249+0000 7fb7baffd700 1 -- 172.21.15.33:0/2846862452 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7bc19a150 con 0x7fb7bc102e00 2024-05-15T14:09:07.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.249+0000 7fb7baffd700 1 --2- 172.21.15.33:0/2846862452 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7bc102e00 0x7fb7bc19fd70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000e9c0 tx=0x7fb7b000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:07.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.249+0000 7fb7b8ff9700 1 -- 172.21.15.33:0/2846862452 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b000cbd0 con 0x7fb7bc102e00 2024-05-15T14:09:07.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.249+0000 7fb7b8ff9700 1 -- 172.21.15.33:0/2846862452 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7b000ee60 con 0x7fb7bc102e00 2024-05-15T14:09:07.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.249+0000 7fb7c2041700 1 -- 172.21.15.33:0/2846862452 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7bc19a440 con 0x7fb7bc102e00 2024-05-15T14:09:07.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.249+0000 7fb7b8ff9700 1 -- 172.21.15.33:0/2846862452 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b00055c0 con 0x7fb7bc102e00 2024-05-15T14:09:07.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.249+0000 7fb7c2041700 1 -- 172.21.15.33:0/2846862452 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7bc111ba0 con 0x7fb7bc102e00 2024-05-15T14:09:07.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.252+0000 7fb7b8ff9700 1 -- 172.21.15.33:0/2846862452 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb7b0004750 con 0x7fb7bc102e00 2024-05-15T14:09:07.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.253+0000 7fb7b8ff9700 1 --2- 172.21.15.33:0/2846862452 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fb7a40b2b20 0x7fb7a40b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:07.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.253+0000 7fb7cc8d7b80 1 Processor -- start 2024-05-15T14:09:07.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.253+0000 7fb7cc8d7b80 1 -- start start 2024-05-15T14:09:07.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.253+0000 7fb7b8ff9700 1 -- 172.21.15.33:0/2846862452 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8586+0+0 (secure 0 0 0) 0x7fb7b00e28c0 con 0x7fb7bc102e00 2024-05-15T14:09:07.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.253+0000 7fb7bb7fe700 1 --2- 172.21.15.33:0/2846862452 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fb7a40b2b20 0x7fb7a40b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:07.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.254+0000 7fb7cc8d7b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad950b4b0 0x560ad950b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:07.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.254+0000 7fb7cc8d7b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad950bdd0 0x560ad9506d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:07.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.254+0000 7fb7cc8d7b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad9507350 0x560ad95077b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:07.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.254+0000 7fb7cc8d7b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560ad9464f00 con 0x560ad950b4b0 2024-05-15T14:09:07.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.254+0000 7fb7cc8d7b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x560ad9454aa0 con 0x560ad9507350 2024-05-15T14:09:07.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.254+0000 7fb7cc8d7b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x560ad9458ae0 con 0x560ad950bdd0 2024-05-15T14:09:07.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.254+0000 7fb7baffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad950bdd0 0x560ad9506d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:07.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.254+0000 7fb7baffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad950bdd0 0x560ad9506d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:56396/0 (socket says 172.21.15.33:56396) 2024-05-15T14:09:07.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.254+0000 7fb7baffd700 1 -- 172.21.15.33:0/576269558 learned_addr learned my addr 172.21.15.33:0/576269558 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:09:07.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.254+0000 7fb7bb7fe700 1 --2- 172.21.15.33:0/2846862452 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fb7a40b2b20 0x7fb7a40b4fe0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac00b2e0 tx=0x7fb7ac01e040 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:07.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.254+0000 7fb7bb7fe700 1 --2- 172.21.15.33:0/576269558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad950b4b0 0x560ad950b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:07.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.254+0000 7fb7bbfff700 1 --2- 172.21.15.33:0/576269558 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad9507350 0x560ad95077b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:07.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.254+0000 7fb7baffd700 1 -- 172.21.15.33:0/576269558 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad9507350 msgr2=0x560ad95077b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:07.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.254+0000 7fb7baffd700 1 --2- 172.21.15.33:0/576269558 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad9507350 0x560ad95077b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:07.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.254+0000 7fb7baffd700 1 -- 172.21.15.33:0/576269558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad950b4b0 msgr2=0x560ad950b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:07.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.254+0000 7fb7baffd700 1 --2- 172.21.15.33:0/576269558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad950b4b0 0x560ad950b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:07.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.254+0000 7fb7baffd700 1 -- 172.21.15.33:0/576269558 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ad93fe140 con 0x560ad950bdd0 2024-05-15T14:09:07.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.254+0000 7fb7bb7fe700 1 --2- 172.21.15.33:0/576269558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad950b4b0 0x560ad950b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:09:07.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.254+0000 7fb7baffd700 1 --2- 172.21.15.33:0/576269558 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad950bdd0 0x560ad9506d30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb7b00a9800 tx=0x7fb7b00a9270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6999a4a7a5ad037b server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:07.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.255+0000 7fb7a1ffb700 1 -- 172.21.15.33:0/576269558 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b00b22b0 con 0x560ad950bdd0 2024-05-15T14:09:07.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.255+0000 7fb7a1ffb700 1 -- 172.21.15.33:0/576269558 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7b00aa200 con 0x560ad950bdd0 2024-05-15T14:09:07.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.255+0000 7fb7a1ffb700 1 -- 172.21.15.33:0/576269558 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b00b24d0 con 0x560ad950bdd0 2024-05-15T14:09:07.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.255+0000 7fb7cc8d7b80 1 -- 172.21.15.33:0/576269558 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad950bdd0 msgr2=0x560ad9506d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:07.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.255+0000 7fb7cc8d7b80 1 --2- 172.21.15.33:0/576269558 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad950bdd0 0x560ad9506d30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb7b00a9800 tx=0x7fb7b00a9270 comp rx=0 tx=0).stop 2024-05-15T14:09:07.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.255+0000 7fb7cc8d7b80 1 -- 172.21.15.33:0/576269558 shutdown_connections 2024-05-15T14:09:07.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.255+0000 7fb7cc8d7b80 1 --2- 172.21.15.33:0/576269558 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad9507350 0x560ad95077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:07.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.255+0000 7fb7cc8d7b80 1 --2- 172.21.15.33:0/576269558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad950b4b0 0x560ad950b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:07.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.255+0000 7fb7cc8d7b80 1 --2- 172.21.15.33:0/576269558 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad950bdd0 0x560ad9506d30 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:07.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.255+0000 7fb7cc8d7b80 1 -- 172.21.15.33:0/576269558 >> 172.21.15.33:0/576269558 conn(0x560ad9433f40 msgr2=0x560ad9501530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:07.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.256+0000 7fb7cc8d7b80 1 -- 172.21.15.33:0/576269558 shutdown_connections 2024-05-15T14:09:07.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.256+0000 7fb7cc8d7b80 1 -- 172.21.15.33:0/576269558 wait complete. 2024-05-15T14:09:07.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.256+0000 7fb7cc8d7b80 1 -- start start 2024-05-15T14:09:07.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.256+0000 7fb7cc8d7b80 1 Processor -- start 2024-05-15T14:09:07.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.257+0000 7fb7cc8d7b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad9507350 0x560ad9512c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:07.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.257+0000 7fb7cc8d7b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad950b4b0 0x560ad950ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:07.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.257+0000 7fb7cc8d7b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad950bdd0 0x560ad9510520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:07.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.257+0000 7fb7cc8d7b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560ad935b130 con 0x560ad950b4b0 2024-05-15T14:09:07.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.257+0000 7fb7cc8d7b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x560ad93bcf30 con 0x560ad9507350 2024-05-15T14:09:07.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.257+0000 7fb7cc8d7b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x560ad9513910 con 0x560ad950bdd0 2024-05-15T14:09:07.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.257+0000 7fb7cc8d7b80 10 client.0 _ll_register_callbacks cb 0x560ad93efae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:09:07.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.257+0000 7fb7baffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad950b4b0 0x560ad950ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:07.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.257+0000 7fb7baffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad950b4b0 0x560ad950ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37480/0 (socket says 172.21.15.33:37480) 2024-05-15T14:09:07.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.257+0000 7fb7baffd700 1 -- 172.21.15.33:0/1906159279 learned_addr learned my addr 172.21.15.33:0/1906159279 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:09:07.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.257+0000 7fb7bb7fe700 1 --2- 172.21.15.33:0/1906159279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad9507350 0x560ad9512c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:07.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.257+0000 7fb7bbfff700 1 --2- 172.21.15.33:0/1906159279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad950bdd0 0x560ad9510520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:07.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.257+0000 7fb7baffd700 1 -- 172.21.15.33:0/1906159279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad950bdd0 msgr2=0x560ad9510520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:07.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.257+0000 7fb7baffd700 1 --2- 172.21.15.33:0/1906159279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad950bdd0 0x560ad9510520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:07.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.257+0000 7fb7baffd700 1 -- 172.21.15.33:0/1906159279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad9507350 msgr2=0x560ad9512c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:07.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.257+0000 7fb7baffd700 1 --2- 172.21.15.33:0/1906159279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad9507350 0x560ad9512c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:07.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.257+0000 7fb7baffd700 1 -- 172.21.15.33:0/1906159279 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ad93a4a60 con 0x560ad950b4b0 2024-05-15T14:09:07.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.257+0000 7fb7bbfff700 1 --2- 172.21.15.33:0/1906159279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad950bdd0 0x560ad9510520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:09:07.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.257+0000 7fb7baffd700 1 --2- 172.21.15.33:0/1906159279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad950b4b0 0x560ad950ffe0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb7b00a5b80 tx=0x7fb7b00b2700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615cac84911357ea server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:07.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.257+0000 7fb79affd700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:09:07.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.257+0000 7fb79affd700 1 -- 172.21.15.33:0/1906159279 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b00c8020 con 0x560ad950b4b0 2024-05-15T14:09:07.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.257+0000 7fb79affd700 1 -- 172.21.15.33:0/1906159279 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8586+0+0 (secure 0 0 0) 0x7fb7b00a8f70 con 0x560ad950b4b0 2024-05-15T14:09:07.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.258+0000 7fb7cc8d7b80 1 -- 172.21.15.33:0/1906159279 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7b00aa200 con 0x560ad950b4b0 2024-05-15T14:09:07.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.258+0000 7fb79affd700 1 -- 172.21.15.33:0/1906159279 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb7b008d050 con 0x560ad950b4b0 2024-05-15T14:09:07.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.258+0000 7fb7cc8d7b80 10 client.16401 fetch_fsmap learned FSMap version 34 2024-05-15T14:09:07.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.258+0000 7fb7cc8d7b80 1 -- 172.21.15.33:0/1906159279 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560ad940e8f0 con 0x560ad950b4b0 2024-05-15T14:09:07.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.258+0000 7fb79affd700 1 -- 172.21.15.33:0/1906159279 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7b00aa200 con 0x560ad950b4b0 2024-05-15T14:09:07.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.258+0000 7fb79affd700 1 -- 172.21.15.33:0/1906159279 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fb7b0088030 con 0x560ad950b4b0 2024-05-15T14:09:07.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.258+0000 7fb7cc8d7b80 10 client.16401 fetch_fsmap finished waiting for FSMap version 34 2024-05-15T14:09:07.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.258+0000 7fb7cc8d7b80 10 client.16401 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:09:07.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.258+0000 7fb7cc8d7b80 20 client.16401 populate_metadata read hostname 'smithi033' 2024-05-15T14:09:07.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.258+0000 7fb7cc8d7b80 1 --2- 172.21.15.33:0/1906159279 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x560ad95203c0 0x560ad95227b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:07.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.258+0000 7fb7cc8d7b80 4 client.16401 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:09:07.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.258+0000 7fb7cc8d7b80 1 -- 172.21.15.33:0/1906159279 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ad9466e50 con 0x560ad95203c0 2024-05-15T14:09:07.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.258+0000 7fb7bb7fe700 1 --2- 172.21.15.33:0/1906159279 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x560ad95203c0 0x560ad95227b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:07.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.259+0000 7fb7bb7fe700 1 --2- 172.21.15.33:0/1906159279 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x560ad95203c0 0x560ad95227b0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=259cef872deffbe7 server_cookie=fd997411d4c49978 in_seq=0 out_seq=0 2024-05-15T14:09:07.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.259+0000 7fb79affd700 10 client.16401 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:09:07.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.260+0000 7fb79affd700 1 -- 172.21.15.33:0/1906159279 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560ad9466e50 con 0x560ad95203c0 2024-05-15T14:09:07.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.260+0000 7fb79affd700 10 client.16401 handle_command_reply: tid=0 2024-05-15T14:09:07.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.260+0000 7fb7cc8d7b80 1 client.16401 shutdown 2024-05-15T14:09:07.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.260+0000 7fb7cc8d7b80 1 -- 172.21.15.33:0/1906159279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad950b4b0 msgr2=0x560ad950ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:07.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.260+0000 7fb7cc8d7b80 1 --2- 172.21.15.33:0/1906159279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad950b4b0 0x560ad950ffe0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb7b00a5b80 tx=0x7fb7b00b2700 comp rx=0 tx=0).stop 2024-05-15T14:09:07.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.261+0000 7fb7cc8d7b80 1 -- 172.21.15.33:0/1906159279 shutdown_connections 2024-05-15T14:09:07.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.261+0000 7fb7cc8d7b80 1 --2- 172.21.15.33:0/1906159279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad950bdd0 0x560ad9510520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:07.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.261+0000 7fb7cc8d7b80 1 --2- 172.21.15.33:0/1906159279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad9507350 0x560ad9512c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:07.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.261+0000 7fb7cc8d7b80 1 --2- 172.21.15.33:0/1906159279 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x560ad95203c0 0x560ad95227b0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:07.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.261+0000 7fb79affd700 0 client.16401 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:09:07.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.261+0000 7fb7cc8d7b80 1 --2- 172.21.15.33:0/1906159279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad950b4b0 0x560ad950ffe0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:07.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.261+0000 7fb7cc8d7b80 1 -- 172.21.15.33:0/1906159279 >> 172.21.15.33:0/1906159279 conn(0x560ad9433f40 msgr2=0x560ad94fc050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:07.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.261+0000 7fb7cc8d7b80 1 -- 172.21.15.33:0/1906159279 shutdown_connections 2024-05-15T14:09:07.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.261+0000 7fb7cc8d7b80 1 -- 172.21.15.33:0/1906159279 wait complete. 2024-05-15T14:09:07.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.261+0000 7fb7cc8d7b80 20 client.16401 trim_cache size 0 max 16384 2024-05-15T14:09:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.284+0000 7fb7cc8d7b80 1 Processor -- start 2024-05-15T14:09:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.284+0000 7fb7cc8d7b80 1 -- start start 2024-05-15T14:09:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.284+0000 7fb7cc8d7b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad9507350 0x560ad94fc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.284+0000 7fb7cc8d7b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad950b4b0 0x560ad9504d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.284+0000 7fb7cc8d7b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad950bdd0 0x560ad9505260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.284+0000 7fb7cc8d7b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc1ae640 con 0x560ad950bdd0 2024-05-15T14:09:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.284+0000 7fb7cc8d7b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc1abe40 con 0x560ad9507350 2024-05-15T14:09:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.284+0000 7fb7cc8d7b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc1a9cb0 con 0x560ad950b4b0 2024-05-15T14:09:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.284+0000 7fb7bbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad950bdd0 0x560ad9505260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.284+0000 7fb7baffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad950b4b0 0x560ad9504d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.284+0000 7fb7bbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad950bdd0 0x560ad9505260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37496/0 (socket says 172.21.15.33:37496) 2024-05-15T14:09:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.284+0000 7fb7bbfff700 1 -- 172.21.15.33:0/3549010065 learned_addr learned my addr 172.21.15.33:0/3549010065 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:09:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.284+0000 7fb7baffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad950b4b0 0x560ad9504d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:56414/0 (socket says 172.21.15.33:56414) 2024-05-15T14:09:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.284+0000 7fb7bb7fe700 1 --2- 172.21.15.33:0/3549010065 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad9507350 0x560ad94fc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.284+0000 7fb7baffd700 1 -- 172.21.15.33:0/3549010065 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad9507350 msgr2=0x560ad94fc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:07.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.284+0000 7fb7baffd700 1 --2- 172.21.15.33:0/3549010065 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad9507350 0x560ad94fc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:07.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.284+0000 7fb7baffd700 1 -- 172.21.15.33:0/3549010065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad950bdd0 msgr2=0x560ad9505260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:07.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.284+0000 7fb7baffd700 1 --2- 172.21.15.33:0/3549010065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad950bdd0 0x560ad9505260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:07.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.285+0000 7fb7baffd700 1 -- 172.21.15.33:0/3549010065 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ad940e8f0 con 0x560ad950b4b0 2024-05-15T14:09:07.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.285+0000 7fb7baffd700 1 --2- 172.21.15.33:0/3549010065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad950b4b0 0x560ad9504d20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0009860 tx=0x7fb7b00a5a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fcd3bf6b7031cac server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:07.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.285+0000 7fb7a17fa700 1 -- 172.21.15.33:0/3549010065 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b00b6070 con 0x560ad950b4b0 2024-05-15T14:09:07.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.285+0000 7fb7a17fa700 1 -- 172.21.15.33:0/3549010065 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7b00aa200 con 0x560ad950b4b0 2024-05-15T14:09:07.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.285+0000 7fb7a17fa700 1 -- 172.21.15.33:0/3549010065 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b00c8070 con 0x560ad950b4b0 2024-05-15T14:09:07.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.286+0000 7fb7cc8d7b80 1 -- 172.21.15.33:0/3549010065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad950b4b0 msgr2=0x560ad9504d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:07.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.286+0000 7fb7cc8d7b80 1 --2- 172.21.15.33:0/3549010065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad950b4b0 0x560ad9504d20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0009860 tx=0x7fb7b00a5a00 comp rx=0 tx=0).stop 2024-05-15T14:09:07.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.286+0000 7fb7cc8d7b80 1 -- 172.21.15.33:0/3549010065 shutdown_connections 2024-05-15T14:09:07.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.286+0000 7fb7cc8d7b80 1 --2- 172.21.15.33:0/3549010065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad950bdd0 0x560ad9505260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:07.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.286+0000 7fb7cc8d7b80 1 --2- 172.21.15.33:0/3549010065 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad9507350 0x560ad94fc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:07.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.286+0000 7fb7cc8d7b80 1 --2- 172.21.15.33:0/3549010065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad950b4b0 0x560ad9504d20 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:07.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.286+0000 7fb7cc8d7b80 1 -- 172.21.15.33:0/3549010065 >> 172.21.15.33:0/3549010065 conn(0x560ad9433f40 msgr2=0x560ad951c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:07.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.286+0000 7fb7cc8d7b80 1 -- 172.21.15.33:0/3549010065 shutdown_connections 2024-05-15T14:09:07.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.286+0000 7fb7cc8d7b80 1 -- 172.21.15.33:0/3549010065 wait complete. 2024-05-15T14:09:07.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.286+0000 7fb7cc8d7b80 1 -- start start 2024-05-15T14:09:07.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.287+0000 7fb7cc8d7b80 1 Processor -- start 2024-05-15T14:09:07.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.287+0000 7fb7cc8d7b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad9507350 0x560ad9525bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:07.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.287+0000 7fb7cc8d7b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad950b4b0 0x560ad9521fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:07.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.287+0000 7fb7cc8d7b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad950bdd0 0x560ad95224f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:07.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.287+0000 7fb7baffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad950b4b0 0x560ad9521fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:07.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.287+0000 7fb7cc8d7b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc1aa300 con 0x560ad950bdd0 2024-05-15T14:09:07.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.287+0000 7fb7cc8d7b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc1ab770 con 0x560ad9507350 2024-05-15T14:09:07.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.287+0000 7fb7baffd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad950b4b0 0x560ad9521fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:56422/0 (socket says 172.21.15.33:56422) 2024-05-15T14:09:07.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.287+0000 7fb7bb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad9507350 0x560ad9525bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:07.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.287+0000 7fb7cc8d7b80 1 -- 172.21.15.33:0/2799590645 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc1aa9d0 con 0x560ad950b4b0 2024-05-15T14:09:07.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.287+0000 7fb7baffd700 1 -- 172.21.15.33:0/2799590645 learned_addr learned my addr 172.21.15.33:0/2799590645 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:09:07.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.287+0000 7fb7bbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad950bdd0 0x560ad95224f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:07.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.287+0000 7fb7cc8d7b80 10 client.0 _ll_register_callbacks cb 0x560ad93efae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:09:07.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.287+0000 7fb7baffd700 1 -- 172.21.15.33:0/2799590645 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad9507350 msgr2=0x560ad9525bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:07.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.287+0000 7fb7baffd700 1 --2- 172.21.15.33:0/2799590645 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad9507350 0x560ad9525bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:07.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.287+0000 7fb7baffd700 1 -- 172.21.15.33:0/2799590645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad950bdd0 msgr2=0x560ad95224f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:07.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.287+0000 7fb7baffd700 1 --2- 172.21.15.33:0/2799590645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad950bdd0 0x560ad95224f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:07.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.287+0000 7fb7baffd700 1 -- 172.21.15.33:0/2799590645 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ad9502000 con 0x560ad950b4b0 2024-05-15T14:09:07.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.288+0000 7fb7bb7fe700 1 --2- 172.21.15.33:0/2799590645 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad9507350 0x560ad9525bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:09:07.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.288+0000 7fb7baffd700 1 --2- 172.21.15.33:0/2799590645 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad950b4b0 0x560ad9521fb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb7b00a9800 tx=0x7fb7b00b22d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e2b356f54d7ee84 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:07.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.288+0000 7fb79affd700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-05-15T14:09:07.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.288+0000 7fb79affd700 1 -- 172.21.15.33:0/2799590645 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b00b6070 con 0x560ad950b4b0 2024-05-15T14:09:07.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.288+0000 7fb79affd700 1 -- 172.21.15.33:0/2799590645 <== mon.1 v2:172.21.15.81:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8586+0+0 (secure 0 0 0) 0x7fb7b00a5470 con 0x560ad950b4b0 2024-05-15T14:09:07.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.288+0000 7fb7cc8d7b80 1 -- 172.21.15.33:0/2799590645 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7b00aa200 con 0x560ad950b4b0 2024-05-15T14:09:07.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.288+0000 7fb79affd700 1 -- 172.21.15.33:0/2799590645 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb7b00c8070 con 0x560ad950b4b0 2024-05-15T14:09:07.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.288+0000 7fb7cc8d7b80 10 client.26005 fetch_fsmap learned FSMap version 34 2024-05-15T14:09:07.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.288+0000 7fb7cc8d7b80 1 -- 172.21.15.33:0/2799590645 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560ad9502000 con 0x560ad950b4b0 2024-05-15T14:09:07.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.288+0000 7fb79affd700 1 -- 172.21.15.33:0/2799590645 <== mon.1 v2:172.21.15.81:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7b00aa200 con 0x560ad950b4b0 2024-05-15T14:09:07.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.289+0000 7fb79affd700 1 -- 172.21.15.33:0/2799590645 <== mon.1 v2:172.21.15.81:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fb7b0088070 con 0x560ad950b4b0 2024-05-15T14:09:07.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.289+0000 7fb7cc8d7b80 10 client.26005 fetch_fsmap finished waiting for FSMap version 34 2024-05-15T14:09:07.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.289+0000 7fb7cc8d7b80 10 client.26005 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:09:07.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.289+0000 7fb7cc8d7b80 20 client.26005 populate_metadata read hostname 'smithi033' 2024-05-15T14:09:07.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.289+0000 7fb7cc8d7b80 1 --2- 172.21.15.33:0/2799590645 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x560ad958b950 0x560ad958dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:07.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.289+0000 7fb7cc8d7b80 4 client.26005 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.0, "await_for": 45.0}] 2024-05-15T14:09:07.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.289+0000 7fb7cc8d7b80 1 -- 172.21.15.33:0/2799590645 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.0, "await_for": 45.0}) -- 0x560ad9460630 con 0x560ad958b950 2024-05-15T14:09:07.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.289+0000 7fb7bb7fe700 1 --2- 172.21.15.33:0/2799590645 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x560ad958b950 0x560ad958dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:07.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.289+0000 7fb79affd700 10 client.26005 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:09:07.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.289+0000 7fb7bb7fe700 1 --2- 172.21.15.33:0/2799590645 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x560ad958b950 0x560ad958dd70 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f83b03f1f42e9d7 server_cookie=dd7eb0cfdb1cffd0 in_seq=0 out_seq=0 2024-05-15T14:09:07.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.373+0000 7fb79affd700 1 -- 172.21.15.33:0/2799590645 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560ad9460630 con 0x560ad958b950 2024-05-15T14:09:07.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.373+0000 7fb79affd700 10 client.26005 handle_command_reply: tid=0 2024-05-15T14:09:07.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.373+0000 7fb7cc8d7b80 1 client.26005 shutdown 2024-05-15T14:09:07.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.373+0000 7fb7cc8d7b80 1 -- 172.21.15.33:0/2799590645 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad950b4b0 msgr2=0x560ad9521fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:07.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.373+0000 7fb7cc8d7b80 1 --2- 172.21.15.33:0/2799590645 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad950b4b0 0x560ad9521fb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb7b00a9800 tx=0x7fb7b00b22d0 comp rx=0 tx=0).stop 2024-05-15T14:09:07.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.373+0000 7fb7cc8d7b80 1 -- 172.21.15.33:0/2799590645 shutdown_connections 2024-05-15T14:09:07.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.373+0000 7fb7cc8d7b80 1 --2- 172.21.15.33:0/2799590645 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x560ad958b950 0x560ad958dd70 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:07.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.373+0000 7fb7cc8d7b80 1 --2- 172.21.15.33:0/2799590645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ad950bdd0 0x560ad95224f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:07.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.373+0000 7fb7cc8d7b80 1 --2- 172.21.15.33:0/2799590645 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ad9507350 0x560ad9525bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:07.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.373+0000 7fb79affd700 0 client.26005 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:09:07.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.373+0000 7fb7cc8d7b80 1 --2- 172.21.15.33:0/2799590645 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x560ad950b4b0 0x560ad9521fb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:07.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.373+0000 7fb7cc8d7b80 1 -- 172.21.15.33:0/2799590645 >> 172.21.15.33:0/2799590645 conn(0x560ad9433f40 msgr2=0x560ad95115b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:07.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.373+0000 7fb7cc8d7b80 1 -- 172.21.15.33:0/2799590645 shutdown_connections 2024-05-15T14:09:07.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.373+0000 7fb7cc8d7b80 1 -- 172.21.15.33:0/2799590645 wait complete. 2024-05-15T14:09:07.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.373+0000 7fb7cc8d7b80 20 client.26005 trim_cache size 0 max 16384 2024-05-15T14:09:07.377 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T14:09:07.378 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 34, 2024-05-15T14:09:07.378 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24478, 2024-05-15T14:09:07.378 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 30, 2024-05-15T14:09:07.378 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-05-15T14:09:07.378 INFO:teuthology.orchestra.run.smithi033.stdout: "ad5b83c1": { 2024-05-15T14:09:07.378 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 30, 2024-05-15T14:09:07.378 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-05-15T14:09:07.378 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:09:07.378 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-05-15T14:09:07.378 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:09:07.378 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:09:07.378 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 90.0, 2024-05-15T14:09:07.378 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 138.0, 2024-05-15T14:09:07.379 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-05-15T14:09:07.379 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-05-15T14:09:07.379 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-05-15T14:09:07.379 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:09:07.379 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-05-15T14:09:07.379 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:09:07.379 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:09:07.379 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:09:07.379 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:09:07.379 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:09:07.379 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:09:07.379 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T14:09:07.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.374+0000 7fb7c2041700 1 -- 172.21.15.33:0/2846862452 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fb7a40b2b20 msgr2=0x7fb7a40b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:07.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.374+0000 7fb7c2041700 1 --2- 172.21.15.33:0/2846862452 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fb7a40b2b20 0x7fb7a40b4fe0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac00b2e0 tx=0x7fb7ac01e040 comp rx=0 tx=0).stop 2024-05-15T14:09:07.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.374+0000 7fb7c2041700 1 -- 172.21.15.33:0/2846862452 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7bc102e00 msgr2=0x7fb7bc19fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:07.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.374+0000 7fb7c2041700 1 --2- 172.21.15.33:0/2846862452 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7bc102e00 0x7fb7bc19fd70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000e9c0 tx=0x7fb7b000ecd0 comp rx=0 tx=0).stop 2024-05-15T14:09:07.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.374+0000 7fb7c2041700 1 -- 172.21.15.33:0/2846862452 shutdown_connections 2024-05-15T14:09:07.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.374+0000 7fb7c2041700 1 --2- 172.21.15.33:0/2846862452 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fb7a40b2b20 0x7fb7a40b4fe0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:07.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.374+0000 7fb7c2041700 1 --2- 172.21.15.33:0/2846862452 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7bc1084c0 0x7fb7bc1999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:07.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.374+0000 7fb7c2041700 1 --2- 172.21.15.33:0/2846862452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7bc102440 0x7fb7bc19f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:07.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.374+0000 7fb7c2041700 1 --2- 172.21.15.33:0/2846862452 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7bc102e00 0x7fb7bc19fd70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:07.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.374+0000 7fb7c2041700 1 -- 172.21.15.33:0/2846862452 >> 172.21.15.33:0/2846862452 conn(0x7fb7bc0793e0 msgr2=0x7fb7bc1004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:07.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.374+0000 7fb7c2041700 1 -- 172.21.15.33:0/2846862452 shutdown_connections 2024-05-15T14:09:07.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:07.374+0000 7fb7c2041700 1 -- 172.21.15.33:0/2846862452 wait complete. 2024-05-15T14:09:07.986 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ad5b83c1', quiesce took 6.4 seconds. Will release after: 11.6 2024-05-15T14:09:08.072 INFO:tasks.workunit.client.0.smithi033.stdout: 1 516383 5.82 MB/sec execute 283 sec latency 746.084 ms 2024-05-15T14:09:08.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:08 smithi033 ceph-mon[31650]: pgmap v273: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 274 op/s 2024-05-15T14:09:08.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:08 smithi052 ceph-mon[36190]: pgmap v273: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 274 op/s 2024-05-15T14:09:08.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:08 smithi081 ceph-mon[36434]: pgmap v273: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 274 op/s 2024-05-15T14:09:09.071 INFO:tasks.workunit.client.0.smithi033.stdout: 1 516383 5.80 MB/sec execute 284 sec latency 1746.176 ms 2024-05-15T14:09:10.072 INFO:tasks.workunit.client.0.smithi033.stdout: 1 516383 5.78 MB/sec execute 285 sec latency 2746.304 ms 2024-05-15T14:09:10.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:10 smithi033 ceph-mon[31650]: pgmap v274: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 274 op/s 2024-05-15T14:09:10.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:10 smithi052 ceph-mon[36190]: pgmap v274: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 274 op/s 2024-05-15T14:09:10.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:10 smithi081 ceph-mon[36434]: pgmap v274: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 274 op/s 2024-05-15T14:09:11.072 INFO:tasks.workunit.client.0.smithi033.stdout: 1 516383 5.76 MB/sec execute 286 sec latency 3746.428 ms 2024-05-15T14:09:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.267+0000 7fb0862bd640 1 -- 172.21.15.33:0/2521532195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb080102370 msgr2=0x7fb08010e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.267+0000 7fb0862bd640 1 --2- 172.21.15.33:0/2521532195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb080102370 0x7fb08010e830 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fb078009cd0 tx=0x7fb07802a8e0 comp rx=0 tx=0).stop 2024-05-15T14:09:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.267+0000 7fb0862bd640 1 -- 172.21.15.33:0/2521532195 shutdown_connections 2024-05-15T14:09:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.267+0000 7fb0862bd640 1 --2- 172.21.15.33:0/2521532195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb080102370 0x7fb08010e830 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.267+0000 7fb0862bd640 1 --2- 172.21.15.33:0/2521532195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb0801019b0 0x7fb080101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.267+0000 7fb0862bd640 1 --2- 172.21.15.33:0/2521532195 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb080107970 0x7fb080107d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.267+0000 7fb0862bd640 1 -- 172.21.15.33:0/2521532195 >> 172.21.15.33:0/2521532195 conn(0x7fb080078aa0 msgr2=0x7fb080078eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.267+0000 7fb0862bd640 1 -- 172.21.15.33:0/2521532195 shutdown_connections 2024-05-15T14:09:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.267+0000 7fb0862bd640 1 -- 172.21.15.33:0/2521532195 wait complete. 2024-05-15T14:09:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.267+0000 7fb0862bd640 1 Processor -- start 2024-05-15T14:09:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.268+0000 7fb0862bd640 1 -- start start 2024-05-15T14:09:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.268+0000 7fb0862bd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0801019b0 0x7fb0801991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.268+0000 7fb0862bd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb080102370 0x7fb080199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.268+0000 7fb0862bd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb080107970 0x7fb08019da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.268+0000 7fb0862bd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb080111c30 con 0x7fb080107970 2024-05-15T14:09:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.268+0000 7fb0862bd640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb080111ab0 con 0x7fb080102370 2024-05-15T14:09:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.268+0000 7fb0862bd640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb080111db0 con 0x7fb0801019b0 2024-05-15T14:09:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.268+0000 7fb08485a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb080107970 0x7fb08019da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.268+0000 7fb08485a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb080107970 0x7fb08019da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37530/0 (socket says 172.21.15.33:37530) 2024-05-15T14:09:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.268+0000 7fb08485a640 1 -- 172.21.15.33:0/2673857547 learned_addr learned my addr 172.21.15.33:0/2673857547 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:09:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.268+0000 7fb07f7fe640 1 --2- 172.21.15.33:0/2673857547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb080102370 0x7fb080199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.268+0000 7fb07ffff640 1 --2- 172.21.15.33:0/2673857547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0801019b0 0x7fb0801991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.268+0000 7fb08485a640 1 -- 172.21.15.33:0/2673857547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0801019b0 msgr2=0x7fb0801991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.268+0000 7fb08485a640 1 --2- 172.21.15.33:0/2673857547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0801019b0 0x7fb0801991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.268+0000 7fb08485a640 1 -- 172.21.15.33:0/2673857547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb080102370 msgr2=0x7fb080199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.268+0000 7fb08485a640 1 --2- 172.21.15.33:0/2673857547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb080102370 0x7fb080199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.268+0000 7fb08485a640 1 -- 172.21.15.33:0/2673857547 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb08019e1b0 con 0x7fb080107970 2024-05-15T14:09:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.269+0000 7fb08485a640 1 --2- 172.21.15.33:0/2673857547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb080107970 0x7fb08019da80 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb078033040 tx=0x7fb078003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.269+0000 7fb07d7fa640 1 -- 172.21.15.33:0/2673857547 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb078004040 con 0x7fb080107970 2024-05-15T14:09:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.269+0000 7fb07d7fa640 1 -- 172.21.15.33:0/2673857547 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb078004690 con 0x7fb080107970 2024-05-15T14:09:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.269+0000 7fb0862bd640 1 -- 172.21.15.33:0/2673857547 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb08019e440 con 0x7fb080107970 2024-05-15T14:09:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.269+0000 7fb07d7fa640 1 -- 172.21.15.33:0/2673857547 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb078030e20 con 0x7fb080107970 2024-05-15T14:09:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.269+0000 7fb0862bd640 1 -- 172.21.15.33:0/2673857547 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0801a5d30 con 0x7fb080107970 2024-05-15T14:09:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.271+0000 7fb07d7fa640 1 -- 172.21.15.33:0/2673857547 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb0780305a0 con 0x7fb080107970 2024-05-15T14:09:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.272+0000 7fb088fd5740 1 Processor -- start 2024-05-15T14:09:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.272+0000 7fb088fd5740 1 -- start start 2024-05-15T14:09:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.272+0000 7fb088fd5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29402c7d0 0x55e29402cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.272+0000 7fb088fd5740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e29402d0f0 0x55e29403b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.272+0000 7fb088fd5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e29403bee0 0x55e29403e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.272+0000 7fb088fd5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e293f9fc30 con 0x55e29402c7d0 2024-05-15T14:09:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.272+0000 7fb088fd5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e293f8f7d0 con 0x55e29403bee0 2024-05-15T14:09:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.272+0000 7fb088fd5740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55e293f93810 con 0x55e29402d0f0 2024-05-15T14:09:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.272+0000 7fb07f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29402c7d0 0x55e29402cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.272+0000 7fb07f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29402c7d0 0x55e29402cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37536/0 (socket says 172.21.15.33:37536) 2024-05-15T14:09:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.272+0000 7fb07f7fe640 1 -- 172.21.15.33:0/300595751 learned_addr learned my addr 172.21.15.33:0/300595751 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:09:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.272+0000 7fb07ffff640 1 --2- 172.21.15.33:0/300595751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e29402d0f0 0x55e29403b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.272+0000 7fb07d7fa640 1 --2- 172.21.15.33:0/2673857547 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fb0680b2950 0x7fb0680b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.273+0000 7fb07f7fe640 1 -- 172.21.15.33:0/300595751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e29402d0f0 msgr2=0x55e29403b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.273+0000 7fb07f7fe640 1 --2- 172.21.15.33:0/300595751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e29402d0f0 0x55e29403b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.273+0000 7fb07f7fe640 1 -- 172.21.15.33:0/300595751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e29403bee0 msgr2=0x55e29403e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.273+0000 7fb07f7fe640 1 --2- 172.21.15.33:0/300595751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e29403bee0 0x55e29403e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.273+0000 7fb08485a640 1 --2- 172.21.15.33:0/2673857547 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fb0680b2950 0x7fb0680b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.273+0000 7fb07f7fe640 1 -- 172.21.15.33:0/300595751 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e293ca4210 con 0x55e29402c7d0 2024-05-15T14:09:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.273+0000 7fb07ffff640 1 --2- 172.21.15.33:0/300595751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e29402d0f0 0x55e29403b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:09:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.273+0000 7fb07d7fa640 1 -- 172.21.15.33:0/2673857547 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fb07802e030 con 0x7fb080107970 2024-05-15T14:09:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.273+0000 7fb07f7fe640 1 --2- 172.21.15.33:0/300595751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29402c7d0 0x55e29402cbb0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb07400bd80 tx=0x7fb0740127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3ed254bdca67850 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.273+0000 7fb0667fc640 1 -- 172.21.15.33:0/300595751 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07400be70 con 0x55e29402c7d0 2024-05-15T14:09:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.273+0000 7fb0667fc640 1 -- 172.21.15.33:0/300595751 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb074016070 con 0x55e29402c7d0 2024-05-15T14:09:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.273+0000 7fb0667fc640 1 -- 172.21.15.33:0/300595751 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0740107a0 con 0x55e29402c7d0 2024-05-15T14:09:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.274+0000 7fb088fd5740 1 -- 172.21.15.33:0/300595751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29402c7d0 msgr2=0x55e29402cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.274+0000 7fb08485a640 1 --2- 172.21.15.33:0/2673857547 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fb0680b2950 0x7fb0680b4e10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb0780caa30 tx=0x7fb07802a8e0 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.274+0000 7fb088fd5740 1 --2- 172.21.15.33:0/300595751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29402c7d0 0x55e29402cbb0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb07400bd80 tx=0x7fb0740127c0 comp rx=0 tx=0).stop 2024-05-15T14:09:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.274+0000 7fb088fd5740 1 -- 172.21.15.33:0/300595751 shutdown_connections 2024-05-15T14:09:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.274+0000 7fb088fd5740 1 --2- 172.21.15.33:0/300595751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e29403bee0 0x55e29403e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.274+0000 7fb088fd5740 1 --2- 172.21.15.33:0/300595751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e29402d0f0 0x55e29403b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.274+0000 7fb088fd5740 1 --2- 172.21.15.33:0/300595751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29402c7d0 0x55e29402cbb0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.274+0000 7fb088fd5740 1 -- 172.21.15.33:0/300595751 >> 172.21.15.33:0/300595751 conn(0x55e294028e00 msgr2=0x55e2940291e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.274+0000 7fb088fd5740 1 -- 172.21.15.33:0/300595751 shutdown_connections 2024-05-15T14:09:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.274+0000 7fb088fd5740 1 -- 172.21.15.33:0/300595751 wait complete. 2024-05-15T14:09:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.274+0000 7fb088fd5740 1 -- start start 2024-05-15T14:09:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.275+0000 7fb088fd5740 1 Processor -- start 2024-05-15T14:09:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.275+0000 7fb088fd5740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e29402c7d0 0x55e29403b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.275+0000 7fb088fd5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e29402d0f0 0x55e294038bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.275+0000 7fb088fd5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29403bee0 0x55e2940391d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.275+0000 7fb088fd5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e293c75790 con 0x55e29403bee0 2024-05-15T14:09:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.275+0000 7fb088fd5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e293f4b890 con 0x55e29402d0f0 2024-05-15T14:09:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.275+0000 7fb088fd5740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55e29403ebb0 con 0x55e29402c7d0 2024-05-15T14:09:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.275+0000 7fb088fd5740 10 client.0 _ll_register_callbacks cb 0x55e293f4ed60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:09:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.275+0000 7fb07f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29403bee0 0x55e2940391d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.275+0000 7fb07f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29403bee0 0x55e2940391d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37540/0 (socket says 172.21.15.33:37540) 2024-05-15T14:09:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.275+0000 7fb07f7fe640 1 -- 172.21.15.33:0/992016756 learned_addr learned my addr 172.21.15.33:0/992016756 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:09:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.275+0000 7fb07ffff640 1 --2- 172.21.15.33:0/992016756 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e29402d0f0 0x55e294038bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.275+0000 7fb07f7fe640 1 --2- 172.21.15.33:0/992016756 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e29402c7d0 0x55e29403b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.275+0000 7fb07f7fe640 1 -- 172.21.15.33:0/992016756 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e29402c7d0 msgr2=0x55e29403b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.275+0000 7fb07f7fe640 1 --2- 172.21.15.33:0/992016756 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e29402c7d0 0x55e29403b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.275+0000 7fb07f7fe640 1 -- 172.21.15.33:0/992016756 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e29402d0f0 msgr2=0x55e294038bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.275+0000 7fb07f7fe640 1 --2- 172.21.15.33:0/992016756 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e29402d0f0 0x55e294038bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.275+0000 7fb07f7fe640 1 -- 172.21.15.33:0/992016756 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e293d70620 con 0x55e29403bee0 2024-05-15T14:09:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.275+0000 7fb07f7fe640 1 --2- 172.21.15.33:0/992016756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29403bee0 0x55e2940391d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fb07400b3d0 tx=0x7fb07401bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f9fe2d335517cc server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.275+0000 7fb05f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:09:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.276+0000 7fb05f7fe640 1 -- 172.21.15.33:0/992016756 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb074004440 con 0x55e29403bee0 2024-05-15T14:09:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.276+0000 7fb088fd5740 1 -- 172.21.15.33:0/992016756 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb074016070 con 0x55e29403bee0 2024-05-15T14:09:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.276+0000 7fb05f7fe640 1 -- 172.21.15.33:0/992016756 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fb074005030 con 0x55e29403bee0 2024-05-15T14:09:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.276+0000 7fb05f7fe640 1 -- 172.21.15.33:0/992016756 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb074005230 con 0x55e29403bee0 2024-05-15T14:09:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.276+0000 7fb088fd5740 10 client.16419 fetch_fsmap learned FSMap version 34 2024-05-15T14:09:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.276+0000 7fb088fd5740 1 -- 172.21.15.33:0/992016756 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e29404a700 con 0x55e29403bee0 2024-05-15T14:09:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.276+0000 7fb05f7fe640 1 -- 172.21.15.33:0/992016756 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb074016070 con 0x55e29403bee0 2024-05-15T14:09:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.276+0000 7fb05f7fe640 1 -- 172.21.15.33:0/992016756 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fb0740035f0 con 0x55e29403bee0 2024-05-15T14:09:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.276+0000 7fb088fd5740 10 client.16419 fetch_fsmap finished waiting for FSMap version 34 2024-05-15T14:09:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.276+0000 7fb088fd5740 10 client.16419 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:09:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.276+0000 7fb088fd5740 20 client.16419 populate_metadata read hostname 'smithi033' 2024-05-15T14:09:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.276+0000 7fb088fd5740 1 --2- 172.21.15.33:0/992016756 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e29404bb10 0x55e29404df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.276+0000 7fb088fd5740 4 client.16419 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:09:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.276+0000 7fb088fd5740 1 -- 172.21.15.33:0/992016756 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e293fa1b80 con 0x55e29404bb10 2024-05-15T14:09:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.277+0000 7fb07ffff640 1 --2- 172.21.15.33:0/992016756 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e29404bb10 0x55e29404df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.277+0000 7fb05f7fe640 10 client.16419 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:09:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.277+0000 7fb07ffff640 1 --2- 172.21.15.33:0/992016756 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e29404bb10 0x55e29404df00 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6619e89c4586c211 server_cookie=658fe81fff6329ac in_seq=0 out_seq=0 2024-05-15T14:09:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.278+0000 7fb05f7fe640 1 -- 172.21.15.33:0/992016756 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e293fa1b80 con 0x55e29404bb10 2024-05-15T14:09:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.278+0000 7fb05f7fe640 10 client.16419 handle_command_reply: tid=0 2024-05-15T14:09:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.278+0000 7fb088fd5740 1 client.16419 shutdown 2024-05-15T14:09:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.278+0000 7fb088fd5740 1 -- 172.21.15.33:0/992016756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29403bee0 msgr2=0x55e2940391d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.278+0000 7fb088fd5740 1 --2- 172.21.15.33:0/992016756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29403bee0 0x55e2940391d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fb07400b3d0 tx=0x7fb07401bf90 comp rx=0 tx=0).stop 2024-05-15T14:09:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.278+0000 7fb088fd5740 1 -- 172.21.15.33:0/992016756 shutdown_connections 2024-05-15T14:09:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.278+0000 7fb088fd5740 1 --2- 172.21.15.33:0/992016756 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e29404bb10 0x55e29404df00 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.278+0000 7fb088fd5740 1 --2- 172.21.15.33:0/992016756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29403bee0 0x55e2940391d0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.278+0000 7fb088fd5740 1 --2- 172.21.15.33:0/992016756 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e29402d0f0 0x55e294038bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.278+0000 7fb088fd5740 1 --2- 172.21.15.33:0/992016756 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e29402c7d0 0x55e29403b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.278+0000 7fb05f7fe640 0 client.16419 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:09:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.278+0000 7fb088fd5740 1 -- 172.21.15.33:0/992016756 >> 172.21.15.33:0/992016756 conn(0x55e294028e00 msgr2=0x55e29402ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.278+0000 7fb088fd5740 1 -- 172.21.15.33:0/992016756 shutdown_connections 2024-05-15T14:09:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.278+0000 7fb088fd5740 1 -- 172.21.15.33:0/992016756 wait complete. 2024-05-15T14:09:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.278+0000 7fb088fd5740 20 client.16419 trim_cache size 0 max 16384 2024-05-15T14:09:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.294+0000 7fb088fd5740 1 Processor -- start 2024-05-15T14:09:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.294+0000 7fb088fd5740 1 -- start start 2024-05-15T14:09:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.294+0000 7fb088fd5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e29402c7d0 0x55e294047af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.294+0000 7fb088fd5740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e29402d0f0 0x55e29403b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.294+0000 7fb088fd5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29403bee0 0x55e294053530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.294+0000 7fb088fd5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb0801aeda0 con 0x55e29403bee0 2024-05-15T14:09:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.294+0000 7fb088fd5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb0801ac5e0 con 0x55e29402c7d0 2024-05-15T14:09:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.294+0000 7fb088fd5740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb0801aa450 con 0x55e29402d0f0 2024-05-15T14:09:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.294+0000 7fb07f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29403bee0 0x55e294053530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.294+0000 7fb07f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29403bee0 0x55e294053530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37544/0 (socket says 172.21.15.33:37544) 2024-05-15T14:09:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.294+0000 7fb07f7fe640 1 -- 172.21.15.33:0/2590234288 learned_addr learned my addr 172.21.15.33:0/2590234288 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:09:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.294+0000 7fb07f7fe640 1 --2- 172.21.15.33:0/2590234288 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e29402c7d0 0x55e294047af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.294+0000 7fb07ffff640 1 --2- 172.21.15.33:0/2590234288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e29402d0f0 0x55e29403b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.294+0000 7fb07f7fe640 1 -- 172.21.15.33:0/2590234288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e29402d0f0 msgr2=0x55e29403b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.294+0000 7fb07f7fe640 1 --2- 172.21.15.33:0/2590234288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e29402d0f0 0x55e29403b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.294+0000 7fb07f7fe640 1 -- 172.21.15.33:0/2590234288 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e29402c7d0 msgr2=0x55e294047af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.294+0000 7fb07f7fe640 1 --2- 172.21.15.33:0/2590234288 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e29402c7d0 0x55e294047af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.294+0000 7fb07f7fe640 1 -- 172.21.15.33:0/2590234288 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e29402cf20 con 0x55e29403bee0 2024-05-15T14:09:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.294+0000 7fb07ffff640 1 --2- 172.21.15.33:0/2590234288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e29402d0f0 0x55e29403b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:09:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.295+0000 7fb07f7fe640 1 --2- 172.21.15.33:0/2590234288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29403bee0 0x55e294053530 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fb07400b540 tx=0x7fb0740127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f35816526af1bfa server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.295+0000 7fb065ffb640 1 -- 172.21.15.33:0/2590234288 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb074012cc0 con 0x55e29403bee0 2024-05-15T14:09:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.295+0000 7fb065ffb640 1 -- 172.21.15.33:0/2590234288 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb074016040 con 0x55e29403bee0 2024-05-15T14:09:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.295+0000 7fb065ffb640 1 -- 172.21.15.33:0/2590234288 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb074003c30 con 0x55e29403bee0 2024-05-15T14:09:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.295+0000 7fb088fd5740 1 -- 172.21.15.33:0/2590234288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29403bee0 msgr2=0x55e294053530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.295+0000 7fb088fd5740 1 --2- 172.21.15.33:0/2590234288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29403bee0 0x55e294053530 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fb07400b540 tx=0x7fb0740127c0 comp rx=0 tx=0).stop 2024-05-15T14:09:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.295+0000 7fb088fd5740 1 -- 172.21.15.33:0/2590234288 shutdown_connections 2024-05-15T14:09:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.295+0000 7fb088fd5740 1 --2- 172.21.15.33:0/2590234288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29403bee0 0x55e294053530 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.295+0000 7fb088fd5740 1 --2- 172.21.15.33:0/2590234288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e29402d0f0 0x55e29403b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.295+0000 7fb088fd5740 1 --2- 172.21.15.33:0/2590234288 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e29402c7d0 0x55e294047af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.295+0000 7fb088fd5740 1 -- 172.21.15.33:0/2590234288 >> 172.21.15.33:0/2590234288 conn(0x55e294028e00 msgr2=0x55e2940341d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.295+0000 7fb088fd5740 1 -- 172.21.15.33:0/2590234288 shutdown_connections 2024-05-15T14:09:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.295+0000 7fb088fd5740 1 -- 172.21.15.33:0/2590234288 wait complete. 2024-05-15T14:09:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.296+0000 7fb088fd5740 1 -- start start 2024-05-15T14:09:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.296+0000 7fb088fd5740 1 Processor -- start 2024-05-15T14:09:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.296+0000 7fb088fd5740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e29402c7d0 0x55e2940442c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.296+0000 7fb088fd5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e29402d0f0 0x55e29403f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.296+0000 7fb088fd5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29403bee0 0x55e29403fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.296+0000 7fb088fd5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb0801aaaa0 con 0x55e29403bee0 2024-05-15T14:09:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.296+0000 7fb088fd5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb0801abf10 con 0x55e29402d0f0 2024-05-15T14:09:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.296+0000 7fb088fd5740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb0801ab170 con 0x55e29402c7d0 2024-05-15T14:09:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.296+0000 7fb088fd5740 10 client.0 _ll_register_callbacks cb 0x7fb0740040d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:09:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.296+0000 7fb07f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29403bee0 0x55e29403fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.296+0000 7fb07f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29403bee0 0x55e29403fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37560/0 (socket says 172.21.15.33:37560) 2024-05-15T14:09:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.296+0000 7fb07f7fe640 1 -- 172.21.15.33:0/930298388 learned_addr learned my addr 172.21.15.33:0/930298388 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:09:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.297+0000 7fb07f7fe640 1 --2- 172.21.15.33:0/930298388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e29402c7d0 0x55e2940442c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.297+0000 7fb07ffff640 1 --2- 172.21.15.33:0/930298388 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e29402d0f0 0x55e29403f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.297+0000 7fb07f7fe640 1 -- 172.21.15.33:0/930298388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e29402c7d0 msgr2=0x55e2940442c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.297+0000 7fb07f7fe640 1 --2- 172.21.15.33:0/930298388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e29402c7d0 0x55e2940442c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.297+0000 7fb07f7fe640 1 -- 172.21.15.33:0/930298388 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e29402d0f0 msgr2=0x55e29403f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.297+0000 7fb07f7fe640 1 --2- 172.21.15.33:0/930298388 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e29402d0f0 0x55e29403f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.297+0000 7fb07f7fe640 1 -- 172.21.15.33:0/930298388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2940402b0 con 0x55e29403bee0 2024-05-15T14:09:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.297+0000 7fb07f7fe640 1 --2- 172.21.15.33:0/930298388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29403bee0 0x55e29403fbc0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fb074003fd0 tx=0x7fb07400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab3c1a3b40e490b server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.297+0000 7fb05f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:09:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.297+0000 7fb05f7fe640 1 -- 172.21.15.33:0/930298388 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb074035070 con 0x55e29403bee0 2024-05-15T14:09:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.297+0000 7fb088fd5740 1 -- 172.21.15.33:0/930298388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb074016040 con 0x55e29403bee0 2024-05-15T14:09:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.297+0000 7fb05f7fe640 1 -- 172.21.15.33:0/930298388 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fb0740035f0 con 0x55e29403bee0 2024-05-15T14:09:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.297+0000 7fb05f7fe640 1 -- 172.21.15.33:0/930298388 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb074043050 con 0x55e29403bee0 2024-05-15T14:09:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.297+0000 7fb088fd5740 10 client.16425 fetch_fsmap learned FSMap version 34 2024-05-15T14:09:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.297+0000 7fb088fd5740 1 -- 172.21.15.33:0/930298388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e2940b5f60 con 0x55e29403bee0 2024-05-15T14:09:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.297+0000 7fb05f7fe640 1 -- 172.21.15.33:0/930298388 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb074016040 con 0x55e29403bee0 2024-05-15T14:09:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.298+0000 7fb05f7fe640 1 -- 172.21.15.33:0/930298388 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fb074032480 con 0x55e29403bee0 2024-05-15T14:09:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.298+0000 7fb088fd5740 10 client.16425 fetch_fsmap finished waiting for FSMap version 34 2024-05-15T14:09:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.298+0000 7fb088fd5740 10 client.16425 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:09:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.298+0000 7fb088fd5740 20 client.16425 populate_metadata read hostname 'smithi033' 2024-05-15T14:09:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.298+0000 7fb088fd5740 1 --2- 172.21.15.33:0/930298388 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e2940b65f0 0x55e2940b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.298+0000 7fb088fd5740 4 client.16425 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:09:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.298+0000 7fb088fd5740 1 -- 172.21.15.33:0/930298388 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e293f9b360 con 0x55e2940b65f0 2024-05-15T14:09:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.298+0000 7fb07ffff640 1 --2- 172.21.15.33:0/930298388 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e2940b65f0 0x55e2940b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.298+0000 7fb07ffff640 1 --2- 172.21.15.33:0/930298388 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e2940b65f0 0x55e2940b8a10 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fab41db4a348089e server_cookie=baf1095f618a00 in_seq=0 out_seq=0 2024-05-15T14:09:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.298+0000 7fb05f7fe640 10 client.16425 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:09:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.358+0000 7fb05f7fe640 1 -- 172.21.15.33:0/930298388 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e293f9b360 con 0x55e2940b65f0 2024-05-15T14:09:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.358+0000 7fb05f7fe640 10 client.16425 handle_command_reply: tid=0 2024-05-15T14:09:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.358+0000 7fb088fd5740 1 client.16425 shutdown 2024-05-15T14:09:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.358+0000 7fb088fd5740 1 -- 172.21.15.33:0/930298388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29403bee0 msgr2=0x55e29403fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.358+0000 7fb088fd5740 1 --2- 172.21.15.33:0/930298388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29403bee0 0x55e29403fbc0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fb074003fd0 tx=0x7fb07400b480 comp rx=0 tx=0).stop 2024-05-15T14:09:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.358+0000 7fb088fd5740 1 -- 172.21.15.33:0/930298388 shutdown_connections 2024-05-15T14:09:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.358+0000 7fb088fd5740 1 --2- 172.21.15.33:0/930298388 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55e2940b65f0 0x55e2940b8a10 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.358+0000 7fb088fd5740 1 --2- 172.21.15.33:0/930298388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29403bee0 0x55e29403fbc0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.358+0000 7fb05f7fe640 0 client.16425 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:09:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.358+0000 7fb088fd5740 1 --2- 172.21.15.33:0/930298388 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e29402d0f0 0x55e29403f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.358+0000 7fb088fd5740 1 --2- 172.21.15.33:0/930298388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e29402c7d0 0x55e2940442c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.358+0000 7fb088fd5740 1 -- 172.21.15.33:0/930298388 >> 172.21.15.33:0/930298388 conn(0x55e294028e00 msgr2=0x55e294027870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.358+0000 7fb088fd5740 1 -- 172.21.15.33:0/930298388 shutdown_connections 2024-05-15T14:09:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.358+0000 7fb088fd5740 1 -- 172.21.15.33:0/930298388 wait complete. 2024-05-15T14:09:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.358+0000 7fb088fd5740 20 client.16425 trim_cache size 0 max 16384 2024-05-15T14:09:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:09:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:09:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:09:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.359+0000 7fb0862bd640 1 -- 172.21.15.33:0/2673857547 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fb0680b2950 msgr2=0x7fb0680b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.359+0000 7fb0862bd640 1 --2- 172.21.15.33:0/2673857547 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fb0680b2950 0x7fb0680b4e10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb0780caa30 tx=0x7fb07802a8e0 comp rx=0 tx=0).stop 2024-05-15T14:09:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.359+0000 7fb0862bd640 1 -- 172.21.15.33:0/2673857547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb080107970 msgr2=0x7fb08019da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.359+0000 7fb0862bd640 1 --2- 172.21.15.33:0/2673857547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb080107970 0x7fb08019da80 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb078033040 tx=0x7fb078003970 comp rx=0 tx=0).stop 2024-05-15T14:09:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.359+0000 7fb0862bd640 1 -- 172.21.15.33:0/2673857547 shutdown_connections 2024-05-15T14:09:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.359+0000 7fb0862bd640 1 --2- 172.21.15.33:0/2673857547 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fb0680b2950 0x7fb0680b4e10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.359+0000 7fb0862bd640 1 --2- 172.21.15.33:0/2673857547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb080107970 0x7fb08019da80 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.359+0000 7fb0862bd640 1 --2- 172.21.15.33:0/2673857547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb080102370 0x7fb080199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.359+0000 7fb0862bd640 1 --2- 172.21.15.33:0/2673857547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0801019b0 0x7fb0801991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.359+0000 7fb0862bd640 1 -- 172.21.15.33:0/2673857547 >> 172.21.15.33:0/2673857547 conn(0x7fb080078aa0 msgr2=0x7fb08006b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.359+0000 7fb0862bd640 1 -- 172.21.15.33:0/2673857547 shutdown_connections 2024-05-15T14:09:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:11.359+0000 7fb0862bd640 1 -- 172.21.15.33:0/2673857547 wait complete. 2024-05-15T14:09:11.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:11 smithi052 ceph-mon[36190]: osdmap e91: 12 total, 12 up, 12 in 2024-05-15T14:09:11.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:11 smithi081 ceph-mon[36434]: osdmap e91: 12 total, 12 up, 12 in 2024-05-15T14:09:11.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:11 smithi033 ceph-mon[31650]: osdmap e91: 12 total, 12 up, 12 in 2024-05-15T14:09:12.072 INFO:tasks.workunit.client.0.smithi033.stdout: 1 516383 5.74 MB/sec execute 287 sec latency 4746.502 ms 2024-05-15T14:09:12.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:12 smithi052 ceph-mon[36190]: pgmap v276: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-05-15T14:09:12.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:12 smithi081 ceph-mon[36434]: pgmap v276: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-05-15T14:09:12.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:12 smithi033 ceph-mon[31650]: pgmap v276: 97 pgs: 97 active+clean; 160 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-05-15T14:09:13.072 INFO:tasks.workunit.client.0.smithi033.stdout: 1 516383 5.72 MB/sec execute 288 sec latency 5746.624 ms 2024-05-15T14:09:14.072 INFO:tasks.workunit.client.0.smithi033.stdout: 1 516383 5.70 MB/sec execute 289 sec latency 6746.670 ms 2024-05-15T14:09:14.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:14 smithi052 ceph-mon[36190]: pgmap v277: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-05-15T14:09:14.859 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:14 smithi081 ceph-mon[36434]: pgmap v277: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-05-15T14:09:14.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:14 smithi033 ceph-mon[31650]: pgmap v277: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-05-15T14:09:15.072 INFO:tasks.workunit.client.0.smithi033.stdout: 1 516383 5.68 MB/sec execute 290 sec latency 7746.752 ms 2024-05-15T14:09:16.072 INFO:tasks.workunit.client.0.smithi033.stdout: 1 516383 5.66 MB/sec execute 291 sec latency 8746.825 ms 2024-05-15T14:09:16.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:16 smithi052 ceph-mon[36190]: pgmap v278: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-05-15T14:09:16.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:16 smithi081 ceph-mon[36434]: pgmap v278: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-05-15T14:09:16.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:16 smithi033 ceph-mon[31650]: pgmap v278: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-05-15T14:09:17.072 INFO:tasks.workunit.client.0.smithi033.stdout: 1 516383 5.64 MB/sec execute 292 sec latency 9746.868 ms 2024-05-15T14:09:18.072 INFO:tasks.workunit.client.0.smithi033.stdout: 1 516383 5.62 MB/sec execute 293 sec latency 10746.925 ms 2024-05-15T14:09:18.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:18 smithi052 ceph-mon[36190]: pgmap v279: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 369 KiB/s wr, 52 op/s 2024-05-15T14:09:18.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:18 smithi081 ceph-mon[36434]: pgmap v279: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 369 KiB/s wr, 52 op/s 2024-05-15T14:09:18.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:18 smithi033 ceph-mon[31650]: pgmap v279: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 369 KiB/s wr, 52 op/s 2024-05-15T14:09:19.072 INFO:tasks.workunit.client.0.smithi033.stdout: 1 516383 5.60 MB/sec execute 294 sec latency 11747.030 ms 2024-05-15T14:09:19.587 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T14:09:19.854 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:09:20.073 INFO:tasks.workunit.client.0.smithi033.stdout: 1 516383 5.58 MB/sec execute 295 sec latency 12747.130 ms 2024-05-15T14:09:20.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:20 smithi052 ceph-mon[36190]: pgmap v280: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 369 KiB/s wr, 52 op/s 2024-05-15T14:09:20.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:20 smithi081 ceph-mon[36434]: pgmap v280: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 369 KiB/s wr, 52 op/s 2024-05-15T14:09:20.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:20 smithi033 ceph-mon[31650]: pgmap v280: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 369 KiB/s wr, 52 op/s 2024-05-15T14:09:21.073 INFO:tasks.workunit.client.0.smithi033.stdout: 1 516383 5.56 MB/sec execute 296 sec latency 13747.221 ms 2024-05-15T14:09:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.420+0000 7f890c6d8700 1 -- 172.21.15.33:0/959768915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8904102e00 msgr2=0x7f890410f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.420+0000 7f890c6d8700 1 --2- 172.21.15.33:0/959768915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8904102e00 0x7f890410f2f0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f88fc00b0a0 tx=0x7f88fc00b3b0 comp rx=0 tx=0).stop 2024-05-15T14:09:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.420+0000 7f890c6d8700 1 -- 172.21.15.33:0/959768915 shutdown_connections 2024-05-15T14:09:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.420+0000 7f890c6d8700 1 --2- 172.21.15.33:0/959768915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8904102e00 0x7f890410f2f0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.420+0000 7f890c6d8700 1 --2- 172.21.15.33:0/959768915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8904108520 0x7f8904108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.420+0000 7f890c6d8700 1 --2- 172.21.15.33:0/959768915 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8904102440 0x7f89041028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.420+0000 7f890c6d8700 1 -- 172.21.15.33:0/959768915 >> 172.21.15.33:0/959768915 conn(0x7f89040793f0 msgr2=0x7f8904079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.420+0000 7f890c6d8700 1 -- 172.21.15.33:0/959768915 shutdown_connections 2024-05-15T14:09:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.420+0000 7f890c6d8700 1 -- 172.21.15.33:0/959768915 wait complete. 2024-05-15T14:09:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.421+0000 7f890c6d8700 1 Processor -- start 2024-05-15T14:09:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.421+0000 7f890c6d8700 1 -- start start 2024-05-15T14:09:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.421+0000 7f890c6d8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8904102440 0x7f890419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.421+0000 7f890c6d8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8904102e00 0x7f890419fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.421+0000 7f890c6d8700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8904108520 0x7f8904199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.421+0000 7f890c6d8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8904113720 con 0x7f8904102440 2024-05-15T14:09:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.421+0000 7f890c6d8700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f89041135a0 con 0x7f8904102e00 2024-05-15T14:09:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.421+0000 7f890c6d8700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f89041138a0 con 0x7f8904108520 2024-05-15T14:09:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.421+0000 7f890a474700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8904102440 0x7f890419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.421+0000 7f890a474700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8904102440 0x7f890419f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39760/0 (socket says 172.21.15.33:39760) 2024-05-15T14:09:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.421+0000 7f890a474700 1 -- 172.21.15.33:0/3067644467 learned_addr learned my addr 172.21.15.33:0/3067644467 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:09:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.422+0000 7f890ac75700 1 --2- 172.21.15.33:0/3067644467 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8904108520 0x7f8904199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.422+0000 7f8909c73700 1 --2- 172.21.15.33:0/3067644467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8904102e00 0x7f890419fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.422+0000 7f890a474700 1 -- 172.21.15.33:0/3067644467 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8904108520 msgr2=0x7f8904199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.422+0000 7f890a474700 1 --2- 172.21.15.33:0/3067644467 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8904108520 0x7f8904199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.422+0000 7f890a474700 1 -- 172.21.15.33:0/3067644467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8904102e00 msgr2=0x7f890419fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:21.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.422+0000 7f890a474700 1 --2- 172.21.15.33:0/3067644467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8904102e00 0x7f890419fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:21.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.422+0000 7f890a474700 1 -- 172.21.15.33:0/3067644467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f890419a0e0 con 0x7f8904102440 2024-05-15T14:09:21.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.422+0000 7f890a474700 1 --2- 172.21.15.33:0/3067644467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8904102440 0x7f890419f770 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f890000c9b0 tx=0x7f890000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:21.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.422+0000 7f88fb7fe700 1 -- 172.21.15.33:0/3067644467 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8900007b00 con 0x7f8904102440 2024-05-15T14:09:21.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.422+0000 7f890c6d8700 1 -- 172.21.15.33:0/3067644467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f890419a3d0 con 0x7f8904102440 2024-05-15T14:09:21.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.422+0000 7f88fb7fe700 1 -- 172.21.15.33:0/3067644467 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f890000ce50 con 0x7f8904102440 2024-05-15T14:09:21.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.422+0000 7f88fb7fe700 1 -- 172.21.15.33:0/3067644467 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8900018660 con 0x7f8904102440 2024-05-15T14:09:21.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.422+0000 7f890c6d8700 1 -- 172.21.15.33:0/3067644467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89041a6640 con 0x7f8904102440 2024-05-15T14:09:21.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.423+0000 7f890c6d8700 1 -- 172.21.15.33:0/3067644467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f890406a230 con 0x7f8904102440 2024-05-15T14:09:21.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.425+0000 7f88fb7fe700 1 -- 172.21.15.33:0/3067644467 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f890000f450 con 0x7f8904102440 2024-05-15T14:09:21.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.425+0000 7f88fb7fe700 1 --2- 172.21.15.33:0/3067644467 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f88f00b29f0 0x7f88f00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:21.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.425+0000 7f8909c73700 1 --2- 172.21.15.33:0/3067644467 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f88f00b29f0 0x7f88f00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:21.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.425+0000 7f88fb7fe700 1 -- 172.21.15.33:0/3067644467 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f89000e2770 con 0x7f8904102440 2024-05-15T14:09:21.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.426+0000 7f8909c73700 1 --2- 172.21.15.33:0/3067644467 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f88f00b29f0 0x7f88f00b4eb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8904079980 tx=0x7f88f400b3c0 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.429+0000 7f88fb7fe700 1 -- 172.21.15.33:0/3067644467 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8900014030 con 0x7f8904102440 2024-05-15T14:09:21.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.701+0000 7f890c6d8700 1 -- 172.21.15.33:0/3067644467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f890419ae90 con 0x7f8904102440 2024-05-15T14:09:21.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.701+0000 7f88fb7fe700 1 -- 172.21.15.33:0/3067644467 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11862 (secure 0 0 0) 0x7f890419ae90 con 0x7f8904102440 2024-05-15T14:09:21.705 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T14:09:21.706 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2030117633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2030117633},{"type":"v1","addr":"172.21.15.33:6839","nonce":2030117633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1918979061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1918979061},{"type":"v1","addr":"172.21.15.33:6841","nonce":1918979061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T13:57:19.056417+0000","modified":"2024-05-15T14:08:23.686464+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24463,"mds_2":24461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/1629697841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1629697841},{"type":"v1","addr":"172.21.15.52:6837","nonce":1629697841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.52:6839/2696044471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2696044471},{"type":"v1","addr":"172.21.15.52:6839","nonce":2696044471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.81:6839/1554395940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":1554395940},{"type":"v1","addr":"172.21.15.81:6839","nonce":1554395940}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25915":{"gid":25915,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/1297849376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1297849376},{"type":"v1","addr":"172.21.15.81:6835","nonce":1297849376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24461,24463,24478]},"id":1}]} 2024-05-15T14:09:21.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.704+0000 7f890c6d8700 1 -- 172.21.15.33:0/3067644467 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f88f00b29f0 msgr2=0x7f88f00b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:21.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.704+0000 7f890c6d8700 1 --2- 172.21.15.33:0/3067644467 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f88f00b29f0 0x7f88f00b4eb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8904079980 tx=0x7f88f400b3c0 comp rx=0 tx=0).stop 2024-05-15T14:09:21.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.704+0000 7f890c6d8700 1 -- 172.21.15.33:0/3067644467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8904102440 msgr2=0x7f890419f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:21.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.704+0000 7f890c6d8700 1 --2- 172.21.15.33:0/3067644467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8904102440 0x7f890419f770 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f890000c9b0 tx=0x7f890000ccc0 comp rx=0 tx=0).stop 2024-05-15T14:09:21.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.704+0000 7f890c6d8700 1 -- 172.21.15.33:0/3067644467 shutdown_connections 2024-05-15T14:09:21.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.704+0000 7f890c6d8700 1 --2- 172.21.15.33:0/3067644467 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f88f00b29f0 0x7f88f00b4eb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:21.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.704+0000 7f890c6d8700 1 --2- 172.21.15.33:0/3067644467 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8904108520 0x7f8904199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:21.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.704+0000 7f890c6d8700 1 --2- 172.21.15.33:0/3067644467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8904102440 0x7f890419f770 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:21.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.704+0000 7f890c6d8700 1 --2- 172.21.15.33:0/3067644467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8904102e00 0x7f890419fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:21.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.704+0000 7f890c6d8700 1 -- 172.21.15.33:0/3067644467 >> 172.21.15.33:0/3067644467 conn(0x7f89040793f0 msgr2=0x7f8904100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:21.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.704+0000 7f890c6d8700 1 -- 172.21.15.33:0/3067644467 shutdown_connections 2024-05-15T14:09:21.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:21.704+0000 7f890c6d8700 1 -- 172.21.15.33:0/3067644467 wait complete. 2024-05-15T14:09:21.710 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 34 2024-05-15T14:09:22.073 INFO:tasks.workunit.client.0.smithi033.stdout: 1 516383 5.54 MB/sec execute 297 sec latency 14747.319 ms 2024-05-15T14:09:22.351 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24478 quiesce db --set-id ad5b83c1 --release --await' 2024-05-15T14:09:22.351 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24478 quiesce db --set-id ad5b83c1 --release --await 2024-05-15T14:09:22.606 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:09:22.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:22 smithi033 ceph-mon[31650]: pgmap v281: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 342 KiB/s wr, 49 op/s 2024-05-15T14:09:22.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:22 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/3067644467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:09:22.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:22 smithi052 ceph-mon[36190]: pgmap v281: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 342 KiB/s wr, 49 op/s 2024-05-15T14:09:22.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:22 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/3067644467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:09:22.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:22 smithi081 ceph-mon[36434]: pgmap v281: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 342 KiB/s wr, 49 op/s 2024-05-15T14:09:22.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:22 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/3067644467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:09:23.073 INFO:tasks.workunit.client.0.smithi033.stdout: 1 516383 5.53 MB/sec execute 298 sec latency 15747.416 ms 2024-05-15T14:09:24.073 INFO:tasks.workunit.client.0.smithi033.stdout: 1 516383 5.51 MB/sec execute 299 sec latency 16747.455 ms 2024-05-15T14:09:24.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.310+0000 7fb40aa76700 1 -- 172.21.15.33:0/2626007425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb40406bbe0 msgr2=0x7fb4041103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:24.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.310+0000 7fb40aa76700 1 --2- 172.21.15.33:0/2626007425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb40406bbe0 0x7fb4041103d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc009990 tx=0x7fb3fc009ca0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.310+0000 7fb40aa76700 1 -- 172.21.15.33:0/2626007425 shutdown_connections 2024-05-15T14:09:24.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.310+0000 7fb40aa76700 1 --2- 172.21.15.33:0/2626007425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb40406bbe0 0x7fb4041103d0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.310+0000 7fb40aa76700 1 --2- 172.21.15.33:0/2626007425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4041095b0 0x7fb404109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.310+0000 7fb40aa76700 1 --2- 172.21.15.33:0/2626007425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb40406b220 0x7fb40406b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.310+0000 7fb40aa76700 1 -- 172.21.15.33:0/2626007425 >> 172.21.15.33:0/2626007425 conn(0x7fb4040793f0 msgr2=0x7fb404079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:24.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.310+0000 7fb40aa76700 1 -- 172.21.15.33:0/2626007425 shutdown_connections 2024-05-15T14:09:24.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.310+0000 7fb40aa76700 1 -- 172.21.15.33:0/2626007425 wait complete. 2024-05-15T14:09:24.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.311+0000 7fb40aa76700 1 Processor -- start 2024-05-15T14:09:24.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.311+0000 7fb40aa76700 1 -- start start 2024-05-15T14:09:24.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.311+0000 7fb40aa76700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb40406b220 0x7fb40419f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.311+0000 7fb40aa76700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb40406bbe0 0x7fb40419fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.311+0000 7fb40aa76700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4041095b0 0x7fb404199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.311+0000 7fb40aa76700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb4041136e0 con 0x7fb40406bbe0 2024-05-15T14:09:24.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.311+0000 7fb40aa76700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb404113560 con 0x7fb4041095b0 2024-05-15T14:09:24.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.311+0000 7fb40aa76700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb404113860 con 0x7fb40406b220 2024-05-15T14:09:24.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.312+0000 7fb403fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb40406bbe0 0x7fb40419fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:24.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.312+0000 7fb403fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb40406bbe0 0x7fb40419fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40424/0 (socket says 172.21.15.33:40424) 2024-05-15T14:09:24.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.312+0000 7fb403fff700 1 -- 172.21.15.33:0/4026961431 learned_addr learned my addr 172.21.15.33:0/4026961431 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:09:24.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.312+0000 7fb408812700 1 --2- 172.21.15.33:0/4026961431 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb40406b220 0x7fb40419f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:24.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.312+0000 7fb409013700 1 --2- 172.21.15.33:0/4026961431 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4041095b0 0x7fb404199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:24.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.312+0000 7fb403fff700 1 -- 172.21.15.33:0/4026961431 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb40406b220 msgr2=0x7fb40419f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:24.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.312+0000 7fb403fff700 1 --2- 172.21.15.33:0/4026961431 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb40406b220 0x7fb40419f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.312+0000 7fb403fff700 1 -- 172.21.15.33:0/4026961431 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4041095b0 msgr2=0x7fb404199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:24.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.312+0000 7fb403fff700 1 --2- 172.21.15.33:0/4026961431 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4041095b0 0x7fb404199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.312+0000 7fb403fff700 1 -- 172.21.15.33:0/4026961431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40419a1a0 con 0x7fb40406bbe0 2024-05-15T14:09:24.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.312+0000 7fb403fff700 1 --2- 172.21.15.33:0/4026961431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb40406bbe0 0x7fb40419fdc0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec00d950 tx=0x7fb3ec00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:24.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.312+0000 7fb401ffb700 1 -- 172.21.15.33:0/4026961431 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec016070 con 0x7fb40406bbe0 2024-05-15T14:09:24.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.312+0000 7fb40aa76700 1 -- 172.21.15.33:0/4026961431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb40419a490 con 0x7fb40406bbe0 2024-05-15T14:09:24.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.312+0000 7fb401ffb700 1 -- 172.21.15.33:0/4026961431 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3ec00fd40 con 0x7fb40406bbe0 2024-05-15T14:09:24.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.313+0000 7fb401ffb700 1 -- 172.21.15.33:0/4026961431 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec00e3b0 con 0x7fb40406bbe0 2024-05-15T14:09:24.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.313+0000 7fb40aa76700 1 -- 172.21.15.33:0/4026961431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4041a6750 con 0x7fb40406bbe0 2024-05-15T14:09:24.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.316+0000 7fb401ffb700 1 -- 172.21.15.33:0/4026961431 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb3ec019070 con 0x7fb40406bbe0 2024-05-15T14:09:24.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.317+0000 7fb401ffb700 1 --2- 172.21.15.33:0/4026961431 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fb3f00b29f0 0x7fb3f00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.317+0000 7fb408812700 1 --2- 172.21.15.33:0/4026961431 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fb3f00b29f0 0x7fb3f00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:24.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.317+0000 7fb401ffb700 1 -- 172.21.15.33:0/4026961431 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fb3ec0e2180 con 0x7fb40406bbe0 2024-05-15T14:09:24.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.317+0000 7fb41530cb80 1 Processor -- start 2024-05-15T14:09:24.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.317+0000 7fb408812700 1 --2- 172.21.15.33:0/4026961431 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fb3f00b29f0 0x7fb3f00b4eb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb40419ab80 tx=0x7fb3f4009180 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:24.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.317+0000 7fb41530cb80 1 -- start start 2024-05-15T14:09:24.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.317+0000 7fb41530cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbab66da0 0x564fbab67180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.317+0000 7fb41530cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564fbab676c0 0x564fbab614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.317+0000 7fb41530cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fbab61ad0 0x564fbab61f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.317+0000 7fb41530cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564fbaabfd90 con 0x564fbab61ad0 2024-05-15T14:09:24.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.317+0000 7fb41530cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564fbaaaf930 con 0x564fbab66da0 2024-05-15T14:09:24.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.317+0000 7fb41530cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x564fbaab3970 con 0x564fbab676c0 2024-05-15T14:09:24.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.317+0000 7fb409013700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fbab61ad0 0x564fbab61f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:24.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.317+0000 7fb409013700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fbab61ad0 0x564fbab61f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40434/0 (socket says 172.21.15.33:40434) 2024-05-15T14:09:24.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.317+0000 7fb409013700 1 -- 172.21.15.33:0/3938776489 learned_addr learned my addr 172.21.15.33:0/3938776489 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:09:24.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.317+0000 7fb408812700 1 --2- 172.21.15.33:0/3938776489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbab66da0 0x564fbab67180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:24.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.317+0000 7fb403fff700 1 --2- 172.21.15.33:0/3938776489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564fbab676c0 0x564fbab614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:24.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.317+0000 7fb409013700 1 -- 172.21.15.33:0/3938776489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564fbab676c0 msgr2=0x564fbab614b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:24.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.317+0000 7fb409013700 1 --2- 172.21.15.33:0/3938776489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564fbab676c0 0x564fbab614b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.317+0000 7fb409013700 1 -- 172.21.15.33:0/3938776489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbab66da0 msgr2=0x564fbab67180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:24.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.317+0000 7fb409013700 1 --2- 172.21.15.33:0/3938776489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbab66da0 0x564fbab67180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.317+0000 7fb409013700 1 -- 172.21.15.33:0/3938776489 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fba9ff9d0 con 0x564fbab61ad0 2024-05-15T14:09:24.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.317+0000 7fb408812700 1 --2- 172.21.15.33:0/3938776489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbab66da0 0x564fbab67180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:09:24.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.317+0000 7fb409013700 1 --2- 172.21.15.33:0/3938776489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fbab61ad0 0x564fbab61f50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc015000 tx=0x7fb3fc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5e61617520dea9 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:24.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.317+0000 7fb3faffd700 1 -- 172.21.15.33:0/3938776489 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc003a60 con 0x564fbab61ad0 2024-05-15T14:09:24.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.317+0000 7fb3faffd700 1 -- 172.21.15.33:0/3938776489 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3fc026070 con 0x564fbab61ad0 2024-05-15T14:09:24.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.317+0000 7fb3faffd700 1 -- 172.21.15.33:0/3938776489 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc00bac0 con 0x564fbab61ad0 2024-05-15T14:09:24.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.318+0000 7fb41530cb80 1 -- 172.21.15.33:0/3938776489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fbab61ad0 msgr2=0x564fbab61f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:24.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.318+0000 7fb41530cb80 1 --2- 172.21.15.33:0/3938776489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fbab61ad0 0x564fbab61f50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc015000 tx=0x7fb3fc009f90 comp rx=0 tx=0).stop 2024-05-15T14:09:24.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.318+0000 7fb41530cb80 1 -- 172.21.15.33:0/3938776489 shutdown_connections 2024-05-15T14:09:24.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.318+0000 7fb41530cb80 1 --2- 172.21.15.33:0/3938776489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fbab61ad0 0x564fbab61f50 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.318+0000 7fb41530cb80 1 --2- 172.21.15.33:0/3938776489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbab66da0 0x564fbab67180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.318+0000 7fb41530cb80 1 --2- 172.21.15.33:0/3938776489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564fbab676c0 0x564fbab614b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.318+0000 7fb41530cb80 1 -- 172.21.15.33:0/3938776489 >> 172.21.15.33:0/3938776489 conn(0x564fbaa8c350 msgr2=0x564fbab5bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:24.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.318+0000 7fb41530cb80 1 -- 172.21.15.33:0/3938776489 shutdown_connections 2024-05-15T14:09:24.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.318+0000 7fb41530cb80 1 -- 172.21.15.33:0/3938776489 wait complete. 2024-05-15T14:09:24.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.319+0000 7fb41530cb80 1 -- start start 2024-05-15T14:09:24.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.319+0000 7fb41530cb80 1 Processor -- start 2024-05-15T14:09:24.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.319+0000 7fb41530cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbab61ad0 0x564fbab6d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.319+0000 7fb41530cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fbab66da0 0x564fbab6a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.319+0000 7fb41530cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564fbab676c0 0x564fbab6ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.319+0000 7fb41530cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564fba9b60a0 con 0x564fbab66da0 2024-05-15T14:09:24.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.319+0000 7fb41530cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564fbaa518c0 con 0x564fbab61ad0 2024-05-15T14:09:24.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.319+0000 7fb41530cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x564fbab6e0a0 con 0x564fbab676c0 2024-05-15T14:09:24.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.319+0000 7fb41530cb80 10 client.0 _ll_register_callbacks cb 0x564fbaa46680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:09:24.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.319+0000 7fb403fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fbab66da0 0x564fbab6a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:24.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.319+0000 7fb403fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fbab66da0 0x564fbab6a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40440/0 (socket says 172.21.15.33:40440) 2024-05-15T14:09:24.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.319+0000 7fb403fff700 1 -- 172.21.15.33:0/653388084 learned_addr learned my addr 172.21.15.33:0/653388084 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:09:24.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.319+0000 7fb409013700 1 --2- 172.21.15.33:0/653388084 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564fbab676c0 0x564fbab6ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:24.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.319+0000 7fb408812700 1 --2- 172.21.15.33:0/653388084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbab61ad0 0x564fbab6d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:24.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.320+0000 7fb403fff700 1 -- 172.21.15.33:0/653388084 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564fbab676c0 msgr2=0x564fbab6ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:24.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.320+0000 7fb403fff700 1 --2- 172.21.15.33:0/653388084 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564fbab676c0 0x564fbab6ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.320+0000 7fb403fff700 1 -- 172.21.15.33:0/653388084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbab61ad0 msgr2=0x564fbab6d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:24.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.320+0000 7fb403fff700 1 --2- 172.21.15.33:0/653388084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbab61ad0 0x564fbab6d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.320+0000 7fb403fff700 1 -- 172.21.15.33:0/653388084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fbaa4e790 con 0x564fbab66da0 2024-05-15T14:09:24.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.320+0000 7fb409013700 1 --2- 172.21.15.33:0/653388084 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564fbab676c0 0x564fbab6ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:09:24.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.320+0000 7fb408812700 1 --2- 172.21.15.33:0/653388084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbab61ad0 0x564fbab6d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:09:24.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.320+0000 7fb403fff700 1 --2- 172.21.15.33:0/653388084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fbab66da0 0x564fbab6a720 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec0a6660 tx=0x7fb3ec0a9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=669925c8069bf54c server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:24.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.320+0000 7fb3e3fff700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:09:24.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.320+0000 7fb3e3fff700 1 -- 172.21.15.33:0/653388084 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec020040 con 0x564fbab66da0 2024-05-15T14:09:24.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.320+0000 7fb3e3fff700 1 -- 172.21.15.33:0/653388084 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fb3ec0a9650 con 0x564fbab66da0 2024-05-15T14:09:24.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.320+0000 7fb41530cb80 1 -- 172.21.15.33:0/653388084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3fc026070 con 0x564fbab66da0 2024-05-15T14:09:24.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.320+0000 7fb3e3fff700 1 -- 172.21.15.33:0/653388084 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb3ec0ad7a0 con 0x564fbab66da0 2024-05-15T14:09:24.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.320+0000 7fb41530cb80 10 client.16443 fetch_fsmap learned FSMap version 34 2024-05-15T14:09:24.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.320+0000 7fb41530cb80 1 -- 172.21.15.33:0/653388084 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564fbab78720 con 0x564fbab66da0 2024-05-15T14:09:24.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.320+0000 7fb3e3fff700 1 -- 172.21.15.33:0/653388084 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3fc026070 con 0x564fbab66da0 2024-05-15T14:09:24.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.320+0000 7fb3e3fff700 1 -- 172.21.15.33:0/653388084 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fb3ec0a8650 con 0x564fbab66da0 2024-05-15T14:09:24.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.321+0000 7fb41530cb80 10 client.16443 fetch_fsmap finished waiting for FSMap version 34 2024-05-15T14:09:24.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.321+0000 7fb41530cb80 10 client.16443 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:09:24.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.321+0000 7fb41530cb80 20 client.16443 populate_metadata read hostname 'smithi033' 2024-05-15T14:09:24.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.321+0000 7fb41530cb80 1 --2- 172.21.15.33:0/653388084 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x564fbab7ad60 0x564fbab7d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.321+0000 7fb41530cb80 4 client.16443 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:09:24.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.321+0000 7fb41530cb80 1 -- 172.21.15.33:0/653388084 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fbab2d460 con 0x564fbab7ad60 2024-05-15T14:09:24.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.321+0000 7fb408812700 1 --2- 172.21.15.33:0/653388084 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x564fbab7ad60 0x564fbab7d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:24.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.321+0000 7fb3e3fff700 10 client.16443 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:09:24.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.321+0000 7fb408812700 1 --2- 172.21.15.33:0/653388084 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x564fbab7ad60 0x564fbab7d150 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fa272f045c7b3b7 server_cookie=96c074110f08a617 in_seq=0 out_seq=0 2024-05-15T14:09:24.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.322+0000 7fb3e3fff700 1 -- 172.21.15.33:0/653388084 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564fbab2d460 con 0x564fbab7ad60 2024-05-15T14:09:24.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.322+0000 7fb3e3fff700 10 client.16443 handle_command_reply: tid=0 2024-05-15T14:09:24.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.322+0000 7fb41530cb80 1 client.16443 shutdown 2024-05-15T14:09:24.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.322+0000 7fb41530cb80 1 -- 172.21.15.33:0/653388084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fbab66da0 msgr2=0x564fbab6a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:24.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.322+0000 7fb41530cb80 1 --2- 172.21.15.33:0/653388084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fbab66da0 0x564fbab6a720 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec0a6660 tx=0x7fb3ec0a9040 comp rx=0 tx=0).stop 2024-05-15T14:09:24.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.322+0000 7fb41530cb80 1 -- 172.21.15.33:0/653388084 shutdown_connections 2024-05-15T14:09:24.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.322+0000 7fb41530cb80 1 --2- 172.21.15.33:0/653388084 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564fbab676c0 0x564fbab6ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.322+0000 7fb41530cb80 1 --2- 172.21.15.33:0/653388084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbab61ad0 0x564fbab6d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.322+0000 7fb41530cb80 1 --2- 172.21.15.33:0/653388084 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x564fbab7ad60 0x564fbab7d150 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.322+0000 7fb41530cb80 1 --2- 172.21.15.33:0/653388084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fbab66da0 0x564fbab6a720 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.322+0000 7fb3e3fff700 0 client.16443 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:09:24.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.322+0000 7fb41530cb80 1 -- 172.21.15.33:0/653388084 >> 172.21.15.33:0/653388084 conn(0x564fbaa8c350 msgr2=0x564fbab567d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:24.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.322+0000 7fb41530cb80 1 -- 172.21.15.33:0/653388084 shutdown_connections 2024-05-15T14:09:24.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.323+0000 7fb41530cb80 1 -- 172.21.15.33:0/653388084 wait complete. 2024-05-15T14:09:24.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.323+0000 7fb41530cb80 20 client.16443 trim_cache size 0 max 16384 2024-05-15T14:09:24.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.343+0000 7fb41530cb80 1 Processor -- start 2024-05-15T14:09:24.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.344+0000 7fb41530cb80 1 -- start start 2024-05-15T14:09:24.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.344+0000 7fb41530cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbab61ad0 0x564fbab68350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.344+0000 7fb41530cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fbab66da0 0x564fbab76fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.344+0000 7fb41530cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564fbab676c0 0x564fbab568e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.344+0000 7fb41530cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb4041b0670 con 0x564fbab66da0 2024-05-15T14:09:24.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.344+0000 7fb41530cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb4041ade70 con 0x564fbab61ad0 2024-05-15T14:09:24.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.344+0000 7fb41530cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb4041abce0 con 0x564fbab676c0 2024-05-15T14:09:24.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.344+0000 7fb403fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fbab66da0 0x564fbab76fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:24.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.344+0000 7fb403fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fbab66da0 0x564fbab76fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40452/0 (socket says 172.21.15.33:40452) 2024-05-15T14:09:24.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.344+0000 7fb403fff700 1 -- 172.21.15.33:0/2471197322 learned_addr learned my addr 172.21.15.33:0/2471197322 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:09:24.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.344+0000 7fb408812700 1 --2- 172.21.15.33:0/2471197322 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbab61ad0 0x564fbab68350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:24.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.344+0000 7fb403fff700 1 -- 172.21.15.33:0/2471197322 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564fbab676c0 msgr2=0x564fbab568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:24.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.344+0000 7fb403fff700 1 --2- 172.21.15.33:0/2471197322 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564fbab676c0 0x564fbab568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.344+0000 7fb403fff700 1 -- 172.21.15.33:0/2471197322 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbab61ad0 msgr2=0x564fbab68350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:24.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.344+0000 7fb403fff700 1 --2- 172.21.15.33:0/2471197322 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbab61ad0 0x564fbab68350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.344+0000 7fb403fff700 1 -- 172.21.15.33:0/2471197322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fbab5ca00 con 0x564fbab66da0 2024-05-15T14:09:24.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.344+0000 7fb408812700 1 --2- 172.21.15.33:0/2471197322 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbab61ad0 0x564fbab68350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:09:24.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.344+0000 7fb403fff700 1 --2- 172.21.15.33:0/2471197322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fbab66da0 0x564fbab76fb0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec00b2f0 tx=0x7fb3ec0a53e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7109f5ffc5c4d832 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:24.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.346+0000 7fb3fa7fc700 1 -- 172.21.15.33:0/2471197322 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec020050 con 0x564fbab66da0 2024-05-15T14:09:24.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.346+0000 7fb3fa7fc700 1 -- 172.21.15.33:0/2471197322 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3ec0a59e0 con 0x564fbab66da0 2024-05-15T14:09:24.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.346+0000 7fb3fa7fc700 1 -- 172.21.15.33:0/2471197322 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec0b2020 con 0x564fbab66da0 2024-05-15T14:09:24.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.346+0000 7fb41530cb80 1 -- 172.21.15.33:0/2471197322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fbab66da0 msgr2=0x564fbab76fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:24.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.346+0000 7fb41530cb80 1 --2- 172.21.15.33:0/2471197322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fbab66da0 0x564fbab76fb0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec00b2f0 tx=0x7fb3ec0a53e0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.346+0000 7fb41530cb80 1 -- 172.21.15.33:0/2471197322 shutdown_connections 2024-05-15T14:09:24.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.346+0000 7fb41530cb80 1 --2- 172.21.15.33:0/2471197322 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564fbab676c0 0x564fbab568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.346+0000 7fb41530cb80 1 --2- 172.21.15.33:0/2471197322 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbab61ad0 0x564fbab68350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.346+0000 7fb41530cb80 1 --2- 172.21.15.33:0/2471197322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fbab66da0 0x564fbab76fb0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.346+0000 7fb41530cb80 1 -- 172.21.15.33:0/2471197322 >> 172.21.15.33:0/2471197322 conn(0x564fbaa8c350 msgr2=0x564fbab658e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:24.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.346+0000 7fb41530cb80 1 -- 172.21.15.33:0/2471197322 shutdown_connections 2024-05-15T14:09:24.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.346+0000 7fb41530cb80 1 -- 172.21.15.33:0/2471197322 wait complete. 2024-05-15T14:09:24.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.346+0000 7fb41530cb80 1 -- start start 2024-05-15T14:09:24.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.346+0000 7fb41530cb80 1 Processor -- start 2024-05-15T14:09:24.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.346+0000 7fb41530cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbab61ad0 0x564fbab70300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.346+0000 7fb41530cb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564fbab66da0 0x564fbab70840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.346+0000 7fb41530cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fbab676c0 0x564fbab70d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.346+0000 7fb41530cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb4041ac330 con 0x564fbab676c0 2024-05-15T14:09:24.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.346+0000 7fb41530cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb4041ad7a0 con 0x564fbab61ad0 2024-05-15T14:09:24.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.347+0000 7fb41530cb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb4041aca00 con 0x564fbab66da0 2024-05-15T14:09:24.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.347+0000 7fb41530cb80 10 client.0 _ll_register_callbacks cb 0x564fbaa46680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:09:24.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.347+0000 7fb408812700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbab61ad0 0x564fbab70300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:24.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.347+0000 7fb409013700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fbab676c0 0x564fbab70d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:24.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.347+0000 7fb408812700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbab61ad0 0x564fbab70300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:53484/0 (socket says 172.21.15.33:53484) 2024-05-15T14:09:24.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.347+0000 7fb408812700 1 -- 172.21.15.33:0/351503014 learned_addr learned my addr 172.21.15.33:0/351503014 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:09:24.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.347+0000 7fb403fff700 1 --2- 172.21.15.33:0/351503014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564fbab66da0 0x564fbab70840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.347+0000 7fb408812700 1 -- 172.21.15.33:0/351503014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564fbab66da0 msgr2=0x564fbab70840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.347+0000 7fb408812700 1 --2- 172.21.15.33:0/351503014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564fbab66da0 0x564fbab70840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.347+0000 7fb408812700 1 -- 172.21.15.33:0/351503014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fbab676c0 msgr2=0x564fbab70d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.347+0000 7fb408812700 1 --2- 172.21.15.33:0/351503014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fbab676c0 0x564fbab70d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.347+0000 7fb408812700 1 -- 172.21.15.33:0/351503014 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fbab673d0 con 0x564fbab61ad0 2024-05-15T14:09:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.347+0000 7fb403fff700 1 --2- 172.21.15.33:0/351503014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564fbab66da0 0x564fbab70840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:09:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.347+0000 7fb409013700 1 --2- 172.21.15.33:0/351503014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fbab676c0 0x564fbab70d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:09:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.347+0000 7fb408812700 1 --2- 172.21.15.33:0/351503014 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbab61ad0 0x564fbab70300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4004330 tx=0x7fb3f400b050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=596bf6aa186fad76 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.347+0000 7fb3e3fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-05-15T14:09:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.347+0000 7fb3e3fff700 1 -- 172.21.15.33:0/351503014 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4015a30 con 0x564fbab61ad0 2024-05-15T14:09:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.347+0000 7fb41530cb80 1 -- 172.21.15.33:0/351503014 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3ec0a59e0 con 0x564fbab61ad0 2024-05-15T14:09:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.347+0000 7fb3e3fff700 1 -- 172.21.15.33:0/351503014 <== mon.2 v2:172.21.15.52:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fb3f4015bd0 con 0x564fbab61ad0 2024-05-15T14:09:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.348+0000 7fb3e3fff700 1 -- 172.21.15.33:0/351503014 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb3f4015dd0 con 0x564fbab61ad0 2024-05-15T14:09:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.348+0000 7fb41530cb80 10 client.25919 fetch_fsmap learned FSMap version 34 2024-05-15T14:09:24.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.348+0000 7fb41530cb80 1 -- 172.21.15.33:0/351503014 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564fbabe6720 con 0x564fbab61ad0 2024-05-15T14:09:24.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.348+0000 7fb3e3fff700 1 -- 172.21.15.33:0/351503014 <== mon.2 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3ec0a59e0 con 0x564fbab61ad0 2024-05-15T14:09:24.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.348+0000 7fb3e3fff700 1 -- 172.21.15.33:0/351503014 <== mon.2 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fb3f4022780 con 0x564fbab61ad0 2024-05-15T14:09:24.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.348+0000 7fb41530cb80 10 client.25919 fetch_fsmap finished waiting for FSMap version 34 2024-05-15T14:09:24.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.348+0000 7fb41530cb80 10 client.25919 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:09:24.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.348+0000 7fb41530cb80 20 client.25919 populate_metadata read hostname 'smithi033' 2024-05-15T14:09:24.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.348+0000 7fb41530cb80 1 --2- 172.21.15.33:0/351503014 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x564fbabe6ec0 0x564fbabe92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.348+0000 7fb41530cb80 4 client.25919 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ad5b83c1", "release": true, "await": true}] 2024-05-15T14:09:24.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.348+0000 7fb41530cb80 1 -- 172.21.15.33:0/351503014 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ad5b83c1", "release": true, "await": true}) -- 0x564fbaac1ce0 con 0x564fbabe6ec0 2024-05-15T14:09:24.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.349+0000 7fb403fff700 1 --2- 172.21.15.33:0/351503014 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x564fbabe6ec0 0x564fbabe92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:24.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.349+0000 7fb3e3fff700 10 client.25919 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:09:24.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.349+0000 7fb403fff700 1 --2- 172.21.15.33:0/351503014 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x564fbabe6ec0 0x564fbabe92e0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23ee6382dfd53360 server_cookie=f3b05521ac8c8c48 in_seq=0 out_seq=0 2024-05-15T14:09:24.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.355+0000 7fb3e3fff700 1 -- 172.21.15.33:0/351503014 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564fbaac1ce0 con 0x564fbabe6ec0 2024-05-15T14:09:24.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.355+0000 7fb3e3fff700 10 client.25919 handle_command_reply: tid=0 2024-05-15T14:09:24.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.355+0000 7fb41530cb80 1 client.25919 shutdown 2024-05-15T14:09:24.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.355+0000 7fb41530cb80 1 -- 172.21.15.33:0/351503014 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbab61ad0 msgr2=0x564fbab70300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:24.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.355+0000 7fb41530cb80 1 --2- 172.21.15.33:0/351503014 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbab61ad0 0x564fbab70300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4004330 tx=0x7fb3f400b050 comp rx=0 tx=0).stop 2024-05-15T14:09:24.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.356+0000 7fb41530cb80 1 -- 172.21.15.33:0/351503014 shutdown_connections 2024-05-15T14:09:24.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.356+0000 7fb41530cb80 1 --2- 172.21.15.33:0/351503014 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x564fbabe6ec0 0x564fbabe92e0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.356+0000 7fb41530cb80 1 --2- 172.21.15.33:0/351503014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fbab676c0 0x564fbab70d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.356+0000 7fb41530cb80 1 --2- 172.21.15.33:0/351503014 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbab61ad0 0x564fbab70300 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.356+0000 7fb41530cb80 1 --2- 172.21.15.33:0/351503014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564fbab66da0 0x564fbab70840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.356+0000 7fb3e3fff700 0 client.25919 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:09:24.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.356+0000 7fb41530cb80 1 -- 172.21.15.33:0/351503014 >> 172.21.15.33:0/351503014 conn(0x564fbaa8c350 msgr2=0x564fbab66170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:24.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.356+0000 7fb41530cb80 1 -- 172.21.15.33:0/351503014 shutdown_connections 2024-05-15T14:09:24.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.356+0000 7fb41530cb80 1 -- 172.21.15.33:0/351503014 wait complete. 2024-05-15T14:09:24.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.356+0000 7fb41530cb80 20 client.25919 trim_cache size 0 max 16384 2024-05-15T14:09:24.360 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T14:09:24.360 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 34, 2024-05-15T14:09:24.360 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24478, 2024-05-15T14:09:24.360 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 32, 2024-05-15T14:09:24.360 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-05-15T14:09:24.360 INFO:teuthology.orchestra.run.smithi033.stdout: "ad5b83c1": { 2024-05-15T14:09:24.360 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 32, 2024-05-15T14:09:24.360 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-05-15T14:09:24.360 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:09:24.360 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-05-15T14:09:24.360 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:09:24.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:09:24.360 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 90.0, 2024-05-15T14:09:24.361 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 138.0, 2024-05-15T14:09:24.361 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-05-15T14:09:24.361 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-05-15T14:09:24.361 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-05-15T14:09:24.361 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:09:24.361 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-05-15T14:09:24.361 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:09:24.361 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:09:24.361 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:09:24.361 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:09:24.361 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:09:24.361 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:09:24.361 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T14:09:24.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.356+0000 7fb40aa76700 1 -- 172.21.15.33:0/4026961431 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fb3f00b29f0 msgr2=0x7fb3f00b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:24.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.356+0000 7fb40aa76700 1 --2- 172.21.15.33:0/4026961431 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fb3f00b29f0 0x7fb3f00b4eb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb40419ab80 tx=0x7fb3f4009180 comp rx=0 tx=0).stop 2024-05-15T14:09:24.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.356+0000 7fb40aa76700 1 -- 172.21.15.33:0/4026961431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb40406bbe0 msgr2=0x7fb40419fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:24.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.356+0000 7fb40aa76700 1 --2- 172.21.15.33:0/4026961431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb40406bbe0 0x7fb40419fdc0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec00d950 tx=0x7fb3ec00dc60 comp rx=0 tx=0).stop 2024-05-15T14:09:24.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.357+0000 7fb40aa76700 1 -- 172.21.15.33:0/4026961431 shutdown_connections 2024-05-15T14:09:24.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.357+0000 7fb40aa76700 1 --2- 172.21.15.33:0/4026961431 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fb3f00b29f0 0x7fb3f00b4eb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.357+0000 7fb40aa76700 1 --2- 172.21.15.33:0/4026961431 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb4041095b0 0x7fb404199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.357+0000 7fb40aa76700 1 --2- 172.21.15.33:0/4026961431 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb40406b220 0x7fb40419f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.357+0000 7fb40aa76700 1 --2- 172.21.15.33:0/4026961431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb40406bbe0 0x7fb40419fdc0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.357+0000 7fb40aa76700 1 -- 172.21.15.33:0/4026961431 >> 172.21.15.33:0/4026961431 conn(0x7fb4040793f0 msgr2=0x7fb404101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:24.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.357+0000 7fb40aa76700 1 -- 172.21.15.33:0/4026961431 shutdown_connections 2024-05-15T14:09:24.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:09:24.357+0000 7fb40aa76700 1 -- 172.21.15.33:0/4026961431 wait complete. 2024-05-15T14:09:24.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.457+0000 7ffae0245640 1 -- 172.21.15.33:0/434662817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffad8101930 msgr2=0x7ffad8101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:24.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.457+0000 7ffae0245640 1 --2- 172.21.15.33:0/434662817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffad8101930 0x7ffad8101db0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ffac8009930 tx=0x7ffac802a420 comp rx=0 tx=0).stop 2024-05-15T14:09:24.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.457+0000 7ffae0245640 1 -- 172.21.15.33:0/434662817 shutdown_connections 2024-05-15T14:09:24.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.457+0000 7ffae0245640 1 --2- 172.21.15.33:0/434662817 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffad81022f0 0x7ffad810e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.457+0000 7ffae0245640 1 --2- 172.21.15.33:0/434662817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffad8101930 0x7ffad8101db0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.457+0000 7ffae0245640 1 --2- 172.21.15.33:0/434662817 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffad8107940 0x7ffad8107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.457+0000 7ffae0245640 1 -- 172.21.15.33:0/434662817 >> 172.21.15.33:0/434662817 conn(0x7ffad8078a70 msgr2=0x7ffad8078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:24.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.457+0000 7ffae0245640 1 -- 172.21.15.33:0/434662817 shutdown_connections 2024-05-15T14:09:24.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.457+0000 7ffae0245640 1 -- 172.21.15.33:0/434662817 wait complete. 2024-05-15T14:09:24.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.458+0000 7ffae0245640 1 Processor -- start 2024-05-15T14:09:24.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.458+0000 7ffae0245640 1 -- start start 2024-05-15T14:09:24.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.458+0000 7ffae0245640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffad8101930 0x7ffad8199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.458+0000 7ffae0245640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffad81022f0 0x7ffad81997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.458+0000 7ffae0245640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffad8107940 0x7ffad819db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.458+0000 7ffae0245640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffad8111c50 con 0x7ffad81022f0 2024-05-15T14:09:24.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.458+0000 7ffae0245640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffad8111ad0 con 0x7ffad8107940 2024-05-15T14:09:24.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.458+0000 7ffae0245640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ffad8111dd0 con 0x7ffad8101930 2024-05-15T14:09:24.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.458+0000 7ffadd7e0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffad81022f0 0x7ffad81997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:24.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.458+0000 7ffadd7e0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffad81022f0 0x7ffad81997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40476/0 (socket says 172.21.15.33:40476) 2024-05-15T14:09:24.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.458+0000 7ffadd7e0640 1 -- 172.21.15.33:0/2951233361 learned_addr learned my addr 172.21.15.33:0/2951233361 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:09:24.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.458+0000 7ffaddfe1640 1 --2- 172.21.15.33:0/2951233361 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffad8101930 0x7ffad8199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:24.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.458+0000 7ffade7e2640 1 --2- 172.21.15.33:0/2951233361 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffad8107940 0x7ffad819db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:24.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.458+0000 7ffadd7e0640 1 -- 172.21.15.33:0/2951233361 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffad8101930 msgr2=0x7ffad8199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:24.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.458+0000 7ffadd7e0640 1 --2- 172.21.15.33:0/2951233361 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffad8101930 0x7ffad8199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.458+0000 7ffadd7e0640 1 -- 172.21.15.33:0/2951233361 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffad8107940 msgr2=0x7ffad819db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.458+0000 7ffadd7e0640 1 --2- 172.21.15.33:0/2951233361 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffad8107940 0x7ffad819db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.458+0000 7ffadd7e0640 1 -- 172.21.15.33:0/2951233361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad819e250 con 0x7ffad81022f0 2024-05-15T14:09:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.458+0000 7ffade7e2640 1 --2- 172.21.15.33:0/2951233361 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffad8107940 0x7ffad819db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:09:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.459+0000 7ffadd7e0640 1 --2- 172.21.15.33:0/2951233361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffad81022f0 0x7ffad81997a0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ffac8005ab0 tx=0x7ffac8003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.459+0000 7ffaceffd640 1 -- 172.21.15.33:0/2951233361 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac803f070 con 0x7ffad81022f0 2024-05-15T14:09:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.459+0000 7ffaceffd640 1 -- 172.21.15.33:0/2951233361 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffac8031b50 con 0x7ffad81022f0 2024-05-15T14:09:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.459+0000 7ffae0245640 1 -- 172.21.15.33:0/2951233361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad819e4e0 con 0x7ffad81022f0 2024-05-15T14:09:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.459+0000 7ffaceffd640 1 -- 172.21.15.33:0/2951233361 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac8035030 con 0x7ffad81022f0 2024-05-15T14:09:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.459+0000 7ffae0245640 1 -- 172.21.15.33:0/2951233361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad81a5dd0 con 0x7ffad81022f0 2024-05-15T14:09:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.462+0000 7ffae3036740 1 Processor -- start 2024-05-15T14:09:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.462+0000 7ffae3036740 1 -- start start 2024-05-15T14:09:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.462+0000 7ffaceffd640 1 -- 172.21.15.33:0/2951233361 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ffac8042070 con 0x7ffad81022f0 2024-05-15T14:09:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.462+0000 7ffae3036740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd7f1d7d0 0x561bd7f1dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.462+0000 7ffae3036740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bd7f1e0f0 0x561bd7f2c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.462+0000 7ffae3036740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561bd7f2cee0 0x561bd7f2f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.462+0000 7ffae3036740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561bd7e90c30 con 0x561bd7f1e0f0 2024-05-15T14:09:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.462+0000 7ffae3036740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561bd7e807d0 con 0x561bd7f2cee0 2024-05-15T14:09:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.462+0000 7ffae3036740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x561bd7e84810 con 0x561bd7f1d7d0 2024-05-15T14:09:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.462+0000 7ffaddfe1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bd7f1e0f0 0x561bd7f2c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:24.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.462+0000 7ffade7e2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd7f1d7d0 0x561bd7f1dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.462+0000 7ffade7e2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd7f1d7d0 0x561bd7f1dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:33072/0 (socket says 172.21.15.33:33072) 2024-05-15T14:09:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.462+0000 7ffade7e2640 1 -- 172.21.15.33:0/1808913559 learned_addr learned my addr 172.21.15.33:0/1808913559 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:09:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.462+0000 7ffaddfe1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bd7f1e0f0 0x561bd7f2c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40478/0 (socket says 172.21.15.33:40478) 2024-05-15T14:09:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.462+0000 7ffadd7e0640 1 --2- 172.21.15.33:0/1808913559 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561bd7f2cee0 0x561bd7f2f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.463+0000 7ffade7e2640 1 -- 172.21.15.33:0/1808913559 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561bd7f2cee0 msgr2=0x561bd7f2f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.463+0000 7ffade7e2640 1 --2- 172.21.15.33:0/1808913559 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561bd7f2cee0 0x561bd7f2f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.463+0000 7ffade7e2640 1 -- 172.21.15.33:0/1808913559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bd7f1e0f0 msgr2=0x561bd7f2c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.463+0000 7ffade7e2640 1 --2- 172.21.15.33:0/1808913559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bd7f1e0f0 0x561bd7f2c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.463+0000 7ffade7e2640 1 -- 172.21.15.33:0/1808913559 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bd7b95210 con 0x561bd7f1d7d0 2024-05-15T14:09:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.463+0000 7ffaddfe1640 1 --2- 172.21.15.33:0/1808913559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bd7f1e0f0 0x561bd7f2c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:09:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.463+0000 7ffade7e2640 1 --2- 172.21.15.33:0/1808913559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd7f1d7d0 0x561bd7f1dbb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ffad0009f90 tx=0x7ffad000f7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db5af35572d288b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.463+0000 7ffaceffd640 1 --2- 172.21.15.33:0/2951233361 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ffac40b2950 0x7ffac40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.463+0000 7ffac3fff640 1 -- 172.21.15.33:0/1808913559 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad0003b70 con 0x561bd7f1d7d0 2024-05-15T14:09:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.463+0000 7ffaddfe1640 1 --2- 172.21.15.33:0/2951233361 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ffac40b2950 0x7ffac40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.463+0000 7ffaceffd640 1 -- 172.21.15.33:0/2951233361 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7ffac8102d30 con 0x7ffad81022f0 2024-05-15T14:09:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.463+0000 7ffac3fff640 1 -- 172.21.15.33:0/1808913559 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffad0003d10 con 0x561bd7f1d7d0 2024-05-15T14:09:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.463+0000 7ffac3fff640 1 -- 172.21.15.33:0/1808913559 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad00119c0 con 0x561bd7f1d7d0 2024-05-15T14:09:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.464+0000 7ffae3036740 1 -- 172.21.15.33:0/1808913559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd7f1d7d0 msgr2=0x561bd7f1dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.464+0000 7ffae3036740 1 --2- 172.21.15.33:0/1808913559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd7f1d7d0 0x561bd7f1dbb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ffad0009f90 tx=0x7ffad000f7f0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.464+0000 7ffaddfe1640 1 --2- 172.21.15.33:0/2951233361 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ffac40b2950 0x7ffac40b4e10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ffad40058c0 tx=0x7ffad400a2d0 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.464+0000 7ffae3036740 1 -- 172.21.15.33:0/1808913559 shutdown_connections 2024-05-15T14:09:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.464+0000 7ffae3036740 1 --2- 172.21.15.33:0/1808913559 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561bd7f2cee0 0x561bd7f2f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.464+0000 7ffae3036740 1 --2- 172.21.15.33:0/1808913559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bd7f1e0f0 0x561bd7f2c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.464+0000 7ffae3036740 1 --2- 172.21.15.33:0/1808913559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd7f1d7d0 0x561bd7f1dbb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.464+0000 7ffae3036740 1 -- 172.21.15.33:0/1808913559 >> 172.21.15.33:0/1808913559 conn(0x561bd7f19e00 msgr2=0x561bd7f1a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:24.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.464+0000 7ffae3036740 1 -- 172.21.15.33:0/1808913559 shutdown_connections 2024-05-15T14:09:24.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.464+0000 7ffae3036740 1 -- 172.21.15.33:0/1808913559 wait complete. 2024-05-15T14:09:24.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.464+0000 7ffae3036740 1 -- start start 2024-05-15T14:09:24.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.464+0000 7ffae3036740 1 Processor -- start 2024-05-15T14:09:24.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.465+0000 7ffae3036740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bd7f1d7d0 0x561bd7f2c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.465+0000 7ffae3036740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561bd7f1e0f0 0x561bd7f29ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.465+0000 7ffae3036740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd7f2cee0 0x561bd7f2a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.465+0000 7ffae3036740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561bd7b66790 con 0x561bd7f1d7d0 2024-05-15T14:09:24.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.465+0000 7ffae3036740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561bd7e3c890 con 0x561bd7f1e0f0 2024-05-15T14:09:24.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.465+0000 7ffae3036740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x561bd7f2fbb0 con 0x561bd7f2cee0 2024-05-15T14:09:24.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.465+0000 7ffae3036740 10 client.0 _ll_register_callbacks cb 0x561bd7e3fd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:09:24.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.465+0000 7ffaddfe1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bd7f1d7d0 0x561bd7f2c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:24.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.465+0000 7ffade7e2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd7f2cee0 0x561bd7f2a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.465+0000 7ffaddfe1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bd7f1d7d0 0x561bd7f2c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40490/0 (socket says 172.21.15.33:40490) 2024-05-15T14:09:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.465+0000 7ffaddfe1640 1 -- 172.21.15.33:0/2228072954 learned_addr learned my addr 172.21.15.33:0/2228072954 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:09:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.465+0000 7ffadd7e0640 1 --2- 172.21.15.33:0/2228072954 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561bd7f1e0f0 0x561bd7f29ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.465+0000 7ffaddfe1640 1 -- 172.21.15.33:0/2228072954 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd7f2cee0 msgr2=0x561bd7f2a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.465+0000 7ffaddfe1640 1 --2- 172.21.15.33:0/2228072954 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd7f2cee0 0x561bd7f2a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.465+0000 7ffaddfe1640 1 -- 172.21.15.33:0/2228072954 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561bd7f1e0f0 msgr2=0x561bd7f29ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.465+0000 7ffaddfe1640 1 --2- 172.21.15.33:0/2228072954 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561bd7f1e0f0 0x561bd7f29ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.465+0000 7ffaddfe1640 1 -- 172.21.15.33:0/2228072954 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bd7c61620 con 0x561bd7f1d7d0 2024-05-15T14:09:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.465+0000 7ffade7e2640 1 --2- 172.21.15.33:0/2228072954 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd7f2cee0 0x561bd7f2a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:09:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.465+0000 7ffaddfe1640 1 --2- 172.21.15.33:0/2228072954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bd7f1d7d0 0x561bd7f2c7e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ffad401c9f0 tx=0x7ffad4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09bcb7f862e77d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.465+0000 7ffac17fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:09:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.466+0000 7ffac17fa640 1 -- 172.21.15.33:0/2228072954 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad402c060 con 0x561bd7f1d7d0 2024-05-15T14:09:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.466+0000 7ffac17fa640 1 -- 172.21.15.33:0/2228072954 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7ffad4016030 con 0x561bd7f1d7d0 2024-05-15T14:09:24.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.466+0000 7ffae3036740 1 -- 172.21.15.33:0/2228072954 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffad0003d10 con 0x561bd7f1d7d0 2024-05-15T14:09:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.466+0000 7ffac17fa640 1 -- 172.21.15.33:0/2228072954 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ffad4030470 con 0x561bd7f1d7d0 2024-05-15T14:09:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.466+0000 7ffae3036740 10 client.16461 fetch_fsmap learned FSMap version 34 2024-05-15T14:09:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.466+0000 7ffae3036740 1 -- 172.21.15.33:0/2228072954 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561bd7f3b650 con 0x561bd7f1d7d0 2024-05-15T14:09:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.466+0000 7ffac17fa640 1 -- 172.21.15.33:0/2228072954 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffad0003d10 con 0x561bd7f1d7d0 2024-05-15T14:09:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.466+0000 7ffac17fa640 1 -- 172.21.15.33:0/2228072954 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7ffad4027750 con 0x561bd7f1d7d0 2024-05-15T14:09:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.466+0000 7ffae3036740 10 client.16461 fetch_fsmap finished waiting for FSMap version 34 2024-05-15T14:09:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.466+0000 7ffae3036740 10 client.16461 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:09:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.466+0000 7ffae3036740 20 client.16461 populate_metadata read hostname 'smithi033' 2024-05-15T14:09:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.466+0000 7ffae3036740 1 --2- 172.21.15.33:0/2228072954 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x561bd7f3cb60 0x561bd7f3ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.466+0000 7ffae3036740 4 client.16461 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:09:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.466+0000 7ffae3036740 1 -- 172.21.15.33:0/2228072954 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bd7e92b80 con 0x561bd7f3cb60 2024-05-15T14:09:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.466+0000 7ffadd7e0640 1 --2- 172.21.15.33:0/2228072954 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x561bd7f3cb60 0x561bd7f3ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.467+0000 7ffadd7e0640 1 --2- 172.21.15.33:0/2228072954 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x561bd7f3cb60 0x561bd7f3ef50 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee9a9a8215c4f676 server_cookie=80b151156fc17e9e in_seq=0 out_seq=0 2024-05-15T14:09:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.467+0000 7ffac17fa640 10 client.16461 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:09:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.468+0000 7ffac17fa640 1 -- 172.21.15.33:0/2228072954 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561bd7e92b80 con 0x561bd7f3cb60 2024-05-15T14:09:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.468+0000 7ffac17fa640 10 client.16461 handle_command_reply: tid=0 2024-05-15T14:09:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.468+0000 7ffae3036740 1 client.16461 shutdown 2024-05-15T14:09:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.468+0000 7ffae3036740 1 -- 172.21.15.33:0/2228072954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bd7f1d7d0 msgr2=0x561bd7f2c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.468+0000 7ffae3036740 1 --2- 172.21.15.33:0/2228072954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bd7f1d7d0 0x561bd7f2c7e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ffad401c9f0 tx=0x7ffad4004730 comp rx=0 tx=0).stop 2024-05-15T14:09:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.468+0000 7ffae3036740 1 -- 172.21.15.33:0/2228072954 shutdown_connections 2024-05-15T14:09:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.468+0000 7ffae3036740 1 --2- 172.21.15.33:0/2228072954 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x561bd7f3cb60 0x561bd7f3ef50 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.468+0000 7ffae3036740 1 --2- 172.21.15.33:0/2228072954 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd7f2cee0 0x561bd7f2a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.468+0000 7ffae3036740 1 --2- 172.21.15.33:0/2228072954 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561bd7f1e0f0 0x561bd7f29ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.468+0000 7ffae3036740 1 --2- 172.21.15.33:0/2228072954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bd7f1d7d0 0x561bd7f2c7e0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.468+0000 7ffac17fa640 0 client.16461 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:09:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.468+0000 7ffae3036740 1 -- 172.21.15.33:0/2228072954 >> 172.21.15.33:0/2228072954 conn(0x561bd7f19e00 msgr2=0x561bd7f1fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.468+0000 7ffae3036740 1 -- 172.21.15.33:0/2228072954 shutdown_connections 2024-05-15T14:09:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.468+0000 7ffae3036740 1 -- 172.21.15.33:0/2228072954 wait complete. 2024-05-15T14:09:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.468+0000 7ffae3036740 20 client.16461 trim_cache size 0 max 16384 2024-05-15T14:09:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.484+0000 7ffae3036740 1 Processor -- start 2024-05-15T14:09:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.484+0000 7ffae3036740 1 -- start start 2024-05-15T14:09:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.485+0000 7ffae3036740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561bd7f1d7d0 0x561bd7f2c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.485+0000 7ffae3036740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd7f1e0f0 0x561bd7f18f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.485+0000 7ffae3036740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bd7f2cee0 0x561bd7f44860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.485+0000 7ffae3036740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffad81aee40 con 0x561bd7f2cee0 2024-05-15T14:09:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.485+0000 7ffae3036740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffad81ac680 con 0x561bd7f1d7d0 2024-05-15T14:09:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.485+0000 7ffae3036740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ffad81aa4f0 con 0x561bd7f1e0f0 2024-05-15T14:09:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.485+0000 7ffade7e2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bd7f2cee0 0x561bd7f44860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.485+0000 7ffade7e2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bd7f2cee0 0x561bd7f44860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40494/0 (socket says 172.21.15.33:40494) 2024-05-15T14:09:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.485+0000 7ffade7e2640 1 -- 172.21.15.33:0/1509111785 learned_addr learned my addr 172.21.15.33:0/1509111785 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:09:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.485+0000 7ffadd7e0640 1 --2- 172.21.15.33:0/1509111785 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd7f1e0f0 0x561bd7f18f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.485+0000 7ffade7e2640 1 -- 172.21.15.33:0/1509111785 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd7f1e0f0 msgr2=0x561bd7f18f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.485+0000 7ffade7e2640 1 --2- 172.21.15.33:0/1509111785 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd7f1e0f0 0x561bd7f18f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.485+0000 7ffade7e2640 1 -- 172.21.15.33:0/1509111785 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561bd7f1d7d0 msgr2=0x561bd7f2c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.485+0000 7ffade7e2640 1 --2- 172.21.15.33:0/1509111785 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561bd7f1d7d0 0x561bd7f2c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.485+0000 7ffade7e2640 1 -- 172.21.15.33:0/1509111785 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bd7f19c30 con 0x561bd7f2cee0 2024-05-15T14:09:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.485+0000 7ffade7e2640 1 --2- 172.21.15.33:0/1509111785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bd7f2cee0 0x561bd7f44860 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ffad0009f30 tx=0x7ffad0009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee4c400de709000 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.485+0000 7ffac37fe640 1 -- 172.21.15.33:0/1509111785 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad0004c00 con 0x561bd7f2cee0 2024-05-15T14:09:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.485+0000 7ffac37fe640 1 -- 172.21.15.33:0/1509111785 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffad0004da0 con 0x561bd7f2cee0 2024-05-15T14:09:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.485+0000 7ffac37fe640 1 -- 172.21.15.33:0/1509111785 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad0005570 con 0x561bd7f2cee0 2024-05-15T14:09:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.487+0000 7ffae3036740 1 -- 172.21.15.33:0/1509111785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bd7f2cee0 msgr2=0x561bd7f44860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.487+0000 7ffae3036740 1 --2- 172.21.15.33:0/1509111785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bd7f2cee0 0x561bd7f44860 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ffad0009f30 tx=0x7ffad0009d40 comp rx=0 tx=0).stop 2024-05-15T14:09:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.487+0000 7ffae3036740 1 -- 172.21.15.33:0/1509111785 shutdown_connections 2024-05-15T14:09:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.487+0000 7ffae3036740 1 --2- 172.21.15.33:0/1509111785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bd7f2cee0 0x561bd7f44860 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.487+0000 7ffae3036740 1 --2- 172.21.15.33:0/1509111785 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd7f1e0f0 0x561bd7f18f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.487+0000 7ffae3036740 1 --2- 172.21.15.33:0/1509111785 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561bd7f1d7d0 0x561bd7f2c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.487+0000 7ffae3036740 1 -- 172.21.15.33:0/1509111785 >> 172.21.15.33:0/1509111785 conn(0x561bd7f19e00 msgr2=0x561bd7f18870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.487+0000 7ffae3036740 1 -- 172.21.15.33:0/1509111785 shutdown_connections 2024-05-15T14:09:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.487+0000 7ffae3036740 1 -- 172.21.15.33:0/1509111785 wait complete. 2024-05-15T14:09:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.487+0000 7ffae3036740 1 -- start start 2024-05-15T14:09:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.487+0000 7ffae3036740 1 Processor -- start 2024-05-15T14:09:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.487+0000 7ffae3036740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd7f1d7d0 0x561bd7f35410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.487+0000 7ffae3036740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bd7f1e0f0 0x561bd7f30780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.487+0000 7ffae3036740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561bd7f2cee0 0x561bd7f30ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.487+0000 7ffae3036740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffad81aab40 con 0x561bd7f1e0f0 2024-05-15T14:09:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.487+0000 7ffae3036740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffad81abfb0 con 0x561bd7f2cee0 2024-05-15T14:09:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.487+0000 7ffae3036740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ffad81ab210 con 0x561bd7f1d7d0 2024-05-15T14:09:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.487+0000 7ffae3036740 10 client.0 _ll_register_callbacks cb 0x7ffad401c8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:09:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.487+0000 7ffadd7e0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bd7f1e0f0 0x561bd7f30780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.487+0000 7ffadd7e0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bd7f1e0f0 0x561bd7f30780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40504/0 (socket says 172.21.15.33:40504) 2024-05-15T14:09:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.487+0000 7ffadd7e0640 1 -- 172.21.15.33:0/47322095 learned_addr learned my addr 172.21.15.33:0/47322095 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:09:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.487+0000 7ffaddfe1640 1 --2- 172.21.15.33:0/47322095 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd7f1d7d0 0x561bd7f35410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.487+0000 7ffade7e2640 1 --2- 172.21.15.33:0/47322095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561bd7f2cee0 0x561bd7f30ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.487+0000 7ffadd7e0640 1 -- 172.21.15.33:0/47322095 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd7f1d7d0 msgr2=0x561bd7f35410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.487+0000 7ffadd7e0640 1 --2- 172.21.15.33:0/47322095 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd7f1d7d0 0x561bd7f35410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.487+0000 7ffadd7e0640 1 -- 172.21.15.33:0/47322095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561bd7f2cee0 msgr2=0x561bd7f30ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.487+0000 7ffadd7e0640 1 --2- 172.21.15.33:0/47322095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561bd7f2cee0 0x561bd7f30ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.487+0000 7ffadd7e0640 1 -- 172.21.15.33:0/47322095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bd7f31510 con 0x561bd7f1e0f0 2024-05-15T14:09:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.487+0000 7ffade7e2640 1 --2- 172.21.15.33:0/47322095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561bd7f2cee0 0x561bd7f30ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:09:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.488+0000 7ffadd7e0640 1 --2- 172.21.15.33:0/47322095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bd7f1e0f0 0x561bd7f30780 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ffac8100520 tx=0x7ffac8031f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a8903626858177f server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:24.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.488+0000 7ffac17fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:09:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.488+0000 7ffac17fa640 1 -- 172.21.15.33:0/47322095 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac8049040 con 0x561bd7f1e0f0 2024-05-15T14:09:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.488+0000 7ffac17fa640 1 -- 172.21.15.33:0/47322095 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7ffac803b460 con 0x561bd7f1e0f0 2024-05-15T14:09:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.488+0000 7ffae3036740 1 -- 172.21.15.33:0/47322095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffad0004da0 con 0x561bd7f1e0f0 2024-05-15T14:09:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.488+0000 7ffac17fa640 1 -- 172.21.15.33:0/47322095 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ffac80ce3b0 con 0x561bd7f1e0f0 2024-05-15T14:09:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.488+0000 7ffae3036740 10 client.16467 fetch_fsmap learned FSMap version 34 2024-05-15T14:09:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.488+0000 7ffae3036740 1 -- 172.21.15.33:0/47322095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561bd7fa7510 con 0x561bd7f1e0f0 2024-05-15T14:09:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.488+0000 7ffac17fa640 1 -- 172.21.15.33:0/47322095 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffad0004da0 con 0x561bd7f1e0f0 2024-05-15T14:09:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.488+0000 7ffac17fa640 1 -- 172.21.15.33:0/47322095 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7ffac80d29d0 con 0x561bd7f1e0f0 2024-05-15T14:09:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.488+0000 7ffae3036740 10 client.16467 fetch_fsmap finished waiting for FSMap version 34 2024-05-15T14:09:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.488+0000 7ffae3036740 10 client.16467 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:09:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.488+0000 7ffae3036740 20 client.16467 populate_metadata read hostname 'smithi033' 2024-05-15T14:09:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.488+0000 7ffae3036740 1 --2- 172.21.15.33:0/47322095 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x561bd7fa7ca0 0x561bd7faa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.488+0000 7ffae3036740 4 client.16467 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:09:24.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.488+0000 7ffae3036740 1 -- 172.21.15.33:0/47322095 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561bd7e8c360 con 0x561bd7fa7ca0 2024-05-15T14:09:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.489+0000 7ffaddfe1640 1 --2- 172.21.15.33:0/47322095 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x561bd7fa7ca0 0x561bd7faa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.489+0000 7ffaddfe1640 1 --2- 172.21.15.33:0/47322095 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x561bd7fa7ca0 0x561bd7faa0c0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95ce14e14243d2e4 server_cookie=27446166d79c46b4 in_seq=0 out_seq=0 2024-05-15T14:09:24.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.489+0000 7ffac17fa640 10 client.16467 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:09:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.521+0000 7ffac17fa640 1 -- 172.21.15.33:0/47322095 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561bd7e8c360 con 0x561bd7fa7ca0 2024-05-15T14:09:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.521+0000 7ffac17fa640 10 client.16467 handle_command_reply: tid=0 2024-05-15T14:09:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.521+0000 7ffae3036740 1 client.16467 shutdown 2024-05-15T14:09:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.521+0000 7ffae3036740 1 -- 172.21.15.33:0/47322095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bd7f1e0f0 msgr2=0x561bd7f30780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.521+0000 7ffae3036740 1 --2- 172.21.15.33:0/47322095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bd7f1e0f0 0x561bd7f30780 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ffac8100520 tx=0x7ffac8031f50 comp rx=0 tx=0).stop 2024-05-15T14:09:24.524 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.521+0000 7ffae3036740 1 -- 172.21.15.33:0/47322095 shutdown_connections 2024-05-15T14:09:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.521+0000 7ffae3036740 1 --2- 172.21.15.33:0/47322095 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x561bd7fa7ca0 0x561bd7faa0c0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.521+0000 7ffae3036740 1 --2- 172.21.15.33:0/47322095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561bd7f2cee0 0x561bd7f30ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.521+0000 7ffae3036740 1 --2- 172.21.15.33:0/47322095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bd7f1e0f0 0x561bd7f30780 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.521+0000 7ffac17fa640 0 client.16467 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:09:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.521+0000 7ffae3036740 1 --2- 172.21.15.33:0/47322095 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561bd7f1d7d0 0x561bd7f35410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.521+0000 7ffae3036740 1 -- 172.21.15.33:0/47322095 >> 172.21.15.33:0/47322095 conn(0x561bd7f19e00 msgr2=0x561bd7f19100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.521+0000 7ffae3036740 1 -- 172.21.15.33:0/47322095 shutdown_connections 2024-05-15T14:09:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.521+0000 7ffae3036740 1 -- 172.21.15.33:0/47322095 wait complete. 2024-05-15T14:09:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.521+0000 7ffae3036740 20 client.16467 trim_cache size 0 max 16384 2024-05-15T14:09:24.525 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:09:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:09:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:09:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.523+0000 7ffae0245640 1 -- 172.21.15.33:0/2951233361 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ffac40b2950 msgr2=0x7ffac40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:24.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.523+0000 7ffae0245640 1 --2- 172.21.15.33:0/2951233361 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ffac40b2950 0x7ffac40b4e10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ffad40058c0 tx=0x7ffad400a2d0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.523+0000 7ffae0245640 1 -- 172.21.15.33:0/2951233361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffad81022f0 msgr2=0x7ffad81997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:24.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.523+0000 7ffae0245640 1 --2- 172.21.15.33:0/2951233361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffad81022f0 0x7ffad81997a0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ffac8005ab0 tx=0x7ffac8003b90 comp rx=0 tx=0).stop 2024-05-15T14:09:24.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.523+0000 7ffae0245640 1 -- 172.21.15.33:0/2951233361 shutdown_connections 2024-05-15T14:09:24.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.523+0000 7ffae0245640 1 --2- 172.21.15.33:0/2951233361 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ffac40b2950 0x7ffac40b4e10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.523+0000 7ffae0245640 1 --2- 172.21.15.33:0/2951233361 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffad8107940 0x7ffad819db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.523+0000 7ffae0245640 1 --2- 172.21.15.33:0/2951233361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffad81022f0 0x7ffad81997a0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.523+0000 7ffae0245640 1 --2- 172.21.15.33:0/2951233361 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffad8101930 0x7ffad8199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:24.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.523+0000 7ffae0245640 1 -- 172.21.15.33:0/2951233361 >> 172.21.15.33:0/2951233361 conn(0x7ffad8078a70 msgr2=0x7ffad806a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:24.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.523+0000 7ffae0245640 1 -- 172.21.15.33:0/2951233361 shutdown_connections 2024-05-15T14:09:24.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:24.523+0000 7ffae0245640 1 -- 172.21.15.33:0/2951233361 wait complete. 2024-05-15T14:09:24.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:24 smithi033 ceph-mon[31650]: pgmap v282: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 417 KiB/s wr, 55 op/s 2024-05-15T14:09:24.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:24 smithi052 ceph-mon[36190]: pgmap v282: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 417 KiB/s wr, 55 op/s 2024-05-15T14:09:24.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:24 smithi081 ceph-mon[36434]: pgmap v282: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 417 KiB/s wr, 55 op/s 2024-05-15T14:09:25.073 INFO:tasks.workunit.client.0.smithi033.stdout: 1 517995 5.52 MB/sec execute 300 sec latency 17034.857 ms 2024-05-15T14:09:25.198 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ad5b83c1', total seconds elapsed: 23.6 2024-05-15T14:09:26.073 INFO:tasks.workunit.client.0.smithi033.stdout: 1 519462 5.53 MB/sec execute 301 sec latency 12.802 ms 2024-05-15T14:09:26.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:26 smithi052 ceph-mon[36190]: pgmap v283: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 236 KiB/s wr, 19 op/s 2024-05-15T14:09:26.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:26 smithi081 ceph-mon[36434]: pgmap v283: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 236 KiB/s wr, 19 op/s 2024-05-15T14:09:26.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:26 smithi033 ceph-mon[31650]: pgmap v283: 97 pgs: 97 active+clean; 154 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 236 KiB/s wr, 19 op/s 2024-05-15T14:09:27.073 INFO:tasks.workunit.client.0.smithi033.stdout: 1 521685 5.55 MB/sec execute 302 sec latency 28.121 ms 2024-05-15T14:09:28.073 INFO:tasks.workunit.client.0.smithi033.stdout: 1 523157 5.56 MB/sec execute 303 sec latency 13.431 ms 2024-05-15T14:09:28.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:28 smithi052 ceph-mon[36190]: pgmap v284: 97 pgs: 97 active+clean; 159 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 875 KiB/s wr, 91 op/s 2024-05-15T14:09:28.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:28 smithi081 ceph-mon[36434]: pgmap v284: 97 pgs: 97 active+clean; 159 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 875 KiB/s wr, 91 op/s 2024-05-15T14:09:28.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:28 smithi033 ceph-mon[31650]: pgmap v284: 97 pgs: 97 active+clean; 159 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 875 KiB/s wr, 91 op/s 2024-05-15T14:09:29.073 INFO:tasks.workunit.client.0.smithi033.stdout: 1 524992 5.57 MB/sec execute 304 sec latency 58.119 ms 2024-05-15T14:09:30.073 INFO:tasks.workunit.client.0.smithi033.stdout: 1 526422 5.58 MB/sec execute 305 sec latency 16.936 ms 2024-05-15T14:09:30.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:30 smithi052 ceph-mon[36190]: pgmap v285: 97 pgs: 97 active+clean; 159 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 748 KiB/s wr, 83 op/s 2024-05-15T14:09:30.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:30 smithi081 ceph-mon[36434]: pgmap v285: 97 pgs: 97 active+clean; 159 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 748 KiB/s wr, 83 op/s 2024-05-15T14:09:30.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:30 smithi033 ceph-mon[31650]: pgmap v285: 97 pgs: 97 active+clean; 159 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 748 KiB/s wr, 83 op/s 2024-05-15T14:09:31.073 INFO:tasks.workunit.client.0.smithi033.stdout: 1 528531 5.60 MB/sec execute 306 sec latency 27.296 ms 2024-05-15T14:09:32.073 INFO:tasks.workunit.client.0.smithi033.stdout: 1 530017 5.61 MB/sec execute 307 sec latency 16.924 ms 2024-05-15T14:09:32.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:32 smithi052 ceph-mon[36190]: pgmap v286: 97 pgs: 97 active+clean; 159 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 748 KiB/s wr, 83 op/s 2024-05-15T14:09:32.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:32 smithi081 ceph-mon[36434]: pgmap v286: 97 pgs: 97 active+clean; 159 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 748 KiB/s wr, 83 op/s 2024-05-15T14:09:32.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:32 smithi033 ceph-mon[31650]: pgmap v286: 97 pgs: 97 active+clean; 159 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 748 KiB/s wr, 83 op/s 2024-05-15T14:09:33.073 INFO:tasks.workunit.client.0.smithi033.stdout: 1 532114 5.62 MB/sec execute 308 sec latency 25.263 ms 2024-05-15T14:09:34.073 INFO:tasks.workunit.client.0.smithi033.stdout: 1 533413 5.63 MB/sec execute 309 sec latency 13.645 ms 2024-05-15T14:09:34.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:34 smithi052 ceph-mon[36190]: pgmap v287: 97 pgs: 97 active+clean; 165 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 197 op/s 2024-05-15T14:09:34.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:34 smithi081 ceph-mon[36434]: pgmap v287: 97 pgs: 97 active+clean; 165 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 197 op/s 2024-05-15T14:09:34.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:34 smithi033 ceph-mon[31650]: pgmap v287: 97 pgs: 97 active+clean; 165 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 197 op/s 2024-05-15T14:09:35.073 INFO:tasks.workunit.client.0.smithi033.stdout: 1 535336 5.64 MB/sec execute 310 sec latency 65.992 ms 2024-05-15T14:09:36.073 INFO:tasks.workunit.client.0.smithi033.stdout: 1 537018 5.66 MB/sec execute 311 sec latency 9.956 ms 2024-05-15T14:09:36.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:36 smithi052 ceph-mon[36190]: pgmap v288: 97 pgs: 97 active+clean; 165 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.9 MiB/s wr, 186 op/s 2024-05-15T14:09:36.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:36 smithi081 ceph-mon[36434]: pgmap v288: 97 pgs: 97 active+clean; 165 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.9 MiB/s wr, 186 op/s 2024-05-15T14:09:36.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:36 smithi033 ceph-mon[31650]: pgmap v288: 97 pgs: 97 active+clean; 165 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.9 MiB/s wr, 186 op/s 2024-05-15T14:09:37.073 INFO:tasks.workunit.client.0.smithi033.stdout: 1 538860 5.66 MB/sec execute 312 sec latency 25.806 ms 2024-05-15T14:09:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.625+0000 7fbe57ef8640 1 -- 172.21.15.33:0/808306021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe501022f0 msgr2=0x7fbe5010e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.625+0000 7fbe57ef8640 1 --2- 172.21.15.33:0/808306021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe501022f0 0x7fbe5010e800 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbe48009be0 tx=0x7fbe4802a6c0 comp rx=0 tx=0).stop 2024-05-15T14:09:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.625+0000 7fbe57ef8640 1 -- 172.21.15.33:0/808306021 shutdown_connections 2024-05-15T14:09:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.625+0000 7fbe57ef8640 1 --2- 172.21.15.33:0/808306021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe501022f0 0x7fbe5010e800 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.625+0000 7fbe57ef8640 1 --2- 172.21.15.33:0/808306021 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe50101930 0x7fbe50101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.625+0000 7fbe57ef8640 1 --2- 172.21.15.33:0/808306021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe50107940 0x7fbe50107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.625+0000 7fbe57ef8640 1 -- 172.21.15.33:0/808306021 >> 172.21.15.33:0/808306021 conn(0x7fbe50078a70 msgr2=0x7fbe50078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.625+0000 7fbe57ef8640 1 -- 172.21.15.33:0/808306021 shutdown_connections 2024-05-15T14:09:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.625+0000 7fbe57ef8640 1 -- 172.21.15.33:0/808306021 wait complete. 2024-05-15T14:09:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.626+0000 7fbe57ef8640 1 Processor -- start 2024-05-15T14:09:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.626+0000 7fbe57ef8640 1 -- start start 2024-05-15T14:09:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.626+0000 7fbe57ef8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe50101930 0x7fbe5019b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.626+0000 7fbe57ef8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe501022f0 0x7fbe5019b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.626+0000 7fbe57ef8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe50107940 0x7fbe5019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.626+0000 7fbe57ef8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbe50110af0 con 0x7fbe50101930 2024-05-15T14:09:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.626+0000 7fbe57ef8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbe50110970 con 0x7fbe50107940 2024-05-15T14:09:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.626+0000 7fbe57ef8640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbe50110c70 con 0x7fbe501022f0 2024-05-15T14:09:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.626+0000 7fbe55c94640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe50101930 0x7fbe5019b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.626+0000 7fbe55493640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe501022f0 0x7fbe5019b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.626+0000 7fbe55c94640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe50101930 0x7fbe5019b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54692/0 (socket says 172.21.15.33:54692) 2024-05-15T14:09:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.626+0000 7fbe55c94640 1 -- 172.21.15.33:0/4120333181 learned_addr learned my addr 172.21.15.33:0/4120333181 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:09:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.626+0000 7fbe56495640 1 --2- 172.21.15.33:0/4120333181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe50107940 0x7fbe5019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.627+0000 7fbe55493640 1 -- 172.21.15.33:0/4120333181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe50107940 msgr2=0x7fbe5019fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.627+0000 7fbe55493640 1 --2- 172.21.15.33:0/4120333181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe50107940 0x7fbe5019fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.627+0000 7fbe55493640 1 -- 172.21.15.33:0/4120333181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe50101930 msgr2=0x7fbe5019b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.627+0000 7fbe55493640 1 --2- 172.21.15.33:0/4120333181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe50101930 0x7fbe5019b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.627+0000 7fbe55493640 1 -- 172.21.15.33:0/4120333181 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe501a0400 con 0x7fbe501022f0 2024-05-15T14:09:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.627+0000 7fbe55493640 1 --2- 172.21.15.33:0/4120333181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe501022f0 0x7fbe5019b9f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbe40009990 tx=0x7fbe40009e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.627+0000 7fbe46ffd640 1 -- 172.21.15.33:0/4120333181 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe40005220 con 0x7fbe501022f0 2024-05-15T14:09:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.627+0000 7fbe57ef8640 1 -- 172.21.15.33:0/4120333181 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe501a06f0 con 0x7fbe501022f0 2024-05-15T14:09:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.627+0000 7fbe46ffd640 1 -- 172.21.15.33:0/4120333181 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe4000cc80 con 0x7fbe501022f0 2024-05-15T14:09:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.628+0000 7fbe57ef8640 1 -- 172.21.15.33:0/4120333181 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe501087b0 con 0x7fbe501022f0 2024-05-15T14:09:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.628+0000 7fbe46ffd640 1 -- 172.21.15.33:0/4120333181 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe40004c80 con 0x7fbe501022f0 2024-05-15T14:09:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.630+0000 7fbe5ac0f740 1 Processor -- start 2024-05-15T14:09:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.630+0000 7fbe5ac0f740 1 -- start start 2024-05-15T14:09:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.631+0000 7fbe5ac0f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f023b88f0 0x558f023b8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.631+0000 7fbe5ac0f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f023b9210 0x558f023c3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.631+0000 7fbe5ac0f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f023c3ad0 0x558f023c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.631+0000 7fbe5ac0f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558f0232bc30 con 0x558f023b88f0 2024-05-15T14:09:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.631+0000 7fbe5ac0f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558f0231b7d0 con 0x558f023c3ad0 2024-05-15T14:09:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.631+0000 7fbe5ac0f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x558f0231f810 con 0x558f023b9210 2024-05-15T14:09:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.631+0000 7fbe56495640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f023b88f0 0x558f023b8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.631+0000 7fbe56495640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f023b88f0 0x558f023b8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54696/0 (socket says 172.21.15.33:54696) 2024-05-15T14:09:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.631+0000 7fbe55c94640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f023b9210 0x558f023c3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.631+0000 7fbe56495640 1 -- 172.21.15.33:0/4068984926 learned_addr learned my addr 172.21.15.33:0/4068984926 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:09:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.631+0000 7fbe55c94640 1 -- 172.21.15.33:0/4068984926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f023c3ad0 msgr2=0x558f023c5ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-15T14:09:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.631+0000 7fbe55c94640 1 --2- 172.21.15.33:0/4068984926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f023c3ad0 0x558f023c5ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.631+0000 7fbe55c94640 1 -- 172.21.15.33:0/4068984926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f023b88f0 msgr2=0x558f023b8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.631+0000 7fbe55c94640 1 --2- 172.21.15.33:0/4068984926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f023b88f0 0x558f023b8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.631+0000 7fbe55c94640 1 -- 172.21.15.33:0/4068984926 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f02030210 con 0x558f023b9210 2024-05-15T14:09:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.632+0000 7fbe55c94640 1 --2- 172.21.15.33:0/4068984926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f023b9210 0x558f023c3590 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c014f90 tx=0x7fbe4c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=118a7efaf1037933 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.632+0000 7fbe46ffd640 1 -- 172.21.15.33:0/4120333181 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbe4000c7d0 con 0x7fbe501022f0 2024-05-15T14:09:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.632+0000 7fbe3bfff640 1 -- 172.21.15.33:0/4068984926 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c004310 con 0x558f023b9210 2024-05-15T14:09:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.632+0000 7fbe3bfff640 1 -- 172.21.15.33:0/4068984926 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe4c0044b0 con 0x558f023b9210 2024-05-15T14:09:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.632+0000 7fbe3bfff640 1 -- 172.21.15.33:0/4068984926 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c00ee50 con 0x558f023b9210 2024-05-15T14:09:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.633+0000 7fbe46ffd640 1 --2- 172.21.15.33:0/4120333181 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbe3c0b2950 0x7fbe3c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.633+0000 7fbe5ac0f740 1 -- 172.21.15.33:0/4068984926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f023b9210 msgr2=0x558f023c3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.633+0000 7fbe5ac0f740 1 --2- 172.21.15.33:0/4068984926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f023b9210 0x558f023c3590 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c014f90 tx=0x7fbe4c009260 comp rx=0 tx=0).stop 2024-05-15T14:09:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.633+0000 7fbe56495640 1 --2- 172.21.15.33:0/4120333181 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbe3c0b2950 0x7fbe3c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.633+0000 7fbe46ffd640 1 -- 172.21.15.33:0/4120333181 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fbe400f2c60 con 0x7fbe501022f0 2024-05-15T14:09:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.633+0000 7fbe5ac0f740 1 -- 172.21.15.33:0/4068984926 shutdown_connections 2024-05-15T14:09:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.633+0000 7fbe5ac0f740 1 --2- 172.21.15.33:0/4068984926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f023c3ad0 0x558f023c5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.633+0000 7fbe5ac0f740 1 --2- 172.21.15.33:0/4068984926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f023b9210 0x558f023c3590 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.633+0000 7fbe5ac0f740 1 --2- 172.21.15.33:0/4068984926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f023b88f0 0x558f023b8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.633+0000 7fbe5ac0f740 1 -- 172.21.15.33:0/4068984926 >> 172.21.15.33:0/4068984926 conn(0x558f023b4e00 msgr2=0x558f023b51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.633+0000 7fbe5ac0f740 1 -- 172.21.15.33:0/4068984926 shutdown_connections 2024-05-15T14:09:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.633+0000 7fbe5ac0f740 1 -- 172.21.15.33:0/4068984926 wait complete. 2024-05-15T14:09:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.633+0000 7fbe56495640 1 --2- 172.21.15.33:0/4120333181 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbe3c0b2950 0x7fbe3c0b4e10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbe480099a0 tx=0x7fbe48005fd0 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.634+0000 7fbe5ac0f740 1 -- start start 2024-05-15T14:09:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.634+0000 7fbe5ac0f740 1 Processor -- start 2024-05-15T14:09:37.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.634+0000 7fbe5ac0f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f023b88f0 0x558f023c3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:37.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.634+0000 7fbe5ac0f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f023b9210 0x558f023c0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:37.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.634+0000 7fbe5ac0f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f023c3ad0 0x558f023c0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:37.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.634+0000 7fbe5ac0f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558f02001790 con 0x558f023b9210 2024-05-15T14:09:37.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.634+0000 7fbe5ac0f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558f022d7890 con 0x558f023c3ad0 2024-05-15T14:09:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.634+0000 7fbe5ac0f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x558f023c67a0 con 0x558f023b88f0 2024-05-15T14:09:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.634+0000 7fbe5ac0f740 10 client.0 _ll_register_callbacks cb 0x558f022dad60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:09:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.634+0000 7fbe55c94640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f023b88f0 0x558f023c3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.634+0000 7fbe55c94640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f023b88f0 0x558f023c3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:49702/0 (socket says 172.21.15.33:49702) 2024-05-15T14:09:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.634+0000 7fbe55c94640 1 -- 172.21.15.33:0/3758413067 learned_addr learned my addr 172.21.15.33:0/3758413067 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:09:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.634+0000 7fbe56495640 1 --2- 172.21.15.33:0/3758413067 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f023c3ad0 0x558f023c0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.635+0000 7fbe55493640 1 --2- 172.21.15.33:0/3758413067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f023b9210 0x558f023c0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.635+0000 7fbe55c94640 1 -- 172.21.15.33:0/3758413067 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f023c3ad0 msgr2=0x558f023c0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.635+0000 7fbe55c94640 1 --2- 172.21.15.33:0/3758413067 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f023c3ad0 0x558f023c0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.635+0000 7fbe55c94640 1 -- 172.21.15.33:0/3758413067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f023b9210 msgr2=0x558f023c0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.635+0000 7fbe55c94640 1 --2- 172.21.15.33:0/3758413067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f023b9210 0x558f023c0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.635+0000 7fbe55c94640 1 -- 172.21.15.33:0/3758413067 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f020fc620 con 0x558f023b88f0 2024-05-15T14:09:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.635+0000 7fbe55493640 1 --2- 172.21.15.33:0/3758413067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f023b9210 0x558f023c0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:09:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.635+0000 7fbe55c94640 1 --2- 172.21.15.33:0/3758413067 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f023b88f0 0x558f023c3480 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c009230 tx=0x7fbe4c00eb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64119bb8efc85d5e server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.635+0000 7fbe397fa640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-05-15T14:09:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.635+0000 7fbe397fa640 1 -- 172.21.15.33:0/3758413067 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c02d060 con 0x558f023b88f0 2024-05-15T14:09:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.635+0000 7fbe5ac0f740 1 -- 172.21.15.33:0/3758413067 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe4c0044b0 con 0x558f023b88f0 2024-05-15T14:09:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.635+0000 7fbe397fa640 1 -- 172.21.15.33:0/3758413067 <== mon.1 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fbe4c005390 con 0x558f023b88f0 2024-05-15T14:09:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.635+0000 7fbe397fa640 1 -- 172.21.15.33:0/3758413067 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbe4c005590 con 0x558f023b88f0 2024-05-15T14:09:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.635+0000 7fbe5ac0f740 10 client.26068 fetch_fsmap learned FSMap version 34 2024-05-15T14:09:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.635+0000 7fbe5ac0f740 1 -- 172.21.15.33:0/3758413067 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558f023d2200 con 0x558f023b88f0 2024-05-15T14:09:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.636+0000 7fbe397fa640 1 -- 172.21.15.33:0/3758413067 <== mon.1 v2:172.21.15.81:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe4c0044b0 con 0x558f023b88f0 2024-05-15T14:09:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.636+0000 7fbe397fa640 1 -- 172.21.15.33:0/3758413067 <== mon.1 v2:172.21.15.81:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fbe4c014250 con 0x558f023b88f0 2024-05-15T14:09:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.637+0000 7fbe5ac0f740 10 client.26068 fetch_fsmap finished waiting for FSMap version 34 2024-05-15T14:09:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.637+0000 7fbe5ac0f740 10 client.26068 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:09:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.637+0000 7fbe5ac0f740 20 client.26068 populate_metadata read hostname 'smithi033' 2024-05-15T14:09:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.637+0000 7fbe5ac0f740 1 --2- 172.21.15.33:0/3758413067 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558f023d3610 0x558f023d5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.637+0000 7fbe5ac0f740 4 client.26068 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:09:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.637+0000 7fbe5ac0f740 1 -- 172.21.15.33:0/3758413067 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f0232db80 con 0x558f023d3610 2024-05-15T14:09:37.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.637+0000 7fbe55493640 1 --2- 172.21.15.33:0/3758413067 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558f023d3610 0x558f023d5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:37.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.637+0000 7fbe55493640 1 --2- 172.21.15.33:0/3758413067 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558f023d3610 0x558f023d5a00 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14c49342982defad server_cookie=89103c8e6db0c4d4 in_seq=0 out_seq=0 2024-05-15T14:09:37.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.637+0000 7fbe397fa640 10 client.26068 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:09:37.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.638+0000 7fbe397fa640 1 -- 172.21.15.33:0/3758413067 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558f0232db80 con 0x558f023d3610 2024-05-15T14:09:37.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.638+0000 7fbe397fa640 10 client.26068 handle_command_reply: tid=0 2024-05-15T14:09:37.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.638+0000 7fbe5ac0f740 1 client.26068 shutdown 2024-05-15T14:09:37.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.638+0000 7fbe5ac0f740 1 -- 172.21.15.33:0/3758413067 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f023b88f0 msgr2=0x558f023c3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:37.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.638+0000 7fbe5ac0f740 1 --2- 172.21.15.33:0/3758413067 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f023b88f0 0x558f023c3480 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c009230 tx=0x7fbe4c00eb70 comp rx=0 tx=0).stop 2024-05-15T14:09:37.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.638+0000 7fbe5ac0f740 1 -- 172.21.15.33:0/3758413067 shutdown_connections 2024-05-15T14:09:37.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.638+0000 7fbe5ac0f740 1 --2- 172.21.15.33:0/3758413067 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558f023d3610 0x558f023d5a00 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:37.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.638+0000 7fbe5ac0f740 1 --2- 172.21.15.33:0/3758413067 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f023c3ad0 0x558f023c0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:37.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.638+0000 7fbe5ac0f740 1 --2- 172.21.15.33:0/3758413067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f023b9210 0x558f023c0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:37.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.638+0000 7fbe397fa640 0 client.26068 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:09:37.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.638+0000 7fbe5ac0f740 1 --2- 172.21.15.33:0/3758413067 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f023b88f0 0x558f023c3480 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:37.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.638+0000 7fbe5ac0f740 1 -- 172.21.15.33:0/3758413067 >> 172.21.15.33:0/3758413067 conn(0x558f023b4e00 msgr2=0x558f023b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:37.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.638+0000 7fbe5ac0f740 1 -- 172.21.15.33:0/3758413067 shutdown_connections 2024-05-15T14:09:37.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.638+0000 7fbe5ac0f740 1 -- 172.21.15.33:0/3758413067 wait complete. 2024-05-15T14:09:37.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.639+0000 7fbe5ac0f740 20 client.26068 trim_cache size 0 max 16384 2024-05-15T14:09:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.655+0000 7fbe5ac0f740 1 Processor -- start 2024-05-15T14:09:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.655+0000 7fbe5ac0f740 1 -- start start 2024-05-15T14:09:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.655+0000 7fbe5ac0f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f023b88f0 0x558f023cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.655+0000 7fbe5ac0f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f023b9210 0x558f023aff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.655+0000 7fbe5ac0f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f023c3ad0 0x558f023c2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.655+0000 7fbe5ac0f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbe501aee80 con 0x558f023c3ad0 2024-05-15T14:09:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.655+0000 7fbe5ac0f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbe501ac6c0 con 0x558f023b88f0 2024-05-15T14:09:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.655+0000 7fbe5ac0f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbe501aa5b0 con 0x558f023b9210 2024-05-15T14:09:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.655+0000 7fbe56495640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f023c3ad0 0x558f023c2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.655+0000 7fbe55493640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f023b9210 0x558f023aff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.655+0000 7fbe55c94640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f023b88f0 0x558f023cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.655+0000 7fbe55493640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f023b9210 0x558f023aff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:49704/0 (socket says 172.21.15.33:49704) 2024-05-15T14:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.655+0000 7fbe55493640 1 -- 172.21.15.33:0/1978578402 learned_addr learned my addr 172.21.15.33:0/1978578402 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.656+0000 7fbe55493640 1 -- 172.21.15.33:0/1978578402 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f023b88f0 msgr2=0x558f023cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.656+0000 7fbe55493640 1 --2- 172.21.15.33:0/1978578402 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f023b88f0 0x558f023cf670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.656+0000 7fbe55493640 1 -- 172.21.15.33:0/1978578402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f023c3ad0 msgr2=0x558f023c2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.656+0000 7fbe55493640 1 --2- 172.21.15.33:0/1978578402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f023c3ad0 0x558f023c2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.656+0000 7fbe55493640 1 -- 172.21.15.33:0/1978578402 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f023b5f90 con 0x558f023b9210 2024-05-15T14:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.656+0000 7fbe55c94640 1 --2- 172.21.15.33:0/1978578402 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f023b88f0 0x558f023cf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.656+0000 7fbe56495640 1 --2- 172.21.15.33:0/1978578402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f023c3ad0 0x558f023c2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.656+0000 7fbe55493640 1 --2- 172.21.15.33:0/1978578402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f023b9210 0x558f023aff90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbe400b7250 tx=0x7fbe40005b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b18374159483388 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.656+0000 7fbe3b7fe640 1 -- 172.21.15.33:0/1978578402 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe400c2d90 con 0x558f023b9210 2024-05-15T14:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.656+0000 7fbe3b7fe640 1 -- 172.21.15.33:0/1978578402 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe400053c0 con 0x558f023b9210 2024-05-15T14:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.656+0000 7fbe3b7fe640 1 -- 172.21.15.33:0/1978578402 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe400f0af0 con 0x558f023b9210 2024-05-15T14:09:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.657+0000 7fbe5ac0f740 1 -- 172.21.15.33:0/1978578402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f023b9210 msgr2=0x558f023aff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.657+0000 7fbe5ac0f740 1 --2- 172.21.15.33:0/1978578402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f023b9210 0x558f023aff90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbe400b7250 tx=0x7fbe40005b90 comp rx=0 tx=0).stop 2024-05-15T14:09:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.657+0000 7fbe5ac0f740 1 -- 172.21.15.33:0/1978578402 shutdown_connections 2024-05-15T14:09:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.657+0000 7fbe5ac0f740 1 --2- 172.21.15.33:0/1978578402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f023c3ad0 0x558f023c2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.657+0000 7fbe5ac0f740 1 --2- 172.21.15.33:0/1978578402 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f023b9210 0x558f023aff90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.657+0000 7fbe5ac0f740 1 --2- 172.21.15.33:0/1978578402 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f023b88f0 0x558f023cf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.657+0000 7fbe5ac0f740 1 -- 172.21.15.33:0/1978578402 >> 172.21.15.33:0/1978578402 conn(0x558f023b4e00 msgr2=0x558f023bab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.657+0000 7fbe5ac0f740 1 -- 172.21.15.33:0/1978578402 shutdown_connections 2024-05-15T14:09:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.657+0000 7fbe5ac0f740 1 -- 172.21.15.33:0/1978578402 wait complete. 2024-05-15T14:09:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.657+0000 7fbe5ac0f740 1 -- start start 2024-05-15T14:09:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.657+0000 7fbe5ac0f740 1 Processor -- start 2024-05-15T14:09:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.658+0000 7fbe5ac0f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f023b88f0 0x558f023cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.658+0000 7fbe5ac0f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f023b9210 0x558f023bcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.658+0000 7fbe5ac0f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f023c3ad0 0x558f023bd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.658+0000 7fbe5ac0f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbe501aac00 con 0x558f023b88f0 2024-05-15T14:09:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.658+0000 7fbe5ac0f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbe501abff0 con 0x558f023c3ad0 2024-05-15T14:09:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.658+0000 7fbe5ac0f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbe501ab250 con 0x558f023b9210 2024-05-15T14:09:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.658+0000 7fbe5ac0f740 10 client.0 _ll_register_callbacks cb 0x7fbe4c014d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:09:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.658+0000 7fbe56495640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f023c3ad0 0x558f023bd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.658+0000 7fbe56495640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f023c3ad0 0x558f023bd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:53618/0 (socket says 172.21.15.33:53618) 2024-05-15T14:09:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.658+0000 7fbe56495640 1 -- 172.21.15.33:0/3430098641 learned_addr learned my addr 172.21.15.33:0/3430098641 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:09:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.658+0000 7fbe55c94640 1 --2- 172.21.15.33:0/3430098641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f023b88f0 0x558f023cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.658+0000 7fbe55493640 1 --2- 172.21.15.33:0/3430098641 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f023b9210 0x558f023bcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.658+0000 7fbe56495640 1 -- 172.21.15.33:0/3430098641 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f023b9210 msgr2=0x558f023bcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.658+0000 7fbe56495640 1 --2- 172.21.15.33:0/3430098641 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f023b9210 0x558f023bcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.658+0000 7fbe56495640 1 -- 172.21.15.33:0/3430098641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f023b88f0 msgr2=0x558f023cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.658+0000 7fbe56495640 1 --2- 172.21.15.33:0/3430098641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f023b88f0 0x558f023cbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.658+0000 7fbe56495640 1 -- 172.21.15.33:0/3430098641 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f023cc470 con 0x558f023c3ad0 2024-05-15T14:09:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.658+0000 7fbe55c94640 1 --2- 172.21.15.33:0/3430098641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f023b88f0 0x558f023cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:09:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.659+0000 7fbe56495640 1 --2- 172.21.15.33:0/3430098641 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f023c3ad0 0x558f023bd330 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbe48009be0 tx=0x7fbe48009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ac0fd63bd742a79 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:37.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.659+0000 7fbe397fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-05-15T14:09:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.659+0000 7fbe397fa640 1 -- 172.21.15.33:0/3430098641 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe48036040 con 0x558f023c3ad0 2024-05-15T14:09:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.659+0000 7fbe5ac0f740 1 -- 172.21.15.33:0/3430098641 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe400053c0 con 0x558f023c3ad0 2024-05-15T14:09:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.659+0000 7fbe397fa640 1 -- 172.21.15.33:0/3430098641 <== mon.2 v2:172.21.15.52:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fbe48004040 con 0x558f023c3ad0 2024-05-15T14:09:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.659+0000 7fbe397fa640 1 -- 172.21.15.33:0/3430098641 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbe48004240 con 0x558f023c3ad0 2024-05-15T14:09:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.659+0000 7fbe5ac0f740 10 client.25943 fetch_fsmap learned FSMap version 34 2024-05-15T14:09:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.659+0000 7fbe5ac0f740 1 -- 172.21.15.33:0/3430098641 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558f0244e710 con 0x558f023c3ad0 2024-05-15T14:09:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.659+0000 7fbe397fa640 1 -- 172.21.15.33:0/3430098641 <== mon.2 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe400053c0 con 0x558f023c3ad0 2024-05-15T14:09:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.660+0000 7fbe397fa640 1 -- 172.21.15.33:0/3430098641 <== mon.2 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fbe48054020 con 0x558f023c3ad0 2024-05-15T14:09:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.660+0000 7fbe5ac0f740 10 client.25943 fetch_fsmap finished waiting for FSMap version 34 2024-05-15T14:09:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.660+0000 7fbe5ac0f740 10 client.25943 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:09:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.660+0000 7fbe5ac0f740 20 client.25943 populate_metadata read hostname 'smithi033' 2024-05-15T14:09:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.660+0000 7fbe5ac0f740 1 --2- 172.21.15.33:0/3430098641 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558f0244ee90 0x558f024512b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.660+0000 7fbe5ac0f740 4 client.25943 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:09:37.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.660+0000 7fbe5ac0f740 1 -- 172.21.15.33:0/3430098641 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558f02327360 con 0x558f0244ee90 2024-05-15T14:09:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.660+0000 7fbe55c94640 1 --2- 172.21.15.33:0/3430098641 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558f0244ee90 0x558f024512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.660+0000 7fbe55c94640 1 --2- 172.21.15.33:0/3430098641 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558f0244ee90 0x558f024512b0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f79a7f7421e5454b server_cookie=b644b7c9d9ed4d3 in_seq=0 out_seq=0 2024-05-15T14:09:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.660+0000 7fbe397fa640 10 client.25943 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:09:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.728+0000 7fbe397fa640 1 -- 172.21.15.33:0/3430098641 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558f02327360 con 0x558f0244ee90 2024-05-15T14:09:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.728+0000 7fbe397fa640 10 client.25943 handle_command_reply: tid=0 2024-05-15T14:09:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.728+0000 7fbe5ac0f740 1 client.25943 shutdown 2024-05-15T14:09:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.728+0000 7fbe5ac0f740 1 -- 172.21.15.33:0/3430098641 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f023c3ad0 msgr2=0x558f023bd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.728+0000 7fbe5ac0f740 1 --2- 172.21.15.33:0/3430098641 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f023c3ad0 0x558f023bd330 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbe48009be0 tx=0x7fbe48009500 comp rx=0 tx=0).stop 2024-05-15T14:09:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.729+0000 7fbe5ac0f740 1 -- 172.21.15.33:0/3430098641 shutdown_connections 2024-05-15T14:09:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.729+0000 7fbe5ac0f740 1 --2- 172.21.15.33:0/3430098641 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558f0244ee90 0x558f024512b0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.729+0000 7fbe5ac0f740 1 --2- 172.21.15.33:0/3430098641 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f023c3ad0 0x558f023bd330 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.729+0000 7fbe5ac0f740 1 --2- 172.21.15.33:0/3430098641 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558f023b9210 0x558f023bcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.729+0000 7fbe5ac0f740 1 --2- 172.21.15.33:0/3430098641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f023b88f0 0x558f023cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.729+0000 7fbe5ac0f740 1 -- 172.21.15.33:0/3430098641 >> 172.21.15.33:0/3430098641 conn(0x558f023b4e00 msgr2=0x558f023b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.729+0000 7fbe397fa640 0 client.25943 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:09:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.729+0000 7fbe5ac0f740 1 -- 172.21.15.33:0/3430098641 shutdown_connections 2024-05-15T14:09:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.729+0000 7fbe5ac0f740 1 -- 172.21.15.33:0/3430098641 wait complete. 2024-05-15T14:09:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.729+0000 7fbe5ac0f740 20 client.25943 trim_cache size 0 max 16384 2024-05-15T14:09:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:09:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:09:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:09:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.730+0000 7fbe57ef8640 1 -- 172.21.15.33:0/4120333181 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbe3c0b2950 msgr2=0x7fbe3c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.730+0000 7fbe57ef8640 1 --2- 172.21.15.33:0/4120333181 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbe3c0b2950 0x7fbe3c0b4e10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbe480099a0 tx=0x7fbe48005fd0 comp rx=0 tx=0).stop 2024-05-15T14:09:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.730+0000 7fbe57ef8640 1 -- 172.21.15.33:0/4120333181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe501022f0 msgr2=0x7fbe5019b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.730+0000 7fbe57ef8640 1 --2- 172.21.15.33:0/4120333181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe501022f0 0x7fbe5019b9f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbe40009990 tx=0x7fbe40009e50 comp rx=0 tx=0).stop 2024-05-15T14:09:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.730+0000 7fbe57ef8640 1 -- 172.21.15.33:0/4120333181 shutdown_connections 2024-05-15T14:09:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.730+0000 7fbe57ef8640 1 --2- 172.21.15.33:0/4120333181 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbe3c0b2950 0x7fbe3c0b4e10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:37.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.730+0000 7fbe57ef8640 1 --2- 172.21.15.33:0/4120333181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe50107940 0x7fbe5019fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:37.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.730+0000 7fbe57ef8640 1 --2- 172.21.15.33:0/4120333181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe501022f0 0x7fbe5019b9f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:37.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.730+0000 7fbe57ef8640 1 --2- 172.21.15.33:0/4120333181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe50101930 0x7fbe5019b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:37.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.730+0000 7fbe57ef8640 1 -- 172.21.15.33:0/4120333181 >> 172.21.15.33:0/4120333181 conn(0x7fbe50078a70 msgr2=0x7fbe5010cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:37.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.730+0000 7fbe57ef8640 1 -- 172.21.15.33:0/4120333181 shutdown_connections 2024-05-15T14:09:37.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:37.730+0000 7fbe57ef8640 1 -- 172.21.15.33:0/4120333181 wait complete. 2024-05-15T14:09:37.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:37 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:09:37.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:37 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:09:37.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:37 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:09:38.073 INFO:tasks.workunit.client.0.smithi033.stdout: 1 540409 5.68 MB/sec execute 313 sec latency 42.233 ms 2024-05-15T14:09:38.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:38 smithi052 ceph-mon[36190]: pgmap v289: 97 pgs: 97 active+clean; 172 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.3 MiB/s wr, 298 op/s 2024-05-15T14:09:38.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:38 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:09:38.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:38 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:09:38.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:38 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:09:38.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:38 smithi081 ceph-mon[36434]: pgmap v289: 97 pgs: 97 active+clean; 172 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.3 MiB/s wr, 298 op/s 2024-05-15T14:09:38.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:38 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:09:38.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:38 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:09:38.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:38 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:09:38.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:38 smithi033 ceph-mon[31650]: pgmap v289: 97 pgs: 97 active+clean; 172 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.3 MiB/s wr, 298 op/s 2024-05-15T14:09:38.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:38 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:09:38.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:38 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:09:38.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:38 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:09:39.073 INFO:tasks.workunit.client.0.smithi033.stdout: 1 542458 5.69 MB/sec execute 314 sec latency 16.421 ms 2024-05-15T14:09:40.073 INFO:tasks.workunit.client.0.smithi033.stdout: 1 544106 5.71 MB/sec execute 315 sec latency 31.878 ms 2024-05-15T14:09:40.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:40 smithi052 ceph-mon[36190]: pgmap v290: 97 pgs: 97 active+clean; 172 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 227 op/s 2024-05-15T14:09:40.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:40 smithi081 ceph-mon[36434]: pgmap v290: 97 pgs: 97 active+clean; 172 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 227 op/s 2024-05-15T14:09:40.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:40 smithi033 ceph-mon[31650]: pgmap v290: 97 pgs: 97 active+clean; 172 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 227 op/s 2024-05-15T14:09:41.074 INFO:tasks.workunit.client.0.smithi033.stdout: 1 546288 5.72 MB/sec execute 316 sec latency 24.601 ms 2024-05-15T14:09:42.074 INFO:tasks.workunit.client.0.smithi033.stdout: 1 547804 5.73 MB/sec execute 317 sec latency 19.396 ms 2024-05-15T14:09:42.842 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:42 smithi052 ceph-mon[36190]: pgmap v291: 97 pgs: 97 active+clean; 172 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 227 op/s 2024-05-15T14:09:42.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:42 smithi081 ceph-mon[36434]: pgmap v291: 97 pgs: 97 active+clean; 172 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 227 op/s 2024-05-15T14:09:42.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:42 smithi033 ceph-mon[31650]: pgmap v291: 97 pgs: 97 active+clean; 172 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 227 op/s 2024-05-15T14:09:43.074 INFO:tasks.workunit.client.0.smithi033.stdout: 1 549666 5.74 MB/sec execute 318 sec latency 32.091 ms 2024-05-15T14:09:43.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:43 smithi033 ceph-mon[31650]: pgmap v292: 97 pgs: 97 active+clean; 168 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.8 MiB/s wr, 369 op/s 2024-05-15T14:09:44.074 INFO:tasks.workunit.client.0.smithi033.stdout: 1 551290 5.76 MB/sec execute 319 sec latency 20.315 ms 2024-05-15T14:09:44.090 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:43 smithi052 ceph-mon[36190]: pgmap v292: 97 pgs: 97 active+clean; 168 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.8 MiB/s wr, 369 op/s 2024-05-15T14:09:44.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:43 smithi081 ceph-mon[36434]: pgmap v292: 97 pgs: 97 active+clean; 168 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.8 MiB/s wr, 369 op/s 2024-05-15T14:09:45.074 INFO:tasks.workunit.client.0.smithi033.stdout: 1 553196 5.76 MB/sec execute 320 sec latency 26.068 ms 2024-05-15T14:09:46.074 INFO:tasks.workunit.client.0.smithi033.stdout: 1 554754 5.78 MB/sec execute 321 sec latency 12.836 ms 2024-05-15T14:09:46.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:46 smithi033 ceph-mon[31650]: pgmap v293: 97 pgs: 97 active+clean; 168 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 255 op/s 2024-05-15T14:09:46.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:46 smithi052 ceph-mon[36190]: pgmap v293: 97 pgs: 97 active+clean; 168 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 255 op/s 2024-05-15T14:09:46.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:46 smithi081 ceph-mon[36434]: pgmap v293: 97 pgs: 97 active+clean; 168 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 255 op/s 2024-05-15T14:09:47.074 INFO:tasks.workunit.client.0.smithi033.stdout: 1 556646 5.78 MB/sec execute 322 sec latency 29.951 ms 2024-05-15T14:09:48.074 INFO:tasks.workunit.client.0.smithi033.stdout: 1 558325 5.80 MB/sec execute 323 sec latency 10.934 ms 2024-05-15T14:09:48.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:48 smithi033 ceph-mon[31650]: pgmap v294: 97 pgs: 97 active+clean; 175 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.3 MiB/s wr, 347 op/s 2024-05-15T14:09:48.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:48 smithi052 ceph-mon[36190]: pgmap v294: 97 pgs: 97 active+clean; 175 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.3 MiB/s wr, 347 op/s 2024-05-15T14:09:48.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:48 smithi081 ceph-mon[36434]: pgmap v294: 97 pgs: 97 active+clean; 175 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.3 MiB/s wr, 347 op/s 2024-05-15T14:09:49.074 INFO:tasks.workunit.client.0.smithi033.stdout: 1 560581 5.82 MB/sec execute 324 sec latency 15.858 ms 2024-05-15T14:09:50.074 INFO:tasks.workunit.client.0.smithi033.stdout: 1 562156 5.83 MB/sec execute 325 sec latency 14.104 ms 2024-05-15T14:09:50.642 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:50 smithi052 ceph-mon[36190]: pgmap v295: 97 pgs: 97 active+clean; 175 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 234 op/s 2024-05-15T14:09:50.658 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:50 smithi081 ceph-mon[36434]: pgmap v295: 97 pgs: 97 active+clean; 175 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 234 op/s 2024-05-15T14:09:50.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:50 smithi033 ceph-mon[31650]: pgmap v295: 97 pgs: 97 active+clean; 175 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 234 op/s 2024-05-15T14:09:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.830+0000 7ff27c8a0640 1 -- 172.21.15.33:0/2680031008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff278101930 msgr2=0x7ff278101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.830+0000 7ff27c8a0640 1 --2- 172.21.15.33:0/2680031008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff278101930 0x7ff278101db0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff26c009930 tx=0x7ff26c02a420 comp rx=0 tx=0).stop 2024-05-15T14:09:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.830+0000 7ff27c8a0640 1 -- 172.21.15.33:0/2680031008 shutdown_connections 2024-05-15T14:09:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.830+0000 7ff27c8a0640 1 --2- 172.21.15.33:0/2680031008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2781022f0 0x7ff27810e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.830+0000 7ff27c8a0640 1 --2- 172.21.15.33:0/2680031008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff278101930 0x7ff278101db0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.830+0000 7ff27c8a0640 1 --2- 172.21.15.33:0/2680031008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff278107940 0x7ff278107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.830+0000 7ff27c8a0640 1 -- 172.21.15.33:0/2680031008 >> 172.21.15.33:0/2680031008 conn(0x7ff278078a70 msgr2=0x7ff278078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.830+0000 7ff27c8a0640 1 -- 172.21.15.33:0/2680031008 shutdown_connections 2024-05-15T14:09:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.830+0000 7ff27c8a0640 1 -- 172.21.15.33:0/2680031008 wait complete. 2024-05-15T14:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.831+0000 7ff27c8a0640 1 Processor -- start 2024-05-15T14:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.831+0000 7ff27c8a0640 1 -- start start 2024-05-15T14:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.831+0000 7ff27c8a0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff278101930 0x7ff278199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.831+0000 7ff27c8a0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2781022f0 0x7ff2781997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.831+0000 7ff27c8a0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff278107940 0x7ff27819dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.831+0000 7ff27c8a0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff278111c00 con 0x7ff278101930 2024-05-15T14:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.831+0000 7ff27c8a0640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff278111a80 con 0x7ff278107940 2024-05-15T14:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.831+0000 7ff27c8a0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ff278111d80 con 0x7ff2781022f0 2024-05-15T14:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.831+0000 7ff27659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff278101930 0x7ff278199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.831+0000 7ff275d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2781022f0 0x7ff2781997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.831+0000 7ff27659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff278101930 0x7ff278199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48944/0 (socket says 172.21.15.33:48944) 2024-05-15T14:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.831+0000 7ff27659c640 1 -- 172.21.15.33:0/580031864 learned_addr learned my addr 172.21.15.33:0/580031864 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.831+0000 7ff276d9d640 1 --2- 172.21.15.33:0/580031864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff278107940 0x7ff27819dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.832+0000 7ff275d9b640 1 -- 172.21.15.33:0/580031864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff278107940 msgr2=0x7ff27819dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.832+0000 7ff275d9b640 1 --2- 172.21.15.33:0/580031864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff278107940 0x7ff27819dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.832+0000 7ff275d9b640 1 -- 172.21.15.33:0/580031864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff278101930 msgr2=0x7ff278199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.832+0000 7ff275d9b640 1 --2- 172.21.15.33:0/580031864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff278101930 0x7ff278199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.832+0000 7ff275d9b640 1 -- 172.21.15.33:0/580031864 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff27819e210 con 0x7ff2781022f0 2024-05-15T14:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.832+0000 7ff275d9b640 1 --2- 172.21.15.33:0/580031864 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2781022f0 0x7ff2781997b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff26c031440 tx=0x7ff26c004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.832+0000 7ff2677fe640 1 -- 172.21.15.33:0/580031864 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26c03f070 con 0x7ff2781022f0 2024-05-15T14:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.832+0000 7ff27c8a0640 1 -- 172.21.15.33:0/580031864 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff27819e4a0 con 0x7ff2781022f0 2024-05-15T14:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.832+0000 7ff2677fe640 1 -- 172.21.15.33:0/580031864 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff26c003aa0 con 0x7ff2781022f0 2024-05-15T14:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.832+0000 7ff2677fe640 1 -- 172.21.15.33:0/580031864 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26c035070 con 0x7ff2781022f0 2024-05-15T14:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.832+0000 7ff27c8a0640 1 -- 172.21.15.33:0/580031864 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2780fe820 con 0x7ff2781022f0 2024-05-15T14:09:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.835+0000 7ff2677fe640 1 -- 172.21.15.33:0/580031864 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff26c03b460 con 0x7ff2781022f0 2024-05-15T14:09:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.835+0000 7ff27f5c2740 1 Processor -- start 2024-05-15T14:09:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.835+0000 7ff27f5c2740 1 -- start start 2024-05-15T14:09:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.836+0000 7ff27f5c2740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556015e518f0 0x556015e51cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.836+0000 7ff27f5c2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556015e52210 0x556015e5c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.836+0000 7ff27f5c2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556015e5cad0 0x556015e5eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.836+0000 7ff27f5c2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556015dc4c30 con 0x556015e5cad0 2024-05-15T14:09:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.836+0000 7ff27f5c2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556015db47d0 con 0x556015e52210 2024-05-15T14:09:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.836+0000 7ff27f5c2740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x556015db8810 con 0x556015e518f0 2024-05-15T14:09:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.836+0000 7ff276d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556015e518f0 0x556015e51cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.836+0000 7ff276d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556015e518f0 0x556015e51cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:39488/0 (socket says 172.21.15.33:39488) 2024-05-15T14:09:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.836+0000 7ff276d9d640 1 -- 172.21.15.33:0/858634966 learned_addr learned my addr 172.21.15.33:0/858634966 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:09:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.836+0000 7ff275d9b640 1 --2- 172.21.15.33:0/858634966 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556015e5cad0 0x556015e5eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.836+0000 7ff27659c640 1 --2- 172.21.15.33:0/858634966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556015e52210 0x556015e5c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.836+0000 7ff276d9d640 1 -- 172.21.15.33:0/858634966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556015e52210 msgr2=0x556015e5c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.836+0000 7ff276d9d640 1 --2- 172.21.15.33:0/858634966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556015e52210 0x556015e5c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.836+0000 7ff276d9d640 1 -- 172.21.15.33:0/858634966 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556015e5cad0 msgr2=0x556015e5eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.836+0000 7ff276d9d640 1 --2- 172.21.15.33:0/858634966 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556015e5cad0 0x556015e5eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.836+0000 7ff276d9d640 1 -- 172.21.15.33:0/858634966 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556015ac9210 con 0x556015e518f0 2024-05-15T14:09:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.836+0000 7ff2677fe640 1 --2- 172.21.15.33:0/580031864 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ff2600b2950 0x7ff2600b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.837+0000 7ff276d9d640 1 --2- 172.21.15.33:0/858634966 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556015e518f0 0x556015e51cd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff270009fd0 tx=0x7ff2700118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e41316c6b22fe4f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.837+0000 7ff27659c640 1 --2- 172.21.15.33:0/580031864 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ff2600b2950 0x7ff2600b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.837+0000 7ff2677fe640 1 -- 172.21.15.33:0/580031864 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7ff26c102a40 con 0x7ff2781022f0 2024-05-15T14:09:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.837+0000 7ff264ff9640 1 -- 172.21.15.33:0/858634966 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff270004d20 con 0x556015e518f0 2024-05-15T14:09:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.837+0000 7ff264ff9640 1 -- 172.21.15.33:0/858634966 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff270004050 con 0x556015e518f0 2024-05-15T14:09:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.837+0000 7ff264ff9640 1 -- 172.21.15.33:0/858634966 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff270005280 con 0x556015e518f0 2024-05-15T14:09:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.838+0000 7ff27f5c2740 1 -- 172.21.15.33:0/858634966 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556015e518f0 msgr2=0x556015e51cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.838+0000 7ff27f5c2740 1 --2- 172.21.15.33:0/858634966 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556015e518f0 0x556015e51cd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff270009fd0 tx=0x7ff2700118d0 comp rx=0 tx=0).stop 2024-05-15T14:09:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.838+0000 7ff27659c640 1 --2- 172.21.15.33:0/580031864 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ff2600b2950 0x7ff2600b4e10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x556015e52e90 tx=0x7ff268012040 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.838+0000 7ff27f5c2740 1 -- 172.21.15.33:0/858634966 shutdown_connections 2024-05-15T14:09:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.838+0000 7ff27f5c2740 1 --2- 172.21.15.33:0/858634966 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556015e5cad0 0x556015e5eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.838+0000 7ff27f5c2740 1 --2- 172.21.15.33:0/858634966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556015e52210 0x556015e5c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.838+0000 7ff27f5c2740 1 --2- 172.21.15.33:0/858634966 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556015e518f0 0x556015e51cd0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.838+0000 7ff27f5c2740 1 -- 172.21.15.33:0/858634966 >> 172.21.15.33:0/858634966 conn(0x556015e4de00 msgr2=0x556015e4e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.838+0000 7ff27f5c2740 1 -- 172.21.15.33:0/858634966 shutdown_connections 2024-05-15T14:09:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.838+0000 7ff27f5c2740 1 -- 172.21.15.33:0/858634966 wait complete. 2024-05-15T14:09:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.838+0000 7ff27f5c2740 1 -- start start 2024-05-15T14:09:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.838+0000 7ff27f5c2740 1 Processor -- start 2024-05-15T14:09:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.838+0000 7ff27f5c2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556015e518f0 0x556015e5c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.838+0000 7ff27f5c2740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556015e52210 0x556015e597c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.838+0000 7ff27f5c2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556015e5cad0 0x556015e59dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.839+0000 7ff27f5c2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556015a9a790 con 0x556015e5cad0 2024-05-15T14:09:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.839+0000 7ff27f5c2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556015d70890 con 0x556015e518f0 2024-05-15T14:09:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.839+0000 7ff27f5c2740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x556015e5f7a0 con 0x556015e52210 2024-05-15T14:09:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.839+0000 7ff275d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556015e52210 0x556015e597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.839+0000 7ff275d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556015e52210 0x556015e597c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:39504/0 (socket says 172.21.15.33:39504) 2024-05-15T14:09:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.839+0000 7ff275d9b640 1 -- 172.21.15.33:0/2908390352 learned_addr learned my addr 172.21.15.33:0/2908390352 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:09:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.839+0000 7ff276d9d640 1 --2- 172.21.15.33:0/2908390352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556015e5cad0 0x556015e59dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.839+0000 7ff27f5c2740 10 client.0 _ll_register_callbacks cb 0x556015d73d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:09:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.839+0000 7ff27659c640 1 --2- 172.21.15.33:0/2908390352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556015e518f0 0x556015e5c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.839+0000 7ff275d9b640 1 -- 172.21.15.33:0/2908390352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556015e518f0 msgr2=0x556015e5c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.839+0000 7ff275d9b640 1 --2- 172.21.15.33:0/2908390352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556015e518f0 0x556015e5c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.839+0000 7ff275d9b640 1 -- 172.21.15.33:0/2908390352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556015e5cad0 msgr2=0x556015e59dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.839+0000 7ff275d9b640 1 --2- 172.21.15.33:0/2908390352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556015e5cad0 0x556015e59dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.839+0000 7ff275d9b640 1 -- 172.21.15.33:0/2908390352 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556015b95620 con 0x556015e52210 2024-05-15T14:09:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.839+0000 7ff276d9d640 1 --2- 172.21.15.33:0/2908390352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556015e5cad0 0x556015e59dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:09:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.839+0000 7ff275d9b640 1 --2- 172.21.15.33:0/2908390352 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556015e52210 0x556015e597c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0c9ac0 tx=0x7ff26c009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a929c5f34687c34 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.839+0000 7ff25dffb640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-05-15T14:09:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.839+0000 7ff25dffb640 1 -- 172.21.15.33:0/2908390352 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26c048070 con 0x556015e52210 2024-05-15T14:09:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.839+0000 7ff27f5c2740 1 -- 172.21.15.33:0/2908390352 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff270004050 con 0x556015e52210 2024-05-15T14:09:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.839+0000 7ff25dffb640 1 -- 172.21.15.33:0/2908390352 <== mon.1 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7ff26c0c6050 con 0x556015e52210 2024-05-15T14:09:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.840+0000 7ff25dffb640 1 -- 172.21.15.33:0/2908390352 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff26c0c9710 con 0x556015e52210 2024-05-15T14:09:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.840+0000 7ff27f5c2740 10 client.26086 fetch_fsmap learned FSMap version 34 2024-05-15T14:09:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.840+0000 7ff27f5c2740 1 -- 172.21.15.33:0/2908390352 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556015e6b240 con 0x556015e52210 2024-05-15T14:09:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.840+0000 7ff25dffb640 1 -- 172.21.15.33:0/2908390352 <== mon.1 v2:172.21.15.81:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff270004050 con 0x556015e52210 2024-05-15T14:09:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.840+0000 7ff25dffb640 1 -- 172.21.15.33:0/2908390352 <== mon.1 v2:172.21.15.81:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7ff26c04b060 con 0x556015e52210 2024-05-15T14:09:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.840+0000 7ff27f5c2740 10 client.26086 fetch_fsmap finished waiting for FSMap version 34 2024-05-15T14:09:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.840+0000 7ff27f5c2740 10 client.26086 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:09:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.840+0000 7ff27f5c2740 20 client.26086 populate_metadata read hostname 'smithi033' 2024-05-15T14:09:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.840+0000 7ff27f5c2740 1 --2- 172.21.15.33:0/2908390352 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556015e6c820 0x556015e6ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.840+0000 7ff27f5c2740 4 client.26086 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:09:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.840+0000 7ff27f5c2740 1 -- 172.21.15.33:0/2908390352 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556015dc6b80 con 0x556015e6c820 2024-05-15T14:09:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.840+0000 7ff27659c640 1 --2- 172.21.15.33:0/2908390352 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556015e6c820 0x556015e6ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.841+0000 7ff25dffb640 10 client.26086 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:09:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.841+0000 7ff27659c640 1 --2- 172.21.15.33:0/2908390352 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556015e6c820 0x556015e6ec80 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be4a5c170ea175ad server_cookie=a9b28a536a6771e5 in_seq=0 out_seq=0 2024-05-15T14:09:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.842+0000 7ff25dffb640 1 -- 172.21.15.33:0/2908390352 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556015dc6b80 con 0x556015e6c820 2024-05-15T14:09:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.842+0000 7ff25dffb640 10 client.26086 handle_command_reply: tid=0 2024-05-15T14:09:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.842+0000 7ff27f5c2740 1 client.26086 shutdown 2024-05-15T14:09:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.842+0000 7ff27f5c2740 1 -- 172.21.15.33:0/2908390352 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556015e52210 msgr2=0x556015e597c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.842+0000 7ff27f5c2740 1 --2- 172.21.15.33:0/2908390352 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556015e52210 0x556015e597c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0c9ac0 tx=0x7ff26c009620 comp rx=0 tx=0).stop 2024-05-15T14:09:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.842+0000 7ff27f5c2740 1 -- 172.21.15.33:0/2908390352 shutdown_connections 2024-05-15T14:09:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.842+0000 7ff27f5c2740 1 --2- 172.21.15.33:0/2908390352 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556015e6c820 0x556015e6ec80 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.842+0000 7ff27f5c2740 1 --2- 172.21.15.33:0/2908390352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556015e5cad0 0x556015e59dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.842+0000 7ff25dffb640 0 client.26086 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:09:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.842+0000 7ff27f5c2740 1 --2- 172.21.15.33:0/2908390352 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556015e52210 0x556015e597c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.842+0000 7ff27f5c2740 1 --2- 172.21.15.33:0/2908390352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556015e518f0 0x556015e5c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.842+0000 7ff27f5c2740 1 -- 172.21.15.33:0/2908390352 >> 172.21.15.33:0/2908390352 conn(0x556015e4de00 msgr2=0x556015e4d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.842+0000 7ff27f5c2740 1 -- 172.21.15.33:0/2908390352 shutdown_connections 2024-05-15T14:09:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.842+0000 7ff27f5c2740 1 -- 172.21.15.33:0/2908390352 wait complete. 2024-05-15T14:09:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.842+0000 7ff27f5c2740 20 client.26086 trim_cache size 0 max 16384 2024-05-15T14:09:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.858+0000 7ff27f5c2740 1 Processor -- start 2024-05-15T14:09:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.858+0000 7ff27f5c2740 1 -- start start 2024-05-15T14:09:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.858+0000 7ff27f5c2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556015e518f0 0x556015e53b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.859+0000 7ff27f5c2740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556015e52210 0x556015e68670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.859+0000 7ff27f5c2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556015e5cad0 0x556015e5c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.859+0000 7ff27f5c2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff2781aed70 con 0x556015e518f0 2024-05-15T14:09:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.859+0000 7ff27f5c2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff2781ac5b0 con 0x556015e5cad0 2024-05-15T14:09:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.859+0000 7ff27f5c2740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ff2781aa420 con 0x556015e52210 2024-05-15T14:09:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.859+0000 7ff27659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556015e518f0 0x556015e53b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.859+0000 7ff27659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556015e518f0 0x556015e53b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48964/0 (socket says 172.21.15.33:48964) 2024-05-15T14:09:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.859+0000 7ff27659c640 1 -- 172.21.15.33:0/887794634 learned_addr learned my addr 172.21.15.33:0/887794634 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:09:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.859+0000 7ff275d9b640 1 --2- 172.21.15.33:0/887794634 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556015e52210 0x556015e68670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.859+0000 7ff276d9d640 1 --2- 172.21.15.33:0/887794634 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556015e5cad0 0x556015e5c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.859+0000 7ff27659c640 1 -- 172.21.15.33:0/887794634 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556015e52210 msgr2=0x556015e68670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.859+0000 7ff27659c640 1 --2- 172.21.15.33:0/887794634 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556015e52210 0x556015e68670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.859+0000 7ff27659c640 1 -- 172.21.15.33:0/887794634 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556015e5cad0 msgr2=0x556015e5c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.859+0000 7ff27659c640 1 --2- 172.21.15.33:0/887794634 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556015e5cad0 0x556015e5c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.859+0000 7ff27659c640 1 -- 172.21.15.33:0/887794634 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556015e52040 con 0x556015e518f0 2024-05-15T14:09:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.859+0000 7ff276d9d640 1 --2- 172.21.15.33:0/887794634 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556015e5cad0 0x556015e5c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:09:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.859+0000 7ff27659c640 1 --2- 172.21.15.33:0/887794634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556015e518f0 0x556015e53b30 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x556015e6f4f0 tx=0x7ff26801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90d6d559cd491a21 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.860+0000 7ff25ffff640 1 -- 172.21.15.33:0/887794634 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff268003e50 con 0x556015e518f0 2024-05-15T14:09:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.860+0000 7ff25ffff640 1 -- 172.21.15.33:0/887794634 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff268032070 con 0x556015e518f0 2024-05-15T14:09:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.860+0000 7ff25ffff640 1 -- 172.21.15.33:0/887794634 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26801d960 con 0x556015e518f0 2024-05-15T14:09:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.860+0000 7ff27f5c2740 1 -- 172.21.15.33:0/887794634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556015e518f0 msgr2=0x556015e53b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.860+0000 7ff27f5c2740 1 --2- 172.21.15.33:0/887794634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556015e518f0 0x556015e53b30 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x556015e6f4f0 tx=0x7ff26801a000 comp rx=0 tx=0).stop 2024-05-15T14:09:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.860+0000 7ff27f5c2740 1 -- 172.21.15.33:0/887794634 shutdown_connections 2024-05-15T14:09:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.860+0000 7ff27f5c2740 1 --2- 172.21.15.33:0/887794634 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556015e5cad0 0x556015e5c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.860+0000 7ff27f5c2740 1 --2- 172.21.15.33:0/887794634 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556015e52210 0x556015e68670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.860+0000 7ff27f5c2740 1 --2- 172.21.15.33:0/887794634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556015e518f0 0x556015e53b30 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.860+0000 7ff27f5c2740 1 -- 172.21.15.33:0/887794634 >> 172.21.15.33:0/887794634 conn(0x556015e4de00 msgr2=0x556015e4e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.861+0000 7ff27f5c2740 1 -- 172.21.15.33:0/887794634 shutdown_connections 2024-05-15T14:09:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.861+0000 7ff27f5c2740 1 -- 172.21.15.33:0/887794634 wait complete. 2024-05-15T14:09:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.861+0000 7ff27f5c2740 1 -- start start 2024-05-15T14:09:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.861+0000 7ff27f5c2740 1 Processor -- start 2024-05-15T14:09:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.861+0000 7ff27f5c2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556015e52210 0x556015e604f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.861+0000 7ff27f5c2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556015e5cad0 0x556015e60a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.861+0000 7ff27f5c2740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556015e60f70 0x556015ed6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.861+0000 7ff27f5c2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff2781aaa70 con 0x556015e52210 2024-05-15T14:09:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.861+0000 7ff27f5c2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff2781abee0 con 0x556015e5cad0 2024-05-15T14:09:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.861+0000 7ff27f5c2740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ff2781ab140 con 0x556015e60f70 2024-05-15T14:09:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.861+0000 7ff27f5c2740 10 client.0 _ll_register_callbacks cb 0x7ff26c009230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:09:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.861+0000 7ff27659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556015e52210 0x556015e604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.861+0000 7ff27659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556015e52210 0x556015e604f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48966/0 (socket says 172.21.15.33:48966) 2024-05-15T14:09:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.861+0000 7ff27659c640 1 -- 172.21.15.33:0/4150999292 learned_addr learned my addr 172.21.15.33:0/4150999292 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:09:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.861+0000 7ff276d9d640 1 --2- 172.21.15.33:0/4150999292 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556015e60f70 0x556015ed6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.862+0000 7ff275d9b640 1 --2- 172.21.15.33:0/4150999292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556015e5cad0 0x556015e60a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.862+0000 7ff27659c640 1 -- 172.21.15.33:0/4150999292 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556015e60f70 msgr2=0x556015ed6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.862+0000 7ff27659c640 1 --2- 172.21.15.33:0/4150999292 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556015e60f70 0x556015ed6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.862+0000 7ff27659c640 1 -- 172.21.15.33:0/4150999292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556015e5cad0 msgr2=0x556015e60a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:09:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.862+0000 7ff27659c640 1 --2- 172.21.15.33:0/4150999292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556015e5cad0 0x556015e60a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.862+0000 7ff27659c640 1 -- 172.21.15.33:0/4150999292 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556015ed6640 con 0x556015e52210 2024-05-15T14:09:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.862+0000 7ff275d9b640 1 --2- 172.21.15.33:0/4150999292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556015e5cad0 0x556015e60a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:09:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.862+0000 7ff27659c640 1 --2- 172.21.15.33:0/4150999292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556015e52210 0x556015e604f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff268004370 tx=0x7ff268003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b370105536c33e77 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:09:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.862+0000 7ff25dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:09:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.862+0000 7ff25dffb640 1 -- 172.21.15.33:0/4150999292 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff268005450 con 0x556015e52210 2024-05-15T14:09:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.862+0000 7ff27f5c2740 1 -- 172.21.15.33:0/4150999292 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff268032070 con 0x556015e52210 2024-05-15T14:09:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.862+0000 7ff25dffb640 1 -- 172.21.15.33:0/4150999292 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7ff2680055f0 con 0x556015e52210 2024-05-15T14:09:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.862+0000 7ff25dffb640 1 -- 172.21.15.33:0/4150999292 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff2680046c0 con 0x556015e52210 2024-05-15T14:09:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.862+0000 7ff27f5c2740 10 client.16503 fetch_fsmap learned FSMap version 34 2024-05-15T14:09:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.862+0000 7ff27f5c2740 1 -- 172.21.15.33:0/4150999292 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556015ed77d0 con 0x556015e52210 2024-05-15T14:09:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.862+0000 7ff25dffb640 1 -- 172.21.15.33:0/4150999292 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff268032070 con 0x556015e52210 2024-05-15T14:09:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.862+0000 7ff25dffb640 1 -- 172.21.15.33:0/4150999292 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7ff26801db10 con 0x556015e52210 2024-05-15T14:09:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.863+0000 7ff27f5c2740 10 client.16503 fetch_fsmap finished waiting for FSMap version 34 2024-05-15T14:09:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.863+0000 7ff27f5c2740 10 client.16503 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:09:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.863+0000 7ff27f5c2740 20 client.16503 populate_metadata read hostname 'smithi033' 2024-05-15T14:09:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.863+0000 7ff27f5c2740 1 --2- 172.21.15.33:0/4150999292 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556015ed7f10 0x556015eda330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:09:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.863+0000 7ff27f5c2740 4 client.16503 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:09:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.863+0000 7ff27f5c2740 1 -- 172.21.15.33:0/4150999292 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556015dc0360 con 0x556015ed7f10 2024-05-15T14:09:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.863+0000 7ff275d9b640 1 --2- 172.21.15.33:0/4150999292 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556015ed7f10 0x556015eda330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:09:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.863+0000 7ff25dffb640 10 client.16503 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:09:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.863+0000 7ff275d9b640 1 --2- 172.21.15.33:0/4150999292 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556015ed7f10 0x556015eda330 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9474bca5a6499c29 server_cookie=c20ff85144b291f6 in_seq=0 out_seq=0 2024-05-15T14:09:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.926+0000 7ff25dffb640 1 -- 172.21.15.33:0/4150999292 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556015dc0360 con 0x556015ed7f10 2024-05-15T14:09:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.926+0000 7ff25dffb640 10 client.16503 handle_command_reply: tid=0 2024-05-15T14:09:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.926+0000 7ff27f5c2740 1 client.16503 shutdown 2024-05-15T14:09:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.926+0000 7ff27f5c2740 1 -- 172.21.15.33:0/4150999292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556015e52210 msgr2=0x556015e604f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.926+0000 7ff27f5c2740 1 --2- 172.21.15.33:0/4150999292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556015e52210 0x556015e604f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff268004370 tx=0x7ff268003e80 comp rx=0 tx=0).stop 2024-05-15T14:09:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.926+0000 7ff27f5c2740 1 -- 172.21.15.33:0/4150999292 shutdown_connections 2024-05-15T14:09:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.926+0000 7ff27f5c2740 1 --2- 172.21.15.33:0/4150999292 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x556015ed7f10 0x556015eda330 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.926+0000 7ff27f5c2740 1 --2- 172.21.15.33:0/4150999292 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556015e60f70 0x556015ed6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.926+0000 7ff27f5c2740 1 --2- 172.21.15.33:0/4150999292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556015e5cad0 0x556015e60a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.926+0000 7ff27f5c2740 1 --2- 172.21.15.33:0/4150999292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556015e52210 0x556015e604f0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.926+0000 7ff25dffb640 0 client.16503 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:09:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.926+0000 7ff27f5c2740 1 -- 172.21.15.33:0/4150999292 >> 172.21.15.33:0/4150999292 conn(0x556015e4de00 msgr2=0x556015e4d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.926+0000 7ff27f5c2740 1 -- 172.21.15.33:0/4150999292 shutdown_connections 2024-05-15T14:09:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.926+0000 7ff27f5c2740 1 -- 172.21.15.33:0/4150999292 wait complete. 2024-05-15T14:09:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.927+0000 7ff27f5c2740 20 client.16503 trim_cache size 0 max 16384 2024-05-15T14:09:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:09:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:09:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:09:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.927+0000 7ff27c8a0640 1 -- 172.21.15.33:0/580031864 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ff2600b2950 msgr2=0x7ff2600b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.927+0000 7ff27c8a0640 1 --2- 172.21.15.33:0/580031864 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ff2600b2950 0x7ff2600b4e10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x556015e52e90 tx=0x7ff268012040 comp rx=0 tx=0).stop 2024-05-15T14:09:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.927+0000 7ff27c8a0640 1 -- 172.21.15.33:0/580031864 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2781022f0 msgr2=0x7ff2781997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:09:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.927+0000 7ff27c8a0640 1 --2- 172.21.15.33:0/580031864 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2781022f0 0x7ff2781997b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff26c031440 tx=0x7ff26c004270 comp rx=0 tx=0).stop 2024-05-15T14:09:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.927+0000 7ff27c8a0640 1 -- 172.21.15.33:0/580031864 shutdown_connections 2024-05-15T14:09:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.927+0000 7ff27c8a0640 1 --2- 172.21.15.33:0/580031864 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ff2600b2950 0x7ff2600b4e10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.927+0000 7ff27c8a0640 1 --2- 172.21.15.33:0/580031864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff278107940 0x7ff27819dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.927+0000 7ff27c8a0640 1 --2- 172.21.15.33:0/580031864 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2781022f0 0x7ff2781997b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.927+0000 7ff27c8a0640 1 --2- 172.21.15.33:0/580031864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff278101930 0x7ff278199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:09:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.927+0000 7ff27c8a0640 1 -- 172.21.15.33:0/580031864 >> 172.21.15.33:0/580031864 conn(0x7ff278078a70 msgr2=0x7ff27806a650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:09:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.927+0000 7ff27c8a0640 1 -- 172.21.15.33:0/580031864 shutdown_connections 2024-05-15T14:09:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:09:50.928+0000 7ff27c8a0640 1 -- 172.21.15.33:0/580031864 wait complete. 2024-05-15T14:09:51.074 INFO:tasks.workunit.client.0.smithi033.stdout: 1 563845 5.83 MB/sec execute 326 sec latency 42.540 ms 2024-05-15T14:09:52.074 INFO:tasks.workunit.client.0.smithi033.stdout: 1 565511 5.85 MB/sec execute 327 sec latency 13.419 ms 2024-05-15T14:09:52.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:52 smithi033 ceph-mon[31650]: pgmap v296: 97 pgs: 97 active+clean; 175 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 234 op/s 2024-05-15T14:09:52.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:52 smithi052 ceph-mon[36190]: pgmap v296: 97 pgs: 97 active+clean; 175 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 234 op/s 2024-05-15T14:09:52.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:52 smithi081 ceph-mon[36434]: pgmap v296: 97 pgs: 97 active+clean; 175 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 234 op/s 2024-05-15T14:09:53.074 INFO:tasks.workunit.client.0.smithi033.stdout: 1 567508 5.86 MB/sec execute 328 sec latency 25.042 ms 2024-05-15T14:09:54.074 INFO:tasks.workunit.client.0.smithi033.stdout: 1 568775 5.87 MB/sec execute 329 sec latency 260.495 ms 2024-05-15T14:09:54.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:54 smithi033 ceph-mon[31650]: pgmap v297: 97 pgs: 97 active+clean; 172 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.7 MiB/s wr, 367 op/s 2024-05-15T14:09:54.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:54 smithi052 ceph-mon[36190]: pgmap v297: 97 pgs: 97 active+clean; 172 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.7 MiB/s wr, 367 op/s 2024-05-15T14:09:54.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:54 smithi081 ceph-mon[36434]: pgmap v297: 97 pgs: 97 active+clean; 172 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.7 MiB/s wr, 367 op/s 2024-05-15T14:09:55.074 INFO:tasks.workunit.client.0.smithi033.stdout: 1 568775 5.85 MB/sec execute 330 sec latency 1260.584 ms 2024-05-15T14:09:56.074 INFO:tasks.workunit.client.0.smithi033.stdout: 1 568775 5.84 MB/sec execute 331 sec latency 2260.678 ms 2024-05-15T14:09:56.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:56 smithi081 ceph-mon[36434]: pgmap v298: 97 pgs: 97 active+clean; 172 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.6 MiB/s wr, 224 op/s 2024-05-15T14:09:56.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:56 smithi033 ceph-mon[31650]: pgmap v298: 97 pgs: 97 active+clean; 172 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.6 MiB/s wr, 224 op/s 2024-05-15T14:09:56.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:56 smithi052 ceph-mon[36190]: pgmap v298: 97 pgs: 97 active+clean; 172 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.6 MiB/s wr, 224 op/s 2024-05-15T14:09:57.075 INFO:tasks.workunit.client.0.smithi033.stdout: 1 568775 5.82 MB/sec execute 332 sec latency 3260.780 ms 2024-05-15T14:09:58.075 INFO:tasks.workunit.client.0.smithi033.stdout: 1 568775 5.80 MB/sec execute 333 sec latency 4260.895 ms 2024-05-15T14:09:58.627 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 28.5 2024-05-15T14:09:58.628 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T14:09:58.656 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:09:58 smithi033 ceph-mon[31650]: pgmap v299: 97 pgs: 97 active+clean; 174 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 260 op/s 2024-05-15T14:09:58.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:09:58 smithi052 ceph-mon[36190]: pgmap v299: 97 pgs: 97 active+clean; 174 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 260 op/s 2024-05-15T14:09:58.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:09:58 smithi081 ceph-mon[36434]: pgmap v299: 97 pgs: 97 active+clean; 174 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 260 op/s 2024-05-15T14:09:58.897 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:09:59.075 INFO:tasks.workunit.client.0.smithi033.stdout: 1 569067 5.78 MB/sec execute 334 sec latency 5008.377 ms 2024-05-15T14:10:00.075 INFO:tasks.workunit.client.0.smithi033.stdout: 1 571105 5.80 MB/sec execute 335 sec latency 42.099 ms 2024-05-15T14:10:00.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:00 smithi033 ceph-mon[31650]: pgmap v300: 97 pgs: 97 active+clean; 174 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.0 MiB/s wr, 168 op/s 2024-05-15T14:10:00.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:00 smithi033 ceph-mon[31650]: overall HEALTH_OK 2024-05-15T14:10:00.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.744+0000 7fb65eb31700 1 -- 172.21.15.33:0/2307585262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb658102810 msgr2=0x7fb658106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:00.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.744+0000 7fb65eb31700 1 --2- 172.21.15.33:0/2307585262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb658102810 0x7fb658106810 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb644009990 tx=0x7fb644009ca0 comp rx=0 tx=0).stop 2024-05-15T14:10:00.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.744+0000 7fb65eb31700 1 -- 172.21.15.33:0/2307585262 shutdown_connections 2024-05-15T14:10:00.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.744+0000 7fb65eb31700 1 --2- 172.21.15.33:0/2307585262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb658106ef0 0x7fb65810eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:00.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.744+0000 7fb65eb31700 1 --2- 172.21.15.33:0/2307585262 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb658101e60 0x7fb658102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:00.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.744+0000 7fb65eb31700 1 --2- 172.21.15.33:0/2307585262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb658102810 0x7fb658106810 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:00.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.744+0000 7fb65eb31700 1 -- 172.21.15.33:0/2307585262 >> 172.21.15.33:0/2307585262 conn(0x7fb658078ab0 msgr2=0x7fb6580ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:00.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.744+0000 7fb65eb31700 1 -- 172.21.15.33:0/2307585262 shutdown_connections 2024-05-15T14:10:00.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.744+0000 7fb65eb31700 1 -- 172.21.15.33:0/2307585262 wait complete. 2024-05-15T14:10:00.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.745+0000 7fb65eb31700 1 Processor -- start 2024-05-15T14:10:00.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.745+0000 7fb65eb31700 1 -- start start 2024-05-15T14:10:00.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.745+0000 7fb65eb31700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb658101e60 0x7fb65819f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:00.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.745+0000 7fb65eb31700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb658102810 0x7fb65819fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:00.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.745+0000 7fb65eb31700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb658106ef0 0x7fb658199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:00.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.745+0000 7fb65eb31700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb658113720 con 0x7fb658101e60 2024-05-15T14:10:00.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.745+0000 7fb65eb31700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581135a0 con 0x7fb658106ef0 2024-05-15T14:10:00.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.745+0000 7fb65eb31700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fb6581138a0 con 0x7fb658102810 2024-05-15T14:10:00.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.745+0000 7fb65c8cd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb658101e60 0x7fb65819f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:00.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.745+0000 7fb65c8cd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb658101e60 0x7fb65819f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35340/0 (socket says 172.21.15.33:35340) 2024-05-15T14:10:00.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.745+0000 7fb65c8cd700 1 -- 172.21.15.33:0/1734191096 learned_addr learned my addr 172.21.15.33:0/1734191096 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:00.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.745+0000 7fb657fff700 1 --2- 172.21.15.33:0/1734191096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb658102810 0x7fb65819fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:00.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.745+0000 7fb65d0ce700 1 --2- 172.21.15.33:0/1734191096 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb658106ef0 0x7fb658199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:00.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.746+0000 7fb657fff700 1 -- 172.21.15.33:0/1734191096 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb658106ef0 msgr2=0x7fb658199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:00.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.746+0000 7fb657fff700 1 --2- 172.21.15.33:0/1734191096 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb658106ef0 0x7fb658199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:00.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.746+0000 7fb657fff700 1 -- 172.21.15.33:0/1734191096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb658101e60 msgr2=0x7fb65819f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:00.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.746+0000 7fb657fff700 1 --2- 172.21.15.33:0/1734191096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb658101e60 0x7fb65819f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:00.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.746+0000 7fb657fff700 1 -- 172.21.15.33:0/1734191096 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb65819a180 con 0x7fb658102810 2024-05-15T14:10:00.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.746+0000 7fb657fff700 1 --2- 172.21.15.33:0/1734191096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb658102810 0x7fb65819fda0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb6440098e0 tx=0x7fb64400bfc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:00.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.746+0000 7fb655ffb700 1 -- 172.21.15.33:0/1734191096 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb644027070 con 0x7fb658102810 2024-05-15T14:10:00.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.746+0000 7fb65eb31700 1 -- 172.21.15.33:0/1734191096 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb65819a410 con 0x7fb658102810 2024-05-15T14:10:00.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.746+0000 7fb655ffb700 1 -- 172.21.15.33:0/1734191096 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb644022470 con 0x7fb658102810 2024-05-15T14:10:00.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.746+0000 7fb655ffb700 1 -- 172.21.15.33:0/1734191096 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb644017650 con 0x7fb658102810 2024-05-15T14:10:00.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.746+0000 7fb65eb31700 1 -- 172.21.15.33:0/1734191096 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb658110110 con 0x7fb658102810 2024-05-15T14:10:00.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.748+0000 7fb65eb31700 1 -- 172.21.15.33:0/1734191096 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6581032f0 con 0x7fb658102810 2024-05-15T14:10:00.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.749+0000 7fb655ffb700 1 -- 172.21.15.33:0/1734191096 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb64402a070 con 0x7fb658102810 2024-05-15T14:10:00.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.750+0000 7fb655ffb700 1 --2- 172.21.15.33:0/1734191096 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fb6480b29f0 0x7fb6480b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:00.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.750+0000 7fb65c8cd700 1 --2- 172.21.15.33:0/1734191096 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fb6480b29f0 0x7fb6480b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:00.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.750+0000 7fb655ffb700 1 -- 172.21.15.33:0/1734191096 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fb6440e3600 con 0x7fb658102810 2024-05-15T14:10:00.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.750+0000 7fb65c8cd700 1 --2- 172.21.15.33:0/1734191096 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fb6480b29f0 0x7fb6480b4eb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb65819ab60 tx=0x7fb64c009260 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:00.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:00.753+0000 7fb655ffb700 1 -- 172.21.15.33:0/1734191096 <== mon.1 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb644013070 con 0x7fb658102810 2024-05-15T14:10:00.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:00 smithi052 ceph-mon[36190]: pgmap v300: 97 pgs: 97 active+clean; 174 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.0 MiB/s wr, 168 op/s 2024-05-15T14:10:00.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:00 smithi052 ceph-mon[36190]: overall HEALTH_OK 2024-05-15T14:10:00.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:00 smithi081 ceph-mon[36434]: pgmap v300: 97 pgs: 97 active+clean; 174 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.0 MiB/s wr, 168 op/s 2024-05-15T14:10:00.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:00 smithi081 ceph-mon[36434]: overall HEALTH_OK 2024-05-15T14:10:01.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:01.061+0000 7fb65eb31700 1 -- 172.21.15.33:0/1734191096 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb65806a230 con 0x7fb658102810 2024-05-15T14:10:01.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:01.062+0000 7fb655ffb700 1 -- 172.21.15.33:0/1734191096 <== mon.1 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11862 (secure 0 0 0) 0x7fb6440ab060 con 0x7fb658102810 2024-05-15T14:10:01.063 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T14:10:01.063 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2030117633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2030117633},{"type":"v1","addr":"172.21.15.33:6839","nonce":2030117633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1918979061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1918979061},{"type":"v1","addr":"172.21.15.33:6841","nonce":1918979061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T13:57:19.056417+0000","modified":"2024-05-15T14:08:23.686464+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24463,"mds_2":24461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/1629697841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1629697841},{"type":"v1","addr":"172.21.15.52:6837","nonce":1629697841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.52:6839/2696044471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2696044471},{"type":"v1","addr":"172.21.15.52:6839","nonce":2696044471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.81:6839/1554395940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":1554395940},{"type":"v1","addr":"172.21.15.81:6839","nonce":1554395940}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25915":{"gid":25915,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/1297849376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1297849376},{"type":"v1","addr":"172.21.15.81:6835","nonce":1297849376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24461,24463,24478]},"id":1}]} 2024-05-15T14:10:01.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:01.064+0000 7fb65eb31700 1 -- 172.21.15.33:0/1734191096 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fb6480b29f0 msgr2=0x7fb6480b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:01.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:01.064+0000 7fb65eb31700 1 --2- 172.21.15.33:0/1734191096 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fb6480b29f0 0x7fb6480b4eb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb65819ab60 tx=0x7fb64c009260 comp rx=0 tx=0).stop 2024-05-15T14:10:01.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:01.064+0000 7fb65eb31700 1 -- 172.21.15.33:0/1734191096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb658102810 msgr2=0x7fb65819fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:01.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:01.064+0000 7fb65eb31700 1 --2- 172.21.15.33:0/1734191096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb658102810 0x7fb65819fda0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb6440098e0 tx=0x7fb64400bfc0 comp rx=0 tx=0).stop 2024-05-15T14:10:01.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:01.064+0000 7fb65eb31700 1 -- 172.21.15.33:0/1734191096 shutdown_connections 2024-05-15T14:10:01.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:01.064+0000 7fb65eb31700 1 --2- 172.21.15.33:0/1734191096 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fb6480b29f0 0x7fb6480b4eb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:01.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:01.064+0000 7fb65eb31700 1 --2- 172.21.15.33:0/1734191096 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb658106ef0 0x7fb658199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:01.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:01.064+0000 7fb65eb31700 1 --2- 172.21.15.33:0/1734191096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb658101e60 0x7fb65819f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:01.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:01.064+0000 7fb65eb31700 1 --2- 172.21.15.33:0/1734191096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb658102810 0x7fb65819fda0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:01.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:01.064+0000 7fb65eb31700 1 -- 172.21.15.33:0/1734191096 >> 172.21.15.33:0/1734191096 conn(0x7fb658078ab0 msgr2=0x7fb6580ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:01.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:01.064+0000 7fb65eb31700 1 -- 172.21.15.33:0/1734191096 shutdown_connections 2024-05-15T14:10:01.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:01.064+0000 7fb65eb31700 1 -- 172.21.15.33:0/1734191096 wait complete. 2024-05-15T14:10:01.067 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 34 2024-05-15T14:10:01.075 INFO:tasks.workunit.client.0.smithi033.stdout: 1 572267 5.81 MB/sec execute 336 sec latency 55.228 ms 2024-05-15T14:10:01.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:01 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/1734191096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:10:01.732 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24478 quiesce db / --timeout 90 --expiration 148.5 --await-for 45.0' 2024-05-15T14:10:01.732 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24478 quiesce db / --timeout 90 --expiration 148.5 --await-for 45.0 2024-05-15T14:10:01.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:01 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/1734191096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:10:01.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:01 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/1734191096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:10:02.044 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:10:02.075 INFO:tasks.workunit.client.0.smithi033.stdout: 1 573651 5.80 MB/sec execute 337 sec latency 24.667 ms 2024-05-15T14:10:02.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:02 smithi033 ceph-mon[31650]: pgmap v301: 97 pgs: 97 active+clean; 174 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.0 MiB/s wr, 168 op/s 2024-05-15T14:10:02.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:02 smithi052 ceph-mon[36190]: pgmap v301: 97 pgs: 97 active+clean; 174 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.0 MiB/s wr, 168 op/s 2024-05-15T14:10:02.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:02 smithi081 ceph-mon[36434]: pgmap v301: 97 pgs: 97 active+clean; 174 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.0 MiB/s wr, 168 op/s 2024-05-15T14:10:03.075 INFO:tasks.workunit.client.0.smithi033.stdout: 1 575772 5.83 MB/sec execute 338 sec latency 13.303 ms 2024-05-15T14:10:03.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.909+0000 7f7467f1a700 1 -- 172.21.15.33:0/785944501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f746006b480 msgr2=0x7f746006b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:03.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.909+0000 7f7467f1a700 1 --2- 172.21.15.33:0/785944501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f746006b480 0x7f746006b860 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f745000b0d0 tx=0x7f745000b3e0 comp rx=0 tx=0).stop 2024-05-15T14:10:03.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.909+0000 7f7467f1a700 1 -- 172.21.15.33:0/785944501 shutdown_connections 2024-05-15T14:10:03.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.909+0000 7f7467f1a700 1 --2- 172.21.15.33:0/785944501 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7460108900 0x7f7460110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:03.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.909+0000 7f7467f1a700 1 --2- 172.21.15.33:0/785944501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f746006b480 0x7f746006b860 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:03.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.909+0000 7f7467f1a700 1 --2- 172.21.15.33:0/785944501 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f746006be30 0x7f74601081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:03.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.909+0000 7f7467f1a700 1 -- 172.21.15.33:0/785944501 >> 172.21.15.33:0/785944501 conn(0x7f74600793e0 msgr2=0x7f74600797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:03.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.909+0000 7f7467f1a700 1 -- 172.21.15.33:0/785944501 shutdown_connections 2024-05-15T14:10:03.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.910+0000 7f7467f1a700 1 -- 172.21.15.33:0/785944501 wait complete. 2024-05-15T14:10:03.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.910+0000 7f7467f1a700 1 Processor -- start 2024-05-15T14:10:03.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.910+0000 7f7467f1a700 1 -- start start 2024-05-15T14:10:03.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.911+0000 7f7467f1a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f746006b480 0x7f7460102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:03.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.911+0000 7f7467f1a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f746006be30 0x7f7460102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:03.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.911+0000 7f7467f1a700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7460108900 0x7f7460102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:03.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.911+0000 7f7467f1a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7460113700 con 0x7f746006b480 2024-05-15T14:10:03.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.911+0000 7f7467f1a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7460113580 con 0x7f746006be30 2024-05-15T14:10:03.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.911+0000 7f7467f1a700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f7460113880 con 0x7f7460108900 2024-05-15T14:10:03.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.911+0000 7f7465cb6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f746006b480 0x7f7460102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:03.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.911+0000 7f7465cb6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f746006b480 0x7f7460102510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43018/0 (socket says 172.21.15.33:43018) 2024-05-15T14:10:03.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.911+0000 7f7465cb6700 1 -- 172.21.15.33:0/2921549393 learned_addr learned my addr 172.21.15.33:0/2921549393 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:03.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.911+0000 7f74654b5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f746006be30 0x7f7460102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:03.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.911+0000 7f74664b7700 1 --2- 172.21.15.33:0/2921549393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7460108900 0x7f7460102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:03.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.911+0000 7f7465cb6700 1 -- 172.21.15.33:0/2921549393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7460108900 msgr2=0x7f7460102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:03.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.911+0000 7f7465cb6700 1 --2- 172.21.15.33:0/2921549393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7460108900 0x7f7460102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:03.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.911+0000 7f7465cb6700 1 -- 172.21.15.33:0/2921549393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f746006be30 msgr2=0x7f7460102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:03.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.911+0000 7f7465cb6700 1 --2- 172.21.15.33:0/2921549393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f746006be30 0x7f7460102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:03.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.911+0000 7f7465cb6700 1 -- 172.21.15.33:0/2921549393 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74601a7fa0 con 0x7f746006b480 2024-05-15T14:10:03.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.911+0000 7f74654b5700 1 --2- 172.21.15.33:0/2921549393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f746006be30 0x7f7460102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:10:03.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.911+0000 7f7465cb6700 1 --2- 172.21.15.33:0/2921549393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f746006b480 0x7f7460102510 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f7450007a20 tx=0x7f7450007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:03.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.911+0000 7f745effd700 1 -- 172.21.15.33:0/2921549393 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745001f070 con 0x7f746006b480 2024-05-15T14:10:03.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.911+0000 7f745effd700 1 -- 172.21.15.33:0/2921549393 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7450004c50 con 0x7f746006b480 2024-05-15T14:10:03.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.911+0000 7f7467f1a700 1 -- 172.21.15.33:0/2921549393 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74601a8230 con 0x7f746006b480 2024-05-15T14:10:03.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.911+0000 7f745effd700 1 -- 172.21.15.33:0/2921549393 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745001b690 con 0x7f746006b480 2024-05-15T14:10:03.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.912+0000 7f7467f1a700 1 -- 172.21.15.33:0/2921549393 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74601a8680 con 0x7f746006b480 2024-05-15T14:10:03.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.915+0000 7f745effd700 1 -- 172.21.15.33:0/2921549393 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7450022070 con 0x7f746006b480 2024-05-15T14:10:03.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.916+0000 7f745effd700 1 --2- 172.21.15.33:0/2921549393 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f74480b2aa0 0x7f74480b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:03.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.916+0000 7f74654b5700 1 --2- 172.21.15.33:0/2921549393 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f74480b2aa0 0x7f74480b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:03.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.916+0000 7f745effd700 1 -- 172.21.15.33:0/2921549393 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f745000f020 con 0x7f746006b480 2024-05-15T14:10:03.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.916+0000 7f74727b0b80 1 Processor -- start 2024-05-15T14:10:03.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.916+0000 7f74727b0b80 1 -- start start 2024-05-15T14:10:03.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.916+0000 7f74654b5700 1 --2- 172.21.15.33:0/2921549393 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f74480b2aa0 0x7f74480b4f60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7460109be0 tx=0x7f7454009260 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:03.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.916+0000 7f74727b0b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ec5c47e30 0x555ec5c48210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:03.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.916+0000 7f74727b0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ec5c48750 0x555ec5c42540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:03.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.917+0000 7f74727b0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555ec5c42b60 0x555ec5c42fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:03.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.917+0000 7f7465cb6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ec5c47e30 0x555ec5c48210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:03.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.917+0000 7f7465cb6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ec5c47e30 0x555ec5c48210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43034/0 (socket says 172.21.15.33:43034) 2024-05-15T14:10:03.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.917+0000 7f7465cb6700 1 -- 172.21.15.33:0/2519404830 learned_addr learned my addr 172.21.15.33:0/2519404830 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:03.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.917+0000 7f74727b0b80 1 -- 172.21.15.33:0/2519404830 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555ec5ba0e20 con 0x555ec5c47e30 2024-05-15T14:10:03.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.917+0000 7f74727b0b80 1 -- 172.21.15.33:0/2519404830 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555ec5b909c0 con 0x555ec5c48750 2024-05-15T14:10:03.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.917+0000 7f74727b0b80 1 -- 172.21.15.33:0/2519404830 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x555ec5b94a00 con 0x555ec5c42b60 2024-05-15T14:10:03.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.917+0000 7f74664b7700 1 --2- 172.21.15.33:0/2519404830 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555ec5c42b60 0x555ec5c42fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:03.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.917+0000 7f74654b5700 1 --2- 172.21.15.33:0/2519404830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ec5c48750 0x555ec5c42540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:03.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.917+0000 7f7465cb6700 1 -- 172.21.15.33:0/2519404830 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555ec5c42b60 msgr2=0x555ec5c42fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:03.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.917+0000 7f7465cb6700 1 --2- 172.21.15.33:0/2519404830 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555ec5c42b60 0x555ec5c42fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:03.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.917+0000 7f7465cb6700 1 -- 172.21.15.33:0/2519404830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ec5c48750 msgr2=0x555ec5c42540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:03.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.917+0000 7f7465cb6700 1 --2- 172.21.15.33:0/2519404830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ec5c48750 0x555ec5c42540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:03.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.917+0000 7f7465cb6700 1 -- 172.21.15.33:0/2519404830 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ec5ae0a60 con 0x555ec5c47e30 2024-05-15T14:10:03.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.917+0000 7f7465cb6700 1 --2- 172.21.15.33:0/2519404830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ec5c47e30 0x555ec5c48210 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f74500a6aa0 tx=0x7f7450007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c7cbf27b1e128d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:03.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.917+0000 7f7447fff700 1 -- 172.21.15.33:0/2519404830 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74500ab120 con 0x555ec5c47e30 2024-05-15T14:10:03.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.917+0000 7f7447fff700 1 -- 172.21.15.33:0/2519404830 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7450009cb0 con 0x555ec5c47e30 2024-05-15T14:10:03.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.917+0000 7f7447fff700 1 -- 172.21.15.33:0/2519404830 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74500cd0b0 con 0x555ec5c47e30 2024-05-15T14:10:03.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.918+0000 7f74727b0b80 1 -- 172.21.15.33:0/2519404830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ec5c47e30 msgr2=0x555ec5c48210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:03.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.918+0000 7f74727b0b80 1 --2- 172.21.15.33:0/2519404830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ec5c47e30 0x555ec5c48210 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f74500a6aa0 tx=0x7f7450007ec0 comp rx=0 tx=0).stop 2024-05-15T14:10:03.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.918+0000 7f74727b0b80 1 -- 172.21.15.33:0/2519404830 shutdown_connections 2024-05-15T14:10:03.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.918+0000 7f74727b0b80 1 --2- 172.21.15.33:0/2519404830 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555ec5c42b60 0x555ec5c42fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:03.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.918+0000 7f74727b0b80 1 --2- 172.21.15.33:0/2519404830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ec5c47e30 0x555ec5c48210 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:03.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.918+0000 7f74727b0b80 1 --2- 172.21.15.33:0/2519404830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ec5c48750 0x555ec5c42540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:03.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.918+0000 7f74727b0b80 1 -- 172.21.15.33:0/2519404830 >> 172.21.15.33:0/2519404830 conn(0x555ec5b6d3e0 msgr2=0x555ec5c3cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:03.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.918+0000 7f74727b0b80 1 -- 172.21.15.33:0/2519404830 shutdown_connections 2024-05-15T14:10:03.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.918+0000 7f74727b0b80 1 -- 172.21.15.33:0/2519404830 wait complete. 2024-05-15T14:10:03.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.919+0000 7f74727b0b80 1 -- start start 2024-05-15T14:10:03.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.919+0000 7f74727b0b80 1 Processor -- start 2024-05-15T14:10:03.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.919+0000 7f74727b0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555ec5c42b60 0x555ec5c4e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:03.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.919+0000 7f74727b0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ec5c47e30 0x555ec5c4b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:03.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.919+0000 7f74727b0b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ec5c48750 0x555ec5c4bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:03.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.919+0000 7f74727b0b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555ec5a97130 con 0x555ec5c48750 2024-05-15T14:10:03.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.919+0000 7f74727b0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555ec5b32950 con 0x555ec5c47e30 2024-05-15T14:10:03.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.919+0000 7f74727b0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x555ec5c4ef50 con 0x555ec5c42b60 2024-05-15T14:10:03.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.919+0000 7f74727b0b80 10 client.0 _ll_register_callbacks cb 0x555ec5b27710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:10:03.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.919+0000 7f74664b7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ec5c48750 0x555ec5c4bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:03.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.919+0000 7f7465cb6700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555ec5c42b60 0x555ec5c4e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:03.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.920+0000 7f74664b7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ec5c48750 0x555ec5c4bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43036/0 (socket says 172.21.15.33:43036) 2024-05-15T14:10:03.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.920+0000 7f74664b7700 1 -- 172.21.15.33:0/2147944019 learned_addr learned my addr 172.21.15.33:0/2147944019 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:03.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.920+0000 7f74654b5700 1 --2- 172.21.15.33:0/2147944019 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ec5c47e30 0x555ec5c4b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:03.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.920+0000 7f74664b7700 1 -- 172.21.15.33:0/2147944019 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555ec5c42b60 msgr2=0x555ec5c4e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:03.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.920+0000 7f74664b7700 1 --2- 172.21.15.33:0/2147944019 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555ec5c42b60 0x555ec5c4e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:03.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.920+0000 7f74664b7700 1 -- 172.21.15.33:0/2147944019 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ec5c47e30 msgr2=0x555ec5c4b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:03.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.920+0000 7f74664b7700 1 --2- 172.21.15.33:0/2147944019 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ec5c47e30 0x555ec5c4b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:03.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.920+0000 7f74664b7700 1 -- 172.21.15.33:0/2147944019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ec5b2f820 con 0x555ec5c48750 2024-05-15T14:10:03.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.920+0000 7f74664b7700 1 --2- 172.21.15.33:0/2147944019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ec5c48750 0x555ec5c4bd90 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f7458010a20 tx=0x7f7458010d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82ffb394b70dbd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:03.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.920+0000 7f74457fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:10:03.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.920+0000 7f74457fa700 1 -- 172.21.15.33:0/2147944019 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745800ae30 con 0x555ec5c48750 2024-05-15T14:10:03.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.920+0000 7f74727b0b80 1 -- 172.21.15.33:0/2147944019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7450009cb0 con 0x555ec5c48750 2024-05-15T14:10:03.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.920+0000 7f74457fa700 1 -- 172.21.15.33:0/2147944019 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f745800faa0 con 0x555ec5c48750 2024-05-15T14:10:03.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.920+0000 7f74457fa700 1 -- 172.21.15.33:0/2147944019 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f745800fca0 con 0x555ec5c48750 2024-05-15T14:10:03.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.920+0000 7f74727b0b80 10 client.16521 fetch_fsmap learned FSMap version 34 2024-05-15T14:10:03.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.920+0000 7f74727b0b80 1 -- 172.21.15.33:0/2147944019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555ec5c59690 con 0x555ec5c48750 2024-05-15T14:10:03.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.921+0000 7f74457fa700 1 -- 172.21.15.33:0/2147944019 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7450009cb0 con 0x555ec5c48750 2024-05-15T14:10:03.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.921+0000 7f74457fa700 1 -- 172.21.15.33:0/2147944019 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f74580053c0 con 0x555ec5c48750 2024-05-15T14:10:03.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.921+0000 7f74727b0b80 10 client.16521 fetch_fsmap finished waiting for FSMap version 34 2024-05-15T14:10:03.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.921+0000 7f74727b0b80 10 client.16521 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:10:03.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.921+0000 7f74727b0b80 20 client.16521 populate_metadata read hostname 'smithi033' 2024-05-15T14:10:03.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.921+0000 7f74727b0b80 1 --2- 172.21.15.33:0/2147944019 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x555ec5c5bd20 0x555ec5c5e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:03.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.921+0000 7f74727b0b80 4 client.16521 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:10:03.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.921+0000 7f74727b0b80 1 -- 172.21.15.33:0/2147944019 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ec5c0e4f0 con 0x555ec5c5bd20 2024-05-15T14:10:03.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.921+0000 7f7465cb6700 1 --2- 172.21.15.33:0/2147944019 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x555ec5c5bd20 0x555ec5c5e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:03.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.922+0000 7f7465cb6700 1 --2- 172.21.15.33:0/2147944019 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x555ec5c5bd20 0x555ec5c5e140 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=feec01bdf520928b server_cookie=927fbc0636260f32 in_seq=0 out_seq=0 2024-05-15T14:10:03.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.922+0000 7f74457fa700 10 client.16521 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:10:03.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.922+0000 7f74457fa700 1 -- 172.21.15.33:0/2147944019 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555ec5c0e4f0 con 0x555ec5c5bd20 2024-05-15T14:10:03.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.922+0000 7f74457fa700 10 client.16521 handle_command_reply: tid=0 2024-05-15T14:10:03.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.922+0000 7f74727b0b80 1 client.16521 shutdown 2024-05-15T14:10:03.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.923+0000 7f74727b0b80 1 -- 172.21.15.33:0/2147944019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ec5c48750 msgr2=0x555ec5c4bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:03.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.923+0000 7f74727b0b80 1 --2- 172.21.15.33:0/2147944019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ec5c48750 0x555ec5c4bd90 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f7458010a20 tx=0x7f7458010d30 comp rx=0 tx=0).stop 2024-05-15T14:10:03.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.923+0000 7f74727b0b80 1 -- 172.21.15.33:0/2147944019 shutdown_connections 2024-05-15T14:10:03.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.923+0000 7f74727b0b80 1 --2- 172.21.15.33:0/2147944019 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x555ec5c5bd20 0x555ec5c5e140 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:03.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.923+0000 7f74727b0b80 1 --2- 172.21.15.33:0/2147944019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ec5c48750 0x555ec5c4bd90 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:03.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.923+0000 7f74727b0b80 1 --2- 172.21.15.33:0/2147944019 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555ec5c42b60 0x555ec5c4e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:03.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.923+0000 7f74727b0b80 1 --2- 172.21.15.33:0/2147944019 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ec5c47e30 0x555ec5c4b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:03.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.923+0000 7f74727b0b80 1 -- 172.21.15.33:0/2147944019 >> 172.21.15.33:0/2147944019 conn(0x555ec5b6d3e0 msgr2=0x555ec5c37860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:03.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.923+0000 7f74457fa700 0 client.16521 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:10:03.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.923+0000 7f74727b0b80 1 -- 172.21.15.33:0/2147944019 shutdown_connections 2024-05-15T14:10:03.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.923+0000 7f74727b0b80 1 -- 172.21.15.33:0/2147944019 wait complete. 2024-05-15T14:10:03.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.923+0000 7f74727b0b80 20 client.16521 trim_cache size 0 max 16384 2024-05-15T14:10:03.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.944+0000 7f74727b0b80 1 Processor -- start 2024-05-15T14:10:03.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.944+0000 7f74727b0b80 1 -- start start 2024-05-15T14:10:03.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.945+0000 7f74727b0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ec5c42b60 0x555ec5c4bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:03.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.945+0000 7f74727b0b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ec5c47e30 0x555ec5c4b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:03.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.945+0000 7f74727b0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555ec5c48750 0x555ec5c37970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:03.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.945+0000 7f74727b0b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f74601b2810 con 0x555ec5c47e30 2024-05-15T14:10:03.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.945+0000 7f74727b0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f74601b0010 con 0x555ec5c42b60 2024-05-15T14:10:03.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.945+0000 7f74727b0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f74601ade80 con 0x555ec5c48750 2024-05-15T14:10:03.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.945+0000 7f74654b5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ec5c47e30 0x555ec5c4b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:03.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.945+0000 7f74654b5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ec5c47e30 0x555ec5c4b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43046/0 (socket says 172.21.15.33:43046) 2024-05-15T14:10:03.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.945+0000 7f74654b5700 1 -- 172.21.15.33:0/2575498062 learned_addr learned my addr 172.21.15.33:0/2575498062 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:03.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.945+0000 7f74664b7700 1 --2- 172.21.15.33:0/2575498062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555ec5c48750 0x555ec5c37970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:03.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.945+0000 7f7465cb6700 1 --2- 172.21.15.33:0/2575498062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ec5c42b60 0x555ec5c4bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:03.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.945+0000 7f74654b5700 1 -- 172.21.15.33:0/2575498062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555ec5c48750 msgr2=0x555ec5c37970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:03.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.945+0000 7f74654b5700 1 --2- 172.21.15.33:0/2575498062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555ec5c48750 0x555ec5c37970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:03.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.945+0000 7f74654b5700 1 -- 172.21.15.33:0/2575498062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ec5c42b60 msgr2=0x555ec5c4bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:03.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.945+0000 7f74654b5700 1 --2- 172.21.15.33:0/2575498062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ec5c42b60 0x555ec5c4bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:03.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.945+0000 7f74654b5700 1 -- 172.21.15.33:0/2575498062 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ec5c5e7c0 con 0x555ec5c47e30 2024-05-15T14:10:03.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.946+0000 7f74654b5700 1 --2- 172.21.15.33:0/2575498062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ec5c47e30 0x555ec5c4b170 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f7454004350 tx=0x7f7454014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745d317553412c12 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:03.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.946+0000 7f74477fe700 1 -- 172.21.15.33:0/2575498062 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7454003b50 con 0x555ec5c47e30 2024-05-15T14:10:03.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.946+0000 7f74477fe700 1 -- 172.21.15.33:0/2575498062 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7454003cf0 con 0x555ec5c47e30 2024-05-15T14:10:03.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.946+0000 7f74477fe700 1 -- 172.21.15.33:0/2575498062 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7454028620 con 0x555ec5c47e30 2024-05-15T14:10:03.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.947+0000 7f74727b0b80 1 -- 172.21.15.33:0/2575498062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ec5c47e30 msgr2=0x555ec5c4b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:03.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.947+0000 7f74727b0b80 1 --2- 172.21.15.33:0/2575498062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ec5c47e30 0x555ec5c4b170 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f7454004350 tx=0x7f7454014000 comp rx=0 tx=0).stop 2024-05-15T14:10:03.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.947+0000 7f74727b0b80 1 -- 172.21.15.33:0/2575498062 shutdown_connections 2024-05-15T14:10:03.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.947+0000 7f74727b0b80 1 --2- 172.21.15.33:0/2575498062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555ec5c48750 0x555ec5c37970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:03.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.947+0000 7f74727b0b80 1 --2- 172.21.15.33:0/2575498062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ec5c42b60 0x555ec5c4bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:03.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.947+0000 7f74727b0b80 1 --2- 172.21.15.33:0/2575498062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ec5c47e30 0x555ec5c4b170 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:03.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.947+0000 7f74727b0b80 1 -- 172.21.15.33:0/2575498062 >> 172.21.15.33:0/2575498062 conn(0x555ec5b6d3e0 msgr2=0x555ec5c4d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:03.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.947+0000 7f74727b0b80 1 -- 172.21.15.33:0/2575498062 shutdown_connections 2024-05-15T14:10:03.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.947+0000 7f74727b0b80 1 -- 172.21.15.33:0/2575498062 wait complete. 2024-05-15T14:10:03.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.947+0000 7f74727b0b80 1 -- start start 2024-05-15T14:10:03.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.947+0000 7f74727b0b80 1 Processor -- start 2024-05-15T14:10:03.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.948+0000 7f74727b0b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555ec5c42b60 0x555ec5c51170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:03.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.948+0000 7f74727b0b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ec5c47e30 0x555ec5c516b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:03.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.948+0000 7f74727b0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ec5c48750 0x555ec5cc65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:03.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.948+0000 7f74727b0b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f74601ae4d0 con 0x555ec5c47e30 2024-05-15T14:10:03.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.948+0000 7f74727b0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f74601af940 con 0x555ec5c48750 2024-05-15T14:10:03.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.948+0000 7f74727b0b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f74601aeba0 con 0x555ec5c42b60 2024-05-15T14:10:03.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.948+0000 7f74727b0b80 10 client.0 _ll_register_callbacks cb 0x555ec5b27710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:10:03.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.948+0000 7f74654b5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ec5c47e30 0x555ec5c516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:03.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.948+0000 7f74654b5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ec5c47e30 0x555ec5c516b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43052/0 (socket says 172.21.15.33:43052) 2024-05-15T14:10:03.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.948+0000 7f74654b5700 1 -- 172.21.15.33:0/2414766660 learned_addr learned my addr 172.21.15.33:0/2414766660 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:03.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.948+0000 7f7465cb6700 1 --2- 172.21.15.33:0/2414766660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555ec5c42b60 0x555ec5c51170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:03.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.948+0000 7f74664b7700 1 --2- 172.21.15.33:0/2414766660 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ec5c48750 0x555ec5cc65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:03.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.948+0000 7f74664b7700 1 -- 172.21.15.33:0/2414766660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555ec5c42b60 msgr2=0x555ec5c51170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:03.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.948+0000 7f74664b7700 1 --2- 172.21.15.33:0/2414766660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555ec5c42b60 0x555ec5c51170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:03.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.948+0000 7f74664b7700 1 -- 172.21.15.33:0/2414766660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ec5c47e30 msgr2=0x555ec5c516b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:03.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.948+0000 7f74664b7700 1 --2- 172.21.15.33:0/2414766660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ec5c47e30 0x555ec5c516b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:03.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.948+0000 7f74664b7700 1 -- 172.21.15.33:0/2414766660 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ec5cc6b10 con 0x555ec5c48750 2024-05-15T14:10:03.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.949+0000 7f74664b7700 1 --2- 172.21.15.33:0/2414766660 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ec5c48750 0x555ec5cc65d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7458005760 tx=0x7f7458022cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=131416064b14069a server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:03.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.949+0000 7f74457fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-05-15T14:10:03.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.949+0000 7f74457fa700 1 -- 172.21.15.33:0/2414766660 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7458030060 con 0x555ec5c48750 2024-05-15T14:10:03.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.949+0000 7f74727b0b80 1 -- 172.21.15.33:0/2414766660 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7454003cf0 con 0x555ec5c48750 2024-05-15T14:10:03.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.949+0000 7f74457fa700 1 -- 172.21.15.33:0/2414766660 <== mon.2 v2:172.21.15.52:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f745800faa0 con 0x555ec5c48750 2024-05-15T14:10:03.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.949+0000 7f74457fa700 1 -- 172.21.15.33:0/2414766660 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f745800a340 con 0x555ec5c48750 2024-05-15T14:10:03.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.949+0000 7f74727b0b80 10 client.25979 fetch_fsmap learned FSMap version 34 2024-05-15T14:10:03.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.949+0000 7f74727b0b80 1 -- 172.21.15.33:0/2414766660 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555ec5cc7cb0 con 0x555ec5c48750 2024-05-15T14:10:03.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.949+0000 7f74457fa700 1 -- 172.21.15.33:0/2414766660 <== mon.2 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7454003cf0 con 0x555ec5c48750 2024-05-15T14:10:03.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.949+0000 7f74457fa700 1 -- 172.21.15.33:0/2414766660 <== mon.2 v2:172.21.15.52:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f7458003a40 con 0x555ec5c48750 2024-05-15T14:10:03.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.949+0000 7f74727b0b80 10 client.25979 fetch_fsmap finished waiting for FSMap version 34 2024-05-15T14:10:03.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.949+0000 7f74727b0b80 10 client.25979 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:10:03.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.949+0000 7f74727b0b80 20 client.25979 populate_metadata read hostname 'smithi033' 2024-05-15T14:10:03.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.949+0000 7f74727b0b80 1 --2- 172.21.15.33:0/2414766660 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x555ec5cc8430 0x555ec5cca850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:03.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.949+0000 7f74727b0b80 4 client.25979 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 148.5, "await_for": 45.0}] 2024-05-15T14:10:03.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.949+0000 7f74727b0b80 1 -- 172.21.15.33:0/2414766660 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 148.5, "await_for": 45.0}) -- 0x555ec5ba2d70 con 0x555ec5cc8430 2024-05-15T14:10:03.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.950+0000 7f7465cb6700 1 --2- 172.21.15.33:0/2414766660 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x555ec5cc8430 0x555ec5cca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:03.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.950+0000 7f7465cb6700 1 --2- 172.21.15.33:0/2414766660 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x555ec5cc8430 0x555ec5cca850 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=713d7935b0cd265d server_cookie=283f65e61378d311 in_seq=0 out_seq=0 2024-05-15T14:10:03.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:03.950+0000 7f74457fa700 10 client.25979 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:10:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.025+0000 7f3ae4445640 1 -- 172.21.15.33:0/3972229408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3adc102a50 msgr2=0x7f3adc102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.025+0000 7f3ae4445640 1 --2- 172.21.15.33:0/3972229408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3adc102a50 0x7f3adc102ed0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3acc009930 tx=0x7f3acc02a420 comp rx=0 tx=0).stop 2024-05-15T14:10:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.025+0000 7f3ae4445640 1 -- 172.21.15.33:0/3972229408 shutdown_connections 2024-05-15T14:10:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.025+0000 7f3ae4445640 1 --2- 172.21.15.33:0/3972229408 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3adc103410 0x7f3adc10f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.025+0000 7f3ae4445640 1 --2- 172.21.15.33:0/3972229408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3adc102a50 0x7f3adc102ed0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.025+0000 7f3ae4445640 1 --2- 172.21.15.33:0/3972229408 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3adc108a60 0x7f3adc108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.025+0000 7f3ae4445640 1 -- 172.21.15.33:0/3972229408 >> 172.21.15.33:0/3972229408 conn(0x7f3adc0fe540 msgr2=0x7f3adc100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.025+0000 7f3ae4445640 1 -- 172.21.15.33:0/3972229408 shutdown_connections 2024-05-15T14:10:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.025+0000 7f3ae4445640 1 -- 172.21.15.33:0/3972229408 wait complete. 2024-05-15T14:10:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.026+0000 7f3ae4445640 1 Processor -- start 2024-05-15T14:10:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.026+0000 7f3ae4445640 1 -- start start 2024-05-15T14:10:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.026+0000 7f3ae4445640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3adc102a50 0x7f3adc1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.026+0000 7f3ae4445640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3adc103410 0x7f3adc1997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.026+0000 7f3ae4445640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3adc108a60 0x7f3adc19db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.026+0000 7f3ae4445640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc112d20 con 0x7f3adc108a60 2024-05-15T14:10:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.026+0000 7f3ae4445640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc112ba0 con 0x7f3adc102a50 2024-05-15T14:10:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.026+0000 7f3ae4445640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc112ea0 con 0x7f3adc103410 2024-05-15T14:10:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.026+0000 7f3ae29e2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3adc108a60 0x7f3adc19db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.026+0000 7f3ae19e0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3adc103410 0x7f3adc1997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.026+0000 7f3ae29e2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3adc108a60 0x7f3adc19db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43066/0 (socket says 172.21.15.33:43066) 2024-05-15T14:10:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.026+0000 7f3ae29e2640 1 -- 172.21.15.33:0/853861266 learned_addr learned my addr 172.21.15.33:0/853861266 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.026+0000 7f3ae21e1640 1 --2- 172.21.15.33:0/853861266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3adc102a50 0x7f3adc1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.026+0000 7f3ae29e2640 1 -- 172.21.15.33:0/853861266 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3adc103410 msgr2=0x7f3adc1997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.026+0000 7f3ae29e2640 1 --2- 172.21.15.33:0/853861266 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3adc103410 0x7f3adc1997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.026+0000 7f3ae29e2640 1 -- 172.21.15.33:0/853861266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3adc102a50 msgr2=0x7f3adc1992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.026+0000 7f3ae29e2640 1 --2- 172.21.15.33:0/853861266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3adc102a50 0x7f3adc1992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.026+0000 7f3ae29e2640 1 -- 172.21.15.33:0/853861266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3adc19e240 con 0x7f3adc108a60 2024-05-15T14:10:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.027+0000 7f3ae29e2640 1 --2- 172.21.15.33:0/853861266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3adc108a60 0x7f3adc19db10 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3ad400cce0 tx=0x7f3ad400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.027+0000 7f3ad37fe640 1 -- 172.21.15.33:0/853861266 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad400f450 con 0x7f3adc108a60 2024-05-15T14:10:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.027+0000 7f3ad37fe640 1 -- 172.21.15.33:0/853861266 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ad400faa0 con 0x7f3adc108a60 2024-05-15T14:10:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.027+0000 7f3ae4445640 1 -- 172.21.15.33:0/853861266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3adc19e530 con 0x7f3adc108a60 2024-05-15T14:10:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.027+0000 7f3ad37fe640 1 -- 172.21.15.33:0/853861266 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad4018b70 con 0x7f3adc108a60 2024-05-15T14:10:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.027+0000 7f3ae4445640 1 -- 172.21.15.33:0/853861266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3adc1a5dc0 con 0x7f3adc108a60 2024-05-15T14:10:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.030+0000 7f3ae7239740 1 Processor -- start 2024-05-15T14:10:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.030+0000 7f3ad37fe640 1 -- 172.21.15.33:0/853861266 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3ad4018d10 con 0x7f3adc108a60 2024-05-15T14:10:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.030+0000 7f3ae7239740 1 -- start start 2024-05-15T14:10:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.031+0000 7f3ae7239740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e6c5af8f0 0x561e6c5afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.031+0000 7f3ae7239740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e6c5b0210 0x561e6c5ba590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.031+0000 7f3ae7239740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e6c5baad0 0x561e6c5bcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.031+0000 7f3ae7239740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561e6c522c30 con 0x561e6c5af8f0 2024-05-15T14:10:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.031+0000 7f3ae7239740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561e6c5127d0 con 0x561e6c5baad0 2024-05-15T14:10:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.031+0000 7f3ae7239740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x561e6c516810 con 0x561e6c5b0210 2024-05-15T14:10:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.031+0000 7f3ae19e0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e6c5af8f0 0x561e6c5afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.031+0000 7f3ae21e1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e6c5b0210 0x561e6c5ba590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.031+0000 7f3ae21e1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e6c5b0210 0x561e6c5ba590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:49056/0 (socket says 172.21.15.33:49056) 2024-05-15T14:10:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.031+0000 7f3ae19e0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e6c5af8f0 0x561e6c5afcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43080/0 (socket says 172.21.15.33:43080) 2024-05-15T14:10:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.031+0000 7f3ae21e1640 1 -- 172.21.15.33:0/2787350804 learned_addr learned my addr 172.21.15.33:0/2787350804 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.031+0000 7f3ae19e0640 1 --2- 172.21.15.33:0/2787350804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e6c5baad0 0x561e6c5bcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.031+0000 7f3ae19e0640 1 -- 172.21.15.33:0/2787350804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e6c5b0210 msgr2=0x561e6c5ba590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.031+0000 7f3ae19e0640 1 --2- 172.21.15.33:0/2787350804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e6c5b0210 0x561e6c5ba590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.031+0000 7f3ae19e0640 1 -- 172.21.15.33:0/2787350804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e6c5baad0 msgr2=0x561e6c5bcec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.031+0000 7f3ae19e0640 1 --2- 172.21.15.33:0/2787350804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e6c5baad0 0x561e6c5bcec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.031+0000 7f3ae19e0640 1 -- 172.21.15.33:0/2787350804 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e6c227210 con 0x561e6c5af8f0 2024-05-15T14:10:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.031+0000 7f3ae21e1640 1 --2- 172.21.15.33:0/2787350804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e6c5b0210 0x561e6c5ba590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:10:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.031+0000 7f3ad37fe640 1 --2- 172.21.15.33:0/853861266 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f3ac80b29d0 0x7f3ac80b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.031+0000 7f3ae21e1640 1 --2- 172.21.15.33:0/853861266 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f3ac80b29d0 0x7f3ac80b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.031+0000 7f3ae19e0640 1 --2- 172.21.15.33:0/2787350804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e6c5af8f0 0x561e6c5afcd0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f3acc031440 tx=0x7f3acc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e41e8a5591b4736 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.032+0000 7f3ad37fe640 1 -- 172.21.15.33:0/853861266 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f3ad40e2930 con 0x7f3adc108a60 2024-05-15T14:10:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.032+0000 7f3ad0bfe640 1 -- 172.21.15.33:0/2787350804 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc044030 con 0x561e6c5af8f0 2024-05-15T14:10:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.032+0000 7f3ad0bfe640 1 -- 172.21.15.33:0/2787350804 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3acc031660 con 0x561e6c5af8f0 2024-05-15T14:10:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.032+0000 7f3ad0bfe640 1 -- 172.21.15.33:0/2787350804 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc035040 con 0x561e6c5af8f0 2024-05-15T14:10:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.032+0000 7f3ae7239740 1 -- 172.21.15.33:0/2787350804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e6c5af8f0 msgr2=0x561e6c5afcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.032+0000 7f3ae7239740 1 --2- 172.21.15.33:0/2787350804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e6c5af8f0 0x561e6c5afcd0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f3acc031440 tx=0x7f3acc005fd0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.032+0000 7f3ae21e1640 1 --2- 172.21.15.33:0/853861266 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f3ac80b29d0 0x7f3ac80b4e90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x561e6c5b0e90 tx=0x7f3ad8005f90 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.033+0000 7f3ae7239740 1 -- 172.21.15.33:0/2787350804 shutdown_connections 2024-05-15T14:10:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.033+0000 7f3ae7239740 1 --2- 172.21.15.33:0/2787350804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e6c5baad0 0x561e6c5bcec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.033+0000 7f3ae7239740 1 --2- 172.21.15.33:0/2787350804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e6c5b0210 0x561e6c5ba590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.033+0000 7f3ae7239740 1 --2- 172.21.15.33:0/2787350804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e6c5af8f0 0x561e6c5afcd0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.033+0000 7f3ae7239740 1 -- 172.21.15.33:0/2787350804 >> 172.21.15.33:0/2787350804 conn(0x561e6c5abe00 msgr2=0x561e6c5ac1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.033+0000 7f3ae7239740 1 -- 172.21.15.33:0/2787350804 shutdown_connections 2024-05-15T14:10:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.033+0000 7f3ae7239740 1 -- 172.21.15.33:0/2787350804 wait complete. 2024-05-15T14:10:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.033+0000 7f3ae7239740 1 -- start start 2024-05-15T14:10:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.033+0000 7f3ae7239740 1 Processor -- start 2024-05-15T14:10:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.033+0000 7f3ae7239740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e6c5af8f0 0x561e6c5ba480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.033+0000 7f3ae7239740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e6c5b0210 0x561e6c5b7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.033+0000 7f3ae7239740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e6c5baad0 0x561e6c5b7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.033+0000 7f3ae7239740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561e6c1f8790 con 0x561e6c5af8f0 2024-05-15T14:10:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.033+0000 7f3ae7239740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561e6c4ce890 con 0x561e6c5b0210 2024-05-15T14:10:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.033+0000 7f3ae7239740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x561e6c5bd740 con 0x561e6c5baad0 2024-05-15T14:10:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.033+0000 7f3ae7239740 10 client.0 _ll_register_callbacks cb 0x561e6c4d1d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:10:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.033+0000 7f3ae21e1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e6c5af8f0 0x561e6c5ba480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.033+0000 7f3ae21e1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e6c5af8f0 0x561e6c5ba480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43092/0 (socket says 172.21.15.33:43092) 2024-05-15T14:10:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.034+0000 7f3ae21e1640 1 -- 172.21.15.33:0/3037829967 learned_addr learned my addr 172.21.15.33:0/3037829967 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.034+0000 7f3ae29e2640 1 --2- 172.21.15.33:0/3037829967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e6c5baad0 0x561e6c5b7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.034+0000 7f3ae21e1640 1 -- 172.21.15.33:0/3037829967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e6c5baad0 msgr2=0x561e6c5b7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.034+0000 7f3ae19e0640 1 --2- 172.21.15.33:0/3037829967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e6c5b0210 0x561e6c5b7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.034+0000 7f3ae21e1640 1 --2- 172.21.15.33:0/3037829967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e6c5baad0 0x561e6c5b7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.034+0000 7f3ae21e1640 1 -- 172.21.15.33:0/3037829967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e6c5b0210 msgr2=0x561e6c5b7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.034+0000 7f3ae21e1640 1 --2- 172.21.15.33:0/3037829967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e6c5b0210 0x561e6c5b7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.034+0000 7f3ae21e1640 1 -- 172.21.15.33:0/3037829967 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e6c2f3620 con 0x561e6c5af8f0 2024-05-15T14:10:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.034+0000 7f3ae21e1640 1 --2- 172.21.15.33:0/3037829967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e6c5af8f0 0x561e6c5ba480 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8005b90 tx=0x7f3ad8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b9793378a285faf server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.034+0000 7f3ac5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:10:04.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.034+0000 7f3ac5ffb640 1 -- 172.21.15.33:0/3037829967 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad800f690 con 0x561e6c5af8f0 2024-05-15T14:10:04.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.034+0000 7f3ac5ffb640 1 -- 172.21.15.33:0/3037829967 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f3ad800f830 con 0x561e6c5af8f0 2024-05-15T14:10:04.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.034+0000 7f3ae7239740 1 -- 172.21.15.33:0/3037829967 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3acc031660 con 0x561e6c5af8f0 2024-05-15T14:10:04.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.034+0000 7f3ac5ffb640 1 -- 172.21.15.33:0/3037829967 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3ad800fa30 con 0x561e6c5af8f0 2024-05-15T14:10:04.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.034+0000 7f3ae7239740 10 client.16539 fetch_fsmap learned FSMap version 34 2024-05-15T14:10:04.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.034+0000 7f3ae7239740 1 -- 172.21.15.33:0/3037829967 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561e6c5c9200 con 0x561e6c5af8f0 2024-05-15T14:10:04.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.035+0000 7f3ac5ffb640 1 -- 172.21.15.33:0/3037829967 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3acc031660 con 0x561e6c5af8f0 2024-05-15T14:10:04.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.035+0000 7f3ac5ffb640 1 -- 172.21.15.33:0/3037829967 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f3ad8027070 con 0x561e6c5af8f0 2024-05-15T14:10:04.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.035+0000 7f3ae7239740 10 client.16539 fetch_fsmap finished waiting for FSMap version 34 2024-05-15T14:10:04.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.035+0000 7f3ae7239740 10 client.16539 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:10:04.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.035+0000 7f3ae7239740 20 client.16539 populate_metadata read hostname 'smithi033' 2024-05-15T14:10:04.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.035+0000 7f3ae7239740 1 --2- 172.21.15.33:0/3037829967 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x561e6c5ca950 0x561e6c5ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:04.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.035+0000 7f3ae7239740 4 client.16539 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:10:04.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.035+0000 7f3ae7239740 1 -- 172.21.15.33:0/3037829967 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e6c524b80 con 0x561e6c5ca950 2024-05-15T14:10:04.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.035+0000 7f3ae19e0640 1 --2- 172.21.15.33:0/3037829967 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x561e6c5ca950 0x561e6c5ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:04.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.035+0000 7f3ae19e0640 1 --2- 172.21.15.33:0/3037829967 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x561e6c5ca950 0x561e6c5ccdd0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=790cfa5da1e03d96 server_cookie=e7f890f7400dd81 in_seq=0 out_seq=0 2024-05-15T14:10:04.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.035+0000 7f3ac5ffb640 10 client.16539 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:10:04.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.038+0000 7f3ac5ffb640 1 -- 172.21.15.33:0/3037829967 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561e6c524b80 con 0x561e6c5ca950 2024-05-15T14:10:04.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.038+0000 7f3ac5ffb640 10 client.16539 handle_command_reply: tid=0 2024-05-15T14:10:04.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.039+0000 7f3ae7239740 1 client.16539 shutdown 2024-05-15T14:10:04.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.039+0000 7f3ae7239740 1 -- 172.21.15.33:0/3037829967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e6c5af8f0 msgr2=0x561e6c5ba480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:04.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.039+0000 7f3ae7239740 1 --2- 172.21.15.33:0/3037829967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e6c5af8f0 0x561e6c5ba480 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8005b90 tx=0x7f3ad8004730 comp rx=0 tx=0).stop 2024-05-15T14:10:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.039+0000 7f3ae7239740 1 -- 172.21.15.33:0/3037829967 shutdown_connections 2024-05-15T14:10:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.039+0000 7f3ae7239740 1 --2- 172.21.15.33:0/3037829967 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x561e6c5ca950 0x561e6c5ccdd0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.039+0000 7f3ae7239740 1 --2- 172.21.15.33:0/3037829967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e6c5baad0 0x561e6c5b7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.039+0000 7f3ae7239740 1 --2- 172.21.15.33:0/3037829967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e6c5b0210 0x561e6c5b7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.039+0000 7f3ac5ffb640 0 client.16539 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:10:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.039+0000 7f3ae7239740 1 --2- 172.21.15.33:0/3037829967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e6c5af8f0 0x561e6c5ba480 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.039+0000 7f3ae7239740 1 -- 172.21.15.33:0/3037829967 >> 172.21.15.33:0/3037829967 conn(0x561e6c5abe00 msgr2=0x561e6c5ab100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.039+0000 7f3ae7239740 1 -- 172.21.15.33:0/3037829967 shutdown_connections 2024-05-15T14:10:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.039+0000 7f3ae7239740 1 -- 172.21.15.33:0/3037829967 wait complete. 2024-05-15T14:10:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.039+0000 7f3ae7239740 20 client.16539 trim_cache size 0 max 16384 2024-05-15T14:10:04.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:04.040+0000 7f74457fa700 1 -- 172.21.15.33:0/2414766660 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555ec5ba2d70 con 0x555ec5cc8430 2024-05-15T14:10:04.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:04.040+0000 7f74457fa700 10 client.25979 handle_command_reply: tid=0 2024-05-15T14:10:04.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:04.040+0000 7f74727b0b80 1 client.25979 shutdown 2024-05-15T14:10:04.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:04.040+0000 7f74727b0b80 1 -- 172.21.15.33:0/2414766660 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ec5c48750 msgr2=0x555ec5cc65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:04.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:04.040+0000 7f74727b0b80 1 --2- 172.21.15.33:0/2414766660 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ec5c48750 0x555ec5cc65d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7458005760 tx=0x7f7458022cc0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:04.040+0000 7f74727b0b80 1 -- 172.21.15.33:0/2414766660 shutdown_connections 2024-05-15T14:10:04.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:04.040+0000 7f74727b0b80 1 --2- 172.21.15.33:0/2414766660 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ec5c48750 0x555ec5cc65d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:04.040+0000 7f74727b0b80 1 --2- 172.21.15.33:0/2414766660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555ec5c42b60 0x555ec5c51170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:04.040+0000 7f74727b0b80 1 --2- 172.21.15.33:0/2414766660 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x555ec5cc8430 0x555ec5cca850 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:04.040+0000 7f74727b0b80 1 --2- 172.21.15.33:0/2414766660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ec5c47e30 0x555ec5c516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:04.040+0000 7f74457fa700 0 client.25979 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:10:04.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:04.040+0000 7f74727b0b80 1 -- 172.21.15.33:0/2414766660 >> 172.21.15.33:0/2414766660 conn(0x555ec5b6d3e0 msgr2=0x555ec5c41530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:04.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:04.040+0000 7f74727b0b80 1 -- 172.21.15.33:0/2414766660 shutdown_connections 2024-05-15T14:10:04.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:04.040+0000 7f74727b0b80 1 -- 172.21.15.33:0/2414766660 wait complete. 2024-05-15T14:10:04.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:04.040+0000 7f74727b0b80 20 client.25979 trim_cache size 0 max 16384 2024-05-15T14:10:04.051 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T14:10:04.051 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 34, 2024-05-15T14:10:04.051 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24478, 2024-05-15T14:10:04.052 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 34, 2024-05-15T14:10:04.052 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-05-15T14:10:04.052 INFO:teuthology.orchestra.run.smithi033.stdout: "5860f899": { 2024-05-15T14:10:04.052 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 34, 2024-05-15T14:10:04.052 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-05-15T14:10:04.052 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:10:04.052 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-05-15T14:10:04.052 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:10:04.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:10:04.052 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 90.0, 2024-05-15T14:10:04.052 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 148.5, 2024-05-15T14:10:04.052 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-05-15T14:10:04.052 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-05-15T14:10:04.052 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-05-15T14:10:04.053 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:10:04.053 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-05-15T14:10:04.053 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:10:04.053 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:10:04.053 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:10:04.053 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:10:04.053 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:10:04.053 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:10:04.053 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T14:10:04.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:04.041+0000 7f7467f1a700 1 -- 172.21.15.33:0/2921549393 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f74480b2aa0 msgr2=0x7f74480b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:04.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:04.041+0000 7f7467f1a700 1 --2- 172.21.15.33:0/2921549393 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f74480b2aa0 0x7f74480b4f60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7460109be0 tx=0x7f7454009260 comp rx=0 tx=0).stop 2024-05-15T14:10:04.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:04.041+0000 7f7467f1a700 1 -- 172.21.15.33:0/2921549393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f746006b480 msgr2=0x7f7460102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:04.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:04.041+0000 7f7467f1a700 1 --2- 172.21.15.33:0/2921549393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f746006b480 0x7f7460102510 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f7450007a20 tx=0x7f7450007a50 comp rx=0 tx=0).stop 2024-05-15T14:10:04.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:04.041+0000 7f7467f1a700 1 -- 172.21.15.33:0/2921549393 shutdown_connections 2024-05-15T14:10:04.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:04.041+0000 7f7467f1a700 1 --2- 172.21.15.33:0/2921549393 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f74480b2aa0 0x7f74480b4f60 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:04.041+0000 7f7467f1a700 1 --2- 172.21.15.33:0/2921549393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7460108900 0x7f7460102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:04.041+0000 7f7467f1a700 1 --2- 172.21.15.33:0/2921549393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f746006b480 0x7f7460102510 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:04.041+0000 7f7467f1a700 1 --2- 172.21.15.33:0/2921549393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f746006be30 0x7f7460102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:04.041+0000 7f7467f1a700 1 -- 172.21.15.33:0/2921549393 >> 172.21.15.33:0/2921549393 conn(0x7f74600793e0 msgr2=0x7f74601003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:04.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:04.041+0000 7f7467f1a700 1 -- 172.21.15.33:0/2921549393 shutdown_connections 2024-05-15T14:10:04.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:04.041+0000 7f7467f1a700 1 -- 172.21.15.33:0/2921549393 wait complete. 2024-05-15T14:10:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.059+0000 7f3ae7239740 1 Processor -- start 2024-05-15T14:10:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.059+0000 7f3ae7239740 1 -- start start 2024-05-15T14:10:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.059+0000 7f3ae7239740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e6c5af8f0 0x561e6c5c6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.059+0000 7f3ae7239740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e6c5b0210 0x561e6c5d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.059+0000 7f3ae7239740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e6c5baad0 0x561e6c5d2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.059+0000 7f3ae7239740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc1b1050 con 0x561e6c5baad0 2024-05-15T14:10:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.059+0000 7f3ae7239740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc1ae890 con 0x561e6c5af8f0 2024-05-15T14:10:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.059+0000 7f3ae7239740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc1ac700 con 0x561e6c5b0210 2024-05-15T14:10:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.060+0000 7f3ae21e1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e6c5af8f0 0x561e6c5c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.060+0000 7f3ae29e2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e6c5baad0 0x561e6c5d2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.060+0000 7f3ae21e1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e6c5af8f0 0x561e6c5c6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:57042/0 (socket says 172.21.15.33:57042) 2024-05-15T14:10:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.060+0000 7f3ae21e1640 1 -- 172.21.15.33:0/3553403237 learned_addr learned my addr 172.21.15.33:0/3553403237 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.060+0000 7f3ae19e0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e6c5b0210 0x561e6c5d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.060+0000 7f3ae21e1640 1 -- 172.21.15.33:0/3553403237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e6c5b0210 msgr2=0x561e6c5d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.060+0000 7f3ae21e1640 1 --2- 172.21.15.33:0/3553403237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e6c5b0210 0x561e6c5d26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.060+0000 7f3ae21e1640 1 -- 172.21.15.33:0/3553403237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e6c5baad0 msgr2=0x561e6c5d2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.060+0000 7f3ae21e1640 1 --2- 172.21.15.33:0/3553403237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e6c5baad0 0x561e6c5d2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.060+0000 7f3ae21e1640 1 -- 172.21.15.33:0/3553403237 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e6c5a7600 con 0x561e6c5af8f0 2024-05-15T14:10:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.060+0000 7f3ae29e2640 1 --2- 172.21.15.33:0/3553403237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e6c5baad0 0x561e6c5d2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:10:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.060+0000 7f3ae21e1640 1 --2- 172.21.15.33:0/3553403237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e6c5af8f0 0x561e6c5c6610 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8004480 tx=0x7f3ad8009df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b3ec1581d149b18 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.060+0000 7f3ac7fff640 1 -- 172.21.15.33:0/3553403237 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad800f890 con 0x561e6c5af8f0 2024-05-15T14:10:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.060+0000 7f3ac7fff640 1 -- 172.21.15.33:0/3553403237 <== mon.2 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ad80298f0 con 0x561e6c5af8f0 2024-05-15T14:10:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.060+0000 7f3ac7fff640 1 -- 172.21.15.33:0/3553403237 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad801f8f0 con 0x561e6c5af8f0 2024-05-15T14:10:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.061+0000 7f3ae7239740 1 -- 172.21.15.33:0/3553403237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e6c5af8f0 msgr2=0x561e6c5c6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.061+0000 7f3ae7239740 1 --2- 172.21.15.33:0/3553403237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e6c5af8f0 0x561e6c5c6610 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8004480 tx=0x7f3ad8009df0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.061+0000 7f3ae7239740 1 -- 172.21.15.33:0/3553403237 shutdown_connections 2024-05-15T14:10:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.061+0000 7f3ae7239740 1 --2- 172.21.15.33:0/3553403237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e6c5baad0 0x561e6c5d2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.061+0000 7f3ae7239740 1 --2- 172.21.15.33:0/3553403237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e6c5b0210 0x561e6c5d26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.061+0000 7f3ae7239740 1 --2- 172.21.15.33:0/3553403237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e6c5af8f0 0x561e6c5c6610 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.061+0000 7f3ae7239740 1 -- 172.21.15.33:0/3553403237 >> 172.21.15.33:0/3553403237 conn(0x561e6c5abe00 msgr2=0x561e6c5b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.061+0000 7f3ae7239740 1 -- 172.21.15.33:0/3553403237 shutdown_connections 2024-05-15T14:10:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.061+0000 7f3ae7239740 1 -- 172.21.15.33:0/3553403237 wait complete. 2024-05-15T14:10:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.062+0000 7f3ae7239740 1 -- start start 2024-05-15T14:10:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.062+0000 7f3ae7239740 1 Processor -- start 2024-05-15T14:10:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.062+0000 7f3ae7239740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e6c5af8f0 0x561e6c5c0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.062+0000 7f3ae7239740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e6c5b0210 0x561e6c5c0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.062+0000 7f3ae7239740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e6c5baad0 0x561e6c634260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.062+0000 7f3ae7239740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc1acd50 con 0x561e6c5af8f0 2024-05-15T14:10:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.062+0000 7f3ae7239740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc1ae1c0 con 0x561e6c5b0210 2024-05-15T14:10:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.062+0000 7f3ae7239740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc1ad420 con 0x561e6c5baad0 2024-05-15T14:10:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.062+0000 7f3ae7239740 10 client.0 _ll_register_callbacks cb 0x7f3ad80044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:10:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.062+0000 7f3ae21e1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e6c5af8f0 0x561e6c5c0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.062+0000 7f3ae29e2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e6c5baad0 0x561e6c634260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.062+0000 7f3ae29e2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e6c5baad0 0x561e6c634260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:49082/0 (socket says 172.21.15.33:49082) 2024-05-15T14:10:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.062+0000 7f3ae21e1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e6c5af8f0 0x561e6c5c0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43118/0 (socket says 172.21.15.33:43118) 2024-05-15T14:10:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.062+0000 7f3ae29e2640 1 -- 172.21.15.33:0/1058756180 learned_addr learned my addr 172.21.15.33:0/1058756180 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.062+0000 7f3ae19e0640 1 --2- 172.21.15.33:0/1058756180 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e6c5b0210 0x561e6c5c0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.062+0000 7f3ae29e2640 1 -- 172.21.15.33:0/1058756180 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e6c5b0210 msgr2=0x561e6c5c0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.062+0000 7f3ae29e2640 1 --2- 172.21.15.33:0/1058756180 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e6c5b0210 0x561e6c5c0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.062+0000 7f3ae29e2640 1 -- 172.21.15.33:0/1058756180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e6c5af8f0 msgr2=0x561e6c5c0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.062+0000 7f3ae29e2640 1 --2- 172.21.15.33:0/1058756180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e6c5af8f0 0x561e6c5c0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.062+0000 7f3ae29e2640 1 -- 172.21.15.33:0/1058756180 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e6c5c1640 con 0x561e6c5baad0 2024-05-15T14:10:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.062+0000 7f3ae21e1640 1 --2- 172.21.15.33:0/1058756180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e6c5af8f0 0x561e6c5c0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:10:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.063+0000 7f3ae29e2640 1 --2- 172.21.15.33:0/1058756180 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e6c5baad0 0x561e6c634260 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3ad40a9b90 tx=0x7f3ad4007670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e7c05374689e73 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.063+0000 7f3ac5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-05-15T14:10:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.063+0000 7f3ac5ffb640 1 -- 172.21.15.33:0/1058756180 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad40a9710 con 0x561e6c5baad0 2024-05-15T14:10:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.063+0000 7f3ae7239740 1 -- 172.21.15.33:0/1058756180 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ad80298f0 con 0x561e6c5baad0 2024-05-15T14:10:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.063+0000 7f3ac5ffb640 1 -- 172.21.15.33:0/1058756180 <== mon.1 v2:172.21.15.81:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f3ad40b27e0 con 0x561e6c5baad0 2024-05-15T14:10:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.063+0000 7f3ac5ffb640 1 -- 172.21.15.33:0/1058756180 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3ad40aa300 con 0x561e6c5baad0 2024-05-15T14:10:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.063+0000 7f3ae7239740 10 client.26134 fetch_fsmap learned FSMap version 34 2024-05-15T14:10:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.063+0000 7f3ae7239740 1 -- 172.21.15.33:0/1058756180 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561e6c635930 con 0x561e6c5baad0 2024-05-15T14:10:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.063+0000 7f3ac5ffb640 1 -- 172.21.15.33:0/1058756180 <== mon.1 v2:172.21.15.81:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ad80298f0 con 0x561e6c5baad0 2024-05-15T14:10:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.063+0000 7f3ac5ffb640 1 -- 172.21.15.33:0/1058756180 <== mon.1 v2:172.21.15.81:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f3ad40a5570 con 0x561e6c5baad0 2024-05-15T14:10:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.063+0000 7f3ae7239740 10 client.26134 fetch_fsmap finished waiting for FSMap version 34 2024-05-15T14:10:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.063+0000 7f3ae7239740 10 client.26134 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:10:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.063+0000 7f3ae7239740 20 client.26134 populate_metadata read hostname 'smithi033' 2024-05-15T14:10:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.063+0000 7f3ae7239740 1 --2- 172.21.15.33:0/1058756180 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x561e6c636060 0x561e6c638480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.063+0000 7f3ae7239740 4 client.26134 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:10:04.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.063+0000 7f3ae7239740 1 -- 172.21.15.33:0/1058756180 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561e6c51e360 con 0x561e6c636060 2024-05-15T14:10:04.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.064+0000 7f3ae21e1640 1 --2- 172.21.15.33:0/1058756180 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x561e6c636060 0x561e6c638480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:04.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.064+0000 7f3ac5ffb640 10 client.26134 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:10:04.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.064+0000 7f3ae21e1640 1 --2- 172.21.15.33:0/1058756180 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x561e6c636060 0x561e6c638480 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=473ae2acb9051b06 server_cookie=8cbee234dcf0de3c in_seq=0 out_seq=0 2024-05-15T14:10:04.075 INFO:tasks.workunit.client.0.smithi033.stdout: 1 576957 5.83 MB/sec execute 339 sec latency 64.829 ms 2024-05-15T14:10:04.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.115+0000 7f3ac5ffb640 1 -- 172.21.15.33:0/1058756180 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561e6c51e360 con 0x561e6c636060 2024-05-15T14:10:04.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.115+0000 7f3ac5ffb640 10 client.26134 handle_command_reply: tid=0 2024-05-15T14:10:04.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.115+0000 7f3ae7239740 1 client.26134 shutdown 2024-05-15T14:10:04.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.115+0000 7f3ae7239740 1 -- 172.21.15.33:0/1058756180 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e6c5baad0 msgr2=0x561e6c634260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:04.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.115+0000 7f3ae7239740 1 --2- 172.21.15.33:0/1058756180 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e6c5baad0 0x561e6c634260 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3ad40a9b90 tx=0x7f3ad4007670 comp rx=0 tx=0).stop 2024-05-15T14:10:04.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.115+0000 7f3ae7239740 1 -- 172.21.15.33:0/1058756180 shutdown_connections 2024-05-15T14:10:04.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.115+0000 7f3ae7239740 1 --2- 172.21.15.33:0/1058756180 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x561e6c636060 0x561e6c638480 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.115+0000 7f3ae7239740 1 --2- 172.21.15.33:0/1058756180 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561e6c5baad0 0x561e6c634260 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.115+0000 7f3ae7239740 1 --2- 172.21.15.33:0/1058756180 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e6c5b0210 0x561e6c5c0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.115+0000 7f3ae7239740 1 --2- 172.21.15.33:0/1058756180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e6c5af8f0 0x561e6c5c0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.115+0000 7f3ae7239740 1 -- 172.21.15.33:0/1058756180 >> 172.21.15.33:0/1058756180 conn(0x561e6c5abe00 msgr2=0x561e6c5c5600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:04.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.115+0000 7f3ac5ffb640 0 client.26134 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:10:04.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.115+0000 7f3ae7239740 1 -- 172.21.15.33:0/1058756180 shutdown_connections 2024-05-15T14:10:04.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.115+0000 7f3ae7239740 1 -- 172.21.15.33:0/1058756180 wait complete. 2024-05-15T14:10:04.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.115+0000 7f3ae7239740 20 client.26134 trim_cache size 0 max 16384 2024-05-15T14:10:04.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:10:04.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:10:04.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:10:04.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.116+0000 7f3ae4445640 1 -- 172.21.15.33:0/853861266 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f3ac80b29d0 msgr2=0x7f3ac80b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:04.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.116+0000 7f3ae4445640 1 --2- 172.21.15.33:0/853861266 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f3ac80b29d0 0x7f3ac80b4e90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x561e6c5b0e90 tx=0x7f3ad8005f90 comp rx=0 tx=0).stop 2024-05-15T14:10:04.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.116+0000 7f3ae4445640 1 -- 172.21.15.33:0/853861266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3adc108a60 msgr2=0x7f3adc19db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:04.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.116+0000 7f3ae4445640 1 --2- 172.21.15.33:0/853861266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3adc108a60 0x7f3adc19db10 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3ad400cce0 tx=0x7f3ad400a620 comp rx=0 tx=0).stop 2024-05-15T14:10:04.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.116+0000 7f3ae4445640 1 -- 172.21.15.33:0/853861266 shutdown_connections 2024-05-15T14:10:04.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.116+0000 7f3ae4445640 1 --2- 172.21.15.33:0/853861266 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f3ac80b29d0 0x7f3ac80b4e90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.116+0000 7f3ae4445640 1 --2- 172.21.15.33:0/853861266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3adc108a60 0x7f3adc19db10 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.116+0000 7f3ae4445640 1 --2- 172.21.15.33:0/853861266 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3adc103410 0x7f3adc1997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.116+0000 7f3ae4445640 1 --2- 172.21.15.33:0/853861266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3adc102a50 0x7f3adc1992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.116+0000 7f3ae4445640 1 -- 172.21.15.33:0/853861266 >> 172.21.15.33:0/853861266 conn(0x7f3adc0fe540 msgr2=0x7f3adc0fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.116+0000 7f3ae4445640 1 -- 172.21.15.33:0/853861266 shutdown_connections 2024-05-15T14:10:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:04.116+0000 7f3ae4445640 1 -- 172.21.15.33:0/853861266 wait complete. 2024-05-15T14:10:04.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:04 smithi033 ceph-mon[31650]: pgmap v302: 97 pgs: 97 active+clean; 178 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 288 op/s 2024-05-15T14:10:04.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:04 smithi052 ceph-mon[36190]: pgmap v302: 97 pgs: 97 active+clean; 178 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 288 op/s 2024-05-15T14:10:04.845 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '5860f899', quiesce took 6.2 seconds. Will release after: 22.3 2024-05-15T14:10:04.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:04 smithi081 ceph-mon[36434]: pgmap v302: 97 pgs: 97 active+clean; 178 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 288 op/s 2024-05-15T14:10:05.075 INFO:tasks.workunit.client.0.smithi033.stdout: 1 576957 5.81 MB/sec execute 340 sec latency 1013.991 ms 2024-05-15T14:10:05.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:05 smithi033 ceph-mon[31650]: osdmap e92: 12 total, 12 up, 12 in 2024-05-15T14:10:05.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:05 smithi052 ceph-mon[36190]: osdmap e92: 12 total, 12 up, 12 in 2024-05-15T14:10:05.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:05 smithi081 ceph-mon[36434]: osdmap e92: 12 total, 12 up, 12 in 2024-05-15T14:10:06.075 INFO:tasks.workunit.client.0.smithi033.stdout: 1 576957 5.79 MB/sec execute 341 sec latency 2014.025 ms 2024-05-15T14:10:06.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:06 smithi033 ceph-mon[31650]: pgmap v304: 97 pgs: 97 active+clean; 178 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-05-15T14:10:06.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:06 smithi052 ceph-mon[36190]: pgmap v304: 97 pgs: 97 active+clean; 178 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-05-15T14:10:06.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:06 smithi081 ceph-mon[36434]: pgmap v304: 97 pgs: 97 active+clean; 178 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-05-15T14:10:07.076 INFO:tasks.workunit.client.0.smithi033.stdout: 1 576957 5.77 MB/sec execute 342 sec latency 3014.132 ms 2024-05-15T14:10:08.076 INFO:tasks.workunit.client.0.smithi033.stdout: 1 576957 5.76 MB/sec execute 343 sec latency 4014.277 ms 2024-05-15T14:10:08.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:08 smithi033 ceph-mon[31650]: pgmap v305: 97 pgs: 97 active+clean; 176 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 230 op/s 2024-05-15T14:10:08.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:08 smithi052 ceph-mon[36190]: pgmap v305: 97 pgs: 97 active+clean; 176 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 230 op/s 2024-05-15T14:10:08.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:08 smithi081 ceph-mon[36434]: pgmap v305: 97 pgs: 97 active+clean; 176 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 230 op/s 2024-05-15T14:10:09.076 INFO:tasks.workunit.client.0.smithi033.stdout: 1 576957 5.74 MB/sec execute 344 sec latency 5014.337 ms 2024-05-15T14:10:10.076 INFO:tasks.workunit.client.0.smithi033.stdout: 1 576957 5.72 MB/sec execute 345 sec latency 6014.451 ms 2024-05-15T14:10:10.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:10 smithi033 ceph-mon[31650]: pgmap v306: 97 pgs: 97 active+clean; 176 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 230 op/s 2024-05-15T14:10:10.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:10 smithi052 ceph-mon[36190]: pgmap v306: 97 pgs: 97 active+clean; 176 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 230 op/s 2024-05-15T14:10:10.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:10 smithi081 ceph-mon[36434]: pgmap v306: 97 pgs: 97 active+clean; 176 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 230 op/s 2024-05-15T14:10:11.076 INFO:tasks.workunit.client.0.smithi033.stdout: 1 576957 5.71 MB/sec execute 346 sec latency 7014.581 ms 2024-05-15T14:10:11.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:11 smithi052 ceph-mon[36190]: osdmap e93: 12 total, 12 up, 12 in 2024-05-15T14:10:11.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:11 smithi081 ceph-mon[36434]: osdmap e93: 12 total, 12 up, 12 in 2024-05-15T14:10:11.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:11 smithi033 ceph-mon[31650]: osdmap e93: 12 total, 12 up, 12 in 2024-05-15T14:10:12.076 INFO:tasks.workunit.client.0.smithi033.stdout: 1 576957 5.69 MB/sec execute 347 sec latency 8014.635 ms 2024-05-15T14:10:12.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:12 smithi052 ceph-mon[36190]: pgmap v308: 97 pgs: 97 active+clean; 176 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 1.3 MiB/s wr, 108 op/s 2024-05-15T14:10:12.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:12 smithi081 ceph-mon[36434]: pgmap v308: 97 pgs: 97 active+clean; 176 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 1.3 MiB/s wr, 108 op/s 2024-05-15T14:10:12.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:12 smithi033 ceph-mon[31650]: pgmap v308: 97 pgs: 97 active+clean; 176 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 1.3 MiB/s wr, 108 op/s 2024-05-15T14:10:13.076 INFO:tasks.workunit.client.0.smithi033.stdout: 1 576957 5.67 MB/sec execute 348 sec latency 9014.746 ms 2024-05-15T14:10:14.076 INFO:tasks.workunit.client.0.smithi033.stdout: 1 576957 5.66 MB/sec execute 349 sec latency 10014.800 ms 2024-05-15T14:10:14.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:14 smithi052 ceph-mon[36190]: pgmap v309: 97 pgs: 97 active+clean; 177 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 2.1 MiB/s wr, 133 op/s 2024-05-15T14:10:14.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:14 smithi081 ceph-mon[36434]: pgmap v309: 97 pgs: 97 active+clean; 177 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 2.1 MiB/s wr, 133 op/s 2024-05-15T14:10:14.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:14 smithi033 ceph-mon[31650]: pgmap v309: 97 pgs: 97 active+clean; 177 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 2.1 MiB/s wr, 133 op/s 2024-05-15T14:10:15.076 INFO:tasks.workunit.client.0.smithi033.stdout: 1 576957 5.64 MB/sec execute 350 sec latency 11014.845 ms 2024-05-15T14:10:16.077 INFO:tasks.workunit.client.0.smithi033.stdout: 1 576957 5.63 MB/sec execute 351 sec latency 12014.946 ms 2024-05-15T14:10:16.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:16 smithi052 ceph-mon[36190]: pgmap v310: 97 pgs: 97 active+clean; 177 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.9 MiB/s wr, 118 op/s 2024-05-15T14:10:16.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:16 smithi081 ceph-mon[36434]: pgmap v310: 97 pgs: 97 active+clean; 177 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.9 MiB/s wr, 118 op/s 2024-05-15T14:10:16.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:16 smithi033 ceph-mon[31650]: pgmap v310: 97 pgs: 97 active+clean; 177 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.9 MiB/s wr, 118 op/s 2024-05-15T14:10:17.076 INFO:tasks.workunit.client.0.smithi033.stdout: 1 576957 5.61 MB/sec execute 352 sec latency 13015.034 ms 2024-05-15T14:10:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.219+0000 7fbbfadf7640 1 -- 172.21.15.33:0/376035565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbf4103ce0 msgr2=0x7fbbf41040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.219+0000 7fbbfadf7640 1 --2- 172.21.15.33:0/376035565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbf4103ce0 0x7fbbf41040c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0009900 tx=0x7fbbf002a480 comp rx=0 tx=0).stop 2024-05-15T14:10:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.219+0000 7fbbfadf7640 1 -- 172.21.15.33:0/376035565 shutdown_connections 2024-05-15T14:10:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.219+0000 7fbbfadf7640 1 --2- 172.21.15.33:0/376035565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbf410d0b0 0x7fbbf410f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.219+0000 7fbbfadf7640 1 --2- 172.21.15.33:0/376035565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbbf4104600 0x7fbbf410cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.219+0000 7fbbfadf7640 1 --2- 172.21.15.33:0/376035565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbf4103ce0 0x7fbbf41040c0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.219+0000 7fbbfadf7640 1 -- 172.21.15.33:0/376035565 >> 172.21.15.33:0/376035565 conn(0x7fbbf4078a70 msgr2=0x7fbbf4078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.219+0000 7fbbfadf7640 1 -- 172.21.15.33:0/376035565 shutdown_connections 2024-05-15T14:10:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.219+0000 7fbbfadf7640 1 -- 172.21.15.33:0/376035565 wait complete. 2024-05-15T14:10:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.219+0000 7fbbfadf7640 1 Processor -- start 2024-05-15T14:10:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.219+0000 7fbbfadf7640 1 -- start start 2024-05-15T14:10:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.220+0000 7fbbfadf7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbbf4103ce0 0x7fbbf41992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.220+0000 7fbbfadf7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbf4104600 0x7fbbf41997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.220+0000 7fbbfadf7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbf410d0b0 0x7fbbf419db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.220+0000 7fbbfadf7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf4112d40 con 0x7fbbf410d0b0 2024-05-15T14:10:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.220+0000 7fbbfadf7640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf4112bc0 con 0x7fbbf4104600 2024-05-15T14:10:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.220+0000 7fbbfadf7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf4112ec0 con 0x7fbbf4103ce0 2024-05-15T14:10:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.220+0000 7fbbf9394640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbf410d0b0 0x7fbbf419db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.220+0000 7fbbf8b93640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbbf4103ce0 0x7fbbf41992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.220+0000 7fbbf9394640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbf410d0b0 0x7fbbf419db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33718/0 (socket says 172.21.15.33:33718) 2024-05-15T14:10:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.220+0000 7fbbf9394640 1 -- 172.21.15.33:0/1731090480 learned_addr learned my addr 172.21.15.33:0/1731090480 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.220+0000 7fbbebfff640 1 --2- 172.21.15.33:0/1731090480 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbf4104600 0x7fbbf41997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.220+0000 7fbbf9394640 1 -- 172.21.15.33:0/1731090480 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbbf4103ce0 msgr2=0x7fbbf41992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.220+0000 7fbbf9394640 1 --2- 172.21.15.33:0/1731090480 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbbf4103ce0 0x7fbbf41992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.220+0000 7fbbf9394640 1 -- 172.21.15.33:0/1731090480 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbf4104600 msgr2=0x7fbbf41997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.220+0000 7fbbf9394640 1 --2- 172.21.15.33:0/1731090480 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbf4104600 0x7fbbf41997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.220+0000 7fbbf9394640 1 -- 172.21.15.33:0/1731090480 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf419e290 con 0x7fbbf410d0b0 2024-05-15T14:10:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.220+0000 7fbbebfff640 1 --2- 172.21.15.33:0/1731090480 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbf4104600 0x7fbbf41997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:10:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.220+0000 7fbbf9394640 1 --2- 172.21.15.33:0/1731090480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbf410d0b0 0x7fbbf419db60 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fbbec00eb40 tx=0x7fbbec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.221+0000 7fbbe9ffb640 1 -- 172.21.15.33:0/1731090480 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbec004b20 con 0x7fbbf410d0b0 2024-05-15T14:10:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.221+0000 7fbbe9ffb640 1 -- 172.21.15.33:0/1731090480 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbec004cc0 con 0x7fbbf410d0b0 2024-05-15T14:10:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.221+0000 7fbbfadf7640 1 -- 172.21.15.33:0/1731090480 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbf419e580 con 0x7fbbf410d0b0 2024-05-15T14:10:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.221+0000 7fbbe9ffb640 1 -- 172.21.15.33:0/1731090480 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbec010720 con 0x7fbbf410d0b0 2024-05-15T14:10:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.221+0000 7fbbfadf7640 1 -- 172.21.15.33:0/1731090480 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbf41a5e10 con 0x7fbbf410d0b0 2024-05-15T14:10:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.223+0000 7fbbe9ffb640 1 -- 172.21.15.33:0/1731090480 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbbec005560 con 0x7fbbf410d0b0 2024-05-15T14:10:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.224+0000 7fbbfdbef740 1 Processor -- start 2024-05-15T14:10:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.224+0000 7fbbfdbef740 1 -- start start 2024-05-15T14:10:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.224+0000 7fbbfdbef740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649e0c87810 0x5649e0c87bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.224+0000 7fbbfdbef740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649e0c88130 0x5649e0c969a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.224+0000 7fbbfdbef740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649e0c96ee0 0x5649e0c992d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.224+0000 7fbbfdbef740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5649e0bfac30 con 0x5649e0c87810 2024-05-15T14:10:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.224+0000 7fbbfdbef740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5649e0bea7d0 con 0x5649e0c96ee0 2024-05-15T14:10:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.224+0000 7fbbfdbef740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5649e0bee810 con 0x5649e0c88130 2024-05-15T14:10:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.224+0000 7fbbebfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649e0c87810 0x5649e0c87bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.224+0000 7fbbe9ffb640 1 --2- 172.21.15.33:0/1731090480 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbbdc0b2950 0x7fbbdc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.224+0000 7fbbebfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649e0c87810 0x5649e0c87bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33720/0 (socket says 172.21.15.33:33720) 2024-05-15T14:10:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.224+0000 7fbbebfff640 1 -- 172.21.15.33:0/2962061072 learned_addr learned my addr 172.21.15.33:0/2962061072 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.224+0000 7fbbf8b93640 1 --2- 172.21.15.33:0/2962061072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649e0c88130 0x5649e0c969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.225+0000 7fbbf9394640 1 --2- 172.21.15.33:0/1731090480 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbbdc0b2950 0x7fbbdc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.225+0000 7fbbebfff640 1 --2- 172.21.15.33:0/2962061072 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649e0c96ee0 0x5649e0c992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.225+0000 7fbbe9ffb640 1 -- 172.21.15.33:0/1731090480 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7fbbec0e2db0 con 0x7fbbf410d0b0 2024-05-15T14:10:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.225+0000 7fbbebfff640 1 -- 172.21.15.33:0/2962061072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649e0c88130 msgr2=0x5649e0c969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.225+0000 7fbbebfff640 1 --2- 172.21.15.33:0/2962061072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649e0c88130 0x5649e0c969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.225+0000 7fbbebfff640 1 -- 172.21.15.33:0/2962061072 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649e0c96ee0 msgr2=0x5649e0c992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.225+0000 7fbbebfff640 1 --2- 172.21.15.33:0/2962061072 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649e0c96ee0 0x5649e0c992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.225+0000 7fbbebfff640 1 -- 172.21.15.33:0/2962061072 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649e08ff210 con 0x5649e0c87810 2024-05-15T14:10:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.225+0000 7fbbf8b93640 1 --2- 172.21.15.33:0/2962061072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649e0c88130 0x5649e0c969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:10:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.225+0000 7fbbebfff640 1 --2- 172.21.15.33:0/2962061072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649e0c87810 0x5649e0c87bf0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fbbe40058c0 tx=0x7fbbe400b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b39c2d1ad681de server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.225+0000 7fbbf9394640 1 --2- 172.21.15.33:0/1731090480 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbbdc0b2950 0x7fbbdc0b4e10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbbec00ceb0 tx=0x7fbbec005900 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.225+0000 7fbbdaffd640 1 -- 172.21.15.33:0/2962061072 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4019030 con 0x5649e0c87810 2024-05-15T14:10:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.225+0000 7fbbdaffd640 1 -- 172.21.15.33:0/2962061072 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbe40119f0 con 0x5649e0c87810 2024-05-15T14:10:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.225+0000 7fbbdaffd640 1 -- 172.21.15.33:0/2962061072 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe401d920 con 0x5649e0c87810 2024-05-15T14:10:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.226+0000 7fbbfdbef740 1 -- 172.21.15.33:0/2962061072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649e0c87810 msgr2=0x5649e0c87bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.226+0000 7fbbfdbef740 1 --2- 172.21.15.33:0/2962061072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649e0c87810 0x5649e0c87bf0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fbbe40058c0 tx=0x7fbbe400b730 comp rx=0 tx=0).stop 2024-05-15T14:10:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.226+0000 7fbbfdbef740 1 -- 172.21.15.33:0/2962061072 shutdown_connections 2024-05-15T14:10:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.226+0000 7fbbfdbef740 1 --2- 172.21.15.33:0/2962061072 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649e0c96ee0 0x5649e0c992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.226+0000 7fbbfdbef740 1 --2- 172.21.15.33:0/2962061072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649e0c88130 0x5649e0c969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.226+0000 7fbbfdbef740 1 --2- 172.21.15.33:0/2962061072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649e0c87810 0x5649e0c87bf0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.226+0000 7fbbfdbef740 1 -- 172.21.15.33:0/2962061072 >> 172.21.15.33:0/2962061072 conn(0x5649e0c83e00 msgr2=0x5649e0c841e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.226+0000 7fbbfdbef740 1 -- 172.21.15.33:0/2962061072 shutdown_connections 2024-05-15T14:10:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.226+0000 7fbbfdbef740 1 -- 172.21.15.33:0/2962061072 wait complete. 2024-05-15T14:10:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.226+0000 7fbbfdbef740 1 -- start start 2024-05-15T14:10:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.226+0000 7fbbfdbef740 1 Processor -- start 2024-05-15T14:10:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.227+0000 7fbbfdbef740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649e0c87810 0x5649e0c96860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.227+0000 7fbbfdbef740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649e0c88130 0x5649e0c93bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.227+0000 7fbbfdbef740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649e0c96ee0 0x5649e0c941f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.227+0000 7fbbfdbef740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5649e08d0790 con 0x5649e0c96ee0 2024-05-15T14:10:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.227+0000 7fbbfdbef740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5649e0ba6890 con 0x5649e0c88130 2024-05-15T14:10:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.227+0000 7fbbfdbef740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5649e0c99bb0 con 0x5649e0c87810 2024-05-15T14:10:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.227+0000 7fbbfdbef740 10 client.0 _ll_register_callbacks cb 0x5649e0ba9d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:10:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.227+0000 7fbbebfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649e0c96ee0 0x5649e0c941f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.227+0000 7fbbebfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649e0c96ee0 0x5649e0c941f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33736/0 (socket says 172.21.15.33:33736) 2024-05-15T14:10:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.227+0000 7fbbebfff640 1 -- 172.21.15.33:0/292688237 learned_addr learned my addr 172.21.15.33:0/292688237 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.227+0000 7fbbebfff640 1 --2- 172.21.15.33:0/292688237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649e0c87810 0x5649e0c96860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.227+0000 7fbbf8b93640 1 --2- 172.21.15.33:0/292688237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649e0c88130 0x5649e0c93bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.227+0000 7fbbebfff640 1 -- 172.21.15.33:0/292688237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649e0c87810 msgr2=0x5649e0c96860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.227+0000 7fbbebfff640 1 --2- 172.21.15.33:0/292688237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649e0c87810 0x5649e0c96860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.227+0000 7fbbebfff640 1 -- 172.21.15.33:0/292688237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649e0c88130 msgr2=0x5649e0c93bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.227+0000 7fbbebfff640 1 --2- 172.21.15.33:0/292688237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649e0c88130 0x5649e0c93bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.227+0000 7fbbebfff640 1 -- 172.21.15.33:0/292688237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649e09cb620 con 0x5649e0c96ee0 2024-05-15T14:10:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.228+0000 7fbbebfff640 1 --2- 172.21.15.33:0/292688237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649e0c96ee0 0x5649e0c941f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4005fd0 tx=0x7fbbe401d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f740d590260a7d07 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.228+0000 7fbbd3fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:10:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.228+0000 7fbbd3fff640 1 -- 172.21.15.33:0/292688237 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4019040 con 0x5649e0c96ee0 2024-05-15T14:10:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.228+0000 7fbbfdbef740 1 -- 172.21.15.33:0/292688237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbe40119f0 con 0x5649e0c96ee0 2024-05-15T14:10:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.228+0000 7fbbd3fff640 1 -- 172.21.15.33:0/292688237 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7fbbe4011e10 con 0x5649e0c96ee0 2024-05-15T14:10:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.228+0000 7fbbd3fff640 1 -- 172.21.15.33:0/292688237 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbbe4011540 con 0x5649e0c96ee0 2024-05-15T14:10:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.228+0000 7fbbfdbef740 10 client.16557 fetch_fsmap learned FSMap version 34 2024-05-15T14:10:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.228+0000 7fbbfdbef740 1 -- 172.21.15.33:0/292688237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5649e0ca5640 con 0x5649e0c96ee0 2024-05-15T14:10:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.228+0000 7fbbd3fff640 1 -- 172.21.15.33:0/292688237 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbe40119f0 con 0x5649e0c96ee0 2024-05-15T14:10:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.228+0000 7fbbd3fff640 1 -- 172.21.15.33:0/292688237 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fbbe402b020 con 0x5649e0c96ee0 2024-05-15T14:10:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.229+0000 7fbbfdbef740 10 client.16557 fetch_fsmap finished waiting for FSMap version 34 2024-05-15T14:10:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.229+0000 7fbbfdbef740 10 client.16557 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:10:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.229+0000 7fbbfdbef740 20 client.16557 populate_metadata read hostname 'smithi033' 2024-05-15T14:10:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.229+0000 7fbbfdbef740 1 --2- 172.21.15.33:0/292688237 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5649e0ca69f0 0x5649e0ca8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.229+0000 7fbbfdbef740 4 client.16557 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:10:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.229+0000 7fbbfdbef740 1 -- 172.21.15.33:0/292688237 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649e0bfcb80 con 0x5649e0ca69f0 2024-05-15T14:10:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.229+0000 7fbbf8b93640 1 --2- 172.21.15.33:0/292688237 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5649e0ca69f0 0x5649e0ca8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.229+0000 7fbbf8b93640 1 --2- 172.21.15.33:0/292688237 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5649e0ca69f0 0x5649e0ca8de0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=994e19d447cd9db9 server_cookie=671d7399e0aeb62 in_seq=0 out_seq=0 2024-05-15T14:10:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.229+0000 7fbbd3fff640 10 client.16557 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:10:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.230+0000 7fbbd3fff640 1 -- 172.21.15.33:0/292688237 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5649e0bfcb80 con 0x5649e0ca69f0 2024-05-15T14:10:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.230+0000 7fbbd3fff640 10 client.16557 handle_command_reply: tid=0 2024-05-15T14:10:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.230+0000 7fbbfdbef740 1 client.16557 shutdown 2024-05-15T14:10:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.230+0000 7fbbfdbef740 1 -- 172.21.15.33:0/292688237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649e0c96ee0 msgr2=0x5649e0c941f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.230+0000 7fbbfdbef740 1 --2- 172.21.15.33:0/292688237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649e0c96ee0 0x5649e0c941f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4005fd0 tx=0x7fbbe401d460 comp rx=0 tx=0).stop 2024-05-15T14:10:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.231+0000 7fbbfdbef740 1 -- 172.21.15.33:0/292688237 shutdown_connections 2024-05-15T14:10:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.231+0000 7fbbfdbef740 1 --2- 172.21.15.33:0/292688237 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5649e0ca69f0 0x5649e0ca8de0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.231+0000 7fbbfdbef740 1 --2- 172.21.15.33:0/292688237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649e0c96ee0 0x5649e0c941f0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.231+0000 7fbbfdbef740 1 --2- 172.21.15.33:0/292688237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649e0c88130 0x5649e0c93bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.231+0000 7fbbd3fff640 0 client.16557 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:10:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.231+0000 7fbbfdbef740 1 --2- 172.21.15.33:0/292688237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649e0c87810 0x5649e0c96860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.231+0000 7fbbfdbef740 1 -- 172.21.15.33:0/292688237 >> 172.21.15.33:0/292688237 conn(0x5649e0c83e00 msgr2=0x5649e0c89c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.231+0000 7fbbfdbef740 1 -- 172.21.15.33:0/292688237 shutdown_connections 2024-05-15T14:10:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.231+0000 7fbbfdbef740 1 -- 172.21.15.33:0/292688237 wait complete. 2024-05-15T14:10:17.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.231+0000 7fbbfdbef740 20 client.16557 trim_cache size 0 max 16384 2024-05-15T14:10:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.246+0000 7fbbfdbef740 1 Processor -- start 2024-05-15T14:10:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.246+0000 7fbbfdbef740 1 -- start start 2024-05-15T14:10:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.247+0000 7fbbfdbef740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649e0c87810 0x5649e0ca2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.247+0000 7fbbfdbef740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649e0c88130 0x5649e0cae570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.247+0000 7fbbfdbef740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649e0c96ee0 0x5649e0caeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.247+0000 7fbbfdbef740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf41aee80 con 0x5649e0c88130 2024-05-15T14:10:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.247+0000 7fbbfdbef740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf41ac6c0 con 0x5649e0c87810 2024-05-15T14:10:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.247+0000 7fbbfdbef740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf41aa530 con 0x5649e0c96ee0 2024-05-15T14:10:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.247+0000 7fbbf8b93640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649e0c88130 0x5649e0cae570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.247+0000 7fbbf8b93640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649e0c88130 0x5649e0cae570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33742/0 (socket says 172.21.15.33:33742) 2024-05-15T14:10:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.247+0000 7fbbf8b93640 1 -- 172.21.15.33:0/4006079042 learned_addr learned my addr 172.21.15.33:0/4006079042 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.247+0000 7fbbebfff640 1 --2- 172.21.15.33:0/4006079042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649e0c87810 0x5649e0ca2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.247+0000 7fbbf8b93640 1 -- 172.21.15.33:0/4006079042 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649e0c96ee0 msgr2=0x5649e0caeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.247+0000 7fbbf8b93640 1 --2- 172.21.15.33:0/4006079042 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649e0c96ee0 0x5649e0caeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.247+0000 7fbbf8b93640 1 -- 172.21.15.33:0/4006079042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649e0c87810 msgr2=0x5649e0ca2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.247+0000 7fbbf8b93640 1 --2- 172.21.15.33:0/4006079042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649e0c87810 0x5649e0ca2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.247+0000 7fbbf8b93640 1 -- 172.21.15.33:0/4006079042 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649e0c849d0 con 0x5649e0c88130 2024-05-15T14:10:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.247+0000 7fbbebfff640 1 --2- 172.21.15.33:0/4006079042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649e0c87810 0x5649e0ca2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:10:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.247+0000 7fbbf8b93640 1 --2- 172.21.15.33:0/4006079042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649e0c88130 0x5649e0cae570 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0005ab0 tx=0x7fbbf0005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a76c15fab62ed3 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.247+0000 7fbbda7fc640 1 -- 172.21.15.33:0/4006079042 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0035040 con 0x5649e0c88130 2024-05-15T14:10:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.247+0000 7fbbda7fc640 1 -- 172.21.15.33:0/4006079042 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbf0004770 con 0x5649e0c88130 2024-05-15T14:10:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.248+0000 7fbbda7fc640 1 -- 172.21.15.33:0/4006079042 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf00455a0 con 0x5649e0c88130 2024-05-15T14:10:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.248+0000 7fbbfdbef740 1 -- 172.21.15.33:0/4006079042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649e0c88130 msgr2=0x5649e0cae570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.248+0000 7fbbfdbef740 1 --2- 172.21.15.33:0/4006079042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649e0c88130 0x5649e0cae570 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0005ab0 tx=0x7fbbf0005a20 comp rx=0 tx=0).stop 2024-05-15T14:10:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.248+0000 7fbbfdbef740 1 -- 172.21.15.33:0/4006079042 shutdown_connections 2024-05-15T14:10:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.248+0000 7fbbfdbef740 1 --2- 172.21.15.33:0/4006079042 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649e0c96ee0 0x5649e0caeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.248+0000 7fbbfdbef740 1 --2- 172.21.15.33:0/4006079042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649e0c88130 0x5649e0cae570 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.248+0000 7fbbfdbef740 1 --2- 172.21.15.33:0/4006079042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649e0c87810 0x5649e0ca2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.248+0000 7fbbfdbef740 1 -- 172.21.15.33:0/4006079042 >> 172.21.15.33:0/4006079042 conn(0x5649e0c83e00 msgr2=0x5649e0c8f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.248+0000 7fbbfdbef740 1 -- 172.21.15.33:0/4006079042 shutdown_connections 2024-05-15T14:10:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.248+0000 7fbbfdbef740 1 -- 172.21.15.33:0/4006079042 wait complete. 2024-05-15T14:10:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.248+0000 7fbbfdbef740 1 -- start start 2024-05-15T14:10:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.249+0000 7fbbfdbef740 1 Processor -- start 2024-05-15T14:10:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.249+0000 7fbbfdbef740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649e0c87810 0x5649e0c9aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.249+0000 7fbbfdbef740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649e0c88130 0x5649e0c9afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.249+0000 7fbbfdbef740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649e0c96ee0 0x5649e0c9b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.249+0000 7fbbfdbef740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf41aab80 con 0x5649e0c88130 2024-05-15T14:10:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.249+0000 7fbbfdbef740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf41abff0 con 0x5649e0c96ee0 2024-05-15T14:10:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.249+0000 7fbbfdbef740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf41ab250 con 0x5649e0c87810 2024-05-15T14:10:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.249+0000 7fbbfdbef740 10 client.0 _ll_register_callbacks cb 0x7fbbe4011c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:10:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.249+0000 7fbbf8b93640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649e0c88130 0x5649e0c9afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.249+0000 7fbbf8b93640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649e0c88130 0x5649e0c9afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33746/0 (socket says 172.21.15.33:33746) 2024-05-15T14:10:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.249+0000 7fbbf8b93640 1 -- 172.21.15.33:0/3141398609 learned_addr learned my addr 172.21.15.33:0/3141398609 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.249+0000 7fbbebfff640 1 --2- 172.21.15.33:0/3141398609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649e0c96ee0 0x5649e0c9b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.249+0000 7fbbebfff640 1 --2- 172.21.15.33:0/3141398609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649e0c87810 0x5649e0c9aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.249+0000 7fbbf8b93640 1 -- 172.21.15.33:0/3141398609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649e0c87810 msgr2=0x5649e0c9aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.249+0000 7fbbf8b93640 1 --2- 172.21.15.33:0/3141398609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649e0c87810 0x5649e0c9aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.249+0000 7fbbf8b93640 1 -- 172.21.15.33:0/3141398609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649e0c96ee0 msgr2=0x5649e0c9b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.249+0000 7fbbf8b93640 1 --2- 172.21.15.33:0/3141398609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649e0c96ee0 0x5649e0c9b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.249+0000 7fbbf8b93640 1 -- 172.21.15.33:0/3141398609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649e0d101d0 con 0x5649e0c88130 2024-05-15T14:10:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.249+0000 7fbbebfff640 1 --2- 172.21.15.33:0/3141398609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649e0c87810 0x5649e0c9aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:10:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.250+0000 7fbbf8b93640 1 --2- 172.21.15.33:0/3141398609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649e0c88130 0x5649e0c9afe0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0009230 tx=0x7fbbf0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f93e45efe9cdd92c server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.250+0000 7fbbd3fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:10:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.250+0000 7fbbd3fff640 1 -- 172.21.15.33:0/3141398609 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0053070 con 0x5649e0c88130 2024-05-15T14:10:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.250+0000 7fbbd3fff640 1 -- 172.21.15.33:0/3141398609 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7fbbf0004920 con 0x5649e0c88130 2024-05-15T14:10:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.250+0000 7fbbfdbef740 1 -- 172.21.15.33:0/3141398609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbf0004770 con 0x5649e0c88130 2024-05-15T14:10:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.250+0000 7fbbd3fff640 1 -- 172.21.15.33:0/3141398609 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbbf005f050 con 0x5649e0c88130 2024-05-15T14:10:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.250+0000 7fbbfdbef740 10 client.16563 fetch_fsmap learned FSMap version 34 2024-05-15T14:10:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.250+0000 7fbbfdbef740 1 -- 172.21.15.33:0/3141398609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5649e0d114c0 con 0x5649e0c88130 2024-05-15T14:10:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.250+0000 7fbbd3fff640 1 -- 172.21.15.33:0/3141398609 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbf0004770 con 0x5649e0c88130 2024-05-15T14:10:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.250+0000 7fbbd3fff640 1 -- 172.21.15.33:0/3141398609 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fbbf00035f0 con 0x5649e0c88130 2024-05-15T14:10:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.250+0000 7fbbfdbef740 10 client.16563 fetch_fsmap finished waiting for FSMap version 34 2024-05-15T14:10:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.250+0000 7fbbfdbef740 10 client.16563 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:10:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.250+0000 7fbbfdbef740 20 client.16563 populate_metadata read hostname 'smithi033' 2024-05-15T14:10:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.250+0000 7fbbfdbef740 1 --2- 172.21.15.33:0/3141398609 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5649e0d11c50 0x5649e0d14070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.250+0000 7fbbfdbef740 4 client.16563 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:10:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.250+0000 7fbbfdbef740 1 -- 172.21.15.33:0/3141398609 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5649e0bf6360 con 0x5649e0d11c50 2024-05-15T14:10:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.251+0000 7fbbebfff640 1 --2- 172.21.15.33:0/3141398609 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5649e0d11c50 0x5649e0d14070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.251+0000 7fbbebfff640 1 --2- 172.21.15.33:0/3141398609 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5649e0d11c50 0x5649e0d14070 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8030058ba0ba4bdb server_cookie=d26f0904a5b95124 in_seq=0 out_seq=0 2024-05-15T14:10:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.251+0000 7fbbd3fff640 10 client.16563 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:10:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.302+0000 7fbbd3fff640 1 -- 172.21.15.33:0/3141398609 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5649e0bf6360 con 0x5649e0d11c50 2024-05-15T14:10:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.302+0000 7fbbd3fff640 10 client.16563 handle_command_reply: tid=0 2024-05-15T14:10:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.302+0000 7fbbfdbef740 1 client.16563 shutdown 2024-05-15T14:10:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.302+0000 7fbbfdbef740 1 -- 172.21.15.33:0/3141398609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649e0c88130 msgr2=0x5649e0c9afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.302+0000 7fbbfdbef740 1 --2- 172.21.15.33:0/3141398609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649e0c88130 0x5649e0c9afe0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0009230 tx=0x7fbbf0000bc0 comp rx=0 tx=0).stop 2024-05-15T14:10:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.303+0000 7fbbfdbef740 1 -- 172.21.15.33:0/3141398609 shutdown_connections 2024-05-15T14:10:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.303+0000 7fbbfdbef740 1 --2- 172.21.15.33:0/3141398609 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5649e0d11c50 0x5649e0d14070 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.303+0000 7fbbfdbef740 1 --2- 172.21.15.33:0/3141398609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649e0c96ee0 0x5649e0c9b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.303+0000 7fbbd3fff640 0 client.16563 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:10:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.303+0000 7fbbfdbef740 1 --2- 172.21.15.33:0/3141398609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5649e0c88130 0x5649e0c9afe0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.303+0000 7fbbfdbef740 1 --2- 172.21.15.33:0/3141398609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5649e0c87810 0x5649e0c9aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:17.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.303+0000 7fbbfdbef740 1 -- 172.21.15.33:0/3141398609 >> 172.21.15.33:0/3141398609 conn(0x5649e0c83e00 msgr2=0x5649e0c8e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:17.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.303+0000 7fbbfdbef740 1 -- 172.21.15.33:0/3141398609 shutdown_connections 2024-05-15T14:10:17.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.303+0000 7fbbfdbef740 1 -- 172.21.15.33:0/3141398609 wait complete. 2024-05-15T14:10:17.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.303+0000 7fbbfdbef740 20 client.16563 trim_cache size 0 max 16384 2024-05-15T14:10:17.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:10:17.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:10:17.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:10:17.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.303+0000 7fbbfadf7640 1 -- 172.21.15.33:0/1731090480 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbbdc0b2950 msgr2=0x7fbbdc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:17.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.303+0000 7fbbfadf7640 1 --2- 172.21.15.33:0/1731090480 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbbdc0b2950 0x7fbbdc0b4e10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbbec00ceb0 tx=0x7fbbec005900 comp rx=0 tx=0).stop 2024-05-15T14:10:17.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.303+0000 7fbbfadf7640 1 -- 172.21.15.33:0/1731090480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbf410d0b0 msgr2=0x7fbbf419db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:17.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.303+0000 7fbbfadf7640 1 --2- 172.21.15.33:0/1731090480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbf410d0b0 0x7fbbf419db60 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fbbec00eb40 tx=0x7fbbec00c2d0 comp rx=0 tx=0).stop 2024-05-15T14:10:17.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.303+0000 7fbbfadf7640 1 -- 172.21.15.33:0/1731090480 shutdown_connections 2024-05-15T14:10:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.303+0000 7fbbfadf7640 1 --2- 172.21.15.33:0/1731090480 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbbdc0b2950 0x7fbbdc0b4e10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.303+0000 7fbbfadf7640 1 --2- 172.21.15.33:0/1731090480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbf410d0b0 0x7fbbf419db60 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.303+0000 7fbbfadf7640 1 --2- 172.21.15.33:0/1731090480 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbf4104600 0x7fbbf41997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.303+0000 7fbbfadf7640 1 --2- 172.21.15.33:0/1731090480 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbbf4103ce0 0x7fbbf41992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.304+0000 7fbbfadf7640 1 -- 172.21.15.33:0/1731090480 >> 172.21.15.33:0/1731090480 conn(0x7fbbf4078a70 msgr2=0x7fbbf40fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.304+0000 7fbbfadf7640 1 -- 172.21.15.33:0/1731090480 shutdown_connections 2024-05-15T14:10:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:17.304+0000 7fbbfadf7640 1 -- 172.21.15.33:0/1731090480 wait complete. 2024-05-15T14:10:18.077 INFO:tasks.workunit.client.0.smithi033.stdout: 1 576957 5.59 MB/sec execute 353 sec latency 14015.076 ms 2024-05-15T14:10:18.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:18 smithi052 ceph-mon[36190]: pgmap v311: 97 pgs: 97 active+clean; 177 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.8 MiB/s wr, 68 op/s 2024-05-15T14:10:18.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:18 smithi081 ceph-mon[36434]: pgmap v311: 97 pgs: 97 active+clean; 177 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.8 MiB/s wr, 68 op/s 2024-05-15T14:10:18.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:18 smithi033 ceph-mon[31650]: pgmap v311: 97 pgs: 97 active+clean; 177 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.8 MiB/s wr, 68 op/s 2024-05-15T14:10:19.077 INFO:tasks.workunit.client.0.smithi033.stdout: 1 576957 5.58 MB/sec execute 354 sec latency 15015.183 ms 2024-05-15T14:10:20.077 INFO:tasks.workunit.client.0.smithi033.stdout: 1 576957 5.56 MB/sec execute 355 sec latency 16015.266 ms 2024-05-15T14:10:20.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:20 smithi052 ceph-mon[36190]: pgmap v312: 97 pgs: 97 active+clean; 177 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.8 MiB/s wr, 68 op/s 2024-05-15T14:10:20.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:20 smithi081 ceph-mon[36434]: pgmap v312: 97 pgs: 97 active+clean; 177 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.8 MiB/s wr, 68 op/s 2024-05-15T14:10:20.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:20 smithi033 ceph-mon[31650]: pgmap v312: 97 pgs: 97 active+clean; 177 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.8 MiB/s wr, 68 op/s 2024-05-15T14:10:21.077 INFO:tasks.workunit.client.0.smithi033.stdout: 1 576957 5.55 MB/sec execute 356 sec latency 17015.416 ms 2024-05-15T14:10:22.077 INFO:tasks.workunit.client.0.smithi033.stdout: 1 576957 5.53 MB/sec execute 357 sec latency 18015.528 ms 2024-05-15T14:10:22.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:22 smithi052 ceph-mon[36190]: pgmap v313: 97 pgs: 97 active+clean; 177 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 1.7 MiB/s wr, 63 op/s 2024-05-15T14:10:22.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:22 smithi081 ceph-mon[36434]: pgmap v313: 97 pgs: 97 active+clean; 177 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 1.7 MiB/s wr, 63 op/s 2024-05-15T14:10:22.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:22 smithi033 ceph-mon[31650]: pgmap v313: 97 pgs: 97 active+clean; 177 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 1.7 MiB/s wr, 63 op/s 2024-05-15T14:10:23.077 INFO:tasks.workunit.client.0.smithi033.stdout: 1 576957 5.52 MB/sec execute 358 sec latency 19015.688 ms 2024-05-15T14:10:24.077 INFO:tasks.workunit.client.0.smithi033.stdout: 1 576957 5.50 MB/sec execute 359 sec latency 20015.752 ms 2024-05-15T14:10:24.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:24 smithi052 ceph-mon[36190]: pgmap v314: 97 pgs: 97 active+clean; 178 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 2.3 MiB/s wr, 93 op/s 2024-05-15T14:10:24.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:24 smithi081 ceph-mon[36434]: pgmap v314: 97 pgs: 97 active+clean; 178 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 2.3 MiB/s wr, 93 op/s 2024-05-15T14:10:24.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:24 smithi033 ceph-mon[31650]: pgmap v314: 97 pgs: 97 active+clean; 178 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 2.3 MiB/s wr, 93 op/s 2024-05-15T14:10:25.077 INFO:tasks.workunit.client.0.smithi033.stdout: 1 576957 5.49 MB/sec execute 360 sec latency 21015.809 ms 2024-05-15T14:10:26.077 INFO:tasks.workunit.client.0.smithi033.stdout: 1 576957 5.47 MB/sec execute 361 sec latency 22015.923 ms 2024-05-15T14:10:26.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:26 smithi052 ceph-mon[36190]: pgmap v315: 97 pgs: 97 active+clean; 178 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 1.6 MiB/s wr, 66 op/s 2024-05-15T14:10:26.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:26 smithi081 ceph-mon[36434]: pgmap v315: 97 pgs: 97 active+clean; 178 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 1.6 MiB/s wr, 66 op/s 2024-05-15T14:10:26.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:26 smithi033 ceph-mon[31650]: pgmap v315: 97 pgs: 97 active+clean; 178 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 1.6 MiB/s wr, 66 op/s 2024-05-15T14:10:27.077 INFO:tasks.workunit.client.0.smithi033.stdout: 1 576957 5.46 MB/sec execute 362 sec latency 23016.023 ms 2024-05-15T14:10:27.147 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T14:10:27.414 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:10:28.078 INFO:tasks.workunit.client.0.smithi033.stdout: 1 576957 5.44 MB/sec execute 363 sec latency 24016.134 ms 2024-05-15T14:10:28.804 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:28 smithi033 ceph-mon[31650]: pgmap v316: 97 pgs: 97 active+clean; 176 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 2.1 MiB/s wr, 90 op/s 2024-05-15T14:10:28.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:28 smithi052 ceph-mon[36190]: pgmap v316: 97 pgs: 97 active+clean; 176 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 2.1 MiB/s wr, 90 op/s 2024-05-15T14:10:28.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:28 smithi081 ceph-mon[36434]: pgmap v316: 97 pgs: 97 active+clean; 176 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 2.1 MiB/s wr, 90 op/s 2024-05-15T14:10:29.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.028+0000 7f7e7c5b9700 1 -- 172.21.15.33:0/2736243364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e741028a0 msgr2=0x7f7e74110310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:29.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.028+0000 7f7e7c5b9700 1 --2- 172.21.15.33:0/2736243364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e741028a0 0x7f7e74110310 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00b0a0 tx=0x7f7e6c00b3b0 comp rx=0 tx=0).stop 2024-05-15T14:10:29.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.028+0000 7f7e7c5b9700 1 -- 172.21.15.33:0/2736243364 shutdown_connections 2024-05-15T14:10:29.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.028+0000 7f7e7c5b9700 1 --2- 172.21.15.33:0/2736243364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e741028a0 0x7f7e74110310 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:29.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.028+0000 7f7e7c5b9700 1 --2- 172.21.15.33:0/2736243364 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e74108600 0x7f7e741089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:29.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.028+0000 7f7e7c5b9700 1 --2- 172.21.15.33:0/2736243364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e74101ee0 0x7f7e74102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:29.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.028+0000 7f7e7c5b9700 1 -- 172.21.15.33:0/2736243364 >> 172.21.15.33:0/2736243364 conn(0x7f7e74077760 msgr2=0x7f7e74077b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:29.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.029+0000 7f7e7c5b9700 1 -- 172.21.15.33:0/2736243364 shutdown_connections 2024-05-15T14:10:29.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.029+0000 7f7e7c5b9700 1 -- 172.21.15.33:0/2736243364 wait complete. 2024-05-15T14:10:29.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.029+0000 7f7e7c5b9700 1 Processor -- start 2024-05-15T14:10:29.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.029+0000 7f7e7c5b9700 1 -- start start 2024-05-15T14:10:29.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.030+0000 7f7e7c5b9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e74101ee0 0x7f7e7419f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:29.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.030+0000 7f7e7c5b9700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e741028a0 0x7f7e7419fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:29.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.030+0000 7f7e7c5b9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e74108600 0x7f7e74199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:29.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.030+0000 7f7e7c5b9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7e74113790 con 0x7f7e74108600 2024-05-15T14:10:29.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.030+0000 7f7e7c5b9700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7e74113610 con 0x7f7e74101ee0 2024-05-15T14:10:29.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.030+0000 7f7e7c5b9700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f7e74113910 con 0x7f7e741028a0 2024-05-15T14:10:29.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.030+0000 7f7e7ab56700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e74108600 0x7f7e74199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:29.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.030+0000 7f7e7ab56700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e74108600 0x7f7e74199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40464/0 (socket says 172.21.15.33:40464) 2024-05-15T14:10:29.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.030+0000 7f7e7ab56700 1 -- 172.21.15.33:0/2990211249 learned_addr learned my addr 172.21.15.33:0/2990211249 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:29.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.030+0000 7f7e7a355700 1 --2- 172.21.15.33:0/2990211249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e74101ee0 0x7f7e7419f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:29.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.030+0000 7f7e79b54700 1 --2- 172.21.15.33:0/2990211249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e741028a0 0x7f7e7419fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:29.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.030+0000 7f7e7ab56700 1 -- 172.21.15.33:0/2990211249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e741028a0 msgr2=0x7f7e7419fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:29.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.030+0000 7f7e7ab56700 1 --2- 172.21.15.33:0/2990211249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e741028a0 0x7f7e7419fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:29.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.030+0000 7f7e7ab56700 1 -- 172.21.15.33:0/2990211249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e74101ee0 msgr2=0x7f7e7419f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:29.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.030+0000 7f7e7ab56700 1 --2- 172.21.15.33:0/2990211249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e74101ee0 0x7f7e7419f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:29.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.030+0000 7f7e7ab56700 1 -- 172.21.15.33:0/2990211249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e7419a190 con 0x7f7e74108600 2024-05-15T14:10:29.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.030+0000 7f7e79b54700 1 --2- 172.21.15.33:0/2990211249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e741028a0 0x7f7e7419fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:10:29.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.030+0000 7f7e7ab56700 1 --2- 172.21.15.33:0/2990211249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e74108600 0x7f7e74199a30 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00b830 tx=0x7f7e6c004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:29.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.031+0000 7f7e6b7fe700 1 -- 172.21.15.33:0/2990211249 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c021070 con 0x7f7e74108600 2024-05-15T14:10:29.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.031+0000 7f7e7c5b9700 1 -- 172.21.15.33:0/2990211249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e7419a420 con 0x7f7e74108600 2024-05-15T14:10:29.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.031+0000 7f7e6b7fe700 1 -- 172.21.15.33:0/2990211249 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e6c007e30 con 0x7f7e74108600 2024-05-15T14:10:29.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.031+0000 7f7e6b7fe700 1 -- 172.21.15.33:0/2990211249 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c0047d0 con 0x7f7e74108600 2024-05-15T14:10:29.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.031+0000 7f7e7c5b9700 1 -- 172.21.15.33:0/2990211249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e741a6740 con 0x7f7e74108600 2024-05-15T14:10:29.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.032+0000 7f7e7c5b9700 1 -- 172.21.15.33:0/2990211249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e7406a230 con 0x7f7e74108600 2024-05-15T14:10:29.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.033+0000 7f7e6b7fe700 1 -- 172.21.15.33:0/2990211249 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7e6c007980 con 0x7f7e74108600 2024-05-15T14:10:29.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.034+0000 7f7e6b7fe700 1 --2- 172.21.15.33:0/2990211249 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f7e600b29f0 0x7f7e600b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:29.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.034+0000 7f7e7a355700 1 --2- 172.21.15.33:0/2990211249 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f7e600b29f0 0x7f7e600b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:29.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.034+0000 7f7e6b7fe700 1 -- 172.21.15.33:0/2990211249 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f7e6c00f020 con 0x7f7e74108600 2024-05-15T14:10:29.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.035+0000 7f7e7a355700 1 --2- 172.21.15.33:0/2990211249 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f7e600b29f0 0x7f7e600b4eb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7e7419ab70 tx=0x7f7e70009190 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:29.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.038+0000 7f7e6b7fe700 1 -- 172.21.15.33:0/2990211249 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7e6c02a020 con 0x7f7e74108600 2024-05-15T14:10:29.078 INFO:tasks.workunit.client.0.smithi033.stdout: 1 576957 5.43 MB/sec execute 364 sec latency 25016.183 ms 2024-05-15T14:10:29.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.317+0000 7f7e7c5b9700 1 -- 172.21.15.33:0/2990211249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e7402e560 con 0x7f7e74108600 2024-05-15T14:10:29.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.318+0000 7f7e6b7fe700 1 -- 172.21.15.33:0/2990211249 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11862 (secure 0 0 0) 0x7f7e6c02e3e0 con 0x7f7e74108600 2024-05-15T14:10:29.320 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T14:10:29.320 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2030117633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2030117633},{"type":"v1","addr":"172.21.15.33:6839","nonce":2030117633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1918979061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1918979061},{"type":"v1","addr":"172.21.15.33:6841","nonce":1918979061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T13:57:19.056417+0000","modified":"2024-05-15T14:08:23.686464+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24463,"mds_2":24461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/1629697841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1629697841},{"type":"v1","addr":"172.21.15.52:6837","nonce":1629697841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.52:6839/2696044471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2696044471},{"type":"v1","addr":"172.21.15.52:6839","nonce":2696044471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.81:6839/1554395940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":1554395940},{"type":"v1","addr":"172.21.15.81:6839","nonce":1554395940}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25915":{"gid":25915,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/1297849376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1297849376},{"type":"v1","addr":"172.21.15.81:6835","nonce":1297849376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24461,24463,24478]},"id":1}]} 2024-05-15T14:10:29.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.320+0000 7f7e7c5b9700 1 -- 172.21.15.33:0/2990211249 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f7e600b29f0 msgr2=0x7f7e600b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:29.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.320+0000 7f7e7c5b9700 1 --2- 172.21.15.33:0/2990211249 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f7e600b29f0 0x7f7e600b4eb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7e7419ab70 tx=0x7f7e70009190 comp rx=0 tx=0).stop 2024-05-15T14:10:29.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.320+0000 7f7e7c5b9700 1 -- 172.21.15.33:0/2990211249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e74108600 msgr2=0x7f7e74199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:29.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.320+0000 7f7e7c5b9700 1 --2- 172.21.15.33:0/2990211249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e74108600 0x7f7e74199a30 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00b830 tx=0x7f7e6c004ce0 comp rx=0 tx=0).stop 2024-05-15T14:10:29.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.320+0000 7f7e7c5b9700 1 -- 172.21.15.33:0/2990211249 shutdown_connections 2024-05-15T14:10:29.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.320+0000 7f7e7c5b9700 1 --2- 172.21.15.33:0/2990211249 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f7e600b29f0 0x7f7e600b4eb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:29.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.320+0000 7f7e7c5b9700 1 --2- 172.21.15.33:0/2990211249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e74108600 0x7f7e74199a30 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.320+0000 7f7e7c5b9700 1 --2- 172.21.15.33:0/2990211249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e74101ee0 0x7f7e7419f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.320+0000 7f7e7c5b9700 1 --2- 172.21.15.33:0/2990211249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e741028a0 0x7f7e7419fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.320+0000 7f7e7c5b9700 1 -- 172.21.15.33:0/2990211249 >> 172.21.15.33:0/2990211249 conn(0x7f7e74077760 msgr2=0x7f7e741009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.321+0000 7f7e7c5b9700 1 -- 172.21.15.33:0/2990211249 shutdown_connections 2024-05-15T14:10:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:29.321+0000 7f7e7c5b9700 1 -- 172.21.15.33:0/2990211249 wait complete. 2024-05-15T14:10:29.323 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 34 2024-05-15T14:10:29.644 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:29 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/2990211249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:10:29.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:29 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/2990211249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:10:29.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:29 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/2990211249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:10:30.078 INFO:tasks.workunit.client.0.smithi033.stdout: 1 576957 5.41 MB/sec execute 365 sec latency 26016.222 ms 2024-05-15T14:10:30.101 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24478 quiesce db --set-id 5860f899 --release --await' 2024-05-15T14:10:30.102 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24478 quiesce db --set-id 5860f899 --release --await 2024-05-15T14:10:30.355 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:10:30.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.402+0000 7ff128672640 1 -- 172.21.15.33:0/3577108384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff120102a30 msgr2=0x7ff120102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:30.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.402+0000 7ff128672640 1 --2- 172.21.15.33:0/3577108384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff120102a30 0x7ff120102eb0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff110009930 tx=0x7ff11002a420 comp rx=0 tx=0).stop 2024-05-15T14:10:30.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.402+0000 7ff128672640 1 -- 172.21.15.33:0/3577108384 shutdown_connections 2024-05-15T14:10:30.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.402+0000 7ff128672640 1 --2- 172.21.15.33:0/3577108384 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1201033f0 0x7ff12010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:30.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.402+0000 7ff128672640 1 --2- 172.21.15.33:0/3577108384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff120102a30 0x7ff120102eb0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:30.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.402+0000 7ff128672640 1 --2- 172.21.15.33:0/3577108384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff120108a40 0x7ff120108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:30.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.402+0000 7ff128672640 1 -- 172.21.15.33:0/3577108384 >> 172.21.15.33:0/3577108384 conn(0x7ff1200fe4e0 msgr2=0x7ff120100900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:30.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.403+0000 7ff128672640 1 -- 172.21.15.33:0/3577108384 shutdown_connections 2024-05-15T14:10:30.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.403+0000 7ff128672640 1 -- 172.21.15.33:0/3577108384 wait complete. 2024-05-15T14:10:30.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.403+0000 7ff128672640 1 Processor -- start 2024-05-15T14:10:30.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.403+0000 7ff128672640 1 -- start start 2024-05-15T14:10:30.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.403+0000 7ff128672640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff120102a30 0x7ff12010c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:30.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.403+0000 7ff128672640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1201033f0 0x7ff12010c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:30.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.404+0000 7ff128672640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff120108a40 0x7ff1201a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:30.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.404+0000 7ff128672640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff120075f80 con 0x7ff120102a30 2024-05-15T14:10:30.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.404+0000 7ff128672640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff120075e00 con 0x7ff120108a40 2024-05-15T14:10:30.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.404+0000 7ff128672640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ff120076100 con 0x7ff1201033f0 2024-05-15T14:10:30.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.404+0000 7ff12640e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff120102a30 0x7ff12010c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:30.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.404+0000 7ff12640e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff120102a30 0x7ff12010c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40478/0 (socket says 172.21.15.33:40478) 2024-05-15T14:10:30.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.404+0000 7ff12640e640 1 -- 172.21.15.33:0/2091717932 learned_addr learned my addr 172.21.15.33:0/2091717932 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:30.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.404+0000 7ff126c0f640 1 --2- 172.21.15.33:0/2091717932 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff120108a40 0x7ff1201a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:30.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.404+0000 7ff125c0d640 1 --2- 172.21.15.33:0/2091717932 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1201033f0 0x7ff12010c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:30.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.404+0000 7ff12640e640 1 -- 172.21.15.33:0/2091717932 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1201033f0 msgr2=0x7ff12010c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:30.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.404+0000 7ff12640e640 1 --2- 172.21.15.33:0/2091717932 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1201033f0 0x7ff12010c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:30.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.404+0000 7ff12640e640 1 -- 172.21.15.33:0/2091717932 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff120108a40 msgr2=0x7ff1201a53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.404+0000 7ff12640e640 1 --2- 172.21.15.33:0/2091717932 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff120108a40 0x7ff1201a53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.404+0000 7ff12640e640 1 -- 172.21.15.33:0/2091717932 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1201a5b10 con 0x7ff120102a30 2024-05-15T14:10:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.404+0000 7ff125c0d640 1 --2- 172.21.15.33:0/2091717932 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1201033f0 0x7ff12010c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:10:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.404+0000 7ff12640e640 1 --2- 172.21.15.33:0/2091717932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff120102a30 0x7ff12010c3a0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00d950 tx=0x7ff11c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.405+0000 7ff1177fe640 1 -- 172.21.15.33:0/2091717932 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c017070 con 0x7ff120102a30 2024-05-15T14:10:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.405+0000 7ff128672640 1 -- 172.21.15.33:0/2091717932 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1201a5e00 con 0x7ff120102a30 2024-05-15T14:10:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.405+0000 7ff1177fe640 1 -- 172.21.15.33:0/2091717932 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff11c009c30 con 0x7ff120102a30 2024-05-15T14:10:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.405+0000 7ff128672640 1 -- 172.21.15.33:0/2091717932 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1201ac490 con 0x7ff120102a30 2024-05-15T14:10:30.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.405+0000 7ff1177fe640 1 -- 172.21.15.33:0/2091717932 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c00f620 con 0x7ff120102a30 2024-05-15T14:10:30.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.407+0000 7ff1177fe640 1 -- 172.21.15.33:0/2091717932 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff11c00f7c0 con 0x7ff120102a30 2024-05-15T14:10:30.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.408+0000 7ff12b390740 1 Processor -- start 2024-05-15T14:10:30.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.408+0000 7ff12b390740 1 -- start start 2024-05-15T14:10:30.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.408+0000 7ff12b390740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5652b5d9e8f0 0x5652b5d9ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:30.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.408+0000 7ff12b390740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652b5d9f210 0x5652b5da9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:30.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.408+0000 7ff12b390740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b5da9ad0 0x5652b5dabec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:30.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.408+0000 7ff12b390740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5652b5d11c30 con 0x5652b5da9ad0 2024-05-15T14:10:30.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.408+0000 7ff12b390740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5652b5d017d0 con 0x5652b5d9e8f0 2024-05-15T14:10:30.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.408+0000 7ff12b390740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5652b5d05810 con 0x5652b5d9f210 2024-05-15T14:10:30.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.408+0000 7ff125c0d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b5da9ad0 0x5652b5dabec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:30.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.408+0000 7ff125c0d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b5da9ad0 0x5652b5dabec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40492/0 (socket says 172.21.15.33:40492) 2024-05-15T14:10:30.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.408+0000 7ff125c0d640 1 -- 172.21.15.33:0/1244954100 learned_addr learned my addr 172.21.15.33:0/1244954100 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.408+0000 7ff126c0f640 1 --2- 172.21.15.33:0/1244954100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5652b5d9e8f0 0x5652b5d9ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.408+0000 7ff12640e640 1 --2- 172.21.15.33:0/1244954100 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652b5d9f210 0x5652b5da9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.408+0000 7ff125c0d640 1 -- 172.21.15.33:0/1244954100 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652b5d9f210 msgr2=0x5652b5da9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.408+0000 7ff125c0d640 1 --2- 172.21.15.33:0/1244954100 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652b5d9f210 0x5652b5da9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.408+0000 7ff125c0d640 1 -- 172.21.15.33:0/1244954100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5652b5d9e8f0 msgr2=0x5652b5d9ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.408+0000 7ff125c0d640 1 --2- 172.21.15.33:0/1244954100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5652b5d9e8f0 0x5652b5d9ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.408+0000 7ff125c0d640 1 -- 172.21.15.33:0/1244954100 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652b5a16210 con 0x5652b5da9ad0 2024-05-15T14:10:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.409+0000 7ff125c0d640 1 --2- 172.21.15.33:0/1244954100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b5da9ad0 0x5652b5dabec0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ff11002aa10 tx=0x7ff110004140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f085a311e927cd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.409+0000 7ff114ff9640 1 -- 172.21.15.33:0/1244954100 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff110035030 con 0x5652b5da9ad0 2024-05-15T14:10:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.409+0000 7ff1177fe640 1 --2- 172.21.15.33:0/2091717932 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ff10c0b2950 0x7ff10c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.409+0000 7ff114ff9640 1 -- 172.21.15.33:0/1244954100 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1100054f0 con 0x5652b5da9ad0 2024-05-15T14:10:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.409+0000 7ff114ff9640 1 -- 172.21.15.33:0/1244954100 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1100035f0 con 0x5652b5da9ad0 2024-05-15T14:10:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.409+0000 7ff126c0f640 1 --2- 172.21.15.33:0/2091717932 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ff10c0b2950 0x7ff10c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.410+0000 7ff1177fe640 1 -- 172.21.15.33:0/2091717932 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7ff11c0e20c0 con 0x7ff120102a30 2024-05-15T14:10:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.410+0000 7ff12b390740 1 -- 172.21.15.33:0/1244954100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b5da9ad0 msgr2=0x5652b5dabec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.410+0000 7ff126c0f640 1 --2- 172.21.15.33:0/2091717932 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ff10c0b2950 0x7ff10c0b4e10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff118005fd0 tx=0x7ff118009500 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.410+0000 7ff12b390740 1 --2- 172.21.15.33:0/1244954100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b5da9ad0 0x5652b5dabec0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ff11002aa10 tx=0x7ff110004140 comp rx=0 tx=0).stop 2024-05-15T14:10:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.410+0000 7ff12b390740 1 -- 172.21.15.33:0/1244954100 shutdown_connections 2024-05-15T14:10:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.410+0000 7ff12b390740 1 --2- 172.21.15.33:0/1244954100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b5da9ad0 0x5652b5dabec0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.410+0000 7ff12b390740 1 --2- 172.21.15.33:0/1244954100 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652b5d9f210 0x5652b5da9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.410+0000 7ff12b390740 1 --2- 172.21.15.33:0/1244954100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5652b5d9e8f0 0x5652b5d9ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.410+0000 7ff12b390740 1 -- 172.21.15.33:0/1244954100 >> 172.21.15.33:0/1244954100 conn(0x5652b5d9ae00 msgr2=0x5652b5d9b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.410+0000 7ff12b390740 1 -- 172.21.15.33:0/1244954100 shutdown_connections 2024-05-15T14:10:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.410+0000 7ff12b390740 1 -- 172.21.15.33:0/1244954100 wait complete. 2024-05-15T14:10:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.410+0000 7ff12b390740 1 -- start start 2024-05-15T14:10:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.410+0000 7ff12b390740 1 Processor -- start 2024-05-15T14:10:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.411+0000 7ff12b390740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b5d9e8f0 0x5652b5da93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.411+0000 7ff12b390740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5652b5d9f210 0x5652b5da66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.411+0000 7ff12b390740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652b5da9ad0 0x5652b5da6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.411+0000 7ff12b390740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5652b59e7790 con 0x5652b5d9e8f0 2024-05-15T14:10:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.411+0000 7ff12b390740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5652b5cbd890 con 0x5652b5d9f210 2024-05-15T14:10:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.411+0000 7ff12b390740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5652b5dac740 con 0x5652b5da9ad0 2024-05-15T14:10:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.411+0000 7ff12b390740 10 client.0 _ll_register_callbacks cb 0x5652b5cc0d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:10:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.411+0000 7ff12640e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b5d9e8f0 0x5652b5da93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.411+0000 7ff12640e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b5d9e8f0 0x5652b5da93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40500/0 (socket says 172.21.15.33:40500) 2024-05-15T14:10:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.411+0000 7ff12640e640 1 -- 172.21.15.33:0/1104768204 learned_addr learned my addr 172.21.15.33:0/1104768204 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.411+0000 7ff125c0d640 1 --2- 172.21.15.33:0/1104768204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5652b5d9f210 0x5652b5da66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.411+0000 7ff126c0f640 1 --2- 172.21.15.33:0/1104768204 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652b5da9ad0 0x5652b5da6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.411+0000 7ff12640e640 1 -- 172.21.15.33:0/1104768204 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652b5da9ad0 msgr2=0x5652b5da6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.411+0000 7ff12640e640 1 --2- 172.21.15.33:0/1104768204 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652b5da9ad0 0x5652b5da6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.411+0000 7ff12640e640 1 -- 172.21.15.33:0/1104768204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5652b5d9f210 msgr2=0x5652b5da66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.411+0000 7ff12640e640 1 --2- 172.21.15.33:0/1104768204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5652b5d9f210 0x5652b5da66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.411+0000 7ff12640e640 1 -- 172.21.15.33:0/1104768204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652b5ae2620 con 0x5652b5d9e8f0 2024-05-15T14:10:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.411+0000 7ff12640e640 1 --2- 172.21.15.33:0/1104768204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b5d9e8f0 0x5652b5da93e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7ff11c000c00 tx=0x7ff11c0a5380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20cffe6ceb51b5d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.411+0000 7ff109ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:10:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.411+0000 7ff109ffb640 1 -- 172.21.15.33:0/1104768204 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c020070 con 0x5652b5d9e8f0 2024-05-15T14:10:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.411+0000 7ff109ffb640 1 -- 172.21.15.33:0/1104768204 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7ff11c0a85f0 con 0x5652b5d9e8f0 2024-05-15T14:10:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.412+0000 7ff12b390740 1 -- 172.21.15.33:0/1104768204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1100054f0 con 0x5652b5d9e8f0 2024-05-15T14:10:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.412+0000 7ff109ffb640 1 -- 172.21.15.33:0/1104768204 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff11c0dfc90 con 0x5652b5d9e8f0 2024-05-15T14:10:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.412+0000 7ff12b390740 10 client.16581 fetch_fsmap learned FSMap version 34 2024-05-15T14:10:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.412+0000 7ff12b390740 1 -- 172.21.15.33:0/1104768204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5652b5db8180 con 0x5652b5d9e8f0 2024-05-15T14:10:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.412+0000 7ff109ffb640 1 -- 172.21.15.33:0/1104768204 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1100054f0 con 0x5652b5d9e8f0 2024-05-15T14:10:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.412+0000 7ff109ffb640 1 -- 172.21.15.33:0/1104768204 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7ff11c0a4d00 con 0x5652b5d9e8f0 2024-05-15T14:10:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.412+0000 7ff12b390740 10 client.16581 fetch_fsmap finished waiting for FSMap version 34 2024-05-15T14:10:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.412+0000 7ff12b390740 10 client.16581 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:10:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.412+0000 7ff12b390740 20 client.16581 populate_metadata read hostname 'smithi033' 2024-05-15T14:10:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.412+0000 7ff12b390740 1 --2- 172.21.15.33:0/1104768204 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5652b5db96e0 0x5652b5dbbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.412+0000 7ff12b390740 4 client.16581 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:10:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.412+0000 7ff12b390740 1 -- 172.21.15.33:0/1104768204 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652b5d13b80 con 0x5652b5db96e0 2024-05-15T14:10:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.412+0000 7ff125c0d640 1 --2- 172.21.15.33:0/1104768204 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5652b5db96e0 0x5652b5dbbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.413+0000 7ff125c0d640 1 --2- 172.21.15.33:0/1104768204 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5652b5db96e0 0x5652b5dbbb70 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dd5f50a1c3e9b63 server_cookie=1efcc8bdcfcba25c in_seq=0 out_seq=0 2024-05-15T14:10:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.413+0000 7ff109ffb640 10 client.16581 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:10:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.414+0000 7ff109ffb640 1 -- 172.21.15.33:0/1104768204 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5652b5d13b80 con 0x5652b5db96e0 2024-05-15T14:10:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.414+0000 7ff109ffb640 10 client.16581 handle_command_reply: tid=0 2024-05-15T14:10:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.414+0000 7ff12b390740 1 client.16581 shutdown 2024-05-15T14:10:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.414+0000 7ff12b390740 1 -- 172.21.15.33:0/1104768204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b5d9e8f0 msgr2=0x5652b5da93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.414+0000 7ff12b390740 1 --2- 172.21.15.33:0/1104768204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b5d9e8f0 0x5652b5da93e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7ff11c000c00 tx=0x7ff11c0a5380 comp rx=0 tx=0).stop 2024-05-15T14:10:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.414+0000 7ff12b390740 1 -- 172.21.15.33:0/1104768204 shutdown_connections 2024-05-15T14:10:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.414+0000 7ff12b390740 1 --2- 172.21.15.33:0/1104768204 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5652b5db96e0 0x5652b5dbbb70 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.414+0000 7ff12b390740 1 --2- 172.21.15.33:0/1104768204 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652b5da9ad0 0x5652b5da6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.414+0000 7ff12b390740 1 --2- 172.21.15.33:0/1104768204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5652b5d9f210 0x5652b5da66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.414+0000 7ff109ffb640 0 client.16581 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:10:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.414+0000 7ff12b390740 1 --2- 172.21.15.33:0/1104768204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b5d9e8f0 0x5652b5da93e0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.414+0000 7ff12b390740 1 -- 172.21.15.33:0/1104768204 >> 172.21.15.33:0/1104768204 conn(0x5652b5d9ae00 msgr2=0x5652b5d9a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.414+0000 7ff12b390740 1 -- 172.21.15.33:0/1104768204 shutdown_connections 2024-05-15T14:10:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.415+0000 7ff12b390740 1 -- 172.21.15.33:0/1104768204 wait complete. 2024-05-15T14:10:30.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.415+0000 7ff12b390740 20 client.16581 trim_cache size 0 max 16384 2024-05-15T14:10:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.430+0000 7ff12b390740 1 Processor -- start 2024-05-15T14:10:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.430+0000 7ff12b390740 1 -- start start 2024-05-15T14:10:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.430+0000 7ff12b390740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652b5d9e8f0 0x5652b5d9a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.430+0000 7ff12b390740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5652b5d9f210 0x5652b5da1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.430+0000 7ff12b390740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b5da9ad0 0x5652b5da8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.430+0000 7ff12b390740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff104011130 con 0x5652b5da9ad0 2024-05-15T14:10:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.430+0000 7ff12b390740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff10400e970 con 0x5652b5d9f210 2024-05-15T14:10:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.430+0000 7ff12b390740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ff10400c7e0 con 0x5652b5d9e8f0 2024-05-15T14:10:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.430+0000 7ff126c0f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b5da9ad0 0x5652b5da8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:30.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.430+0000 7ff126c0f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b5da9ad0 0x5652b5da8f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40510/0 (socket says 172.21.15.33:40510) 2024-05-15T14:10:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.430+0000 7ff126c0f640 1 -- 172.21.15.33:0/3151286727 learned_addr learned my addr 172.21.15.33:0/3151286727 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.430+0000 7ff12640e640 1 --2- 172.21.15.33:0/3151286727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652b5d9e8f0 0x5652b5d9a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.430+0000 7ff125c0d640 1 --2- 172.21.15.33:0/3151286727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5652b5d9f210 0x5652b5da1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.430+0000 7ff126c0f640 1 -- 172.21.15.33:0/3151286727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652b5d9e8f0 msgr2=0x5652b5d9a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.430+0000 7ff126c0f640 1 --2- 172.21.15.33:0/3151286727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652b5d9e8f0 0x5652b5d9a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.430+0000 7ff126c0f640 1 -- 172.21.15.33:0/3151286727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5652b5d9f210 msgr2=0x5652b5da1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.430+0000 7ff126c0f640 1 --2- 172.21.15.33:0/3151286727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5652b5d9f210 0x5652b5da1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.430+0000 7ff126c0f640 1 -- 172.21.15.33:0/3151286727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652b5d9fb80 con 0x5652b5da9ad0 2024-05-15T14:10:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.431+0000 7ff126c0f640 1 --2- 172.21.15.33:0/3151286727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b5da9ad0 0x5652b5da8f10 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ff118003e80 tx=0x7ff11801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24de4450e53599e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.431+0000 7ff10bfff640 1 -- 172.21.15.33:0/3151286727 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff118004b20 con 0x5652b5da9ad0 2024-05-15T14:10:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.431+0000 7ff10bfff640 1 -- 172.21.15.33:0/3151286727 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff118004cc0 con 0x5652b5da9ad0 2024-05-15T14:10:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.431+0000 7ff10bfff640 1 -- 172.21.15.33:0/3151286727 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1180054e0 con 0x5652b5da9ad0 2024-05-15T14:10:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.431+0000 7ff12b390740 1 -- 172.21.15.33:0/3151286727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b5da9ad0 msgr2=0x5652b5da8f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.431+0000 7ff12b390740 1 --2- 172.21.15.33:0/3151286727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b5da9ad0 0x5652b5da8f10 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ff118003e80 tx=0x7ff11801a000 comp rx=0 tx=0).stop 2024-05-15T14:10:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.431+0000 7ff12b390740 1 -- 172.21.15.33:0/3151286727 shutdown_connections 2024-05-15T14:10:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.431+0000 7ff12b390740 1 --2- 172.21.15.33:0/3151286727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b5da9ad0 0x5652b5da8f10 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.431+0000 7ff12b390740 1 --2- 172.21.15.33:0/3151286727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5652b5d9f210 0x5652b5da1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.431+0000 7ff12b390740 1 --2- 172.21.15.33:0/3151286727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652b5d9e8f0 0x5652b5d9a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.431+0000 7ff12b390740 1 -- 172.21.15.33:0/3151286727 >> 172.21.15.33:0/3151286727 conn(0x5652b5d9ae00 msgr2=0x5652b5da02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.431+0000 7ff12b390740 1 -- 172.21.15.33:0/3151286727 shutdown_connections 2024-05-15T14:10:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.431+0000 7ff12b390740 1 -- 172.21.15.33:0/3151286727 wait complete. 2024-05-15T14:10:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.432+0000 7ff12b390740 1 -- start start 2024-05-15T14:10:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.432+0000 7ff12b390740 1 Processor -- start 2024-05-15T14:10:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.432+0000 7ff12b390740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652b5d9e8f0 0x5652b5db1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.432+0000 7ff12b390740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b5d9f210 0x5652b5dad280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.432+0000 7ff12b390740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5652b5da9ad0 0x5652b5dad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.432+0000 7ff12b390740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff10400ce30 con 0x5652b5d9f210 2024-05-15T14:10:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.432+0000 7ff12b390740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff10400e2a0 con 0x5652b5da9ad0 2024-05-15T14:10:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.432+0000 7ff12b390740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7ff10400d500 con 0x5652b5d9e8f0 2024-05-15T14:10:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.432+0000 7ff12b390740 10 client.0 _ll_register_callbacks cb 0x7ff11c00df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:10:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.432+0000 7ff125c0d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b5d9f210 0x5652b5dad280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.432+0000 7ff125c0d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b5d9f210 0x5652b5dad280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40520/0 (socket says 172.21.15.33:40520) 2024-05-15T14:10:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.432+0000 7ff125c0d640 1 -- 172.21.15.33:0/2946579868 learned_addr learned my addr 172.21.15.33:0/2946579868 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.432+0000 7ff12640e640 1 --2- 172.21.15.33:0/2946579868 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652b5d9e8f0 0x5652b5db1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.432+0000 7ff125c0d640 1 -- 172.21.15.33:0/2946579868 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652b5d9e8f0 msgr2=0x5652b5db1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.432+0000 7ff125c0d640 1 --2- 172.21.15.33:0/2946579868 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652b5d9e8f0 0x5652b5db1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.432+0000 7ff125c0d640 1 -- 172.21.15.33:0/2946579868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5652b5da9ad0 msgr2=0x5652b5dad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.432+0000 7ff125c0d640 1 --2- 172.21.15.33:0/2946579868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5652b5da9ad0 0x5652b5dad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.432+0000 7ff125c0d640 1 -- 172.21.15.33:0/2946579868 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652b5dadff0 con 0x5652b5d9f210 2024-05-15T14:10:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.433+0000 7ff125c0d640 1 --2- 172.21.15.33:0/2946579868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b5d9f210 0x5652b5dad280 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff110031440 tx=0x7ff1100048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c8bba12e0cfbee server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.433+0000 7ff109ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:10:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.433+0000 7ff109ffb640 1 -- 172.21.15.33:0/2946579868 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff110056060 con 0x5652b5d9f210 2024-05-15T14:10:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.433+0000 7ff109ffb640 1 -- 172.21.15.33:0/2946579868 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7ff110005020 con 0x5652b5d9f210 2024-05-15T14:10:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.433+0000 7ff12b390740 1 -- 172.21.15.33:0/2946579868 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff118004cc0 con 0x5652b5d9f210 2024-05-15T14:10:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.433+0000 7ff109ffb640 1 -- 172.21.15.33:0/2946579868 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff110035040 con 0x5652b5d9f210 2024-05-15T14:10:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.433+0000 7ff12b390740 10 client.16587 fetch_fsmap learned FSMap version 34 2024-05-15T14:10:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.433+0000 7ff12b390740 1 -- 172.21.15.33:0/2946579868 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5652b5e24010 con 0x5652b5d9f210 2024-05-15T14:10:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.433+0000 7ff109ffb640 1 -- 172.21.15.33:0/2946579868 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff118004cc0 con 0x5652b5d9f210 2024-05-15T14:10:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.433+0000 7ff109ffb640 1 -- 172.21.15.33:0/2946579868 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7ff1100314a0 con 0x5652b5d9f210 2024-05-15T14:10:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.433+0000 7ff12b390740 10 client.16587 fetch_fsmap finished waiting for FSMap version 34 2024-05-15T14:10:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.433+0000 7ff12b390740 10 client.16587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:10:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.433+0000 7ff12b390740 20 client.16587 populate_metadata read hostname 'smithi033' 2024-05-15T14:10:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.433+0000 7ff12b390740 1 --2- 172.21.15.33:0/2946579868 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5652b5e24410 0x5652b5e26800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.433+0000 7ff12b390740 4 client.16587 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:10:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.433+0000 7ff12b390740 1 -- 172.21.15.33:0/2946579868 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5652b5d0d360 con 0x5652b5e24410 2024-05-15T14:10:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.433+0000 7ff12640e640 1 --2- 172.21.15.33:0/2946579868 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5652b5e24410 0x5652b5e26800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.434+0000 7ff12640e640 1 --2- 172.21.15.33:0/2946579868 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5652b5e24410 0x5652b5e26800 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67604711a63431d7 server_cookie=e2bdb5419e9aa426 in_seq=0 out_seq=0 2024-05-15T14:10:30.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.434+0000 7ff109ffb640 10 client.16587 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:10:30.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.472+0000 7ff109ffb640 1 -- 172.21.15.33:0/2946579868 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5652b5d0d360 con 0x5652b5e24410 2024-05-15T14:10:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.472+0000 7ff109ffb640 10 client.16587 handle_command_reply: tid=0 2024-05-15T14:10:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.472+0000 7ff12b390740 1 client.16587 shutdown 2024-05-15T14:10:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.472+0000 7ff12b390740 1 -- 172.21.15.33:0/2946579868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b5d9f210 msgr2=0x5652b5dad280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.472+0000 7ff12b390740 1 --2- 172.21.15.33:0/2946579868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b5d9f210 0x5652b5dad280 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff110031440 tx=0x7ff1100048d0 comp rx=0 tx=0).stop 2024-05-15T14:10:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.472+0000 7ff12b390740 1 -- 172.21.15.33:0/2946579868 shutdown_connections 2024-05-15T14:10:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.472+0000 7ff12b390740 1 --2- 172.21.15.33:0/2946579868 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5652b5e24410 0x5652b5e26800 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.472+0000 7ff12b390740 1 --2- 172.21.15.33:0/2946579868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5652b5da9ad0 0x5652b5dad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.472+0000 7ff109ffb640 0 client.16587 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:10:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.472+0000 7ff12b390740 1 --2- 172.21.15.33:0/2946579868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5652b5d9f210 0x5652b5dad280 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.472+0000 7ff12b390740 1 --2- 172.21.15.33:0/2946579868 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5652b5d9e8f0 0x5652b5db1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.472+0000 7ff12b390740 1 -- 172.21.15.33:0/2946579868 >> 172.21.15.33:0/2946579868 conn(0x5652b5d9ae00 msgr2=0x5652b5db4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.473+0000 7ff12b390740 1 -- 172.21.15.33:0/2946579868 shutdown_connections 2024-05-15T14:10:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.473+0000 7ff12b390740 1 -- 172.21.15.33:0/2946579868 wait complete. 2024-05-15T14:10:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.473+0000 7ff12b390740 20 client.16587 trim_cache size 0 max 16384 2024-05-15T14:10:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:10:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:10:30.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:10:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.473+0000 7ff128672640 1 -- 172.21.15.33:0/2091717932 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ff10c0b2950 msgr2=0x7ff10c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.473+0000 7ff128672640 1 --2- 172.21.15.33:0/2091717932 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ff10c0b2950 0x7ff10c0b4e10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff118005fd0 tx=0x7ff118009500 comp rx=0 tx=0).stop 2024-05-15T14:10:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.473+0000 7ff128672640 1 -- 172.21.15.33:0/2091717932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff120102a30 msgr2=0x7ff12010c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.473+0000 7ff128672640 1 --2- 172.21.15.33:0/2091717932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff120102a30 0x7ff12010c3a0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00d950 tx=0x7ff11c00de10 comp rx=0 tx=0).stop 2024-05-15T14:10:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.473+0000 7ff128672640 1 -- 172.21.15.33:0/2091717932 shutdown_connections 2024-05-15T14:10:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.473+0000 7ff128672640 1 --2- 172.21.15.33:0/2091717932 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7ff10c0b2950 0x7ff10c0b4e10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.473+0000 7ff128672640 1 --2- 172.21.15.33:0/2091717932 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff120108a40 0x7ff1201a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.473+0000 7ff128672640 1 --2- 172.21.15.33:0/2091717932 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1201033f0 0x7ff12010c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.473+0000 7ff128672640 1 --2- 172.21.15.33:0/2091717932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff120102a30 0x7ff12010c3a0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.473+0000 7ff128672640 1 -- 172.21.15.33:0/2091717932 >> 172.21.15.33:0/2091717932 conn(0x7ff1200fe4e0 msgr2=0x7ff120106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.473+0000 7ff128672640 1 -- 172.21.15.33:0/2091717932 shutdown_connections 2024-05-15T14:10:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:30.473+0000 7ff128672640 1 -- 172.21.15.33:0/2091717932 wait complete. 2024-05-15T14:10:30.739 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:30 smithi033 ceph-mon[31650]: pgmap v317: 97 pgs: 97 active+clean; 176 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 1.3 MiB/s wr, 60 op/s 2024-05-15T14:10:30.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:30 smithi052 ceph-mon[36190]: pgmap v317: 97 pgs: 97 active+clean; 176 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 1.3 MiB/s wr, 60 op/s 2024-05-15T14:10:30.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:30 smithi081 ceph-mon[36434]: pgmap v317: 97 pgs: 97 active+clean; 176 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 1.3 MiB/s wr, 60 op/s 2024-05-15T14:10:31.078 INFO:tasks.workunit.client.0.smithi033.stdout: 1 576957 5.40 MB/sec execute 366 sec latency 27016.258 ms 2024-05-15T14:10:32.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.033+0000 7f98a5b16700 1 -- 172.21.15.33:0/2736617335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a01084b0 msgr2=0x7f98a0108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:32.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.033+0000 7f98a5b16700 1 --2- 172.21.15.33:0/2736617335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a01084b0 0x7f98a0108890 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f9890009990 tx=0x7f9890009ca0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.033+0000 7f98a5b16700 1 -- 172.21.15.33:0/2736617335 shutdown_connections 2024-05-15T14:10:32.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.033+0000 7f98a5b16700 1 --2- 172.21.15.33:0/2736617335 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98a0102e30 0x7f98a010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.033+0000 7f98a5b16700 1 --2- 172.21.15.33:0/2736617335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a01084b0 0x7f98a0108890 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.033+0000 7f98a5b16700 1 --2- 172.21.15.33:0/2736617335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f98a0102470 0x7f98a01028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.033+0000 7f98a5b16700 1 -- 172.21.15.33:0/2736617335 >> 172.21.15.33:0/2736617335 conn(0x7f98a00793f0 msgr2=0x7f98a0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:32.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.033+0000 7f98a5b16700 1 -- 172.21.15.33:0/2736617335 shutdown_connections 2024-05-15T14:10:32.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.033+0000 7f98a5b16700 1 -- 172.21.15.33:0/2736617335 wait complete. 2024-05-15T14:10:32.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.033+0000 7f98a5b16700 1 Processor -- start 2024-05-15T14:10:32.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.034+0000 7f98a5b16700 1 -- start start 2024-05-15T14:10:32.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.034+0000 7f98a5b16700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f98a0102470 0x7f98a019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:32.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.034+0000 7f98a5b16700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98a0102e30 0x7f98a019b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:32.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.034+0000 7f98a5b16700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a01084b0 0x7f98a01a5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:32.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.034+0000 7f98a5b16700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f98a0112690 con 0x7f98a01084b0 2024-05-15T14:10:32.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.034+0000 7f98a5b16700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f98a0112510 con 0x7f98a0102470 2024-05-15T14:10:32.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.034+0000 7f98a5b16700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f98a0112810 con 0x7f98a0102e30 2024-05-15T14:10:32.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.034+0000 7f989ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a01084b0 0x7f98a01a5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:32.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.034+0000 7f989ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a01084b0 0x7f98a01a5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40542/0 (socket says 172.21.15.33:40542) 2024-05-15T14:10:32.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.034+0000 7f989effd700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98a0102e30 0x7f98a019b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:32.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.034+0000 7f989ffff700 1 -- 172.21.15.33:0/3968444403 learned_addr learned my addr 172.21.15.33:0/3968444403 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:32.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.034+0000 7f989f7fe700 1 --2- 172.21.15.33:0/3968444403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f98a0102470 0x7f98a019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:32.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.034+0000 7f989ffff700 1 -- 172.21.15.33:0/3968444403 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98a0102e30 msgr2=0x7f98a019b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:32.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.034+0000 7f989ffff700 1 --2- 172.21.15.33:0/3968444403 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98a0102e30 0x7f98a019b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.034+0000 7f989ffff700 1 -- 172.21.15.33:0/3968444403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f98a0102470 msgr2=0x7f98a019ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:32.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.034+0000 7f989ffff700 1 --2- 172.21.15.33:0/3968444403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f98a0102470 0x7f98a019ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.034+0000 7f989ffff700 1 -- 172.21.15.33:0/3968444403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a01a6110 con 0x7f98a01084b0 2024-05-15T14:10:32.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.034+0000 7f989effd700 1 --2- 172.21.15.33:0/3968444403 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98a0102e30 0x7f98a019b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:10:32.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.035+0000 7f989ffff700 1 --2- 172.21.15.33:0/3968444403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a01084b0 0x7f98a01a5bd0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f9898007ee0 tx=0x7f989800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:32.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.035+0000 7f989cff9700 1 -- 172.21.15.33:0/3968444403 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9898004bb0 con 0x7f98a01084b0 2024-05-15T14:10:32.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.035+0000 7f98a5b16700 1 -- 172.21.15.33:0/3968444403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a01a62e0 con 0x7f98a01084b0 2024-05-15T14:10:32.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.035+0000 7f989cff9700 1 -- 172.21.15.33:0/3968444403 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9898004d50 con 0x7f98a01084b0 2024-05-15T14:10:32.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.035+0000 7f989cff9700 1 -- 172.21.15.33:0/3968444403 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9898005920 con 0x7f98a01084b0 2024-05-15T14:10:32.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.035+0000 7f98a5b16700 1 -- 172.21.15.33:0/3968444403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a01a6820 con 0x7f98a01084b0 2024-05-15T14:10:32.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.037+0000 7f989cff9700 1 -- 172.21.15.33:0/3968444403 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f989801a020 con 0x7f98a01084b0 2024-05-15T14:10:32.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.038+0000 7f989cff9700 1 --2- 172.21.15.33:0/3968444403 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f98880b29f0 0x7f98880b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:32.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.038+0000 7f989cff9700 1 -- 172.21.15.33:0/3968444403 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f98980e1ce0 con 0x7f98a01084b0 2024-05-15T14:10:32.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.038+0000 7f989f7fe700 1 --2- 172.21.15.33:0/3968444403 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f98880b29f0 0x7f98880b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:32.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.038+0000 7f98b03acb80 1 Processor -- start 2024-05-15T14:10:32.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.039+0000 7f98b03acb80 1 -- start start 2024-05-15T14:10:32.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.039+0000 7f989f7fe700 1 --2- 172.21.15.33:0/3968444403 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f98880b29f0 0x7f98880b4eb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f989000b5c0 tx=0x7f9890005fd0 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:32.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.039+0000 7f98b03acb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d99ab640 0x5578d99aba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:32.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.039+0000 7f98b03acb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d99abf60 0x5578d999d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:32.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.039+0000 7f98b03acb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5578d999dad0 0x5578d999df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:32.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.039+0000 7f98b03acb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5578d98fbd90 con 0x5578d99ab640 2024-05-15T14:10:32.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.039+0000 7f98b03acb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5578d98eb930 con 0x5578d99abf60 2024-05-15T14:10:32.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.039+0000 7f98b03acb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5578d98ef970 con 0x5578d999dad0 2024-05-15T14:10:32.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.039+0000 7f989f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d99ab640 0x5578d99aba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:32.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.039+0000 7f989f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d99ab640 0x5578d99aba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40558/0 (socket says 172.21.15.33:40558) 2024-05-15T14:10:32.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.039+0000 7f989f7fe700 1 -- 172.21.15.33:0/760957375 learned_addr learned my addr 172.21.15.33:0/760957375 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:32.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.039+0000 7f989effd700 1 --2- 172.21.15.33:0/760957375 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d99abf60 0x5578d999d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:32.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.039+0000 7f989ffff700 1 --2- 172.21.15.33:0/760957375 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5578d999dad0 0x5578d999df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:32.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.039+0000 7f989f7fe700 1 -- 172.21.15.33:0/760957375 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5578d999dad0 msgr2=0x5578d999df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:32.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.039+0000 7f989f7fe700 1 --2- 172.21.15.33:0/760957375 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5578d999dad0 0x5578d999df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.039+0000 7f989f7fe700 1 -- 172.21.15.33:0/760957375 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d99abf60 msgr2=0x5578d999d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:32.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.039+0000 7f989f7fe700 1 --2- 172.21.15.33:0/760957375 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d99abf60 0x5578d999d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.039+0000 7f989f7fe700 1 -- 172.21.15.33:0/760957375 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578d983b9d0 con 0x5578d99ab640 2024-05-15T14:10:32.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.039+0000 7f989ffff700 1 --2- 172.21.15.33:0/760957375 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5578d999dad0 0x5578d999df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:10:32.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.039+0000 7f989f7fe700 1 --2- 172.21.15.33:0/760957375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d99ab640 0x5578d99aba20 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f989000ba90 tx=0x7f989001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc35f61a8491a90 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:32.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.040+0000 7f9885ffb700 1 -- 172.21.15.33:0/760957375 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890034070 con 0x5578d99ab640 2024-05-15T14:10:32.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.040+0000 7f9885ffb700 1 -- 172.21.15.33:0/760957375 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9890024930 con 0x5578d99ab640 2024-05-15T14:10:32.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.040+0000 7f9885ffb700 1 -- 172.21.15.33:0/760957375 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989002f9a0 con 0x5578d99ab640 2024-05-15T14:10:32.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.040+0000 7f98b03acb80 1 -- 172.21.15.33:0/760957375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d99ab640 msgr2=0x5578d99aba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:32.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.040+0000 7f98b03acb80 1 --2- 172.21.15.33:0/760957375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d99ab640 0x5578d99aba20 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f989000ba90 tx=0x7f989001f000 comp rx=0 tx=0).stop 2024-05-15T14:10:32.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.040+0000 7f98b03acb80 1 -- 172.21.15.33:0/760957375 shutdown_connections 2024-05-15T14:10:32.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.040+0000 7f98b03acb80 1 --2- 172.21.15.33:0/760957375 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5578d999dad0 0x5578d999df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.040+0000 7f98b03acb80 1 --2- 172.21.15.33:0/760957375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d99ab640 0x5578d99aba20 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.040+0000 7f98b03acb80 1 --2- 172.21.15.33:0/760957375 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d99abf60 0x5578d999d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.040+0000 7f98b03acb80 1 -- 172.21.15.33:0/760957375 >> 172.21.15.33:0/760957375 conn(0x5578d98c8350 msgr2=0x5578d9997ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:32.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.040+0000 7f98b03acb80 1 -- 172.21.15.33:0/760957375 shutdown_connections 2024-05-15T14:10:32.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.040+0000 7f98b03acb80 1 -- 172.21.15.33:0/760957375 wait complete. 2024-05-15T14:10:32.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.041+0000 7f98b03acb80 1 -- start start 2024-05-15T14:10:32.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.041+0000 7f98b03acb80 1 Processor -- start 2024-05-15T14:10:32.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.041+0000 7f98b03acb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5578d999dad0 0x5578d99b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:32.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.041+0000 7f98b03acb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d99ab640 0x5578d99aef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:32.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.041+0000 7f98b03acb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d99abf60 0x5578d99af4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:32.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.041+0000 7f98b03acb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5578d97f20a0 con 0x5578d99abf60 2024-05-15T14:10:32.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.041+0000 7f98b03acb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5578d988d8c0 con 0x5578d99ab640 2024-05-15T14:10:32.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.041+0000 7f98b03acb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5578d99b2920 con 0x5578d999dad0 2024-05-15T14:10:32.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.041+0000 7f98b03acb80 10 client.0 _ll_register_callbacks cb 0x5578d9882680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:10:32.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.041+0000 7f989ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d99abf60 0x5578d99af4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:32.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.042+0000 7f989ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d99abf60 0x5578d99af4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40564/0 (socket says 172.21.15.33:40564) 2024-05-15T14:10:32.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.042+0000 7f989ffff700 1 -- 172.21.15.33:0/3727033274 learned_addr learned my addr 172.21.15.33:0/3727033274 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:32.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.042+0000 7f989f7fe700 1 --2- 172.21.15.33:0/3727033274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5578d999dad0 0x5578d99b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:32.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.042+0000 7f989effd700 1 --2- 172.21.15.33:0/3727033274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d99ab640 0x5578d99aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:32.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.042+0000 7f989ffff700 1 -- 172.21.15.33:0/3727033274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5578d999dad0 msgr2=0x5578d99b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:32.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.042+0000 7f989ffff700 1 --2- 172.21.15.33:0/3727033274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5578d999dad0 0x5578d99b1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.042+0000 7f989ffff700 1 -- 172.21.15.33:0/3727033274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d99ab640 msgr2=0x5578d99aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:32.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.042+0000 7f989ffff700 1 --2- 172.21.15.33:0/3727033274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d99ab640 0x5578d99aef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.042+0000 7f989ffff700 1 -- 172.21.15.33:0/3727033274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578d988a790 con 0x5578d99abf60 2024-05-15T14:10:32.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.042+0000 7f989effd700 1 --2- 172.21.15.33:0/3727033274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d99ab640 0x5578d99aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:10:32.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.042+0000 7f989ffff700 1 --2- 172.21.15.33:0/3727033274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d99abf60 0x5578d99af4c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f98980a8f40 tx=0x7f98980a94c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac46a7827426f120 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:32.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.042+0000 7f987effd700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:10:32.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.042+0000 7f987effd700 1 -- 172.21.15.33:0/3727033274 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98980a96b0 con 0x5578d99abf60 2024-05-15T14:10:32.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.042+0000 7f987effd700 1 -- 172.21.15.33:0/3727033274 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f98980dfd80 con 0x5578d99abf60 2024-05-15T14:10:32.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.042+0000 7f98b03acb80 1 -- 172.21.15.33:0/3727033274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9890024930 con 0x5578d99abf60 2024-05-15T14:10:32.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.042+0000 7f987effd700 1 -- 172.21.15.33:0/3727033274 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f98980a8fb0 con 0x5578d99abf60 2024-05-15T14:10:32.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.042+0000 7f98b03acb80 10 client.16599 fetch_fsmap learned FSMap version 34 2024-05-15T14:10:32.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.042+0000 7f98b03acb80 1 -- 172.21.15.33:0/3727033274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5578d99bcff0 con 0x5578d99abf60 2024-05-15T14:10:32.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.043+0000 7f987effd700 1 -- 172.21.15.33:0/3727033274 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9890024930 con 0x5578d99abf60 2024-05-15T14:10:32.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.043+0000 7f987effd700 1 -- 172.21.15.33:0/3727033274 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f98980a8210 con 0x5578d99abf60 2024-05-15T14:10:32.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.043+0000 7f98b03acb80 10 client.16599 fetch_fsmap finished waiting for FSMap version 34 2024-05-15T14:10:32.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.043+0000 7f98b03acb80 10 client.16599 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:10:32.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.043+0000 7f98b03acb80 20 client.16599 populate_metadata read hostname 'smithi033' 2024-05-15T14:10:32.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.043+0000 7f98b03acb80 1 --2- 172.21.15.33:0/3727033274 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5578d99bf680 0x5578d99c1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:32.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.043+0000 7f98b03acb80 4 client.16599 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:10:32.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.043+0000 7f98b03acb80 1 -- 172.21.15.33:0/3727033274 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578d9969460 con 0x5578d99bf680 2024-05-15T14:10:32.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.043+0000 7f989f7fe700 1 --2- 172.21.15.33:0/3727033274 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5578d99bf680 0x5578d99c1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:32.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.044+0000 7f989f7fe700 1 --2- 172.21.15.33:0/3727033274 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5578d99bf680 0x5578d99c1a70 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=391fd642cdd8f643 server_cookie=9af028d5ae570fd in_seq=0 out_seq=0 2024-05-15T14:10:32.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.044+0000 7f987effd700 10 client.16599 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:10:32.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.044+0000 7f987effd700 1 -- 172.21.15.33:0/3727033274 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5578d9969460 con 0x5578d99bf680 2024-05-15T14:10:32.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.044+0000 7f987effd700 10 client.16599 handle_command_reply: tid=0 2024-05-15T14:10:32.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.044+0000 7f98b03acb80 1 client.16599 shutdown 2024-05-15T14:10:32.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.044+0000 7f98b03acb80 1 -- 172.21.15.33:0/3727033274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d99abf60 msgr2=0x5578d99af4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:32.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.044+0000 7f98b03acb80 1 --2- 172.21.15.33:0/3727033274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d99abf60 0x5578d99af4c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f98980a8f40 tx=0x7f98980a94c0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.044+0000 7f98b03acb80 1 -- 172.21.15.33:0/3727033274 shutdown_connections 2024-05-15T14:10:32.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.044+0000 7f98b03acb80 1 --2- 172.21.15.33:0/3727033274 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5578d99bf680 0x5578d99c1a70 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.045+0000 7f98b03acb80 1 --2- 172.21.15.33:0/3727033274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d99abf60 0x5578d99af4c0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.045+0000 7f98b03acb80 1 --2- 172.21.15.33:0/3727033274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5578d999dad0 0x5578d99b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.045+0000 7f98b03acb80 1 --2- 172.21.15.33:0/3727033274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d99ab640 0x5578d99aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.045+0000 7f98b03acb80 1 -- 172.21.15.33:0/3727033274 >> 172.21.15.33:0/3727033274 conn(0x5578d98c8350 msgr2=0x5578d99927d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:32.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.045+0000 7f987effd700 0 client.16599 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:10:32.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.045+0000 7f98b03acb80 1 -- 172.21.15.33:0/3727033274 shutdown_connections 2024-05-15T14:10:32.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.045+0000 7f98b03acb80 1 -- 172.21.15.33:0/3727033274 wait complete. 2024-05-15T14:10:32.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.045+0000 7f98b03acb80 20 client.16599 trim_cache size 0 max 16384 2024-05-15T14:10:32.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.065+0000 7f98b03acb80 1 Processor -- start 2024-05-15T14:10:32.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.065+0000 7f98b03acb80 1 -- start start 2024-05-15T14:10:32.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.065+0000 7f98b03acb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d999dad0 0x5578d999d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:32.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.065+0000 7f98b03acb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d99ab640 0x5578d99bb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:32.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.065+0000 7f98b03acb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5578d99abf60 0x5578d99928e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:32.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.065+0000 7f98b03acb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f98a01ae5f0 con 0x5578d999dad0 2024-05-15T14:10:32.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.065+0000 7f98b03acb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f98a01abdf0 con 0x5578d99ab640 2024-05-15T14:10:32.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.065+0000 7f98b03acb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f98a01a9c60 con 0x5578d99abf60 2024-05-15T14:10:32.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.065+0000 7f989f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d999dad0 0x5578d999d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:32.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.065+0000 7f989f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d999dad0 0x5578d999d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40568/0 (socket says 172.21.15.33:40568) 2024-05-15T14:10:32.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.065+0000 7f989f7fe700 1 -- 172.21.15.33:0/3362022211 learned_addr learned my addr 172.21.15.33:0/3362022211 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:32.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.066+0000 7f989f7fe700 1 -- 172.21.15.33:0/3362022211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5578d99abf60 msgr2=0x5578d99928e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:32.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.066+0000 7f989f7fe700 1 --2- 172.21.15.33:0/3362022211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5578d99abf60 0x5578d99928e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.066+0000 7f989f7fe700 1 -- 172.21.15.33:0/3362022211 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d99ab640 msgr2=0x5578d99bb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:32.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.066+0000 7f989effd700 1 --2- 172.21.15.33:0/3362022211 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d99ab640 0x5578d99bb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:32.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.066+0000 7f989f7fe700 1 --2- 172.21.15.33:0/3362022211 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d99ab640 0x5578d99bb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.066+0000 7f989f7fe700 1 -- 172.21.15.33:0/3362022211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578d9998560 con 0x5578d999dad0 2024-05-15T14:10:32.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.066+0000 7f989effd700 1 --2- 172.21.15.33:0/3362022211 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d99ab640 0x5578d99bb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:10:32.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.066+0000 7f989f7fe700 1 --2- 172.21.15.33:0/3362022211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d999dad0 0x5578d999d4b0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f989000a010 tx=0x7f989002f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ba96d383209b3c server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:32.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.066+0000 7f98857fa700 1 -- 172.21.15.33:0/3362022211 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890034040 con 0x5578d999dad0 2024-05-15T14:10:32.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.066+0000 7f98857fa700 1 -- 172.21.15.33:0/3362022211 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9890024d10 con 0x5578d999dad0 2024-05-15T14:10:32.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.066+0000 7f98857fa700 1 -- 172.21.15.33:0/3362022211 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989002fe20 con 0x5578d999dad0 2024-05-15T14:10:32.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.067+0000 7f98b03acb80 1 -- 172.21.15.33:0/3362022211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d999dad0 msgr2=0x5578d999d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:32.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.067+0000 7f98b03acb80 1 --2- 172.21.15.33:0/3362022211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d999dad0 0x5578d999d4b0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f989000a010 tx=0x7f989002f580 comp rx=0 tx=0).stop 2024-05-15T14:10:32.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.067+0000 7f98b03acb80 1 -- 172.21.15.33:0/3362022211 shutdown_connections 2024-05-15T14:10:32.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.067+0000 7f98b03acb80 1 --2- 172.21.15.33:0/3362022211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5578d99abf60 0x5578d99928e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.067+0000 7f98b03acb80 1 --2- 172.21.15.33:0/3362022211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d999dad0 0x5578d999d4b0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.067+0000 7f98b03acb80 1 --2- 172.21.15.33:0/3362022211 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d99ab640 0x5578d99bb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.067+0000 7f98b03acb80 1 -- 172.21.15.33:0/3362022211 >> 172.21.15.33:0/3362022211 conn(0x5578d98c8350 msgr2=0x5578d99b0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:32.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.067+0000 7f98b03acb80 1 -- 172.21.15.33:0/3362022211 shutdown_connections 2024-05-15T14:10:32.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.067+0000 7f98b03acb80 1 -- 172.21.15.33:0/3362022211 wait complete. 2024-05-15T14:10:32.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.067+0000 7f98b03acb80 1 -- start start 2024-05-15T14:10:32.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.067+0000 7f98b03acb80 1 Processor -- start 2024-05-15T14:10:32.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.068+0000 7f98b03acb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5578d999dad0 0x5578d99b7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:32.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.068+0000 7f98b03acb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d99ab640 0x5578d99b4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:32.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.068+0000 7f98b03acb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d99abf60 0x5578d99b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:32.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.068+0000 7f98b03acb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f98a01aa2b0 con 0x5578d99ab640 2024-05-15T14:10:32.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.068+0000 7f98b03acb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f98a01ab720 con 0x5578d99abf60 2024-05-15T14:10:32.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.068+0000 7f98b03acb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f98a01aa980 con 0x5578d999dad0 2024-05-15T14:10:32.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.068+0000 7f98b03acb80 10 client.0 _ll_register_callbacks cb 0x5578d9882680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:10:32.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.068+0000 7f989effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d99ab640 0x5578d99b4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:32.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.068+0000 7f989effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d99ab640 0x5578d99b4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40578/0 (socket says 172.21.15.33:40578) 2024-05-15T14:10:32.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.068+0000 7f989effd700 1 -- 172.21.15.33:0/936047434 learned_addr learned my addr 172.21.15.33:0/936047434 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:32.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.068+0000 7f989ffff700 1 --2- 172.21.15.33:0/936047434 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d99abf60 0x5578d99b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:32.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.068+0000 7f989f7fe700 1 --2- 172.21.15.33:0/936047434 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5578d999dad0 0x5578d99b7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:32.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.068+0000 7f989effd700 1 -- 172.21.15.33:0/936047434 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5578d999dad0 msgr2=0x5578d99b7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:32.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.068+0000 7f989effd700 1 --2- 172.21.15.33:0/936047434 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5578d999dad0 0x5578d99b7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.068+0000 7f989effd700 1 -- 172.21.15.33:0/936047434 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d99abf60 msgr2=0x5578d99b50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:32.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.068+0000 7f989effd700 1 --2- 172.21.15.33:0/936047434 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d99abf60 0x5578d99b50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.068+0000 7f989effd700 1 -- 172.21.15.33:0/936047434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578d99b5700 con 0x5578d99ab640 2024-05-15T14:10:32.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.068+0000 7f989f7fe700 1 --2- 172.21.15.33:0/936047434 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5578d999dad0 0x5578d99b7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:10:32.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.068+0000 7f989ffff700 1 --2- 172.21.15.33:0/936047434 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d99abf60 0x5578d99b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:10:32.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.068+0000 7f989effd700 1 --2- 172.21.15.33:0/936047434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d99ab640 0x5578d99b4b80 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f98a00799d0 tx=0x7f9894013cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d049bad1f2927 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:32.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.068+0000 7f987effd700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:10:32.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.069+0000 7f987effd700 1 -- 172.21.15.33:0/936047434 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894013ea0 con 0x5578d99ab640 2024-05-15T14:10:32.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.069+0000 7f987effd700 1 -- 172.21.15.33:0/936047434 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f9894011b10 con 0x5578d99ab640 2024-05-15T14:10:32.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.069+0000 7f98b03acb80 1 -- 172.21.15.33:0/936047434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9890024d10 con 0x5578d99ab640 2024-05-15T14:10:32.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.069+0000 7f987effd700 1 -- 172.21.15.33:0/936047434 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9894011d10 con 0x5578d99ab640 2024-05-15T14:10:32.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.069+0000 7f98b03acb80 10 client.16605 fetch_fsmap learned FSMap version 34 2024-05-15T14:10:32.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.069+0000 7f98b03acb80 1 -- 172.21.15.33:0/936047434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5578d9a2ad90 con 0x5578d99ab640 2024-05-15T14:10:32.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.069+0000 7f987effd700 1 -- 172.21.15.33:0/936047434 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9890024d10 con 0x5578d99ab640 2024-05-15T14:10:32.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.069+0000 7f987effd700 1 -- 172.21.15.33:0/936047434 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f9894025420 con 0x5578d99ab640 2024-05-15T14:10:32.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.069+0000 7f98b03acb80 10 client.16605 fetch_fsmap finished waiting for FSMap version 34 2024-05-15T14:10:32.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.069+0000 7f98b03acb80 10 client.16605 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:10:32.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.069+0000 7f98b03acb80 20 client.16605 populate_metadata read hostname 'smithi033' 2024-05-15T14:10:32.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.069+0000 7f98b03acb80 1 --2- 172.21.15.33:0/936047434 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5578d9a2b5b0 0x5578d9a2d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:32.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.070+0000 7f98b03acb80 4 client.16605 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "5860f899", "release": true, "await": true}] 2024-05-15T14:10:32.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.070+0000 7f98b03acb80 1 -- 172.21.15.33:0/936047434 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "quiesce db", "set_id": "5860f899", "release": true, "await": true}) -- 0x5578d98fdce0 con 0x5578d9a2b5b0 2024-05-15T14:10:32.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.070+0000 7f989f7fe700 1 --2- 172.21.15.33:0/936047434 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5578d9a2b5b0 0x5578d9a2d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:32.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.070+0000 7f989f7fe700 1 --2- 172.21.15.33:0/936047434 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5578d9a2b5b0 0x5578d9a2d9d0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15dfddf7e14c69c7 server_cookie=25e3b9907bc93f6f in_seq=0 out_seq=0 2024-05-15T14:10:32.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.070+0000 7f987effd700 10 client.16605 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:10:32.078 INFO:tasks.workunit.client.0.smithi033.stdout: 1 576957 5.38 MB/sec execute 367 sec latency 28016.290 ms 2024-05-15T14:10:32.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.079+0000 7f987effd700 1 -- 172.21.15.33:0/936047434 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5578d98fdce0 con 0x5578d9a2b5b0 2024-05-15T14:10:32.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.079+0000 7f987effd700 10 client.16605 handle_command_reply: tid=0 2024-05-15T14:10:32.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.079+0000 7f98b03acb80 1 client.16605 shutdown 2024-05-15T14:10:32.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.079+0000 7f98b03acb80 1 -- 172.21.15.33:0/936047434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d99ab640 msgr2=0x5578d99b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:32.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.079+0000 7f98b03acb80 1 --2- 172.21.15.33:0/936047434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d99ab640 0x5578d99b4b80 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f98a00799d0 tx=0x7f9894013cc0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.079+0000 7f98b03acb80 1 -- 172.21.15.33:0/936047434 shutdown_connections 2024-05-15T14:10:32.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.079+0000 7f98b03acb80 1 --2- 172.21.15.33:0/936047434 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d99abf60 0x5578d99b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.079+0000 7f98b03acb80 1 --2- 172.21.15.33:0/936047434 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5578d999dad0 0x5578d99b7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.079+0000 7f98b03acb80 1 --2- 172.21.15.33:0/936047434 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5578d9a2b5b0 0x5578d9a2d9d0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.079+0000 7f98b03acb80 1 --2- 172.21.15.33:0/936047434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578d99ab640 0x5578d99b4b80 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.079+0000 7f987effd700 0 client.16605 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:10:32.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.079+0000 7f98b03acb80 1 -- 172.21.15.33:0/936047434 >> 172.21.15.33:0/936047434 conn(0x5578d98c8350 msgr2=0x5578d99ae800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:32.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.080+0000 7f98b03acb80 1 -- 172.21.15.33:0/936047434 shutdown_connections 2024-05-15T14:10:32.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.080+0000 7f98b03acb80 1 -- 172.21.15.33:0/936047434 wait complete. 2024-05-15T14:10:32.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.080+0000 7f98b03acb80 20 client.16605 trim_cache size 0 max 16384 2024-05-15T14:10:32.083 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T14:10:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 34, 2024-05-15T14:10:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24478, 2024-05-15T14:10:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 36, 2024-05-15T14:10:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-05-15T14:10:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: "5860f899": { 2024-05-15T14:10:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 36, 2024-05-15T14:10:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-05-15T14:10:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:10:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-05-15T14:10:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:10:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:10:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 90.0, 2024-05-15T14:10:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 148.5, 2024-05-15T14:10:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-05-15T14:10:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-05-15T14:10:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-05-15T14:10:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:10:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-05-15T14:10:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:10:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:10:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:10:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:10:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:10:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:10:32.084 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T14:10:32.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.080+0000 7f98a5b16700 1 -- 172.21.15.33:0/3968444403 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f98880b29f0 msgr2=0x7f98880b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:32.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.080+0000 7f98a5b16700 1 --2- 172.21.15.33:0/3968444403 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f98880b29f0 0x7f98880b4eb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f989000b5c0 tx=0x7f9890005fd0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.080+0000 7f98a5b16700 1 -- 172.21.15.33:0/3968444403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a01084b0 msgr2=0x7f98a01a5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:32.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.080+0000 7f98a5b16700 1 --2- 172.21.15.33:0/3968444403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a01084b0 0x7f98a01a5bd0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f9898007ee0 tx=0x7f989800d3b0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.081+0000 7f98a5b16700 1 -- 172.21.15.33:0/3968444403 shutdown_connections 2024-05-15T14:10:32.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.081+0000 7f98a5b16700 1 --2- 172.21.15.33:0/3968444403 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f98880b29f0 0x7f98880b4eb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.081+0000 7f98a5b16700 1 --2- 172.21.15.33:0/3968444403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a01084b0 0x7f98a01a5bd0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.081+0000 7f98a5b16700 1 --2- 172.21.15.33:0/3968444403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f98a0102470 0x7f98a019ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.081+0000 7f98a5b16700 1 --2- 172.21.15.33:0/3968444403 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98a0102e30 0x7f98a019b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:32.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.081+0000 7f98a5b16700 1 -- 172.21.15.33:0/3968444403 >> 172.21.15.33:0/3968444403 conn(0x7f98a00793f0 msgr2=0x7f98a0100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:32.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.081+0000 7f98a5b16700 1 -- 172.21.15.33:0/3968444403 shutdown_connections 2024-05-15T14:10:32.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:10:32.081+0000 7f98a5b16700 1 -- 172.21.15.33:0/3968444403 wait complete. 2024-05-15T14:10:32.689 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:32 smithi033 ceph-mon[31650]: pgmap v318: 97 pgs: 97 active+clean; 176 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 1.3 MiB/s wr, 60 op/s 2024-05-15T14:10:32.757 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '5860f899', total seconds elapsed: 34.1 2024-05-15T14:10:32.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:32 smithi052 ceph-mon[36190]: pgmap v318: 97 pgs: 97 active+clean; 176 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 1.3 MiB/s wr, 60 op/s 2024-05-15T14:10:32.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:32 smithi081 ceph-mon[36434]: pgmap v318: 97 pgs: 97 active+clean; 176 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 1.3 MiB/s wr, 60 op/s 2024-05-15T14:10:33.078 INFO:tasks.workunit.client.0.smithi033.stdout: 1 579096 5.40 MB/sec execute 368 sec latency 28016.948 ms 2024-05-15T14:10:34.078 INFO:tasks.workunit.client.0.smithi033.stdout: 1 580445 5.40 MB/sec execute 369 sec latency 22.389 ms 2024-05-15T14:10:34.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:34 smithi052 ceph-mon[36190]: pgmap v319: 97 pgs: 97 active+clean; 170 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 1.4 MiB/s wr, 87 op/s 2024-05-15T14:10:34.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:34 smithi081 ceph-mon[36434]: pgmap v319: 97 pgs: 97 active+clean; 170 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 1.4 MiB/s wr, 87 op/s 2024-05-15T14:10:34.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:34 smithi033 ceph-mon[31650]: pgmap v319: 97 pgs: 97 active+clean; 170 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 1.4 MiB/s wr, 87 op/s 2024-05-15T14:10:35.078 INFO:tasks.workunit.client.0.smithi033.stdout: 1 582629 5.42 MB/sec execute 370 sec latency 14.926 ms 2024-05-15T14:10:36.078 INFO:tasks.workunit.client.0.smithi033.stdout: 1 583936 5.43 MB/sec execute 371 sec latency 34.990 ms 2024-05-15T14:10:36.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:36 smithi052 ceph-mon[36190]: pgmap v320: 97 pgs: 97 active+clean; 170 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 638 KiB/s wr, 51 op/s 2024-05-15T14:10:36.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:36 smithi081 ceph-mon[36434]: pgmap v320: 97 pgs: 97 active+clean; 170 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 638 KiB/s wr, 51 op/s 2024-05-15T14:10:36.940 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:36 smithi033 ceph-mon[31650]: pgmap v320: 97 pgs: 97 active+clean; 170 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 638 KiB/s wr, 51 op/s 2024-05-15T14:10:37.078 INFO:tasks.workunit.client.0.smithi033.stdout: 1 585831 5.44 MB/sec execute 372 sec latency 18.765 ms 2024-05-15T14:10:38.078 INFO:tasks.workunit.client.0.smithi033.stdout: 1 587250 5.44 MB/sec execute 373 sec latency 15.127 ms 2024-05-15T14:10:38.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:38 smithi052 ceph-mon[36190]: pgmap v321: 97 pgs: 97 active+clean; 177 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.7 MiB/s wr, 188 op/s 2024-05-15T14:10:38.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:38 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:10:38.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:38 smithi081 ceph-mon[36434]: pgmap v321: 97 pgs: 97 active+clean; 177 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.7 MiB/s wr, 188 op/s 2024-05-15T14:10:38.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:38 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:10:38.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:38 smithi033 ceph-mon[31650]: pgmap v321: 97 pgs: 97 active+clean; 177 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.7 MiB/s wr, 188 op/s 2024-05-15T14:10:38.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:38 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:10:39.078 INFO:tasks.workunit.client.0.smithi033.stdout: 1 589060 5.45 MB/sec execute 374 sec latency 28.149 ms 2024-05-15T14:10:39.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:39 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:10:39.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:39 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:10:39.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:39 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:10:39.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:39 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:10:39.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:39 smithi033 ceph-mon[31650]: pgmap v322: 97 pgs: 97 active+clean; 177 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.1 MiB/s wr, 165 op/s 2024-05-15T14:10:39.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:39 smithi033 ceph-mon[31650]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:10:40.042 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:39 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:10:40.042 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:39 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:10:40.042 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:39 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:10:40.042 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:39 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:10:40.042 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:39 smithi052 ceph-mon[36190]: pgmap v322: 97 pgs: 97 active+clean; 177 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.1 MiB/s wr, 165 op/s 2024-05-15T14:10:40.043 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:39 smithi052 ceph-mon[36190]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:10:40.078 INFO:tasks.workunit.client.0.smithi033.stdout: 1 590401 5.46 MB/sec execute 375 sec latency 34.017 ms 2024-05-15T14:10:40.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:39 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:10:40.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:39 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:10:40.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:39 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:10:40.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:39 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:10:40.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:39 smithi081 ceph-mon[36434]: pgmap v322: 97 pgs: 97 active+clean; 177 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.1 MiB/s wr, 165 op/s 2024-05-15T14:10:40.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:39 smithi081 ceph-mon[36434]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:10:41.078 INFO:tasks.workunit.client.0.smithi033.stdout: 1 592113 5.46 MB/sec execute 376 sec latency 28.355 ms 2024-05-15T14:10:41.340 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:41 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:10:41.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:41 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:10:41.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:41 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:10:41.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:41 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:10:41.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:41 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:41.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:41 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:41.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:41 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:41.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:41 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:41.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:41 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:10:41.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:41 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:10:41.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:41 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:10:41.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:41 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:10:41.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:41 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:10:41.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:41 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:10:41.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:41 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:41.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:41 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:41.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:41 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:41.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:41 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:41.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:41 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:10:41.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:41 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:10:41.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:41 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:10:41.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:41 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:10:41.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:41 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:10:41.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:41 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:10:41.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:41 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:41.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:41 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:41.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:41 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:41.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:41 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:41.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:41 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:10:41.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:41 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:10:42.078 INFO:tasks.workunit.client.0.smithi033.stdout: 1 593604 5.48 MB/sec execute 377 sec latency 19.907 ms 2024-05-15T14:10:42.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:42 smithi052 ceph-mon[36190]: Adjusting osd_memory_target on smithi081 to 176.2M 2024-05-15T14:10:42.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:42 smithi052 ceph-mon[36190]: Unable to set osd_memory_target on smithi081 to 184783155: error parsing value: Value '184783155' is below minimum 939524096 2024-05-15T14:10:42.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:42 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:42.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:42 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:42.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:42 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:42.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:42 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:42.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:42 smithi052 ceph-mon[36190]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-05-15T14:10:42.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:42 smithi052 ceph-mon[36190]: Unable to set osd_memory_target on smithi033 to 185879142: error parsing value: Value '185879142' is below minimum 939524096 2024-05-15T14:10:42.341 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:42 smithi052 ceph-mon[36190]: pgmap v323: 97 pgs: 97 active+clean; 177 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.1 MiB/s wr, 165 op/s 2024-05-15T14:10:42.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:42 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:42.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:42 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:42.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:42 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:42.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:42 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:42.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:42 smithi052 ceph-mon[36190]: Adjusting osd_memory_target on smithi052 to 177.2M 2024-05-15T14:10:42.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:42 smithi052 ceph-mon[36190]: Unable to set osd_memory_target on smithi052 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-05-15T14:10:42.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:42 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:10:42.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:42 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:10:42.342 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:42 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:10:42.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:42 smithi081 ceph-mon[36434]: Adjusting osd_memory_target on smithi081 to 176.2M 2024-05-15T14:10:42.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:42 smithi081 ceph-mon[36434]: Unable to set osd_memory_target on smithi081 to 184783155: error parsing value: Value '184783155' is below minimum 939524096 2024-05-15T14:10:42.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:42 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:42.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:42 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:42.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:42 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:42.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:42 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:42.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:42 smithi081 ceph-mon[36434]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-05-15T14:10:42.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:42 smithi081 ceph-mon[36434]: Unable to set osd_memory_target on smithi033 to 185879142: error parsing value: Value '185879142' is below minimum 939524096 2024-05-15T14:10:42.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:42 smithi081 ceph-mon[36434]: pgmap v323: 97 pgs: 97 active+clean; 177 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.1 MiB/s wr, 165 op/s 2024-05-15T14:10:42.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:42 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:42.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:42 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:42.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:42 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:42.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:42 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:42.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:42 smithi081 ceph-mon[36434]: Adjusting osd_memory_target on smithi052 to 177.2M 2024-05-15T14:10:42.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:42 smithi081 ceph-mon[36434]: Unable to set osd_memory_target on smithi052 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-05-15T14:10:42.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:42 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:10:42.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:42 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:10:42.358 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:42 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:10:42.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:42 smithi033 ceph-mon[31650]: Adjusting osd_memory_target on smithi081 to 176.2M 2024-05-15T14:10:42.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:42 smithi033 ceph-mon[31650]: Unable to set osd_memory_target on smithi081 to 184783155: error parsing value: Value '184783155' is below minimum 939524096 2024-05-15T14:10:42.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:42 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:42.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:42 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:42.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:42 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:42.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:42 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:42.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:42 smithi033 ceph-mon[31650]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-05-15T14:10:42.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:42 smithi033 ceph-mon[31650]: Unable to set osd_memory_target on smithi033 to 185879142: error parsing value: Value '185879142' is below minimum 939524096 2024-05-15T14:10:42.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:42 smithi033 ceph-mon[31650]: pgmap v323: 97 pgs: 97 active+clean; 177 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.1 MiB/s wr, 165 op/s 2024-05-15T14:10:42.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:42 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:42.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:42 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:42.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:42 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:42.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:42 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-15T14:10:42.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:42 smithi033 ceph-mon[31650]: Adjusting osd_memory_target on smithi052 to 177.2M 2024-05-15T14:10:42.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:42 smithi033 ceph-mon[31650]: Unable to set osd_memory_target on smithi052 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-05-15T14:10:42.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:42 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:10:42.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:42 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:10:42.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:42 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:10:43.078 INFO:tasks.workunit.client.0.smithi033.stdout: 1 595121 5.48 MB/sec execute 378 sec latency 32.557 ms 2024-05-15T14:10:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.570+0000 7f29adc54640 1 -- 172.21.15.33:0/1279907173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29a81045e0 msgr2=0x7f29a810cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.570+0000 7f29adc54640 1 --2- 172.21.15.33:0/1279907173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29a81045e0 0x7f29a810cac0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f2990009900 tx=0x7f299002a480 comp rx=0 tx=0).stop 2024-05-15T14:10:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.570+0000 7f29adc54640 1 -- 172.21.15.33:0/1279907173 shutdown_connections 2024-05-15T14:10:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.570+0000 7f29adc54640 1 --2- 172.21.15.33:0/1279907173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f29a810d090 0x7f29a810f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.570+0000 7f29adc54640 1 --2- 172.21.15.33:0/1279907173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29a81045e0 0x7f29a810cac0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.570+0000 7f29adc54640 1 --2- 172.21.15.33:0/1279907173 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29a8103cc0 0x7f29a81040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.570+0000 7f29adc54640 1 -- 172.21.15.33:0/1279907173 >> 172.21.15.33:0/1279907173 conn(0x7f29a8078a70 msgr2=0x7f29a8078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.570+0000 7f29adc54640 1 -- 172.21.15.33:0/1279907173 shutdown_connections 2024-05-15T14:10:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.570+0000 7f29adc54640 1 -- 172.21.15.33:0/1279907173 wait complete. 2024-05-15T14:10:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.571+0000 7f29adc54640 1 Processor -- start 2024-05-15T14:10:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.571+0000 7f29adc54640 1 -- start start 2024-05-15T14:10:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.571+0000 7f29adc54640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f29a8103cc0 0x7f29a819b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.571+0000 7f29adc54640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29a81045e0 0x7f29a819b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.571+0000 7f29adc54640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29a810d090 0x7f29a819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.571+0000 7f29adc54640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f29a8112cd0 con 0x7f29a810d090 2024-05-15T14:10:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.571+0000 7f29adc54640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f29a8112b50 con 0x7f29a81045e0 2024-05-15T14:10:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.571+0000 7f29adc54640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f29a8112e50 con 0x7f29a8103cc0 2024-05-15T14:10:43.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.572+0000 7f29a7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29a810d090 0x7f29a819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:43.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.572+0000 7f29a77fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f29a8103cc0 0x7f29a819b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:43.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.572+0000 7f29a6ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29a81045e0 0x7f29a819b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:43.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.572+0000 7f29a6ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29a81045e0 0x7f29a819b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:39174/0 (socket says 172.21.15.33:39174) 2024-05-15T14:10:43.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.572+0000 7f29a7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29a810d090 0x7f29a819fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39120/0 (socket says 172.21.15.33:39120) 2024-05-15T14:10:43.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.572+0000 7f29a6ffd640 1 -- 172.21.15.33:0/121059078 learned_addr learned my addr 172.21.15.33:0/121059078 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:43.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.572+0000 7f29a7fff640 1 -- 172.21.15.33:0/121059078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f29a8103cc0 msgr2=0x7f29a819b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:43.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.572+0000 7f29a7fff640 1 --2- 172.21.15.33:0/121059078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f29a8103cc0 0x7f29a819b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:43.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.572+0000 7f29a7fff640 1 -- 172.21.15.33:0/121059078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29a81045e0 msgr2=0x7f29a819b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:43.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.572+0000 7f29a7fff640 1 --2- 172.21.15.33:0/121059078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29a81045e0 0x7f29a819b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:43.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.572+0000 7f29a7fff640 1 -- 172.21.15.33:0/121059078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a81a0400 con 0x7f29a810d090 2024-05-15T14:10:43.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.572+0000 7f29a77fe640 1 --2- 172.21.15.33:0/121059078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f29a8103cc0 0x7f29a819b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:10:43.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.572+0000 7f29a7fff640 1 --2- 172.21.15.33:0/121059078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29a810d090 0x7f29a819fcd0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f29a0005fd0 tx=0x7f29a000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:43.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.572+0000 7f29a4ff9640 1 -- 172.21.15.33:0/121059078 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a000cd30 con 0x7f29a810d090 2024-05-15T14:10:43.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.572+0000 7f29adc54640 1 -- 172.21.15.33:0/121059078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29a81a06f0 con 0x7f29a810d090 2024-05-15T14:10:43.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.572+0000 7f29a4ff9640 1 -- 172.21.15.33:0/121059078 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29a0005020 con 0x7f29a810d090 2024-05-15T14:10:43.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.572+0000 7f29a4ff9640 1 -- 172.21.15.33:0/121059078 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a0010690 con 0x7f29a810d090 2024-05-15T14:10:43.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.572+0000 7f29adc54640 1 -- 172.21.15.33:0/121059078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29a81a7f80 con 0x7f29a810d090 2024-05-15T14:10:43.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.575+0000 7f29a4ff9640 1 -- 172.21.15.33:0/121059078 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f29a00054d0 con 0x7f29a810d090 2024-05-15T14:10:43.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.575+0000 7f29b096a740 1 Processor -- start 2024-05-15T14:10:43.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.575+0000 7f29b096a740 1 -- start start 2024-05-15T14:10:43.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.575+0000 7f29b096a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566fd8558f0 0x5566fd855cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:43.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.575+0000 7f29b096a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566fd856210 0x5566fd860590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:43.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.575+0000 7f29b096a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566fd860ad0 0x5566fd862ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:43.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.575+0000 7f29b096a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5566fd7c8c30 con 0x5566fd8558f0 2024-05-15T14:10:43.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.575+0000 7f29b096a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5566fd7b87d0 con 0x5566fd860ad0 2024-05-15T14:10:43.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.575+0000 7f29b096a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5566fd7bc810 con 0x5566fd856210 2024-05-15T14:10:43.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.575+0000 7f29a6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566fd8558f0 0x5566fd855cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.575+0000 7f29a77fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566fd856210 0x5566fd860590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.576+0000 7f29a77fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566fd856210 0x5566fd860590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:50944/0 (socket says 172.21.15.33:50944) 2024-05-15T14:10:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.576+0000 7f29a6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566fd8558f0 0x5566fd855cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39128/0 (socket says 172.21.15.33:39128) 2024-05-15T14:10:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.576+0000 7f29a77fe640 1 -- 172.21.15.33:0/2347864505 learned_addr learned my addr 172.21.15.33:0/2347864505 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.576+0000 7f29a6ffd640 1 --2- 172.21.15.33:0/2347864505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566fd860ad0 0x5566fd862ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.576+0000 7f29a77fe640 1 -- 172.21.15.33:0/2347864505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566fd860ad0 msgr2=0x5566fd862ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.576+0000 7f29a77fe640 1 --2- 172.21.15.33:0/2347864505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566fd860ad0 0x5566fd862ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.576+0000 7f29a77fe640 1 -- 172.21.15.33:0/2347864505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566fd8558f0 msgr2=0x5566fd855cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.576+0000 7f29a77fe640 1 --2- 172.21.15.33:0/2347864505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566fd8558f0 0x5566fd855cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.576+0000 7f29a77fe640 1 -- 172.21.15.33:0/2347864505 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566fd4cd210 con 0x5566fd856210 2024-05-15T14:10:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.576+0000 7f29a6ffd640 1 --2- 172.21.15.33:0/2347864505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566fd860ad0 0x5566fd862ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:10:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.576+0000 7f29a4ff9640 1 --2- 172.21.15.33:0/121059078 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f29940b2950 0x7f29940b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.576+0000 7f29a77fe640 1 --2- 172.21.15.33:0/2347864505 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566fd856210 0x5566fd860590 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f299800f4c0 tx=0x7f299800f980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=397518378741fd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.576+0000 7f29a6ffd640 1 --2- 172.21.15.33:0/121059078 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f29940b2950 0x7f29940b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.576+0000 7f29a4ff9640 1 -- 172.21.15.33:0/121059078 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f29a00e2c80 con 0x7f29a810d090 2024-05-15T14:10:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.576+0000 7f299dffb640 1 -- 172.21.15.33:0/2347864505 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2998005220 con 0x5566fd856210 2024-05-15T14:10:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.576+0000 7f299dffb640 1 -- 172.21.15.33:0/2347864505 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29980047a0 con 0x5566fd856210 2024-05-15T14:10:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.576+0000 7f299dffb640 1 -- 172.21.15.33:0/2347864505 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2998010ab0 con 0x5566fd856210 2024-05-15T14:10:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.577+0000 7f29b096a740 1 -- 172.21.15.33:0/2347864505 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566fd856210 msgr2=0x5566fd860590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.577+0000 7f29a6ffd640 1 --2- 172.21.15.33:0/121059078 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f29940b2950 0x7f29940b4e10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2990031440 tx=0x7f2990005e50 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.577+0000 7f29b096a740 1 --2- 172.21.15.33:0/2347864505 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566fd856210 0x5566fd860590 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f299800f4c0 tx=0x7f299800f980 comp rx=0 tx=0).stop 2024-05-15T14:10:43.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.577+0000 7f29b096a740 1 -- 172.21.15.33:0/2347864505 shutdown_connections 2024-05-15T14:10:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.577+0000 7f29b096a740 1 --2- 172.21.15.33:0/2347864505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566fd860ad0 0x5566fd862ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.577+0000 7f29b096a740 1 --2- 172.21.15.33:0/2347864505 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566fd856210 0x5566fd860590 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.577+0000 7f29b096a740 1 --2- 172.21.15.33:0/2347864505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566fd8558f0 0x5566fd855cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.577+0000 7f29b096a740 1 -- 172.21.15.33:0/2347864505 >> 172.21.15.33:0/2347864505 conn(0x5566fd851e00 msgr2=0x5566fd8521e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.577+0000 7f29b096a740 1 -- 172.21.15.33:0/2347864505 shutdown_connections 2024-05-15T14:10:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.577+0000 7f29b096a740 1 -- 172.21.15.33:0/2347864505 wait complete. 2024-05-15T14:10:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.577+0000 7f29b096a740 1 -- start start 2024-05-15T14:10:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.578+0000 7f29b096a740 1 Processor -- start 2024-05-15T14:10:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.578+0000 7f29b096a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566fd8558f0 0x5566fd860390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.578+0000 7f29b096a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566fd856210 0x5566fd85d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:43.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.578+0000 7f29b096a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566fd860ad0 0x5566fd85dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.578+0000 7f29b096a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5566fd49e790 con 0x5566fd856210 2024-05-15T14:10:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.578+0000 7f29b096a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5566fd774890 con 0x5566fd860ad0 2024-05-15T14:10:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.578+0000 7f29b096a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5566fd8637a0 con 0x5566fd8558f0 2024-05-15T14:10:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.578+0000 7f29b096a740 10 client.0 _ll_register_callbacks cb 0x5566fd777d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:10:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.578+0000 7f29a6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566fd856210 0x5566fd85d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.578+0000 7f29a77fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566fd8558f0 0x5566fd860390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.578+0000 7f29a6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566fd856210 0x5566fd85d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39132/0 (socket says 172.21.15.33:39132) 2024-05-15T14:10:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.578+0000 7f29a6ffd640 1 -- 172.21.15.33:0/3181647238 learned_addr learned my addr 172.21.15.33:0/3181647238 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.578+0000 7f29a77fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566fd8558f0 0x5566fd860390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:50946/0 (socket says 172.21.15.33:50946) 2024-05-15T14:10:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.578+0000 7f29a7fff640 1 --2- 172.21.15.33:0/3181647238 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566fd860ad0 0x5566fd85dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.578+0000 7f29a6ffd640 1 -- 172.21.15.33:0/3181647238 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566fd8558f0 msgr2=0x5566fd860390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.578+0000 7f29a6ffd640 1 --2- 172.21.15.33:0/3181647238 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566fd8558f0 0x5566fd860390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.578+0000 7f29a6ffd640 1 -- 172.21.15.33:0/3181647238 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566fd860ad0 msgr2=0x5566fd85dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.578+0000 7f29a6ffd640 1 --2- 172.21.15.33:0/3181647238 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566fd860ad0 0x5566fd85dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.578+0000 7f29a6ffd640 1 -- 172.21.15.33:0/3181647238 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566fd599620 con 0x5566fd856210 2024-05-15T14:10:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.578+0000 7f29a7fff640 1 --2- 172.21.15.33:0/3181647238 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566fd860ad0 0x5566fd85dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:10:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.578+0000 7f29a77fe640 1 --2- 172.21.15.33:0/3181647238 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566fd8558f0 0x5566fd860390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:10:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.579+0000 7f29a6ffd640 1 --2- 172.21.15.33:0/3181647238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566fd856210 0x5566fd85d680 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f2990000c00 tx=0x7f29900058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d14f0809f3a0ff server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.579+0000 7f2986ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:10:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.579+0000 7f2986ffd640 1 -- 172.21.15.33:0/3181647238 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299004d030 con 0x5566fd856210 2024-05-15T14:10:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.579+0000 7f2986ffd640 1 -- 172.21.15.33:0/3181647238 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f2990003800 con 0x5566fd856210 2024-05-15T14:10:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.579+0000 7f29b096a740 1 -- 172.21.15.33:0/3181647238 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29980047a0 con 0x5566fd856210 2024-05-15T14:10:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.579+0000 7f2986ffd640 1 -- 172.21.15.33:0/3181647238 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2990035040 con 0x5566fd856210 2024-05-15T14:10:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.579+0000 7f29b096a740 10 client.16617 fetch_fsmap learned FSMap version 35 2024-05-15T14:10:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.579+0000 7f29b096a740 1 -- 172.21.15.33:0/3181647238 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5566fd86f120 con 0x5566fd856210 2024-05-15T14:10:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.579+0000 7f2986ffd640 1 -- 172.21.15.33:0/3181647238 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29980047a0 con 0x5566fd856210 2024-05-15T14:10:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.579+0000 7f2986ffd640 1 -- 172.21.15.33:0/3181647238 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f299003d9b0 con 0x5566fd856210 2024-05-15T14:10:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.580+0000 7f29b096a740 10 client.16617 fetch_fsmap finished waiting for FSMap version 35 2024-05-15T14:10:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.580+0000 7f29b096a740 10 client.16617 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:10:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.580+0000 7f29b096a740 20 client.16617 populate_metadata read hostname 'smithi033' 2024-05-15T14:10:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.580+0000 7f29b096a740 1 --2- 172.21.15.33:0/3181647238 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5566fd870630 0x5566fd872a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.580+0000 7f29b096a740 4 client.16617 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:10:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.580+0000 7f29b096a740 1 -- 172.21.15.33:0/3181647238 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566fd7cab80 con 0x5566fd870630 2024-05-15T14:10:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.580+0000 7f29a77fe640 1 --2- 172.21.15.33:0/3181647238 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5566fd870630 0x5566fd872a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.580+0000 7f29a77fe640 1 --2- 172.21.15.33:0/3181647238 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5566fd870630 0x5566fd872a80 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3836229bf38f588 server_cookie=3259b83ef7383628 in_seq=0 out_seq=0 2024-05-15T14:10:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.580+0000 7f2986ffd640 10 client.16617 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:10:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.581+0000 7f2986ffd640 1 -- 172.21.15.33:0/3181647238 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5566fd7cab80 con 0x5566fd870630 2024-05-15T14:10:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.581+0000 7f2986ffd640 10 client.16617 handle_command_reply: tid=0 2024-05-15T14:10:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.581+0000 7f29b096a740 1 client.16617 shutdown 2024-05-15T14:10:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.581+0000 7f29b096a740 1 -- 172.21.15.33:0/3181647238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566fd856210 msgr2=0x5566fd85d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.581+0000 7f29b096a740 1 --2- 172.21.15.33:0/3181647238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566fd856210 0x5566fd85d680 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f2990000c00 tx=0x7f29900058d0 comp rx=0 tx=0).stop 2024-05-15T14:10:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.581+0000 7f29b096a740 1 -- 172.21.15.33:0/3181647238 shutdown_connections 2024-05-15T14:10:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.581+0000 7f29b096a740 1 --2- 172.21.15.33:0/3181647238 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5566fd870630 0x5566fd872a80 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.581+0000 7f29b096a740 1 --2- 172.21.15.33:0/3181647238 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566fd860ad0 0x5566fd85dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.581+0000 7f2986ffd640 0 client.16617 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:10:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.581+0000 7f29b096a740 1 --2- 172.21.15.33:0/3181647238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566fd856210 0x5566fd85d680 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.581+0000 7f29b096a740 1 --2- 172.21.15.33:0/3181647238 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566fd8558f0 0x5566fd860390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.581+0000 7f29b096a740 1 -- 172.21.15.33:0/3181647238 >> 172.21.15.33:0/3181647238 conn(0x5566fd851e00 msgr2=0x5566fd851100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.581+0000 7f29b096a740 1 -- 172.21.15.33:0/3181647238 shutdown_connections 2024-05-15T14:10:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.581+0000 7f29b096a740 1 -- 172.21.15.33:0/3181647238 wait complete. 2024-05-15T14:10:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.581+0000 7f29b096a740 20 client.16617 trim_cache size 0 max 16384 2024-05-15T14:10:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.597+0000 7f29b096a740 1 Processor -- start 2024-05-15T14:10:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.597+0000 7f29b096a740 1 -- start start 2024-05-15T14:10:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.597+0000 7f29b096a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566fd8558f0 0x5566fd858dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.598+0000 7f29b096a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566fd856210 0x5566fd84cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.598+0000 7f29b096a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566fd860ad0 0x5566fd8579b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.598+0000 7f29b096a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f29a81b3250 con 0x5566fd860ad0 2024-05-15T14:10:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.598+0000 7f29b096a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f29a81b0ac0 con 0x5566fd8558f0 2024-05-15T14:10:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.598+0000 7f29b096a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f29a81ae930 con 0x5566fd856210 2024-05-15T14:10:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.598+0000 7f29a7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566fd860ad0 0x5566fd8579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.598+0000 7f29a77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566fd8558f0 0x5566fd858dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.598+0000 7f29a6ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566fd856210 0x5566fd84cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.598+0000 7f29a77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566fd8558f0 0x5566fd858dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:39216/0 (socket says 172.21.15.33:39216) 2024-05-15T14:10:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.598+0000 7f29a7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566fd860ad0 0x5566fd8579b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39144/0 (socket says 172.21.15.33:39144) 2024-05-15T14:10:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.598+0000 7f29a6ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566fd856210 0x5566fd84cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:50958/0 (socket says 172.21.15.33:50958) 2024-05-15T14:10:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.598+0000 7f29a77fe640 1 -- 172.21.15.33:0/2342239618 learned_addr learned my addr 172.21.15.33:0/2342239618 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.598+0000 7f29a77fe640 1 -- 172.21.15.33:0/2342239618 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566fd856210 msgr2=0x5566fd84cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.598+0000 7f29a77fe640 1 --2- 172.21.15.33:0/2342239618 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566fd856210 0x5566fd84cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.598+0000 7f29a77fe640 1 -- 172.21.15.33:0/2342239618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566fd860ad0 msgr2=0x5566fd8579b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.598+0000 7f29a77fe640 1 --2- 172.21.15.33:0/2342239618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566fd860ad0 0x5566fd8579b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.598+0000 7f29a77fe640 1 -- 172.21.15.33:0/2342239618 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566fd85c630 con 0x5566fd8558f0 2024-05-15T14:10:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.598+0000 7f29a6ffd640 1 --2- 172.21.15.33:0/2342239618 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566fd856210 0x5566fd84cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:10:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.598+0000 7f29a7fff640 1 --2- 172.21.15.33:0/2342239618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566fd860ad0 0x5566fd8579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:10:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.598+0000 7f29a77fe640 1 --2- 172.21.15.33:0/2342239618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566fd8558f0 0x5566fd858dc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f299800a5c0 tx=0x7f299800a2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e2683230c3d83c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.598+0000 7f299d7fa640 1 -- 172.21.15.33:0/2342239618 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29980109c0 con 0x5566fd8558f0 2024-05-15T14:10:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.599+0000 7f299d7fa640 1 -- 172.21.15.33:0/2342239618 <== mon.2 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f299802a7f0 con 0x5566fd8558f0 2024-05-15T14:10:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.599+0000 7f299d7fa640 1 -- 172.21.15.33:0/2342239618 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2998004e80 con 0x5566fd8558f0 2024-05-15T14:10:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.599+0000 7f29b096a740 1 -- 172.21.15.33:0/2342239618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566fd8558f0 msgr2=0x5566fd858dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.599+0000 7f29b096a740 1 --2- 172.21.15.33:0/2342239618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566fd8558f0 0x5566fd858dc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f299800a5c0 tx=0x7f299800a2d0 comp rx=0 tx=0).stop 2024-05-15T14:10:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.599+0000 7f29b096a740 1 -- 172.21.15.33:0/2342239618 shutdown_connections 2024-05-15T14:10:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.599+0000 7f29b096a740 1 --2- 172.21.15.33:0/2342239618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566fd860ad0 0x5566fd8579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.599+0000 7f29b096a740 1 --2- 172.21.15.33:0/2342239618 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566fd856210 0x5566fd84cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.599+0000 7f29b096a740 1 --2- 172.21.15.33:0/2342239618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566fd8558f0 0x5566fd858dc0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.599+0000 7f29b096a740 1 -- 172.21.15.33:0/2342239618 >> 172.21.15.33:0/2342239618 conn(0x5566fd851e00 msgr2=0x5566fd8572a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.599+0000 7f29b096a740 1 -- 172.21.15.33:0/2342239618 shutdown_connections 2024-05-15T14:10:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.599+0000 7f29b096a740 1 -- 172.21.15.33:0/2342239618 wait complete. 2024-05-15T14:10:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.600+0000 7f29b096a740 1 -- start start 2024-05-15T14:10:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.600+0000 7f29b096a740 1 Processor -- start 2024-05-15T14:10:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.600+0000 7f29b096a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566fd8558f0 0x5566fd868d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.600+0000 7f29b096a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566fd856210 0x5566fd8640c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:43.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.600+0000 7f29b096a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566fd860ad0 0x5566fd864600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.600+0000 7f29b096a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f29a81aef80 con 0x5566fd860ad0 2024-05-15T14:10:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.600+0000 7f29b096a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f29a81b03f0 con 0x5566fd8558f0 2024-05-15T14:10:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.600+0000 7f29b096a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f29a81af650 con 0x5566fd856210 2024-05-15T14:10:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.600+0000 7f29b096a740 10 client.0 _ll_register_callbacks cb 0x7f2990031780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:10:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.600+0000 7f29a7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566fd860ad0 0x5566fd864600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.600+0000 7f29a77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566fd8558f0 0x5566fd868d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.600+0000 7f29a6ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566fd856210 0x5566fd8640c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.600+0000 7f29a77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566fd8558f0 0x5566fd868d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:39226/0 (socket says 172.21.15.33:39226) 2024-05-15T14:10:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.600+0000 7f29a77fe640 1 -- 172.21.15.33:0/441319384 learned_addr learned my addr 172.21.15.33:0/441319384 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.600+0000 7f29a6ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566fd856210 0x5566fd8640c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:50970/0 (socket says 172.21.15.33:50970) 2024-05-15T14:10:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.600+0000 7f29a7fff640 1 -- 172.21.15.33:0/441319384 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566fd856210 msgr2=0x5566fd8640c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.600+0000 7f29a7fff640 1 --2- 172.21.15.33:0/441319384 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566fd856210 0x5566fd8640c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.600+0000 7f29a7fff640 1 -- 172.21.15.33:0/441319384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566fd8558f0 msgr2=0x5566fd868d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.600+0000 7f29a7fff640 1 --2- 172.21.15.33:0/441319384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566fd8558f0 0x5566fd868d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.600+0000 7f29a7fff640 1 -- 172.21.15.33:0/441319384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566fd856040 con 0x5566fd860ad0 2024-05-15T14:10:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.601+0000 7f29a7fff640 1 --2- 172.21.15.33:0/441319384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566fd860ad0 0x5566fd864600 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f29a00e03c0 tx=0x7f29a00aa550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c3b8b080191b26 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.601+0000 7f2986ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:10:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.601+0000 7f2986ffd640 1 -- 172.21.15.33:0/441319384 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a006d070 con 0x5566fd860ad0 2024-05-15T14:10:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.601+0000 7f2986ffd640 1 -- 172.21.15.33:0/441319384 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f29a00b2b30 con 0x5566fd860ad0 2024-05-15T14:10:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.601+0000 7f29b096a740 1 -- 172.21.15.33:0/441319384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f299802a7f0 con 0x5566fd860ad0 2024-05-15T14:10:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.601+0000 7f2986ffd640 1 -- 172.21.15.33:0/441319384 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f29a00ae300 con 0x5566fd860ad0 2024-05-15T14:10:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.601+0000 7f29b096a740 10 client.16623 fetch_fsmap learned FSMap version 35 2024-05-15T14:10:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.601+0000 7f29b096a740 1 -- 172.21.15.33:0/441319384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5566fd8dabf0 con 0x5566fd860ad0 2024-05-15T14:10:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.601+0000 7f2986ffd640 1 -- 172.21.15.33:0/441319384 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f299802a7f0 con 0x5566fd860ad0 2024-05-15T14:10:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.601+0000 7f2986ffd640 1 -- 172.21.15.33:0/441319384 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f29a00ad070 con 0x5566fd860ad0 2024-05-15T14:10:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.601+0000 7f29b096a740 10 client.16623 fetch_fsmap finished waiting for FSMap version 35 2024-05-15T14:10:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.601+0000 7f29b096a740 10 client.16623 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:10:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.601+0000 7f29b096a740 20 client.16623 populate_metadata read hostname 'smithi033' 2024-05-15T14:10:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.601+0000 7f29b096a740 1 --2- 172.21.15.33:0/441319384 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5566fd8db340 0x5566fd8dd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.601+0000 7f29b096a740 4 client.16623 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:10:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.601+0000 7f29b096a740 1 -- 172.21.15.33:0/441319384 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5566fd7c4360 con 0x5566fd8db340 2024-05-15T14:10:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.601+0000 7f29a77fe640 1 --2- 172.21.15.33:0/441319384 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5566fd8db340 0x5566fd8dd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.602+0000 7f29a77fe640 1 --2- 172.21.15.33:0/441319384 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5566fd8db340 0x5566fd8dd760 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65d4f2a1e7b877e1 server_cookie=11a3b8b2e0d7507d in_seq=0 out_seq=0 2024-05-15T14:10:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.602+0000 7f2986ffd640 10 client.16623 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:10:43.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.654+0000 7f2986ffd640 1 -- 172.21.15.33:0/441319384 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5566fd7c4360 con 0x5566fd8db340 2024-05-15T14:10:43.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.654+0000 7f2986ffd640 10 client.16623 handle_command_reply: tid=0 2024-05-15T14:10:43.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.654+0000 7f29b096a740 1 client.16623 shutdown 2024-05-15T14:10:43.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.654+0000 7f29b096a740 1 -- 172.21.15.33:0/441319384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566fd860ad0 msgr2=0x5566fd864600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:43.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.654+0000 7f29b096a740 1 --2- 172.21.15.33:0/441319384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566fd860ad0 0x5566fd864600 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f29a00e03c0 tx=0x7f29a00aa550 comp rx=0 tx=0).stop 2024-05-15T14:10:43.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.654+0000 7f29b096a740 1 -- 172.21.15.33:0/441319384 shutdown_connections 2024-05-15T14:10:43.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.654+0000 7f29b096a740 1 --2- 172.21.15.33:0/441319384 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5566fd8db340 0x5566fd8dd760 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.655+0000 7f2986ffd640 0 client.16623 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:10:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.655+0000 7f29b096a740 1 --2- 172.21.15.33:0/441319384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566fd860ad0 0x5566fd864600 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.655+0000 7f29b096a740 1 --2- 172.21.15.33:0/441319384 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5566fd856210 0x5566fd8640c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.655+0000 7f29b096a740 1 --2- 172.21.15.33:0/441319384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566fd8558f0 0x5566fd868d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.655+0000 7f29b096a740 1 -- 172.21.15.33:0/441319384 >> 172.21.15.33:0/441319384 conn(0x5566fd851e00 msgr2=0x5566fd857b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.655+0000 7f29b096a740 1 -- 172.21.15.33:0/441319384 shutdown_connections 2024-05-15T14:10:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.655+0000 7f29b096a740 1 -- 172.21.15.33:0/441319384 wait complete. 2024-05-15T14:10:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.655+0000 7f29b096a740 20 client.16623 trim_cache size 0 max 16384 2024-05-15T14:10:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:10:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:10:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:10:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.656+0000 7f29adc54640 1 -- 172.21.15.33:0/121059078 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f29940b2950 msgr2=0x7f29940b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.656+0000 7f29adc54640 1 --2- 172.21.15.33:0/121059078 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f29940b2950 0x7f29940b4e10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2990031440 tx=0x7f2990005e50 comp rx=0 tx=0).stop 2024-05-15T14:10:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.656+0000 7f29adc54640 1 -- 172.21.15.33:0/121059078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29a810d090 msgr2=0x7f29a819fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.656+0000 7f29adc54640 1 --2- 172.21.15.33:0/121059078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29a810d090 0x7f29a819fcd0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f29a0005fd0 tx=0x7f29a000ee50 comp rx=0 tx=0).stop 2024-05-15T14:10:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.656+0000 7f29adc54640 1 -- 172.21.15.33:0/121059078 shutdown_connections 2024-05-15T14:10:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.656+0000 7f29adc54640 1 --2- 172.21.15.33:0/121059078 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f29940b2950 0x7f29940b4e10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.656+0000 7f29adc54640 1 --2- 172.21.15.33:0/121059078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29a810d090 0x7f29a819fcd0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.656+0000 7f29adc54640 1 --2- 172.21.15.33:0/121059078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29a81045e0 0x7f29a819b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.656+0000 7f29adc54640 1 --2- 172.21.15.33:0/121059078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f29a8103cc0 0x7f29a819b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.656+0000 7f29adc54640 1 -- 172.21.15.33:0/121059078 >> 172.21.15.33:0/121059078 conn(0x7f29a8078a70 msgr2=0x7f29a8108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.656+0000 7f29adc54640 1 -- 172.21.15.33:0/121059078 shutdown_connections 2024-05-15T14:10:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:43.656+0000 7f29adc54640 1 -- 172.21.15.33:0/121059078 wait complete. 2024-05-15T14:10:44.078 INFO:tasks.workunit.client.0.smithi033.stdout: 1 596965 5.50 MB/sec execute 379 sec latency 101.814 ms 2024-05-15T14:10:44.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:44 smithi081 ceph-mon[36434]: pgmap v324: 97 pgs: 97 active+clean; 184 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.9 MiB/s wr, 275 op/s 2024-05-15T14:10:44.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:44 smithi081 ceph-mon[36434]: daemon mds.f restarted 2024-05-15T14:10:44.607 INFO:journalctl@ceph.mds.f.smithi081.stdout:May 15 14:10:44 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-f[66264]: ignoring --setuser ceph since I am not root 2024-05-15T14:10:44.608 INFO:journalctl@ceph.mds.f.smithi081.stdout:May 15 14:10:44 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-f[66264]: ignoring --setgroup ceph since I am not root 2024-05-15T14:10:44.608 INFO:journalctl@ceph.mds.f.smithi081.stdout:May 15 14:10:44 smithi081 ceph-mds[66268]: -- 172.21.15.81:0/1224545892 <== mon.1 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a3ad12fa40 con 0x55a3ad103000 2024-05-15T14:10:44.608 INFO:journalctl@ceph.mds.f.smithi081.stdout:May 15 14:10:44 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-f[66264]: starting mds.f at 2024-05-15T14:10:44.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:44 smithi033 ceph-mon[31650]: pgmap v324: 97 pgs: 97 active+clean; 184 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.9 MiB/s wr, 275 op/s 2024-05-15T14:10:44.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:44 smithi033 ceph-mon[31650]: daemon mds.f restarted 2024-05-15T14:10:44.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:44 smithi052 ceph-mon[36190]: pgmap v324: 97 pgs: 97 active+clean; 184 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.9 MiB/s wr, 275 op/s 2024-05-15T14:10:44.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:44 smithi052 ceph-mon[36190]: daemon mds.f restarted 2024-05-15T14:10:45.078 INFO:tasks.workunit.client.0.smithi033.stdout: 1 598379 5.50 MB/sec execute 380 sec latency 24.116 ms 2024-05-15T14:10:45.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:45 smithi033 ceph-mon[31650]: osdmap e94: 12 total, 12 up, 12 in 2024-05-15T14:10:45.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:45 smithi033 ceph-mon[31650]: mds.? [v2:172.21.15.81:6834/3674797952,v1:172.21.15.81:6835/3674797952] up:boot 2024-05-15T14:10:45.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:45 smithi033 ceph-mon[31650]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 2 up:standby-replay 7 up:standby 2024-05-15T14:10:45.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:45 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-15T14:10:45.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:45 smithi033 ceph-mon[31650]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:10:45.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:45 smithi052 ceph-mon[36190]: osdmap e94: 12 total, 12 up, 12 in 2024-05-15T14:10:45.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:45 smithi052 ceph-mon[36190]: mds.? [v2:172.21.15.81:6834/3674797952,v1:172.21.15.81:6835/3674797952] up:boot 2024-05-15T14:10:45.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:45 smithi052 ceph-mon[36190]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 2 up:standby-replay 7 up:standby 2024-05-15T14:10:45.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:45 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-15T14:10:45.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:45 smithi052 ceph-mon[36190]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:10:45.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:45 smithi081 ceph-mon[36434]: osdmap e94: 12 total, 12 up, 12 in 2024-05-15T14:10:45.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:45 smithi081 ceph-mon[36434]: mds.? [v2:172.21.15.81:6834/3674797952,v1:172.21.15.81:6835/3674797952] up:boot 2024-05-15T14:10:45.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:45 smithi081 ceph-mon[36434]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 2 up:standby-replay 7 up:standby 2024-05-15T14:10:45.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:45 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-15T14:10:45.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:45 smithi081 ceph-mon[36434]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:10:46.078 INFO:tasks.workunit.client.0.smithi033.stdout: 1 600519 5.52 MB/sec execute 381 sec latency 13.883 ms 2024-05-15T14:10:46.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:46 smithi033 ceph-mon[31650]: pgmap v326: 97 pgs: 97 active+clean; 184 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.1 MiB/s wr, 297 op/s 2024-05-15T14:10:46.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:46 smithi052 ceph-mon[36190]: pgmap v326: 97 pgs: 97 active+clean; 184 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.1 MiB/s wr, 297 op/s 2024-05-15T14:10:46.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:46 smithi081 ceph-mon[36434]: pgmap v326: 97 pgs: 97 active+clean; 184 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.1 MiB/s wr, 297 op/s 2024-05-15T14:10:47.078 INFO:tasks.workunit.client.0.smithi033.stdout: 1 601789 5.52 MB/sec execute 382 sec latency 22.702 ms 2024-05-15T14:10:48.078 INFO:tasks.workunit.client.0.smithi033.stdout: 1 603997 5.54 MB/sec execute 383 sec latency 15.752 ms 2024-05-15T14:10:48.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:48 smithi033 ceph-mon[31650]: pgmap v327: 97 pgs: 97 active+clean; 180 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 298 op/s 2024-05-15T14:10:48.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:48 smithi052 ceph-mon[36190]: pgmap v327: 97 pgs: 97 active+clean; 180 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 298 op/s 2024-05-15T14:10:48.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:48 smithi081 ceph-mon[36434]: pgmap v327: 97 pgs: 97 active+clean; 180 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 298 op/s 2024-05-15T14:10:49.078 INFO:tasks.workunit.client.0.smithi033.stdout: 1 605299 5.54 MB/sec execute 384 sec latency 23.198 ms 2024-05-15T14:10:50.079 INFO:tasks.workunit.client.0.smithi033.stdout: 1 607565 5.56 MB/sec execute 385 sec latency 23.230 ms 2024-05-15T14:10:50.654 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:50 smithi052 ceph-mon[36190]: pgmap v328: 97 pgs: 97 active+clean; 180 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 298 op/s 2024-05-15T14:10:50.654 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:50 smithi052 ceph-mon[36190]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:10:50.659 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:50 smithi081 ceph-mon[36434]: pgmap v328: 97 pgs: 97 active+clean; 180 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 298 op/s 2024-05-15T14:10:50.659 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:50 smithi081 ceph-mon[36434]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:10:50.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:50 smithi033 ceph-mon[31650]: pgmap v328: 97 pgs: 97 active+clean; 180 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 298 op/s 2024-05-15T14:10:50.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:50 smithi033 ceph-mon[31650]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:10:51.079 INFO:tasks.workunit.client.0.smithi033.stdout: 1 608413 5.55 MB/sec execute 386 sec latency 180.542 ms 2024-05-15T14:10:52.079 INFO:tasks.workunit.client.0.smithi033.stdout: 1 608413 5.54 MB/sec execute 387 sec latency 1180.651 ms 2024-05-15T14:10:52.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:52 smithi033 ceph-mon[31650]: pgmap v329: 97 pgs: 97 active+clean; 180 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 298 op/s 2024-05-15T14:10:52.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:52 smithi052 ceph-mon[36190]: pgmap v329: 97 pgs: 97 active+clean; 180 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 298 op/s 2024-05-15T14:10:52.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:52 smithi081 ceph-mon[36434]: pgmap v329: 97 pgs: 97 active+clean; 180 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 298 op/s 2024-05-15T14:10:53.079 INFO:tasks.workunit.client.0.smithi033.stdout: 1 608413 5.52 MB/sec execute 388 sec latency 2180.747 ms 2024-05-15T14:10:54.079 INFO:tasks.workunit.client.0.smithi033.stdout: 1 608850 5.52 MB/sec execute 389 sec latency 2925.529 ms 2024-05-15T14:10:54.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:54 smithi033 ceph-mon[31650]: pgmap v330: 97 pgs: 97 active+clean; 186 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.6 MiB/s wr, 280 op/s 2024-05-15T14:10:54.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:54 smithi052 ceph-mon[36190]: pgmap v330: 97 pgs: 97 active+clean; 186 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.6 MiB/s wr, 280 op/s 2024-05-15T14:10:54.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:54 smithi081 ceph-mon[36434]: pgmap v330: 97 pgs: 97 active+clean; 186 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.6 MiB/s wr, 280 op/s 2024-05-15T14:10:55.079 INFO:tasks.workunit.client.0.smithi033.stdout: 1 611103 5.54 MB/sec execute 390 sec latency 14.206 ms 2024-05-15T14:10:56.079 INFO:tasks.workunit.client.0.smithi033.stdout: 1 611963 5.53 MB/sec execute 391 sec latency 238.111 ms 2024-05-15T14:10:56.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:56 smithi033 ceph-mon[31650]: pgmap v331: 97 pgs: 97 active+clean; 186 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.5 MiB/s wr, 256 op/s 2024-05-15T14:10:56.759 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.758+0000 7f4d90d4e640 1 -- 172.21.15.33:0/1181370569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d8c106d20 msgr2=0x7f4d8c10e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.758+0000 7f4d90d4e640 1 --2- 172.21.15.33:0/1181370569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d8c106d20 0x7f4d8c10e8a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f4d84009cd0 tx=0x7f4d8402a8d0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.758+0000 7f4d90d4e640 1 -- 172.21.15.33:0/1181370569 shutdown_connections 2024-05-15T14:10:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.758+0000 7f4d90d4e640 1 --2- 172.21.15.33:0/1181370569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d8c106d20 0x7f4d8c10e8a0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.758+0000 7f4d90d4e640 1 --2- 172.21.15.33:0/1181370569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d8c102800 0x7f4d8c1065f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.758+0000 7f4d90d4e640 1 --2- 172.21.15.33:0/1181370569 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d8c101e50 0x7f4d8c102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.758+0000 7f4d90d4e640 1 -- 172.21.15.33:0/1181370569 >> 172.21.15.33:0/1181370569 conn(0x7f4d8c078810 msgr2=0x7f4d8c078c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.758+0000 7f4d90d4e640 1 -- 172.21.15.33:0/1181370569 shutdown_connections 2024-05-15T14:10:56.760 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.758+0000 7f4d90d4e640 1 -- 172.21.15.33:0/1181370569 wait complete. 2024-05-15T14:10:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.759+0000 7f4d90d4e640 1 Processor -- start 2024-05-15T14:10:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.759+0000 7f4d90d4e640 1 -- start start 2024-05-15T14:10:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.759+0000 7f4d90d4e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d8c101e50 0x7f4d8c19a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.759+0000 7f4d90d4e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d8c106d20 0x7f4d8c19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.759+0000 7f4d90d4e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d8c19ab10 0x7f4d8c1a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.759+0000 7f4d90d4e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c111c00 con 0x7f4d8c101e50 2024-05-15T14:10:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.759+0000 7f4d90d4e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c111a80 con 0x7f4d8c106d20 2024-05-15T14:10:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.759+0000 7f4d90d4e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c111d80 con 0x7f4d8c19ab10 2024-05-15T14:10:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.759+0000 7f4d8a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d8c101e50 0x7f4d8c19a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.759+0000 7f4d8a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d8c101e50 0x7f4d8c19a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55912/0 (socket says 172.21.15.33:55912) 2024-05-15T14:10:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.759+0000 7f4d8a59c640 1 -- 172.21.15.33:0/1284421888 learned_addr learned my addr 172.21.15.33:0/1284421888 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.759+0000 7f4d83fff640 1 --2- 172.21.15.33:0/1284421888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d8c106d20 0x7f4d8c19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.759+0000 7f4d8ad9d640 1 --2- 172.21.15.33:0/1284421888 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d8c19ab10 0x7f4d8c1a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.759+0000 7f4d8a59c640 1 -- 172.21.15.33:0/1284421888 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d8c19ab10 msgr2=0x7f4d8c1a50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.759+0000 7f4d8a59c640 1 --2- 172.21.15.33:0/1284421888 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d8c19ab10 0x7f4d8c1a50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.759+0000 7f4d8a59c640 1 -- 172.21.15.33:0/1284421888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d8c106d20 msgr2=0x7f4d8c19a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.759+0000 7f4d8a59c640 1 --2- 172.21.15.33:0/1284421888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d8c106d20 0x7f4d8c19a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.759+0000 7f4d8a59c640 1 -- 172.21.15.33:0/1284421888 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d8c1a5890 con 0x7f4d8c101e50 2024-05-15T14:10:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.759+0000 7f4d83fff640 1 --2- 172.21.15.33:0/1284421888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d8c106d20 0x7f4d8c19a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:10:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.760+0000 7f4d8a59c640 1 --2- 172.21.15.33:0/1284421888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d8c101e50 0x7f4d8c19a040 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c00e9f0 tx=0x7f4d7c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.760+0000 7f4d837fe640 1 -- 172.21.15.33:0/1284421888 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c00cd80 con 0x7f4d8c101e50 2024-05-15T14:10:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.760+0000 7f4d837fe640 1 -- 172.21.15.33:0/1284421888 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d7c0108e0 con 0x7f4d8c101e50 2024-05-15T14:10:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.760+0000 7f4d90d4e640 1 -- 172.21.15.33:0/1284421888 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d8c1a5b80 con 0x7f4d8c101e50 2024-05-15T14:10:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.760+0000 7f4d837fe640 1 -- 172.21.15.33:0/1284421888 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c018980 con 0x7f4d8c101e50 2024-05-15T14:10:56.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.760+0000 7f4d90d4e640 1 -- 172.21.15.33:0/1284421888 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d8c1a6120 con 0x7f4d8c101e50 2024-05-15T14:10:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.762+0000 7f4d837fe640 1 -- 172.21.15.33:0/1284421888 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4d7c010430 con 0x7f4d8c101e50 2024-05-15T14:10:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.763+0000 7f4d93a60740 1 Processor -- start 2024-05-15T14:10:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.763+0000 7f4d93a60740 1 -- start start 2024-05-15T14:10:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.763+0000 7f4d93a60740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5575f707f7f0 0x5575f707fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.763+0000 7f4d93a60740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575f7080110 0x5575f708e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.763+0000 7f4d93a60740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575f708eee0 0x5575f70912d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.763+0000 7f4d93a60740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5575f6ff2c30 con 0x5575f708eee0 2024-05-15T14:10:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.763+0000 7f4d93a60740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5575f6fe27d0 con 0x5575f7080110 2024-05-15T14:10:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.763+0000 7f4d93a60740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5575f6fe6810 con 0x5575f707f7f0 2024-05-15T14:10:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.763+0000 7f4d83fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575f708eee0 0x5575f70912d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.763+0000 7f4d83fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575f708eee0 0x5575f70912d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55924/0 (socket says 172.21.15.33:55924) 2024-05-15T14:10:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.763+0000 7f4d83fff640 1 -- 172.21.15.33:0/3658172809 learned_addr learned my addr 172.21.15.33:0/3658172809 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.764+0000 7f4d8ad9d640 1 --2- 172.21.15.33:0/3658172809 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5575f707f7f0 0x5575f707fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.764+0000 7f4d8a59c640 1 --2- 172.21.15.33:0/3658172809 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575f7080110 0x5575f708e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.764+0000 7f4d837fe640 1 --2- 172.21.15.33:0/1284421888 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f4d780b29d0 0x7f4d780b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.764+0000 7f4d83fff640 1 -- 172.21.15.33:0/3658172809 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5575f707f7f0 msgr2=0x5575f707fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.764+0000 7f4d83fff640 1 --2- 172.21.15.33:0/3658172809 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5575f707f7f0 0x5575f707fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.764+0000 7f4d83fff640 1 -- 172.21.15.33:0/3658172809 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575f7080110 msgr2=0x5575f708e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.764+0000 7f4d83fff640 1 --2- 172.21.15.33:0/3658172809 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575f7080110 0x5575f708e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.764+0000 7f4d83fff640 1 -- 172.21.15.33:0/3658172809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575f6cf7210 con 0x5575f708eee0 2024-05-15T14:10:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.764+0000 7f4d8ad9d640 1 --2- 172.21.15.33:0/3658172809 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5575f707f7f0 0x5575f707fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:10:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.764+0000 7f4d8a59c640 1 --2- 172.21.15.33:0/3658172809 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575f7080110 0x5575f708e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:10:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.764+0000 7f4d8ad9d640 1 --2- 172.21.15.33:0/1284421888 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f4d780b29d0 0x7f4d780b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.764+0000 7f4d837fe640 1 -- 172.21.15.33:0/1284421888 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8498+0+0 (secure 0 0 0) 0x7f4d7c0e2a30 con 0x7f4d8c101e50 2024-05-15T14:10:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.764+0000 7f4d83fff640 1 --2- 172.21.15.33:0/3658172809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575f708eee0 0x5575f70912d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400bc80 tx=0x7f4d7400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a172b0e95bb11125 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.764+0000 7f4d80ff9640 1 -- 172.21.15.33:0/3658172809 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d74015070 con 0x5575f708eee0 2024-05-15T14:10:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.764+0000 7f4d80ff9640 1 -- 172.21.15.33:0/3658172809 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d740054d0 con 0x5575f708eee0 2024-05-15T14:10:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.764+0000 7f4d80ff9640 1 -- 172.21.15.33:0/3658172809 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d74011440 con 0x5575f708eee0 2024-05-15T14:10:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.765+0000 7f4d93a60740 1 -- 172.21.15.33:0/3658172809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575f708eee0 msgr2=0x5575f70912d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.765+0000 7f4d8ad9d640 1 --2- 172.21.15.33:0/1284421888 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f4d780b29d0 0x7f4d780b4e90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4d84006010 tx=0x7f4d840095c0 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.765+0000 7f4d93a60740 1 --2- 172.21.15.33:0/3658172809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575f708eee0 0x5575f70912d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400bc80 tx=0x7f4d7400f730 comp rx=0 tx=0).stop 2024-05-15T14:10:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.765+0000 7f4d93a60740 1 -- 172.21.15.33:0/3658172809 shutdown_connections 2024-05-15T14:10:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.765+0000 7f4d93a60740 1 --2- 172.21.15.33:0/3658172809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575f708eee0 0x5575f70912d0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.765+0000 7f4d93a60740 1 --2- 172.21.15.33:0/3658172809 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575f7080110 0x5575f708e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.765+0000 7f4d93a60740 1 --2- 172.21.15.33:0/3658172809 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5575f707f7f0 0x5575f707fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.765+0000 7f4d93a60740 1 -- 172.21.15.33:0/3658172809 >> 172.21.15.33:0/3658172809 conn(0x5575f707be00 msgr2=0x5575f707c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.765+0000 7f4d93a60740 1 -- 172.21.15.33:0/3658172809 shutdown_connections 2024-05-15T14:10:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.765+0000 7f4d93a60740 1 -- 172.21.15.33:0/3658172809 wait complete. 2024-05-15T14:10:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.766+0000 7f4d93a60740 1 -- start start 2024-05-15T14:10:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.766+0000 7f4d93a60740 1 Processor -- start 2024-05-15T14:10:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.766+0000 7f4d93a60740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575f707f7f0 0x5575f708e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.766+0000 7f4d93a60740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5575f7080110 0x5575f708bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.766+0000 7f4d93a60740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575f708eee0 0x5575f708c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.766+0000 7f4d93a60740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5575f6cc8790 con 0x5575f707f7f0 2024-05-15T14:10:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.766+0000 7f4d93a60740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5575f6f9e890 con 0x5575f708eee0 2024-05-15T14:10:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.766+0000 7f4d93a60740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5575f7091bb0 con 0x5575f7080110 2024-05-15T14:10:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.766+0000 7f4d93a60740 10 client.0 _ll_register_callbacks cb 0x5575f6fa1d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:10:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.766+0000 7f4d8a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575f707f7f0 0x5575f708e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.766+0000 7f4d8a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575f707f7f0 0x5575f708e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55934/0 (socket says 172.21.15.33:55934) 2024-05-15T14:10:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.766+0000 7f4d8a59c640 1 -- 172.21.15.33:0/4176237596 learned_addr learned my addr 172.21.15.33:0/4176237596 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.766+0000 7f4d83fff640 1 --2- 172.21.15.33:0/4176237596 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5575f7080110 0x5575f708bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.766+0000 7f4d8a59c640 1 -- 172.21.15.33:0/4176237596 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5575f7080110 msgr2=0x5575f708bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.766+0000 7f4d8a59c640 1 --2- 172.21.15.33:0/4176237596 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5575f7080110 0x5575f708bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.766+0000 7f4d8a59c640 1 -- 172.21.15.33:0/4176237596 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575f708eee0 msgr2=0x5575f708c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.766+0000 7f4d8a59c640 1 --2- 172.21.15.33:0/4176237596 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575f708eee0 0x5575f708c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.766+0000 7f4d8a59c640 1 -- 172.21.15.33:0/4176237596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575f6dc3620 con 0x5575f707f7f0 2024-05-15T14:10:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.767+0000 7f4d83fff640 1 --2- 172.21.15.33:0/4176237596 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5575f7080110 0x5575f708bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:10:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.767+0000 7f4d8a59c640 1 --2- 172.21.15.33:0/4176237596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575f707f7f0 0x5575f708e850 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0a9cb0 tx=0x7f4d7c0a9740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4aa9ad2e68df37 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.767+0000 7f4d71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:10:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.767+0000 7f4d71ffb640 1 -- 172.21.15.33:0/4176237596 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c0c0060 con 0x5575f707f7f0 2024-05-15T14:10:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.767+0000 7f4d71ffb640 1 -- 172.21.15.33:0/4176237596 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8498+0+0 (secure 0 0 0) 0x7f4d7c010a90 con 0x5575f707f7f0 2024-05-15T14:10:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.767+0000 7f4d93a60740 1 -- 172.21.15.33:0/4176237596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d740054d0 con 0x5575f707f7f0 2024-05-15T14:10:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.767+0000 7f4d71ffb640 1 -- 172.21.15.33:0/4176237596 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f4d7c0aa420 con 0x5575f707f7f0 2024-05-15T14:10:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.767+0000 7f4d93a60740 10 client.16641 fetch_fsmap learned FSMap version 38 2024-05-15T14:10:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.767+0000 7f4d93a60740 1 -- 172.21.15.33:0/4176237596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5575f709d650 con 0x5575f707f7f0 2024-05-15T14:10:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.767+0000 7f4d71ffb640 1 -- 172.21.15.33:0/4176237596 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d740054d0 con 0x5575f707f7f0 2024-05-15T14:10:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.767+0000 7f4d71ffb640 1 -- 172.21.15.33:0/4176237596 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7f4d7c018b20 con 0x5575f707f7f0 2024-05-15T14:10:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.767+0000 7f4d93a60740 10 client.16641 fetch_fsmap finished waiting for FSMap version 38 2024-05-15T14:10:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.767+0000 7f4d93a60740 10 client.16641 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:10:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.767+0000 7f4d93a60740 20 client.16641 populate_metadata read hostname 'smithi033' 2024-05-15T14:10:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.767+0000 7f4d93a60740 1 --2- 172.21.15.33:0/4176237596 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5575f709eb10 0x5575f70a0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.767+0000 7f4d93a60740 4 client.16641 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:10:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.767+0000 7f4d93a60740 1 -- 172.21.15.33:0/4176237596 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575f6ff4b80 con 0x5575f709eb10 2024-05-15T14:10:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.768+0000 7f4d83fff640 1 --2- 172.21.15.33:0/4176237596 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5575f709eb10 0x5575f70a0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.768+0000 7f4d83fff640 1 --2- 172.21.15.33:0/4176237596 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5575f709eb10 0x5575f70a0f40 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c7fe592ef7356d9 server_cookie=5984c48f50f7a267 in_seq=0 out_seq=0 2024-05-15T14:10:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.768+0000 7f4d71ffb640 10 client.16641 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:10:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.769+0000 7f4d71ffb640 1 -- 172.21.15.33:0/4176237596 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5575f6ff4b80 con 0x5575f709eb10 2024-05-15T14:10:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.769+0000 7f4d71ffb640 10 client.16641 handle_command_reply: tid=0 2024-05-15T14:10:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.769+0000 7f4d93a60740 1 client.16641 shutdown 2024-05-15T14:10:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.769+0000 7f4d93a60740 1 -- 172.21.15.33:0/4176237596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575f707f7f0 msgr2=0x5575f708e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.769+0000 7f4d93a60740 1 --2- 172.21.15.33:0/4176237596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575f707f7f0 0x5575f708e850 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0a9cb0 tx=0x7f4d7c0a9740 comp rx=0 tx=0).stop 2024-05-15T14:10:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.769+0000 7f4d93a60740 1 -- 172.21.15.33:0/4176237596 shutdown_connections 2024-05-15T14:10:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.769+0000 7f4d93a60740 1 --2- 172.21.15.33:0/4176237596 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5575f709eb10 0x5575f70a0f40 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.769+0000 7f4d93a60740 1 --2- 172.21.15.33:0/4176237596 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575f708eee0 0x5575f708c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.769+0000 7f4d93a60740 1 --2- 172.21.15.33:0/4176237596 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5575f7080110 0x5575f708bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.769+0000 7f4d71ffb640 0 client.16641 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:10:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.769+0000 7f4d93a60740 1 --2- 172.21.15.33:0/4176237596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575f707f7f0 0x5575f708e850 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.769+0000 7f4d93a60740 1 -- 172.21.15.33:0/4176237596 >> 172.21.15.33:0/4176237596 conn(0x5575f707be00 msgr2=0x5575f7081c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.770+0000 7f4d93a60740 1 -- 172.21.15.33:0/4176237596 shutdown_connections 2024-05-15T14:10:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.770+0000 7f4d93a60740 1 -- 172.21.15.33:0/4176237596 wait complete. 2024-05-15T14:10:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.770+0000 7f4d93a60740 20 client.16641 trim_cache size 0 max 16384 2024-05-15T14:10:56.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.785+0000 7f4d93a60740 1 Processor -- start 2024-05-15T14:10:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.785+0000 7f4d93a60740 1 -- start start 2024-05-15T14:10:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.785+0000 7f4d93a60740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575f707f7f0 0x5575f708e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.785+0000 7f4d93a60740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5575f7080110 0x5575f70a6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.785+0000 7f4d93a60740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575f708eee0 0x5575f70a6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.785+0000 7f4d93a60740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c1aee20 con 0x5575f707f7f0 2024-05-15T14:10:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.785+0000 7f4d93a60740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c1ac660 con 0x5575f708eee0 2024-05-15T14:10:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.785+0000 7f4d93a60740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c1aa4b0 con 0x5575f7080110 2024-05-15T14:10:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.785+0000 7f4d8a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575f707f7f0 0x5575f708e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.785+0000 7f4d8a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575f707f7f0 0x5575f708e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55944/0 (socket says 172.21.15.33:55944) 2024-05-15T14:10:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.785+0000 7f4d8a59c640 1 -- 172.21.15.33:0/2049797146 learned_addr learned my addr 172.21.15.33:0/2049797146 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.785+0000 7f4d8ad9d640 1 --2- 172.21.15.33:0/2049797146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575f708eee0 0x5575f70a6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.785+0000 7f4d8a59c640 1 -- 172.21.15.33:0/2049797146 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5575f7080110 msgr2=0x5575f70a6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.785+0000 7f4d8a59c640 1 --2- 172.21.15.33:0/2049797146 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5575f7080110 0x5575f70a6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.785+0000 7f4d8a59c640 1 -- 172.21.15.33:0/2049797146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575f708eee0 msgr2=0x5575f70a6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.785+0000 7f4d8a59c640 1 --2- 172.21.15.33:0/2049797146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575f708eee0 0x5575f70a6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.786+0000 7f4d8a59c640 1 -- 172.21.15.33:0/2049797146 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575f708c3d0 con 0x5575f707f7f0 2024-05-15T14:10:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.786+0000 7f4d8a59c640 1 --2- 172.21.15.33:0/2049797146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575f707f7f0 0x5575f708e380 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0a9a20 tx=0x7f4d7c0a9af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d52430baa168f087 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.786+0000 7f4d73fff640 1 -- 172.21.15.33:0/2049797146 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c0c0020 con 0x5575f707f7f0 2024-05-15T14:10:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.786+0000 7f4d73fff640 1 -- 172.21.15.33:0/2049797146 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d7c0ae0d0 con 0x5575f707f7f0 2024-05-15T14:10:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.786+0000 7f4d73fff640 1 -- 172.21.15.33:0/2049797146 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c018b20 con 0x5575f707f7f0 2024-05-15T14:10:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.786+0000 7f4d93a60740 1 -- 172.21.15.33:0/2049797146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575f707f7f0 msgr2=0x5575f708e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.786+0000 7f4d93a60740 1 --2- 172.21.15.33:0/2049797146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575f707f7f0 0x5575f708e380 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0a9a20 tx=0x7f4d7c0a9af0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.786+0000 7f4d93a60740 1 -- 172.21.15.33:0/2049797146 shutdown_connections 2024-05-15T14:10:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.786+0000 7f4d93a60740 1 --2- 172.21.15.33:0/2049797146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575f708eee0 0x5575f70a6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.786+0000 7f4d93a60740 1 --2- 172.21.15.33:0/2049797146 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5575f7080110 0x5575f70a6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.786+0000 7f4d93a60740 1 --2- 172.21.15.33:0/2049797146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575f707f7f0 0x5575f708e380 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.786+0000 7f4d93a60740 1 -- 172.21.15.33:0/2049797146 >> 172.21.15.33:0/2049797146 conn(0x5575f707be00 msgr2=0x5575f7081c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.787+0000 7f4d93a60740 1 -- 172.21.15.33:0/2049797146 shutdown_connections 2024-05-15T14:10:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.787+0000 7f4d93a60740 1 -- 172.21.15.33:0/2049797146 wait complete. 2024-05-15T14:10:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.787+0000 7f4d93a60740 1 -- start start 2024-05-15T14:10:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.787+0000 7f4d93a60740 1 Processor -- start 2024-05-15T14:10:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.787+0000 7f4d93a60740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575f707f7f0 0x5575f7102710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.787+0000 7f4d93a60740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575f7080110 0x5575f7102c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.787+0000 7f4d93a60740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5575f708eee0 0x5575f7093110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.787+0000 7f4d93a60740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c1aab20 con 0x5575f707f7f0 2024-05-15T14:10:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.787+0000 7f4d93a60740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c1abf90 con 0x5575f7080110 2024-05-15T14:10:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.787+0000 7f4d93a60740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c1ab1f0 con 0x5575f708eee0 2024-05-15T14:10:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.787+0000 7f4d93a60740 10 client.0 _ll_register_callbacks cb 0x7f4d7c0a9670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:10:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.787+0000 7f4d8a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575f707f7f0 0x5575f7102710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.787+0000 7f4d8a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575f707f7f0 0x5575f7102710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55956/0 (socket says 172.21.15.33:55956) 2024-05-15T14:10:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.787+0000 7f4d8a59c640 1 -- 172.21.15.33:0/1392755492 learned_addr learned my addr 172.21.15.33:0/1392755492 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:10:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.787+0000 7f4d8ad9d640 1 --2- 172.21.15.33:0/1392755492 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5575f708eee0 0x5575f7093110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.788+0000 7f4d8a59c640 1 -- 172.21.15.33:0/1392755492 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5575f708eee0 msgr2=0x5575f7093110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.788+0000 7f4d8a59c640 1 --2- 172.21.15.33:0/1392755492 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5575f708eee0 0x5575f7093110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.788+0000 7f4d8a59c640 1 -- 172.21.15.33:0/1392755492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575f7080110 msgr2=0x5575f7102c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:10:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.788+0000 7f4d8a59c640 1 --2- 172.21.15.33:0/1392755492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575f7080110 0x5575f7102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.788+0000 7f4d8a59c640 1 -- 172.21.15.33:0/1392755492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575f70937d0 con 0x5575f707f7f0 2024-05-15T14:10:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.788+0000 7f4d8a59c640 1 --2- 172.21.15.33:0/1392755492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575f707f7f0 0x5575f7102710 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c018eb0 tx=0x7f4d7c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809e4da12f7e2303 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:10:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.788+0000 7f4d71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:10:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.788+0000 7f4d71ffb640 1 -- 172.21.15.33:0/1392755492 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c0c0070 con 0x5575f707f7f0 2024-05-15T14:10:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.788+0000 7f4d71ffb640 1 -- 172.21.15.33:0/1392755492 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8498+0+0 (secure 0 0 0) 0x7f4d7c0a9190 con 0x5575f707f7f0 2024-05-15T14:10:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.788+0000 7f4d93a60740 1 -- 172.21.15.33:0/1392755492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d7c0ae0d0 con 0x5575f707f7f0 2024-05-15T14:10:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.788+0000 7f4d71ffb640 1 -- 172.21.15.33:0/1392755492 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f4d7c0b2b80 con 0x5575f707f7f0 2024-05-15T14:10:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.788+0000 7f4d93a60740 10 client.16647 fetch_fsmap learned FSMap version 38 2024-05-15T14:10:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.788+0000 7f4d93a60740 1 -- 172.21.15.33:0/1392755492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5575f7109e50 con 0x5575f707f7f0 2024-05-15T14:10:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.788+0000 7f4d71ffb640 1 -- 172.21.15.33:0/1392755492 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d7c0ae0d0 con 0x5575f707f7f0 2024-05-15T14:10:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.788+0000 7f4d71ffb640 1 -- 172.21.15.33:0/1392755492 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7f4d7c0b9030 con 0x5575f707f7f0 2024-05-15T14:10:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.789+0000 7f4d93a60740 10 client.16647 fetch_fsmap finished waiting for FSMap version 38 2024-05-15T14:10:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.789+0000 7f4d93a60740 10 client.16647 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:10:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.789+0000 7f4d93a60740 20 client.16647 populate_metadata read hostname 'smithi033' 2024-05-15T14:10:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.789+0000 7f4d93a60740 1 --2- 172.21.15.33:0/1392755492 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5575f710a350 0x5575f710c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:10:56.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.789+0000 7f4d93a60740 4 client.16647 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:10:56.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.789+0000 7f4d93a60740 1 -- 172.21.15.33:0/1392755492 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5575f6fee360 con 0x5575f710a350 2024-05-15T14:10:56.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.789+0000 7f4d83fff640 1 --2- 172.21.15.33:0/1392755492 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5575f710a350 0x5575f710c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:10:56.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.789+0000 7f4d83fff640 1 --2- 172.21.15.33:0/1392755492 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5575f710a350 0x5575f710c770 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=812bc72bbd95251d server_cookie=6d42e972d2cf0c90 in_seq=0 out_seq=0 2024-05-15T14:10:56.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.789+0000 7f4d71ffb640 10 client.16647 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:10:56.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.837+0000 7f4d71ffb640 1 -- 172.21.15.33:0/1392755492 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5575f6fee360 con 0x5575f710a350 2024-05-15T14:10:56.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.837+0000 7f4d71ffb640 10 client.16647 handle_command_reply: tid=0 2024-05-15T14:10:56.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.837+0000 7f4d93a60740 1 client.16647 shutdown 2024-05-15T14:10:56.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.837+0000 7f4d93a60740 1 -- 172.21.15.33:0/1392755492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575f707f7f0 msgr2=0x5575f7102710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:56.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.837+0000 7f4d93a60740 1 --2- 172.21.15.33:0/1392755492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575f707f7f0 0x5575f7102710 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c018eb0 tx=0x7f4d7c000bc0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.837+0000 7f4d93a60740 1 -- 172.21.15.33:0/1392755492 shutdown_connections 2024-05-15T14:10:56.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.837+0000 7f4d93a60740 1 --2- 172.21.15.33:0/1392755492 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5575f710a350 0x5575f710c770 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.837+0000 7f4d93a60740 1 --2- 172.21.15.33:0/1392755492 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5575f708eee0 0x5575f7093110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.837+0000 7f4d93a60740 1 --2- 172.21.15.33:0/1392755492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575f7080110 0x5575f7102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.837+0000 7f4d71ffb640 0 client.16647 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:10:56.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.837+0000 7f4d93a60740 1 --2- 172.21.15.33:0/1392755492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5575f707f7f0 0x5575f7102710 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.837+0000 7f4d93a60740 1 -- 172.21.15.33:0/1392755492 >> 172.21.15.33:0/1392755492 conn(0x5575f707be00 msgr2=0x5575f707a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:56.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.837+0000 7f4d93a60740 1 -- 172.21.15.33:0/1392755492 shutdown_connections 2024-05-15T14:10:56.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.837+0000 7f4d93a60740 1 -- 172.21.15.33:0/1392755492 wait complete. 2024-05-15T14:10:56.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.837+0000 7f4d93a60740 20 client.16647 trim_cache size 0 max 16384 2024-05-15T14:10:56.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:10:56.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:10:56.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:10:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.837+0000 7f4d90d4e640 1 -- 172.21.15.33:0/1284421888 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f4d780b29d0 msgr2=0x7f4d780b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.837+0000 7f4d90d4e640 1 --2- 172.21.15.33:0/1284421888 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f4d780b29d0 0x7f4d780b4e90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4d84006010 tx=0x7f4d840095c0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.837+0000 7f4d90d4e640 1 -- 172.21.15.33:0/1284421888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d8c101e50 msgr2=0x7f4d8c19a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:10:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.837+0000 7f4d90d4e640 1 --2- 172.21.15.33:0/1284421888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d8c101e50 0x7f4d8c19a040 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c00e9f0 tx=0x7f4d7c00eeb0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.838+0000 7f4d90d4e640 1 -- 172.21.15.33:0/1284421888 shutdown_connections 2024-05-15T14:10:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.838+0000 7f4d90d4e640 1 --2- 172.21.15.33:0/1284421888 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f4d780b29d0 0x7f4d780b4e90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.838+0000 7f4d90d4e640 1 --2- 172.21.15.33:0/1284421888 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4d8c19ab10 0x7f4d8c1a50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.838+0000 7f4d90d4e640 1 --2- 172.21.15.33:0/1284421888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d8c106d20 0x7f4d8c19a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.838+0000 7f4d90d4e640 1 --2- 172.21.15.33:0/1284421888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d8c101e50 0x7f4d8c19a040 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:10:56.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.838+0000 7f4d90d4e640 1 -- 172.21.15.33:0/1284421888 >> 172.21.15.33:0/1284421888 conn(0x7f4d8c078810 msgr2=0x7f4d8c105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:10:56.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.838+0000 7f4d90d4e640 1 -- 172.21.15.33:0/1284421888 shutdown_connections 2024-05-15T14:10:56.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:10:56.838+0000 7f4d90d4e640 1 -- 172.21.15.33:0/1284421888 wait complete. 2024-05-15T14:10:56.842 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:56 smithi052 ceph-mon[36190]: pgmap v331: 97 pgs: 97 active+clean; 186 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.5 MiB/s wr, 256 op/s 2024-05-15T14:10:56.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:56 smithi081 ceph-mon[36434]: pgmap v331: 97 pgs: 97 active+clean; 186 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.5 MiB/s wr, 256 op/s 2024-05-15T14:10:57.079 INFO:tasks.workunit.client.0.smithi033.stdout: 1 611963 5.52 MB/sec execute 392 sec latency 1238.202 ms 2024-05-15T14:10:58.079 INFO:tasks.workunit.client.0.smithi033.stdout: 1 611963 5.50 MB/sec execute 393 sec latency 2238.279 ms 2024-05-15T14:10:58.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:10:58 smithi033 ceph-mon[31650]: pgmap v332: 97 pgs: 97 active+clean; 186 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.7 MiB/s wr, 309 op/s 2024-05-15T14:10:58.842 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:10:58 smithi052 ceph-mon[36190]: pgmap v332: 97 pgs: 97 active+clean; 186 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.7 MiB/s wr, 309 op/s 2024-05-15T14:10:58.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:10:58 smithi081 ceph-mon[36434]: pgmap v332: 97 pgs: 97 active+clean; 186 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.7 MiB/s wr, 309 op/s 2024-05-15T14:10:59.080 INFO:tasks.workunit.client.0.smithi033.stdout: 1 612304 5.50 MB/sec execute 394 sec latency 3036.565 ms 2024-05-15T14:11:00.079 INFO:tasks.workunit.client.0.smithi033.stdout: 1 614276 5.51 MB/sec execute 395 sec latency 16.122 ms 2024-05-15T14:11:00.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:00 smithi033 ceph-mon[31650]: pgmap v333: 97 pgs: 97 active+clean; 186 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 952 KiB/s wr, 170 op/s 2024-05-15T14:11:00.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:00 smithi052 ceph-mon[36190]: pgmap v333: 97 pgs: 97 active+clean; 186 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 952 KiB/s wr, 170 op/s 2024-05-15T14:11:00.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:00 smithi081 ceph-mon[36434]: pgmap v333: 97 pgs: 97 active+clean; 186 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 952 KiB/s wr, 170 op/s 2024-05-15T14:11:01.079 INFO:tasks.workunit.client.0.smithi033.stdout: 1 614804 5.50 MB/sec execute 396 sec latency 581.578 ms 2024-05-15T14:11:02.079 INFO:tasks.workunit.client.0.smithi033.stdout: 1 614804 5.49 MB/sec execute 397 sec latency 1581.662 ms 2024-05-15T14:11:02.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:02 smithi033 ceph-mon[31650]: pgmap v334: 97 pgs: 97 active+clean; 186 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 952 KiB/s wr, 170 op/s 2024-05-15T14:11:02.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:02 smithi052 ceph-mon[36190]: pgmap v334: 97 pgs: 97 active+clean; 186 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 952 KiB/s wr, 170 op/s 2024-05-15T14:11:02.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:02 smithi081 ceph-mon[36434]: pgmap v334: 97 pgs: 97 active+clean; 186 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 952 KiB/s wr, 170 op/s 2024-05-15T14:11:03.079 INFO:tasks.workunit.client.0.smithi033.stdout: 1 614804 5.48 MB/sec execute 398 sec latency 2581.699 ms 2024-05-15T14:11:04.079 INFO:tasks.workunit.client.0.smithi033.stdout: 1 614961 5.46 MB/sec execute 399 sec latency 3328.720 ms 2024-05-15T14:11:04.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:04 smithi033 ceph-mon[31650]: pgmap v335: 97 pgs: 97 active+clean; 188 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.2 MiB/s wr, 228 op/s 2024-05-15T14:11:04.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:04 smithi033 ceph-mon[31650]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:11:04.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:04 smithi052 ceph-mon[36190]: pgmap v335: 97 pgs: 97 active+clean; 188 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.2 MiB/s wr, 228 op/s 2024-05-15T14:11:04.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:04 smithi052 ceph-mon[36190]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:11:04.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:04 smithi081 ceph-mon[36434]: pgmap v335: 97 pgs: 97 active+clean; 188 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.2 MiB/s wr, 228 op/s 2024-05-15T14:11:04.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:04 smithi081 ceph-mon[36434]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:11:05.080 INFO:tasks.workunit.client.0.smithi033.stdout: 1 615505 5.45 MB/sec execute 400 sec latency 348.084 ms 2024-05-15T14:11:06.080 INFO:tasks.workunit.client.0.smithi033.stdout: 1 615505 5.44 MB/sec execute 401 sec latency 1348.201 ms 2024-05-15T14:11:06.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:06 smithi033 ceph-mon[31650]: pgmap v336: 97 pgs: 97 active+clean; 188 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 593 KiB/s wr, 132 op/s 2024-05-15T14:11:06.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:06 smithi052 ceph-mon[36190]: pgmap v336: 97 pgs: 97 active+clean; 188 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 593 KiB/s wr, 132 op/s 2024-05-15T14:11:06.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:06 smithi081 ceph-mon[36434]: pgmap v336: 97 pgs: 97 active+clean; 188 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 593 KiB/s wr, 132 op/s 2024-05-15T14:11:07.080 INFO:tasks.workunit.client.0.smithi033.stdout: 1 615505 5.43 MB/sec execute 402 sec latency 2348.299 ms 2024-05-15T14:11:08.080 INFO:tasks.workunit.client.0.smithi033.stdout: 1 615505 5.41 MB/sec execute 403 sec latency 3348.403 ms 2024-05-15T14:11:08.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:08 smithi033 ceph-mon[31650]: pgmap v337: 97 pgs: 97 active+clean; 195 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 180 op/s 2024-05-15T14:11:08.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:08 smithi052 ceph-mon[36190]: pgmap v337: 97 pgs: 97 active+clean; 195 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 180 op/s 2024-05-15T14:11:08.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:08 smithi081 ceph-mon[36434]: pgmap v337: 97 pgs: 97 active+clean; 195 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 180 op/s 2024-05-15T14:11:09.080 INFO:tasks.workunit.client.0.smithi033.stdout: 1 615943 5.41 MB/sec execute 404 sec latency 4095.902 ms 2024-05-15T14:11:09.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.963+0000 7f76d2319640 1 -- 172.21.15.33:0/857603495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76cc102160 msgr2=0x7f76cc10eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:09.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.963+0000 7f76d2319640 1 --2- 172.21.15.33:0/857603495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76cc102160 0x7f76cc10eeb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f76c0009860 tx=0x7f76c002a340 comp rx=0 tx=0).stop 2024-05-15T14:11:09.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.963+0000 7f76d2319640 1 -- 172.21.15.33:0/857603495 shutdown_connections 2024-05-15T14:11:09.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.963+0000 7f76d2319640 1 --2- 172.21.15.33:0/857603495 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76cc10f3f0 0x7f76cc1117e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:09.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.963+0000 7f76d2319640 1 --2- 172.21.15.33:0/857603495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76cc102160 0x7f76cc10eeb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:09.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.963+0000 7f76d2319640 1 --2- 172.21.15.33:0/857603495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76cc101840 0x7f76cc101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:09.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.963+0000 7f76d2319640 1 -- 172.21.15.33:0/857603495 >> 172.21.15.33:0/857603495 conn(0x7f76cc078a70 msgr2=0x7f76cc078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:09.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.963+0000 7f76d2319640 1 -- 172.21.15.33:0/857603495 shutdown_connections 2024-05-15T14:11:09.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.963+0000 7f76d2319640 1 -- 172.21.15.33:0/857603495 wait complete. 2024-05-15T14:11:09.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.963+0000 7f76d2319640 1 Processor -- start 2024-05-15T14:11:09.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.963+0000 7f76d2319640 1 -- start start 2024-05-15T14:11:09.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.964+0000 7f76d2319640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76cc101840 0x7f76cc19b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:09.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.964+0000 7f76d2319640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76cc102160 0x7f76cc19b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:09.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.964+0000 7f76d2319640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76cc10f3f0 0x7f76cc19fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:09.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.964+0000 7f76d2319640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f76cc114f00 con 0x7f76cc10f3f0 2024-05-15T14:11:09.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.964+0000 7f76d2319640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f76cc114d80 con 0x7f76cc101840 2024-05-15T14:11:09.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.964+0000 7f76d2319640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f76cc115080 con 0x7f76cc102160 2024-05-15T14:11:09.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.964+0000 7f76d08b6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76cc10f3f0 0x7f76cc19fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:09.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.964+0000 7f76cbfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76cc101840 0x7f76cc19b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:09.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.964+0000 7f76cb7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76cc102160 0x7f76cc19b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:09.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.964+0000 7f76cbfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76cc101840 0x7f76cc19b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:52656/0 (socket says 172.21.15.33:52656) 2024-05-15T14:11:09.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.964+0000 7f76d08b6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76cc10f3f0 0x7f76cc19fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60278/0 (socket says 172.21.15.33:60278) 2024-05-15T14:11:09.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.964+0000 7f76cbfff640 1 -- 172.21.15.33:0/1286360628 learned_addr learned my addr 172.21.15.33:0/1286360628 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:11:09.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.964+0000 7f76cbfff640 1 -- 172.21.15.33:0/1286360628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76cc102160 msgr2=0x7f76cc19b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:09.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.964+0000 7f76cbfff640 1 --2- 172.21.15.33:0/1286360628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76cc102160 0x7f76cc19b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.964+0000 7f76cbfff640 1 -- 172.21.15.33:0/1286360628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76cc10f3f0 msgr2=0x7f76cc19fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.964+0000 7f76cbfff640 1 --2- 172.21.15.33:0/1286360628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76cc10f3f0 0x7f76cc19fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.964+0000 7f76cbfff640 1 -- 172.21.15.33:0/1286360628 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76cc1a0490 con 0x7f76cc101840 2024-05-15T14:11:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.964+0000 7f76cbfff640 1 --2- 172.21.15.33:0/1286360628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76cc101840 0x7f76cc19b4a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f76bc00e9c0 tx=0x7f76bc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.965+0000 7f76c97fa640 1 -- 172.21.15.33:0/1286360628 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76bc00ccc0 con 0x7f76cc101840 2024-05-15T14:11:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.965+0000 7f76c97fa640 1 -- 172.21.15.33:0/1286360628 <== mon.2 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76bc0054d0 con 0x7f76cc101840 2024-05-15T14:11:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.965+0000 7f76d2319640 1 -- 172.21.15.33:0/1286360628 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76cc1a0780 con 0x7f76cc101840 2024-05-15T14:11:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.965+0000 7f76c97fa640 1 -- 172.21.15.33:0/1286360628 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76bc010430 con 0x7f76cc101840 2024-05-15T14:11:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.965+0000 7f76d2319640 1 -- 172.21.15.33:0/1286360628 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76cc1a8010 con 0x7f76cc101840 2024-05-15T14:11:09.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.967+0000 7f76c97fa640 1 -- 172.21.15.33:0/1286360628 <== mon.2 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f76bc004f80 con 0x7f76cc101840 2024-05-15T14:11:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.968+0000 7f76d503c740 1 Processor -- start 2024-05-15T14:11:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.968+0000 7f76d503c740 1 -- start start 2024-05-15T14:11:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.968+0000 7f76d503c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593521528f0 0x559352152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.968+0000 7f76d503c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559352153210 0x55935215d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.968+0000 7f76d503c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55935215dad0 0x55935215fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.968+0000 7f76d503c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5593520c5c30 con 0x55935215dad0 2024-05-15T14:11:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.968+0000 7f76d503c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5593520b57d0 con 0x5593521528f0 2024-05-15T14:11:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.968+0000 7f76d503c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5593520b9810 con 0x559352153210 2024-05-15T14:11:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.968+0000 7f76cbfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559352153210 0x55935215d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.968+0000 7f76d08b6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593521528f0 0x559352152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.968+0000 7f76cb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55935215dad0 0x55935215fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.968+0000 7f76cbfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559352153210 0x55935215d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:60502/0 (socket says 172.21.15.33:60502) 2024-05-15T14:11:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.968+0000 7f76cbfff640 1 -- 172.21.15.33:0/1131339621 learned_addr learned my addr 172.21.15.33:0/1131339621 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:11:09.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.968+0000 7f76d08b6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593521528f0 0x559352152cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:52670/0 (socket says 172.21.15.33:52670) 2024-05-15T14:11:09.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.968+0000 7f76c97fa640 1 --2- 172.21.15.33:0/1286360628 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f76b40b29d0 0x7f76b40b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:09.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.968+0000 7f76cb7fe640 1 -- 172.21.15.33:0/1131339621 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559352153210 msgr2=0x55935215d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:09.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.968+0000 7f76cb7fe640 1 --2- 172.21.15.33:0/1131339621 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559352153210 0x55935215d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:09.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.968+0000 7f76cb7fe640 1 -- 172.21.15.33:0/1131339621 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593521528f0 msgr2=0x559352152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:09.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.968+0000 7f76cb7fe640 1 --2- 172.21.15.33:0/1131339621 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593521528f0 0x559352152cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:09.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.968+0000 7f76cb7fe640 1 -- 172.21.15.33:0/1131339621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559351dca210 con 0x55935215dad0 2024-05-15T14:11:09.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.968+0000 7f76d08b6640 1 --2- 172.21.15.33:0/1286360628 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f76b40b29d0 0x7f76b40b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:09.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.969+0000 7f76cb7fe640 1 --2- 172.21.15.33:0/1131339621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55935215dad0 0x55935215fec0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f76c0005940 tx=0x7f76c0005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=187e5d0fbe2a4a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:09.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.969+0000 7f76c97fa640 1 -- 172.21.15.33:0/1286360628 <== mon.2 v2:172.21.15.52:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8586+0+0 (secure 0 0 0) 0x7f76bc0e2990 con 0x7f76cc101840 2024-05-15T14:11:09.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.969+0000 7f76b27fc640 1 -- 172.21.15.33:0/1131339621 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0041070 con 0x55935215dad0 2024-05-15T14:11:09.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.969+0000 7f76d08b6640 1 --2- 172.21.15.33:0/1286360628 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f76b40b29d0 0x7f76b40b4e90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f76c4005c40 tx=0x7f76c4004a40 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:09.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.969+0000 7f76b27fc640 1 -- 172.21.15.33:0/1131339621 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76c00044d0 con 0x55935215dad0 2024-05-15T14:11:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.969+0000 7f76b27fc640 1 -- 172.21.15.33:0/1131339621 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0037040 con 0x55935215dad0 2024-05-15T14:11:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.970+0000 7f76d503c740 1 -- 172.21.15.33:0/1131339621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55935215dad0 msgr2=0x55935215fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.970+0000 7f76d503c740 1 --2- 172.21.15.33:0/1131339621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55935215dad0 0x55935215fec0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f76c0005940 tx=0x7f76c0005b10 comp rx=0 tx=0).stop 2024-05-15T14:11:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.970+0000 7f76d503c740 1 -- 172.21.15.33:0/1131339621 shutdown_connections 2024-05-15T14:11:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.970+0000 7f76d503c740 1 --2- 172.21.15.33:0/1131339621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55935215dad0 0x55935215fec0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.970+0000 7f76d503c740 1 --2- 172.21.15.33:0/1131339621 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559352153210 0x55935215d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.970+0000 7f76d503c740 1 --2- 172.21.15.33:0/1131339621 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593521528f0 0x559352152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.970+0000 7f76d503c740 1 -- 172.21.15.33:0/1131339621 >> 172.21.15.33:0/1131339621 conn(0x55935214ee00 msgr2=0x55935214f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.970+0000 7f76d503c740 1 -- 172.21.15.33:0/1131339621 shutdown_connections 2024-05-15T14:11:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.970+0000 7f76d503c740 1 -- 172.21.15.33:0/1131339621 wait complete. 2024-05-15T14:11:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.970+0000 7f76d503c740 1 -- start start 2024-05-15T14:11:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.970+0000 7f76d503c740 1 Processor -- start 2024-05-15T14:11:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.971+0000 7f76d503c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5593521528f0 0x55935215d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.971+0000 7f76d503c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559352153210 0x55935215a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.971+0000 7f76d503c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55935215dad0 0x55935215ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.971+0000 7f76d503c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559351d9b790 con 0x559352153210 2024-05-15T14:11:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.971+0000 7f76d503c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559352071890 con 0x55935215dad0 2024-05-15T14:11:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.971+0000 7f76d503c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x559352160740 con 0x5593521528f0 2024-05-15T14:11:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.971+0000 7f76d503c740 10 client.0 _ll_register_callbacks cb 0x559352074d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:11:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.971+0000 7f76cb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559352153210 0x55935215a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.971+0000 7f76cb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559352153210 0x55935215a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60298/0 (socket says 172.21.15.33:60298) 2024-05-15T14:11:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.971+0000 7f76cb7fe640 1 -- 172.21.15.33:0/4119174332 learned_addr learned my addr 172.21.15.33:0/4119174332 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:11:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.971+0000 7f76cbfff640 1 --2- 172.21.15.33:0/4119174332 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5593521528f0 0x55935215d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.971+0000 7f76d08b6640 1 --2- 172.21.15.33:0/4119174332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55935215dad0 0x55935215ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.971+0000 7f76cb7fe640 1 -- 172.21.15.33:0/4119174332 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5593521528f0 msgr2=0x55935215d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.971+0000 7f76cb7fe640 1 --2- 172.21.15.33:0/4119174332 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5593521528f0 0x55935215d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.971+0000 7f76cb7fe640 1 -- 172.21.15.33:0/4119174332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55935215dad0 msgr2=0x55935215ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.971+0000 7f76cb7fe640 1 --2- 172.21.15.33:0/4119174332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55935215dad0 0x55935215ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.971+0000 7f76cb7fe640 1 -- 172.21.15.33:0/4119174332 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559351e96620 con 0x559352153210 2024-05-15T14:11:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.971+0000 7f76cbfff640 1 --2- 172.21.15.33:0/4119174332 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5593521528f0 0x55935215d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:11:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.971+0000 7f76cb7fe640 1 --2- 172.21.15.33:0/4119174332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559352153210 0x55935215a730 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f76c002a930 tx=0x7f76c0004090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=791ec1eac021136a server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.971+0000 7f76ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:11:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.971+0000 7f76ab7fe640 1 -- 172.21.15.33:0/4119174332 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0037030 con 0x559352153210 2024-05-15T14:11:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.972+0000 7f76ab7fe640 1 -- 172.21.15.33:0/4119174332 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8586+0+0 (secure 0 0 0) 0x7f76c0030650 con 0x559352153210 2024-05-15T14:11:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.972+0000 7f76d503c740 1 -- 172.21.15.33:0/4119174332 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76c00044d0 con 0x559352153210 2024-05-15T14:11:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.972+0000 7f76ab7fe640 1 -- 172.21.15.33:0/4119174332 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f76c0053050 con 0x559352153210 2024-05-15T14:11:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.972+0000 7f76d503c740 10 client.16659 fetch_fsmap learned FSMap version 39 2024-05-15T14:11:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.972+0000 7f76d503c740 1 -- 172.21.15.33:0/4119174332 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55935216c160 con 0x559352153210 2024-05-15T14:11:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.972+0000 7f76ab7fe640 1 -- 172.21.15.33:0/4119174332 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76c00044d0 con 0x559352153210 2024-05-15T14:11:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.972+0000 7f76ab7fe640 1 -- 172.21.15.33:0/4119174332 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f76c0003a50 con 0x559352153210 2024-05-15T14:11:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.972+0000 7f76d503c740 10 client.16659 fetch_fsmap finished waiting for FSMap version 39 2024-05-15T14:11:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.972+0000 7f76d503c740 10 client.16659 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:11:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.972+0000 7f76d503c740 20 client.16659 populate_metadata read hostname 'smithi033' 2024-05-15T14:11:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.972+0000 7f76d503c740 1 --2- 172.21.15.33:0/4119174332 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55935216d4b0 0x55935216f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.972+0000 7f76d503c740 4 client.16659 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:11:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.972+0000 7f76d503c740 1 -- 172.21.15.33:0/4119174332 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593520c7b80 con 0x55935216d4b0 2024-05-15T14:11:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.973+0000 7f76cbfff640 1 --2- 172.21.15.33:0/4119174332 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55935216d4b0 0x55935216f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.973+0000 7f76cbfff640 1 --2- 172.21.15.33:0/4119174332 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55935216d4b0 0x55935216f8a0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc0350a00b3b02e3 server_cookie=58204ee481b71c8b in_seq=0 out_seq=0 2024-05-15T14:11:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.973+0000 7f76ab7fe640 10 client.16659 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:11:09.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.973+0000 7f76ab7fe640 1 -- 172.21.15.33:0/4119174332 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5593520c7b80 con 0x55935216d4b0 2024-05-15T14:11:09.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.973+0000 7f76ab7fe640 10 client.16659 handle_command_reply: tid=0 2024-05-15T14:11:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.974+0000 7f76d503c740 1 client.16659 shutdown 2024-05-15T14:11:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.974+0000 7f76d503c740 1 -- 172.21.15.33:0/4119174332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559352153210 msgr2=0x55935215a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.974+0000 7f76d503c740 1 --2- 172.21.15.33:0/4119174332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559352153210 0x55935215a730 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f76c002a930 tx=0x7f76c0004090 comp rx=0 tx=0).stop 2024-05-15T14:11:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.974+0000 7f76d503c740 1 -- 172.21.15.33:0/4119174332 shutdown_connections 2024-05-15T14:11:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.974+0000 7f76d503c740 1 --2- 172.21.15.33:0/4119174332 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55935216d4b0 0x55935216f8a0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.974+0000 7f76d503c740 1 --2- 172.21.15.33:0/4119174332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55935215dad0 0x55935215ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.974+0000 7f76d503c740 1 --2- 172.21.15.33:0/4119174332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559352153210 0x55935215a730 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.974+0000 7f76d503c740 1 --2- 172.21.15.33:0/4119174332 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5593521528f0 0x55935215d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.974+0000 7f76d503c740 1 -- 172.21.15.33:0/4119174332 >> 172.21.15.33:0/4119174332 conn(0x55935214ee00 msgr2=0x55935214e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.974+0000 7f76ab7fe640 0 client.16659 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:11:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.974+0000 7f76d503c740 1 -- 172.21.15.33:0/4119174332 shutdown_connections 2024-05-15T14:11:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.974+0000 7f76d503c740 1 -- 172.21.15.33:0/4119174332 wait complete. 2024-05-15T14:11:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.974+0000 7f76d503c740 20 client.16659 trim_cache size 0 max 16384 2024-05-15T14:11:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.990+0000 7f76d503c740 1 Processor -- start 2024-05-15T14:11:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.990+0000 7f76d503c740 1 -- start start 2024-05-15T14:11:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.990+0000 7f76d503c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593521528f0 0x559352174f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.991+0000 7f76d503c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559352153210 0x5593521754d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.991+0000 7f76d503c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55935215dad0 0x559352157990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.991+0000 7f76d503c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f76cc1b3360 con 0x559352153210 2024-05-15T14:11:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.991+0000 7f76d503c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f76cc1b0bd0 con 0x5593521528f0 2024-05-15T14:11:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.991+0000 7f76d503c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f76cc1aea40 con 0x55935215dad0 2024-05-15T14:11:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.991+0000 7f76cb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559352153210 0x5593521754d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.991+0000 7f76d08b6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55935215dad0 0x559352157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.991+0000 7f76cb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559352153210 0x5593521754d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60312/0 (socket says 172.21.15.33:60312) 2024-05-15T14:11:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.991+0000 7f76d08b6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55935215dad0 0x559352157990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:60522/0 (socket says 172.21.15.33:60522) 2024-05-15T14:11:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.991+0000 7f76cb7fe640 1 -- 172.21.15.33:0/2713406131 learned_addr learned my addr 172.21.15.33:0/2713406131 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:11:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.991+0000 7f76cbfff640 1 --2- 172.21.15.33:0/2713406131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593521528f0 0x559352174f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.991+0000 7f76cb7fe640 1 -- 172.21.15.33:0/2713406131 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55935215dad0 msgr2=0x559352157990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.991+0000 7f76cb7fe640 1 --2- 172.21.15.33:0/2713406131 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55935215dad0 0x559352157990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.991+0000 7f76cb7fe640 1 -- 172.21.15.33:0/2713406131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593521528f0 msgr2=0x559352174f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.991+0000 7f76cb7fe640 1 --2- 172.21.15.33:0/2713406131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593521528f0 0x559352174f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.991+0000 7f76cb7fe640 1 -- 172.21.15.33:0/2713406131 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55935214f9d0 con 0x559352153210 2024-05-15T14:11:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.991+0000 7f76d08b6640 1 --2- 172.21.15.33:0/2713406131 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55935215dad0 0x559352157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:11:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.991+0000 7f76cb7fe640 1 --2- 172.21.15.33:0/2713406131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559352153210 0x5593521754d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f76c0000c00 tx=0x7f76c0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9602a7e7b8b2a67b server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.991+0000 7f76b1ffb640 1 -- 172.21.15.33:0/2713406131 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0037070 con 0x559352153210 2024-05-15T14:11:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.991+0000 7f76b1ffb640 1 -- 172.21.15.33:0/2713406131 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76c0030900 con 0x559352153210 2024-05-15T14:11:09.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.991+0000 7f76b1ffb640 1 -- 172.21.15.33:0/2713406131 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0041040 con 0x559352153210 2024-05-15T14:11:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.992+0000 7f76d503c740 1 -- 172.21.15.33:0/2713406131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559352153210 msgr2=0x5593521754d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.992+0000 7f76d503c740 1 --2- 172.21.15.33:0/2713406131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559352153210 0x5593521754d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f76c0000c00 tx=0x7f76c0005f90 comp rx=0 tx=0).stop 2024-05-15T14:11:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.992+0000 7f76d503c740 1 -- 172.21.15.33:0/2713406131 shutdown_connections 2024-05-15T14:11:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.992+0000 7f76d503c740 1 --2- 172.21.15.33:0/2713406131 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55935215dad0 0x559352157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.992+0000 7f76d503c740 1 --2- 172.21.15.33:0/2713406131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559352153210 0x5593521754d0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.992+0000 7f76d503c740 1 --2- 172.21.15.33:0/2713406131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593521528f0 0x559352174f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.992+0000 7f76d503c740 1 -- 172.21.15.33:0/2713406131 >> 172.21.15.33:0/2713406131 conn(0x55935214ee00 msgr2=0x559352168600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.992+0000 7f76d503c740 1 -- 172.21.15.33:0/2713406131 shutdown_connections 2024-05-15T14:11:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.992+0000 7f76d503c740 1 -- 172.21.15.33:0/2713406131 wait complete. 2024-05-15T14:11:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.993+0000 7f76d503c740 1 -- start start 2024-05-15T14:11:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.993+0000 7f76d503c740 1 Processor -- start 2024-05-15T14:11:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.993+0000 7f76d503c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593521528f0 0x559352165f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.993+0000 7f76d503c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559352153210 0x5593521632d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.993+0000 7f76d503c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55935215dad0 0x559352163830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.993+0000 7f76d503c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f76cc1af090 con 0x559352153210 2024-05-15T14:11:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.993+0000 7f76d503c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f76cc1b0500 con 0x5593521528f0 2024-05-15T14:11:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.993+0000 7f76d503c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f76cc1af760 con 0x55935215dad0 2024-05-15T14:11:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.993+0000 7f76d503c740 10 client.0 _ll_register_callbacks cb 0x7f76c40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:11:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.993+0000 7f76cb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559352153210 0x5593521632d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.993+0000 7f76cbfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593521528f0 0x559352165f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.993+0000 7f76cbfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593521528f0 0x559352165f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:52686/0 (socket says 172.21.15.33:52686) 2024-05-15T14:11:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.993+0000 7f76cbfff640 1 -- 172.21.15.33:0/701371616 learned_addr learned my addr 172.21.15.33:0/701371616 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:11:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.993+0000 7f76d08b6640 1 --2- 172.21.15.33:0/701371616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55935215dad0 0x559352163830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.994+0000 7f76cb7fe640 1 -- 172.21.15.33:0/701371616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55935215dad0 msgr2=0x559352163830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.994+0000 7f76cb7fe640 1 --2- 172.21.15.33:0/701371616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55935215dad0 0x559352163830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.994+0000 7f76cb7fe640 1 -- 172.21.15.33:0/701371616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593521528f0 msgr2=0x559352165f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.994+0000 7f76cb7fe640 1 --2- 172.21.15.33:0/701371616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593521528f0 0x559352165f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.994+0000 7f76cb7fe640 1 -- 172.21.15.33:0/701371616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593521640b0 con 0x559352153210 2024-05-15T14:11:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.994+0000 7f76cbfff640 1 --2- 172.21.15.33:0/701371616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593521528f0 0x559352165f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:11:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.994+0000 7f76cb7fe640 1 --2- 172.21.15.33:0/701371616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559352153210 0x5593521632d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f76c002a860 tx=0x7f76c00056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a709986d31c88f09 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.994+0000 7f76ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:11:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.994+0000 7f76ab7fe640 1 -- 172.21.15.33:0/701371616 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c005b020 con 0x559352153210 2024-05-15T14:11:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.994+0000 7f76d503c740 1 -- 172.21.15.33:0/701371616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76c0030900 con 0x559352153210 2024-05-15T14:11:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.994+0000 7f76ab7fe640 1 -- 172.21.15.33:0/701371616 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8586+0+0 (secure 0 0 0) 0x7f76c00037f0 con 0x559352153210 2024-05-15T14:11:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.994+0000 7f76ab7fe640 1 -- 172.21.15.33:0/701371616 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f76c0066050 con 0x559352153210 2024-05-15T14:11:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.994+0000 7f76d503c740 10 client.16665 fetch_fsmap learned FSMap version 39 2024-05-15T14:11:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.994+0000 7f76d503c740 1 -- 172.21.15.33:0/701371616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5593521d7bd0 con 0x559352153210 2024-05-15T14:11:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.994+0000 7f76ab7fe640 1 -- 172.21.15.33:0/701371616 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76c0030900 con 0x559352153210 2024-05-15T14:11:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.995+0000 7f76ab7fe640 1 -- 172.21.15.33:0/701371616 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f76c0045440 con 0x559352153210 2024-05-15T14:11:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.995+0000 7f76d503c740 10 client.16665 fetch_fsmap finished waiting for FSMap version 39 2024-05-15T14:11:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.995+0000 7f76d503c740 10 client.16665 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:11:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.995+0000 7f76d503c740 20 client.16665 populate_metadata read hostname 'smithi033' 2024-05-15T14:11:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.995+0000 7f76d503c740 1 --2- 172.21.15.33:0/701371616 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5593521d8220 0x5593521da640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.995+0000 7f76d503c740 4 client.16665 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:11:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.995+0000 7f76d503c740 1 -- 172.21.15.33:0/701371616 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5593520c1360 con 0x5593521d8220 2024-05-15T14:11:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.995+0000 7f76cbfff640 1 --2- 172.21.15.33:0/701371616 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5593521d8220 0x5593521da640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.995+0000 7f76cbfff640 1 --2- 172.21.15.33:0/701371616 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5593521d8220 0x5593521da640 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61f6035634d3a596 server_cookie=c1c1efbb3fbf544a in_seq=0 out_seq=0 2024-05-15T14:11:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:09.995+0000 7f76ab7fe640 10 client.16665 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:11:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:10.031+0000 7f76ab7fe640 1 -- 172.21.15.33:0/701371616 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5593520c1360 con 0x5593521d8220 2024-05-15T14:11:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:10.031+0000 7f76ab7fe640 10 client.16665 handle_command_reply: tid=0 2024-05-15T14:11:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:10.031+0000 7f76d503c740 1 client.16665 shutdown 2024-05-15T14:11:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:10.031+0000 7f76d503c740 1 -- 172.21.15.33:0/701371616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559352153210 msgr2=0x5593521632d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:10.031+0000 7f76d503c740 1 --2- 172.21.15.33:0/701371616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559352153210 0x5593521632d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f76c002a860 tx=0x7f76c00056e0 comp rx=0 tx=0).stop 2024-05-15T14:11:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:10.031+0000 7f76d503c740 1 -- 172.21.15.33:0/701371616 shutdown_connections 2024-05-15T14:11:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:10.031+0000 7f76d503c740 1 --2- 172.21.15.33:0/701371616 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5593521d8220 0x5593521da640 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:10.032+0000 7f76d503c740 1 --2- 172.21.15.33:0/701371616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55935215dad0 0x559352163830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:10.032+0000 7f76ab7fe640 0 client.16665 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:11:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:10.032+0000 7f76d503c740 1 --2- 172.21.15.33:0/701371616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559352153210 0x5593521632d0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:10.032+0000 7f76d503c740 1 --2- 172.21.15.33:0/701371616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593521528f0 0x559352165f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:10.032+0000 7f76d503c740 1 -- 172.21.15.33:0/701371616 >> 172.21.15.33:0/701371616 conn(0x55935214ee00 msgr2=0x559352168e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:10.032+0000 7f76d503c740 1 -- 172.21.15.33:0/701371616 shutdown_connections 2024-05-15T14:11:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:10.032+0000 7f76d503c740 1 -- 172.21.15.33:0/701371616 wait complete. 2024-05-15T14:11:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:10.032+0000 7f76d503c740 20 client.16665 trim_cache size 0 max 16384 2024-05-15T14:11:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:11:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:11:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:11:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:10.032+0000 7f76d2319640 1 -- 172.21.15.33:0/1286360628 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f76b40b29d0 msgr2=0x7f76b40b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:10.032+0000 7f76d2319640 1 --2- 172.21.15.33:0/1286360628 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f76b40b29d0 0x7f76b40b4e90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f76c4005c40 tx=0x7f76c4004a40 comp rx=0 tx=0).stop 2024-05-15T14:11:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:10.032+0000 7f76d2319640 1 -- 172.21.15.33:0/1286360628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76cc101840 msgr2=0x7f76cc19b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:10.032+0000 7f76d2319640 1 --2- 172.21.15.33:0/1286360628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76cc101840 0x7f76cc19b4a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f76bc00e9c0 tx=0x7f76bc00ee80 comp rx=0 tx=0).stop 2024-05-15T14:11:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:10.033+0000 7f76d2319640 1 -- 172.21.15.33:0/1286360628 shutdown_connections 2024-05-15T14:11:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:10.033+0000 7f76d2319640 1 --2- 172.21.15.33:0/1286360628 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f76b40b29d0 0x7f76b40b4e90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:10.033+0000 7f76d2319640 1 --2- 172.21.15.33:0/1286360628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76cc10f3f0 0x7f76cc19fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:10.033+0000 7f76d2319640 1 --2- 172.21.15.33:0/1286360628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76cc102160 0x7f76cc19b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:10.033+0000 7f76d2319640 1 --2- 172.21.15.33:0/1286360628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f76cc101840 0x7f76cc19b4a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:10.033+0000 7f76d2319640 1 -- 172.21.15.33:0/1286360628 >> 172.21.15.33:0/1286360628 conn(0x7f76cc078a70 msgr2=0x7f76cc110a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:10.033+0000 7f76d2319640 1 -- 172.21.15.33:0/1286360628 shutdown_connections 2024-05-15T14:11:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:10.033+0000 7f76d2319640 1 -- 172.21.15.33:0/1286360628 wait complete. 2024-05-15T14:11:10.080 INFO:tasks.workunit.client.0.smithi033.stdout: 1 617211 5.41 MB/sec execute 405 sec latency 86.578 ms 2024-05-15T14:11:10.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:10 smithi033 ceph-mon[31650]: pgmap v338: 97 pgs: 97 active+clean; 195 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 1.0 MiB/s wr, 105 op/s 2024-05-15T14:11:10.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:10 smithi033 ceph-mon[31650]: osdmap e95: 12 total, 12 up, 12 in 2024-05-15T14:11:10.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:10 smithi052 ceph-mon[36190]: pgmap v338: 97 pgs: 97 active+clean; 195 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 1.0 MiB/s wr, 105 op/s 2024-05-15T14:11:10.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:10 smithi052 ceph-mon[36190]: osdmap e95: 12 total, 12 up, 12 in 2024-05-15T14:11:10.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:10 smithi081 ceph-mon[36434]: pgmap v338: 97 pgs: 97 active+clean; 195 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 1.0 MiB/s wr, 105 op/s 2024-05-15T14:11:10.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:10 smithi081 ceph-mon[36434]: osdmap e95: 12 total, 12 up, 12 in 2024-05-15T14:11:11.080 INFO:tasks.workunit.client.0.smithi033.stdout: 1 618773 5.42 MB/sec execute 406 sec latency 14.674 ms 2024-05-15T14:11:12.080 INFO:tasks.workunit.client.0.smithi033.stdout: 1 620580 5.42 MB/sec execute 407 sec latency 38.439 ms 2024-05-15T14:11:12.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:12 smithi052 ceph-mon[36190]: pgmap v340: 97 pgs: 97 active+clean; 195 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-05-15T14:11:12.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:12 smithi081 ceph-mon[36434]: pgmap v340: 97 pgs: 97 active+clean; 195 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-05-15T14:11:12.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:12 smithi033 ceph-mon[31650]: pgmap v340: 97 pgs: 97 active+clean; 195 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-05-15T14:11:13.080 INFO:tasks.workunit.client.0.smithi033.stdout: 1 622080 5.44 MB/sec execute 408 sec latency 18.663 ms 2024-05-15T14:11:14.080 INFO:tasks.workunit.client.0.smithi033.stdout: 1 623896 5.44 MB/sec execute 409 sec latency 14.264 ms 2024-05-15T14:11:14.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:14 smithi052 ceph-mon[36190]: pgmap v341: 97 pgs: 97 active+clean; 190 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 206 op/s 2024-05-15T14:11:14.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:14 smithi081 ceph-mon[36434]: pgmap v341: 97 pgs: 97 active+clean; 190 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 206 op/s 2024-05-15T14:11:14.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:14 smithi033 ceph-mon[31650]: pgmap v341: 97 pgs: 97 active+clean; 190 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 206 op/s 2024-05-15T14:11:15.080 INFO:tasks.workunit.client.0.smithi033.stdout: 1 625482 5.46 MB/sec execute 410 sec latency 10.238 ms 2024-05-15T14:11:16.080 INFO:tasks.workunit.client.0.smithi033.stdout: 1 626780 5.46 MB/sec execute 411 sec latency 21.382 ms 2024-05-15T14:11:16.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:16 smithi052 ceph-mon[36190]: pgmap v342: 97 pgs: 97 active+clean; 190 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 206 op/s 2024-05-15T14:11:16.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:16 smithi052 ceph-mon[36190]: osdmap e96: 12 total, 12 up, 12 in 2024-05-15T14:11:16.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:16 smithi081 ceph-mon[36434]: pgmap v342: 97 pgs: 97 active+clean; 190 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 206 op/s 2024-05-15T14:11:16.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:16 smithi081 ceph-mon[36434]: osdmap e96: 12 total, 12 up, 12 in 2024-05-15T14:11:16.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:16 smithi033 ceph-mon[31650]: pgmap v342: 97 pgs: 97 active+clean; 190 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 206 op/s 2024-05-15T14:11:16.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:16 smithi033 ceph-mon[31650]: osdmap e96: 12 total, 12 up, 12 in 2024-05-15T14:11:17.080 INFO:tasks.workunit.client.0.smithi033.stdout: 1 628853 5.48 MB/sec execute 412 sec latency 28.056 ms 2024-05-15T14:11:18.081 INFO:tasks.workunit.client.0.smithi033.stdout: 1 630123 5.48 MB/sec execute 413 sec latency 23.417 ms 2024-05-15T14:11:18.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:18 smithi052 ceph-mon[36190]: pgmap v344: 97 pgs: 97 active+clean; 195 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.7 MiB/s wr, 381 op/s 2024-05-15T14:11:18.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:18 smithi081 ceph-mon[36434]: pgmap v344: 97 pgs: 97 active+clean; 195 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.7 MiB/s wr, 381 op/s 2024-05-15T14:11:18.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:18 smithi033 ceph-mon[31650]: pgmap v344: 97 pgs: 97 active+clean; 195 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.7 MiB/s wr, 381 op/s 2024-05-15T14:11:19.080 INFO:tasks.workunit.client.0.smithi033.stdout: 1 632226 5.49 MB/sec execute 414 sec latency 24.858 ms 2024-05-15T14:11:20.080 INFO:tasks.workunit.client.0.smithi033.stdout: 1 633146 5.49 MB/sec execute 415 sec latency 16.108 ms 2024-05-15T14:11:20.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:20 smithi052 ceph-mon[36190]: pgmap v345: 97 pgs: 97 active+clean; 195 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 308 op/s 2024-05-15T14:11:20.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:20 smithi081 ceph-mon[36434]: pgmap v345: 97 pgs: 97 active+clean; 195 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 308 op/s 2024-05-15T14:11:20.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:20 smithi033 ceph-mon[31650]: pgmap v345: 97 pgs: 97 active+clean; 195 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 308 op/s 2024-05-15T14:11:21.082 INFO:tasks.workunit.client.0.smithi033.stdout: 1 635065 5.50 MB/sec execute 416 sec latency 23.660 ms 2024-05-15T14:11:22.080 INFO:tasks.workunit.client.0.smithi033.stdout: 1 636298 5.50 MB/sec execute 417 sec latency 45.450 ms 2024-05-15T14:11:22.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:22 smithi052 ceph-mon[36190]: pgmap v346: 97 pgs: 97 active+clean; 195 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 305 op/s 2024-05-15T14:11:22.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:22 smithi081 ceph-mon[36434]: pgmap v346: 97 pgs: 97 active+clean; 195 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 305 op/s 2024-05-15T14:11:22.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:22 smithi033 ceph-mon[31650]: pgmap v346: 97 pgs: 97 active+clean; 195 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 305 op/s 2024-05-15T14:11:23.080 INFO:tasks.workunit.client.0.smithi033.stdout: 1 638145 5.51 MB/sec execute 418 sec latency 39.864 ms 2024-05-15T14:11:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.129+0000 7f1ee9319640 1 -- 172.21.15.33:0/1015423542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee4103d20 msgr2=0x7f1ee4104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.129+0000 7f1ee9319640 1 --2- 172.21.15.33:0/1015423542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee4103d20 0x7f1ee4104100 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4009900 tx=0x7f1ed402a480 comp rx=0 tx=0).stop 2024-05-15T14:11:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.129+0000 7f1ee9319640 1 -- 172.21.15.33:0/1015423542 shutdown_connections 2024-05-15T14:11:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.129+0000 7f1ee9319640 1 --2- 172.21.15.33:0/1015423542 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ee410d060 0x7f1ee410f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.129+0000 7f1ee9319640 1 --2- 172.21.15.33:0/1015423542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ee4104640 0x7f1ee410cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.129+0000 7f1ee9319640 1 --2- 172.21.15.33:0/1015423542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee4103d20 0x7f1ee4104100 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.129+0000 7f1ee9319640 1 -- 172.21.15.33:0/1015423542 >> 172.21.15.33:0/1015423542 conn(0x7f1ee4078aa0 msgr2=0x7f1ee4078eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.129+0000 7f1ee9319640 1 -- 172.21.15.33:0/1015423542 shutdown_connections 2024-05-15T14:11:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.129+0000 7f1ee9319640 1 -- 172.21.15.33:0/1015423542 wait complete. 2024-05-15T14:11:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.130+0000 7f1ee9319640 1 Processor -- start 2024-05-15T14:11:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.130+0000 7f1ee9319640 1 -- start start 2024-05-15T14:11:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.130+0000 7f1ee9319640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ee4103d20 0x7f1ee4101970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.130+0000 7f1ee9319640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ee4104640 0x7f1ee4101eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.130+0000 7f1ee9319640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee410d060 0x7f1ee41023f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.130+0000 7f1ee9319640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee4112cf0 con 0x7f1ee410d060 2024-05-15T14:11:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.130+0000 7f1ee9319640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee4112b70 con 0x7f1ee4103d20 2024-05-15T14:11:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.130+0000 7f1ee9319640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee4112e70 con 0x7f1ee4104640 2024-05-15T14:11:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.130+0000 7f1ee37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee410d060 0x7f1ee41023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.130+0000 7f1ee27fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ee4104640 0x7f1ee4101eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:23.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.130+0000 7f1ee2ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ee4103d20 0x7f1ee4101970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:23.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.130+0000 7f1ee37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee410d060 0x7f1ee41023f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49072/0 (socket says 172.21.15.33:49072) 2024-05-15T14:11:23.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.130+0000 7f1ee37fe640 1 -- 172.21.15.33:0/83445512 learned_addr learned my addr 172.21.15.33:0/83445512 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:11:23.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.131+0000 7f1ee2ffd640 1 -- 172.21.15.33:0/83445512 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ee4104640 msgr2=0x7f1ee4101eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:23.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.131+0000 7f1ee2ffd640 1 --2- 172.21.15.33:0/83445512 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ee4104640 0x7f1ee4101eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:23.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.131+0000 7f1ee2ffd640 1 -- 172.21.15.33:0/83445512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee410d060 msgr2=0x7f1ee41023f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:23.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.131+0000 7f1ee2ffd640 1 --2- 172.21.15.33:0/83445512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee410d060 0x7f1ee41023f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:23.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.131+0000 7f1ee2ffd640 1 -- 172.21.15.33:0/83445512 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ee41a7730 con 0x7f1ee4103d20 2024-05-15T14:11:23.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.131+0000 7f1ee27fc640 1 --2- 172.21.15.33:0/83445512 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ee4104640 0x7f1ee4101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:11:23.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.131+0000 7f1ee37fe640 1 --2- 172.21.15.33:0/83445512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee410d060 0x7f1ee41023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:11:23.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.131+0000 7f1ee2ffd640 1 --2- 172.21.15.33:0/83445512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ee4103d20 0x7f1ee4101970 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1ed402a9a0 tx=0x7f1ed4031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:23.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.131+0000 7f1ecbfff640 1 -- 172.21.15.33:0/83445512 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed403f030 con 0x7f1ee4103d20 2024-05-15T14:11:23.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.131+0000 7f1ee9319640 1 -- 172.21.15.33:0/83445512 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ee41a79c0 con 0x7f1ee4103d20 2024-05-15T14:11:23.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.131+0000 7f1ecbfff640 1 -- 172.21.15.33:0/83445512 <== mon.2 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ed4004490 con 0x7f1ee4103d20 2024-05-15T14:11:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.131+0000 7f1ecbfff640 1 -- 172.21.15.33:0/83445512 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed4035070 con 0x7f1ee4103d20 2024-05-15T14:11:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.131+0000 7f1ee9319640 1 -- 172.21.15.33:0/83445512 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ee41a7eb0 con 0x7f1ee4103d20 2024-05-15T14:11:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.134+0000 7f1ecbfff640 1 -- 172.21.15.33:0/83445512 <== mon.2 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1ed4042030 con 0x7f1ee4103d20 2024-05-15T14:11:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.134+0000 7f1eec037740 1 Processor -- start 2024-05-15T14:11:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.134+0000 7f1eec037740 1 -- start start 2024-05-15T14:11:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.134+0000 7f1eec037740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5600d0bc08f0 0x5600d0bc0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.134+0000 7f1eec037740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600d0bc1210 0x5600d0bcb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.134+0000 7f1eec037740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600d0bcbad0 0x5600d0bcdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.134+0000 7f1eec037740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5600d0b33c30 con 0x5600d0bcbad0 2024-05-15T14:11:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.134+0000 7f1eec037740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5600d0b237d0 con 0x5600d0bc1210 2024-05-15T14:11:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.134+0000 7f1eec037740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5600d0b27810 con 0x5600d0bc08f0 2024-05-15T14:11:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.134+0000 7f1ee37fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5600d0bc08f0 0x5600d0bc0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.134+0000 7f1ee2ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600d0bc1210 0x5600d0bcb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.134+0000 7f1ee27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600d0bcbad0 0x5600d0bcdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.135+0000 7f1ee37fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5600d0bc08f0 0x5600d0bc0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:45386/0 (socket says 172.21.15.33:45386) 2024-05-15T14:11:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.135+0000 7f1ee2ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600d0bc1210 0x5600d0bcb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:36660/0 (socket says 172.21.15.33:36660) 2024-05-15T14:11:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.135+0000 7f1ee27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600d0bcbad0 0x5600d0bcdec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49088/0 (socket says 172.21.15.33:49088) 2024-05-15T14:11:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.135+0000 7f1ee37fe640 1 -- 172.21.15.33:0/1842418983 learned_addr learned my addr 172.21.15.33:0/1842418983 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:11:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.135+0000 7f1ecbfff640 1 --2- 172.21.15.33:0/83445512 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f1ecc0b2980 0x7f1ecc0b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.135+0000 7f1ee37fe640 1 -- 172.21.15.33:0/1842418983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600d0bc1210 msgr2=0x5600d0bcb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.135+0000 7f1ee37fe640 1 --2- 172.21.15.33:0/1842418983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600d0bc1210 0x5600d0bcb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.135+0000 7f1ee37fe640 1 -- 172.21.15.33:0/1842418983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600d0bcbad0 msgr2=0x5600d0bcdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.135+0000 7f1ee37fe640 1 --2- 172.21.15.33:0/1842418983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600d0bcbad0 0x5600d0bcdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.135+0000 7f1ee37fe640 1 -- 172.21.15.33:0/1842418983 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600d0838210 con 0x5600d0bc08f0 2024-05-15T14:11:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.135+0000 7f1ee27fc640 1 --2- 172.21.15.33:0/1842418983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600d0bcbad0 0x5600d0bcdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:11:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.135+0000 7f1ee37fe640 1 --2- 172.21.15.33:0/83445512 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f1ecc0b2980 0x7f1ecc0b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.135+0000 7f1ecbfff640 1 -- 172.21.15.33:0/83445512 <== mon.2 v2:172.21.15.52:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7f1ed4102820 con 0x7f1ee4103d20 2024-05-15T14:11:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.135+0000 7f1ee37fe640 1 --2- 172.21.15.33:0/1842418983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5600d0bc08f0 0x5600d0bc0cd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1edc0119d0 tx=0x7f1edc011e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc545cdb1beafeea server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.135+0000 7f1ec97fa640 1 -- 172.21.15.33:0/1842418983 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1edc017070 con 0x5600d0bc08f0 2024-05-15T14:11:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.135+0000 7f1ee37fe640 1 --2- 172.21.15.33:0/83445512 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f1ecc0b2980 0x7f1ecc0b4e40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1edc023000 tx=0x7f1edc024000 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.135+0000 7f1ec97fa640 1 -- 172.21.15.33:0/1842418983 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1edc0108e0 con 0x5600d0bc08f0 2024-05-15T14:11:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.135+0000 7f1ec97fa640 1 -- 172.21.15.33:0/1842418983 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1edc00e610 con 0x5600d0bc08f0 2024-05-15T14:11:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.136+0000 7f1eec037740 1 -- 172.21.15.33:0/1842418983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5600d0bc08f0 msgr2=0x5600d0bc0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.136+0000 7f1eec037740 1 --2- 172.21.15.33:0/1842418983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5600d0bc08f0 0x5600d0bc0cd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1edc0119d0 tx=0x7f1edc011e90 comp rx=0 tx=0).stop 2024-05-15T14:11:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.136+0000 7f1eec037740 1 -- 172.21.15.33:0/1842418983 shutdown_connections 2024-05-15T14:11:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.136+0000 7f1eec037740 1 --2- 172.21.15.33:0/1842418983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600d0bcbad0 0x5600d0bcdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.136+0000 7f1eec037740 1 --2- 172.21.15.33:0/1842418983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600d0bc1210 0x5600d0bcb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.136+0000 7f1eec037740 1 --2- 172.21.15.33:0/1842418983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5600d0bc08f0 0x5600d0bc0cd0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.136+0000 7f1eec037740 1 -- 172.21.15.33:0/1842418983 >> 172.21.15.33:0/1842418983 conn(0x5600d0bbce00 msgr2=0x5600d0bbd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.136+0000 7f1eec037740 1 -- 172.21.15.33:0/1842418983 shutdown_connections 2024-05-15T14:11:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.136+0000 7f1eec037740 1 -- 172.21.15.33:0/1842418983 wait complete. 2024-05-15T14:11:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.136+0000 7f1eec037740 1 -- start start 2024-05-15T14:11:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.137+0000 7f1eec037740 1 Processor -- start 2024-05-15T14:11:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.137+0000 7f1eec037740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5600d0bc08f0 0x5600d0bcb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.137+0000 7f1eec037740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600d0bc1210 0x5600d0bc8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.137+0000 7f1eec037740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600d0bcbad0 0x5600d0bc8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.137+0000 7f1eec037740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5600d0809790 con 0x5600d0bcbad0 2024-05-15T14:11:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.137+0000 7f1eec037740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5600d0adf890 con 0x5600d0bc1210 2024-05-15T14:11:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.137+0000 7f1eec037740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5600d0bce7a0 con 0x5600d0bc08f0 2024-05-15T14:11:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.137+0000 7f1eec037740 10 client.0 _ll_register_callbacks cb 0x5600d0ae2d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:11:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.137+0000 7f1ee2ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5600d0bc08f0 0x5600d0bcb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.137+0000 7f1ee37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600d0bcbad0 0x5600d0bc8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.137+0000 7f1ee27fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600d0bc1210 0x5600d0bc8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.137+0000 7f1ee2ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5600d0bc08f0 0x5600d0bcb460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:45402/0 (socket says 172.21.15.33:45402) 2024-05-15T14:11:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.137+0000 7f1ee2ffd640 1 -- 172.21.15.33:0/3628143428 learned_addr learned my addr 172.21.15.33:0/3628143428 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:11:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.137+0000 7f1ee37fe640 1 -- 172.21.15.33:0/3628143428 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5600d0bc08f0 msgr2=0x5600d0bcb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.137+0000 7f1ee37fe640 1 --2- 172.21.15.33:0/3628143428 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5600d0bc08f0 0x5600d0bcb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.137+0000 7f1ee37fe640 1 -- 172.21.15.33:0/3628143428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600d0bc1210 msgr2=0x5600d0bc8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.137+0000 7f1ee37fe640 1 --2- 172.21.15.33:0/3628143428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600d0bc1210 0x5600d0bc8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.137+0000 7f1ee37fe640 1 -- 172.21.15.33:0/3628143428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600d0904620 con 0x5600d0bcbad0 2024-05-15T14:11:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.137+0000 7f1ee2ffd640 1 --2- 172.21.15.33:0/3628143428 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5600d0bc08f0 0x5600d0bcb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:11:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.137+0000 7f1ee37fe640 1 --2- 172.21.15.33:0/3628143428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600d0bcbad0 0x5600d0bc8cd0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1edc0049d0 tx=0x7f1edc00ff00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698941443e145e27 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.138+0000 7f1ec6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:11:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.138+0000 7f1ec6ff5640 1 -- 172.21.15.33:0/3628143428 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1edc017040 con 0x5600d0bcbad0 2024-05-15T14:11:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.138+0000 7f1ec6ff5640 1 -- 172.21.15.33:0/3628143428 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7f1edc00fc40 con 0x5600d0bcbad0 2024-05-15T14:11:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.138+0000 7f1eec037740 1 -- 172.21.15.33:0/3628143428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1edc0108e0 con 0x5600d0bcbad0 2024-05-15T14:11:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.138+0000 7f1ec6ff5640 1 -- 172.21.15.33:0/3628143428 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1edc004c50 con 0x5600d0bcbad0 2024-05-15T14:11:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.138+0000 7f1eec037740 10 client.16677 fetch_fsmap learned FSMap version 39 2024-05-15T14:11:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.138+0000 7f1eec037740 1 -- 172.21.15.33:0/3628143428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5600d0bda1e0 con 0x5600d0bcbad0 2024-05-15T14:11:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.138+0000 7f1ec6ff5640 1 -- 172.21.15.33:0/3628143428 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1edc0108e0 con 0x5600d0bcbad0 2024-05-15T14:11:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.138+0000 7f1ec6ff5640 1 -- 172.21.15.33:0/3628143428 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f1edc01b070 con 0x5600d0bcbad0 2024-05-15T14:11:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.138+0000 7f1eec037740 10 client.16677 fetch_fsmap finished waiting for FSMap version 39 2024-05-15T14:11:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.138+0000 7f1eec037740 10 client.16677 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:11:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.138+0000 7f1eec037740 20 client.16677 populate_metadata read hostname 'smithi033' 2024-05-15T14:11:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.138+0000 7f1eec037740 1 --2- 172.21.15.33:0/3628143428 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5600d0bdb640 0x5600d0bdda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.138+0000 7f1eec037740 4 client.16677 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:11:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.138+0000 7f1eec037740 1 -- 172.21.15.33:0/3628143428 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600d0b35b80 con 0x5600d0bdb640 2024-05-15T14:11:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.139+0000 7f1ee2ffd640 1 --2- 172.21.15.33:0/3628143428 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5600d0bdb640 0x5600d0bdda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.139+0000 7f1ec6ff5640 10 client.16677 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:11:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.139+0000 7f1ee2ffd640 1 --2- 172.21.15.33:0/3628143428 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5600d0bdb640 0x5600d0bdda30 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd838f63da6b129d server_cookie=bf20b6757cd5afb4 in_seq=0 out_seq=0 2024-05-15T14:11:23.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.140+0000 7f1ec6ff5640 1 -- 172.21.15.33:0/3628143428 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5600d0b35b80 con 0x5600d0bdb640 2024-05-15T14:11:23.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.140+0000 7f1ec6ff5640 10 client.16677 handle_command_reply: tid=0 2024-05-15T14:11:23.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.140+0000 7f1eec037740 1 client.16677 shutdown 2024-05-15T14:11:23.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.140+0000 7f1eec037740 1 -- 172.21.15.33:0/3628143428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600d0bcbad0 msgr2=0x5600d0bc8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:23.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.140+0000 7f1eec037740 1 --2- 172.21.15.33:0/3628143428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600d0bcbad0 0x5600d0bc8cd0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1edc0049d0 tx=0x7f1edc00ff00 comp rx=0 tx=0).stop 2024-05-15T14:11:23.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.140+0000 7f1eec037740 1 -- 172.21.15.33:0/3628143428 shutdown_connections 2024-05-15T14:11:23.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.140+0000 7f1eec037740 1 --2- 172.21.15.33:0/3628143428 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5600d0bdb640 0x5600d0bdda30 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:23.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.140+0000 7f1eec037740 1 --2- 172.21.15.33:0/3628143428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600d0bcbad0 0x5600d0bc8cd0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:23.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.140+0000 7f1eec037740 1 --2- 172.21.15.33:0/3628143428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600d0bc1210 0x5600d0bc8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:23.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.140+0000 7f1ec6ff5640 0 client.16677 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:11:23.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.140+0000 7f1eec037740 1 --2- 172.21.15.33:0/3628143428 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5600d0bc08f0 0x5600d0bcb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:23.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.140+0000 7f1eec037740 1 -- 172.21.15.33:0/3628143428 >> 172.21.15.33:0/3628143428 conn(0x5600d0bbce00 msgr2=0x5600d0bbc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:23.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.140+0000 7f1eec037740 1 -- 172.21.15.33:0/3628143428 shutdown_connections 2024-05-15T14:11:23.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.140+0000 7f1eec037740 1 -- 172.21.15.33:0/3628143428 wait complete. 2024-05-15T14:11:23.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.140+0000 7f1eec037740 20 client.16677 trim_cache size 0 max 16384 2024-05-15T14:11:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.158+0000 7f1eec037740 1 Processor -- start 2024-05-15T14:11:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.158+0000 7f1eec037740 1 -- start start 2024-05-15T14:11:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.158+0000 7f1eec037740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600d0bc08f0 0x5600d0bd7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.158+0000 7f1eec037740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5600d0bc1210 0x5600d0bcaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.158+0000 7f1eec037740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600d0bcbad0 0x5600d0be3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.158+0000 7f1eec037740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee41b32d0 con 0x5600d0bc08f0 2024-05-15T14:11:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.158+0000 7f1eec037740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee41b0b40 con 0x5600d0bcbad0 2024-05-15T14:11:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.158+0000 7f1eec037740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee41ae9b0 con 0x5600d0bc1210 2024-05-15T14:11:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.158+0000 7f1ee2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600d0bc08f0 0x5600d0bd7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.158+0000 7f1ee2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600d0bc08f0 0x5600d0bd7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49116/0 (socket says 172.21.15.33:49116) 2024-05-15T14:11:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.158+0000 7f1ee2ffd640 1 -- 172.21.15.33:0/2851169204 learned_addr learned my addr 172.21.15.33:0/2851169204 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:11:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.158+0000 7f1ee27fc640 1 --2- 172.21.15.33:0/2851169204 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5600d0bc1210 0x5600d0bcaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.158+0000 7f1ee37fe640 1 --2- 172.21.15.33:0/2851169204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600d0bcbad0 0x5600d0be3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.158+0000 7f1ee2ffd640 1 -- 172.21.15.33:0/2851169204 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5600d0bc1210 msgr2=0x5600d0bcaf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.158+0000 7f1ee2ffd640 1 --2- 172.21.15.33:0/2851169204 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5600d0bc1210 0x5600d0bcaf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.158+0000 7f1ee2ffd640 1 -- 172.21.15.33:0/2851169204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600d0bcbad0 msgr2=0x5600d0be3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.158+0000 7f1ee2ffd640 1 --2- 172.21.15.33:0/2851169204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600d0bcbad0 0x5600d0be3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.158+0000 7f1ee2ffd640 1 -- 172.21.15.33:0/2851169204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600d0bbd9d0 con 0x5600d0bc08f0 2024-05-15T14:11:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.158+0000 7f1ee2ffd640 1 --2- 172.21.15.33:0/2851169204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600d0bc08f0 0x5600d0bd7670 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1ed40c98a0 tx=0x7f1ed4104d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fde182d5c93ae96 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.159+0000 7f1ec8ff9640 1 -- 172.21.15.33:0/2851169204 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed4108040 con 0x5600d0bc08f0 2024-05-15T14:11:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.159+0000 7f1ec8ff9640 1 -- 172.21.15.33:0/2851169204 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ed40d2290 con 0x5600d0bc08f0 2024-05-15T14:11:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.159+0000 7f1ec8ff9640 1 -- 172.21.15.33:0/2851169204 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed4049040 con 0x5600d0bc08f0 2024-05-15T14:11:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.159+0000 7f1eec037740 1 -- 172.21.15.33:0/2851169204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600d0bc08f0 msgr2=0x5600d0bd7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.159+0000 7f1eec037740 1 --2- 172.21.15.33:0/2851169204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600d0bc08f0 0x5600d0bd7670 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1ed40c98a0 tx=0x7f1ed4104d30 comp rx=0 tx=0).stop 2024-05-15T14:11:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.159+0000 7f1eec037740 1 -- 172.21.15.33:0/2851169204 shutdown_connections 2024-05-15T14:11:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.159+0000 7f1eec037740 1 --2- 172.21.15.33:0/2851169204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600d0bcbad0 0x5600d0be3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.159+0000 7f1eec037740 1 --2- 172.21.15.33:0/2851169204 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5600d0bc1210 0x5600d0bcaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.159+0000 7f1eec037740 1 --2- 172.21.15.33:0/2851169204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600d0bc08f0 0x5600d0bd7670 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.159+0000 7f1eec037740 1 -- 172.21.15.33:0/2851169204 >> 172.21.15.33:0/2851169204 conn(0x5600d0bbce00 msgr2=0x5600d0bc2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.159+0000 7f1eec037740 1 -- 172.21.15.33:0/2851169204 shutdown_connections 2024-05-15T14:11:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.159+0000 7f1eec037740 1 -- 172.21.15.33:0/2851169204 wait complete. 2024-05-15T14:11:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.160+0000 7f1eec037740 1 -- start start 2024-05-15T14:11:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.160+0000 7f1eec037740 1 Processor -- start 2024-05-15T14:11:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.160+0000 7f1eec037740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600d0bc08f0 0x5600d0bcedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.160+0000 7f1eec037740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600d0bc1210 0x5600d0bcf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.160+0000 7f1eec037740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5600d0bcbad0 0x5600d0c3dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.160+0000 7f1eec037740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee41af000 con 0x5600d0bc1210 2024-05-15T14:11:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.160+0000 7f1eec037740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee41b0470 con 0x5600d0bc08f0 2024-05-15T14:11:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.160+0000 7f1eec037740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee41af6d0 con 0x5600d0bcbad0 2024-05-15T14:11:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.160+0000 7f1eec037740 10 client.0 _ll_register_callbacks cb 0x7f1ed40c6270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:11:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.160+0000 7f1ee27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600d0bc1210 0x5600d0bcf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.160+0000 7f1ee37fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5600d0bcbad0 0x5600d0c3dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.160+0000 7f1ee27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600d0bc1210 0x5600d0bcf300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49120/0 (socket says 172.21.15.33:49120) 2024-05-15T14:11:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.160+0000 7f1ee27fc640 1 -- 172.21.15.33:0/3622582835 learned_addr learned my addr 172.21.15.33:0/3622582835 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:11:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.160+0000 7f1ee2ffd640 1 --2- 172.21.15.33:0/3622582835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600d0bc08f0 0x5600d0bcedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.160+0000 7f1ee27fc640 1 -- 172.21.15.33:0/3622582835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5600d0bcbad0 msgr2=0x5600d0c3dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.160+0000 7f1ee27fc640 1 --2- 172.21.15.33:0/3622582835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5600d0bcbad0 0x5600d0c3dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.160+0000 7f1ee27fc640 1 -- 172.21.15.33:0/3622582835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600d0bc08f0 msgr2=0x5600d0bcedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.160+0000 7f1ee27fc640 1 --2- 172.21.15.33:0/3622582835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600d0bc08f0 0x5600d0bcedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.160+0000 7f1ee27fc640 1 -- 172.21.15.33:0/3622582835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600d0c3e3e0 con 0x5600d0bc1210 2024-05-15T14:11:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.161+0000 7f1ee27fc640 1 --2- 172.21.15.33:0/3622582835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600d0bc1210 0x5600d0bcf300 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x5600d0bbdf90 tx=0x7f1ed800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfae2faf2ac3bb1a server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.161+0000 7f1ec6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:11:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.161+0000 7f1ec6ff5640 1 -- 172.21.15.33:0/3622582835 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed8023060 con 0x5600d0bc1210 2024-05-15T14:11:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.161+0000 7f1ec6ff5640 1 -- 172.21.15.33:0/3622582835 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7f1ed8004250 con 0x5600d0bc1210 2024-05-15T14:11:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.161+0000 7f1eec037740 1 -- 172.21.15.33:0/3622582835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5600d0b2f360 con 0x5600d0bc1210 2024-05-15T14:11:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.161+0000 7f1ec6ff5640 1 -- 172.21.15.33:0/3622582835 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1ed8004450 con 0x5600d0bc1210 2024-05-15T14:11:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.161+0000 7f1eec037740 10 client.16683 fetch_fsmap learned FSMap version 39 2024-05-15T14:11:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.161+0000 7f1eec037740 1 -- 172.21.15.33:0/3622582835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5600d0c45830 con 0x5600d0bc1210 2024-05-15T14:11:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.161+0000 7f1ec6ff5640 1 -- 172.21.15.33:0/3622582835 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5600d0b2f360 con 0x5600d0bc1210 2024-05-15T14:11:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.161+0000 7f1ec6ff5640 1 -- 172.21.15.33:0/3622582835 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f1ed80105c0 con 0x5600d0bc1210 2024-05-15T14:11:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.161+0000 7f1eec037740 10 client.16683 fetch_fsmap finished waiting for FSMap version 39 2024-05-15T14:11:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.161+0000 7f1eec037740 10 client.16683 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:11:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.161+0000 7f1eec037740 20 client.16683 populate_metadata read hostname 'smithi033' 2024-05-15T14:11:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.161+0000 7f1eec037740 1 --2- 172.21.15.33:0/3622582835 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5600d0c46070 0x5600d0c48490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.161+0000 7f1eec037740 4 client.16683 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:11:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.161+0000 7f1eec037740 1 -- 172.21.15.33:0/3622582835 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5600d0aa95f0 con 0x5600d0c46070 2024-05-15T14:11:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.162+0000 7f1ee2ffd640 1 --2- 172.21.15.33:0/3622582835 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5600d0c46070 0x5600d0c48490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.162+0000 7f1ee2ffd640 1 --2- 172.21.15.33:0/3622582835 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5600d0c46070 0x5600d0c48490 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e360fa6e5a5691a server_cookie=4e999ab3d2922b41 in_seq=0 out_seq=0 2024-05-15T14:11:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.162+0000 7f1ec6ff5640 10 client.16683 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:11:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.213+0000 7f1ec6ff5640 1 -- 172.21.15.33:0/3622582835 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5600d0aa95f0 con 0x5600d0c46070 2024-05-15T14:11:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.213+0000 7f1ec6ff5640 10 client.16683 handle_command_reply: tid=0 2024-05-15T14:11:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.213+0000 7f1eec037740 1 client.16683 shutdown 2024-05-15T14:11:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.213+0000 7f1eec037740 1 -- 172.21.15.33:0/3622582835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600d0bc1210 msgr2=0x5600d0bcf300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.213+0000 7f1eec037740 1 --2- 172.21.15.33:0/3622582835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600d0bc1210 0x5600d0bcf300 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x5600d0bbdf90 tx=0x7f1ed800f790 comp rx=0 tx=0).stop 2024-05-15T14:11:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.213+0000 7f1eec037740 1 -- 172.21.15.33:0/3622582835 shutdown_connections 2024-05-15T14:11:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.213+0000 7f1eec037740 1 --2- 172.21.15.33:0/3622582835 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5600d0c46070 0x5600d0c48490 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.213+0000 7f1eec037740 1 --2- 172.21.15.33:0/3622582835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5600d0bcbad0 0x5600d0c3dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.213+0000 7f1ec6ff5640 0 client.16683 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:11:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.213+0000 7f1eec037740 1 --2- 172.21.15.33:0/3622582835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600d0bc1210 0x5600d0bcf300 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.213+0000 7f1eec037740 1 --2- 172.21.15.33:0/3622582835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600d0bc08f0 0x5600d0bcedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.213+0000 7f1eec037740 1 -- 172.21.15.33:0/3622582835 >> 172.21.15.33:0/3622582835 conn(0x5600d0bbce00 msgr2=0x5600d0bc22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.213+0000 7f1eec037740 1 -- 172.21.15.33:0/3622582835 shutdown_connections 2024-05-15T14:11:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.213+0000 7f1eec037740 1 -- 172.21.15.33:0/3622582835 wait complete. 2024-05-15T14:11:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.214+0000 7f1eec037740 20 client.16683 trim_cache size 0 max 16384 2024-05-15T14:11:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:11:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:11:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:11:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.214+0000 7f1ee9319640 1 -- 172.21.15.33:0/83445512 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f1ecc0b2980 msgr2=0x7f1ecc0b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.214+0000 7f1ee9319640 1 --2- 172.21.15.33:0/83445512 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f1ecc0b2980 0x7f1ecc0b4e40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1edc023000 tx=0x7f1edc024000 comp rx=0 tx=0).stop 2024-05-15T14:11:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.214+0000 7f1ee9319640 1 -- 172.21.15.33:0/83445512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ee4103d20 msgr2=0x7f1ee4101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.214+0000 7f1ee9319640 1 --2- 172.21.15.33:0/83445512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ee4103d20 0x7f1ee4101970 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1ed402a9a0 tx=0x7f1ed4031a20 comp rx=0 tx=0).stop 2024-05-15T14:11:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.214+0000 7f1ee9319640 1 -- 172.21.15.33:0/83445512 shutdown_connections 2024-05-15T14:11:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.214+0000 7f1ee9319640 1 --2- 172.21.15.33:0/83445512 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f1ecc0b2980 0x7f1ecc0b4e40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.214+0000 7f1ee9319640 1 --2- 172.21.15.33:0/83445512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ee410d060 0x7f1ee41023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.214+0000 7f1ee9319640 1 --2- 172.21.15.33:0/83445512 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ee4104640 0x7f1ee4101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.214+0000 7f1ee9319640 1 --2- 172.21.15.33:0/83445512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ee4103d20 0x7f1ee4101970 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.214+0000 7f1ee9319640 1 -- 172.21.15.33:0/83445512 >> 172.21.15.33:0/83445512 conn(0x7f1ee4078aa0 msgr2=0x7f1ee40feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.214+0000 7f1ee9319640 1 -- 172.21.15.33:0/83445512 shutdown_connections 2024-05-15T14:11:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:23.215+0000 7f1ee9319640 1 -- 172.21.15.33:0/83445512 wait complete. 2024-05-15T14:11:24.081 INFO:tasks.workunit.client.0.smithi033.stdout: 1 639621 5.52 MB/sec execute 419 sec latency 100.126 ms 2024-05-15T14:11:24.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:24 smithi052 ceph-mon[36190]: pgmap v347: 97 pgs: 97 active+clean; 211 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 308 op/s 2024-05-15T14:11:24.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:24 smithi081 ceph-mon[36434]: pgmap v347: 97 pgs: 97 active+clean; 211 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 308 op/s 2024-05-15T14:11:24.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:24 smithi033 ceph-mon[31650]: pgmap v347: 97 pgs: 97 active+clean; 211 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 308 op/s 2024-05-15T14:11:25.081 INFO:tasks.workunit.client.0.smithi033.stdout: 1 641032 5.52 MB/sec execute 420 sec latency 22.418 ms 2024-05-15T14:11:25.687 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.3 2024-05-15T14:11:25.687 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T14:11:25.963 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:11:26.081 INFO:tasks.workunit.client.0.smithi033.stdout: 1 643051 5.54 MB/sec execute 421 sec latency 23.085 ms 2024-05-15T14:11:26.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:26 smithi033 ceph-mon[31650]: pgmap v348: 97 pgs: 97 active+clean; 211 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 308 op/s 2024-05-15T14:11:26.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:26 smithi052 ceph-mon[36190]: pgmap v348: 97 pgs: 97 active+clean; 211 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 308 op/s 2024-05-15T14:11:26.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:26 smithi081 ceph-mon[36434]: pgmap v348: 97 pgs: 97 active+clean; 211 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.4 MiB/s wr, 308 op/s 2024-05-15T14:11:27.081 INFO:tasks.workunit.client.0.smithi033.stdout: 1 644288 5.54 MB/sec execute 422 sec latency 28.780 ms 2024-05-15T14:11:27.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.852+0000 7f078a685700 1 -- 172.21.15.33:0/2822656971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0784102440 msgr2=0x7f07841028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:27.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.852+0000 7f078a685700 1 --2- 172.21.15.33:0/2822656971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0784102440 0x7f07841028c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f0778009990 tx=0x7f0778009ca0 comp rx=0 tx=0).stop 2024-05-15T14:11:27.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.852+0000 7f078a685700 1 -- 172.21.15.33:0/2822656971 shutdown_connections 2024-05-15T14:11:27.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.852+0000 7f078a685700 1 --2- 172.21.15.33:0/2822656971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0784102e00 0x7f078410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:27.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.852+0000 7f078a685700 1 --2- 172.21.15.33:0/2822656971 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0784108520 0x7f0784108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:27.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.852+0000 7f078a685700 1 --2- 172.21.15.33:0/2822656971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0784102440 0x7f07841028c0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:27.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.852+0000 7f078a685700 1 -- 172.21.15.33:0/2822656971 >> 172.21.15.33:0/2822656971 conn(0x7f07840793f0 msgr2=0x7f0784079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:27.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.852+0000 7f078a685700 1 -- 172.21.15.33:0/2822656971 shutdown_connections 2024-05-15T14:11:27.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.852+0000 7f078a685700 1 -- 172.21.15.33:0/2822656971 wait complete. 2024-05-15T14:11:27.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.853+0000 7f078a685700 1 Processor -- start 2024-05-15T14:11:27.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.853+0000 7f078a685700 1 -- start start 2024-05-15T14:11:27.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.853+0000 7f078a685700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0784102440 0x7f078419f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:27.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.853+0000 7f078a685700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0784102e00 0x7f078419fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:27.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.853+0000 7f078a685700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0784108520 0x7f0784199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:27.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.853+0000 7f078a685700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f07841125f0 con 0x7f0784108520 2024-05-15T14:11:27.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.853+0000 7f078a685700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0784112470 con 0x7f0784102440 2024-05-15T14:11:27.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.853+0000 7f078a685700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0784112770 con 0x7f0784102e00 2024-05-15T14:11:27.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.853+0000 7f0788c22700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0784108520 0x7f0784199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:27.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.853+0000 7f0783fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0784102440 0x7f078419f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:27.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.853+0000 7f07837fe700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0784102e00 0x7f078419fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:27.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.853+0000 7f0783fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0784102440 0x7f078419f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:36696/0 (socket says 172.21.15.33:36696) 2024-05-15T14:11:27.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.853+0000 7f0788c22700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0784108520 0x7f0784199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49128/0 (socket says 172.21.15.33:49128) 2024-05-15T14:11:27.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.853+0000 7f0783fff700 1 -- 172.21.15.33:0/4021164827 learned_addr learned my addr 172.21.15.33:0/4021164827 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:11:27.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.854+0000 7f0788c22700 1 -- 172.21.15.33:0/4021164827 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0784102e00 msgr2=0x7f078419fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:27.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.854+0000 7f0788c22700 1 --2- 172.21.15.33:0/4021164827 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0784102e00 0x7f078419fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:27.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.854+0000 7f0788c22700 1 -- 172.21.15.33:0/4021164827 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0784102440 msgr2=0x7f078419f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:27.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.854+0000 7f0788c22700 1 --2- 172.21.15.33:0/4021164827 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0784102440 0x7f078419f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:27.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.854+0000 7f0788c22700 1 -- 172.21.15.33:0/4021164827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f078419a1b0 con 0x7f0784108520 2024-05-15T14:11:27.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.854+0000 7f07837fe700 1 --2- 172.21.15.33:0/4021164827 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0784102e00 0x7f078419fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:11:27.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.854+0000 7f0783fff700 1 --2- 172.21.15.33:0/4021164827 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0784102440 0x7f078419f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:11:27.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.854+0000 7f0788c22700 1 --2- 172.21.15.33:0/4021164827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0784108520 0x7f0784199a50 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f077c00efc0 tx=0x7f077c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:27.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.854+0000 7f07817fa700 1 -- 172.21.15.33:0/4021164827 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c0092e0 con 0x7f0784108520 2024-05-15T14:11:27.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.854+0000 7f07817fa700 1 -- 172.21.15.33:0/4021164827 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f077c004bb0 con 0x7f0784108520 2024-05-15T14:11:27.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.854+0000 7f078a685700 1 -- 172.21.15.33:0/4021164827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f078419a440 con 0x7f0784108520 2024-05-15T14:11:27.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.854+0000 7f07817fa700 1 -- 172.21.15.33:0/4021164827 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c004020 con 0x7f0784108520 2024-05-15T14:11:27.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.854+0000 7f078a685700 1 -- 172.21.15.33:0/4021164827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07841a6760 con 0x7f0784108520 2024-05-15T14:11:27.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.855+0000 7f078a685700 1 -- 172.21.15.33:0/4021164827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f078406a230 con 0x7f0784108520 2024-05-15T14:11:27.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.857+0000 7f07817fa700 1 -- 172.21.15.33:0/4021164827 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f077c01b030 con 0x7f0784108520 2024-05-15T14:11:27.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.858+0000 7f07817fa700 1 --2- 172.21.15.33:0/4021164827 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f076c0b29f0 0x7f076c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:27.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.858+0000 7f0783fff700 1 --2- 172.21.15.33:0/4021164827 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f076c0b29f0 0x7f076c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:27.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.858+0000 7f07817fa700 1 -- 172.21.15.33:0/4021164827 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7f077c0e2b50 con 0x7f0784108520 2024-05-15T14:11:27.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.858+0000 7f0783fff700 1 --2- 172.21.15.33:0/4021164827 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f076c0b29f0 0x7f076c0b4eb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f078419ab90 tx=0x7f0774009260 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:27.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:27.862+0000 7f07817fa700 1 -- 172.21.15.33:0/4021164827 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f077c022020 con 0x7f0784108520 2024-05-15T14:11:28.081 INFO:tasks.workunit.client.0.smithi033.stdout: 1 646459 5.56 MB/sec execute 423 sec latency 19.888 ms 2024-05-15T14:11:28.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:28.157+0000 7f078a685700 1 -- 172.21.15.33:0/4021164827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f078402e560 con 0x7f0784108520 2024-05-15T14:11:28.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:28.158+0000 7f07817fa700 1 -- 172.21.15.33:0/4021164827 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11861 (secure 0 0 0) 0x7f077c010040 con 0x7f0784108520 2024-05-15T14:11:28.161 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T14:11:28.161 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2030117633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2030117633},{"type":"v1","addr":"172.21.15.33:6839","nonce":2030117633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1918979061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1918979061},{"type":"v1","addr":"172.21.15.33:6841","nonce":1918979061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T13:57:19.056417+0000","modified":"2024-05-15T14:11:03.273793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24463,"mds_2":24461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/1629697841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1629697841},{"type":"v1","addr":"172.21.15.52:6837","nonce":1629697841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.52:6839/2696044471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2696044471},{"type":"v1","addr":"172.21.15.52:6839","nonce":2696044471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.81:6839/1554395940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":1554395940},{"type":"v1","addr":"172.21.15.81:6839","nonce":1554395940}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26206":{"gid":26206,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/3674797952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3674797952},{"type":"v1","addr":"172.21.15.81:6835","nonce":3674797952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24463,24478,24461]},"id":1}]} 2024-05-15T14:11:28.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:28.161+0000 7f078a685700 1 -- 172.21.15.33:0/4021164827 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f076c0b29f0 msgr2=0x7f076c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:28.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:28.161+0000 7f078a685700 1 --2- 172.21.15.33:0/4021164827 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f076c0b29f0 0x7f076c0b4eb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f078419ab90 tx=0x7f0774009260 comp rx=0 tx=0).stop 2024-05-15T14:11:28.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:28.161+0000 7f078a685700 1 -- 172.21.15.33:0/4021164827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0784108520 msgr2=0x7f0784199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:28.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:28.161+0000 7f078a685700 1 --2- 172.21.15.33:0/4021164827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0784108520 0x7f0784199a50 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f077c00efc0 tx=0x7f077c00c3a0 comp rx=0 tx=0).stop 2024-05-15T14:11:28.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:28.161+0000 7f078a685700 1 -- 172.21.15.33:0/4021164827 shutdown_connections 2024-05-15T14:11:28.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:28.161+0000 7f078a685700 1 --2- 172.21.15.33:0/4021164827 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f076c0b29f0 0x7f076c0b4eb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:28.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:28.161+0000 7f078a685700 1 --2- 172.21.15.33:0/4021164827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0784108520 0x7f0784199a50 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:28.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:28.161+0000 7f078a685700 1 --2- 172.21.15.33:0/4021164827 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0784102440 0x7f078419f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:28.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:28.161+0000 7f078a685700 1 --2- 172.21.15.33:0/4021164827 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0784102e00 0x7f078419fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:28.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:28.161+0000 7f078a685700 1 -- 172.21.15.33:0/4021164827 >> 172.21.15.33:0/4021164827 conn(0x7f07840793f0 msgr2=0x7f078406b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:28.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:28.161+0000 7f078a685700 1 -- 172.21.15.33:0/4021164827 shutdown_connections 2024-05-15T14:11:28.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:28.161+0000 7f078a685700 1 -- 172.21.15.33:0/4021164827 wait complete. 2024-05-15T14:11:28.165 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 39 2024-05-15T14:11:28.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:28 smithi052 ceph-mon[36190]: pgmap v349: 97 pgs: 97 active+clean; 203 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.9 MiB/s wr, 410 op/s 2024-05-15T14:11:28.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:28 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/4021164827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:11:28.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:28 smithi081 ceph-mon[36434]: pgmap v349: 97 pgs: 97 active+clean; 203 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.9 MiB/s wr, 410 op/s 2024-05-15T14:11:28.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:28 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/4021164827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:11:28.923 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:28 smithi033 ceph-mon[31650]: pgmap v349: 97 pgs: 97 active+clean; 203 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.9 MiB/s wr, 410 op/s 2024-05-15T14:11:28.923 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:28 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/4021164827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:11:28.954 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24478 quiesce db / --timeout 90 --expiration 145.3 --await-for 45.0' 2024-05-15T14:11:28.955 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24478 quiesce db / --timeout 90 --expiration 145.3 --await-for 45.0 2024-05-15T14:11:29.081 INFO:tasks.workunit.client.0.smithi033.stdout: 1 647734 5.56 MB/sec execute 424 sec latency 19.888 ms 2024-05-15T14:11:29.224 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:11:30.081 INFO:tasks.workunit.client.0.smithi033.stdout: 1 649981 5.58 MB/sec execute 425 sec latency 26.822 ms 2024-05-15T14:11:30.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:30 smithi052 ceph-mon[36190]: pgmap v350: 97 pgs: 97 active+clean; 203 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.5 MiB/s wr, 273 op/s 2024-05-15T14:11:30.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:30 smithi081 ceph-mon[36434]: pgmap v350: 97 pgs: 97 active+clean; 203 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.5 MiB/s wr, 273 op/s 2024-05-15T14:11:30.879 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:30 smithi033 ceph-mon[31650]: pgmap v350: 97 pgs: 97 active+clean; 203 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.5 MiB/s wr, 273 op/s 2024-05-15T14:11:31.081 INFO:tasks.workunit.client.0.smithi033.stdout: 1 651276 5.58 MB/sec execute 426 sec latency 44.793 ms 2024-05-15T14:11:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.350+0000 7fa29b0d0700 1 -- 172.21.15.33:0/1616711328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa294102440 msgr2=0x7fa2941028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.350+0000 7fa29b0d0700 1 --2- 172.21.15.33:0/1616711328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa294102440 0x7fa2941028c0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa284009990 tx=0x7fa284009ca0 comp rx=0 tx=0).stop 2024-05-15T14:11:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.350+0000 7fa29b0d0700 1 -- 172.21.15.33:0/1616711328 shutdown_connections 2024-05-15T14:11:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.350+0000 7fa29b0d0700 1 --2- 172.21.15.33:0/1616711328 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa294102e00 0x7fa29410f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.350+0000 7fa29b0d0700 1 --2- 172.21.15.33:0/1616711328 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2941084c0 0x7fa2941088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.350+0000 7fa29b0d0700 1 --2- 172.21.15.33:0/1616711328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa294102440 0x7fa2941028c0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.350+0000 7fa29b0d0700 1 -- 172.21.15.33:0/1616711328 >> 172.21.15.33:0/1616711328 conn(0x7fa2940793e0 msgr2=0x7fa2940797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.350+0000 7fa29b0d0700 1 -- 172.21.15.33:0/1616711328 shutdown_connections 2024-05-15T14:11:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.350+0000 7fa29b0d0700 1 -- 172.21.15.33:0/1616711328 wait complete. 2024-05-15T14:11:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.351+0000 7fa29b0d0700 1 Processor -- start 2024-05-15T14:11:31.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.351+0000 7fa29b0d0700 1 -- start start 2024-05-15T14:11:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.352+0000 7fa29b0d0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa294102440 0x7fa29419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.352+0000 7fa29b0d0700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa294102e00 0x7fa29419ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.352+0000 7fa29b0d0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2941084c0 0x7fa294198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.352+0000 7fa29b0d0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa294112670 con 0x7fa2941084c0 2024-05-15T14:11:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.352+0000 7fa29b0d0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa2941124f0 con 0x7fa294102440 2024-05-15T14:11:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.352+0000 7fa29b0d0700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa2941127f0 con 0x7fa294102e00 2024-05-15T14:11:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.352+0000 7fa29966d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2941084c0 0x7fa294198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.352+0000 7fa28bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa294102e00 0x7fa29419ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.352+0000 7fa29966d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2941084c0 0x7fa294198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49140/0 (socket says 172.21.15.33:49140) 2024-05-15T14:11:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.352+0000 7fa28bfff700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa294102e00 0x7fa29419ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:45476/0 (socket says 172.21.15.33:45476) 2024-05-15T14:11:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.352+0000 7fa29966d700 1 -- 172.21.15.33:0/3485331779 learned_addr learned my addr 172.21.15.33:0/3485331779 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:11:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.352+0000 7fa298e6c700 1 --2- 172.21.15.33:0/3485331779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa294102440 0x7fa29419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.352+0000 7fa29966d700 1 -- 172.21.15.33:0/3485331779 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa294102e00 msgr2=0x7fa29419ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:31.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.352+0000 7fa29966d700 1 --2- 172.21.15.33:0/3485331779 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa294102e00 0x7fa29419ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.352+0000 7fa29966d700 1 -- 172.21.15.33:0/3485331779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa294102440 msgr2=0x7fa29419e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.352+0000 7fa29966d700 1 --2- 172.21.15.33:0/3485331779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa294102440 0x7fa29419e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.352+0000 7fa29966d700 1 -- 172.21.15.33:0/3485331779 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa294198ff0 con 0x7fa2941084c0 2024-05-15T14:11:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.352+0000 7fa298e6c700 1 --2- 172.21.15.33:0/3485331779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa294102440 0x7fa29419e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:11:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.352+0000 7fa28bfff700 1 --2- 172.21.15.33:0/3485331779 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa294102e00 0x7fa29419ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:11:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.352+0000 7fa29966d700 1 --2- 172.21.15.33:0/3485331779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2941084c0 0x7fa294198890 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fa28c00b550 tx=0x7fa28c00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.352+0000 7fa289ffb700 1 -- 172.21.15.33:0/3485331779 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c015020 con 0x7fa2941084c0 2024-05-15T14:11:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.353+0000 7fa29b0d0700 1 -- 172.21.15.33:0/3485331779 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2941992e0 con 0x7fa2941084c0 2024-05-15T14:11:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.353+0000 7fa289ffb700 1 -- 172.21.15.33:0/3485331779 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa28c011ce0 con 0x7fa2941084c0 2024-05-15T14:11:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.353+0000 7fa289ffb700 1 -- 172.21.15.33:0/3485331779 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c010dd0 con 0x7fa2941084c0 2024-05-15T14:11:31.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.353+0000 7fa29b0d0700 1 -- 172.21.15.33:0/3485331779 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa29406b220 con 0x7fa2941084c0 2024-05-15T14:11:31.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.355+0000 7fa289ffb700 1 -- 172.21.15.33:0/3485331779 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa28c018020 con 0x7fa2941084c0 2024-05-15T14:11:31.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.356+0000 7fa289ffb700 1 --2- 172.21.15.33:0/3485331779 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fa27c0b2970 0x7fa27c0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:31.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.356+0000 7fa298e6c700 1 --2- 172.21.15.33:0/3485331779 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fa27c0b2970 0x7fa27c0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:31.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.356+0000 7fa289ffb700 1 -- 172.21.15.33:0/3485331779 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7fa28c0e1a60 con 0x7fa2941084c0 2024-05-15T14:11:31.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.357+0000 7fa298e6c700 1 --2- 172.21.15.33:0/3485331779 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fa27c0b2970 0x7fa27c0b4e30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa2941999d0 tx=0x7fa29000b040 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:31.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.357+0000 7fa2a5966b80 1 Processor -- start 2024-05-15T14:11:31.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.357+0000 7fa2a5966b80 1 -- start start 2024-05-15T14:11:31.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.358+0000 7fa2a5966b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b82e8ff4b0 0x55b82e8ff890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:31.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.358+0000 7fa2a5966b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b82e8ffdd0 0x55b82e8fad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:31.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.358+0000 7fa2a5966b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b82e8fb350 0x55b82e8fb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:31.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.358+0000 7fa2a5966b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b82e858f00 con 0x55b82e8ffdd0 2024-05-15T14:11:31.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.358+0000 7fa2a5966b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b82e848aa0 con 0x55b82e8fb350 2024-05-15T14:11:31.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.358+0000 7fa2a5966b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55b82e84cae0 con 0x55b82e8ff4b0 2024-05-15T14:11:31.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.358+0000 7fa298e6c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b82e8ff4b0 0x55b82e8ff890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:31.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.358+0000 7fa29966d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b82e8fb350 0x55b82e8fb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:31.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.358+0000 7fa298e6c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b82e8ff4b0 0x55b82e8ff890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:45486/0 (socket says 172.21.15.33:45486) 2024-05-15T14:11:31.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.358+0000 7fa298e6c700 1 -- 172.21.15.33:0/159323270 learned_addr learned my addr 172.21.15.33:0/159323270 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:11:31.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.358+0000 7fa28bfff700 1 --2- 172.21.15.33:0/159323270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b82e8ffdd0 0x55b82e8fad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:31.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.358+0000 7fa29966d700 1 -- 172.21.15.33:0/159323270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b82e8ff4b0 msgr2=0x55b82e8ff890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:31.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.358+0000 7fa29966d700 1 --2- 172.21.15.33:0/159323270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b82e8ff4b0 0x55b82e8ff890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:31.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.358+0000 7fa29966d700 1 -- 172.21.15.33:0/159323270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b82e8ffdd0 msgr2=0x55b82e8fad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:31.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.358+0000 7fa29966d700 1 --2- 172.21.15.33:0/159323270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b82e8ffdd0 0x55b82e8fad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:31.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.358+0000 7fa29966d700 1 -- 172.21.15.33:0/159323270 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b82e7f2140 con 0x55b82e8fb350 2024-05-15T14:11:31.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.358+0000 7fa298e6c700 1 --2- 172.21.15.33:0/159323270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b82e8ff4b0 0x55b82e8ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:11:31.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.358+0000 7fa28bfff700 1 --2- 172.21.15.33:0/159323270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b82e8ffdd0 0x55b82e8fad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:11:31.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.358+0000 7fa29966d700 1 --2- 172.21.15.33:0/159323270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b82e8fb350 0x55b82e8fb7b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa28c0df2b0 tx=0x7fa28c0a89d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29cfa81e590ee281 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:31.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.359+0000 7fa27affd700 1 -- 172.21.15.33:0/159323270 <== mon.2 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c0ab070 con 0x55b82e8fb350 2024-05-15T14:11:31.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.359+0000 7fa27affd700 1 -- 172.21.15.33:0/159323270 <== mon.2 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa28c0ad1f0 con 0x55b82e8fb350 2024-05-15T14:11:31.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.359+0000 7fa27affd700 1 -- 172.21.15.33:0/159323270 <== mon.2 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c0b1a30 con 0x55b82e8fb350 2024-05-15T14:11:31.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.359+0000 7fa2a5966b80 1 -- 172.21.15.33:0/159323270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b82e8fb350 msgr2=0x55b82e8fb7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:31.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.359+0000 7fa2a5966b80 1 --2- 172.21.15.33:0/159323270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b82e8fb350 0x55b82e8fb7b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa28c0df2b0 tx=0x7fa28c0a89d0 comp rx=0 tx=0).stop 2024-05-15T14:11:31.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.360+0000 7fa2a5966b80 1 -- 172.21.15.33:0/159323270 shutdown_connections 2024-05-15T14:11:31.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.360+0000 7fa2a5966b80 1 --2- 172.21.15.33:0/159323270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b82e8fb350 0x55b82e8fb7b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:31.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.360+0000 7fa2a5966b80 1 --2- 172.21.15.33:0/159323270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b82e8ff4b0 0x55b82e8ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:31.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.360+0000 7fa2a5966b80 1 --2- 172.21.15.33:0/159323270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b82e8ffdd0 0x55b82e8fad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:31.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.360+0000 7fa2a5966b80 1 -- 172.21.15.33:0/159323270 >> 172.21.15.33:0/159323270 conn(0x55b82e827f40 msgr2=0x55b82e8f5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:31.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.360+0000 7fa2a5966b80 1 -- 172.21.15.33:0/159323270 shutdown_connections 2024-05-15T14:11:31.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.360+0000 7fa2a5966b80 1 -- 172.21.15.33:0/159323270 wait complete. 2024-05-15T14:11:31.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.360+0000 7fa2a5966b80 1 -- start start 2024-05-15T14:11:31.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.360+0000 7fa2a5966b80 1 Processor -- start 2024-05-15T14:11:31.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.360+0000 7fa2a5966b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b82e8fb350 0x55b82e906bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:31.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.361+0000 7fa2a5966b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b82e8ff4b0 0x55b82e903f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:31.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.361+0000 7fa2a5966b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b82e8ffdd0 0x55b82e904490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:31.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.361+0000 7fa2a5966b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b82e74f130 con 0x55b82e8ffdd0 2024-05-15T14:11:31.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.361+0000 7fa2a5966b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b82e7b0f30 con 0x55b82e8ff4b0 2024-05-15T14:11:31.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.361+0000 7fa2a5966b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55b82e907910 con 0x55b82e8fb350 2024-05-15T14:11:31.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.361+0000 7fa2a5966b80 10 client.0 _ll_register_callbacks cb 0x55b82e7e3ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:11:31.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.361+0000 7fa298e6c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b82e8fb350 0x55b82e906bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:31.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.361+0000 7fa298e6c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b82e8fb350 0x55b82e906bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:45500/0 (socket says 172.21.15.33:45500) 2024-05-15T14:11:31.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.361+0000 7fa298e6c700 1 -- 172.21.15.33:0/4163212920 learned_addr learned my addr 172.21.15.33:0/4163212920 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:11:31.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.361+0000 7fa28bfff700 1 --2- 172.21.15.33:0/4163212920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b82e8ff4b0 0x55b82e903f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:31.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.361+0000 7fa29966d700 1 --2- 172.21.15.33:0/4163212920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b82e8ffdd0 0x55b82e904490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:31.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.361+0000 7fa298e6c700 1 -- 172.21.15.33:0/4163212920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b82e8ff4b0 msgr2=0x55b82e903f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:31.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.361+0000 7fa298e6c700 1 --2- 172.21.15.33:0/4163212920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b82e8ff4b0 0x55b82e903f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:31.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.361+0000 7fa298e6c700 1 -- 172.21.15.33:0/4163212920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b82e8ffdd0 msgr2=0x55b82e904490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:31.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.361+0000 7fa298e6c700 1 --2- 172.21.15.33:0/4163212920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b82e8ffdd0 0x55b82e904490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:31.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.361+0000 7fa298e6c700 1 -- 172.21.15.33:0/4163212920 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b82e798a60 con 0x55b82e8fb350 2024-05-15T14:11:31.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.361+0000 7fa298e6c700 1 --2- 172.21.15.33:0/4163212920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b82e8fb350 0x55b82e906bb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa290015d80 tx=0x7fa290004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae29157a5de0a4bf server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:31.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.361+0000 7fa273fff700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-05-15T14:11:31.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.361+0000 7fa273fff700 1 -- 172.21.15.33:0/4163212920 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa290026070 con 0x55b82e8fb350 2024-05-15T14:11:31.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.362+0000 7fa2a5966b80 1 -- 172.21.15.33:0/4163212920 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa28c0ad1f0 con 0x55b82e8fb350 2024-05-15T14:11:31.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.362+0000 7fa273fff700 1 -- 172.21.15.33:0/4163212920 <== mon.1 v2:172.21.15.81:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7fa290016d60 con 0x55b82e8fb350 2024-05-15T14:11:31.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.362+0000 7fa273fff700 1 -- 172.21.15.33:0/4163212920 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa290031050 con 0x55b82e8fb350 2024-05-15T14:11:31.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.362+0000 7fa2a5966b80 10 client.26275 fetch_fsmap learned FSMap version 39 2024-05-15T14:11:31.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.362+0000 7fa2a5966b80 1 -- 172.21.15.33:0/4163212920 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b82e8028f0 con 0x55b82e8fb350 2024-05-15T14:11:31.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.362+0000 7fa273fff700 1 -- 172.21.15.33:0/4163212920 <== mon.1 v2:172.21.15.81:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa28c0ad1f0 con 0x55b82e8fb350 2024-05-15T14:11:31.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.362+0000 7fa273fff700 1 -- 172.21.15.33:0/4163212920 <== mon.1 v2:172.21.15.81:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fa2900223f0 con 0x55b82e8fb350 2024-05-15T14:11:31.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.362+0000 7fa2a5966b80 10 client.26275 fetch_fsmap finished waiting for FSMap version 39 2024-05-15T14:11:31.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.362+0000 7fa2a5966b80 10 client.26275 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:11:31.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.362+0000 7fa2a5966b80 20 client.26275 populate_metadata read hostname 'smithi033' 2024-05-15T14:11:31.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.362+0000 7fa2a5966b80 1 --2- 172.21.15.33:0/4163212920 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55b82e91ccf0 0x55b82e91f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:31.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.362+0000 7fa2a5966b80 4 client.26275 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:11:31.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.362+0000 7fa2a5966b80 1 -- 172.21.15.33:0/4163212920 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b82e85ae50 con 0x55b82e91ccf0 2024-05-15T14:11:31.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.362+0000 7fa28bfff700 1 --2- 172.21.15.33:0/4163212920 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55b82e91ccf0 0x55b82e91f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:31.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.363+0000 7fa28bfff700 1 --2- 172.21.15.33:0/4163212920 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55b82e91ccf0 0x55b82e91f0e0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e762e3aefd1c1d11 server_cookie=27a46a0c97cbb913 in_seq=0 out_seq=0 2024-05-15T14:11:31.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.363+0000 7fa273fff700 10 client.26275 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:11:31.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.364+0000 7fa273fff700 1 -- 172.21.15.33:0/4163212920 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b82e85ae50 con 0x55b82e91ccf0 2024-05-15T14:11:31.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.364+0000 7fa273fff700 10 client.26275 handle_command_reply: tid=0 2024-05-15T14:11:31.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.364+0000 7fa2a5966b80 1 client.26275 shutdown 2024-05-15T14:11:31.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.364+0000 7fa2a5966b80 1 -- 172.21.15.33:0/4163212920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b82e8fb350 msgr2=0x55b82e906bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:31.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.364+0000 7fa2a5966b80 1 --2- 172.21.15.33:0/4163212920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b82e8fb350 0x55b82e906bb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa290015d80 tx=0x7fa290004ad0 comp rx=0 tx=0).stop 2024-05-15T14:11:31.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.364+0000 7fa2a5966b80 1 -- 172.21.15.33:0/4163212920 shutdown_connections 2024-05-15T14:11:31.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.364+0000 7fa2a5966b80 1 --2- 172.21.15.33:0/4163212920 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55b82e91ccf0 0x55b82e91f0e0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:31.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.364+0000 7fa2a5966b80 1 --2- 172.21.15.33:0/4163212920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b82e8ffdd0 0x55b82e904490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:31.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.364+0000 7fa2a5966b80 1 --2- 172.21.15.33:0/4163212920 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b82e8fb350 0x55b82e906bb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:31.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.364+0000 7fa2a5966b80 1 --2- 172.21.15.33:0/4163212920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b82e8ff4b0 0x55b82e903f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:31.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.364+0000 7fa273fff700 0 client.26275 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:11:31.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.364+0000 7fa2a5966b80 1 -- 172.21.15.33:0/4163212920 >> 172.21.15.33:0/4163212920 conn(0x55b82e827f40 msgr2=0x55b82e8f0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:31.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.364+0000 7fa2a5966b80 1 -- 172.21.15.33:0/4163212920 shutdown_connections 2024-05-15T14:11:31.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.364+0000 7fa2a5966b80 1 -- 172.21.15.33:0/4163212920 wait complete. 2024-05-15T14:11:31.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.364+0000 7fa2a5966b80 20 client.26275 trim_cache size 0 max 16384 2024-05-15T14:11:31.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.386+0000 7fa2a5966b80 1 Processor -- start 2024-05-15T14:11:31.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.386+0000 7fa2a5966b80 1 -- start start 2024-05-15T14:11:31.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.386+0000 7fa2a5966b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b82e8fb350 0x55b82e8f0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:31.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.386+0000 7fa2a5966b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b82e8ff4b0 0x55b82e8f8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:31.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.386+0000 7fa2a5966b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b82e8ffdd0 0x55b82e8f9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:31.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.386+0000 7fa2a5966b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa2941ae1e0 con 0x55b82e8fb350 2024-05-15T14:11:31.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.386+0000 7fa2a5966b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa2941aba10 con 0x55b82e8ff4b0 2024-05-15T14:11:31.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.386+0000 7fa2a5966b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa2941a9880 con 0x55b82e8ffdd0 2024-05-15T14:11:31.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.386+0000 7fa298e6c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b82e8fb350 0x55b82e8f0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:31.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.386+0000 7fa298e6c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b82e8fb350 0x55b82e8f0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49182/0 (socket says 172.21.15.33:49182) 2024-05-15T14:11:31.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.386+0000 7fa298e6c700 1 -- 172.21.15.33:0/1537300287 learned_addr learned my addr 172.21.15.33:0/1537300287 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:11:31.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.386+0000 7fa29966d700 1 --2- 172.21.15.33:0/1537300287 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b82e8ffdd0 0x55b82e8f9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:31.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.387+0000 7fa28bfff700 1 --2- 172.21.15.33:0/1537300287 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b82e8ff4b0 0x55b82e8f8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:31.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.387+0000 7fa298e6c700 1 -- 172.21.15.33:0/1537300287 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b82e8ffdd0 msgr2=0x55b82e8f9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:31.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.387+0000 7fa298e6c700 1 --2- 172.21.15.33:0/1537300287 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b82e8ffdd0 0x55b82e8f9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:31.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.387+0000 7fa298e6c700 1 -- 172.21.15.33:0/1537300287 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b82e8ff4b0 msgr2=0x55b82e8f8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:31.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.387+0000 7fa298e6c700 1 --2- 172.21.15.33:0/1537300287 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b82e8ff4b0 0x55b82e8f8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:31.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.387+0000 7fa298e6c700 1 -- 172.21.15.33:0/1537300287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b82e9067f0 con 0x55b82e8fb350 2024-05-15T14:11:31.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.387+0000 7fa298e6c700 1 --2- 172.21.15.33:0/1537300287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b82e8fb350 0x55b82e8f0160 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fa290009960 tx=0x7fa290022680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede88386c0879e8c server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:31.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.387+0000 7fa27a7fc700 1 -- 172.21.15.33:0/1537300287 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa290039020 con 0x55b82e8fb350 2024-05-15T14:11:31.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.387+0000 7fa27a7fc700 1 -- 172.21.15.33:0/1537300287 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa29001d040 con 0x55b82e8fb350 2024-05-15T14:11:31.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.387+0000 7fa27a7fc700 1 -- 172.21.15.33:0/1537300287 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa290026040 con 0x55b82e8fb350 2024-05-15T14:11:31.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.388+0000 7fa2a5966b80 1 -- 172.21.15.33:0/1537300287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b82e8fb350 msgr2=0x55b82e8f0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:31.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.388+0000 7fa2a5966b80 1 --2- 172.21.15.33:0/1537300287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b82e8fb350 0x55b82e8f0160 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fa290009960 tx=0x7fa290022680 comp rx=0 tx=0).stop 2024-05-15T14:11:31.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.388+0000 7fa2a5966b80 1 -- 172.21.15.33:0/1537300287 shutdown_connections 2024-05-15T14:11:31.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.388+0000 7fa2a5966b80 1 --2- 172.21.15.33:0/1537300287 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b82e8ffdd0 0x55b82e8f9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:31.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.388+0000 7fa2a5966b80 1 --2- 172.21.15.33:0/1537300287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b82e8fb350 0x55b82e8f0160 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:31.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.388+0000 7fa2a5966b80 1 --2- 172.21.15.33:0/1537300287 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b82e8ff4b0 0x55b82e8f8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:31.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.388+0000 7fa2a5966b80 1 -- 172.21.15.33:0/1537300287 >> 172.21.15.33:0/1537300287 conn(0x55b82e827f40 msgr2=0x55b82e902840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:31.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.388+0000 7fa2a5966b80 1 -- 172.21.15.33:0/1537300287 shutdown_connections 2024-05-15T14:11:31.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.388+0000 7fa2a5966b80 1 -- 172.21.15.33:0/1537300287 wait complete. 2024-05-15T14:11:31.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.388+0000 7fa2a5966b80 1 -- start start 2024-05-15T14:11:31.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.388+0000 7fa2a5966b80 1 Processor -- start 2024-05-15T14:11:31.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.389+0000 7fa2a5966b80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b82e8fb350 0x55b82e9103f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:31.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.389+0000 7fa2a5966b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b82e8ff4b0 0x55b82e910930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:31.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.389+0000 7fa2a5966b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b82e8ffdd0 0x55b82e910e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:31.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.389+0000 7fa2a5966b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa2941a9ed0 con 0x55b82e8ff4b0 2024-05-15T14:11:31.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.389+0000 7fa2a5966b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa2941ab340 con 0x55b82e8ffdd0 2024-05-15T14:11:31.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.389+0000 7fa2a5966b80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fa2941aa5a0 con 0x55b82e8fb350 2024-05-15T14:11:31.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.389+0000 7fa2a5966b80 10 client.0 _ll_register_callbacks cb 0x55b82e7e3ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:11:31.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.389+0000 7fa298e6c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b82e8fb350 0x55b82e9103f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:31.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.389+0000 7fa28bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b82e8ff4b0 0x55b82e910930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:31.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.389+0000 7fa298e6c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b82e8fb350 0x55b82e9103f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:45530/0 (socket says 172.21.15.33:45530) 2024-05-15T14:11:31.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.389+0000 7fa298e6c700 1 -- 172.21.15.33:0/1817378612 learned_addr learned my addr 172.21.15.33:0/1817378612 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:11:31.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.389+0000 7fa29966d700 1 --2- 172.21.15.33:0/1817378612 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b82e8ffdd0 0x55b82e910e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:31.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.389+0000 7fa298e6c700 1 -- 172.21.15.33:0/1817378612 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b82e8ffdd0 msgr2=0x55b82e910e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:31.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.389+0000 7fa298e6c700 1 --2- 172.21.15.33:0/1817378612 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b82e8ffdd0 0x55b82e910e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:31.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.389+0000 7fa298e6c700 1 -- 172.21.15.33:0/1817378612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b82e8ff4b0 msgr2=0x55b82e910930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:31.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.389+0000 7fa298e6c700 1 --2- 172.21.15.33:0/1817378612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b82e8ff4b0 0x55b82e910930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:31.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.389+0000 7fa298e6c700 1 -- 172.21.15.33:0/1817378612 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b82e987600 con 0x55b82e8fb350 2024-05-15T14:11:31.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.389+0000 7fa28bfff700 1 --2- 172.21.15.33:0/1817378612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b82e8ff4b0 0x55b82e910930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:11:31.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.389+0000 7fa298e6c700 1 --2- 172.21.15.33:0/1817378612 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b82e8fb350 0x55b82e9103f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa290005d50 tx=0x7fa290005fe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f105dd4bac13b37 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:31.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.389+0000 7fa273fff700 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-05-15T14:11:31.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.390+0000 7fa273fff700 1 -- 172.21.15.33:0/1817378612 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa290039030 con 0x55b82e8fb350 2024-05-15T14:11:31.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.390+0000 7fa2a5966b80 1 -- 172.21.15.33:0/1817378612 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa29001d040 con 0x55b82e8fb350 2024-05-15T14:11:31.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.390+0000 7fa273fff700 1 -- 172.21.15.33:0/1817378612 <== mon.1 v2:172.21.15.81:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7fa290022ab0 con 0x55b82e8fb350 2024-05-15T14:11:31.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.390+0000 7fa273fff700 1 -- 172.21.15.33:0/1817378612 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa290045050 con 0x55b82e8fb350 2024-05-15T14:11:31.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.390+0000 7fa2a5966b80 10 client.26281 fetch_fsmap learned FSMap version 39 2024-05-15T14:11:31.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.390+0000 7fa2a5966b80 1 -- 172.21.15.33:0/1817378612 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b82e9888e0 con 0x55b82e8fb350 2024-05-15T14:11:31.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.390+0000 7fa273fff700 1 -- 172.21.15.33:0/1817378612 <== mon.1 v2:172.21.15.81:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa29001d040 con 0x55b82e8fb350 2024-05-15T14:11:31.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.390+0000 7fa273fff700 1 -- 172.21.15.33:0/1817378612 <== mon.1 v2:172.21.15.81:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fa290004020 con 0x55b82e8fb350 2024-05-15T14:11:31.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.390+0000 7fa2a5966b80 10 client.26281 fetch_fsmap finished waiting for FSMap version 39 2024-05-15T14:11:31.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.390+0000 7fa2a5966b80 10 client.26281 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:11:31.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.390+0000 7fa2a5966b80 20 client.26281 populate_metadata read hostname 'smithi033' 2024-05-15T14:11:31.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.390+0000 7fa2a5966b80 1 --2- 172.21.15.33:0/1817378612 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55b82e988f20 0x55b82e98b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:31.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.390+0000 7fa2a5966b80 4 client.26281 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.3, "await_for": 45.0}] 2024-05-15T14:11:31.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.390+0000 7fa2a5966b80 1 -- 172.21.15.33:0/1817378612 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.3, "await_for": 45.0}) -- 0x55b82e854630 con 0x55b82e988f20 2024-05-15T14:11:31.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.390+0000 7fa28bfff700 1 --2- 172.21.15.33:0/1817378612 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55b82e988f20 0x55b82e98b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:31.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.391+0000 7fa273fff700 10 client.26281 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:11:31.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.391+0000 7fa28bfff700 1 --2- 172.21.15.33:0/1817378612 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55b82e988f20 0x55b82e98b340 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7eaa3f8d3bbd9193 server_cookie=4f39a990e8718ec7 in_seq=0 out_seq=0 2024-05-15T14:11:31.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.472+0000 7fa273fff700 1 -- 172.21.15.33:0/1817378612 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b82e854630 con 0x55b82e988f20 2024-05-15T14:11:31.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.472+0000 7fa273fff700 10 client.26281 handle_command_reply: tid=0 2024-05-15T14:11:31.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.472+0000 7fa2a5966b80 1 client.26281 shutdown 2024-05-15T14:11:31.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.472+0000 7fa2a5966b80 1 -- 172.21.15.33:0/1817378612 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b82e8fb350 msgr2=0x55b82e9103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:31.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.472+0000 7fa2a5966b80 1 --2- 172.21.15.33:0/1817378612 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b82e8fb350 0x55b82e9103f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa290005d50 tx=0x7fa290005fe0 comp rx=0 tx=0).stop 2024-05-15T14:11:31.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.472+0000 7fa2a5966b80 1 -- 172.21.15.33:0/1817378612 shutdown_connections 2024-05-15T14:11:31.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.472+0000 7fa2a5966b80 1 --2- 172.21.15.33:0/1817378612 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b82e8ffdd0 0x55b82e910e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:31.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.472+0000 7fa2a5966b80 1 --2- 172.21.15.33:0/1817378612 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b82e8fb350 0x55b82e9103f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:31.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.472+0000 7fa2a5966b80 1 --2- 172.21.15.33:0/1817378612 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55b82e988f20 0x55b82e98b340 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:31.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.472+0000 7fa2a5966b80 1 --2- 172.21.15.33:0/1817378612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b82e8ff4b0 0x55b82e910930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:31.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.472+0000 7fa273fff700 0 client.26281 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:11:31.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.472+0000 7fa2a5966b80 1 -- 172.21.15.33:0/1817378612 >> 172.21.15.33:0/1817378612 conn(0x55b82e827f40 msgr2=0x55b82e9030d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:31.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.472+0000 7fa2a5966b80 1 -- 172.21.15.33:0/1817378612 shutdown_connections 2024-05-15T14:11:31.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.472+0000 7fa2a5966b80 1 -- 172.21.15.33:0/1817378612 wait complete. 2024-05-15T14:11:31.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.472+0000 7fa2a5966b80 20 client.26281 trim_cache size 0 max 16384 2024-05-15T14:11:31.477 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T14:11:31.477 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 39, 2024-05-15T14:11:31.477 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24478, 2024-05-15T14:11:31.477 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 38, 2024-05-15T14:11:31.477 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-05-15T14:11:31.477 INFO:teuthology.orchestra.run.smithi033.stdout: "69488f58": { 2024-05-15T14:11:31.477 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 38, 2024-05-15T14:11:31.477 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-05-15T14:11:31.477 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:11:31.477 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-05-15T14:11:31.477 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:11:31.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:11:31.477 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 90.0, 2024-05-15T14:11:31.478 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 145.3, 2024-05-15T14:11:31.478 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-05-15T14:11:31.478 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-05-15T14:11:31.478 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-05-15T14:11:31.478 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:11:31.478 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-05-15T14:11:31.478 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:11:31.478 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:11:31.478 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:11:31.478 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:11:31.478 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:11:31.478 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:11:31.478 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T14:11:31.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.473+0000 7fa29b0d0700 1 -- 172.21.15.33:0/3485331779 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fa27c0b2970 msgr2=0x7fa27c0b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:31.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.473+0000 7fa29b0d0700 1 --2- 172.21.15.33:0/3485331779 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fa27c0b2970 0x7fa27c0b4e30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa2941999d0 tx=0x7fa29000b040 comp rx=0 tx=0).stop 2024-05-15T14:11:31.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.473+0000 7fa29b0d0700 1 -- 172.21.15.33:0/3485331779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2941084c0 msgr2=0x7fa294198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:31.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.473+0000 7fa29b0d0700 1 --2- 172.21.15.33:0/3485331779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2941084c0 0x7fa294198890 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fa28c00b550 tx=0x7fa28c00b860 comp rx=0 tx=0).stop 2024-05-15T14:11:31.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.473+0000 7fa29b0d0700 1 -- 172.21.15.33:0/3485331779 shutdown_connections 2024-05-15T14:11:31.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.473+0000 7fa29b0d0700 1 --2- 172.21.15.33:0/3485331779 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fa27c0b2970 0x7fa27c0b4e30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:31.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.473+0000 7fa29b0d0700 1 --2- 172.21.15.33:0/3485331779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2941084c0 0x7fa294198890 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:31.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.473+0000 7fa29b0d0700 1 --2- 172.21.15.33:0/3485331779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa294102440 0x7fa29419e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:31.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.473+0000 7fa29b0d0700 1 --2- 172.21.15.33:0/3485331779 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa294102e00 0x7fa29419ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:31.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.473+0000 7fa29b0d0700 1 -- 172.21.15.33:0/3485331779 >> 172.21.15.33:0/3485331779 conn(0x7fa2940793e0 msgr2=0x7fa294100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:31.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.473+0000 7fa29b0d0700 1 -- 172.21.15.33:0/3485331779 shutdown_connections 2024-05-15T14:11:31.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:31.473+0000 7fa29b0d0700 1 -- 172.21.15.33:0/3485331779 wait complete. 2024-05-15T14:11:32.081 INFO:tasks.workunit.client.0.smithi033.stdout: 1 651485 5.57 MB/sec execute 427 sec latency 590.134 ms 2024-05-15T14:11:32.195 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '69488f58', quiesce took 6.5 seconds. Will release after: 18.8 2024-05-15T14:11:32.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:32 smithi052 ceph-mon[36190]: pgmap v351: 97 pgs: 97 active+clean; 203 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.5 MiB/s wr, 273 op/s 2024-05-15T14:11:32.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:32 smithi081 ceph-mon[36434]: pgmap v351: 97 pgs: 97 active+clean; 203 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.5 MiB/s wr, 273 op/s 2024-05-15T14:11:32.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:32 smithi033 ceph-mon[31650]: pgmap v351: 97 pgs: 97 active+clean; 203 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.5 MiB/s wr, 273 op/s 2024-05-15T14:11:33.081 INFO:tasks.workunit.client.0.smithi033.stdout: 1 651485 5.55 MB/sec execute 428 sec latency 1590.250 ms 2024-05-15T14:11:34.081 INFO:tasks.workunit.client.0.smithi033.stdout: 1 651485 5.54 MB/sec execute 429 sec latency 2590.291 ms 2024-05-15T14:11:34.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:34 smithi052 ceph-mon[36190]: pgmap v352: 97 pgs: 97 active+clean; 207 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.2 MiB/s wr, 385 op/s 2024-05-15T14:11:34.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:34 smithi081 ceph-mon[36434]: pgmap v352: 97 pgs: 97 active+clean; 207 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.2 MiB/s wr, 385 op/s 2024-05-15T14:11:34.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:34 smithi033 ceph-mon[31650]: pgmap v352: 97 pgs: 97 active+clean; 207 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.2 MiB/s wr, 385 op/s 2024-05-15T14:11:35.081 INFO:tasks.workunit.client.0.smithi033.stdout: 1 651485 5.53 MB/sec execute 430 sec latency 3590.367 ms 2024-05-15T14:11:36.081 INFO:tasks.workunit.client.0.smithi033.stdout: 1 651485 5.51 MB/sec execute 431 sec latency 4590.510 ms 2024-05-15T14:11:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.317+0000 7fcf71083640 1 -- 172.21.15.33:0/3294734260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf6c10d090 msgr2=0x7fcf6c10f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.317+0000 7fcf71083640 1 --2- 172.21.15.33:0/3294734260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf6c10d090 0x7fcf6c10f4c0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fcf64009ca0 tx=0x7fcf6402a950 comp rx=0 tx=0).stop 2024-05-15T14:11:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.317+0000 7fcf71083640 1 -- 172.21.15.33:0/3294734260 shutdown_connections 2024-05-15T14:11:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.317+0000 7fcf71083640 1 --2- 172.21.15.33:0/3294734260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf6c10d090 0x7fcf6c10f4c0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.317+0000 7fcf71083640 1 --2- 172.21.15.33:0/3294734260 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf6c1045e0 0x7fcf6c10cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.317+0000 7fcf71083640 1 --2- 172.21.15.33:0/3294734260 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf6c103cc0 0x7fcf6c1040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.317+0000 7fcf71083640 1 -- 172.21.15.33:0/3294734260 >> 172.21.15.33:0/3294734260 conn(0x7fcf6c078a70 msgr2=0x7fcf6c078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.317+0000 7fcf71083640 1 -- 172.21.15.33:0/3294734260 shutdown_connections 2024-05-15T14:11:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.317+0000 7fcf71083640 1 -- 172.21.15.33:0/3294734260 wait complete. 2024-05-15T14:11:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.318+0000 7fcf71083640 1 Processor -- start 2024-05-15T14:11:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.318+0000 7fcf71083640 1 -- start start 2024-05-15T14:11:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.318+0000 7fcf71083640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf6c103cc0 0x7fcf6c199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.318+0000 7fcf71083640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf6c1045e0 0x7fcf6c199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.318+0000 7fcf71083640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf6c10d090 0x7fcf6c19dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.318+0000 7fcf71083640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c112cd0 con 0x7fcf6c1045e0 2024-05-15T14:11:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.318+0000 7fcf71083640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c112b50 con 0x7fcf6c103cc0 2024-05-15T14:11:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.318+0000 7fcf71083640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c112e50 con 0x7fcf6c10d090 2024-05-15T14:11:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.319+0000 7fcf6a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf6c1045e0 0x7fcf6c199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.319+0000 7fcf6a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf6c1045e0 0x7fcf6c199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54292/0 (socket says 172.21.15.33:54292) 2024-05-15T14:11:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.319+0000 7fcf6a59c640 1 -- 172.21.15.33:0/1832218502 learned_addr learned my addr 172.21.15.33:0/1832218502 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:11:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.319+0000 7fcf6b59e640 1 --2- 172.21.15.33:0/1832218502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf6c10d090 0x7fcf6c19dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.319+0000 7fcf6ad9d640 1 --2- 172.21.15.33:0/1832218502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf6c103cc0 0x7fcf6c199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.319+0000 7fcf6a59c640 1 -- 172.21.15.33:0/1832218502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf6c10d090 msgr2=0x7fcf6c19dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.319+0000 7fcf6a59c640 1 --2- 172.21.15.33:0/1832218502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf6c10d090 0x7fcf6c19dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.319+0000 7fcf6a59c640 1 -- 172.21.15.33:0/1832218502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf6c103cc0 msgr2=0x7fcf6c199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.319+0000 7fcf6a59c640 1 --2- 172.21.15.33:0/1832218502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf6c103cc0 0x7fcf6c199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.319+0000 7fcf6a59c640 1 -- 172.21.15.33:0/1832218502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf6c19e200 con 0x7fcf6c1045e0 2024-05-15T14:11:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.319+0000 7fcf6a59c640 1 --2- 172.21.15.33:0/1832218502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf6c1045e0 0x7fcf6c199750 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fcf6000d770 tx=0x7fcf6000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.319+0000 7fcf53fff640 1 -- 172.21.15.33:0/1832218502 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf60017070 con 0x7fcf6c1045e0 2024-05-15T14:11:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.319+0000 7fcf71083640 1 -- 172.21.15.33:0/1832218502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf6c19e4f0 con 0x7fcf6c1045e0 2024-05-15T14:11:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.319+0000 7fcf53fff640 1 -- 172.21.15.33:0/1832218502 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf600054d0 con 0x7fcf6c1045e0 2024-05-15T14:11:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.319+0000 7fcf53fff640 1 -- 172.21.15.33:0/1832218502 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf60004c30 con 0x7fcf6c1045e0 2024-05-15T14:11:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.320+0000 7fcf71083640 1 -- 172.21.15.33:0/1832218502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf6c1a5d80 con 0x7fcf6c1045e0 2024-05-15T14:11:36.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.322+0000 7fcf53fff640 1 -- 172.21.15.33:0/1832218502 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fcf60004df0 con 0x7fcf6c1045e0 2024-05-15T14:11:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.322+0000 7fcf73da4740 1 Processor -- start 2024-05-15T14:11:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.322+0000 7fcf73da4740 1 -- start start 2024-05-15T14:11:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.323+0000 7fcf73da4740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592aa6b68f0 0x5592aa6b6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.323+0000 7fcf73da4740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5592aa6b7210 0x5592aa6c1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.323+0000 7fcf73da4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592aa6c1ad0 0x5592aa6c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.323+0000 7fcf73da4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5592aa629c30 con 0x5592aa6c1ad0 2024-05-15T14:11:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.323+0000 7fcf73da4740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5592aa6197d0 con 0x5592aa6b68f0 2024-05-15T14:11:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.323+0000 7fcf73da4740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5592aa61d810 con 0x5592aa6b7210 2024-05-15T14:11:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.323+0000 7fcf6a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592aa6c1ad0 0x5592aa6c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.323+0000 7fcf6ad9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5592aa6b7210 0x5592aa6c1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.323+0000 7fcf6ad9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5592aa6b7210 0x5592aa6c1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:32996/0 (socket says 172.21.15.33:32996) 2024-05-15T14:11:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.323+0000 7fcf6ad9d640 1 -- 172.21.15.33:0/2501768692 learned_addr learned my addr 172.21.15.33:0/2501768692 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:11:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.323+0000 7fcf6b59e640 1 --2- 172.21.15.33:0/2501768692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592aa6b68f0 0x5592aa6b6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.323+0000 7fcf6ad9d640 1 -- 172.21.15.33:0/2501768692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592aa6b68f0 msgr2=0x5592aa6b6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.323+0000 7fcf6ad9d640 1 --2- 172.21.15.33:0/2501768692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592aa6b68f0 0x5592aa6b6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.323+0000 7fcf6ad9d640 1 -- 172.21.15.33:0/2501768692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592aa6c1ad0 msgr2=0x5592aa6c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.323+0000 7fcf6ad9d640 1 --2- 172.21.15.33:0/2501768692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592aa6c1ad0 0x5592aa6c3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.323+0000 7fcf6ad9d640 1 -- 172.21.15.33:0/2501768692 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592aa32e210 con 0x5592aa6b7210 2024-05-15T14:11:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.323+0000 7fcf53fff640 1 --2- 172.21.15.33:0/1832218502 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fcf540b2950 0x7fcf540b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.324+0000 7fcf6b59e640 1 --2- 172.21.15.33:0/1832218502 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fcf540b2950 0x7fcf540b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.324+0000 7fcf6ad9d640 1 --2- 172.21.15.33:0/2501768692 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5592aa6b7210 0x5592aa6c1590 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c00bc90 tx=0x7fcf5c00f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b65522e24578c531 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.324+0000 7fcf53fff640 1 -- 172.21.15.33:0/1832218502 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7fcf600e2050 con 0x7fcf6c1045e0 2024-05-15T14:11:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.324+0000 7fcf517fa640 1 -- 172.21.15.33:0/2501768692 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c015070 con 0x5592aa6b7210 2024-05-15T14:11:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.324+0000 7fcf517fa640 1 -- 172.21.15.33:0/2501768692 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf5c0118f0 con 0x5592aa6b7210 2024-05-15T14:11:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.324+0000 7fcf517fa640 1 -- 172.21.15.33:0/2501768692 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c0107d0 con 0x5592aa6b7210 2024-05-15T14:11:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.325+0000 7fcf73da4740 1 -- 172.21.15.33:0/2501768692 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5592aa6b7210 msgr2=0x5592aa6c1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.325+0000 7fcf73da4740 1 --2- 172.21.15.33:0/2501768692 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5592aa6b7210 0x5592aa6c1590 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c00bc90 tx=0x7fcf5c00f730 comp rx=0 tx=0).stop 2024-05-15T14:11:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.325+0000 7fcf6b59e640 1 --2- 172.21.15.33:0/1832218502 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fcf540b2950 0x7fcf540b4e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcf64005e30 tx=0x7fcf64005c50 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.325+0000 7fcf73da4740 1 -- 172.21.15.33:0/2501768692 shutdown_connections 2024-05-15T14:11:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.325+0000 7fcf73da4740 1 --2- 172.21.15.33:0/2501768692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592aa6c1ad0 0x5592aa6c3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.325+0000 7fcf73da4740 1 --2- 172.21.15.33:0/2501768692 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5592aa6b7210 0x5592aa6c1590 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.325+0000 7fcf73da4740 1 --2- 172.21.15.33:0/2501768692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592aa6b68f0 0x5592aa6b6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.325+0000 7fcf73da4740 1 -- 172.21.15.33:0/2501768692 >> 172.21.15.33:0/2501768692 conn(0x5592aa6b2e00 msgr2=0x5592aa6b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.325+0000 7fcf73da4740 1 -- 172.21.15.33:0/2501768692 shutdown_connections 2024-05-15T14:11:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.325+0000 7fcf73da4740 1 -- 172.21.15.33:0/2501768692 wait complete. 2024-05-15T14:11:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.325+0000 7fcf73da4740 1 -- start start 2024-05-15T14:11:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.325+0000 7fcf73da4740 1 Processor -- start 2024-05-15T14:11:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.325+0000 7fcf73da4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592aa6b68f0 0x5592aa6c1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.325+0000 7fcf73da4740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5592aa6b7210 0x5592aa6be6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.325+0000 7fcf73da4740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592aa6c1ad0 0x5592aa6bec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.325+0000 7fcf73da4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5592aa2ff790 con 0x5592aa6b68f0 2024-05-15T14:11:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.325+0000 7fcf73da4740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5592aa5d5890 con 0x5592aa6c1ad0 2024-05-15T14:11:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.325+0000 7fcf73da4740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5592aa6c47a0 con 0x5592aa6b7210 2024-05-15T14:11:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.326+0000 7fcf73da4740 10 client.0 _ll_register_callbacks cb 0x5592aa5d8d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:11:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.326+0000 7fcf6ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592aa6b68f0 0x5592aa6c1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.326+0000 7fcf6ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592aa6b68f0 0x5592aa6c1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54310/0 (socket says 172.21.15.33:54310) 2024-05-15T14:11:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.326+0000 7fcf6ad9d640 1 -- 172.21.15.33:0/1040440009 learned_addr learned my addr 172.21.15.33:0/1040440009 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:11:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.326+0000 7fcf6a59c640 1 --2- 172.21.15.33:0/1040440009 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5592aa6b7210 0x5592aa6be6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.326+0000 7fcf6b59e640 1 --2- 172.21.15.33:0/1040440009 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592aa6c1ad0 0x5592aa6bec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.326+0000 7fcf6ad9d640 1 -- 172.21.15.33:0/1040440009 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5592aa6b7210 msgr2=0x5592aa6be6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.326+0000 7fcf6ad9d640 1 --2- 172.21.15.33:0/1040440009 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5592aa6b7210 0x5592aa6be6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.326+0000 7fcf6ad9d640 1 -- 172.21.15.33:0/1040440009 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592aa6c1ad0 msgr2=0x5592aa6bec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.326+0000 7fcf6ad9d640 1 --2- 172.21.15.33:0/1040440009 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592aa6c1ad0 0x5592aa6bec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.326+0000 7fcf6ad9d640 1 -- 172.21.15.33:0/1040440009 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592aa3fa620 con 0x5592aa6b68f0 2024-05-15T14:11:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.326+0000 7fcf6b59e640 1 --2- 172.21.15.33:0/1040440009 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592aa6c1ad0 0x5592aa6bec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:11:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.326+0000 7fcf6ad9d640 1 --2- 172.21.15.33:0/1040440009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592aa6b68f0 0x5592aa6c1400 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c011670 tx=0x7fcf5c0116a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9aab12ca4045bb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.326+0000 7fcf4eff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:11:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.326+0000 7fcf4eff5640 1 -- 172.21.15.33:0/1040440009 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c028060 con 0x5592aa6b68f0 2024-05-15T14:11:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.326+0000 7fcf4eff5640 1 -- 172.21.15.33:0/1040440009 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7fcf5c00bd80 con 0x5592aa6b68f0 2024-05-15T14:11:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.326+0000 7fcf73da4740 1 -- 172.21.15.33:0/1040440009 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf5c0118f0 con 0x5592aa6b68f0 2024-05-15T14:11:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.327+0000 7fcf4eff5640 1 -- 172.21.15.33:0/1040440009 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcf5c015070 con 0x5592aa6b68f0 2024-05-15T14:11:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.327+0000 7fcf73da4740 10 client.16719 fetch_fsmap learned FSMap version 39 2024-05-15T14:11:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.327+0000 7fcf73da4740 1 -- 172.21.15.33:0/1040440009 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5592aa6d01a0 con 0x5592aa6b68f0 2024-05-15T14:11:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.327+0000 7fcf4eff5640 1 -- 172.21.15.33:0/1040440009 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf5c0118f0 con 0x5592aa6b68f0 2024-05-15T14:11:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.327+0000 7fcf4eff5640 1 -- 172.21.15.33:0/1040440009 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fcf5c005460 con 0x5592aa6b68f0 2024-05-15T14:11:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.327+0000 7fcf73da4740 10 client.16719 fetch_fsmap finished waiting for FSMap version 39 2024-05-15T14:11:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.327+0000 7fcf73da4740 10 client.16719 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:11:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.327+0000 7fcf73da4740 20 client.16719 populate_metadata read hostname 'smithi033' 2024-05-15T14:11:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.327+0000 7fcf73da4740 1 --2- 172.21.15.33:0/1040440009 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5592aa6d1600 0x5592aa6d39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.327+0000 7fcf73da4740 4 client.16719 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:11:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.327+0000 7fcf73da4740 1 -- 172.21.15.33:0/1040440009 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592aa62bb80 con 0x5592aa6d1600 2024-05-15T14:11:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.327+0000 7fcf6a59c640 1 --2- 172.21.15.33:0/1040440009 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5592aa6d1600 0x5592aa6d39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.328+0000 7fcf6a59c640 1 --2- 172.21.15.33:0/1040440009 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5592aa6d1600 0x5592aa6d39f0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7429565f3b279581 server_cookie=b9c170302e560b40 in_seq=0 out_seq=0 2024-05-15T14:11:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.328+0000 7fcf4eff5640 10 client.16719 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:11:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.329+0000 7fcf4eff5640 1 -- 172.21.15.33:0/1040440009 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5592aa62bb80 con 0x5592aa6d1600 2024-05-15T14:11:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.329+0000 7fcf4eff5640 10 client.16719 handle_command_reply: tid=0 2024-05-15T14:11:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.329+0000 7fcf73da4740 1 client.16719 shutdown 2024-05-15T14:11:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.329+0000 7fcf73da4740 1 -- 172.21.15.33:0/1040440009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592aa6b68f0 msgr2=0x5592aa6c1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.329+0000 7fcf73da4740 1 --2- 172.21.15.33:0/1040440009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592aa6b68f0 0x5592aa6c1400 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c011670 tx=0x7fcf5c0116a0 comp rx=0 tx=0).stop 2024-05-15T14:11:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.329+0000 7fcf73da4740 1 -- 172.21.15.33:0/1040440009 shutdown_connections 2024-05-15T14:11:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.329+0000 7fcf73da4740 1 --2- 172.21.15.33:0/1040440009 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5592aa6d1600 0x5592aa6d39f0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.329+0000 7fcf73da4740 1 --2- 172.21.15.33:0/1040440009 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592aa6c1ad0 0x5592aa6bec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.329+0000 7fcf73da4740 1 --2- 172.21.15.33:0/1040440009 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5592aa6b7210 0x5592aa6be6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.329+0000 7fcf4eff5640 0 client.16719 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:11:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.329+0000 7fcf73da4740 1 --2- 172.21.15.33:0/1040440009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592aa6b68f0 0x5592aa6c1400 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.329+0000 7fcf73da4740 1 -- 172.21.15.33:0/1040440009 >> 172.21.15.33:0/1040440009 conn(0x5592aa6b2e00 msgr2=0x5592aa6b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.329+0000 7fcf73da4740 1 -- 172.21.15.33:0/1040440009 shutdown_connections 2024-05-15T14:11:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.329+0000 7fcf73da4740 1 -- 172.21.15.33:0/1040440009 wait complete. 2024-05-15T14:11:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.329+0000 7fcf73da4740 20 client.16719 trim_cache size 0 max 16384 2024-05-15T14:11:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.344+0000 7fcf73da4740 1 Processor -- start 2024-05-15T14:11:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.344+0000 7fcf73da4740 1 -- start start 2024-05-15T14:11:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.345+0000 7fcf73da4740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5592aa6b68f0 0x5592aa6b9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.345+0000 7fcf73da4740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592aa6b7210 0x5592aa6b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.345+0000 7fcf73da4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592aa6c1ad0 0x5592aa6c0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.345+0000 7fcf73da4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c1b0fc0 con 0x5592aa6c1ad0 2024-05-15T14:11:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.345+0000 7fcf73da4740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c1ae830 con 0x5592aa6b7210 2024-05-15T14:11:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.345+0000 7fcf73da4740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c1ac6a0 con 0x5592aa6b68f0 2024-05-15T14:11:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.345+0000 7fcf6b59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592aa6c1ad0 0x5592aa6c0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.345+0000 7fcf6b59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592aa6c1ad0 0x5592aa6c0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54326/0 (socket says 172.21.15.33:54326) 2024-05-15T14:11:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.345+0000 7fcf6b59e640 1 -- 172.21.15.33:0/1484805284 learned_addr learned my addr 172.21.15.33:0/1484805284 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:11:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.345+0000 7fcf6a59c640 1 --2- 172.21.15.33:0/1484805284 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592aa6b7210 0x5592aa6b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.345+0000 7fcf6ad9d640 1 --2- 172.21.15.33:0/1484805284 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5592aa6b68f0 0x5592aa6b9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.345+0000 7fcf6b59e640 1 -- 172.21.15.33:0/1484805284 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5592aa6b68f0 msgr2=0x5592aa6b9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.345+0000 7fcf6b59e640 1 --2- 172.21.15.33:0/1484805284 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5592aa6b68f0 0x5592aa6b9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.345+0000 7fcf6b59e640 1 -- 172.21.15.33:0/1484805284 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592aa6b7210 msgr2=0x5592aa6b1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.345+0000 7fcf6b59e640 1 --2- 172.21.15.33:0/1484805284 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592aa6b7210 0x5592aa6b1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.345+0000 7fcf6b59e640 1 -- 172.21.15.33:0/1484805284 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592aa6b2c30 con 0x5592aa6c1ad0 2024-05-15T14:11:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.345+0000 7fcf6a59c640 1 --2- 172.21.15.33:0/1484805284 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592aa6b7210 0x5592aa6b1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:11:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.345+0000 7fcf6b59e640 1 --2- 172.21.15.33:0/1484805284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592aa6c1ad0 0x5592aa6c0f30 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c06b350 tx=0x7fcf64042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4a54d080e6a93f server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.345+0000 7fcf50ff9640 1 -- 172.21.15.33:0/1484805284 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf64034040 con 0x5592aa6c1ad0 2024-05-15T14:11:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.346+0000 7fcf50ff9640 1 -- 172.21.15.33:0/1484805284 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf64003e20 con 0x5592aa6c1ad0 2024-05-15T14:11:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.346+0000 7fcf50ff9640 1 -- 172.21.15.33:0/1484805284 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf640306a0 con 0x5592aa6c1ad0 2024-05-15T14:11:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.346+0000 7fcf73da4740 1 -- 172.21.15.33:0/1484805284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592aa6c1ad0 msgr2=0x5592aa6c0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.346+0000 7fcf73da4740 1 --2- 172.21.15.33:0/1484805284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592aa6c1ad0 0x5592aa6c0f30 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c06b350 tx=0x7fcf64042000 comp rx=0 tx=0).stop 2024-05-15T14:11:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.346+0000 7fcf73da4740 1 -- 172.21.15.33:0/1484805284 shutdown_connections 2024-05-15T14:11:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.346+0000 7fcf73da4740 1 --2- 172.21.15.33:0/1484805284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592aa6c1ad0 0x5592aa6c0f30 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.346+0000 7fcf73da4740 1 --2- 172.21.15.33:0/1484805284 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592aa6b7210 0x5592aa6b1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.346+0000 7fcf73da4740 1 --2- 172.21.15.33:0/1484805284 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5592aa6b68f0 0x5592aa6b9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.346+0000 7fcf73da4740 1 -- 172.21.15.33:0/1484805284 >> 172.21.15.33:0/1484805284 conn(0x5592aa6b2e00 msgr2=0x5592aa6b1870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.346+0000 7fcf73da4740 1 -- 172.21.15.33:0/1484805284 shutdown_connections 2024-05-15T14:11:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.346+0000 7fcf73da4740 1 -- 172.21.15.33:0/1484805284 wait complete. 2024-05-15T14:11:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.346+0000 7fcf73da4740 1 -- start start 2024-05-15T14:11:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.347+0000 7fcf73da4740 1 Processor -- start 2024-05-15T14:11:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.347+0000 7fcf73da4740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592aa6b68f0 0x5592aa6c9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.347+0000 7fcf73da4740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5592aa6b7210 0x5592aa6c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.347+0000 7fcf73da4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592aa6c1ad0 0x5592aa6c4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.347+0000 7fcf73da4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c1accf0 con 0x5592aa6c1ad0 2024-05-15T14:11:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.347+0000 7fcf73da4740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c1ae160 con 0x5592aa6b68f0 2024-05-15T14:11:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.347+0000 7fcf73da4740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c1ad3c0 con 0x5592aa6b7210 2024-05-15T14:11:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.347+0000 7fcf73da4740 10 client.0 _ll_register_callbacks cb 0x7fcf5c00bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:11:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.347+0000 7fcf6b59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592aa6c1ad0 0x5592aa6c4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.347+0000 7fcf6b59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592aa6c1ad0 0x5592aa6c4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54340/0 (socket says 172.21.15.33:54340) 2024-05-15T14:11:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.347+0000 7fcf6b59e640 1 -- 172.21.15.33:0/3128354064 learned_addr learned my addr 172.21.15.33:0/3128354064 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:11:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.347+0000 7fcf6a59c640 1 --2- 172.21.15.33:0/3128354064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5592aa6b7210 0x5592aa6c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.347+0000 7fcf6ad9d640 1 --2- 172.21.15.33:0/3128354064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592aa6b68f0 0x5592aa6c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.347+0000 7fcf6b59e640 1 -- 172.21.15.33:0/3128354064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5592aa6b7210 msgr2=0x5592aa6c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.347+0000 7fcf6b59e640 1 --2- 172.21.15.33:0/3128354064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5592aa6b7210 0x5592aa6c9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.347+0000 7fcf6b59e640 1 -- 172.21.15.33:0/3128354064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592aa6b68f0 msgr2=0x5592aa6c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.347+0000 7fcf6b59e640 1 --2- 172.21.15.33:0/3128354064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592aa6b68f0 0x5592aa6c9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.347+0000 7fcf6b59e640 1 -- 172.21.15.33:0/3128354064 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592aa6c57a0 con 0x5592aa6c1ad0 2024-05-15T14:11:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.347+0000 7fcf6ad9d640 1 --2- 172.21.15.33:0/3128354064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592aa6b68f0 0x5592aa6c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:11:36.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.347+0000 7fcf6b59e640 1 --2- 172.21.15.33:0/3128354064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592aa6c1ad0 0x5592aa6c4fd0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fcf6402f810 tx=0x7fcf6402f8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c5320bf3a069b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.347+0000 7fcf4eff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:11:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.348+0000 7fcf4eff5640 1 -- 172.21.15.33:0/3128354064 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6405e060 con 0x5592aa6c1ad0 2024-05-15T14:11:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.348+0000 7fcf4eff5640 1 -- 172.21.15.33:0/3128354064 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7fcf64004350 con 0x5592aa6c1ad0 2024-05-15T14:11:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.348+0000 7fcf73da4740 1 -- 172.21.15.33:0/3128354064 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf64003e20 con 0x5592aa6c1ad0 2024-05-15T14:11:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.348+0000 7fcf4eff5640 1 -- 172.21.15.33:0/3128354064 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcf64034050 con 0x5592aa6c1ad0 2024-05-15T14:11:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.348+0000 7fcf73da4740 10 client.16725 fetch_fsmap learned FSMap version 39 2024-05-15T14:11:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.348+0000 7fcf73da4740 1 -- 172.21.15.33:0/3128354064 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5592aa73b6b0 con 0x5592aa6c1ad0 2024-05-15T14:11:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.348+0000 7fcf4eff5640 1 -- 172.21.15.33:0/3128354064 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf64003e20 con 0x5592aa6c1ad0 2024-05-15T14:11:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.348+0000 7fcf4eff5640 1 -- 172.21.15.33:0/3128354064 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fcf6403a4c0 con 0x5592aa6c1ad0 2024-05-15T14:11:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.348+0000 7fcf73da4740 10 client.16725 fetch_fsmap finished waiting for FSMap version 39 2024-05-15T14:11:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.348+0000 7fcf73da4740 10 client.16725 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:11:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.348+0000 7fcf73da4740 20 client.16725 populate_metadata read hostname 'smithi033' 2024-05-15T14:11:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.348+0000 7fcf73da4740 1 --2- 172.21.15.33:0/3128354064 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5592aa73bec0 0x5592aa73e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.348+0000 7fcf73da4740 4 client.16725 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:11:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.348+0000 7fcf73da4740 1 -- 172.21.15.33:0/3128354064 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5592aa625360 con 0x5592aa73bec0 2024-05-15T14:11:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.349+0000 7fcf6ad9d640 1 --2- 172.21.15.33:0/3128354064 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5592aa73bec0 0x5592aa73e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.349+0000 7fcf6ad9d640 1 --2- 172.21.15.33:0/3128354064 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5592aa73bec0 0x5592aa73e2e0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73815c351595850b server_cookie=9e3e84093ba5751c in_seq=0 out_seq=0 2024-05-15T14:11:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.349+0000 7fcf4eff5640 10 client.16725 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:11:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.397+0000 7fcf4eff5640 1 -- 172.21.15.33:0/3128354064 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5592aa625360 con 0x5592aa73bec0 2024-05-15T14:11:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.397+0000 7fcf4eff5640 10 client.16725 handle_command_reply: tid=0 2024-05-15T14:11:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.397+0000 7fcf73da4740 1 client.16725 shutdown 2024-05-15T14:11:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.397+0000 7fcf73da4740 1 -- 172.21.15.33:0/3128354064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592aa6c1ad0 msgr2=0x5592aa6c4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.397+0000 7fcf73da4740 1 --2- 172.21.15.33:0/3128354064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592aa6c1ad0 0x5592aa6c4fd0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fcf6402f810 tx=0x7fcf6402f8e0 comp rx=0 tx=0).stop 2024-05-15T14:11:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.397+0000 7fcf73da4740 1 -- 172.21.15.33:0/3128354064 shutdown_connections 2024-05-15T14:11:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.397+0000 7fcf73da4740 1 --2- 172.21.15.33:0/3128354064 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5592aa73bec0 0x5592aa73e2e0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.397+0000 7fcf73da4740 1 --2- 172.21.15.33:0/3128354064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592aa6c1ad0 0x5592aa6c4fd0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.397+0000 7fcf73da4740 1 --2- 172.21.15.33:0/3128354064 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5592aa6b7210 0x5592aa6c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.397+0000 7fcf73da4740 1 --2- 172.21.15.33:0/3128354064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592aa6b68f0 0x5592aa6c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.397+0000 7fcf73da4740 1 -- 172.21.15.33:0/3128354064 >> 172.21.15.33:0/3128354064 conn(0x5592aa6b2e00 msgr2=0x5592aa6b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.397+0000 7fcf4eff5640 0 client.16725 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:11:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.397+0000 7fcf73da4740 1 -- 172.21.15.33:0/3128354064 shutdown_connections 2024-05-15T14:11:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.397+0000 7fcf73da4740 1 -- 172.21.15.33:0/3128354064 wait complete. 2024-05-15T14:11:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.398+0000 7fcf73da4740 20 client.16725 trim_cache size 0 max 16384 2024-05-15T14:11:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:11:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:11:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:11:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.398+0000 7fcf71083640 1 -- 172.21.15.33:0/1832218502 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fcf540b2950 msgr2=0x7fcf540b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.398+0000 7fcf71083640 1 --2- 172.21.15.33:0/1832218502 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fcf540b2950 0x7fcf540b4e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcf64005e30 tx=0x7fcf64005c50 comp rx=0 tx=0).stop 2024-05-15T14:11:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.398+0000 7fcf71083640 1 -- 172.21.15.33:0/1832218502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf6c1045e0 msgr2=0x7fcf6c199750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.398+0000 7fcf71083640 1 --2- 172.21.15.33:0/1832218502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf6c1045e0 0x7fcf6c199750 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fcf6000d770 tx=0x7fcf6000dc30 comp rx=0 tx=0).stop 2024-05-15T14:11:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.398+0000 7fcf71083640 1 -- 172.21.15.33:0/1832218502 shutdown_connections 2024-05-15T14:11:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.398+0000 7fcf71083640 1 --2- 172.21.15.33:0/1832218502 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fcf540b2950 0x7fcf540b4e10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.398+0000 7fcf71083640 1 --2- 172.21.15.33:0/1832218502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf6c10d090 0x7fcf6c19dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.398+0000 7fcf71083640 1 --2- 172.21.15.33:0/1832218502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf6c1045e0 0x7fcf6c199750 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.398+0000 7fcf71083640 1 --2- 172.21.15.33:0/1832218502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf6c103cc0 0x7fcf6c199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.398+0000 7fcf71083640 1 -- 172.21.15.33:0/1832218502 >> 172.21.15.33:0/1832218502 conn(0x7fcf6c078a70 msgr2=0x7fcf6c0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.399+0000 7fcf71083640 1 -- 172.21.15.33:0/1832218502 shutdown_connections 2024-05-15T14:11:36.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:36.399+0000 7fcf71083640 1 -- 172.21.15.33:0/1832218502 wait complete. 2024-05-15T14:11:36.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:36 smithi052 ceph-mon[36190]: pgmap v353: 97 pgs: 97 active+clean; 207 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.7 MiB/s wr, 258 op/s 2024-05-15T14:11:36.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:36 smithi081 ceph-mon[36434]: pgmap v353: 97 pgs: 97 active+clean; 207 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.7 MiB/s wr, 258 op/s 2024-05-15T14:11:36.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:36 smithi033 ceph-mon[31650]: pgmap v353: 97 pgs: 97 active+clean; 207 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.7 MiB/s wr, 258 op/s 2024-05-15T14:11:37.082 INFO:tasks.workunit.client.0.smithi033.stdout: 1 651485 5.50 MB/sec execute 432 sec latency 5590.620 ms 2024-05-15T14:11:38.082 INFO:tasks.workunit.client.0.smithi033.stdout: 1 651485 5.49 MB/sec execute 433 sec latency 6590.739 ms 2024-05-15T14:11:38.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:38 smithi052 ceph-mon[36190]: pgmap v354: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.8 MiB/s wr, 284 op/s 2024-05-15T14:11:38.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:38 smithi081 ceph-mon[36434]: pgmap v354: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.8 MiB/s wr, 284 op/s 2024-05-15T14:11:38.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:38 smithi033 ceph-mon[31650]: pgmap v354: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.8 MiB/s wr, 284 op/s 2024-05-15T14:11:39.082 INFO:tasks.workunit.client.0.smithi033.stdout: 1 651485 5.48 MB/sec execute 434 sec latency 7590.827 ms 2024-05-15T14:11:40.082 INFO:tasks.workunit.client.0.smithi033.stdout: 1 651485 5.46 MB/sec execute 435 sec latency 8590.892 ms 2024-05-15T14:11:40.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:40 smithi052 ceph-mon[36190]: pgmap v355: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 745 KiB/s wr, 137 op/s 2024-05-15T14:11:40.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:40 smithi081 ceph-mon[36434]: pgmap v355: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 745 KiB/s wr, 137 op/s 2024-05-15T14:11:40.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:40 smithi033 ceph-mon[31650]: pgmap v355: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 745 KiB/s wr, 137 op/s 2024-05-15T14:11:41.082 INFO:tasks.workunit.client.0.smithi033.stdout: 1 651485 5.45 MB/sec execute 436 sec latency 9591.013 ms 2024-05-15T14:11:42.084 INFO:tasks.workunit.client.0.smithi033.stdout: 1 651485 5.44 MB/sec execute 437 sec latency 10591.050 ms 2024-05-15T14:11:42.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:42 smithi052 ceph-mon[36190]: pgmap v356: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 745 KiB/s wr, 137 op/s 2024-05-15T14:11:42.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:42 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:11:42.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:42 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:11:42.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:42 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:11:42.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:42 smithi081 ceph-mon[36434]: pgmap v356: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 745 KiB/s wr, 137 op/s 2024-05-15T14:11:42.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:42 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:11:42.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:42 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:11:42.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:42 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:11:42.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:42 smithi033 ceph-mon[31650]: pgmap v356: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 745 KiB/s wr, 137 op/s 2024-05-15T14:11:42.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:42 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:11:42.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:42 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:11:42.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:42 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:11:43.082 INFO:tasks.workunit.client.0.smithi033.stdout: 1 651485 5.43 MB/sec execute 438 sec latency 11591.121 ms 2024-05-15T14:11:43.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:43 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:11:43.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:43 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:11:43.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:43 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:11:43.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:43 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:11:43.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:43 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:11:43.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:43 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:11:43.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:43 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:11:43.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:43 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:11:43.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:43 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:11:44.082 INFO:tasks.workunit.client.0.smithi033.stdout: 1 651485 5.41 MB/sec execute 439 sec latency 12591.193 ms 2024-05-15T14:11:44.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:44 smithi052 ceph-mon[36190]: pgmap v357: 97 pgs: 97 active+clean; 206 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-05-15T14:11:44.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:44 smithi081 ceph-mon[36434]: pgmap v357: 97 pgs: 97 active+clean; 206 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-05-15T14:11:44.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:44 smithi033 ceph-mon[31650]: pgmap v357: 97 pgs: 97 active+clean; 206 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-05-15T14:11:45.082 INFO:tasks.workunit.client.0.smithi033.stdout: 1 651485 5.40 MB/sec execute 440 sec latency 13591.301 ms 2024-05-15T14:11:46.082 INFO:tasks.workunit.client.0.smithi033.stdout: 1 651485 5.39 MB/sec execute 441 sec latency 14591.421 ms 2024-05-15T14:11:46.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:46 smithi052 ceph-mon[36190]: pgmap v358: 97 pgs: 97 active+clean; 206 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 429 KiB/s wr, 39 op/s 2024-05-15T14:11:46.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:46 smithi081 ceph-mon[36434]: pgmap v358: 97 pgs: 97 active+clean; 206 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 429 KiB/s wr, 39 op/s 2024-05-15T14:11:46.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:46 smithi033 ceph-mon[31650]: pgmap v358: 97 pgs: 97 active+clean; 206 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 429 KiB/s wr, 39 op/s 2024-05-15T14:11:47.083 INFO:tasks.workunit.client.0.smithi033.stdout: 1 651485 5.38 MB/sec execute 442 sec latency 15591.536 ms 2024-05-15T14:11:48.082 INFO:tasks.workunit.client.0.smithi033.stdout: 1 651485 5.36 MB/sec execute 443 sec latency 16591.638 ms 2024-05-15T14:11:48.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:48 smithi052 ceph-mon[36190]: pgmap v359: 97 pgs: 97 active+clean; 209 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 1.1 MiB/s wr, 61 op/s 2024-05-15T14:11:48.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:48 smithi081 ceph-mon[36434]: pgmap v359: 97 pgs: 97 active+clean; 209 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 1.1 MiB/s wr, 61 op/s 2024-05-15T14:11:48.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:48 smithi033 ceph-mon[31650]: pgmap v359: 97 pgs: 97 active+clean; 209 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 1.1 MiB/s wr, 61 op/s 2024-05-15T14:11:49.083 INFO:tasks.workunit.client.0.smithi033.stdout: 1 651485 5.35 MB/sec execute 444 sec latency 17591.718 ms 2024-05-15T14:11:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.499+0000 7fc8dd746640 1 -- 172.21.15.33:0/3985926335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8d8104ec0 msgr2=0x7fc8d81052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.499+0000 7fc8dd746640 1 --2- 172.21.15.33:0/3985926335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8d8104ec0 0x7fc8d81052a0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8009900 tx=0x7fc8c802a410 comp rx=0 tx=0).stop 2024-05-15T14:11:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.499+0000 7fc8dd746640 1 -- 172.21.15.33:0/3985926335 shutdown_connections 2024-05-15T14:11:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.499+0000 7fc8dd746640 1 --2- 172.21.15.33:0/3985926335 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8d8109f90 0x7fc8d8111b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.499+0000 7fc8dd746640 1 --2- 172.21.15.33:0/3985926335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8d8105870 0x7fc8d81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.499+0000 7fc8dd746640 1 --2- 172.21.15.33:0/3985926335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8d8104ec0 0x7fc8d81052a0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.499+0000 7fc8dd746640 1 -- 172.21.15.33:0/3985926335 >> 172.21.15.33:0/3985926335 conn(0x7fc8d8100740 msgr2=0x7fc8d8102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.499+0000 7fc8dd746640 1 -- 172.21.15.33:0/3985926335 shutdown_connections 2024-05-15T14:11:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.499+0000 7fc8dd746640 1 -- 172.21.15.33:0/3985926335 wait complete. 2024-05-15T14:11:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.500+0000 7fc8dd746640 1 Processor -- start 2024-05-15T14:11:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.500+0000 7fc8dd746640 1 -- start start 2024-05-15T14:11:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.500+0000 7fc8dd746640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8d8104ec0 0x7fc8d819b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.500+0000 7fc8dd746640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8d8109f90 0x7fc8d819ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.500+0000 7fc8dd746640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8d819fdb0 0x7fc8d81a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.500+0000 7fc8dd746640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d8114ef0 con 0x7fc8d8109f90 2024-05-15T14:11:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.500+0000 7fc8dd746640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d8114d70 con 0x7fc8d819fdb0 2024-05-15T14:11:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.500+0000 7fc8dd746640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d8115070 con 0x7fc8d8104ec0 2024-05-15T14:11:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.500+0000 7fc8d67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8d8109f90 0x7fc8d819ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.501+0000 7fc8d67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8d8109f90 0x7fc8d819ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35286/0 (socket says 172.21.15.33:35286) 2024-05-15T14:11:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.501+0000 7fc8d67fc640 1 -- 172.21.15.33:0/578238146 learned_addr learned my addr 172.21.15.33:0/578238146 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:11:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.501+0000 7fc8d6ffd640 1 --2- 172.21.15.33:0/578238146 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8d8104ec0 0x7fc8d819b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.501+0000 7fc8d77fe640 1 --2- 172.21.15.33:0/578238146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8d819fdb0 0x7fc8d81a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.501+0000 7fc8d67fc640 1 -- 172.21.15.33:0/578238146 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8d8104ec0 msgr2=0x7fc8d819b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.501+0000 7fc8d67fc640 1 --2- 172.21.15.33:0/578238146 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8d8104ec0 0x7fc8d819b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.501+0000 7fc8d67fc640 1 -- 172.21.15.33:0/578238146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8d819fdb0 msgr2=0x7fc8d81a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.501+0000 7fc8d67fc640 1 --2- 172.21.15.33:0/578238146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8d819fdb0 0x7fc8d81a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.501+0000 7fc8d67fc640 1 -- 172.21.15.33:0/578238146 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8d81a08a0 con 0x7fc8d8109f90 2024-05-15T14:11:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.501+0000 7fc8d77fe640 1 --2- 172.21.15.33:0/578238146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8d819fdb0 0x7fc8d81a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:11:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.501+0000 7fc8d67fc640 1 --2- 172.21.15.33:0/578238146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8d8109f90 0x7fc8d819ba30 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc8c000d770 tx=0x7fc8c000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.501+0000 7fc8cffff640 1 -- 172.21.15.33:0/578238146 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0017070 con 0x7fc8d8109f90 2024-05-15T14:11:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.501+0000 7fc8cffff640 1 -- 172.21.15.33:0/578238146 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8c0009c30 con 0x7fc8d8109f90 2024-05-15T14:11:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.501+0000 7fc8cffff640 1 -- 172.21.15.33:0/578238146 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c000f620 con 0x7fc8d8109f90 2024-05-15T14:11:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.501+0000 7fc8dd746640 1 -- 172.21.15.33:0/578238146 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8d81a0b90 con 0x7fc8d8109f90 2024-05-15T14:11:49.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.501+0000 7fc8dd746640 1 -- 172.21.15.33:0/578238146 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8d81a8380 con 0x7fc8d8109f90 2024-05-15T14:11:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.504+0000 7fc8e0467740 1 Processor -- start 2024-05-15T14:11:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.504+0000 7fc8cffff640 1 -- 172.21.15.33:0/578238146 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc8c001a070 con 0x7fc8d8109f90 2024-05-15T14:11:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.504+0000 7fc8e0467740 1 -- start start 2024-05-15T14:11:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.504+0000 7fc8e0467740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d56f2f8f0 0x555d56f2fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.504+0000 7fc8e0467740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d56f30210 0x555d56f3a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.504+0000 7fc8e0467740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d56f3aad0 0x555d56f3cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.504+0000 7fc8e0467740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555d56ea2c30 con 0x555d56f2f8f0 2024-05-15T14:11:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.504+0000 7fc8e0467740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555d56e927d0 con 0x555d56f30210 2024-05-15T14:11:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.504+0000 7fc8e0467740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x555d56e96810 con 0x555d56f3aad0 2024-05-15T14:11:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.505+0000 7fc8d77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d56f2f8f0 0x555d56f2fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.505+0000 7fc8d77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d56f2f8f0 0x555d56f2fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35300/0 (socket says 172.21.15.33:35300) 2024-05-15T14:11:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.505+0000 7fc8d77fe640 1 -- 172.21.15.33:0/1010065219 learned_addr learned my addr 172.21.15.33:0/1010065219 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:11:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.505+0000 7fc8d67fc640 1 --2- 172.21.15.33:0/1010065219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d56f3aad0 0x555d56f3cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.505+0000 7fc8d6ffd640 1 --2- 172.21.15.33:0/1010065219 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d56f30210 0x555d56f3a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.505+0000 7fc8d77fe640 1 -- 172.21.15.33:0/1010065219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d56f3aad0 msgr2=0x555d56f3cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.505+0000 7fc8d77fe640 1 --2- 172.21.15.33:0/1010065219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d56f3aad0 0x555d56f3cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.505+0000 7fc8d77fe640 1 -- 172.21.15.33:0/1010065219 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d56f30210 msgr2=0x555d56f3a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.505+0000 7fc8d77fe640 1 --2- 172.21.15.33:0/1010065219 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d56f30210 0x555d56f3a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.505+0000 7fc8d77fe640 1 -- 172.21.15.33:0/1010065219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d56ba7210 con 0x555d56f2f8f0 2024-05-15T14:11:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.505+0000 7fc8cffff640 1 --2- 172.21.15.33:0/578238146 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc8c40b2950 0x7fc8c40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.505+0000 7fc8d77fe640 1 --2- 172.21.15.33:0/1010065219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d56f2f8f0 0x555d56f2fcd0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009f30 tx=0x7fc8d0010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8af35128ad0997a server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.505+0000 7fc8d6ffd640 1 --2- 172.21.15.33:0/578238146 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc8c40b2950 0x7fc8c40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.505+0000 7fc8cffff640 1 -- 172.21.15.33:0/578238146 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7fc8c00e2150 con 0x7fc8d8109f90 2024-05-15T14:11:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.505+0000 7fc8cd7fa640 1 -- 172.21.15.33:0/1010065219 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d0016070 con 0x555d56f2f8f0 2024-05-15T14:11:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.505+0000 7fc8cd7fa640 1 -- 172.21.15.33:0/1010065219 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8d001bb10 con 0x555d56f2f8f0 2024-05-15T14:11:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.505+0000 7fc8cd7fa640 1 -- 172.21.15.33:0/1010065219 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d0012bc0 con 0x555d56f2f8f0 2024-05-15T14:11:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.506+0000 7fc8e0467740 1 -- 172.21.15.33:0/1010065219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d56f2f8f0 msgr2=0x555d56f2fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.506+0000 7fc8e0467740 1 --2- 172.21.15.33:0/1010065219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d56f2f8f0 0x555d56f2fcd0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009f30 tx=0x7fc8d0010800 comp rx=0 tx=0).stop 2024-05-15T14:11:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.506+0000 7fc8d6ffd640 1 --2- 172.21.15.33:0/578238146 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc8c40b2950 0x7fc8c40b4e10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8031440 tx=0x7fc8c8005890 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.506+0000 7fc8e0467740 1 -- 172.21.15.33:0/1010065219 shutdown_connections 2024-05-15T14:11:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.506+0000 7fc8e0467740 1 --2- 172.21.15.33:0/1010065219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d56f3aad0 0x555d56f3cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.506+0000 7fc8e0467740 1 --2- 172.21.15.33:0/1010065219 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d56f30210 0x555d56f3a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.506+0000 7fc8e0467740 1 --2- 172.21.15.33:0/1010065219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d56f2f8f0 0x555d56f2fcd0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.506+0000 7fc8e0467740 1 -- 172.21.15.33:0/1010065219 >> 172.21.15.33:0/1010065219 conn(0x555d56f2be00 msgr2=0x555d56f2c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.506+0000 7fc8e0467740 1 -- 172.21.15.33:0/1010065219 shutdown_connections 2024-05-15T14:11:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.506+0000 7fc8e0467740 1 -- 172.21.15.33:0/1010065219 wait complete. 2024-05-15T14:11:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.507+0000 7fc8e0467740 1 -- start start 2024-05-15T14:11:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.507+0000 7fc8e0467740 1 Processor -- start 2024-05-15T14:11:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.507+0000 7fc8e0467740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d56f2f8f0 0x555d56f3a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.507+0000 7fc8e0467740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d56f30210 0x555d56f377e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.507+0000 7fc8e0467740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d56f3aad0 0x555d56f37df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.507+0000 7fc8e0467740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555d56b78790 con 0x555d56f30210 2024-05-15T14:11:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.507+0000 7fc8e0467740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555d56e4e890 con 0x555d56f3aad0 2024-05-15T14:11:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.507+0000 7fc8e0467740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x555d56f3d7a0 con 0x555d56f2f8f0 2024-05-15T14:11:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.507+0000 7fc8e0467740 10 client.0 _ll_register_callbacks cb 0x555d56e51d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:11:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.507+0000 7fc8d67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d56f30210 0x555d56f377e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.507+0000 7fc8d67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d56f30210 0x555d56f377e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35312/0 (socket says 172.21.15.33:35312) 2024-05-15T14:11:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.507+0000 7fc8d67fc640 1 -- 172.21.15.33:0/3044321373 learned_addr learned my addr 172.21.15.33:0/3044321373 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:11:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.507+0000 7fc8d77fe640 1 --2- 172.21.15.33:0/3044321373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d56f3aad0 0x555d56f37df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.507+0000 7fc8d6ffd640 1 --2- 172.21.15.33:0/3044321373 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d56f2f8f0 0x555d56f3a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.507+0000 7fc8d67fc640 1 -- 172.21.15.33:0/3044321373 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d56f2f8f0 msgr2=0x555d56f3a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.507+0000 7fc8d67fc640 1 --2- 172.21.15.33:0/3044321373 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d56f2f8f0 0x555d56f3a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.507+0000 7fc8d67fc640 1 -- 172.21.15.33:0/3044321373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d56f3aad0 msgr2=0x555d56f37df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.507+0000 7fc8d67fc640 1 --2- 172.21.15.33:0/3044321373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d56f3aad0 0x555d56f37df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.508+0000 7fc8d67fc640 1 -- 172.21.15.33:0/3044321373 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d56c73620 con 0x555d56f30210 2024-05-15T14:11:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.508+0000 7fc8d67fc640 1 --2- 172.21.15.33:0/3044321373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d56f30210 0x555d56f377e0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00a93b0 tx=0x7fc8c0010f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7414abb9a1db4f6f server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.508+0000 7fc8be7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:11:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.508+0000 7fc8be7fc640 1 -- 172.21.15.33:0/3044321373 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0021070 con 0x555d56f30210 2024-05-15T14:11:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.508+0000 7fc8be7fc640 1 -- 172.21.15.33:0/3044321373 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7fc8c00b0d60 con 0x555d56f30210 2024-05-15T14:11:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.508+0000 7fc8e0467740 1 -- 172.21.15.33:0/3044321373 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8d001bb10 con 0x555d56f30210 2024-05-15T14:11:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.508+0000 7fc8be7fc640 1 -- 172.21.15.33:0/3044321373 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc8c00a8d70 con 0x555d56f30210 2024-05-15T14:11:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.508+0000 7fc8e0467740 10 client.16737 fetch_fsmap learned FSMap version 39 2024-05-15T14:11:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.508+0000 7fc8e0467740 1 -- 172.21.15.33:0/3044321373 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555d56f49280 con 0x555d56f30210 2024-05-15T14:11:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.508+0000 7fc8be7fc640 1 -- 172.21.15.33:0/3044321373 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8d001bb10 con 0x555d56f30210 2024-05-15T14:11:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.508+0000 7fc8be7fc640 1 -- 172.21.15.33:0/3044321373 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fc8c00a8680 con 0x555d56f30210 2024-05-15T14:11:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.508+0000 7fc8e0467740 10 client.16737 fetch_fsmap finished waiting for FSMap version 39 2024-05-15T14:11:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.508+0000 7fc8e0467740 10 client.16737 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:11:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.508+0000 7fc8e0467740 20 client.16737 populate_metadata read hostname 'smithi033' 2024-05-15T14:11:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.508+0000 7fc8e0467740 1 --2- 172.21.15.33:0/3044321373 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x555d56f4a790 0x555d56f4cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.508+0000 7fc8e0467740 4 client.16737 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:11:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.508+0000 7fc8e0467740 1 -- 172.21.15.33:0/3044321373 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d56ea4b80 con 0x555d56f4a790 2024-05-15T14:11:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.509+0000 7fc8d6ffd640 1 --2- 172.21.15.33:0/3044321373 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x555d56f4a790 0x555d56f4cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.509+0000 7fc8be7fc640 10 client.16737 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:11:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.509+0000 7fc8d6ffd640 1 --2- 172.21.15.33:0/3044321373 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x555d56f4a790 0x555d56f4cbc0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3c872d0eab59d8d server_cookie=e093a119f620565d in_seq=0 out_seq=0 2024-05-15T14:11:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.510+0000 7fc8be7fc640 1 -- 172.21.15.33:0/3044321373 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555d56ea4b80 con 0x555d56f4a790 2024-05-15T14:11:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.510+0000 7fc8be7fc640 10 client.16737 handle_command_reply: tid=0 2024-05-15T14:11:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.510+0000 7fc8e0467740 1 client.16737 shutdown 2024-05-15T14:11:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.510+0000 7fc8e0467740 1 -- 172.21.15.33:0/3044321373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d56f30210 msgr2=0x555d56f377e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.510+0000 7fc8e0467740 1 --2- 172.21.15.33:0/3044321373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d56f30210 0x555d56f377e0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00a93b0 tx=0x7fc8c0010f80 comp rx=0 tx=0).stop 2024-05-15T14:11:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.510+0000 7fc8e0467740 1 -- 172.21.15.33:0/3044321373 shutdown_connections 2024-05-15T14:11:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.510+0000 7fc8e0467740 1 --2- 172.21.15.33:0/3044321373 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x555d56f4a790 0x555d56f4cbc0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.510+0000 7fc8e0467740 1 --2- 172.21.15.33:0/3044321373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d56f3aad0 0x555d56f37df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.510+0000 7fc8e0467740 1 --2- 172.21.15.33:0/3044321373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d56f30210 0x555d56f377e0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.510+0000 7fc8be7fc640 0 client.16737 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:11:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.510+0000 7fc8e0467740 1 --2- 172.21.15.33:0/3044321373 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d56f2f8f0 0x555d56f3a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.510+0000 7fc8e0467740 1 -- 172.21.15.33:0/3044321373 >> 172.21.15.33:0/3044321373 conn(0x555d56f2be00 msgr2=0x555d56f2b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.510+0000 7fc8e0467740 1 -- 172.21.15.33:0/3044321373 shutdown_connections 2024-05-15T14:11:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.510+0000 7fc8e0467740 1 -- 172.21.15.33:0/3044321373 wait complete. 2024-05-15T14:11:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.510+0000 7fc8e0467740 20 client.16737 trim_cache size 0 max 16384 2024-05-15T14:11:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.526+0000 7fc8e0467740 1 Processor -- start 2024-05-15T14:11:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.526+0000 7fc8e0467740 1 -- start start 2024-05-15T14:11:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.526+0000 7fc8e0467740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d56f2f8f0 0x555d56f46670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.526+0000 7fc8e0467740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d56f30210 0x555d56f52490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.526+0000 7fc8e0467740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d56f3aad0 0x555d56f529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.526+0000 7fc8e0467740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d81b3330 con 0x555d56f30210 2024-05-15T14:11:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.526+0000 7fc8e0467740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d81b0ba0 con 0x555d56f3aad0 2024-05-15T14:11:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.526+0000 7fc8e0467740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d81aea10 con 0x555d56f2f8f0 2024-05-15T14:11:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.526+0000 7fc8d67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d56f30210 0x555d56f52490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.526+0000 7fc8d67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d56f30210 0x555d56f52490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35316/0 (socket says 172.21.15.33:35316) 2024-05-15T14:11:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.526+0000 7fc8d67fc640 1 -- 172.21.15.33:0/3530624100 learned_addr learned my addr 172.21.15.33:0/3530624100 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:11:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.526+0000 7fc8d6ffd640 1 --2- 172.21.15.33:0/3530624100 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d56f2f8f0 0x555d56f46670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.526+0000 7fc8d77fe640 1 --2- 172.21.15.33:0/3530624100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d56f3aad0 0x555d56f529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.526+0000 7fc8d67fc640 1 -- 172.21.15.33:0/3530624100 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d56f2f8f0 msgr2=0x555d56f46670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.527+0000 7fc8d67fc640 1 --2- 172.21.15.33:0/3530624100 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d56f2f8f0 0x555d56f46670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.527+0000 7fc8d67fc640 1 -- 172.21.15.33:0/3530624100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d56f3aad0 msgr2=0x555d56f529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.527+0000 7fc8d67fc640 1 --2- 172.21.15.33:0/3530624100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d56f3aad0 0x555d56f529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.527+0000 7fc8d67fc640 1 -- 172.21.15.33:0/3530624100 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d56f27830 con 0x555d56f30210 2024-05-15T14:11:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.527+0000 7fc8d6ffd640 1 --2- 172.21.15.33:0/3530624100 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d56f2f8f0 0x555d56f46670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:11:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.527+0000 7fc8d77fe640 1 --2- 172.21.15.33:0/3530624100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d56f3aad0 0x555d56f529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:11:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.527+0000 7fc8d67fc640 1 --2- 172.21.15.33:0/3530624100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d56f30210 0x555d56f52490 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00a6690 tx=0x7fc8c00ad850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a36ffd078ce62c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.527+0000 7fc8ccff9640 1 -- 172.21.15.33:0/3530624100 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0021070 con 0x555d56f30210 2024-05-15T14:11:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.527+0000 7fc8ccff9640 1 -- 172.21.15.33:0/3530624100 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8c0009de0 con 0x555d56f30210 2024-05-15T14:11:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.527+0000 7fc8ccff9640 1 -- 172.21.15.33:0/3530624100 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c00a8b90 con 0x555d56f30210 2024-05-15T14:11:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.528+0000 7fc8e0467740 1 -- 172.21.15.33:0/3530624100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d56f30210 msgr2=0x555d56f52490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.528+0000 7fc8e0467740 1 --2- 172.21.15.33:0/3530624100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d56f30210 0x555d56f52490 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00a6690 tx=0x7fc8c00ad850 comp rx=0 tx=0).stop 2024-05-15T14:11:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.528+0000 7fc8e0467740 1 -- 172.21.15.33:0/3530624100 shutdown_connections 2024-05-15T14:11:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.528+0000 7fc8e0467740 1 --2- 172.21.15.33:0/3530624100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d56f3aad0 0x555d56f529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.528+0000 7fc8e0467740 1 --2- 172.21.15.33:0/3530624100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d56f30210 0x555d56f52490 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.528+0000 7fc8e0467740 1 --2- 172.21.15.33:0/3530624100 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d56f2f8f0 0x555d56f46670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.528+0000 7fc8e0467740 1 -- 172.21.15.33:0/3530624100 >> 172.21.15.33:0/3530624100 conn(0x555d56f2be00 msgr2=0x555d56f360b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.528+0000 7fc8e0467740 1 -- 172.21.15.33:0/3530624100 shutdown_connections 2024-05-15T14:11:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.528+0000 7fc8e0467740 1 -- 172.21.15.33:0/3530624100 wait complete. 2024-05-15T14:11:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.528+0000 7fc8e0467740 1 -- start start 2024-05-15T14:11:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.528+0000 7fc8e0467740 1 Processor -- start 2024-05-15T14:11:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.528+0000 7fc8e0467740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d56f2f8f0 0x555d56f40710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.528+0000 7fc8e0467740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d56f30210 0x555d56f40c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.528+0000 7fc8e0467740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d56f3aad0 0x555d56f41190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.528+0000 7fc8e0467740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d81af060 con 0x555d56f30210 2024-05-15T14:11:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.528+0000 7fc8e0467740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d81b04d0 con 0x555d56f3aad0 2024-05-15T14:11:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.528+0000 7fc8e0467740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d81af730 con 0x555d56f2f8f0 2024-05-15T14:11:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.528+0000 7fc8e0467740 10 client.0 _ll_register_callbacks cb 0x7fc8c00dfd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:11:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.529+0000 7fc8d67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d56f30210 0x555d56f40c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.529+0000 7fc8d67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d56f30210 0x555d56f40c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35320/0 (socket says 172.21.15.33:35320) 2024-05-15T14:11:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.529+0000 7fc8d67fc640 1 -- 172.21.15.33:0/207054539 learned_addr learned my addr 172.21.15.33:0/207054539 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:11:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.529+0000 7fc8d6ffd640 1 --2- 172.21.15.33:0/207054539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d56f2f8f0 0x555d56f40710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.529+0000 7fc8d77fe640 1 --2- 172.21.15.33:0/207054539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d56f3aad0 0x555d56f41190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.529+0000 7fc8d67fc640 1 -- 172.21.15.33:0/207054539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d56f2f8f0 msgr2=0x555d56f40710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.529+0000 7fc8d67fc640 1 --2- 172.21.15.33:0/207054539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d56f2f8f0 0x555d56f40710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.529+0000 7fc8d67fc640 1 -- 172.21.15.33:0/207054539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d56f3aad0 msgr2=0x555d56f41190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.529+0000 7fc8d67fc640 1 --2- 172.21.15.33:0/207054539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d56f3aad0 0x555d56f41190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.529+0000 7fc8d67fc640 1 -- 172.21.15.33:0/207054539 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d56fb41b0 con 0x555d56f30210 2024-05-15T14:11:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.529+0000 7fc8d6ffd640 1 --2- 172.21.15.33:0/207054539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d56f2f8f0 0x555d56f40710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:11:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.529+0000 7fc8d77fe640 1 --2- 172.21.15.33:0/207054539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d56f3aad0 0x555d56f41190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:11:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.529+0000 7fc8d67fc640 1 --2- 172.21.15.33:0/207054539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d56f30210 0x555d56f40c50 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00b23e0 tx=0x7fc8c00a95a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107cce2ea195b781 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.529+0000 7fc8be7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:11:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.529+0000 7fc8be7fc640 1 -- 172.21.15.33:0/207054539 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c00c2070 con 0x555d56f30210 2024-05-15T14:11:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.529+0000 7fc8be7fc640 1 -- 172.21.15.33:0/207054539 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7fc8c00a4fa0 con 0x555d56f30210 2024-05-15T14:11:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.529+0000 7fc8e0467740 1 -- 172.21.15.33:0/207054539 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8c0009de0 con 0x555d56f30210 2024-05-15T14:11:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.529+0000 7fc8be7fc640 1 -- 172.21.15.33:0/207054539 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc8c0021070 con 0x555d56f30210 2024-05-15T14:11:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.529+0000 7fc8e0467740 10 client.16743 fetch_fsmap learned FSMap version 39 2024-05-15T14:11:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.529+0000 7fc8e0467740 1 -- 172.21.15.33:0/207054539 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555d56fb5410 con 0x555d56f30210 2024-05-15T14:11:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.530+0000 7fc8be7fc640 1 -- 172.21.15.33:0/207054539 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8c0009de0 con 0x555d56f30210 2024-05-15T14:11:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.530+0000 7fc8be7fc640 1 -- 172.21.15.33:0/207054539 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fc8c00a80f0 con 0x555d56f30210 2024-05-15T14:11:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.530+0000 7fc8e0467740 10 client.16743 fetch_fsmap finished waiting for FSMap version 39 2024-05-15T14:11:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.530+0000 7fc8e0467740 10 client.16743 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:11:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.530+0000 7fc8e0467740 20 client.16743 populate_metadata read hostname 'smithi033' 2024-05-15T14:11:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.530+0000 7fc8e0467740 1 --2- 172.21.15.33:0/207054539 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x555d56fb58c0 0x555d56fb7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.530+0000 7fc8e0467740 4 client.16743 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:11:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.530+0000 7fc8e0467740 1 -- 172.21.15.33:0/207054539 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555d56e9e360 con 0x555d56fb58c0 2024-05-15T14:11:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.530+0000 7fc8d6ffd640 1 --2- 172.21.15.33:0/207054539 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x555d56fb58c0 0x555d56fb7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.530+0000 7fc8d6ffd640 1 --2- 172.21.15.33:0/207054539 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x555d56fb58c0 0x555d56fb7ce0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=452851b11f767db3 server_cookie=843e626ea66952a2 in_seq=0 out_seq=0 2024-05-15T14:11:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.530+0000 7fc8be7fc640 10 client.16743 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:11:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.563+0000 7fc8be7fc640 1 -- 172.21.15.33:0/207054539 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555d56e9e360 con 0x555d56fb58c0 2024-05-15T14:11:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.563+0000 7fc8be7fc640 10 client.16743 handle_command_reply: tid=0 2024-05-15T14:11:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.563+0000 7fc8e0467740 1 client.16743 shutdown 2024-05-15T14:11:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.564+0000 7fc8e0467740 1 -- 172.21.15.33:0/207054539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d56f30210 msgr2=0x555d56f40c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.564+0000 7fc8e0467740 1 --2- 172.21.15.33:0/207054539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d56f30210 0x555d56f40c50 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00b23e0 tx=0x7fc8c00a95a0 comp rx=0 tx=0).stop 2024-05-15T14:11:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.564+0000 7fc8e0467740 1 -- 172.21.15.33:0/207054539 shutdown_connections 2024-05-15T14:11:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.564+0000 7fc8e0467740 1 --2- 172.21.15.33:0/207054539 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x555d56fb58c0 0x555d56fb7ce0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.564+0000 7fc8e0467740 1 --2- 172.21.15.33:0/207054539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555d56f3aad0 0x555d56f41190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.564+0000 7fc8be7fc640 0 client.16743 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:11:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.564+0000 7fc8e0467740 1 --2- 172.21.15.33:0/207054539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d56f30210 0x555d56f40c50 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.564+0000 7fc8e0467740 1 --2- 172.21.15.33:0/207054539 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d56f2f8f0 0x555d56f40710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.564+0000 7fc8e0467740 1 -- 172.21.15.33:0/207054539 >> 172.21.15.33:0/207054539 conn(0x555d56f2be00 msgr2=0x555d56f45660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.564+0000 7fc8e0467740 1 -- 172.21.15.33:0/207054539 shutdown_connections 2024-05-15T14:11:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.564+0000 7fc8e0467740 1 -- 172.21.15.33:0/207054539 wait complete. 2024-05-15T14:11:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.564+0000 7fc8e0467740 20 client.16743 trim_cache size 0 max 16384 2024-05-15T14:11:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:11:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:11:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:11:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.564+0000 7fc8dd746640 1 -- 172.21.15.33:0/578238146 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc8c40b2950 msgr2=0x7fc8c40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.564+0000 7fc8dd746640 1 --2- 172.21.15.33:0/578238146 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc8c40b2950 0x7fc8c40b4e10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8031440 tx=0x7fc8c8005890 comp rx=0 tx=0).stop 2024-05-15T14:11:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.564+0000 7fc8dd746640 1 -- 172.21.15.33:0/578238146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8d8109f90 msgr2=0x7fc8d819ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.564+0000 7fc8dd746640 1 --2- 172.21.15.33:0/578238146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8d8109f90 0x7fc8d819ba30 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc8c000d770 tx=0x7fc8c000dc30 comp rx=0 tx=0).stop 2024-05-15T14:11:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.564+0000 7fc8dd746640 1 -- 172.21.15.33:0/578238146 shutdown_connections 2024-05-15T14:11:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.564+0000 7fc8dd746640 1 --2- 172.21.15.33:0/578238146 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fc8c40b2950 0x7fc8c40b4e10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.564+0000 7fc8dd746640 1 --2- 172.21.15.33:0/578238146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8d819fdb0 0x7fc8d81a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.564+0000 7fc8dd746640 1 --2- 172.21.15.33:0/578238146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8d8109f90 0x7fc8d819ba30 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.564+0000 7fc8dd746640 1 --2- 172.21.15.33:0/578238146 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8d8104ec0 0x7fc8d819b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.564+0000 7fc8dd746640 1 -- 172.21.15.33:0/578238146 >> 172.21.15.33:0/578238146 conn(0x7fc8d8100740 msgr2=0x7fc8d8101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.565+0000 7fc8dd746640 1 -- 172.21.15.33:0/578238146 shutdown_connections 2024-05-15T14:11:49.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:11:49.565+0000 7fc8dd746640 1 -- 172.21.15.33:0/578238146 wait complete. 2024-05-15T14:11:49.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:49 smithi081 ceph-mon[36434]: pgmap v360: 97 pgs: 97 active+clean; 209 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 1.0 MiB/s wr, 34 op/s 2024-05-15T14:11:49.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:49 smithi033 ceph-mon[31650]: pgmap v360: 97 pgs: 97 active+clean; 209 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 1.0 MiB/s wr, 34 op/s 2024-05-15T14:11:50.083 INFO:tasks.workunit.client.0.smithi033.stdout: 1 651485 5.34 MB/sec execute 445 sec latency 18591.780 ms 2024-05-15T14:11:50.090 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:49 smithi052 ceph-mon[36190]: pgmap v360: 97 pgs: 97 active+clean; 209 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 1.0 MiB/s wr, 34 op/s 2024-05-15T14:11:50.996 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T14:11:51.083 INFO:tasks.workunit.client.0.smithi033.stdout: 1 651485 5.33 MB/sec execute 446 sec latency 19591.845 ms 2024-05-15T14:11:51.260 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:11:52.083 INFO:tasks.workunit.client.0.smithi033.stdout: 1 651485 5.32 MB/sec execute 447 sec latency 20591.950 ms 2024-05-15T14:11:52.657 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:52 smithi033 ceph-mon[31650]: pgmap v361: 97 pgs: 97 active+clean; 209 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 1.0 MiB/s wr, 34 op/s 2024-05-15T14:11:52.794 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:52 smithi052 ceph-mon[36190]: pgmap v361: 97 pgs: 97 active+clean; 209 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 1.0 MiB/s wr, 34 op/s 2024-05-15T14:11:52.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:52 smithi081 ceph-mon[36434]: pgmap v361: 97 pgs: 97 active+clean; 209 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 1.0 MiB/s wr, 34 op/s 2024-05-15T14:11:52.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.987+0000 7f7d9ebe2700 1 -- 172.21.15.33:0/1505220997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d981095b0 msgr2=0x7f7d98109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:52.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.987+0000 7f7d9ebe2700 1 --2- 172.21.15.33:0/1505220997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d981095b0 0x7f7d98109990 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f7d94009990 tx=0x7f7d94009ca0 comp rx=0 tx=0).stop 2024-05-15T14:11:52.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.987+0000 7f7d9ebe2700 1 -- 172.21.15.33:0/1505220997 shutdown_connections 2024-05-15T14:11:52.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.987+0000 7f7d9ebe2700 1 --2- 172.21.15.33:0/1505220997 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d9806c670 0x7f7d981103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:52.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.987+0000 7f7d9ebe2700 1 --2- 172.21.15.33:0/1505220997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d981095b0 0x7f7d98109990 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:52.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.987+0000 7f7d9ebe2700 1 --2- 172.21.15.33:0/1505220997 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d9806bcb0 0x7f7d9806c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:52.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.987+0000 7f7d9ebe2700 1 -- 172.21.15.33:0/1505220997 >> 172.21.15.33:0/1505220997 conn(0x7f7d980793f0 msgr2=0x7f7d98079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:52.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.987+0000 7f7d9ebe2700 1 -- 172.21.15.33:0/1505220997 shutdown_connections 2024-05-15T14:11:52.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.987+0000 7f7d9ebe2700 1 -- 172.21.15.33:0/1505220997 wait complete. 2024-05-15T14:11:52.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.988+0000 7f7d9ebe2700 1 Processor -- start 2024-05-15T14:11:52.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.988+0000 7f7d9ebe2700 1 -- start start 2024-05-15T14:11:52.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.988+0000 7f7d9ebe2700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d9806bcb0 0x7f7d98102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:52.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.988+0000 7f7d9ebe2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d9806c670 0x7f7d98102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:52.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.988+0000 7f7d9ebe2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d981095b0 0x7f7d98102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:52.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.988+0000 7f7d9ebe2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7d98113810 con 0x7f7d981095b0 2024-05-15T14:11:52.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.988+0000 7f7d9ebe2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7d98113690 con 0x7f7d9806c670 2024-05-15T14:11:52.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.988+0000 7f7d9ebe2700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f7d98113990 con 0x7f7d9806bcb0 2024-05-15T14:11:52.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.988+0000 7f7d9d17f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d981095b0 0x7f7d98102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:52.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.988+0000 7f7d9d17f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d981095b0 0x7f7d98102f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54308/0 (socket says 172.21.15.33:54308) 2024-05-15T14:11:52.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.988+0000 7f7d9d17f700 1 -- 172.21.15.33:0/1575638425 learned_addr learned my addr 172.21.15.33:0/1575638425 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:11:52.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.988+0000 7f7d9c97e700 1 --2- 172.21.15.33:0/1575638425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d9806bcb0 0x7f7d98102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:52.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.988+0000 7f7d8ffff700 1 --2- 172.21.15.33:0/1575638425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d9806c670 0x7f7d98102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:52.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.988+0000 7f7d9d17f700 1 -- 172.21.15.33:0/1575638425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d9806bcb0 msgr2=0x7f7d98102510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:52.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.988+0000 7f7d9d17f700 1 --2- 172.21.15.33:0/1575638425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d9806bcb0 0x7f7d98102510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:52.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.988+0000 7f7d9d17f700 1 -- 172.21.15.33:0/1575638425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d9806c670 msgr2=0x7f7d98102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:52.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.988+0000 7f7d9d17f700 1 --2- 172.21.15.33:0/1575638425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d9806c670 0x7f7d98102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:52.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.988+0000 7f7d9d17f700 1 -- 172.21.15.33:0/1575638425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d981a82f0 con 0x7f7d981095b0 2024-05-15T14:11:52.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.989+0000 7f7d9d17f700 1 --2- 172.21.15.33:0/1575638425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d981095b0 0x7f7d98102f90 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f7d9000b970 tx=0x7f7d9000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:52.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.989+0000 7f7d8dffb700 1 -- 172.21.15.33:0/1575638425 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9000d860 con 0x7f7d981095b0 2024-05-15T14:11:52.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.989+0000 7f7d8dffb700 1 -- 172.21.15.33:0/1575638425 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d9000e430 con 0x7f7d981095b0 2024-05-15T14:11:52.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.989+0000 7f7d8dffb700 1 -- 172.21.15.33:0/1575638425 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d90017660 con 0x7f7d981095b0 2024-05-15T14:11:52.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.989+0000 7f7d9ebe2700 1 -- 172.21.15.33:0/1575638425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d981a85e0 con 0x7f7d981095b0 2024-05-15T14:11:52.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.989+0000 7f7d9ebe2700 1 -- 172.21.15.33:0/1575638425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d981a8b20 con 0x7f7d981095b0 2024-05-15T14:11:52.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.990+0000 7f7d9ebe2700 1 -- 172.21.15.33:0/1575638425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d9810ba90 con 0x7f7d981095b0 2024-05-15T14:11:52.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.991+0000 7f7d8dffb700 1 -- 172.21.15.33:0/1575638425 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7d9000da00 con 0x7f7d981095b0 2024-05-15T14:11:52.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.992+0000 7f7d8dffb700 1 --2- 172.21.15.33:0/1575638425 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f7d800b29f0 0x7f7d800b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:52.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.992+0000 7f7d9c97e700 1 --2- 172.21.15.33:0/1575638425 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f7d800b29f0 0x7f7d800b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:52.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.992+0000 7f7d8dffb700 1 -- 172.21.15.33:0/1575638425 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7f7d900e2230 con 0x7f7d981095b0 2024-05-15T14:11:52.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.993+0000 7f7d9c97e700 1 --2- 172.21.15.33:0/1575638425 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f7d800b29f0 0x7f7d800b4eb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7d94009750 tx=0x7f7d94005fd0 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:52.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:52.997+0000 7f7d8dffb700 1 -- 172.21.15.33:0/1575638425 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7d90013030 con 0x7f7d981095b0 2024-05-15T14:11:53.083 INFO:tasks.workunit.client.0.smithi033.stdout: 1 651485 5.30 MB/sec execute 448 sec latency 21592.012 ms 2024-05-15T14:11:53.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:53.256+0000 7f7d9ebe2700 1 -- 172.21.15.33:0/1575638425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d98104c70 con 0x7f7d981095b0 2024-05-15T14:11:53.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:53.257+0000 7f7d8dffb700 1 -- 172.21.15.33:0/1575638425 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11861 (secure 0 0 0) 0x7f7d98104c70 con 0x7f7d981095b0 2024-05-15T14:11:53.259 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T14:11:53.259 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2030117633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2030117633},{"type":"v1","addr":"172.21.15.33:6839","nonce":2030117633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1918979061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1918979061},{"type":"v1","addr":"172.21.15.33:6841","nonce":1918979061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T13:57:19.056417+0000","modified":"2024-05-15T14:11:03.273793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24463,"mds_2":24461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/1629697841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1629697841},{"type":"v1","addr":"172.21.15.52:6837","nonce":1629697841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.52:6839/2696044471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2696044471},{"type":"v1","addr":"172.21.15.52:6839","nonce":2696044471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.81:6839/1554395940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":1554395940},{"type":"v1","addr":"172.21.15.81:6839","nonce":1554395940}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26206":{"gid":26206,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/3674797952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3674797952},{"type":"v1","addr":"172.21.15.81:6835","nonce":3674797952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24463,24478,24461]},"id":1}]} 2024-05-15T14:11:53.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:53.259+0000 7f7d9ebe2700 1 -- 172.21.15.33:0/1575638425 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f7d800b29f0 msgr2=0x7f7d800b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:53.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:53.259+0000 7f7d9ebe2700 1 --2- 172.21.15.33:0/1575638425 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f7d800b29f0 0x7f7d800b4eb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7d94009750 tx=0x7f7d94005fd0 comp rx=0 tx=0).stop 2024-05-15T14:11:53.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:53.259+0000 7f7d9ebe2700 1 -- 172.21.15.33:0/1575638425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d981095b0 msgr2=0x7f7d98102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:53.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:53.259+0000 7f7d9ebe2700 1 --2- 172.21.15.33:0/1575638425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d981095b0 0x7f7d98102f90 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f7d9000b970 tx=0x7f7d9000bc80 comp rx=0 tx=0).stop 2024-05-15T14:11:53.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:53.259+0000 7f7d9ebe2700 1 -- 172.21.15.33:0/1575638425 shutdown_connections 2024-05-15T14:11:53.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:53.259+0000 7f7d9ebe2700 1 --2- 172.21.15.33:0/1575638425 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f7d800b29f0 0x7f7d800b4eb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:53.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:53.259+0000 7f7d9ebe2700 1 --2- 172.21.15.33:0/1575638425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d981095b0 0x7f7d98102f90 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:53.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:53.259+0000 7f7d9ebe2700 1 --2- 172.21.15.33:0/1575638425 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d9806bcb0 0x7f7d98102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:53.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:53.259+0000 7f7d9ebe2700 1 --2- 172.21.15.33:0/1575638425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d9806c670 0x7f7d98102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:53.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:53.259+0000 7f7d9ebe2700 1 -- 172.21.15.33:0/1575638425 >> 172.21.15.33:0/1575638425 conn(0x7f7d980793f0 msgr2=0x7f7d98101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:53.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:53.259+0000 7f7d9ebe2700 1 -- 172.21.15.33:0/1575638425 shutdown_connections 2024-05-15T14:11:53.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:53.259+0000 7f7d9ebe2700 1 -- 172.21.15.33:0/1575638425 wait complete. 2024-05-15T14:11:53.262 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 39 2024-05-15T14:11:53.663 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:53 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/1575638425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:11:53.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:53 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/1575638425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:11:53.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:53 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/1575638425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:11:53.937 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24478 quiesce db --set-id 69488f58 --release --await' 2024-05-15T14:11:53.937 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24478 quiesce db --set-id 69488f58 --release --await 2024-05-15T14:11:54.083 INFO:tasks.workunit.client.0.smithi033.stdout: 1 651485 5.29 MB/sec execute 449 sec latency 22592.060 ms 2024-05-15T14:11:54.202 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:11:54.669 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:54 smithi033 ceph-mon[31650]: pgmap v362: 97 pgs: 97 active+clean; 198 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 1.1 MiB/s wr, 56 op/s 2024-05-15T14:11:54.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:54 smithi052 ceph-mon[36190]: pgmap v362: 97 pgs: 97 active+clean; 198 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 1.1 MiB/s wr, 56 op/s 2024-05-15T14:11:54.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:54 smithi081 ceph-mon[36434]: pgmap v362: 97 pgs: 97 active+clean; 198 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 1.1 MiB/s wr, 56 op/s 2024-05-15T14:11:55.083 INFO:tasks.workunit.client.0.smithi033.stdout: 1 651485 5.28 MB/sec execute 450 sec latency 23592.144 ms 2024-05-15T14:11:56.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.029+0000 7f4b8c6f5700 1 -- 172.21.15.33:0/1191873520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b84102440 msgr2=0x7f4b841028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:56.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.029+0000 7f4b8c6f5700 1 --2- 172.21.15.33:0/1191873520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b84102440 0x7f4b841028c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f4b78009990 tx=0x7f4b78009ca0 comp rx=0 tx=0).stop 2024-05-15T14:11:56.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.029+0000 7f4b8c6f5700 1 -- 172.21.15.33:0/1191873520 shutdown_connections 2024-05-15T14:11:56.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.029+0000 7f4b8c6f5700 1 --2- 172.21.15.33:0/1191873520 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b84102e00 0x7f4b8410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:56.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.029+0000 7f4b8c6f5700 1 --2- 172.21.15.33:0/1191873520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b84108520 0x7f4b84108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:56.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.029+0000 7f4b8c6f5700 1 --2- 172.21.15.33:0/1191873520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b84102440 0x7f4b841028c0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:56.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.029+0000 7f4b8c6f5700 1 -- 172.21.15.33:0/1191873520 >> 172.21.15.33:0/1191873520 conn(0x7f4b840793f0 msgr2=0x7f4b84079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:56.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.029+0000 7f4b8c6f5700 1 -- 172.21.15.33:0/1191873520 shutdown_connections 2024-05-15T14:11:56.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.029+0000 7f4b8c6f5700 1 -- 172.21.15.33:0/1191873520 wait complete. 2024-05-15T14:11:56.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.030+0000 7f4b8c6f5700 1 Processor -- start 2024-05-15T14:11:56.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.030+0000 7f4b8c6f5700 1 -- start start 2024-05-15T14:11:56.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.030+0000 7f4b8c6f5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b84102440 0x7f4b84104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:56.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.030+0000 7f4b8c6f5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b84102e00 0x7f4b841045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:56.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.030+0000 7f4b8c6f5700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b84108520 0x7f4b84104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:56.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.030+0000 7f4b8c6f5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4b841125d0 con 0x7f4b84102440 2024-05-15T14:11:56.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.030+0000 7f4b8c6f5700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4b84112450 con 0x7f4b84102e00 2024-05-15T14:11:56.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.030+0000 7f4b8c6f5700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4b84112750 con 0x7f4b84108520 2024-05-15T14:11:56.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.030+0000 7f4b8a491700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b84102440 0x7f4b84104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.030+0000 7f4b8a491700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b84102440 0x7f4b84104080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54320/0 (socket says 172.21.15.33:54320) 2024-05-15T14:11:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.030+0000 7f4b8a491700 1 -- 172.21.15.33:0/2828884012 learned_addr learned my addr 172.21.15.33:0/2828884012 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:11:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.031+0000 7f4b8ac92700 1 --2- 172.21.15.33:0/2828884012 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b84108520 0x7f4b84104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.031+0000 7f4b89c90700 1 --2- 172.21.15.33:0/2828884012 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b84102e00 0x7f4b841045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.031+0000 7f4b8a491700 1 -- 172.21.15.33:0/2828884012 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b84108520 msgr2=0x7f4b84104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.031+0000 7f4b8a491700 1 --2- 172.21.15.33:0/2828884012 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b84108520 0x7f4b84104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.031+0000 7f4b8a491700 1 -- 172.21.15.33:0/2828884012 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b84102e00 msgr2=0x7f4b841045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.031+0000 7f4b8a491700 1 --2- 172.21.15.33:0/2828884012 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b84102e00 0x7f4b841045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.031+0000 7f4b8a491700 1 -- 172.21.15.33:0/2828884012 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b841a8180 con 0x7f4b84102440 2024-05-15T14:11:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.031+0000 7f4b8ac92700 1 --2- 172.21.15.33:0/2828884012 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b84108520 0x7f4b84104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:11:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.031+0000 7f4b8a491700 1 --2- 172.21.15.33:0/2828884012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b84102440 0x7f4b84104080 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f4b7400ada0 tx=0x7f4b7400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.031+0000 7f4b837fe700 1 -- 172.21.15.33:0/2828884012 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b74017020 con 0x7f4b84102440 2024-05-15T14:11:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.031+0000 7f4b837fe700 1 -- 172.21.15.33:0/2828884012 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b7400ce30 con 0x7f4b84102440 2024-05-15T14:11:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.031+0000 7f4b837fe700 1 -- 172.21.15.33:0/2828884012 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b740055a0 con 0x7f4b84102440 2024-05-15T14:11:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.031+0000 7f4b8c6f5700 1 -- 172.21.15.33:0/2828884012 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b841a8470 con 0x7f4b84102440 2024-05-15T14:11:56.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.031+0000 7f4b8c6f5700 1 -- 172.21.15.33:0/2828884012 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b841a89b0 con 0x7f4b84102440 2024-05-15T14:11:56.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.035+0000 7f4b837fe700 1 -- 172.21.15.33:0/2828884012 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4b74004c40 con 0x7f4b84102440 2024-05-15T14:11:56.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.035+0000 7f4b837fe700 1 --2- 172.21.15.33:0/2828884012 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f4b6c0b29f0 0x7f4b6c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:56.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.036+0000 7f4b89c90700 1 --2- 172.21.15.33:0/2828884012 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f4b6c0b29f0 0x7f4b6c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:56.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.036+0000 7f4b96f8bb80 1 Processor -- start 2024-05-15T14:11:56.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.036+0000 7f4b837fe700 1 -- 172.21.15.33:0/2828884012 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7f4b740e1df0 con 0x7f4b84102440 2024-05-15T14:11:56.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.036+0000 7f4b96f8bb80 1 -- start start 2024-05-15T14:11:56.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.036+0000 7f4b89c90700 1 --2- 172.21.15.33:0/2828884012 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f4b6c0b29f0 0x7f4b6c0b4eb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4b78006010 tx=0x7f4b7801a040 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:56.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.036+0000 7f4b96f8bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588162d7da0 0x5588162d8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:56.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.036+0000 7f4b96f8bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588162d86c0 0x5588162d24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:56.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.036+0000 7f4b96f8bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588162d2ad0 0x5588162d2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:56.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.036+0000 7f4b96f8bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558816230d90 con 0x5588162d7da0 2024-05-15T14:11:56.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.036+0000 7f4b96f8bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558816220930 con 0x5588162d86c0 2024-05-15T14:11:56.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.036+0000 7f4b96f8bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x558816224970 con 0x5588162d2ad0 2024-05-15T14:11:56.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.036+0000 7f4b8a491700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588162d7da0 0x5588162d8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:56.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.036+0000 7f4b8a491700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588162d7da0 0x5588162d8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54324/0 (socket says 172.21.15.33:54324) 2024-05-15T14:11:56.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.036+0000 7f4b8a491700 1 -- 172.21.15.33:0/1984591964 learned_addr learned my addr 172.21.15.33:0/1984591964 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:11:56.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.036+0000 7f4b89c90700 1 --2- 172.21.15.33:0/1984591964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588162d86c0 0x5588162d24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:56.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.037+0000 7f4b8ac92700 1 --2- 172.21.15.33:0/1984591964 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588162d2ad0 0x5588162d2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:56.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.037+0000 7f4b8a491700 1 -- 172.21.15.33:0/1984591964 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588162d2ad0 msgr2=0x5588162d2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:56.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.037+0000 7f4b8a491700 1 --2- 172.21.15.33:0/1984591964 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588162d2ad0 0x5588162d2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:56.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.037+0000 7f4b8a491700 1 -- 172.21.15.33:0/1984591964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588162d86c0 msgr2=0x5588162d24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:56.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.037+0000 7f4b8a491700 1 --2- 172.21.15.33:0/1984591964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588162d86c0 0x5588162d24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:56.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.037+0000 7f4b8a491700 1 -- 172.21.15.33:0/1984591964 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588161709d0 con 0x5588162d7da0 2024-05-15T14:11:56.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.037+0000 7f4b8a491700 1 --2- 172.21.15.33:0/1984591964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588162d7da0 0x5588162d8180 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f4b740a90e0 tx=0x7f4b740a8ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3e63e2a129de85 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:56.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.037+0000 7f4b80ff9700 1 -- 172.21.15.33:0/1984591964 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b740c3070 con 0x5588162d7da0 2024-05-15T14:11:56.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.037+0000 7f4b80ff9700 1 -- 172.21.15.33:0/1984591964 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b740ad500 con 0x5588162d7da0 2024-05-15T14:11:56.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.037+0000 7f4b80ff9700 1 -- 172.21.15.33:0/1984591964 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b740b1a90 con 0x5588162d7da0 2024-05-15T14:11:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.038+0000 7f4b96f8bb80 1 -- 172.21.15.33:0/1984591964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588162d7da0 msgr2=0x5588162d8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.038+0000 7f4b96f8bb80 1 --2- 172.21.15.33:0/1984591964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588162d7da0 0x5588162d8180 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f4b740a90e0 tx=0x7f4b740a8ce0 comp rx=0 tx=0).stop 2024-05-15T14:11:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.038+0000 7f4b96f8bb80 1 -- 172.21.15.33:0/1984591964 shutdown_connections 2024-05-15T14:11:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.038+0000 7f4b96f8bb80 1 --2- 172.21.15.33:0/1984591964 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588162d2ad0 0x5588162d2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.038+0000 7f4b96f8bb80 1 --2- 172.21.15.33:0/1984591964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588162d7da0 0x5588162d8180 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.038+0000 7f4b96f8bb80 1 --2- 172.21.15.33:0/1984591964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588162d86c0 0x5588162d24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.038+0000 7f4b96f8bb80 1 -- 172.21.15.33:0/1984591964 >> 172.21.15.33:0/1984591964 conn(0x5588161fd350 msgr2=0x5588162ccce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.038+0000 7f4b96f8bb80 1 -- 172.21.15.33:0/1984591964 shutdown_connections 2024-05-15T14:11:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.038+0000 7f4b96f8bb80 1 -- 172.21.15.33:0/1984591964 wait complete. 2024-05-15T14:11:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.038+0000 7f4b96f8bb80 1 -- start start 2024-05-15T14:11:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.039+0000 7f4b96f8bb80 1 Processor -- start 2024-05-15T14:11:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.039+0000 7f4b96f8bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588162d2ad0 0x5588162de450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.039+0000 7f4b96f8bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588162d7da0 0x5588162db7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.039+0000 7f4b96f8bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588162d86c0 0x5588162dbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.039+0000 7f4b96f8bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5588161270a0 con 0x5588162d86c0 2024-05-15T14:11:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.039+0000 7f4b96f8bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5588161c28c0 con 0x5588162d7da0 2024-05-15T14:11:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.039+0000 7f4b96f8bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x5588162df100 con 0x5588162d2ad0 2024-05-15T14:11:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.039+0000 7f4b96f8bb80 10 client.0 _ll_register_callbacks cb 0x5588161b7680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:11:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.039+0000 7f4b8ac92700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588162d86c0 0x5588162dbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.039+0000 7f4b8ac92700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588162d86c0 0x5588162dbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54342/0 (socket says 172.21.15.33:54342) 2024-05-15T14:11:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.039+0000 7f4b8ac92700 1 -- 172.21.15.33:0/2274534430 learned_addr learned my addr 172.21.15.33:0/2274534430 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:11:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.039+0000 7f4b89c90700 1 --2- 172.21.15.33:0/2274534430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588162d7da0 0x5588162db7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.039+0000 7f4b8a491700 1 --2- 172.21.15.33:0/2274534430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588162d2ad0 0x5588162de450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.039+0000 7f4b8ac92700 1 -- 172.21.15.33:0/2274534430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588162d2ad0 msgr2=0x5588162de450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:56.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.039+0000 7f4b8ac92700 1 --2- 172.21.15.33:0/2274534430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588162d2ad0 0x5588162de450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:56.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.039+0000 7f4b8ac92700 1 -- 172.21.15.33:0/2274534430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588162d7da0 msgr2=0x5588162db7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:56.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.039+0000 7f4b8ac92700 1 --2- 172.21.15.33:0/2274534430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588162d7da0 0x5588162db7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:56.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.039+0000 7f4b8ac92700 1 -- 172.21.15.33:0/2274534430 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588161bf790 con 0x5588162d86c0 2024-05-15T14:11:56.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.039+0000 7f4b8a491700 1 --2- 172.21.15.33:0/2274534430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588162d2ad0 0x5588162de450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:11:56.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.039+0000 7f4b8ac92700 1 --2- 172.21.15.33:0/2274534430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588162d86c0 0x5588162dbd30 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c010a40 tx=0x7f4b7c010e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b875b7198ebd0c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:56.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.039+0000 7f4b69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:11:56.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.040+0000 7f4b69ffb700 1 -- 172.21.15.33:0/2274534430 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c00ea50 con 0x5588162d86c0 2024-05-15T14:11:56.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.040+0000 7f4b69ffb700 1 -- 172.21.15.33:0/2274534430 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7f4b7c00ebf0 con 0x5588162d86c0 2024-05-15T14:11:56.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.040+0000 7f4b96f8bb80 1 -- 172.21.15.33:0/2274534430 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b740ad500 con 0x5588162d86c0 2024-05-15T14:11:56.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.040+0000 7f4b69ffb700 1 -- 172.21.15.33:0/2274534430 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4b7c00ee30 con 0x5588162d86c0 2024-05-15T14:11:56.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.040+0000 7f4b96f8bb80 10 client.16761 fetch_fsmap learned FSMap version 39 2024-05-15T14:11:56.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.040+0000 7f4b96f8bb80 1 -- 172.21.15.33:0/2274534430 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5588162e98f0 con 0x5588162d86c0 2024-05-15T14:11:56.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.040+0000 7f4b69ffb700 1 -- 172.21.15.33:0/2274534430 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b740ad500 con 0x5588162d86c0 2024-05-15T14:11:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.040+0000 7f4b69ffb700 1 -- 172.21.15.33:0/2274534430 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f4b7c011510 con 0x5588162d86c0 2024-05-15T14:11:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.040+0000 7f4b96f8bb80 10 client.16761 fetch_fsmap finished waiting for FSMap version 39 2024-05-15T14:11:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.040+0000 7f4b96f8bb80 10 client.16761 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:11:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.040+0000 7f4b96f8bb80 20 client.16761 populate_metadata read hostname 'smithi033' 2024-05-15T14:11:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.040+0000 7f4b96f8bb80 1 --2- 172.21.15.33:0/2274534430 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5588162ebf30 0x5588162ee380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.040+0000 7f4b96f8bb80 4 client.16761 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:11:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.040+0000 7f4b96f8bb80 1 -- 172.21.15.33:0/2274534430 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55881629e460 con 0x5588162ebf30 2024-05-15T14:11:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.041+0000 7f4b8a491700 1 --2- 172.21.15.33:0/2274534430 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5588162ebf30 0x5588162ee380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.041+0000 7f4b69ffb700 10 client.16761 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:11:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.041+0000 7f4b8a491700 1 --2- 172.21.15.33:0/2274534430 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5588162ebf30 0x5588162ee380 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=986462696b62d6db server_cookie=867098387a733f47 in_seq=0 out_seq=0 2024-05-15T14:11:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.042+0000 7f4b69ffb700 1 -- 172.21.15.33:0/2274534430 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55881629e460 con 0x5588162ebf30 2024-05-15T14:11:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.042+0000 7f4b69ffb700 10 client.16761 handle_command_reply: tid=0 2024-05-15T14:11:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.042+0000 7f4b96f8bb80 1 client.16761 shutdown 2024-05-15T14:11:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.042+0000 7f4b96f8bb80 1 -- 172.21.15.33:0/2274534430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588162d86c0 msgr2=0x5588162dbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.042+0000 7f4b96f8bb80 1 --2- 172.21.15.33:0/2274534430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588162d86c0 0x5588162dbd30 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c010a40 tx=0x7f4b7c010e00 comp rx=0 tx=0).stop 2024-05-15T14:11:56.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.042+0000 7f4b96f8bb80 1 -- 172.21.15.33:0/2274534430 shutdown_connections 2024-05-15T14:11:56.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.042+0000 7f4b96f8bb80 1 --2- 172.21.15.33:0/2274534430 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5588162ebf30 0x5588162ee380 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:56.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.042+0000 7f4b96f8bb80 1 --2- 172.21.15.33:0/2274534430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588162d86c0 0x5588162dbd30 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:56.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.042+0000 7f4b96f8bb80 1 --2- 172.21.15.33:0/2274534430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588162d2ad0 0x5588162de450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:56.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.042+0000 7f4b96f8bb80 1 --2- 172.21.15.33:0/2274534430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588162d7da0 0x5588162db7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:56.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.042+0000 7f4b96f8bb80 1 -- 172.21.15.33:0/2274534430 >> 172.21.15.33:0/2274534430 conn(0x5588161fd350 msgr2=0x5588162c77d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:56.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.042+0000 7f4b69ffb700 0 client.16761 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:11:56.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.042+0000 7f4b96f8bb80 1 -- 172.21.15.33:0/2274534430 shutdown_connections 2024-05-15T14:11:56.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.042+0000 7f4b96f8bb80 1 -- 172.21.15.33:0/2274534430 wait complete. 2024-05-15T14:11:56.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.042+0000 7f4b96f8bb80 20 client.16761 trim_cache size 0 max 16384 2024-05-15T14:11:56.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.062+0000 7f4b96f8bb80 1 Processor -- start 2024-05-15T14:11:56.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.062+0000 7f4b96f8bb80 1 -- start start 2024-05-15T14:11:56.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.063+0000 7f4b96f8bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588162d2ad0 0x5588162dbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:56.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.063+0000 7f4b96f8bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588162d7da0 0x5588162d9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:56.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.063+0000 7f4b96f8bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588162d86c0 0x5588162c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:56.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.063+0000 7f4b96f8bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4b841b2a40 con 0x5588162d86c0 2024-05-15T14:11:56.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.063+0000 7f4b96f8bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4b841b0270 con 0x5588162d2ad0 2024-05-15T14:11:56.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.063+0000 7f4b96f8bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4b841ae0e0 con 0x5588162d7da0 2024-05-15T14:11:56.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.063+0000 7f4b8ac92700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588162d86c0 0x5588162c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:56.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.063+0000 7f4b8ac92700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588162d86c0 0x5588162c78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54356/0 (socket says 172.21.15.33:54356) 2024-05-15T14:11:56.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.063+0000 7f4b8ac92700 1 -- 172.21.15.33:0/153891430 learned_addr learned my addr 172.21.15.33:0/153891430 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:11:56.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.063+0000 7f4b8a491700 1 --2- 172.21.15.33:0/153891430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588162d2ad0 0x5588162dbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:56.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.063+0000 7f4b89c90700 1 --2- 172.21.15.33:0/153891430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588162d7da0 0x5588162d9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:56.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.063+0000 7f4b8ac92700 1 -- 172.21.15.33:0/153891430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588162d7da0 msgr2=0x5588162d9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:56.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.063+0000 7f4b8ac92700 1 --2- 172.21.15.33:0/153891430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588162d7da0 0x5588162d9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:56.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.063+0000 7f4b8ac92700 1 -- 172.21.15.33:0/153891430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588162d2ad0 msgr2=0x5588162dbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:56.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.063+0000 7f4b8ac92700 1 --2- 172.21.15.33:0/153891430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588162d2ad0 0x5588162dbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:56.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.063+0000 7f4b8ac92700 1 -- 172.21.15.33:0/153891430 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588162da7d0 con 0x5588162d86c0 2024-05-15T14:11:56.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.063+0000 7f4b8a491700 1 --2- 172.21.15.33:0/153891430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588162d2ad0 0x5588162dbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:11:56.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.063+0000 7f4b8ac92700 1 --2- 172.21.15.33:0/153891430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588162d86c0 0x5588162c78e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c010710 tx=0x7f4b7c00cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d0bf1676237cd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:56.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.063+0000 7f4b6bfff700 1 -- 172.21.15.33:0/153891430 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c029070 con 0x5588162d86c0 2024-05-15T14:11:56.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.063+0000 7f4b6bfff700 1 -- 172.21.15.33:0/153891430 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b7c00ec50 con 0x5588162d86c0 2024-05-15T14:11:56.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.063+0000 7f4b6bfff700 1 -- 172.21.15.33:0/153891430 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c0116a0 con 0x5588162d86c0 2024-05-15T14:11:56.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.064+0000 7f4b96f8bb80 1 -- 172.21.15.33:0/153891430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588162d86c0 msgr2=0x5588162c78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:56.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.064+0000 7f4b96f8bb80 1 --2- 172.21.15.33:0/153891430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588162d86c0 0x5588162c78e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c010710 tx=0x7f4b7c00cf30 comp rx=0 tx=0).stop 2024-05-15T14:11:56.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.064+0000 7f4b96f8bb80 1 -- 172.21.15.33:0/153891430 shutdown_connections 2024-05-15T14:11:56.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.064+0000 7f4b96f8bb80 1 --2- 172.21.15.33:0/153891430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588162d86c0 0x5588162c78e0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:56.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.064+0000 7f4b96f8bb80 1 --2- 172.21.15.33:0/153891430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588162d2ad0 0x5588162dbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:56.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.064+0000 7f4b96f8bb80 1 --2- 172.21.15.33:0/153891430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588162d7da0 0x5588162d9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:56.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.064+0000 7f4b96f8bb80 1 -- 172.21.15.33:0/153891430 >> 172.21.15.33:0/153891430 conn(0x5588161fd350 msgr2=0x5588162d69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:56.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.064+0000 7f4b96f8bb80 1 -- 172.21.15.33:0/153891430 shutdown_connections 2024-05-15T14:11:56.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.064+0000 7f4b96f8bb80 1 -- 172.21.15.33:0/153891430 wait complete. 2024-05-15T14:11:56.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.064+0000 7f4b96f8bb80 1 -- start start 2024-05-15T14:11:56.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.065+0000 7f4b96f8bb80 1 Processor -- start 2024-05-15T14:11:56.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.065+0000 7f4b96f8bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588162d2ad0 0x5588162e1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:56.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.065+0000 7f4b96f8bb80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588162d7da0 0x5588162e18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:56.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.065+0000 7f4b96f8bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588162d86c0 0x5588162e1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:56.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.065+0000 7f4b96f8bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4b841ae730 con 0x5588162d2ad0 2024-05-15T14:11:56.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.065+0000 7f4b96f8bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4b841afba0 con 0x5588162d86c0 2024-05-15T14:11:56.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.065+0000 7f4b96f8bb80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4b841aee00 con 0x5588162d7da0 2024-05-15T14:11:56.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.065+0000 7f4b96f8bb80 10 client.0 _ll_register_callbacks cb 0x5588161b7680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:11:56.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.065+0000 7f4b8a491700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588162d2ad0 0x5588162e1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:56.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.065+0000 7f4b8a491700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588162d2ad0 0x5588162e1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54364/0 (socket says 172.21.15.33:54364) 2024-05-15T14:11:56.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.065+0000 7f4b8a491700 1 -- 172.21.15.33:0/3849577213 learned_addr learned my addr 172.21.15.33:0/3849577213 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:11:56.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.065+0000 7f4b89c90700 1 --2- 172.21.15.33:0/3849577213 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588162d7da0 0x5588162e18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:56.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.065+0000 7f4b8ac92700 1 --2- 172.21.15.33:0/3849577213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588162d86c0 0x5588162e1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:56.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.065+0000 7f4b8a491700 1 -- 172.21.15.33:0/3849577213 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588162d7da0 msgr2=0x5588162e18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:56.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.065+0000 7f4b8a491700 1 --2- 172.21.15.33:0/3849577213 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588162d7da0 0x5588162e18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:56.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.065+0000 7f4b8a491700 1 -- 172.21.15.33:0/3849577213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588162d86c0 msgr2=0x5588162e1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:11:56.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.065+0000 7f4b8a491700 1 --2- 172.21.15.33:0/3849577213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588162d86c0 0x5588162e1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:56.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.065+0000 7f4b8a491700 1 -- 172.21.15.33:0/3849577213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588163568e0 con 0x5588162d2ad0 2024-05-15T14:11:56.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.066+0000 7f4b8a491700 1 --2- 172.21.15.33:0/3849577213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588162d2ad0 0x5588162e1360 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f4b7400a0a0 tx=0x7f4b740a8990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2629d5dc708851af server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:11:56.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.066+0000 7f4b69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:11:56.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.066+0000 7f4b69ffb700 1 -- 172.21.15.33:0/3849577213 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b740bd060 con 0x5588162d2ad0 2024-05-15T14:11:56.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.066+0000 7f4b69ffb700 1 -- 172.21.15.33:0/3849577213 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7f4b740a93b0 con 0x5588162d2ad0 2024-05-15T14:11:56.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.066+0000 7f4b96f8bb80 1 -- 172.21.15.33:0/3849577213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b7c00ec50 con 0x5588162d2ad0 2024-05-15T14:11:56.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.066+0000 7f4b69ffb700 1 -- 172.21.15.33:0/3849577213 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4b740b8c70 con 0x5588162d2ad0 2024-05-15T14:11:56.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.066+0000 7f4b96f8bb80 10 client.16767 fetch_fsmap learned FSMap version 39 2024-05-15T14:11:56.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.066+0000 7f4b96f8bb80 1 -- 172.21.15.33:0/3849577213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558816357be0 con 0x5588162d2ad0 2024-05-15T14:11:56.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.066+0000 7f4b69ffb700 1 -- 172.21.15.33:0/3849577213 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b7c00ec50 con 0x5588162d2ad0 2024-05-15T14:11:56.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.066+0000 7f4b69ffb700 1 -- 172.21.15.33:0/3849577213 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f4b74012070 con 0x5588162d2ad0 2024-05-15T14:11:56.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.066+0000 7f4b96f8bb80 10 client.16767 fetch_fsmap finished waiting for FSMap version 39 2024-05-15T14:11:56.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.066+0000 7f4b96f8bb80 10 client.16767 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:11:56.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.066+0000 7f4b96f8bb80 20 client.16767 populate_metadata read hostname 'smithi033' 2024-05-15T14:11:56.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.066+0000 7f4b96f8bb80 1 --2- 172.21.15.33:0/3849577213 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558816357e70 0x55881635a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:11:56.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.066+0000 7f4b96f8bb80 4 client.16767 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "69488f58", "release": true, "await": true}] 2024-05-15T14:11:56.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.066+0000 7f4b96f8bb80 1 -- 172.21.15.33:0/3849577213 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "quiesce db", "set_id": "69488f58", "release": true, "await": true}) -- 0x558816232ce0 con 0x558816357e70 2024-05-15T14:11:56.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.067+0000 7f4b89c90700 1 --2- 172.21.15.33:0/3849577213 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558816357e70 0x55881635a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:11:56.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.067+0000 7f4b89c90700 1 --2- 172.21.15.33:0/3849577213 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558816357e70 0x55881635a260 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6479230607f1e10a server_cookie=9432b789ca970f73 in_seq=0 out_seq=0 2024-05-15T14:11:56.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.067+0000 7f4b69ffb700 10 client.16767 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:11:56.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.075+0000 7f4b69ffb700 1 -- 172.21.15.33:0/3849577213 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558816232ce0 con 0x558816357e70 2024-05-15T14:11:56.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.075+0000 7f4b69ffb700 10 client.16767 handle_command_reply: tid=0 2024-05-15T14:11:56.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.075+0000 7f4b96f8bb80 1 client.16767 shutdown 2024-05-15T14:11:56.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.075+0000 7f4b96f8bb80 1 -- 172.21.15.33:0/3849577213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588162d2ad0 msgr2=0x5588162e1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:56.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.075+0000 7f4b96f8bb80 1 --2- 172.21.15.33:0/3849577213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588162d2ad0 0x5588162e1360 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f4b7400a0a0 tx=0x7f4b740a8990 comp rx=0 tx=0).stop 2024-05-15T14:11:56.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.075+0000 7f4b96f8bb80 1 -- 172.21.15.33:0/3849577213 shutdown_connections 2024-05-15T14:11:56.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.075+0000 7f4b96f8bb80 1 --2- 172.21.15.33:0/3849577213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588162d86c0 0x5588162e1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:56.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.075+0000 7f4b96f8bb80 1 --2- 172.21.15.33:0/3849577213 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x558816357e70 0x55881635a260 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:56.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.075+0000 7f4b96f8bb80 1 --2- 172.21.15.33:0/3849577213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5588162d2ad0 0x5588162e1360 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:56.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.075+0000 7f4b96f8bb80 1 --2- 172.21.15.33:0/3849577213 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5588162d7da0 0x5588162e18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:56.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.075+0000 7f4b96f8bb80 1 -- 172.21.15.33:0/3849577213 >> 172.21.15.33:0/3849577213 conn(0x5588161fd350 msgr2=0x5588162d91b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:56.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.075+0000 7f4b69ffb700 0 client.16767 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:11:56.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.075+0000 7f4b96f8bb80 1 -- 172.21.15.33:0/3849577213 shutdown_connections 2024-05-15T14:11:56.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.075+0000 7f4b96f8bb80 1 -- 172.21.15.33:0/3849577213 wait complete. 2024-05-15T14:11:56.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.075+0000 7f4b96f8bb80 20 client.16767 trim_cache size 0 max 16384 2024-05-15T14:11:56.079 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T14:11:56.079 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 39, 2024-05-15T14:11:56.080 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24478, 2024-05-15T14:11:56.080 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 40, 2024-05-15T14:11:56.080 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-05-15T14:11:56.080 INFO:teuthology.orchestra.run.smithi033.stdout: "69488f58": { 2024-05-15T14:11:56.080 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 40, 2024-05-15T14:11:56.080 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-05-15T14:11:56.080 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:11:56.080 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-05-15T14:11:56.080 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:11:56.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:11:56.080 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 90.0, 2024-05-15T14:11:56.080 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 145.3, 2024-05-15T14:11:56.080 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-05-15T14:11:56.080 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-05-15T14:11:56.081 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-05-15T14:11:56.081 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-05-15T14:11:56.081 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-05-15T14:11:56.081 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-05-15T14:11:56.081 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:11:56.081 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:11:56.081 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:11:56.081 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:11:56.081 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:11:56.081 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T14:11:56.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.076+0000 7f4b8c6f5700 1 -- 172.21.15.33:0/2828884012 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f4b6c0b29f0 msgr2=0x7f4b6c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:56.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.076+0000 7f4b8c6f5700 1 --2- 172.21.15.33:0/2828884012 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f4b6c0b29f0 0x7f4b6c0b4eb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4b78006010 tx=0x7f4b7801a040 comp rx=0 tx=0).stop 2024-05-15T14:11:56.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.076+0000 7f4b8c6f5700 1 -- 172.21.15.33:0/2828884012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b84102440 msgr2=0x7f4b84104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:11:56.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.076+0000 7f4b8c6f5700 1 --2- 172.21.15.33:0/2828884012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b84102440 0x7f4b84104080 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f4b7400ada0 tx=0x7f4b7400c330 comp rx=0 tx=0).stop 2024-05-15T14:11:56.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.076+0000 7f4b8c6f5700 1 -- 172.21.15.33:0/2828884012 shutdown_connections 2024-05-15T14:11:56.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.076+0000 7f4b8c6f5700 1 --2- 172.21.15.33:0/2828884012 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f4b6c0b29f0 0x7f4b6c0b4eb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:56.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.076+0000 7f4b8c6f5700 1 --2- 172.21.15.33:0/2828884012 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4b84108520 0x7f4b84104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:56.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.076+0000 7f4b8c6f5700 1 --2- 172.21.15.33:0/2828884012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b84102440 0x7f4b84104080 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:56.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.076+0000 7f4b8c6f5700 1 --2- 172.21.15.33:0/2828884012 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b84102e00 0x7f4b841045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:11:56.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.076+0000 7f4b8c6f5700 1 -- 172.21.15.33:0/2828884012 >> 172.21.15.33:0/2828884012 conn(0x7f4b840793f0 msgr2=0x7f4b8406b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:11:56.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.076+0000 7f4b8c6f5700 1 -- 172.21.15.33:0/2828884012 shutdown_connections 2024-05-15T14:11:56.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:11:56.076+0000 7f4b8c6f5700 1 -- 172.21.15.33:0/2828884012 wait complete. 2024-05-15T14:11:56.083 INFO:tasks.workunit.client.0.smithi033.stdout: 1 651500 5.27 MB/sec execute 451 sec latency 24590.847 ms 2024-05-15T14:11:56.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:56 smithi033 ceph-mon[31650]: pgmap v363: 97 pgs: 97 active+clean; 198 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 728 KiB/s wr, 43 op/s 2024-05-15T14:11:56.787 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '69488f58', total seconds elapsed: 31.1 2024-05-15T14:11:56.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:56 smithi052 ceph-mon[36190]: pgmap v363: 97 pgs: 97 active+clean; 198 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 728 KiB/s wr, 43 op/s 2024-05-15T14:11:56.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:56 smithi081 ceph-mon[36434]: pgmap v363: 97 pgs: 97 active+clean; 198 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 728 KiB/s wr, 43 op/s 2024-05-15T14:11:57.083 INFO:tasks.workunit.client.0.smithi033.stdout: 1 653654 5.29 MB/sec execute 452 sec latency 17.901 ms 2024-05-15T14:11:58.083 INFO:tasks.workunit.client.0.smithi033.stdout: 1 654829 5.29 MB/sec execute 453 sec latency 36.772 ms 2024-05-15T14:11:58.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:11:58 smithi033 ceph-mon[31650]: pgmap v364: 97 pgs: 97 active+clean; 199 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 1.4 MiB/s wr, 95 op/s 2024-05-15T14:11:58.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:11:58 smithi052 ceph-mon[36190]: pgmap v364: 97 pgs: 97 active+clean; 199 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 1.4 MiB/s wr, 95 op/s 2024-05-15T14:11:58.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:11:58 smithi081 ceph-mon[36434]: pgmap v364: 97 pgs: 97 active+clean; 199 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 1.4 MiB/s wr, 95 op/s 2024-05-15T14:11:59.083 INFO:tasks.workunit.client.0.smithi033.stdout: 1 656710 5.30 MB/sec execute 454 sec latency 18.300 ms 2024-05-15T14:12:00.083 INFO:tasks.workunit.client.0.smithi033.stdout: 1 658055 5.30 MB/sec execute 455 sec latency 17.151 ms 2024-05-15T14:12:00.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:00 smithi033 ceph-mon[31650]: pgmap v365: 97 pgs: 97 active+clean; 199 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 783 KiB/s wr, 73 op/s 2024-05-15T14:12:00.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:00 smithi052 ceph-mon[36190]: pgmap v365: 97 pgs: 97 active+clean; 199 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 783 KiB/s wr, 73 op/s 2024-05-15T14:12:00.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:00 smithi081 ceph-mon[36434]: pgmap v365: 97 pgs: 97 active+clean; 199 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 783 KiB/s wr, 73 op/s 2024-05-15T14:12:01.083 INFO:tasks.workunit.client.0.smithi033.stdout: 1 659474 5.30 MB/sec execute 456 sec latency 24.733 ms 2024-05-15T14:12:02.083 INFO:tasks.workunit.client.0.smithi033.stdout: 1 661064 5.31 MB/sec execute 457 sec latency 11.217 ms 2024-05-15T14:12:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.662+0000 7f3f66699640 1 -- 172.21.15.33:0/1012788926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f60102370 msgr2=0x7f3f6010e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.662+0000 7f3f66699640 1 --2- 172.21.15.33:0/1012788926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f60102370 0x7f3f6010e830 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3f58009cd0 tx=0x7f3f5802a8e0 comp rx=0 tx=0).stop 2024-05-15T14:12:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.662+0000 7f3f66699640 1 -- 172.21.15.33:0/1012788926 shutdown_connections 2024-05-15T14:12:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.662+0000 7f3f66699640 1 --2- 172.21.15.33:0/1012788926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f60102370 0x7f3f6010e830 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.662+0000 7f3f66699640 1 --2- 172.21.15.33:0/1012788926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f601019b0 0x7f3f60101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.662+0000 7f3f66699640 1 --2- 172.21.15.33:0/1012788926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f60107970 0x7f3f60107d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.662+0000 7f3f66699640 1 -- 172.21.15.33:0/1012788926 >> 172.21.15.33:0/1012788926 conn(0x7f3f60078aa0 msgr2=0x7f3f60078eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.662+0000 7f3f66699640 1 -- 172.21.15.33:0/1012788926 shutdown_connections 2024-05-15T14:12:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.662+0000 7f3f66699640 1 -- 172.21.15.33:0/1012788926 wait complete. 2024-05-15T14:12:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.663+0000 7f3f66699640 1 Processor -- start 2024-05-15T14:12:02.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.663+0000 7f3f66699640 1 -- start start 2024-05-15T14:12:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.663+0000 7f3f66699640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f601019b0 0x7f3f60197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.663+0000 7f3f66699640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f60102370 0x7f3f60197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:02.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.663+0000 7f3f66699640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f60107970 0x7f3f6019b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.663+0000 7f3f66699640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3f60110b20 con 0x7f3f60107970 2024-05-15T14:12:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.663+0000 7f3f66699640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3f601109a0 con 0x7f3f601019b0 2024-05-15T14:12:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.663+0000 7f3f66699640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3f60110ca0 con 0x7f3f60102370 2024-05-15T14:12:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.663+0000 7f3f64c36640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f60107970 0x7f3f6019b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.664+0000 7f3f5f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f60102370 0x7f3f60197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.664+0000 7f3f5ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f601019b0 0x7f3f60197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.664+0000 7f3f5f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f60102370 0x7f3f60197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:58446/0 (socket says 172.21.15.33:58446) 2024-05-15T14:12:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.664+0000 7f3f64c36640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f60107970 0x7f3f6019b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36622/0 (socket says 172.21.15.33:36622) 2024-05-15T14:12:02.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.664+0000 7f3f5f7fe640 1 -- 172.21.15.33:0/407460213 learned_addr learned my addr 172.21.15.33:0/407460213 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:12:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.664+0000 7f3f5f7fe640 1 -- 172.21.15.33:0/407460213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f601019b0 msgr2=0x7f3f60197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.664+0000 7f3f5f7fe640 1 --2- 172.21.15.33:0/407460213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f601019b0 0x7f3f60197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.664+0000 7f3f5f7fe640 1 -- 172.21.15.33:0/407460213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f60107970 msgr2=0x7f3f6019b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.664+0000 7f3f5f7fe640 1 --2- 172.21.15.33:0/407460213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f60107970 0x7f3f6019b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.664+0000 7f3f5f7fe640 1 -- 172.21.15.33:0/407460213 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f6019bfe0 con 0x7f3f60102370 2024-05-15T14:12:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.664+0000 7f3f5f7fe640 1 --2- 172.21.15.33:0/407460213 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f60102370 0x7f3f60197580 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3f5400e9f0 tx=0x7f3f5400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.664+0000 7f3f5d7fa640 1 -- 172.21.15.33:0/407460213 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5400cd80 con 0x7f3f60102370 2024-05-15T14:12:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.664+0000 7f3f66699640 1 -- 172.21.15.33:0/407460213 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f6019c2d0 con 0x7f3f60102370 2024-05-15T14:12:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.664+0000 7f3f5d7fa640 1 -- 172.21.15.33:0/407460213 <== mon.1 v2:172.21.15.81:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f540108e0 con 0x7f3f60102370 2024-05-15T14:12:02.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.664+0000 7f3f5d7fa640 1 -- 172.21.15.33:0/407460213 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f54018980 con 0x7f3f60102370 2024-05-15T14:12:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.664+0000 7f3f66699640 1 -- 172.21.15.33:0/407460213 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f601a3b60 con 0x7f3f60102370 2024-05-15T14:12:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.667+0000 7f3f693ba740 1 Processor -- start 2024-05-15T14:12:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.667+0000 7f3f693ba740 1 -- start start 2024-05-15T14:12:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.667+0000 7f3f5d7fa640 1 -- 172.21.15.33:0/407460213 <== mon.1 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3f54010430 con 0x7f3f60102370 2024-05-15T14:12:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.667+0000 7f3f693ba740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6600db7f0 0x55f6600dbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.667+0000 7f3f693ba740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6600dc110 0x55f6600ea9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.667+0000 7f3f693ba740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f6600eaee0 0x55f6600ed2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.667+0000 7f3f693ba740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f66004ec30 con 0x55f6600dc110 2024-05-15T14:12:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.667+0000 7f3f693ba740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f66003e7d0 con 0x55f6600db7f0 2024-05-15T14:12:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.667+0000 7f3f693ba740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55f660042810 con 0x55f6600eaee0 2024-05-15T14:12:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.667+0000 7f3f5ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6600dc110 0x55f6600ea9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.667+0000 7f3f64c36640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6600db7f0 0x55f6600dbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.667+0000 7f3f64c36640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6600db7f0 0x55f6600dbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:41548/0 (socket says 172.21.15.33:41548) 2024-05-15T14:12:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.667+0000 7f3f5ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6600dc110 0x55f6600ea9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36636/0 (socket says 172.21.15.33:36636) 2024-05-15T14:12:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.667+0000 7f3f64c36640 1 -- 172.21.15.33:0/3319462878 learned_addr learned my addr 172.21.15.33:0/3319462878 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:12:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.667+0000 7f3f5f7fe640 1 --2- 172.21.15.33:0/3319462878 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f6600eaee0 0x55f6600ed2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.667+0000 7f3f5ffff640 1 -- 172.21.15.33:0/3319462878 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f6600eaee0 msgr2=0x55f6600ed2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:12:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.667+0000 7f3f5ffff640 1 --2- 172.21.15.33:0/3319462878 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f6600eaee0 0x55f6600ed2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.667+0000 7f3f5ffff640 1 -- 172.21.15.33:0/3319462878 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6600db7f0 msgr2=0x55f6600dbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:12:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.667+0000 7f3f5ffff640 1 --2- 172.21.15.33:0/3319462878 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6600db7f0 0x55f6600dbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.668+0000 7f3f5ffff640 1 -- 172.21.15.33:0/3319462878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f65fd53210 con 0x55f6600dc110 2024-05-15T14:12:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.668+0000 7f3f64c36640 1 --2- 172.21.15.33:0/3319462878 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6600db7f0 0x55f6600dbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:12:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.668+0000 7f3f5f7fe640 1 --2- 172.21.15.33:0/3319462878 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f6600eaee0 0x55f6600ed2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:12:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.668+0000 7f3f5ffff640 1 --2- 172.21.15.33:0/3319462878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6600dc110 0x55f6600ea9a0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f3f5000bc20 tx=0x7f3f50005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9fc865bfbc2b68f server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.668+0000 7f3f5d7fa640 1 --2- 172.21.15.33:0/407460213 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f3f480b2950 0x7f3f480b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.668+0000 7f3f467fc640 1 -- 172.21.15.33:0/3319462878 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5001a070 con 0x55f6600dc110 2024-05-15T14:12:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.668+0000 7f3f467fc640 1 -- 172.21.15.33:0/3319462878 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f50009e50 con 0x55f6600dc110 2024-05-15T14:12:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.668+0000 7f3f64c36640 1 --2- 172.21.15.33:0/407460213 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f3f480b2950 0x7f3f480b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.668+0000 7f3f5d7fa640 1 -- 172.21.15.33:0/407460213 <== mon.1 v2:172.21.15.81:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7f3f540e2950 con 0x7f3f60102370 2024-05-15T14:12:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.668+0000 7f3f467fc640 1 -- 172.21.15.33:0/3319462878 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f500156d0 con 0x55f6600dc110 2024-05-15T14:12:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.669+0000 7f3f64c36640 1 --2- 172.21.15.33:0/407460213 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f3f480b2950 0x7f3f480b4e10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3f58005e10 tx=0x7f3f5803d040 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:02.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.669+0000 7f3f693ba740 1 -- 172.21.15.33:0/3319462878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6600dc110 msgr2=0x55f6600ea9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.669+0000 7f3f693ba740 1 --2- 172.21.15.33:0/3319462878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6600dc110 0x55f6600ea9a0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f3f5000bc20 tx=0x7f3f50005f90 comp rx=0 tx=0).stop 2024-05-15T14:12:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.669+0000 7f3f693ba740 1 -- 172.21.15.33:0/3319462878 shutdown_connections 2024-05-15T14:12:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.669+0000 7f3f693ba740 1 --2- 172.21.15.33:0/3319462878 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f6600eaee0 0x55f6600ed2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.669+0000 7f3f693ba740 1 --2- 172.21.15.33:0/3319462878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6600dc110 0x55f6600ea9a0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.669+0000 7f3f693ba740 1 --2- 172.21.15.33:0/3319462878 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6600db7f0 0x55f6600dbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.669+0000 7f3f693ba740 1 -- 172.21.15.33:0/3319462878 >> 172.21.15.33:0/3319462878 conn(0x55f6600d7e00 msgr2=0x55f6600d81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.669+0000 7f3f693ba740 1 -- 172.21.15.33:0/3319462878 shutdown_connections 2024-05-15T14:12:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.669+0000 7f3f693ba740 1 -- 172.21.15.33:0/3319462878 wait complete. 2024-05-15T14:12:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.669+0000 7f3f693ba740 1 -- start start 2024-05-15T14:12:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.669+0000 7f3f693ba740 1 Processor -- start 2024-05-15T14:12:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.670+0000 7f3f693ba740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6600db7f0 0x55f6600ea770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.670+0000 7f3f693ba740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6600dc110 0x55f6600e7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.670+0000 7f3f693ba740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f6600eaee0 0x55f6600e7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.670+0000 7f3f693ba740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f65fd24790 con 0x55f6600dc110 2024-05-15T14:12:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.670+0000 7f3f693ba740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f65fffa890 con 0x55f6600db7f0 2024-05-15T14:12:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.670+0000 7f3f693ba740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55f6600edbb0 con 0x55f6600eaee0 2024-05-15T14:12:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.670+0000 7f3f693ba740 10 client.0 _ll_register_callbacks cb 0x55f65fffdd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:12:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.670+0000 7f3f5ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6600db7f0 0x55f6600ea770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.670+0000 7f3f5f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6600dc110 0x55f6600e7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.670+0000 7f3f5ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6600db7f0 0x55f6600ea770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.33:41550/0 (socket says 172.21.15.33:41550) 2024-05-15T14:12:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.670+0000 7f3f5ffff640 1 -- 172.21.15.33:0/1862228604 learned_addr learned my addr 172.21.15.33:0/1862228604 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:12:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.670+0000 7f3f64c36640 1 --2- 172.21.15.33:0/1862228604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f6600eaee0 0x55f6600e7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.670+0000 7f3f5f7fe640 1 -- 172.21.15.33:0/1862228604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f6600eaee0 msgr2=0x55f6600e7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:12:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.670+0000 7f3f5f7fe640 1 --2- 172.21.15.33:0/1862228604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f6600eaee0 0x55f6600e7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.670+0000 7f3f5f7fe640 1 -- 172.21.15.33:0/1862228604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6600db7f0 msgr2=0x55f6600ea770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:12:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.670+0000 7f3f5f7fe640 1 --2- 172.21.15.33:0/1862228604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6600db7f0 0x55f6600ea770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.670+0000 7f3f5f7fe640 1 -- 172.21.15.33:0/1862228604 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f65fe1f620 con 0x55f6600dc110 2024-05-15T14:12:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.670+0000 7f3f64c36640 1 --2- 172.21.15.33:0/1862228604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f6600eaee0 0x55f6600e7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:12:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.670+0000 7f3f5ffff640 1 --2- 172.21.15.33:0/1862228604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6600db7f0 0x55f6600ea770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:12:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.670+0000 7f3f5f7fe640 1 --2- 172.21.15.33:0/1862228604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6600dc110 0x55f6600e7a60 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f3f540e0090 tx=0x7f3f540a9bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4957219b496359a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.670+0000 7f3f3f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:12:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.670+0000 7f3f3f7fe640 1 -- 172.21.15.33:0/1862228604 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f540a97e0 con 0x55f6600dc110 2024-05-15T14:12:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.671+0000 7f3f3f7fe640 1 -- 172.21.15.33:0/1862228604 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7f3f540a62c0 con 0x55f6600dc110 2024-05-15T14:12:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.671+0000 7f3f693ba740 1 -- 172.21.15.33:0/1862228604 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f50009e50 con 0x55f6600dc110 2024-05-15T14:12:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.671+0000 7f3f3f7fe640 1 -- 172.21.15.33:0/1862228604 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3f540a9cd0 con 0x55f6600dc110 2024-05-15T14:12:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.671+0000 7f3f693ba740 10 client.16779 fetch_fsmap learned FSMap version 39 2024-05-15T14:12:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.671+0000 7f3f693ba740 1 -- 172.21.15.33:0/1862228604 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f6600f9570 con 0x55f6600dc110 2024-05-15T14:12:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.671+0000 7f3f3f7fe640 1 -- 172.21.15.33:0/1862228604 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f50009e50 con 0x55f6600dc110 2024-05-15T14:12:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.671+0000 7f3f3f7fe640 1 -- 172.21.15.33:0/1862228604 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f3f540a5640 con 0x55f6600dc110 2024-05-15T14:12:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.671+0000 7f3f693ba740 10 client.16779 fetch_fsmap finished waiting for FSMap version 39 2024-05-15T14:12:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.671+0000 7f3f693ba740 10 client.16779 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:12:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.671+0000 7f3f693ba740 20 client.16779 populate_metadata read hostname 'smithi033' 2024-05-15T14:12:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.671+0000 7f3f693ba740 1 --2- 172.21.15.33:0/1862228604 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55f6600faa30 0x55f6600fce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.671+0000 7f3f693ba740 4 client.16779 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:12:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.671+0000 7f3f693ba740 1 -- 172.21.15.33:0/1862228604 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f660050b80 con 0x55f6600faa30 2024-05-15T14:12:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.671+0000 7f3f5ffff640 1 --2- 172.21.15.33:0/1862228604 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55f6600faa30 0x55f6600fce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.672+0000 7f3f3f7fe640 10 client.16779 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:12:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.672+0000 7f3f5ffff640 1 --2- 172.21.15.33:0/1862228604 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55f6600faa30 0x55f6600fce80 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db6929cfbccef7ad server_cookie=a871db38f41a4373 in_seq=0 out_seq=0 2024-05-15T14:12:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.672+0000 7f3f3f7fe640 1 -- 172.21.15.33:0/1862228604 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f660050b80 con 0x55f6600faa30 2024-05-15T14:12:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.672+0000 7f3f3f7fe640 10 client.16779 handle_command_reply: tid=0 2024-05-15T14:12:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.672+0000 7f3f693ba740 1 client.16779 shutdown 2024-05-15T14:12:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.673+0000 7f3f693ba740 1 -- 172.21.15.33:0/1862228604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6600dc110 msgr2=0x55f6600e7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.673+0000 7f3f693ba740 1 --2- 172.21.15.33:0/1862228604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6600dc110 0x55f6600e7a60 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f3f540e0090 tx=0x7f3f540a9bd0 comp rx=0 tx=0).stop 2024-05-15T14:12:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.673+0000 7f3f693ba740 1 -- 172.21.15.33:0/1862228604 shutdown_connections 2024-05-15T14:12:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.673+0000 7f3f693ba740 1 --2- 172.21.15.33:0/1862228604 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55f6600faa30 0x55f6600fce80 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.673+0000 7f3f693ba740 1 --2- 172.21.15.33:0/1862228604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f6600eaee0 0x55f6600e7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.673+0000 7f3f693ba740 1 --2- 172.21.15.33:0/1862228604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6600dc110 0x55f6600e7a60 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.673+0000 7f3f693ba740 1 --2- 172.21.15.33:0/1862228604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6600db7f0 0x55f6600ea770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.673+0000 7f3f3f7fe640 0 client.16779 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:12:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.673+0000 7f3f693ba740 1 -- 172.21.15.33:0/1862228604 >> 172.21.15.33:0/1862228604 conn(0x55f6600d7e00 msgr2=0x55f6600ddc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.673+0000 7f3f693ba740 1 -- 172.21.15.33:0/1862228604 shutdown_connections 2024-05-15T14:12:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.673+0000 7f3f693ba740 1 -- 172.21.15.33:0/1862228604 wait complete. 2024-05-15T14:12:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.673+0000 7f3f693ba740 20 client.16779 trim_cache size 0 max 16384 2024-05-15T14:12:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.688+0000 7f3f693ba740 1 Processor -- start 2024-05-15T14:12:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.688+0000 7f3f693ba740 1 -- start start 2024-05-15T14:12:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.688+0000 7f3f693ba740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f6600db7f0 0x55f6600d7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.688+0000 7f3f693ba740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6600dc110 0x55f6600ea2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.688+0000 7f3f693ba740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6600eaee0 0x55f660102790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.688+0000 7f3f693ba740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3f601aeee0 con 0x55f6600dc110 2024-05-15T14:12:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.688+0000 7f3f693ba740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3f601ac750 con 0x55f6600eaee0 2024-05-15T14:12:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.688+0000 7f3f693ba740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3f601aa5c0 con 0x55f6600db7f0 2024-05-15T14:12:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.688+0000 7f3f5f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6600dc110 0x55f6600ea2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.688+0000 7f3f5f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6600dc110 0x55f6600ea2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36668/0 (socket says 172.21.15.33:36668) 2024-05-15T14:12:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.688+0000 7f3f5f7fe640 1 -- 172.21.15.33:0/1605786129 learned_addr learned my addr 172.21.15.33:0/1605786129 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:12:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.688+0000 7f3f5ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f6600db7f0 0x55f6600d7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.688+0000 7f3f64c36640 1 --2- 172.21.15.33:0/1605786129 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6600eaee0 0x55f660102790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.689+0000 7f3f5f7fe640 1 -- 172.21.15.33:0/1605786129 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f6600db7f0 msgr2=0x55f6600d7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:12:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.689+0000 7f3f5f7fe640 1 --2- 172.21.15.33:0/1605786129 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f6600db7f0 0x55f6600d7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.689+0000 7f3f5f7fe640 1 -- 172.21.15.33:0/1605786129 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6600eaee0 msgr2=0x55f660102790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:12:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.689+0000 7f3f5f7fe640 1 --2- 172.21.15.33:0/1605786129 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6600eaee0 0x55f660102790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.689+0000 7f3f5f7fe640 1 -- 172.21.15.33:0/1605786129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6600d7c30 con 0x55f6600dc110 2024-05-15T14:12:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.689+0000 7f3f5f7fe640 1 --2- 172.21.15.33:0/1605786129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6600dc110 0x55f6600ea2a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f3f540e0690 tx=0x7f3f540a6170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55dca4f4ba326c7e server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.689+0000 7f3f45ffb640 1 -- 172.21.15.33:0/1605786129 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f540a97e0 con 0x55f6600dc110 2024-05-15T14:12:02.694 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:02 smithi033 ceph-mon[31650]: pgmap v366: 97 pgs: 97 active+clean; 199 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 783 KiB/s wr, 73 op/s 2024-05-15T14:12:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.689+0000 7f3f45ffb640 1 -- 172.21.15.33:0/1605786129 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f540c42e0 con 0x55f6600dc110 2024-05-15T14:12:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.689+0000 7f3f45ffb640 1 -- 172.21.15.33:0/1605786129 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f54018b20 con 0x55f6600dc110 2024-05-15T14:12:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.690+0000 7f3f693ba740 1 -- 172.21.15.33:0/1605786129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6600dc110 msgr2=0x55f6600ea2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.690+0000 7f3f693ba740 1 --2- 172.21.15.33:0/1605786129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6600dc110 0x55f6600ea2a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f3f540e0690 tx=0x7f3f540a6170 comp rx=0 tx=0).stop 2024-05-15T14:12:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.690+0000 7f3f693ba740 1 -- 172.21.15.33:0/1605786129 shutdown_connections 2024-05-15T14:12:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.690+0000 7f3f693ba740 1 --2- 172.21.15.33:0/1605786129 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6600eaee0 0x55f660102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.690+0000 7f3f693ba740 1 --2- 172.21.15.33:0/1605786129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6600dc110 0x55f6600ea2a0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.690+0000 7f3f693ba740 1 --2- 172.21.15.33:0/1605786129 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f6600db7f0 0x55f6600d7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.690+0000 7f3f693ba740 1 -- 172.21.15.33:0/1605786129 >> 172.21.15.33:0/1605786129 conn(0x55f6600d7e00 msgr2=0x55f6600e2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.690+0000 7f3f693ba740 1 -- 172.21.15.33:0/1605786129 shutdown_connections 2024-05-15T14:12:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.690+0000 7f3f693ba740 1 -- 172.21.15.33:0/1605786129 wait complete. 2024-05-15T14:12:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.690+0000 7f3f693ba740 1 -- start start 2024-05-15T14:12:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.690+0000 7f3f693ba740 1 Processor -- start 2024-05-15T14:12:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.690+0000 7f3f693ba740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f6600db7f0 0x55f6600f3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.690+0000 7f3f693ba740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6600dc110 0x55f6600ee7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.690+0000 7f3f693ba740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6600eaee0 0x55f6600eed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.690+0000 7f3f693ba740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3f601aac10 con 0x55f6600dc110 2024-05-15T14:12:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.690+0000 7f3f693ba740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3f601ac080 con 0x55f6600eaee0 2024-05-15T14:12:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.691+0000 7f3f693ba740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3f601ab2e0 con 0x55f6600db7f0 2024-05-15T14:12:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.691+0000 7f3f693ba740 10 client.0 _ll_register_callbacks cb 0x7f3f540a5bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:12:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.691+0000 7f3f5ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f6600db7f0 0x55f6600f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.691+0000 7f3f5f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6600dc110 0x55f6600ee7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.691+0000 7f3f5ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f6600db7f0 0x55f6600f3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.33:58480/0 (socket says 172.21.15.33:58480) 2024-05-15T14:12:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.691+0000 7f3f5ffff640 1 -- 172.21.15.33:0/1768038251 learned_addr learned my addr 172.21.15.33:0/1768038251 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:12:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.691+0000 7f3f64c36640 1 --2- 172.21.15.33:0/1768038251 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6600eaee0 0x55f6600eed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.691+0000 7f3f5ffff640 1 -- 172.21.15.33:0/1768038251 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6600eaee0 msgr2=0x55f6600eed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:12:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.691+0000 7f3f5ffff640 1 --2- 172.21.15.33:0/1768038251 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6600eaee0 0x55f6600eed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.691+0000 7f3f5ffff640 1 -- 172.21.15.33:0/1768038251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6600dc110 msgr2=0x55f6600ee7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:12:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.691+0000 7f3f5ffff640 1 --2- 172.21.15.33:0/1768038251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6600dc110 0x55f6600ee7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.691+0000 7f3f5ffff640 1 -- 172.21.15.33:0/1768038251 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6600ef590 con 0x55f6600db7f0 2024-05-15T14:12:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.691+0000 7f3f5f7fe640 1 --2- 172.21.15.33:0/1768038251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6600dc110 0x55f6600ee7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:12:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.691+0000 7f3f5ffff640 1 --2- 172.21.15.33:0/1768038251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f6600db7f0 0x55f6600f3450 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3f50000c00 tx=0x7f3f500055d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd277b4f618622b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.691+0000 7f3f3f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2024-05-15T14:12:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.691+0000 7f3f3f7fe640 1 -- 172.21.15.33:0/1768038251 <== mon.1 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5001a040 con 0x55f6600db7f0 2024-05-15T14:12:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.691+0000 7f3f693ba740 1 -- 172.21.15.33:0/1768038251 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f540c42e0 con 0x55f6600db7f0 2024-05-15T14:12:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.691+0000 7f3f3f7fe640 1 -- 172.21.15.33:0/1768038251 <== mon.1 v2:172.21.15.81:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8542+0+0 (secure 0 0 0) 0x7f3f5002b800 con 0x55f6600db7f0 2024-05-15T14:12:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.692+0000 7f3f3f7fe640 1 -- 172.21.15.33:0/1768038251 <== mon.1 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3f50005c50 con 0x55f6600db7f0 2024-05-15T14:12:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.692+0000 7f3f693ba740 10 client.26350 fetch_fsmap learned FSMap version 39 2024-05-15T14:12:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.692+0000 7f3f693ba740 1 -- 172.21.15.33:0/1768038251 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f660165420 con 0x55f6600db7f0 2024-05-15T14:12:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.692+0000 7f3f3f7fe640 1 -- 172.21.15.33:0/1768038251 <== mon.1 v2:172.21.15.81:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f540c42e0 con 0x55f6600db7f0 2024-05-15T14:12:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.692+0000 7f3f3f7fe640 1 -- 172.21.15.33:0/1768038251 <== mon.1 v2:172.21.15.81:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f3f50032070 con 0x55f6600db7f0 2024-05-15T14:12:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.692+0000 7f3f693ba740 10 client.26350 fetch_fsmap finished waiting for FSMap version 39 2024-05-15T14:12:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.692+0000 7f3f693ba740 10 client.26350 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-15T14:12:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.692+0000 7f3f693ba740 20 client.26350 populate_metadata read hostname 'smithi033' 2024-05-15T14:12:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.692+0000 7f3f693ba740 1 --2- 172.21.15.33:0/1768038251 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55f660165a30 0x55f660167e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.692+0000 7f3f693ba740 4 client.26350 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-15T14:12:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.692+0000 7f3f693ba740 1 -- 172.21.15.33:0/1768038251 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f66004a360 con 0x55f660165a30 2024-05-15T14:12:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.692+0000 7f3f5f7fe640 1 --2- 172.21.15.33:0/1768038251 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55f660165a30 0x55f660167e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.692+0000 7f3f3f7fe640 10 client.26350 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:12:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.692+0000 7f3f5f7fe640 1 --2- 172.21.15.33:0/1768038251 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55f660165a30 0x55f660167e50 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7f53e55abccd0bf server_cookie=1fb1ed60b5e7a334 in_seq=0 out_seq=0 2024-05-15T14:12:02.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.742+0000 7f3f3f7fe640 1 -- 172.21.15.33:0/1768038251 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f66004a360 con 0x55f660165a30 2024-05-15T14:12:02.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.742+0000 7f3f3f7fe640 10 client.26350 handle_command_reply: tid=0 2024-05-15T14:12:02.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.742+0000 7f3f693ba740 1 client.26350 shutdown 2024-05-15T14:12:02.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.742+0000 7f3f693ba740 1 -- 172.21.15.33:0/1768038251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f6600db7f0 msgr2=0x55f6600f3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:02.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.742+0000 7f3f693ba740 1 --2- 172.21.15.33:0/1768038251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f6600db7f0 0x55f6600f3450 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3f50000c00 tx=0x7f3f500055d0 comp rx=0 tx=0).stop 2024-05-15T14:12:02.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.742+0000 7f3f693ba740 1 -- 172.21.15.33:0/1768038251 shutdown_connections 2024-05-15T14:12:02.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.742+0000 7f3f693ba740 1 --2- 172.21.15.33:0/1768038251 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55f660165a30 0x55f660167e50 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:02.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.742+0000 7f3f693ba740 1 --2- 172.21.15.33:0/1768038251 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6600eaee0 0x55f6600eed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:02.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.742+0000 7f3f693ba740 1 --2- 172.21.15.33:0/1768038251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f6600dc110 0x55f6600ee7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:02.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.742+0000 7f3f693ba740 1 --2- 172.21.15.33:0/1768038251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f6600db7f0 0x55f6600f3450 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:02.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.742+0000 7f3f3f7fe640 0 client.26350 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:12:02.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.742+0000 7f3f693ba740 1 -- 172.21.15.33:0/1768038251 >> 172.21.15.33:0/1768038251 conn(0x55f6600d7e00 msgr2=0x55f6600d7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:02.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.742+0000 7f3f693ba740 1 -- 172.21.15.33:0/1768038251 shutdown_connections 2024-05-15T14:12:02.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.742+0000 7f3f693ba740 1 -- 172.21.15.33:0/1768038251 wait complete. 2024-05-15T14:12:02.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.742+0000 7f3f693ba740 20 client.26350 trim_cache size 0 max 16384 2024-05-15T14:12:02.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-05-15T14:12:02.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-05-15T14:12:02.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-05-15T14:12:02.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.743+0000 7f3f66699640 1 -- 172.21.15.33:0/407460213 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f3f480b2950 msgr2=0x7f3f480b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:02.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.743+0000 7f3f66699640 1 --2- 172.21.15.33:0/407460213 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f3f480b2950 0x7f3f480b4e10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3f58005e10 tx=0x7f3f5803d040 comp rx=0 tx=0).stop 2024-05-15T14:12:02.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.743+0000 7f3f66699640 1 -- 172.21.15.33:0/407460213 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f60102370 msgr2=0x7f3f60197580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:02.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.743+0000 7f3f66699640 1 --2- 172.21.15.33:0/407460213 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f60102370 0x7f3f60197580 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3f5400e9f0 tx=0x7f3f5400eeb0 comp rx=0 tx=0).stop 2024-05-15T14:12:02.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.743+0000 7f3f66699640 1 -- 172.21.15.33:0/407460213 shutdown_connections 2024-05-15T14:12:02.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.743+0000 7f3f66699640 1 --2- 172.21.15.33:0/407460213 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f3f480b2950 0x7f3f480b4e10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:02.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.743+0000 7f3f66699640 1 --2- 172.21.15.33:0/407460213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f60107970 0x7f3f6019b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:02.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.743+0000 7f3f66699640 1 --2- 172.21.15.33:0/407460213 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3f60102370 0x7f3f60197580 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.743+0000 7f3f66699640 1 --2- 172.21.15.33:0/407460213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f601019b0 0x7f3f60197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.743+0000 7f3f66699640 1 -- 172.21.15.33:0/407460213 >> 172.21.15.33:0/407460213 conn(0x7f3f60078aa0 msgr2=0x7f3f60104280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.743+0000 7f3f66699640 1 -- 172.21.15.33:0/407460213 shutdown_connections 2024-05-15T14:12:02.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-05-15T14:12:02.743+0000 7f3f66699640 1 -- 172.21.15.33:0/407460213 wait complete. 2024-05-15T14:12:02.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-15T14:12:02.799 INFO:tasks.workunit.client.0.smithi033.stdout:[662055] write failed on handle 12093 (Resource temporarily unavailable) 2024-05-15T14:12:02.800 INFO:tasks.workunit.client.0.smithi033.stdout:Child failed with status 1 2024-05-15T14:12:02.800 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-05-15T14:12:02.800 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-15T14:12:02.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:02 smithi052 ceph-mon[36190]: pgmap v366: 97 pgs: 97 active+clean; 199 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 783 KiB/s wr, 73 op/s 2024-05-15T14:12:02.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:02 smithi081 ceph-mon[36434]: pgmap v366: 97 pgs: 97 active+clean; 199 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 783 KiB/s wr, 73 op/s 2024-05-15T14:12:03.144 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi033 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-05-15T14:12:03.403 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=54515f164fe745a8a0a3d4b11c46fa42 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_9b63866860f0b71216b2c0a73aedd2e0a8a5c98b/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi033 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-05-15T14:12:03.406 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-05-15T14:12:03.416 INFO:tasks.quiescer:joining Quiescers 2024-05-15T14:12:03.417 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-05-15T14:12:03.417 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T14:12:03.670 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:12:04.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:04 smithi033 ceph-mon[31650]: pgmap v367: 97 pgs: 97 active+clean; 206 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 239 op/s 2024-05-15T14:12:04.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:04 smithi052 ceph-mon[36190]: pgmap v367: 97 pgs: 97 active+clean; 206 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 239 op/s 2024-05-15T14:12:04.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:04 smithi081 ceph-mon[36434]: pgmap v367: 97 pgs: 97 active+clean; 206 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 239 op/s 2024-05-15T14:12:05.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.328+0000 7fbcd309f700 1 -- 172.21.15.33:0/633926859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbccc102e30 msgr2=0x7fbccc10f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:05.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.328+0000 7fbcd309f700 1 --2- 172.21.15.33:0/633926859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbccc102e30 0x7fbccc10f2f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4009990 tx=0x7fbcc4009ca0 comp rx=0 tx=0).stop 2024-05-15T14:12:05.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.328+0000 7fbcd309f700 1 -- 172.21.15.33:0/633926859 shutdown_connections 2024-05-15T14:12:05.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.328+0000 7fbcd309f700 1 --2- 172.21.15.33:0/633926859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbccc102e30 0x7fbccc10f2f0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:05.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.328+0000 7fbcd309f700 1 --2- 172.21.15.33:0/633926859 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbccc1084b0 0x7fbccc108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:05.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.328+0000 7fbcd309f700 1 --2- 172.21.15.33:0/633926859 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbccc102470 0x7fbccc1028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:05.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.328+0000 7fbcd309f700 1 -- 172.21.15.33:0/633926859 >> 172.21.15.33:0/633926859 conn(0x7fbccc0793f0 msgr2=0x7fbccc079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:05.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.328+0000 7fbcd309f700 1 -- 172.21.15.33:0/633926859 shutdown_connections 2024-05-15T14:12:05.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.328+0000 7fbcd309f700 1 -- 172.21.15.33:0/633926859 wait complete. 2024-05-15T14:12:05.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.329+0000 7fbcd309f700 1 Processor -- start 2024-05-15T14:12:05.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.329+0000 7fbcd309f700 1 -- start start 2024-05-15T14:12:05.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.329+0000 7fbcd309f700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbccc102470 0x7fbccc19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:05.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.329+0000 7fbcd309f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbccc102e30 0x7fbccc19af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:05.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.329+0000 7fbcd309f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbccc1084b0 0x7fbccc1a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:05.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.329+0000 7fbcd309f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc1125f0 con 0x7fbccc102e30 2024-05-15T14:12:05.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.329+0000 7fbcd309f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc112470 con 0x7fbccc1084b0 2024-05-15T14:12:05.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.329+0000 7fbcd309f700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc112770 con 0x7fbccc102470 2024-05-15T14:12:05.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.329+0000 7fbccbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbccc102e30 0x7fbccc19af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:05.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.329+0000 7fbcd0e3b700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbccc102470 0x7fbccc19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:05.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.329+0000 7fbccbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbccc102e30 0x7fbccc19af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36694/0 (socket says 172.21.15.33:36694) 2024-05-15T14:12:05.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.329+0000 7fbccbfff700 1 -- 172.21.15.33:0/2167129838 learned_addr learned my addr 172.21.15.33:0/2167129838 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:12:05.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.329+0000 7fbcd163c700 1 --2- 172.21.15.33:0/2167129838 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbccc1084b0 0x7fbccc1a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:05.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.329+0000 7fbccbfff700 1 -- 172.21.15.33:0/2167129838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbccc102470 msgr2=0x7fbccc19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:05.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.329+0000 7fbccbfff700 1 --2- 172.21.15.33:0/2167129838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbccc102470 0x7fbccc19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:05.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.329+0000 7fbccbfff700 1 -- 172.21.15.33:0/2167129838 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbccc1084b0 msgr2=0x7fbccc1a5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:05.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.329+0000 7fbccbfff700 1 --2- 172.21.15.33:0/2167129838 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbccc1084b0 0x7fbccc1a5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:05.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.329+0000 7fbccbfff700 1 -- 172.21.15.33:0/2167129838 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbccc1a5fd0 con 0x7fbccc102e30 2024-05-15T14:12:05.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.329+0000 7fbcd163c700 1 --2- 172.21.15.33:0/2167129838 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbccc1084b0 0x7fbccc1a5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:12:05.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.330+0000 7fbcd0e3b700 1 --2- 172.21.15.33:0/2167129838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbccc102470 0x7fbccc19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:12:05.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.330+0000 7fbccbfff700 1 --2- 172.21.15.33:0/2167129838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbccc102e30 0x7fbccc19af50 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400e9c0 tx=0x7fbcb400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:05.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.330+0000 7fbcc9ffb700 1 -- 172.21.15.33:0/2167129838 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb400cbd0 con 0x7fbccc102e30 2024-05-15T14:12:05.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.330+0000 7fbcc9ffb700 1 -- 172.21.15.33:0/2167129838 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcb400cd70 con 0x7fbccc102e30 2024-05-15T14:12:05.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.330+0000 7fbcd309f700 1 -- 172.21.15.33:0/2167129838 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbccc198860 con 0x7fbccc102e30 2024-05-15T14:12:05.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.330+0000 7fbcc9ffb700 1 -- 172.21.15.33:0/2167129838 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4005610 con 0x7fbccc102e30 2024-05-15T14:12:05.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.330+0000 7fbcd309f700 1 -- 172.21.15.33:0/2167129838 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbccc198d50 con 0x7fbccc102e30 2024-05-15T14:12:05.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.331+0000 7fbcd309f700 1 -- 172.21.15.33:0/2167129838 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbccc06a230 con 0x7fbccc102e30 2024-05-15T14:12:05.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.332+0000 7fbcc9ffb700 1 -- 172.21.15.33:0/2167129838 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbcb4004750 con 0x7fbccc102e30 2024-05-15T14:12:05.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.333+0000 7fbcc9ffb700 1 --2- 172.21.15.33:0/2167129838 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbcb80b29f0 0x7fbcb80b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:05.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.333+0000 7fbcd0e3b700 1 --2- 172.21.15.33:0/2167129838 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbcb80b29f0 0x7fbcb80b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:05.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.333+0000 7fbcc9ffb700 1 -- 172.21.15.33:0/2167129838 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8586+0+0 (secure 0 0 0) 0x7fbcb40e2980 con 0x7fbccc102e30 2024-05-15T14:12:05.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.334+0000 7fbcd0e3b700 1 --2- 172.21.15.33:0/2167129838 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbcb80b29f0 0x7fbcb80b4eb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc009fd0 tx=0x7fbcbc009270 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:05.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.338+0000 7fbcc9ffb700 1 -- 172.21.15.33:0/2167129838 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbcb40aa320 con 0x7fbccc102e30 2024-05-15T14:12:05.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.605+0000 7fbcd309f700 1 -- 172.21.15.33:0/2167129838 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbccc02e720 con 0x7fbccc102e30 2024-05-15T14:12:05.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.605+0000 7fbcc9ffb700 1 -- 172.21.15.33:0/2167129838 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11861 (secure 0 0 0) 0x7fbcb40a9c20 con 0x7fbccc102e30 2024-05-15T14:12:05.607 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T14:12:05.608 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2030117633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2030117633},{"type":"v1","addr":"172.21.15.33:6839","nonce":2030117633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1918979061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1918979061},{"type":"v1","addr":"172.21.15.33:6841","nonce":1918979061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T13:57:19.056417+0000","modified":"2024-05-15T14:11:03.273793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24463,"mds_2":24461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/1629697841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1629697841},{"type":"v1","addr":"172.21.15.52:6837","nonce":1629697841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.52:6839/2696044471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2696044471},{"type":"v1","addr":"172.21.15.52:6839","nonce":2696044471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.81:6839/1554395940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":1554395940},{"type":"v1","addr":"172.21.15.81:6839","nonce":1554395940}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26206":{"gid":26206,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/3674797952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3674797952},{"type":"v1","addr":"172.21.15.81:6835","nonce":3674797952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24463,24478,24461]},"id":1}]} 2024-05-15T14:12:05.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.608+0000 7fbcd309f700 1 -- 172.21.15.33:0/2167129838 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbcb80b29f0 msgr2=0x7fbcb80b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:05.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.608+0000 7fbcd309f700 1 --2- 172.21.15.33:0/2167129838 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbcb80b29f0 0x7fbcb80b4eb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc009fd0 tx=0x7fbcbc009270 comp rx=0 tx=0).stop 2024-05-15T14:12:05.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.608+0000 7fbcd309f700 1 -- 172.21.15.33:0/2167129838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbccc102e30 msgr2=0x7fbccc19af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:05.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.608+0000 7fbcd309f700 1 --2- 172.21.15.33:0/2167129838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbccc102e30 0x7fbccc19af50 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400e9c0 tx=0x7fbcb400ecd0 comp rx=0 tx=0).stop 2024-05-15T14:12:05.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.608+0000 7fbcd309f700 1 -- 172.21.15.33:0/2167129838 shutdown_connections 2024-05-15T14:12:05.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.608+0000 7fbcd309f700 1 --2- 172.21.15.33:0/2167129838 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbcb80b29f0 0x7fbcb80b4eb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:05.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.608+0000 7fbcd309f700 1 --2- 172.21.15.33:0/2167129838 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbccc1084b0 0x7fbccc1a5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:05.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.608+0000 7fbcd309f700 1 --2- 172.21.15.33:0/2167129838 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbccc102470 0x7fbccc19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:05.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.608+0000 7fbcd309f700 1 --2- 172.21.15.33:0/2167129838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbccc102e30 0x7fbccc19af50 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:05.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.608+0000 7fbcd309f700 1 -- 172.21.15.33:0/2167129838 >> 172.21.15.33:0/2167129838 conn(0x7fbccc0793f0 msgr2=0x7fbccc100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:05.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.608+0000 7fbcd309f700 1 -- 172.21.15.33:0/2167129838 shutdown_connections 2024-05-15T14:12:05.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:05.608+0000 7fbcd309f700 1 -- 172.21.15.33:0/2167129838 wait complete. 2024-05-15T14:12:05.613 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 39 2024-05-15T14:12:05.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:05 smithi033 ceph-mon[31650]: osdmap e97: 12 total, 12 up, 12 in 2024-05-15T14:12:05.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:05 smithi052 ceph-mon[36190]: osdmap e97: 12 total, 12 up, 12 in 2024-05-15T14:12:05.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:05 smithi081 ceph-mon[36434]: osdmap e97: 12 total, 12 up, 12 in 2024-05-15T14:12:06.150 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24478 quiesce db --cancel --all' 2024-05-15T14:12:06.150 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- timeout 105 ceph tell mds.24478 quiesce db --cancel --all 2024-05-15T14:12:06.407 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:12:06.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:06 smithi033 ceph-mon[31650]: pgmap v369: 97 pgs: 97 active+clean; 206 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 261 op/s 2024-05-15T14:12:06.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:06 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/2167129838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:12:06.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:06 smithi052 ceph-mon[36190]: pgmap v369: 97 pgs: 97 active+clean; 206 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 261 op/s 2024-05-15T14:12:06.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:06 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/2167129838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:12:06.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:06 smithi081 ceph-mon[36434]: pgmap v369: 97 pgs: 97 active+clean; 206 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 261 op/s 2024-05-15T14:12:06.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:06 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/2167129838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:12:08.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.226+0000 7f4ac4803700 1 -- 172.21.15.33:0/3032792324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ac0105040 msgr2=0x7f4ac010d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:08.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.226+0000 7f4ac4803700 1 --2- 172.21.15.33:0/3032792324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ac0105040 0x7f4ac010d530 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4009990 tx=0x7f4ab4009ca0 comp rx=0 tx=0).stop 2024-05-15T14:12:08.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.226+0000 7f4ac4803700 1 -- 172.21.15.33:0/3032792324 shutdown_connections 2024-05-15T14:12:08.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.226+0000 7f4ac4803700 1 --2- 172.21.15.33:0/3032792324 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ac010db00 0x7f4ac010ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:08.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.226+0000 7f4ac4803700 1 --2- 172.21.15.33:0/3032792324 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ac0104720 0x7f4ac0104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:08.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.226+0000 7f4ac4803700 1 --2- 172.21.15.33:0/3032792324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ac0105040 0x7f4ac010d530 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:08.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.226+0000 7f4ac4803700 1 -- 172.21.15.33:0/3032792324 >> 172.21.15.33:0/3032792324 conn(0x7f4ac00fdef0 msgr2=0x7f4ac0100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:08.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.226+0000 7f4ac4803700 1 -- 172.21.15.33:0/3032792324 shutdown_connections 2024-05-15T14:12:08.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.226+0000 7f4ac4803700 1 -- 172.21.15.33:0/3032792324 wait complete. 2024-05-15T14:12:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.227+0000 7f4ac4803700 1 Processor -- start 2024-05-15T14:12:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.227+0000 7f4ac4803700 1 -- start start 2024-05-15T14:12:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.227+0000 7f4ac4803700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ac0104720 0x7f4ac019fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.227+0000 7f4ac4803700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ac0105040 0x7f4ac019ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.227+0000 7f4ac4803700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ac010db00 0x7f4ac0199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.227+0000 7f4ac4803700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac0113790 con 0x7f4ac010db00 2024-05-15T14:12:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.227+0000 7f4ac4803700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac0113610 con 0x7f4ac0105040 2024-05-15T14:12:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.227+0000 7f4ac4803700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac0113910 con 0x7f4ac0104720 2024-05-15T14:12:08.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.227+0000 7f4abf59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ac010db00 0x7f4ac0199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:08.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.227+0000 7f4abf59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ac010db00 0x7f4ac0199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36710/0 (socket says 172.21.15.33:36710) 2024-05-15T14:12:08.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.227+0000 7f4abf59e700 1 -- 172.21.15.33:0/1750167095 learned_addr learned my addr 172.21.15.33:0/1750167095 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:12:08.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.227+0000 7f4abe59c700 1 --2- 172.21.15.33:0/1750167095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ac0105040 0x7f4ac019ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:08.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.228+0000 7f4abed9d700 1 --2- 172.21.15.33:0/1750167095 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ac0104720 0x7f4ac019fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:08.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.228+0000 7f4abf59e700 1 -- 172.21.15.33:0/1750167095 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ac0104720 msgr2=0x7f4ac019fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:08.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.228+0000 7f4abf59e700 1 --2- 172.21.15.33:0/1750167095 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ac0104720 0x7f4ac019fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:08.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.228+0000 7f4abf59e700 1 -- 172.21.15.33:0/1750167095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ac0105040 msgr2=0x7f4ac019ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:08.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.228+0000 7f4abf59e700 1 --2- 172.21.15.33:0/1750167095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ac0105040 0x7f4ac019ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:08.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.228+0000 7f4abf59e700 1 -- 172.21.15.33:0/1750167095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac019a3f0 con 0x7f4ac010db00 2024-05-15T14:12:08.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.228+0000 7f4abf59e700 1 --2- 172.21.15.33:0/1750167095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ac010db00 0x7f4ac0199b10 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8007ee0 tx=0x7f4ab800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:08.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.228+0000 7f4aa7fff700 1 -- 172.21.15.33:0/1750167095 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8004bb0 con 0x7f4ac010db00 2024-05-15T14:12:08.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.228+0000 7f4aa7fff700 1 -- 172.21.15.33:0/1750167095 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ab8004d50 con 0x7f4ac010db00 2024-05-15T14:12:08.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.228+0000 7f4ac4803700 1 -- 172.21.15.33:0/1750167095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ac019a6e0 con 0x7f4ac010db00 2024-05-15T14:12:08.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.228+0000 7f4aa7fff700 1 -- 172.21.15.33:0/1750167095 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8005920 con 0x7f4ac010db00 2024-05-15T14:12:08.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.228+0000 7f4ac4803700 1 -- 172.21.15.33:0/1750167095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ac01a69c0 con 0x7f4ac010db00 2024-05-15T14:12:08.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.231+0000 7f4aa7fff700 1 -- 172.21.15.33:0/1750167095 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4ab801a020 con 0x7f4ac010db00 2024-05-15T14:12:08.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.232+0000 7f4aa7fff700 1 --2- 172.21.15.33:0/1750167095 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f4aa80b2a70 0x7f4aa80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:08.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.232+0000 7f4abed9d700 1 --2- 172.21.15.33:0/1750167095 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f4aa80b2a70 0x7f4aa80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:08.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.232+0000 7f4aa7fff700 1 -- 172.21.15.33:0/1750167095 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8586+0+0 (secure 0 0 0) 0x7f4ab80e1c90 con 0x7f4ac010db00 2024-05-15T14:12:08.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.232+0000 7f4acf89ab80 1 Processor -- start 2024-05-15T14:12:08.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.232+0000 7f4acf89ab80 1 -- start start 2024-05-15T14:12:08.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.232+0000 7f4abed9d700 1 --2- 172.21.15.33:0/1750167095 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f4aa80b2a70 0x7f4aa80b4f30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0009a10 tx=0x7f4ab0009190 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:08.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.232+0000 7f4acf89ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55852ff8d360 0x55852ff8d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:08.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.233+0000 7f4acf89ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55852ff8dc80 0x55852ff916c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:08.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.233+0000 7f4acf89ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852ff91c00 0x55852ff92040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:08.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.233+0000 7f4acf89ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55852fee6de0 con 0x55852ff8d360 2024-05-15T14:12:08.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.233+0000 7f4acf89ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55852fed6980 con 0x55852ff8dc80 2024-05-15T14:12:08.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.233+0000 7f4acf89ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55852feda9c0 con 0x55852ff91c00 2024-05-15T14:12:08.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.233+0000 7f4abed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55852ff8d360 0x55852ff8d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:08.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.233+0000 7f4abed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55852ff8d360 0x55852ff8d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36712/0 (socket says 172.21.15.33:36712) 2024-05-15T14:12:08.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.233+0000 7f4abed9d700 1 -- 172.21.15.33:0/2708686441 learned_addr learned my addr 172.21.15.33:0/2708686441 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:12:08.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.233+0000 7f4abe59c700 1 --2- 172.21.15.33:0/2708686441 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55852ff8dc80 0x55852ff916c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:08.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.233+0000 7f4abf59e700 1 --2- 172.21.15.33:0/2708686441 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852ff91c00 0x55852ff92040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:08.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.233+0000 7f4abed9d700 1 -- 172.21.15.33:0/2708686441 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852ff91c00 msgr2=0x55852ff92040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:12:08.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.233+0000 7f4abed9d700 1 --2- 172.21.15.33:0/2708686441 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852ff91c00 0x55852ff92040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:08.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.233+0000 7f4abed9d700 1 -- 172.21.15.33:0/2708686441 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55852ff8dc80 msgr2=0x55852ff916c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:12:08.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.233+0000 7f4abed9d700 1 --2- 172.21.15.33:0/2708686441 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55852ff8dc80 0x55852ff916c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:08.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.233+0000 7f4abed9d700 1 -- 172.21.15.33:0/2708686441 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55852fe80020 con 0x55852ff8d360 2024-05-15T14:12:08.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.233+0000 7f4abf59e700 1 --2- 172.21.15.33:0/2708686441 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852ff91c00 0x55852ff92040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:12:08.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.233+0000 7f4abed9d700 1 --2- 172.21.15.33:0/2708686441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55852ff8d360 0x55852ff8d740 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4ab000ea00 tx=0x7f4ab0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c377aac8c0907f server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:08.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.233+0000 7f4aa57fa700 1 -- 172.21.15.33:0/2708686441 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab001b040 con 0x55852ff8d360 2024-05-15T14:12:08.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.234+0000 7f4aa57fa700 1 -- 172.21.15.33:0/2708686441 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ab0024070 con 0x55852ff8d360 2024-05-15T14:12:08.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.234+0000 7f4aa57fa700 1 -- 172.21.15.33:0/2708686441 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab0013740 con 0x55852ff8d360 2024-05-15T14:12:08.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.234+0000 7f4acf89ab80 1 -- 172.21.15.33:0/2708686441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55852ff8d360 msgr2=0x55852ff8d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:08.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.234+0000 7f4acf89ab80 1 --2- 172.21.15.33:0/2708686441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55852ff8d360 0x55852ff8d740 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4ab000ea00 tx=0x7f4ab0011000 comp rx=0 tx=0).stop 2024-05-15T14:12:08.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.234+0000 7f4acf89ab80 1 -- 172.21.15.33:0/2708686441 shutdown_connections 2024-05-15T14:12:08.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.234+0000 7f4acf89ab80 1 --2- 172.21.15.33:0/2708686441 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852ff91c00 0x55852ff92040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:08.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.234+0000 7f4acf89ab80 1 --2- 172.21.15.33:0/2708686441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55852ff8d360 0x55852ff8d740 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:08.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.234+0000 7f4acf89ab80 1 --2- 172.21.15.33:0/2708686441 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55852ff8dc80 0x55852ff916c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:08.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.234+0000 7f4acf89ab80 1 -- 172.21.15.33:0/2708686441 >> 172.21.15.33:0/2708686441 conn(0x55852feb5e20 msgr2=0x55852ff835c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:08.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.234+0000 7f4acf89ab80 1 -- 172.21.15.33:0/2708686441 shutdown_connections 2024-05-15T14:12:08.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.234+0000 7f4acf89ab80 1 -- 172.21.15.33:0/2708686441 wait complete. 2024-05-15T14:12:08.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.235+0000 7f4acf89ab80 1 -- start start 2024-05-15T14:12:08.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.235+0000 7f4acf89ab80 1 Processor -- start 2024-05-15T14:12:08.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.235+0000 7f4acf89ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55852ff8d360 0x55852ff99040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:08.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.235+0000 7f4acf89ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55852ff8dc80 0x55852ff963e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:08.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.235+0000 7f4acf89ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852ff91c00 0x55852ff96920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:08.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.235+0000 7f4acf89ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55852fddd010 con 0x55852ff8dc80 2024-05-15T14:12:08.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.235+0000 7f4acf89ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55852fe3ee10 con 0x55852ff8d360 2024-05-15T14:12:08.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.235+0000 7f4acf89ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x55852ff99e00 con 0x55852ff91c00 2024-05-15T14:12:08.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.235+0000 7f4acf89ab80 10 client.0 _ll_register_callbacks cb 0x55852fe719c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:12:08.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.235+0000 7f4abe59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55852ff8dc80 0x55852ff963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:08.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.235+0000 7f4abe59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55852ff8dc80 0x55852ff963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36722/0 (socket says 172.21.15.33:36722) 2024-05-15T14:12:08.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.235+0000 7f4abe59c700 1 -- 172.21.15.33:0/3099873163 learned_addr learned my addr 172.21.15.33:0/3099873163 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:12:08.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.235+0000 7f4abed9d700 1 --2- 172.21.15.33:0/3099873163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55852ff8d360 0x55852ff99040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:08.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.236+0000 7f4abf59e700 1 --2- 172.21.15.33:0/3099873163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852ff91c00 0x55852ff96920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:08.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.236+0000 7f4abe59c700 1 -- 172.21.15.33:0/3099873163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852ff91c00 msgr2=0x55852ff96920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:12:08.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.236+0000 7f4abe59c700 1 --2- 172.21.15.33:0/3099873163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852ff91c00 0x55852ff96920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:08.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.236+0000 7f4abe59c700 1 -- 172.21.15.33:0/3099873163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55852ff8d360 msgr2=0x55852ff99040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:12:08.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.236+0000 7f4abe59c700 1 --2- 172.21.15.33:0/3099873163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55852ff8d360 0x55852ff99040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:08.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.236+0000 7f4abe59c700 1 -- 172.21.15.33:0/3099873163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55852fe26940 con 0x55852ff8dc80 2024-05-15T14:12:08.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.236+0000 7f4abe59c700 1 --2- 172.21.15.33:0/3099873163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55852ff8dc80 0x55852ff963e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4009990 tx=0x7f4ab400bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730d505d11fa9024 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:08.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.236+0000 7f4aa2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:12:08.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.236+0000 7f4aa2ff5700 1 -- 172.21.15.33:0/3099873163 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4003a60 con 0x55852ff8dc80 2024-05-15T14:12:08.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.236+0000 7f4aa2ff5700 1 -- 172.21.15.33:0/3099873163 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8586+0+0 (secure 0 0 0) 0x7f4ab4003c00 con 0x55852ff8dc80 2024-05-15T14:12:08.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.236+0000 7f4acf89ab80 1 -- 172.21.15.33:0/3099873163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ab0024070 con 0x55852ff8dc80 2024-05-15T14:12:08.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.236+0000 7f4aa2ff5700 1 -- 172.21.15.33:0/3099873163 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4ab4031050 con 0x55852ff8dc80 2024-05-15T14:12:08.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.236+0000 7f4acf89ab80 10 client.16803 fetch_fsmap learned FSMap version 39 2024-05-15T14:12:08.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.236+0000 7f4acf89ab80 1 -- 172.21.15.33:0/3099873163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55852fe907d0 con 0x55852ff8dc80 2024-05-15T14:12:08.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.236+0000 7f4aa2ff5700 1 -- 172.21.15.33:0/3099873163 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ab0024070 con 0x55852ff8dc80 2024-05-15T14:12:08.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.237+0000 7f4aa2ff5700 1 -- 172.21.15.33:0/3099873163 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f4ab400f800 con 0x55852ff8dc80 2024-05-15T14:12:08.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.237+0000 7f4acf89ab80 10 client.16803 fetch_fsmap finished waiting for FSMap version 39 2024-05-15T14:12:08.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.237+0000 7f4acf89ab80 10 client.16803 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:12:08.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.237+0000 7f4acf89ab80 20 client.16803 populate_metadata read hostname 'smithi033' 2024-05-15T14:12:08.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.237+0000 7f4acf89ab80 1 --2- 172.21.15.33:0/3099873163 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55852ffa6960 0x55852ffa8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:08.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.237+0000 7f4acf89ab80 4 client.16803 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-15T14:12:08.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.237+0000 7f4acf89ab80 1 -- 172.21.15.33:0/3099873163 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55852fee8d30 con 0x55852ffa6960 2024-05-15T14:12:08.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.237+0000 7f4abed9d700 1 --2- 172.21.15.33:0/3099873163 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55852ffa6960 0x55852ffa8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:08.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.237+0000 7f4abed9d700 1 --2- 172.21.15.33:0/3099873163 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55852ffa6960 0x55852ffa8d50 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8d93e57f9b6ce6b server_cookie=c1f526d683c054f2 in_seq=0 out_seq=0 2024-05-15T14:12:08.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.237+0000 7f4aa2ff5700 10 client.16803 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:12:08.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.238+0000 7f4aa2ff5700 1 -- 172.21.15.33:0/3099873163 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55852fee8d30 con 0x55852ffa6960 2024-05-15T14:12:08.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.238+0000 7f4aa2ff5700 10 client.16803 handle_command_reply: tid=0 2024-05-15T14:12:08.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.238+0000 7f4acf89ab80 1 client.16803 shutdown 2024-05-15T14:12:08.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.238+0000 7f4acf89ab80 1 -- 172.21.15.33:0/3099873163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55852ff8dc80 msgr2=0x55852ff963e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:08.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.238+0000 7f4acf89ab80 1 --2- 172.21.15.33:0/3099873163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55852ff8dc80 0x55852ff963e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4009990 tx=0x7f4ab400bf20 comp rx=0 tx=0).stop 2024-05-15T14:12:08.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.238+0000 7f4acf89ab80 1 -- 172.21.15.33:0/3099873163 shutdown_connections 2024-05-15T14:12:08.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.238+0000 7f4acf89ab80 1 --2- 172.21.15.33:0/3099873163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852ff91c00 0x55852ff96920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:08.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.238+0000 7f4acf89ab80 1 --2- 172.21.15.33:0/3099873163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55852ff8d360 0x55852ff99040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:08.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.238+0000 7f4acf89ab80 1 --2- 172.21.15.33:0/3099873163 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x55852ffa6960 0x55852ffa8d50 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:08.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.238+0000 7f4acf89ab80 1 --2- 172.21.15.33:0/3099873163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55852ff8dc80 0x55852ff963e0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:08.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.239+0000 7f4aa2ff5700 0 client.16803 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:12:08.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.239+0000 7f4acf89ab80 1 -- 172.21.15.33:0/3099873163 >> 172.21.15.33:0/3099873163 conn(0x55852feb5e20 msgr2=0x55852ff7e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:08.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.239+0000 7f4acf89ab80 1 -- 172.21.15.33:0/3099873163 shutdown_connections 2024-05-15T14:12:08.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.239+0000 7f4acf89ab80 1 -- 172.21.15.33:0/3099873163 wait complete. 2024-05-15T14:12:08.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.239+0000 7f4acf89ab80 20 client.16803 trim_cache size 0 max 16384 2024-05-15T14:12:08.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.259+0000 7f4acf89ab80 1 Processor -- start 2024-05-15T14:12:08.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.259+0000 7f4acf89ab80 1 -- start start 2024-05-15T14:12:08.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.259+0000 7f4acf89ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852ff8d360 0x55852ffae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:08.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.259+0000 7f4acf89ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55852ff8dc80 0x55852ffaeb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:08.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.259+0000 7f4acf89ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55852ff91c00 0x55852ff9f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:08.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.259+0000 7f4acf89ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac01ae5c0 con 0x55852ff8dc80 2024-05-15T14:12:08.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.259+0000 7f4acf89ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac01abdf0 con 0x55852ff91c00 2024-05-15T14:12:08.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.259+0000 7f4acf89ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac01a9c60 con 0x55852ff8d360 2024-05-15T14:12:08.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.259+0000 7f4abe59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55852ff8dc80 0x55852ffaeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:08.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.259+0000 7f4abe59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55852ff8dc80 0x55852ffaeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36738/0 (socket says 172.21.15.33:36738) 2024-05-15T14:12:08.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.259+0000 7f4abe59c700 1 -- 172.21.15.33:0/3920308251 learned_addr learned my addr 172.21.15.33:0/3920308251 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:12:08.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.259+0000 7f4abe59c700 1 -- 172.21.15.33:0/3920308251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852ff8d360 msgr2=0x55852ffae5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:12:08.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.259+0000 7f4abed9d700 1 --2- 172.21.15.33:0/3920308251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852ff8d360 0x55852ffae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:08.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.259+0000 7f4abe59c700 1 --2- 172.21.15.33:0/3920308251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852ff8d360 0x55852ffae5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:08.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.259+0000 7f4abf59e700 1 --2- 172.21.15.33:0/3920308251 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55852ff91c00 0x55852ff9f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:08.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.259+0000 7f4abe59c700 1 -- 172.21.15.33:0/3920308251 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55852ff91c00 msgr2=0x55852ff9f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:12:08.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.259+0000 7f4abe59c700 1 --2- 172.21.15.33:0/3920308251 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55852ff91c00 0x55852ff9f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:08.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.259+0000 7f4abe59c700 1 -- 172.21.15.33:0/3920308251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55852ff953a0 con 0x55852ff8dc80 2024-05-15T14:12:08.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.259+0000 7f4abf59e700 1 --2- 172.21.15.33:0/3920308251 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55852ff91c00 0x55852ff9f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:12:08.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.259+0000 7f4abe59c700 1 --2- 172.21.15.33:0/3920308251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55852ff8dc80 0x55852ffaeb10 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40098e0 tx=0x7f4ab40037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e646306ffb170eb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:08.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.260+0000 7f4aa4ff9700 1 -- 172.21.15.33:0/3920308251 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4039070 con 0x55852ff8dc80 2024-05-15T14:12:08.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.260+0000 7f4aa4ff9700 1 -- 172.21.15.33:0/3920308251 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ab400fe10 con 0x55852ff8dc80 2024-05-15T14:12:08.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.260+0000 7f4aa4ff9700 1 -- 172.21.15.33:0/3920308251 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4005320 con 0x55852ff8dc80 2024-05-15T14:12:08.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.260+0000 7f4acf89ab80 1 -- 172.21.15.33:0/3920308251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55852ff8dc80 msgr2=0x55852ffaeb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:08.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.260+0000 7f4acf89ab80 1 --2- 172.21.15.33:0/3920308251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55852ff8dc80 0x55852ffaeb10 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40098e0 tx=0x7f4ab40037c0 comp rx=0 tx=0).stop 2024-05-15T14:12:08.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.260+0000 7f4acf89ab80 1 -- 172.21.15.33:0/3920308251 shutdown_connections 2024-05-15T14:12:08.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.260+0000 7f4acf89ab80 1 --2- 172.21.15.33:0/3920308251 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55852ff91c00 0x55852ff9f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:08.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.260+0000 7f4acf89ab80 1 --2- 172.21.15.33:0/3920308251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852ff8d360 0x55852ffae5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:08.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.260+0000 7f4acf89ab80 1 --2- 172.21.15.33:0/3920308251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55852ff8dc80 0x55852ffaeb10 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:08.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.260+0000 7f4acf89ab80 1 -- 172.21.15.33:0/3920308251 >> 172.21.15.33:0/3920308251 conn(0x55852feb5e20 msgr2=0x55852ff885f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:08.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.260+0000 7f4acf89ab80 1 -- 172.21.15.33:0/3920308251 shutdown_connections 2024-05-15T14:12:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.260+0000 7f4acf89ab80 1 -- 172.21.15.33:0/3920308251 wait complete. 2024-05-15T14:12:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.261+0000 7f4acf89ab80 1 -- start start 2024-05-15T14:12:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.261+0000 7f4acf89ab80 1 Processor -- start 2024-05-15T14:12:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.261+0000 7f4acf89ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55852ff8d360 0x55852ff9f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.261+0000 7f4acf89ab80 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852ff8dc80 0x55852ff9daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.261+0000 7f4acf89ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55852ff91c00 0x55852ff9e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.261+0000 7f4acf89ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac01aa2b0 con 0x55852ff8d360 2024-05-15T14:12:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.261+0000 7f4acf89ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac01ab720 con 0x55852ff91c00 2024-05-15T14:12:08.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.261+0000 7f4acf89ab80 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac01aa980 con 0x55852ff8dc80 2024-05-15T14:12:08.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.261+0000 7f4acf89ab80 10 client.0 _ll_register_callbacks cb 0x55852fe719c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-15T14:12:08.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.261+0000 7f4abed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55852ff8d360 0x55852ff9f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:08.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.261+0000 7f4abed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55852ff8d360 0x55852ff9f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36742/0 (socket says 172.21.15.33:36742) 2024-05-15T14:12:08.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.261+0000 7f4abed9d700 1 -- 172.21.15.33:0/4229779355 learned_addr learned my addr 172.21.15.33:0/4229779355 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:12:08.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.261+0000 7f4abe59c700 1 --2- 172.21.15.33:0/4229779355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852ff8dc80 0x55852ff9daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:08.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.261+0000 7f4abed9d700 1 -- 172.21.15.33:0/4229779355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852ff8dc80 msgr2=0x55852ff9daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:12:08.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.262+0000 7f4abf59e700 1 --2- 172.21.15.33:0/4229779355 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55852ff91c00 0x55852ff9e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:08.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.262+0000 7f4abed9d700 1 --2- 172.21.15.33:0/4229779355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852ff8dc80 0x55852ff9daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:08.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.262+0000 7f4abed9d700 1 -- 172.21.15.33:0/4229779355 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55852ff91c00 msgr2=0x55852ff9e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-15T14:12:08.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.262+0000 7f4abed9d700 1 --2- 172.21.15.33:0/4229779355 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55852ff91c00 0x55852ff9e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:08.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.262+0000 7f4abed9d700 1 -- 172.21.15.33:0/4229779355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585300111a0 con 0x55852ff8d360 2024-05-15T14:12:08.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.262+0000 7f4abe59c700 1 --2- 172.21.15.33:0/4229779355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852ff8dc80 0x55852ff9daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:12:08.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.262+0000 7f4abed9d700 1 --2- 172.21.15.33:0/4229779355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55852ff8d360 0x55852ff9f6a0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0015e70 tx=0x7f4ab0015500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0155dd275336d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:08.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.262+0000 7f4aa2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-15T14:12:08.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.262+0000 7f4aa2ff5700 1 -- 172.21.15.33:0/4229779355 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab00159a0 con 0x55852ff8d360 2024-05-15T14:12:08.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.262+0000 7f4aa2ff5700 1 -- 172.21.15.33:0/4229779355 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8586+0+0 (secure 0 0 0) 0x7f4ab0033080 con 0x55852ff8d360 2024-05-15T14:12:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.262+0000 7f4acf89ab80 1 -- 172.21.15.33:0/4229779355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ab400fe10 con 0x55852ff8d360 2024-05-15T14:12:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.262+0000 7f4aa2ff5700 1 -- 172.21.15.33:0/4229779355 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4ab0015b40 con 0x55852ff8d360 2024-05-15T14:12:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.262+0000 7f4acf89ab80 10 client.16809 fetch_fsmap learned FSMap version 39 2024-05-15T14:12:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.262+0000 7f4acf89ab80 1 -- 172.21.15.33:0/4229779355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558530012250 con 0x55852ff8d360 2024-05-15T14:12:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.262+0000 7f4aa2ff5700 1 -- 172.21.15.33:0/4229779355 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ab400fe10 con 0x55852ff8d360 2024-05-15T14:12:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.262+0000 7f4aa2ff5700 1 -- 172.21.15.33:0/4229779355 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f4ab003a070 con 0x55852ff8d360 2024-05-15T14:12:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.263+0000 7f4acf89ab80 10 client.16809 fetch_fsmap finished waiting for FSMap version 39 2024-05-15T14:12:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.263+0000 7f4acf89ab80 10 client.16809 resolve_mds: validated gid 24478 aka daemon mds.i 2024-05-15T14:12:08.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.263+0000 7f4acf89ab80 20 client.16809 populate_metadata read hostname 'smithi033' 2024-05-15T14:12:08.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.263+0000 7f4acf89ab80 1 --2- 172.21.15.33:0/4229779355 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5585300126b0 0x558530014ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:08.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.263+0000 7f4acf89ab80 4 client.16809 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-05-15T14:12:08.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.263+0000 7f4acf89ab80 1 -- 172.21.15.33:0/4229779355 --> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55852fee2510 con 0x5585300126b0 2024-05-15T14:12:08.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.263+0000 7f4abe59c700 1 --2- 172.21.15.33:0/4229779355 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5585300126b0 0x558530014ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:08.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.263+0000 7f4abe59c700 1 --2- 172.21.15.33:0/4229779355 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5585300126b0 0x558530014ad0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc0b99394ac68bcb server_cookie=c860199698e2c707 in_seq=0 out_seq=0 2024-05-15T14:12:08.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.263+0000 7f4aa2ff5700 10 client.16809 ms_handle_connect on v2:172.21.15.81:6838/1554395940 2024-05-15T14:12:08.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.264+0000 7f4aa2ff5700 1 -- 172.21.15.33:0/4229779355 <== mds.0 v2:172.21.15.81:6838/1554395940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55852fee2510 con 0x5585300126b0 2024-05-15T14:12:08.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.264+0000 7f4aa2ff5700 10 client.16809 handle_command_reply: tid=0 2024-05-15T14:12:08.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.264+0000 7f4acf89ab80 1 client.16809 shutdown 2024-05-15T14:12:08.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.264+0000 7f4acf89ab80 1 -- 172.21.15.33:0/4229779355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55852ff8d360 msgr2=0x55852ff9f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:08.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.264+0000 7f4acf89ab80 1 --2- 172.21.15.33:0/4229779355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55852ff8d360 0x55852ff9f6a0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0015e70 tx=0x7f4ab0015500 comp rx=0 tx=0).stop 2024-05-15T14:12:08.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.264+0000 7f4acf89ab80 1 -- 172.21.15.33:0/4229779355 shutdown_connections 2024-05-15T14:12:08.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.264+0000 7f4acf89ab80 1 --2- 172.21.15.33:0/4229779355 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55852ff91c00 0x55852ff9e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:08.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.264+0000 7f4acf89ab80 1 --2- 172.21.15.33:0/4229779355 >> [v2:172.21.15.81:6838/1554395940,v1:172.21.15.81:6839/1554395940] conn(0x5585300126b0 0x558530014ad0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:08.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.264+0000 7f4acf89ab80 1 --2- 172.21.15.33:0/4229779355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55852ff8d360 0x55852ff9f6a0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:08.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.264+0000 7f4acf89ab80 1 --2- 172.21.15.33:0/4229779355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55852ff8dc80 0x55852ff9daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:08.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.264+0000 7f4acf89ab80 1 -- 172.21.15.33:0/4229779355 >> 172.21.15.33:0/4229779355 conn(0x55852feb5e20 msgr2=0x55852ff7d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:08.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.264+0000 7f4aa2ff5700 0 client.16809 ms_handle_reset on v2:172.21.15.81:6838/1554395940 2024-05-15T14:12:08.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.264+0000 7f4acf89ab80 1 -- 172.21.15.33:0/4229779355 shutdown_connections 2024-05-15T14:12:08.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.264+0000 7f4acf89ab80 1 -- 172.21.15.33:0/4229779355 wait complete. 2024-05-15T14:12:08.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.264+0000 7f4acf89ab80 20 client.16809 trim_cache size 0 max 16384 2024-05-15T14:12:08.275 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T14:12:08.276 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 39, 2024-05-15T14:12:08.276 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24478, 2024-05-15T14:12:08.276 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 40, 2024-05-15T14:12:08.276 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": {} 2024-05-15T14:12:08.276 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T14:12:08.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.265+0000 7f4ac4803700 1 -- 172.21.15.33:0/1750167095 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f4aa80b2a70 msgr2=0x7f4aa80b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:08.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.265+0000 7f4ac4803700 1 --2- 172.21.15.33:0/1750167095 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f4aa80b2a70 0x7f4aa80b4f30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0009a10 tx=0x7f4ab0009190 comp rx=0 tx=0).stop 2024-05-15T14:12:08.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.265+0000 7f4ac4803700 1 -- 172.21.15.33:0/1750167095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ac010db00 msgr2=0x7f4ac0199b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:08.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.265+0000 7f4ac4803700 1 --2- 172.21.15.33:0/1750167095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ac010db00 0x7f4ac0199b10 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8007ee0 tx=0x7f4ab800d3b0 comp rx=0 tx=0).stop 2024-05-15T14:12:08.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.265+0000 7f4ac4803700 1 -- 172.21.15.33:0/1750167095 shutdown_connections 2024-05-15T14:12:08.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.265+0000 7f4ac4803700 1 --2- 172.21.15.33:0/1750167095 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f4aa80b2a70 0x7f4aa80b4f30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:08.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.265+0000 7f4ac4803700 1 --2- 172.21.15.33:0/1750167095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ac010db00 0x7f4ac0199b10 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:08.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.265+0000 7f4ac4803700 1 --2- 172.21.15.33:0/1750167095 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4ac0104720 0x7f4ac019fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:08.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.265+0000 7f4ac4803700 1 --2- 172.21.15.33:0/1750167095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4ac0105040 0x7f4ac019ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:08.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.265+0000 7f4ac4803700 1 -- 172.21.15.33:0/1750167095 >> 172.21.15.33:0/1750167095 conn(0x7f4ac00fdef0 msgr2=0x7f4ac00fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:08.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.265+0000 7f4ac4803700 1 -- 172.21.15.33:0/1750167095 shutdown_connections 2024-05-15T14:12:08.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:08.265+0000 7f4ac4803700 1 -- 172.21.15.33:0/1750167095 wait complete. 2024-05-15T14:12:08.636 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:08 smithi033 ceph-mon[31650]: pgmap v370: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 258 op/s 2024-05-15T14:12:08.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:08 smithi052 ceph-mon[36190]: pgmap v370: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 258 op/s 2024-05-15T14:12:08.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:08 smithi081 ceph-mon[36434]: pgmap v370: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 258 op/s 2024-05-15T14:12:09.004 INFO:tasks.quiescer:done joining Quiescers 2024-05-15T14:12:09.005 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-05-15T14:12:09.015 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi033.front.sepia.ceph.com 2024-05-15T14:12:09.024 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-05-15T14:12:09.034 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-05-15T14:12:09.034 INFO:teuthology.task.exec:Executing custom commands... 2024-05-15T14:12:09.034 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-05-15T14:12:09.034 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-05-15T14:12:09.097 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-05-15T14:12:09.167 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-15T14:12:09.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.316+0000 7f4823b5f640 1 -- 172.21.15.33:0/3904549311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f481c108a70 msgr2=0x7f481c108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:09.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.316+0000 7f4823b5f640 1 --2- 172.21.15.33:0/3904549311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f481c108a70 0x7f481c108e50 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f4814009cd0 tx=0x7f481402a8e0 comp rx=0 tx=0).stop 2024-05-15T14:12:09.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.316+0000 7f4823b5f640 1 -- 172.21.15.33:0/3904549311 shutdown_connections 2024-05-15T14:12:09.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.316+0000 7f4823b5f640 1 --2- 172.21.15.33:0/3904549311 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f481c103420 0x7f481c10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:09.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.316+0000 7f4823b5f640 1 --2- 172.21.15.33:0/3904549311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f481c102a60 0x7f481c102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:09.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.316+0000 7f4823b5f640 1 --2- 172.21.15.33:0/3904549311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f481c108a70 0x7f481c108e50 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:09.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.316+0000 7f4823b5f640 1 -- 172.21.15.33:0/3904549311 >> 172.21.15.33:0/3904549311 conn(0x7f481c0fe530 msgr2=0x7f481c100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:09.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.316+0000 7f4823b5f640 1 -- 172.21.15.33:0/3904549311 shutdown_connections 2024-05-15T14:12:09.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.316+0000 7f4823b5f640 1 -- 172.21.15.33:0/3904549311 wait complete. 2024-05-15T14:12:09.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.317+0000 7f4823b5f640 1 Processor -- start 2024-05-15T14:12:09.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.317+0000 7f4823b5f640 1 -- start start 2024-05-15T14:12:09.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.317+0000 7f4823b5f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f481c102a60 0x7f481c199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:09.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.317+0000 7f4823b5f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f481c103420 0x7f481c1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:09.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.317+0000 7f4823b5f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f481c108a70 0x7f481c19db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:09.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.317+0000 7f4823b5f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f481c112d50 con 0x7f481c108a70 2024-05-15T14:12:09.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.317+0000 7f4823b5f640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f481c112bd0 con 0x7f481c103420 2024-05-15T14:12:09.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.317+0000 7f4823b5f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f481c112ed0 con 0x7f481c102a60 2024-05-15T14:12:09.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.317+0000 7f48220fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f481c108a70 0x7f481c19db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:09.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.317+0000 7f48220fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f481c108a70 0x7f481c19db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36774/0 (socket says 172.21.15.33:36774) 2024-05-15T14:12:09.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.317+0000 7f48218fb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f481c102a60 0x7f481c199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:09.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.317+0000 7f48220fc640 1 -- 172.21.15.33:0/4075352804 learned_addr learned my addr 172.21.15.33:0/4075352804 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:12:09.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.317+0000 7f48210fa640 1 --2- 172.21.15.33:0/4075352804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f481c103420 0x7f481c1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:09.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.317+0000 7f48220fc640 1 -- 172.21.15.33:0/4075352804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f481c102a60 msgr2=0x7f481c199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:09.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.317+0000 7f48220fc640 1 --2- 172.21.15.33:0/4075352804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f481c102a60 0x7f481c199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:09.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.317+0000 7f48220fc640 1 -- 172.21.15.33:0/4075352804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f481c103420 msgr2=0x7f481c1997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:09.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.317+0000 7f48220fc640 1 --2- 172.21.15.33:0/4075352804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f481c103420 0x7f481c1997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:09.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.317+0000 7f48220fc640 1 -- 172.21.15.33:0/4075352804 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481c19e280 con 0x7f481c108a70 2024-05-15T14:12:09.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.318+0000 7f48220fc640 1 --2- 172.21.15.33:0/4075352804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f481c108a70 0x7f481c19db50 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f480c00c970 tx=0x7f480c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:09.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.318+0000 7f481affd640 1 -- 172.21.15.33:0/4075352804 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c007bb0 con 0x7f481c108a70 2024-05-15T14:12:09.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.318+0000 7f481affd640 1 -- 172.21.15.33:0/4075352804 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f480c007d50 con 0x7f481c108a70 2024-05-15T14:12:09.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.318+0000 7f481affd640 1 -- 172.21.15.33:0/4075352804 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c004e60 con 0x7f481c108a70 2024-05-15T14:12:09.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.318+0000 7f4823b5f640 1 -- 172.21.15.33:0/4075352804 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f481c19e570 con 0x7f481c108a70 2024-05-15T14:12:09.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.318+0000 7f4823b5f640 1 -- 172.21.15.33:0/4075352804 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f481c1a5e00 con 0x7f481c108a70 2024-05-15T14:12:09.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.319+0000 7f4823b5f640 1 -- 172.21.15.33:0/4075352804 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f481c051030 con 0x7f481c108a70 2024-05-15T14:12:09.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.320+0000 7f481affd640 1 -- 172.21.15.33:0/4075352804 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f480c005020 con 0x7f481c108a70 2024-05-15T14:12:09.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.321+0000 7f481affd640 1 --2- 172.21.15.33:0/4075352804 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f48040b29d0 0x7f48040b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:09.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.321+0000 7f48218fb640 1 --2- 172.21.15.33:0/4075352804 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f48040b29d0 0x7f48040b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:09.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.321+0000 7f481affd640 1 -- 172.21.15.33:0/4075352804 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8586+0+0 (secure 0 0 0) 0x7f480c0e2280 con 0x7f481c108a70 2024-05-15T14:12:09.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.322+0000 7f48218fb640 1 --2- 172.21.15.33:0/4075352804 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f48040b29d0 0x7f48040b4e90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4814009ca0 tx=0x7f4814009380 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:09.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.326+0000 7f481affd640 1 -- 172.21.15.33:0/4075352804 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f480c014030 con 0x7f481c108a70 2024-05-15T14:12:09.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.544+0000 7f4823b5f640 1 -- 172.21.15.33:0/4075352804 --> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f481c062e90 con 0x7f48040b29d0 2024-05-15T14:12:09.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.552+0000 7f481affd640 1 -- 172.21.15.33:0/4075352804 <== mgr.15144 v2:172.21.15.33:6800/1796050977 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f481c062e90 con 0x7f48040b29d0 2024-05-15T14:12:09.554 INFO:teuthology.orchestra.run.smithi033.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-15T00:00:00", "created": "2024-05-15T14:00:22", "first": "2024-05-15T14:01:00", "last": "2024-05-15T14:12:00", "last_pruned": "2024-05-15T14:12:00", "created_count": 12, "pruned_count": 6, "active": true} 2024-05-15T14:12:09.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.555+0000 7f4823b5f640 1 -- 172.21.15.33:0/4075352804 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f48040b29d0 msgr2=0x7f48040b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:09.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.555+0000 7f4823b5f640 1 --2- 172.21.15.33:0/4075352804 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f48040b29d0 0x7f48040b4e90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4814009ca0 tx=0x7f4814009380 comp rx=0 tx=0).stop 2024-05-15T14:12:09.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.555+0000 7f4823b5f640 1 -- 172.21.15.33:0/4075352804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f481c108a70 msgr2=0x7f481c19db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:09.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.555+0000 7f4823b5f640 1 --2- 172.21.15.33:0/4075352804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f481c108a70 0x7f481c19db50 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f480c00c970 tx=0x7f480c00ce30 comp rx=0 tx=0).stop 2024-05-15T14:12:09.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.555+0000 7f4823b5f640 1 -- 172.21.15.33:0/4075352804 shutdown_connections 2024-05-15T14:12:09.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.555+0000 7f4823b5f640 1 --2- 172.21.15.33:0/4075352804 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f48040b29d0 0x7f48040b4e90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:09.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.555+0000 7f4823b5f640 1 --2- 172.21.15.33:0/4075352804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f481c108a70 0x7f481c19db50 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:09.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.555+0000 7f4823b5f640 1 --2- 172.21.15.33:0/4075352804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f481c103420 0x7f481c1997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:09.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.555+0000 7f4823b5f640 1 --2- 172.21.15.33:0/4075352804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f481c102a60 0x7f481c199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:09.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.555+0000 7f4823b5f640 1 -- 172.21.15.33:0/4075352804 >> 172.21.15.33:0/4075352804 conn(0x7f481c0fe530 msgr2=0x7f481c0fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:09.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.555+0000 7f4823b5f640 1 -- 172.21.15.33:0/4075352804 shutdown_connections 2024-05-15T14:12:09.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.555+0000 7f4823b5f640 1 -- 172.21.15.33:0/4075352804 wait complete. 2024-05-15T14:12:09.567 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-15T14:12:09.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.672+0000 7fd1af69b640 1 -- 172.21.15.33:0/1557141901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1a8104600 msgr2=0x7fd1a810cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:09.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.672+0000 7fd1af69b640 1 --2- 172.21.15.33:0/1557141901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1a8104600 0x7fd1a810cae0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fd198009900 tx=0x7fd19802a480 comp rx=0 tx=0).stop 2024-05-15T14:12:09.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.672+0000 7fd1af69b640 1 -- 172.21.15.33:0/1557141901 shutdown_connections 2024-05-15T14:12:09.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.672+0000 7fd1af69b640 1 --2- 172.21.15.33:0/1557141901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1a810d0b0 0x7fd1a810f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:09.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.672+0000 7fd1af69b640 1 --2- 172.21.15.33:0/1557141901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1a8104600 0x7fd1a810cae0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:09.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.672+0000 7fd1af69b640 1 --2- 172.21.15.33:0/1557141901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1a8103ce0 0x7fd1a81040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:09.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.672+0000 7fd1af69b640 1 -- 172.21.15.33:0/1557141901 >> 172.21.15.33:0/1557141901 conn(0x7fd1a8078ad0 msgr2=0x7fd1a8078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:09.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.672+0000 7fd1af69b640 1 -- 172.21.15.33:0/1557141901 shutdown_connections 2024-05-15T14:12:09.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.672+0000 7fd1af69b640 1 -- 172.21.15.33:0/1557141901 wait complete. 2024-05-15T14:12:09.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.673+0000 7fd1af69b640 1 Processor -- start 2024-05-15T14:12:09.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.673+0000 7fd1af69b640 1 -- start start 2024-05-15T14:12:09.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.673+0000 7fd1af69b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1a8103ce0 0x7fd1a8101930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:09.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.673+0000 7fd1af69b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1a8104600 0x7fd1a8101e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:09.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.673+0000 7fd1af69b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1a810d0b0 0x7fd1a81023b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:09.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.673+0000 7fd1af69b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a8112cf0 con 0x7fd1a8103ce0 2024-05-15T14:12:09.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.673+0000 7fd1af69b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a8112b70 con 0x7fd1a8104600 2024-05-15T14:12:09.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.673+0000 7fd1af69b640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a8112e70 con 0x7fd1a810d0b0 2024-05-15T14:12:09.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.673+0000 7fd1ad437640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1a8103ce0 0x7fd1a8101930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:09.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.673+0000 7fd1ad437640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1a8103ce0 0x7fd1a8101930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36798/0 (socket says 172.21.15.33:36798) 2024-05-15T14:12:09.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.673+0000 7fd1ad437640 1 -- 172.21.15.33:0/2785889507 learned_addr learned my addr 172.21.15.33:0/2785889507 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:12:09.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.673+0000 7fd1acc36640 1 --2- 172.21.15.33:0/2785889507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1a8104600 0x7fd1a8101e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:09.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.673+0000 7fd1adc38640 1 --2- 172.21.15.33:0/2785889507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1a810d0b0 0x7fd1a81023b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:09.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.673+0000 7fd1ad437640 1 -- 172.21.15.33:0/2785889507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1a810d0b0 msgr2=0x7fd1a81023b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:09.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.673+0000 7fd1ad437640 1 --2- 172.21.15.33:0/2785889507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1a810d0b0 0x7fd1a81023b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:09.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.673+0000 7fd1ad437640 1 -- 172.21.15.33:0/2785889507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1a8104600 msgr2=0x7fd1a8101e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:09.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.673+0000 7fd1ad437640 1 --2- 172.21.15.33:0/2785889507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1a8104600 0x7fd1a8101e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:09.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.673+0000 7fd1ad437640 1 -- 172.21.15.33:0/2785889507 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1a81a76d0 con 0x7fd1a8103ce0 2024-05-15T14:12:09.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.673+0000 7fd1adc38640 1 --2- 172.21.15.33:0/2785889507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1a810d0b0 0x7fd1a81023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:12:09.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.673+0000 7fd1ad437640 1 --2- 172.21.15.33:0/2785889507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1a8103ce0 0x7fd1a8101930 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fd1a400c920 tx=0x7fd1a400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:09.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.674+0000 7fd19e7fc640 1 -- 172.21.15.33:0/2785889507 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4007c70 con 0x7fd1a8103ce0 2024-05-15T14:12:09.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.674+0000 7fd1af69b640 1 -- 172.21.15.33:0/2785889507 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1a81a79c0 con 0x7fd1a8103ce0 2024-05-15T14:12:09.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.674+0000 7fd19e7fc640 1 -- 172.21.15.33:0/2785889507 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1a4007e10 con 0x7fd1a8103ce0 2024-05-15T14:12:09.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.674+0000 7fd19e7fc640 1 -- 172.21.15.33:0/2785889507 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4018820 con 0x7fd1a8103ce0 2024-05-15T14:12:09.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.674+0000 7fd1af69b640 1 -- 172.21.15.33:0/2785889507 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1a81a7ee0 con 0x7fd1a8103ce0 2024-05-15T14:12:09.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.675+0000 7fd1af69b640 1 -- 172.21.15.33:0/2785889507 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1a806b8b0 con 0x7fd1a8103ce0 2024-05-15T14:12:09.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.676+0000 7fd19e7fc640 1 -- 172.21.15.33:0/2785889507 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd1a400f450 con 0x7fd1a8103ce0 2024-05-15T14:12:09.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.677+0000 7fd19e7fc640 1 --2- 172.21.15.33:0/2785889507 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fd1940b2980 0x7fd1940b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:09.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.677+0000 7fd1acc36640 1 --2- 172.21.15.33:0/2785889507 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fd1940b2980 0x7fd1940b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:09.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.677+0000 7fd19e7fc640 1 -- 172.21.15.33:0/2785889507 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8586+0+0 (secure 0 0 0) 0x7fd1a40e2ab0 con 0x7fd1a8103ce0 2024-05-15T14:12:09.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.677+0000 7fd1acc36640 1 --2- 172.21.15.33:0/2785889507 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fd1940b2980 0x7fd1940b4e40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd198031440 tx=0x7fd1980095f0 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:09.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.680+0000 7fd19e7fc640 1 -- 172.21.15.33:0/2785889507 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd1a4014030 con 0x7fd1a8103ce0 2024-05-15T14:12:09.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.894+0000 7fd1af69b640 1 -- 172.21.15.33:0/2785889507 --> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fd1a8062e90 con 0x7fd1940b2980 2024-05-15T14:12:09.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.901+0000 7fd19e7fc640 1 -- 172.21.15.33:0/2785889507 <== mgr.15144 v2:172.21.15.33:6800/1796050977 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fd1a8062e90 con 0x7fd1940b2980 2024-05-15T14:12:09.903 INFO:teuthology.orchestra.run.smithi033.stdout:/ 1m 6m3h 2024-05-15T14:12:09.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.904+0000 7fd1af69b640 1 -- 172.21.15.33:0/2785889507 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fd1940b2980 msgr2=0x7fd1940b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:09.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.904+0000 7fd1af69b640 1 --2- 172.21.15.33:0/2785889507 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fd1940b2980 0x7fd1940b4e40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd198031440 tx=0x7fd1980095f0 comp rx=0 tx=0).stop 2024-05-15T14:12:09.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.904+0000 7fd1af69b640 1 -- 172.21.15.33:0/2785889507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1a8103ce0 msgr2=0x7fd1a8101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:09.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.904+0000 7fd1af69b640 1 --2- 172.21.15.33:0/2785889507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1a8103ce0 0x7fd1a8101930 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fd1a400c920 tx=0x7fd1a400cde0 comp rx=0 tx=0).stop 2024-05-15T14:12:09.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.904+0000 7fd1af69b640 1 -- 172.21.15.33:0/2785889507 shutdown_connections 2024-05-15T14:12:09.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.904+0000 7fd1af69b640 1 --2- 172.21.15.33:0/2785889507 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fd1940b2980 0x7fd1940b4e40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:09.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.904+0000 7fd1af69b640 1 --2- 172.21.15.33:0/2785889507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1a810d0b0 0x7fd1a81023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:09.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.904+0000 7fd1af69b640 1 --2- 172.21.15.33:0/2785889507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1a8104600 0x7fd1a8101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:09.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.904+0000 7fd1af69b640 1 --2- 172.21.15.33:0/2785889507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1a8103ce0 0x7fd1a8101930 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:09.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.904+0000 7fd1af69b640 1 -- 172.21.15.33:0/2785889507 >> 172.21.15.33:0/2785889507 conn(0x7fd1a8078ad0 msgr2=0x7fd1a80feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:09.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.904+0000 7fd1af69b640 1 -- 172.21.15.33:0/2785889507 shutdown_connections 2024-05-15T14:12:09.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:09.904+0000 7fd1af69b640 1 -- 172.21.15.33:0/2785889507 wait complete. 2024-05-15T14:12:09.917 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-15T14:12:09.928 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-15T14:12:10.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.031+0000 7f0c87d1c640 1 -- 172.21.15.33:0/2414870578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c80108c50 msgr2=0x7f0c80109030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:10.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.031+0000 7f0c87d1c640 1 --2- 172.21.15.33:0/2414870578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c80108c50 0x7f0c80109030 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f0c74009930 tx=0x7f0c7402a420 comp rx=0 tx=0).stop 2024-05-15T14:12:10.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.031+0000 7f0c87d1c640 1 -- 172.21.15.33:0/2414870578 shutdown_connections 2024-05-15T14:12:10.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.031+0000 7f0c87d1c640 1 --2- 172.21.15.33:0/2414870578 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c80101cd0 0x7f0c8010f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:10.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.031+0000 7f0c87d1c640 1 --2- 172.21.15.33:0/2414870578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c80101310 0x7f0c80101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:10.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.031+0000 7f0c87d1c640 1 --2- 172.21.15.33:0/2414870578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c80108c50 0x7f0c80109030 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:10.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.031+0000 7f0c87d1c640 1 -- 172.21.15.33:0/2414870578 >> 172.21.15.33:0/2414870578 conn(0x7f0c80076c00 msgr2=0x7f0c80077010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:10.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.031+0000 7f0c87d1c640 1 -- 172.21.15.33:0/2414870578 shutdown_connections 2024-05-15T14:12:10.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.031+0000 7f0c87d1c640 1 -- 172.21.15.33:0/2414870578 wait complete. 2024-05-15T14:12:10.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.032+0000 7f0c87d1c640 1 Processor -- start 2024-05-15T14:12:10.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.032+0000 7f0c87d1c640 1 -- start start 2024-05-15T14:12:10.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.032+0000 7f0c87d1c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c80101310 0x7f0c80199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:10.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.032+0000 7f0c87d1c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c80101cd0 0x7f0c801997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:10.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.032+0000 7f0c87d1c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c80108c50 0x7f0c8019db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:10.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.032+0000 7f0c87d1c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0c80112d20 con 0x7f0c80101cd0 2024-05-15T14:12:10.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.032+0000 7f0c87d1c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0c80112ba0 con 0x7f0c80108c50 2024-05-15T14:12:10.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.032+0000 7f0c87d1c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0c80112ea0 con 0x7f0c80101310 2024-05-15T14:12:10.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.032+0000 7f0c852b7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c80101cd0 0x7f0c801997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:10.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.032+0000 7f0c852b7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c80101cd0 0x7f0c801997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36808/0 (socket says 172.21.15.33:36808) 2024-05-15T14:12:10.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.033+0000 7f0c852b7640 1 -- 172.21.15.33:0/3945652406 learned_addr learned my addr 172.21.15.33:0/3945652406 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:12:10.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.033+0000 7f0c85ab8640 1 --2- 172.21.15.33:0/3945652406 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c80101310 0x7f0c80199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:10.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.033+0000 7f0c862b9640 1 --2- 172.21.15.33:0/3945652406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c80108c50 0x7f0c8019db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:10.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.033+0000 7f0c852b7640 1 -- 172.21.15.33:0/3945652406 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c80101310 msgr2=0x7f0c80199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:10.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.033+0000 7f0c852b7640 1 --2- 172.21.15.33:0/3945652406 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c80101310 0x7f0c80199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:10.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.033+0000 7f0c852b7640 1 -- 172.21.15.33:0/3945652406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c80108c50 msgr2=0x7f0c8019db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:10.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.033+0000 7f0c852b7640 1 --2- 172.21.15.33:0/3945652406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c80108c50 0x7f0c8019db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:10.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.033+0000 7f0c852b7640 1 -- 172.21.15.33:0/3945652406 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c8019e270 con 0x7f0c80101cd0 2024-05-15T14:12:10.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.033+0000 7f0c852b7640 1 --2- 172.21.15.33:0/3945652406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c80101cd0 0x7f0c801997c0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c00e9f0 tx=0x7f0c7c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:10.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.033+0000 7f0c72ffd640 1 -- 172.21.15.33:0/3945652406 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c00cd80 con 0x7f0c80101cd0 2024-05-15T14:12:10.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.033+0000 7f0c72ffd640 1 -- 172.21.15.33:0/3945652406 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c7c0108e0 con 0x7f0c80101cd0 2024-05-15T14:12:10.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.033+0000 7f0c87d1c640 1 -- 172.21.15.33:0/3945652406 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c8019e560 con 0x7f0c80101cd0 2024-05-15T14:12:10.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.033+0000 7f0c72ffd640 1 -- 172.21.15.33:0/3945652406 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c018980 con 0x7f0c80101cd0 2024-05-15T14:12:10.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.033+0000 7f0c87d1c640 1 -- 172.21.15.33:0/3945652406 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c801a5df0 con 0x7f0c80101cd0 2024-05-15T14:12:10.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.034+0000 7f0c87d1c640 1 -- 172.21.15.33:0/3945652406 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c80069d50 con 0x7f0c80101cd0 2024-05-15T14:12:10.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.036+0000 7f0c72ffd640 1 -- 172.21.15.33:0/3945652406 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0c7c010430 con 0x7f0c80101cd0 2024-05-15T14:12:10.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.036+0000 7f0c72ffd640 1 --2- 172.21.15.33:0/3945652406 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0c6c0b2960 0x7f0c6c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:10.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.036+0000 7f0c72ffd640 1 -- 172.21.15.33:0/3945652406 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8586+0+0 (secure 0 0 0) 0x7f0c7c0e2eb0 con 0x7f0c80101cd0 2024-05-15T14:12:10.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.036+0000 7f0c85ab8640 1 --2- 172.21.15.33:0/3945652406 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0c6c0b2960 0x7f0c6c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:10.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.037+0000 7f0c85ab8640 1 --2- 172.21.15.33:0/3945652406 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0c6c0b2960 0x7f0c6c0b4e20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0c74009900 tx=0x7f0c74000bc0 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:10.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.040+0000 7f0c72ffd640 1 -- 172.21.15.33:0/3945652406 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0c7c0aa850 con 0x7f0c80101cd0 2024-05-15T14:12:10.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.231+0000 7f0c87d1c640 1 -- 172.21.15.33:0/3945652406 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0c8010afd0 con 0x7f0c80101cd0 2024-05-15T14:12:10.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.233+0000 7f0c72ffd640 1 -- 172.21.15.33:0/3945652406 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+23128 (secure 0 0 0) 0x7f0c7c0aa150 con 0x7f0c80101cd0 2024-05-15T14:12:10.236 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T14:12:10.236 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":97,"fsid":"bca78ca4-12c1-11ef-bc98-c7b262605968","created":"2024-05-15T13:49:10.327613+0000","modified":"2024-05-15T14:12:04.411979+0000","last_up_change":"2024-05-15T13:56:16.097711+0000","last_in_change":"2024-05-15T13:55:56.722143+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-15T13:52:32.970844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-15T13:57:12.324802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-15T13:57:15.898406+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":97,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b434111-810f-450a-b63d-c97080e9f8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6803","nonce":887409130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6805","nonce":887409130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6809","nonce":887409130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6807","nonce":887409130}]},"public_addr":"172.21.15.33:6803/887409130","cluster_addr":"172.21.15.33:6805/887409130","heartbeat_back_addr":"172.21.15.33:6809/887409130","heartbeat_front_addr":"172.21.15.33:6807/887409130","state":["exists","up"]},{"osd":1,"uuid":"0b578d4e-a841-4b27-bc95-d6c3bae580fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6801","nonce":2794180003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6803","nonce":2794180003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6807","nonce":2794180003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6805","nonce":2794180003}]},"public_addr":"172.21.15.52:6801/2794180003","cluster_addr":"172.21.15.52:6803/2794180003","heartbeat_back_addr":"172.21.15.52:6807/2794180003","heartbeat_front_addr":"172.21.15.52:6805/2794180003","state":["exists","up"]},{"osd":2,"uuid":"0e9c9390-ed33-464b-8f39-bca75941e05e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6801","nonce":315610978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6803","nonce":315610978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6807","nonce":315610978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6805","nonce":315610978}]},"public_addr":"172.21.15.81:6801/315610978","cluster_addr":"172.21.15.81:6803/315610978","heartbeat_back_addr":"172.21.15.81:6807/315610978","heartbeat_front_addr":"172.21.15.81:6805/315610978","state":["exists","up"]},{"osd":3,"uuid":"ecdb24c1-fb15-42a5-9bae-436bbae9512a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6811","nonce":818533501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6813","nonce":818533501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6817","nonce":818533501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6815","nonce":818533501}]},"public_addr":"172.21.15.33:6811/818533501","cluster_addr":"172.21.15.33:6813/818533501","heartbeat_back_addr":"172.21.15.33:6817/818533501","heartbeat_front_addr":"172.21.15.33:6815/818533501","state":["exists","up"]},{"osd":4,"uuid":"706752a0-b17f-4563-a7a9-d5078bf0f3b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6809","nonce":3010133776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6811","nonce":3010133776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6815","nonce":3010133776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6813","nonce":3010133776}]},"public_addr":"172.21.15.52:6809/3010133776","cluster_addr":"172.21.15.52:6811/3010133776","heartbeat_back_addr":"172.21.15.52:6815/3010133776","heartbeat_front_addr":"172.21.15.52:6813/3010133776","state":["exists","up"]},{"osd":5,"uuid":"3d45ab97-f364-4903-974d-8acc961363d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6809","nonce":2132616297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6811","nonce":2132616297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6815","nonce":2132616297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6813","nonce":2132616297}]},"public_addr":"172.21.15.81:6809/2132616297","cluster_addr":"172.21.15.81:6811/2132616297","heartbeat_back_addr":"172.21.15.81:6815/2132616297","heartbeat_front_addr":"172.21.15.81:6813/2132616297","state":["exists","up"]},{"osd":6,"uuid":"3be524c5-dcf9-4742-b896-ea9a44a6b391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6819","nonce":1260691801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6821","nonce":1260691801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6825","nonce":1260691801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6823","nonce":1260691801}]},"public_addr":"172.21.15.33:6819/1260691801","cluster_addr":"172.21.15.33:6821/1260691801","heartbeat_back_addr":"172.21.15.33:6825/1260691801","heartbeat_front_addr":"172.21.15.33:6823/1260691801","state":["exists","up"]},{"osd":7,"uuid":"bda2f920-4f2c-4eed-8127-83c1973676e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6817","nonce":1654595813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6819","nonce":1654595813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6823","nonce":1654595813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6821","nonce":1654595813}]},"public_addr":"172.21.15.52:6817/1654595813","cluster_addr":"172.21.15.52:6819/1654595813","heartbeat_back_addr":"172.21.15.52:6823/1654595813","heartbeat_front_addr":"172.21.15.52:6821/1654595813","state":["exists","up"]},{"osd":8,"uuid":"550585b8-0df4-49ee-95c8-3d084744dbd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6817","nonce":3009661748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6819","nonce":3009661748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6823","nonce":3009661748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6821","nonce":3009661748}]},"public_addr":"172.21.15.81:6817/3009661748","cluster_addr":"172.21.15.81:6819/3009661748","heartbeat_back_addr":"172.21.15.81:6823/3009661748","heartbeat_front_addr":"172.21.15.81:6821/3009661748","state":["exists","up"]},{"osd":9,"uuid":"1d781846-26f3-4b23-b22c-c994a8509a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6827","nonce":2843220858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6829","nonce":2843220858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6833","nonce":2843220858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6831","nonce":2843220858}]},"public_addr":"172.21.15.33:6827/2843220858","cluster_addr":"172.21.15.33:6829/2843220858","heartbeat_back_addr":"172.21.15.33:6833/2843220858","heartbeat_front_addr":"172.21.15.33:6831/2843220858","state":["exists","up"]},{"osd":10,"uuid":"af58c940-fdc4-418b-a638-7dc44438b34c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6825","nonce":2422349831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6827","nonce":2422349831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6831","nonce":2422349831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6829","nonce":2422349831}]},"public_addr":"172.21.15.52:6825/2422349831","cluster_addr":"172.21.15.52:6827/2422349831","heartbeat_back_addr":"172.21.15.52:6831/2422349831","heartbeat_front_addr":"172.21.15.52:6829/2422349831","state":["exists","up"]},{"osd":11,"uuid":"ab84aaea-2086-4842-8bf4-628842fce46a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6825","nonce":326313090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6827","nonce":326313090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6831","nonce":326313090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6829","nonce":326313090}]},"public_addr":"172.21.15.81:6825/326313090","cluster_addr":"172.21.15.81:6827/326313090","heartbeat_back_addr":"172.21.15.81:6831/326313090","heartbeat_front_addr":"172.21.15.81:6829/326313090","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:51:41.493136+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:52:05.784773+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:52:29.769429+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:52:54.891620+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:53:19.412922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:53:43.921841+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:54:09.486732+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:54:34.064370+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:54:58.655304+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:55:24.548693+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:55:49.941191+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:56:14.919558+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1564238110":"2024-05-16T14:00:21.157290+0000","172.21.15.33:6800/1797827152":"2024-05-16T14:00:21.157290+0000","172.21.15.33:0/436121571":"2024-05-16T13:50:04.881656+0000","172.21.15.33:0/2647407310":"2024-05-16T14:00:21.157290+0000","172.21.15.33:0/2326878092":"2024-05-16T14:00:21.157290+0000","172.21.15.33:6801/2680375465":"2024-05-16T13:50:04.881656+0000","172.21.15.33:0/2259371407":"2024-05-16T14:00:21.157290+0000","172.21.15.33:6800/2680375465":"2024-05-16T13:50:04.881656+0000","172.21.15.33:0/1022787179":"2024-05-16T13:49:37.564643+0000","172.21.15.33:0/3720711369":"2024-05-16T13:50:04.881656+0000","172.21.15.33:0/1282466838":"2024-05-16T13:49:37.564643+0000","172.21.15.33:0/3022248969":"2024-05-16T13:49:37.564643+0000","172.21.15.33:6801/3303495364":"2024-05-16T13:49:37.564643+0000","172.21.15.33:6801/1797827152":"2024-05-16T14:00:21.157290+0000","172.21.15.33:6800/3303495364":"2024-05-16T13:49:37.564643+0000","172.21.15.33:0/4282812692":"2024-05-16T13:50:04.881656+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-15T14:12:10.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.235+0000 7f0c87d1c640 1 -- 172.21.15.33:0/3945652406 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0c6c0b2960 msgr2=0x7f0c6c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:10.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.235+0000 7f0c87d1c640 1 --2- 172.21.15.33:0/3945652406 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0c6c0b2960 0x7f0c6c0b4e20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0c74009900 tx=0x7f0c74000bc0 comp rx=0 tx=0).stop 2024-05-15T14:12:10.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.235+0000 7f0c87d1c640 1 -- 172.21.15.33:0/3945652406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c80101cd0 msgr2=0x7f0c801997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:10.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.235+0000 7f0c87d1c640 1 --2- 172.21.15.33:0/3945652406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c80101cd0 0x7f0c801997c0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c00e9f0 tx=0x7f0c7c00eeb0 comp rx=0 tx=0).stop 2024-05-15T14:12:10.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.235+0000 7f0c87d1c640 1 -- 172.21.15.33:0/3945652406 shutdown_connections 2024-05-15T14:12:10.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.235+0000 7f0c87d1c640 1 --2- 172.21.15.33:0/3945652406 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0c6c0b2960 0x7f0c6c0b4e20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:10.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.235+0000 7f0c87d1c640 1 --2- 172.21.15.33:0/3945652406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c80108c50 0x7f0c8019db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:10.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.235+0000 7f0c87d1c640 1 --2- 172.21.15.33:0/3945652406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c80101cd0 0x7f0c801997c0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:10.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.235+0000 7f0c87d1c640 1 --2- 172.21.15.33:0/3945652406 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c80101310 0x7f0c80199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:10.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.235+0000 7f0c87d1c640 1 -- 172.21.15.33:0/3945652406 >> 172.21.15.33:0/3945652406 conn(0x7f0c80076c00 msgr2=0x7f0c80100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:10.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.235+0000 7f0c87d1c640 1 -- 172.21.15.33:0/3945652406 shutdown_connections 2024-05-15T14:12:10.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.235+0000 7f0c87d1c640 1 -- 172.21.15.33:0/3945652406 wait complete. 2024-05-15T14:12:10.249 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-15T13:52:32.970844+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-15T13:57:12.324802+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-15T13:57:15.898406+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 97, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-15T14:12:10.250 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-15T14:12:10.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.356+0000 7fefbf3c1640 1 -- 172.21.15.33:0/164283970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefb810d1d0 msgr2=0x7fefb810f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:10.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.356+0000 7fefbf3c1640 1 --2- 172.21.15.33:0/164283970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefb810d1d0 0x7fefb810f660 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fefb0009900 tx=0x7fefb002a410 comp rx=0 tx=0).stop 2024-05-15T14:12:10.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.357+0000 7fefbf3c1640 1 -- 172.21.15.33:0/164283970 shutdown_connections 2024-05-15T14:12:10.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.357+0000 7fefbf3c1640 1 --2- 172.21.15.33:0/164283970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefb810d1d0 0x7fefb810f660 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:10.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.357+0000 7fefbf3c1640 1 --2- 172.21.15.33:0/164283970 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fefb8104670 0x7fefb810cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:10.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.357+0000 7fefbf3c1640 1 --2- 172.21.15.33:0/164283970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefb8103cc0 0x7fefb81040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:10.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.357+0000 7fefbf3c1640 1 -- 172.21.15.33:0/164283970 >> 172.21.15.33:0/164283970 conn(0x7fefb8078a70 msgr2=0x7fefb8078e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:10.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.357+0000 7fefbf3c1640 1 -- 172.21.15.33:0/164283970 shutdown_connections 2024-05-15T14:12:10.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.357+0000 7fefbf3c1640 1 -- 172.21.15.33:0/164283970 wait complete. 2024-05-15T14:12:10.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.357+0000 7fefbf3c1640 1 Processor -- start 2024-05-15T14:12:10.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.357+0000 7fefbf3c1640 1 -- start start 2024-05-15T14:12:10.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.357+0000 7fefbf3c1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefb8103cc0 0x7fefb8199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:10.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.357+0000 7fefbf3c1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fefb810d1d0 0x7fefb8199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:10.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.357+0000 7fefbf3c1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefb819dcc0 0x7fefb819e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:10.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.357+0000 7fefbf3c1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fefb8112ed0 con 0x7fefb8103cc0 2024-05-15T14:12:10.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.357+0000 7fefbf3c1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fefb8112d50 con 0x7fefb819dcc0 2024-05-15T14:12:10.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.357+0000 7fefbf3c1640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fefb8113050 con 0x7fefb810d1d0 2024-05-15T14:12:10.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.358+0000 7fefbd15d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefb8103cc0 0x7fefb8199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:10.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.358+0000 7fefbd15d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefb8103cc0 0x7fefb8199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36828/0 (socket says 172.21.15.33:36828) 2024-05-15T14:12:10.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.358+0000 7fefbd15d640 1 -- 172.21.15.33:0/1084167140 learned_addr learned my addr 172.21.15.33:0/1084167140 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:12:10.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.358+0000 7fefbc95c640 1 --2- 172.21.15.33:0/1084167140 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fefb810d1d0 0x7fefb8199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:10.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.358+0000 7fefbd15d640 1 -- 172.21.15.33:0/1084167140 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fefb810d1d0 msgr2=0x7fefb8199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:10.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.358+0000 7fefbd95e640 1 --2- 172.21.15.33:0/1084167140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefb819dcc0 0x7fefb819e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:10.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.358+0000 7fefbd15d640 1 --2- 172.21.15.33:0/1084167140 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fefb810d1d0 0x7fefb8199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:10.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.358+0000 7fefbd15d640 1 -- 172.21.15.33:0/1084167140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefb819dcc0 msgr2=0x7fefb819e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:10.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.358+0000 7fefbd15d640 1 --2- 172.21.15.33:0/1084167140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefb819dcc0 0x7fefb819e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:10.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.358+0000 7fefbd15d640 1 -- 172.21.15.33:0/1084167140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb819e7b0 con 0x7fefb8103cc0 2024-05-15T14:12:10.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.358+0000 7fefbc95c640 1 --2- 172.21.15.33:0/1084167140 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fefb810d1d0 0x7fefb8199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:12:10.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.358+0000 7fefbd15d640 1 --2- 172.21.15.33:0/1084167140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefb8103cc0 0x7fefb8199450 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fefb400db80 tx=0x7fefb400b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:10.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.358+0000 7fefae7fc640 1 -- 172.21.15.33:0/1084167140 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb4017070 con 0x7fefb8103cc0 2024-05-15T14:12:10.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.358+0000 7fefae7fc640 1 -- 172.21.15.33:0/1084167140 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefb4003b70 con 0x7fefb8103cc0 2024-05-15T14:12:10.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.358+0000 7fefbf3c1640 1 -- 172.21.15.33:0/1084167140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefb819eaa0 con 0x7fefb8103cc0 2024-05-15T14:12:10.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.358+0000 7fefae7fc640 1 -- 172.21.15.33:0/1084167140 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb40106c0 con 0x7fefb8103cc0 2024-05-15T14:12:10.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.358+0000 7fefbf3c1640 1 -- 172.21.15.33:0/1084167140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefb81a6230 con 0x7fefb8103cc0 2024-05-15T14:12:10.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.359+0000 7fefbf3c1640 1 -- 172.21.15.33:0/1084167140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefb8069d50 con 0x7fefb8103cc0 2024-05-15T14:12:10.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.361+0000 7fefae7fc640 1 -- 172.21.15.33:0/1084167140 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fefb4004020 con 0x7fefb8103cc0 2024-05-15T14:12:10.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.362+0000 7fefae7fc640 1 --2- 172.21.15.33:0/1084167140 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fefa40b2960 0x7fefa40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:10.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.362+0000 7fefbc95c640 1 --2- 172.21.15.33:0/1084167140 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fefa40b2960 0x7fefa40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:10.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.362+0000 7fefae7fc640 1 -- 172.21.15.33:0/1084167140 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8586+0+0 (secure 0 0 0) 0x7fefb40e2ab0 con 0x7fefb8103cc0 2024-05-15T14:12:10.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.363+0000 7fefbc95c640 1 --2- 172.21.15.33:0/1084167140 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fefa40b2960 0x7fefa40b4e20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fefa8006fd0 tx=0x7fefa8008040 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:10.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.366+0000 7fefae7fc640 1 -- 172.21.15.33:0/1084167140 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fefb40aa450 con 0x7fefb8103cc0 2024-05-15T14:12:10.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.559+0000 7fefbf3c1640 1 -- 172.21.15.33:0/1084167140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fefb81081b0 con 0x7fefb8103cc0 2024-05-15T14:12:10.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.559+0000 7fefae7fc640 1 -- 172.21.15.33:0/1084167140 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) ==== 93+0+10 (secure 0 0 0) 0x7fefb40a9d50 con 0x7fefb8103cc0 2024-05-15T14:12:10.562 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-05-15T14:12:10.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.561+0000 7fefbf3c1640 1 -- 172.21.15.33:0/1084167140 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fefa40b2960 msgr2=0x7fefa40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:10.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.561+0000 7fefbf3c1640 1 --2- 172.21.15.33:0/1084167140 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fefa40b2960 0x7fefa40b4e20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fefa8006fd0 tx=0x7fefa8008040 comp rx=0 tx=0).stop 2024-05-15T14:12:10.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.561+0000 7fefbf3c1640 1 -- 172.21.15.33:0/1084167140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefb8103cc0 msgr2=0x7fefb8199450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:10.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.561+0000 7fefbf3c1640 1 --2- 172.21.15.33:0/1084167140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefb8103cc0 0x7fefb8199450 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fefb400db80 tx=0x7fefb400b630 comp rx=0 tx=0).stop 2024-05-15T14:12:10.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.561+0000 7fefbf3c1640 1 -- 172.21.15.33:0/1084167140 shutdown_connections 2024-05-15T14:12:10.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.561+0000 7fefbf3c1640 1 --2- 172.21.15.33:0/1084167140 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fefa40b2960 0x7fefa40b4e20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:10.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.561+0000 7fefbf3c1640 1 --2- 172.21.15.33:0/1084167140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fefb819dcc0 0x7fefb819e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:10.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.561+0000 7fefbf3c1640 1 --2- 172.21.15.33:0/1084167140 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fefb810d1d0 0x7fefb8199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:10.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.561+0000 7fefbf3c1640 1 --2- 172.21.15.33:0/1084167140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefb8103cc0 0x7fefb8199450 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:10.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.561+0000 7fefbf3c1640 1 -- 172.21.15.33:0/1084167140 >> 172.21.15.33:0/1084167140 conn(0x7fefb8078a70 msgr2=0x7fefb80fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:10.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.562+0000 7fefbf3c1640 1 -- 172.21.15.33:0/1084167140 shutdown_connections 2024-05-15T14:12:10.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.562+0000 7fefbf3c1640 1 -- 172.21.15.33:0/1084167140 wait complete. 2024-05-15T14:12:10.575 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:10 smithi033 ceph-mon[31650]: pgmap v371: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 258 op/s 2024-05-15T14:12:10.575 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:10 smithi033 ceph-mon[31650]: from='client.16815 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T14:12:10.575 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:10 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/3945652406' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T14:12:10.576 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-15T14:12:10.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.683+0000 7f178c86d640 1 -- 172.21.15.33:0/2793438386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1788101920 msgr2=0x7f1788101da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:10.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.683+0000 7f178c86d640 1 --2- 172.21.15.33:0/2793438386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1788101920 0x7f1788101da0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f177c009930 tx=0x7f177c02a420 comp rx=0 tx=0).stop 2024-05-15T14:12:10.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.683+0000 7f178c86d640 1 -- 172.21.15.33:0/2793438386 shutdown_connections 2024-05-15T14:12:10.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.683+0000 7f178c86d640 1 --2- 172.21.15.33:0/2793438386 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17881022e0 0x7f178810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:10.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.683+0000 7f178c86d640 1 --2- 172.21.15.33:0/2793438386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1788101920 0x7f1788101da0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:10.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.683+0000 7f178c86d640 1 --2- 172.21.15.33:0/2793438386 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1788107930 0x7f1788107d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:10.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.683+0000 7f178c86d640 1 -- 172.21.15.33:0/2793438386 >> 172.21.15.33:0/2793438386 conn(0x7f1788078b00 msgr2=0x7f1788078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:10.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.683+0000 7f178c86d640 1 -- 172.21.15.33:0/2793438386 shutdown_connections 2024-05-15T14:12:10.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.683+0000 7f178c86d640 1 -- 172.21.15.33:0/2793438386 wait complete. 2024-05-15T14:12:10.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.684+0000 7f178c86d640 1 Processor -- start 2024-05-15T14:12:10.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.684+0000 7f178c86d640 1 -- start start 2024-05-15T14:12:10.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.684+0000 7f178c86d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1788101920 0x7f1788199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:10.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.684+0000 7f178c86d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17881022e0 0x7f1788199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:10.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.684+0000 7f178c86d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1788107930 0x7f178819db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:10.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.684+0000 7f178c86d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1788111bf0 con 0x7f17881022e0 2024-05-15T14:12:10.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.684+0000 7f178c86d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1788111a70 con 0x7f1788107930 2024-05-15T14:12:10.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.684+0000 7f178c86d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f1788111d70 con 0x7f1788101920 2024-05-15T14:12:10.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.684+0000 7f1785d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17881022e0 0x7f1788199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:10.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.684+0000 7f1785d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17881022e0 0x7f1788199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36844/0 (socket says 172.21.15.33:36844) 2024-05-15T14:12:10.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.684+0000 7f1785d9b640 1 -- 172.21.15.33:0/1134281336 learned_addr learned my addr 172.21.15.33:0/1134281336 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:12:10.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.684+0000 7f178659c640 1 --2- 172.21.15.33:0/1134281336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1788101920 0x7f1788199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:10.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.684+0000 7f1786d9d640 1 --2- 172.21.15.33:0/1134281336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1788107930 0x7f178819db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:10.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.684+0000 7f1785d9b640 1 -- 172.21.15.33:0/1134281336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1788101920 msgr2=0x7f1788199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:10.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.684+0000 7f1785d9b640 1 --2- 172.21.15.33:0/1134281336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1788101920 0x7f1788199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:10.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.684+0000 7f1785d9b640 1 -- 172.21.15.33:0/1134281336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1788107930 msgr2=0x7f178819db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:10.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.684+0000 7f1785d9b640 1 --2- 172.21.15.33:0/1134281336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1788107930 0x7f178819db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:10.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.684+0000 7f1785d9b640 1 -- 172.21.15.33:0/1134281336 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f178819e230 con 0x7f17881022e0 2024-05-15T14:12:10.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.684+0000 7f1786d9d640 1 --2- 172.21.15.33:0/1134281336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1788107930 0x7f178819db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:12:10.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.684+0000 7f1785d9b640 1 --2- 172.21.15.33:0/1134281336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17881022e0 0x7f1788199780 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f177c005940 tx=0x7f177c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:10.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.685+0000 7f17777fe640 1 -- 172.21.15.33:0/1134281336 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177c03f030 con 0x7f17881022e0 2024-05-15T14:12:10.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.685+0000 7f17777fe640 1 -- 172.21.15.33:0/1134281336 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f177c003cb0 con 0x7f17881022e0 2024-05-15T14:12:10.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.685+0000 7f17777fe640 1 -- 172.21.15.33:0/1134281336 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177c035070 con 0x7f17881022e0 2024-05-15T14:12:10.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.685+0000 7f178c86d640 1 -- 172.21.15.33:0/1134281336 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f178819e4c0 con 0x7f17881022e0 2024-05-15T14:12:10.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.685+0000 7f178c86d640 1 -- 172.21.15.33:0/1134281336 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17881a5db0 con 0x7f17881022e0 2024-05-15T14:12:10.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.686+0000 7f178c86d640 1 -- 172.21.15.33:0/1134281336 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1788069900 con 0x7f17881022e0 2024-05-15T14:12:10.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.687+0000 7f17777fe640 1 -- 172.21.15.33:0/1134281336 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f177c042030 con 0x7f17881022e0 2024-05-15T14:12:10.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.688+0000 7f17777fe640 1 --2- 172.21.15.33:0/1134281336 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f17700b2960 0x7f17700b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:10.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.688+0000 7f178659c640 1 --2- 172.21.15.33:0/1134281336 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f17700b2960 0x7f17700b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:10.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.689+0000 7f17777fe640 1 -- 172.21.15.33:0/1134281336 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8542+0+0 (secure 0 0 0) 0x7f177c102ec0 con 0x7f17881022e0 2024-05-15T14:12:10.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.689+0000 7f178659c640 1 --2- 172.21.15.33:0/1134281336 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f17700b2960 0x7f17700b4e20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1778009ae0 tx=0x7f1778005f90 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:10.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.693+0000 7f17777fe640 1 -- 172.21.15.33:0/1134281336 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f177c0ca890 con 0x7f17881022e0 2024-05-15T14:12:10.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:10 smithi052 ceph-mon[36190]: pgmap v371: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 258 op/s 2024-05-15T14:12:10.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:10 smithi052 ceph-mon[36190]: from='client.16815 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T14:12:10.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:10 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/3945652406' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T14:12:10.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:10 smithi081 ceph-mon[36434]: pgmap v371: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 258 op/s 2024-05-15T14:12:10.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:10 smithi081 ceph-mon[36434]: from='client.16815 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T14:12:10.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:10 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/3945652406' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T14:12:10.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.884+0000 7f178c86d640 1 -- 172.21.15.33:0/1134281336 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f17881a6010 con 0x7f17881022e0 2024-05-15T14:12:10.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.885+0000 7f17777fe640 1 -- 172.21.15.33:0/1134281336 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) ==== 104+0+11 (secure 0 0 0) 0x7f177c0ca190 con 0x7f17881022e0 2024-05-15T14:12:10.888 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-05-15T14:12:10.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.887+0000 7f178c86d640 1 -- 172.21.15.33:0/1134281336 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f17700b2960 msgr2=0x7f17700b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:10.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.887+0000 7f178c86d640 1 --2- 172.21.15.33:0/1134281336 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f17700b2960 0x7f17700b4e20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1778009ae0 tx=0x7f1778005f90 comp rx=0 tx=0).stop 2024-05-15T14:12:10.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.887+0000 7f178c86d640 1 -- 172.21.15.33:0/1134281336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17881022e0 msgr2=0x7f1788199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:10.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.887+0000 7f178c86d640 1 --2- 172.21.15.33:0/1134281336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17881022e0 0x7f1788199780 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f177c005940 tx=0x7f177c0042f0 comp rx=0 tx=0).stop 2024-05-15T14:12:10.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.887+0000 7f178c86d640 1 -- 172.21.15.33:0/1134281336 shutdown_connections 2024-05-15T14:12:10.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.887+0000 7f178c86d640 1 --2- 172.21.15.33:0/1134281336 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f17700b2960 0x7f17700b4e20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:10.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.887+0000 7f178c86d640 1 --2- 172.21.15.33:0/1134281336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1788107930 0x7f178819db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:10.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.887+0000 7f178c86d640 1 --2- 172.21.15.33:0/1134281336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17881022e0 0x7f1788199780 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:10.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.887+0000 7f178c86d640 1 --2- 172.21.15.33:0/1134281336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1788101920 0x7f1788199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:10.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.887+0000 7f178c86d640 1 -- 172.21.15.33:0/1134281336 >> 172.21.15.33:0/1134281336 conn(0x7f1788078b00 msgr2=0x7f1788100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:10.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.887+0000 7f178c86d640 1 -- 172.21.15.33:0/1134281336 shutdown_connections 2024-05-15T14:12:10.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:10.887+0000 7f178c86d640 1 -- 172.21.15.33:0/1134281336 wait complete. 2024-05-15T14:12:10.901 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-15T14:12:11.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.007+0000 7f2ecc447640 1 -- 172.21.15.33:0/2913877602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ec410d020 msgr2=0x7f2ec410f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:11.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.007+0000 7f2ecc447640 1 --2- 172.21.15.33:0/2913877602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ec410d020 0x7f2ec410f4a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc009900 tx=0x7f2ebc02a480 comp rx=0 tx=0).stop 2024-05-15T14:12:11.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.007+0000 7f2ecc447640 1 -- 172.21.15.33:0/2913877602 shutdown_connections 2024-05-15T14:12:11.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.007+0000 7f2ecc447640 1 --2- 172.21.15.33:0/2913877602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ec410d020 0x7f2ec410f4a0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:11.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.007+0000 7f2ecc447640 1 --2- 172.21.15.33:0/2913877602 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ec4104600 0x7f2ec410cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:11.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.007+0000 7f2ecc447640 1 --2- 172.21.15.33:0/2913877602 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ec4103ce0 0x7f2ec41040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:11.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.007+0000 7f2ecc447640 1 -- 172.21.15.33:0/2913877602 >> 172.21.15.33:0/2913877602 conn(0x7f2ec4078b00 msgr2=0x7f2ec4078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:11.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.007+0000 7f2ecc447640 1 -- 172.21.15.33:0/2913877602 shutdown_connections 2024-05-15T14:12:11.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.007+0000 7f2ecc447640 1 -- 172.21.15.33:0/2913877602 wait complete. 2024-05-15T14:12:11.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.008+0000 7f2ecc447640 1 Processor -- start 2024-05-15T14:12:11.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.008+0000 7f2ecc447640 1 -- start start 2024-05-15T14:12:11.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.008+0000 7f2ecc447640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ec4103ce0 0x7f2ec4199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:11.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.008+0000 7f2ecc447640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ec4104600 0x7f2ec41997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:11.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.008+0000 7f2ecc447640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ec410d020 0x7f2ec419db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:11.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.008+0000 7f2ecc447640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec4112cb0 con 0x7f2ec410d020 2024-05-15T14:12:11.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.008+0000 7f2ecc447640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec4112b30 con 0x7f2ec4103ce0 2024-05-15T14:12:11.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.008+0000 7f2ecc447640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec4112e30 con 0x7f2ec4104600 2024-05-15T14:12:11.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.008+0000 7f2eca9e4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ec410d020 0x7f2ec419db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:11.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.008+0000 7f2eca9e4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ec410d020 0x7f2ec419db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36854/0 (socket says 172.21.15.33:36854) 2024-05-15T14:12:11.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.008+0000 7f2eca9e4640 1 -- 172.21.15.33:0/2114073653 learned_addr learned my addr 172.21.15.33:0/2114073653 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:12:11.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.008+0000 7f2ec99e2640 1 --2- 172.21.15.33:0/2114073653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ec4104600 0x7f2ec41997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:11.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.008+0000 7f2eca1e3640 1 --2- 172.21.15.33:0/2114073653 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ec4103ce0 0x7f2ec4199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:11.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.008+0000 7f2eca9e4640 1 -- 172.21.15.33:0/2114073653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ec4104600 msgr2=0x7f2ec41997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:11.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.008+0000 7f2eca9e4640 1 --2- 172.21.15.33:0/2114073653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ec4104600 0x7f2ec41997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:11.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.008+0000 7f2eca9e4640 1 -- 172.21.15.33:0/2114073653 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ec4103ce0 msgr2=0x7f2ec4199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:11.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.008+0000 7f2eca9e4640 1 --2- 172.21.15.33:0/2114073653 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ec4103ce0 0x7f2ec4199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:11.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.008+0000 7f2eca9e4640 1 -- 172.21.15.33:0/2114073653 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec419e260 con 0x7f2ec410d020 2024-05-15T14:12:11.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.009+0000 7f2eca9e4640 1 --2- 172.21.15.33:0/2114073653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ec410d020 0x7f2ec419db30 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc004770 tx=0x7f2ebc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:11.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.009+0000 7f2ec37fe640 1 -- 172.21.15.33:0/2114073653 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc03f030 con 0x7f2ec410d020 2024-05-15T14:12:11.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.009+0000 7f2ec37fe640 1 -- 172.21.15.33:0/2114073653 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ebc004440 con 0x7f2ec410d020 2024-05-15T14:12:11.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.009+0000 7f2ecc447640 1 -- 172.21.15.33:0/2114073653 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec419e4f0 con 0x7f2ec410d020 2024-05-15T14:12:11.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.009+0000 7f2ec37fe640 1 -- 172.21.15.33:0/2114073653 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc035070 con 0x7f2ec410d020 2024-05-15T14:12:11.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.009+0000 7f2ecc447640 1 -- 172.21.15.33:0/2114073653 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec41a5de0 con 0x7f2ec410d020 2024-05-15T14:12:11.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.010+0000 7f2ecc447640 1 -- 172.21.15.33:0/2114073653 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ec4069900 con 0x7f2ec410d020 2024-05-15T14:12:11.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.011+0000 7f2ec37fe640 1 -- 172.21.15.33:0/2114073653 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2ebc042030 con 0x7f2ec410d020 2024-05-15T14:12:11.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.012+0000 7f2ec37fe640 1 --2- 172.21.15.33:0/2114073653 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f2eac0b2960 0x7f2eac0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:11.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.012+0000 7f2ec37fe640 1 -- 172.21.15.33:0/2114073653 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8542+0+0 (secure 0 0 0) 0x7f2ebc102a50 con 0x7f2ec410d020 2024-05-15T14:12:11.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.012+0000 7f2eca1e3640 1 --2- 172.21.15.33:0/2114073653 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f2eac0b2960 0x7f2eac0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:11.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.012+0000 7f2eca1e3640 1 --2- 172.21.15.33:0/2114073653 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f2eac0b2960 0x7f2eac0b4e20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4005fd0 tx=0x7f2eb4009500 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:11.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.016+0000 7f2ec37fe640 1 -- 172.21.15.33:0/2114073653 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2ebc0ca3c0 con 0x7f2ec410d020 2024-05-15T14:12:11.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.209+0000 7f2ecc447640 1 -- 172.21.15.33:0/2114073653 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2ec41081a0 con 0x7f2ec410d020 2024-05-15T14:12:11.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.210+0000 7f2ec37fe640 1 -- 172.21.15.33:0/2114073653 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) ==== 100+0+11 (secure 0 0 0) 0x7f2ebc0c9cc0 con 0x7f2ec410d020 2024-05-15T14:12:11.213 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-05-15T14:12:11.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.212+0000 7f2ecc447640 1 -- 172.21.15.33:0/2114073653 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f2eac0b2960 msgr2=0x7f2eac0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:11.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.212+0000 7f2ecc447640 1 --2- 172.21.15.33:0/2114073653 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f2eac0b2960 0x7f2eac0b4e20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4005fd0 tx=0x7f2eb4009500 comp rx=0 tx=0).stop 2024-05-15T14:12:11.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.212+0000 7f2ecc447640 1 -- 172.21.15.33:0/2114073653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ec410d020 msgr2=0x7f2ec419db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:11.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.212+0000 7f2ecc447640 1 --2- 172.21.15.33:0/2114073653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ec410d020 0x7f2ec419db30 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc004770 tx=0x7f2ebc004040 comp rx=0 tx=0).stop 2024-05-15T14:12:11.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.212+0000 7f2ecc447640 1 -- 172.21.15.33:0/2114073653 shutdown_connections 2024-05-15T14:12:11.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.212+0000 7f2ecc447640 1 --2- 172.21.15.33:0/2114073653 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f2eac0b2960 0x7f2eac0b4e20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:11.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.212+0000 7f2ecc447640 1 --2- 172.21.15.33:0/2114073653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ec410d020 0x7f2ec419db30 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:11.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.212+0000 7f2ecc447640 1 --2- 172.21.15.33:0/2114073653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2ec4104600 0x7f2ec41997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:11.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.212+0000 7f2ecc447640 1 --2- 172.21.15.33:0/2114073653 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ec4103ce0 0x7f2ec4199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:11.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.212+0000 7f2ecc447640 1 -- 172.21.15.33:0/2114073653 >> 172.21.15.33:0/2114073653 conn(0x7f2ec4078b00 msgr2=0x7f2ec40ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:11.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.212+0000 7f2ecc447640 1 -- 172.21.15.33:0/2114073653 shutdown_connections 2024-05-15T14:12:11.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.212+0000 7f2ecc447640 1 -- 172.21.15.33:0/2114073653 wait complete. 2024-05-15T14:12:11.226 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-15T14:12:11.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.329+0000 7f00ee304640 1 -- 172.21.15.33:0/2872042814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00e8101ba0 msgr2=0x7f00e8102020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:11.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.329+0000 7f00ee304640 1 --2- 172.21.15.33:0/2872042814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00e8101ba0 0x7f00e8102020 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f00dc009930 tx=0x7f00dc02a420 comp rx=0 tx=0).stop 2024-05-15T14:12:11.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.329+0000 7f00ee304640 1 -- 172.21.15.33:0/2872042814 shutdown_connections 2024-05-15T14:12:11.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.329+0000 7f00ee304640 1 --2- 172.21.15.33:0/2872042814 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f00e8102560 0x7f00e810e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:11.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.329+0000 7f00ee304640 1 --2- 172.21.15.33:0/2872042814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00e8101ba0 0x7f00e8102020 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:11.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.329+0000 7f00ee304640 1 --2- 172.21.15.33:0/2872042814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00e8106950 0x7f00e8106d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:11.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.329+0000 7f00ee304640 1 -- 172.21.15.33:0/2872042814 >> 172.21.15.33:0/2872042814 conn(0x7f00e80787a0 msgr2=0x7f00e8078bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:11.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.329+0000 7f00ee304640 1 -- 172.21.15.33:0/2872042814 shutdown_connections 2024-05-15T14:12:11.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.329+0000 7f00ee304640 1 -- 172.21.15.33:0/2872042814 wait complete. 2024-05-15T14:12:11.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.330+0000 7f00ee304640 1 Processor -- start 2024-05-15T14:12:11.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.330+0000 7f00ee304640 1 -- start start 2024-05-15T14:12:11.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.330+0000 7f00ee304640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00e8101ba0 0x7f00e81981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:11.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.330+0000 7f00ee304640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00e8102560 0x7f00e8198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:11.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.330+0000 7f00ee304640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f00e8106950 0x7f00e819cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:11.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.330+0000 7f00ee304640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f00e8111cb0 con 0x7f00e8101ba0 2024-05-15T14:12:11.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.330+0000 7f00ee304640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f00e8111b30 con 0x7f00e8102560 2024-05-15T14:12:11.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.330+0000 7f00ee304640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f00e8111e30 con 0x7f00e8106950 2024-05-15T14:12:11.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.330+0000 7f00e7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00e8101ba0 0x7f00e81981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:11.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.330+0000 7f00e7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00e8101ba0 0x7f00e81981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36866/0 (socket says 172.21.15.33:36866) 2024-05-15T14:12:11.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.330+0000 7f00e7fff640 1 -- 172.21.15.33:0/3261472208 learned_addr learned my addr 172.21.15.33:0/3261472208 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:12:11.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.331+0000 7f00e77fe640 1 --2- 172.21.15.33:0/3261472208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00e8102560 0x7f00e8198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:11.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.331+0000 7f00ec8a1640 1 --2- 172.21.15.33:0/3261472208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f00e8106950 0x7f00e819cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:11.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.331+0000 7f00e7fff640 1 -- 172.21.15.33:0/3261472208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f00e8106950 msgr2=0x7f00e819cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:11.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.331+0000 7f00e7fff640 1 --2- 172.21.15.33:0/3261472208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f00e8106950 0x7f00e819cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:11.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.331+0000 7f00e7fff640 1 -- 172.21.15.33:0/3261472208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00e8102560 msgr2=0x7f00e8198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:11.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.331+0000 7f00e7fff640 1 --2- 172.21.15.33:0/3261472208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00e8102560 0x7f00e8198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:11.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.331+0000 7f00e7fff640 1 -- 172.21.15.33:0/3261472208 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e819d1e0 con 0x7f00e8101ba0 2024-05-15T14:12:11.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.331+0000 7f00e77fe640 1 --2- 172.21.15.33:0/3261472208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00e8102560 0x7f00e8198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:12:11.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.331+0000 7f00ec8a1640 1 --2- 172.21.15.33:0/3261472208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f00e8106950 0x7f00e819cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-15T14:12:11.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.331+0000 7f00e7fff640 1 --2- 172.21.15.33:0/3261472208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00e8101ba0 0x7f00e81981f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f00d800d950 tx=0x7f00d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:11.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.331+0000 7f00e57fa640 1 -- 172.21.15.33:0/3261472208 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d8017070 con 0x7f00e8101ba0 2024-05-15T14:12:11.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.331+0000 7f00e57fa640 1 -- 172.21.15.33:0/3261472208 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00d8009c30 con 0x7f00e8101ba0 2024-05-15T14:12:11.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.331+0000 7f00ee304640 1 -- 172.21.15.33:0/3261472208 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e819d4d0 con 0x7f00e8101ba0 2024-05-15T14:12:11.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.331+0000 7f00e57fa640 1 -- 172.21.15.33:0/3261472208 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d800f620 con 0x7f00e8101ba0 2024-05-15T14:12:11.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.331+0000 7f00ee304640 1 -- 172.21.15.33:0/3261472208 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e81a4d60 con 0x7f00e8101ba0 2024-05-15T14:12:11.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.332+0000 7f00ee304640 1 -- 172.21.15.33:0/3261472208 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00e8069d50 con 0x7f00e8101ba0 2024-05-15T14:12:11.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.334+0000 7f00e57fa640 1 -- 172.21.15.33:0/3261472208 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f00d8010460 con 0x7f00e8101ba0 2024-05-15T14:12:11.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.335+0000 7f00e57fa640 1 --2- 172.21.15.33:0/3261472208 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f00d00b2960 0x7f00d00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:11.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.335+0000 7f00e77fe640 1 --2- 172.21.15.33:0/3261472208 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f00d00b2960 0x7f00d00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:11.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.335+0000 7f00e57fa640 1 -- 172.21.15.33:0/3261472208 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8542+0+0 (secure 0 0 0) 0x7f00d80e2310 con 0x7f00e8101ba0 2024-05-15T14:12:11.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.335+0000 7f00e77fe640 1 --2- 172.21.15.33:0/3261472208 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f00d00b2960 0x7f00d00b4e20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f00dc000c00 tx=0x7f00dc031400 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:11.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.339+0000 7f00e57fa640 1 -- 172.21.15.33:0/3261472208 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f00d80a9ce0 con 0x7f00e8101ba0 2024-05-15T14:12:11.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.582+0000 7f00ee304640 1 -- 172.21.15.33:0/3261472208 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f00e8109e60 con 0x7f00e8101ba0 2024-05-15T14:12:11.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.592+0000 7f00e57fa640 1 -- 172.21.15.33:0/3261472208 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+816172 (secure 0 0 0) 0x7f00d8009de0 con 0x7f00e8101ba0 2024-05-15T14:12:11.605 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T14:12:11.605 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T14:12:11.605 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 26, 2024-05-15T14:12:11.605 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.605 INFO:teuthology.orchestra.run.smithi033.stdout: "active_gid": 15144, 2024-05-15T14:12:11.605 INFO:teuthology.orchestra.run.smithi033.stdout: "active_name": "x", 2024-05-15T14:12:11.605 INFO:teuthology.orchestra.run.smithi033.stdout: "active_addrs": { 2024-05-15T14:12:11.605 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-15T14:12:11.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:11.606 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-15T14:12:11.606 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6800", 2024-05-15T14:12:11.606 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 1796050977 2024-05-15T14:12:11.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:11.606 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-15T14:12:11.606 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6801", 2024-05-15T14:12:11.607 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 1796050977 2024-05-15T14:12:11.607 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:11.607 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-15T14:12:11.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.607 INFO:teuthology.orchestra.run.smithi033.stdout: "active_addr": "172.21.15.33:6801/1796050977", 2024-05-15T14:12:11.607 INFO:teuthology.orchestra.run.smithi033.stdout: "active_change": "2024-05-15T14:00:21.157502+0000", 2024-05-15T14:12:11.607 INFO:teuthology.orchestra.run.smithi033.stdout: "active_mgr_features": 4540701547738038271, 2024-05-15T14:12:11.607 INFO:teuthology.orchestra.run.smithi033.stdout: "available": true, 2024-05-15T14:12:11.607 INFO:teuthology.orchestra.run.smithi033.stdout: "standbys": [ 2024-05-15T14:12:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: "gid": 24821, 2024-05-15T14:12:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "y", 2024-05-15T14:12:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_features": 4540701547738038271, 2024-05-15T14:12:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: "available_modules": [ 2024-05-15T14:12:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "alerts", 2024-05-15T14:12:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-15T14:12:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-15T14:12:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-15T14:12:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.610 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.610 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How frequently to reexamine health status", 2024-05-15T14:12:11.610 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.610 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.610 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.610 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:11.610 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:11.612 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:11.612 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:11.612 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:11.612 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:11.612 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:11.612 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.612 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.612 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.613 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.613 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:11.613 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:11.613 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.613 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.613 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.613 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:11.613 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.614 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.614 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.614 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.614 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.614 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.614 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.614 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_destination": { 2024-05-15T14:12:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_destination", 2024-05-15T14:12:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email address to send alerts to", 2024-05-15T14:12:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_from_name": { 2024-05-15T14:12:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_from_name", 2024-05-15T14:12:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.621 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.621 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.621 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "Ceph", 2024-05-15T14:12:11.621 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.621 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.621 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.621 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email From: name", 2024-05-15T14:12:11.621 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_host": { 2024-05-15T14:12:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_host", 2024-05-15T14:12:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP server", 2024-05-15T14:12:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_password": { 2024-05-15T14:12:11.624 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_password", 2024-05-15T14:12:11.624 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.624 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.624 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.624 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:11.624 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.624 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.624 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Password to authenticate with", 2024-05-15T14:12:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_port": { 2024-05-15T14:12:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_port", 2024-05-15T14:12:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "465", 2024-05-15T14:12:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP port", 2024-05-15T14:12:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_sender": { 2024-05-15T14:12:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_sender", 2024-05-15T14:12:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP envelope sender", 2024-05-15T14:12:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_ssl": { 2024-05-15T14:12:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_ssl", 2024-05-15T14:12:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-15T14:12:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_user": { 2024-05-15T14:12:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_user", 2024-05-15T14:12:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "User to authenticate as", 2024-05-15T14:12:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "balancer", 2024-05-15T14:12:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "active": { 2024-05-15T14:12:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "active", 2024-05-15T14:12:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically balance PGs across cluster", 2024-05-15T14:12:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_time": { 2024-05-15T14:12:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_time", 2024-05-15T14:12:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0000", 2024-05-15T14:12:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "beginning time of day to automatically balance", 2024-05-15T14:12:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-15T14:12:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_weekday": { 2024-05-15T14:12:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_weekday", 2024-05-15T14:12:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-15T14:12:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-05-15T14:12:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-05-15T14:12:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-15T14:12:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-15T14:12:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_max_iterations": { 2024-05-15T14:12:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_max_iterations", 2024-05-15T14:12:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-15T14:12:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "25", 2024-05-15T14:12:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-15T14:12:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "250", 2024-05-15T14:12:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-15T14:12:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_metrics": { 2024-05-15T14:12:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_metrics", 2024-05-15T14:12:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "pgs,objects,bytes", 2024-05-15T14:12:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-15T14:12:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-15T14:12:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_step": { 2024-05-15T14:12:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_step", 2024-05-15T14:12:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-15T14:12:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.5", 2024-05-15T14:12:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0.001", 2024-05-15T14:12:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "0.999", 2024-05-15T14:12:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "aggressiveness of optimization", 2024-05-15T14:12:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-15T14:12:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "end_time": { 2024-05-15T14:12:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_time", 2024-05-15T14:12:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2359", 2024-05-15T14:12:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ending time of day to automatically balance", 2024-05-15T14:12:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-15T14:12:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "end_weekday": { 2024-05-15T14:12:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_weekday", 2024-05-15T14:12:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-15T14:12:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-05-15T14:12:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-05-15T14:12:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-15T14:12:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-15T14:12:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "min_score": { 2024-05-15T14:12:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_score", 2024-05-15T14:12:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-15T14:12:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-15T14:12:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-05-15T14:12:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-05-15T14:12:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "upmap", 2024-05-15T14:12:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-compat", 2024-05-15T14:12:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "none", 2024-05-15T14:12:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "read", 2024-05-15T14:12:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap", 2024-05-15T14:12:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap-read" 2024-05-15T14:12:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Balancer mode", 2024-05-15T14:12:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_ids": { 2024-05-15T14:12:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_ids", 2024-05-15T14:12:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-15T14:12:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-15T14:12:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-15T14:12:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-15T14:12:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-15T14:12:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_deviation": { 2024-05-15T14:12:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_deviation", 2024-05-15T14:12:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-15T14:12:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-15T14:12:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-15T14:12:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-15T14:12:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_optimizations": { 2024-05-15T14:12:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_optimizations", 2024-05-15T14:12:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-15T14:12:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-15T14:12:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-15T14:12:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm", 2024-05-15T14:12:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_down_multiplier": { 2024-05-15T14:12:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_down_multiplier", 2024-05-15T14:12:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-15T14:12:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-05-15T14:12:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-15T14:12:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_refresh_rate": { 2024-05-15T14:12:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_refresh_rate", 2024-05-15T14:12:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "20", 2024-05-15T14:12:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-15T14:12:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_starting_port": { 2024-05-15T14:12:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_starting_port", 2024-05-15T14:12:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4721", 2024-05-15T14:12:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-15T14:12:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_ptrace": { 2024-05-15T14:12:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_ptrace", 2024-05-15T14:12:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-15T14:12:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-15T14:12:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_interval": { 2024-05-15T14:12:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_interval", 2024-05-15T14:12:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-15T14:12:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-15T14:12:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_memory_target_ratio": { 2024-05-15T14:12:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_memory_target_ratio", 2024-05-15T14:12:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-15T14:12:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.7", 2024-05-15T14:12:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-15T14:12:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm_log_destination": { 2024-05-15T14:12:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm_log_destination", 2024-05-15T14:12:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "file", 2024-05-15T14:12:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "file,syslog", 2024-05-15T14:12:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "syslog" 2024-05-15T14:12:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-15T14:12:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "cgroups_split": { 2024-05-15T14:12:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cgroups_split", 2024-05-15T14:12:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-15T14:12:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "config_checks_enabled": { 2024-05-15T14:12:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_checks_enabled", 2024-05-15T14:12:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-15T14:12:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: "config_dashboard": { 2024-05-15T14:12:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_dashboard", 2024-05-15T14:12:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-15T14:12:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_alertmanager": { 2024-05-15T14:12:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_alertmanager", 2024-05-15T14:12:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-15T14:12:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-15T14:12:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_base": { 2024-05-15T14:12:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_base", 2024-05-15T14:12:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-15T14:12:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Container image name, without the tag", 2024-05-15T14:12:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_elasticsearch": { 2024-05-15T14:12:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_elasticsearch", 2024-05-15T14:12:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-15T14:12:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "elasticsearch container image", 2024-05-15T14:12:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_grafana": { 2024-05-15T14:12:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_grafana", 2024-05-15T14:12:11.695 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.695 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.695 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.695 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-15T14:12:11.695 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.695 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.695 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-15T14:12:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_haproxy": { 2024-05-15T14:12:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_haproxy", 2024-05-15T14:12:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-15T14:12:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "HAproxy container image", 2024-05-15T14:12:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_agent": { 2024-05-15T14:12:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_agent", 2024-05-15T14:12:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-15T14:12:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger agent container image", 2024-05-15T14:12:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_collector": { 2024-05-15T14:12:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_collector", 2024-05-15T14:12:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-15T14:12:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger collector container image", 2024-05-15T14:12:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_query": { 2024-05-15T14:12:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_query", 2024-05-15T14:12:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-15T14:12:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger query container image", 2024-05-15T14:12:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_keepalived": { 2024-05-15T14:12:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_keepalived", 2024-05-15T14:12:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-15T14:12:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Keepalived container image", 2024-05-15T14:12:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_loki": { 2024-05-15T14:12:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_loki", 2024-05-15T14:12:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-15T14:12:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Loki container image", 2024-05-15T14:12:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_node_exporter": { 2024-05-15T14:12:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_node_exporter", 2024-05-15T14:12:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-15T14:12:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-15T14:12:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_nvmeof": { 2024-05-15T14:12:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_nvmeof", 2024-05-15T14:12:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-15T14:12:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Nvme-of container image", 2024-05-15T14:12:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_prometheus": { 2024-05-15T14:12:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_prometheus", 2024-05-15T14:12:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-15T14:12:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-15T14:12:11.713 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.713 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.713 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.713 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_promtail": { 2024-05-15T14:12:11.713 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_promtail", 2024-05-15T14:12:11.713 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-15T14:12:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Promtail container image", 2024-05-15T14:12:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_samba": { 2024-05-15T14:12:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_samba", 2024-05-15T14:12:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-15T14:12:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Samba/SMB container image", 2024-05-15T14:12:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_snmp_gateway": { 2024-05-15T14:12:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_snmp_gateway", 2024-05-15T14:12:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-15T14:12:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SNMP Gateway container image", 2024-05-15T14:12:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "container_init": { 2024-05-15T14:12:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_init", 2024-05-15T14:12:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Run podman/docker with `--init`", 2024-05-15T14:12:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "daemon_cache_timeout": { 2024-05-15T14:12:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "daemon_cache_timeout", 2024-05-15T14:12:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-15T14:12:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-15T14:12:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "default_cephadm_command_timeout": { 2024-05-15T14:12:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_cephadm_command_timeout", 2024-05-15T14:12:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "900", 2024-05-15T14:12:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-15T14:12:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "default_registry": { 2024-05-15T14:12:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_registry", 2024-05-15T14:12:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io", 2024-05-15T14:12:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-15T14:12:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "device_cache_timeout": { 2024-05-15T14:12:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_cache_timeout", 2024-05-15T14:12:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1800", 2024-05-15T14:12:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache device inventory", 2024-05-15T14:12:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: "device_enhanced_scan": { 2024-05-15T14:12:11.730 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_enhanced_scan", 2024-05-15T14:12:11.730 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.730 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.730 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.730 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:11.730 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.730 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.731 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.731 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-15T14:12:11.731 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.731 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.731 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.732 INFO:teuthology.orchestra.run.smithi033.stdout: "facts_cache_timeout": { 2024-05-15T14:12:11.732 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "facts_cache_timeout", 2024-05-15T14:12:11.732 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:11.732 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.732 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.732 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-15T14:12:11.733 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.733 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.733 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.733 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache host facts data", 2024-05-15T14:12:11.733 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.733 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.733 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.734 INFO:teuthology.orchestra.run.smithi033.stdout: "grafana_dashboards_path": { 2024-05-15T14:12:11.734 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "grafana_dashboards_path", 2024-05-15T14:12:11.734 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.734 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.734 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-15T14:12:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-15T14:12:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.736 INFO:teuthology.orchestra.run.smithi033.stdout: "host_check_interval": { 2024-05-15T14:12:11.736 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "host_check_interval", 2024-05-15T14:12:11.736 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:11.736 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.736 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.737 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-15T14:12:11.737 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.737 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.737 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.737 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to perform a host check", 2024-05-15T14:12:11.737 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.737 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.738 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.738 INFO:teuthology.orchestra.run.smithi033.stdout: "hw_monitoring": { 2024-05-15T14:12:11.738 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hw_monitoring", 2024-05-15T14:12:11.738 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.738 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.738 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.739 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:11.739 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.739 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.739 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.739 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-15T14:12:11.739 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.739 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.739 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.740 INFO:teuthology.orchestra.run.smithi033.stdout: "inventory_list_all": { 2024-05-15T14:12:11.740 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "inventory_list_all", 2024-05-15T14:12:11.740 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.740 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.740 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.740 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:11.740 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.741 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.741 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.741 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-15T14:12:11.741 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.741 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.741 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.742 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:11.742 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:11.742 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.742 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.742 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.742 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:11.743 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.743 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.743 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:11.743 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:11.743 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:11.743 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:11.743 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:11.745 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:11.745 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:11.745 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:11.745 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.745 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.746 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.746 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.746 INFO:teuthology.orchestra.run.smithi033.stdout: "log_refresh_metadata": { 2024-05-15T14:12:11.746 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_refresh_metadata", 2024-05-15T14:12:11.746 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.746 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.747 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.747 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:11.747 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.747 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.747 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.747 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-15T14:12:11.747 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.748 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.748 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.748 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:11.748 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:11.748 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.748 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.748 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.749 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:11.749 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.749 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.749 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.749 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-15T14:12:11.749 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.749 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.750 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.750 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.750 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:11.750 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:11.750 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.750 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.750 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:11.752 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:11.752 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:11.752 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:11.752 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.752 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.752 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.752 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.753 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:11.753 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:11.753 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.753 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.753 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.753 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:11.753 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.754 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.754 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.754 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.754 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.754 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.754 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.754 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.755 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-15T14:12:11.755 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-15T14:12:11.755 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.755 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.755 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.755 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:11.755 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.755 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.756 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.756 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-15T14:12:11.756 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.756 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.756 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.756 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-15T14:12:11.756 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-15T14:12:11.757 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.757 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.757 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.757 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "*", 2024-05-15T14:12:11.757 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.757 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.757 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.758 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-15T14:12:11.758 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.758 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.758 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.758 INFO:teuthology.orchestra.run.smithi033.stdout: "max_count_per_host": { 2024-05-15T14:12:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_count_per_host", 2024-05-15T14:12:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-15T14:12:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.760 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.760 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of daemons per service per host", 2024-05-15T14:12:11.760 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.760 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.760 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.760 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.760 INFO:teuthology.orchestra.run.smithi033.stdout: "max_osd_draining_count": { 2024-05-15T14:12:11.761 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_osd_draining_count", 2024-05-15T14:12:11.761 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:11.761 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.761 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.761 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-15T14:12:11.761 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.761 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.761 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.762 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-15T14:12:11.762 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.762 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.762 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.762 INFO:teuthology.orchestra.run.smithi033.stdout: "migration_current": { 2024-05-15T14:12:11.762 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "migration_current", 2024-05-15T14:12:11.763 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:11.763 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.763 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.763 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:11.763 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.763 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.763 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.764 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "internal - do not modify", 2024-05-15T14:12:11.764 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.764 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.764 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.764 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.764 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-05-15T14:12:11.764 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-05-15T14:12:11.764 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.765 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.765 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.765 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "root", 2024-05-15T14:12:11.765 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.765 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.765 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:11.765 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm-package", 2024-05-15T14:12:11.765 INFO:teuthology.orchestra.run.smithi033.stdout: "root" 2024-05-15T14:12:11.766 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:11.766 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "mode for remote execution of cephadm", 2024-05-15T14:12:11.766 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.766 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.766 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.766 INFO:teuthology.orchestra.run.smithi033.stdout: "oob_default_addr": { 2024-05-15T14:12:11.767 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "oob_default_addr", 2024-05-15T14:12:11.767 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.767 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.767 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.767 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "169.254.1.1", 2024-05-15T14:12:11.767 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.767 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.768 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.768 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-15T14:12:11.768 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.768 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.768 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.768 INFO:teuthology.orchestra.run.smithi033.stdout: "prometheus_alerts_path": { 2024-05-15T14:12:11.768 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus_alerts_path", 2024-05-15T14:12:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-15T14:12:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.770 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-15T14:12:11.770 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.770 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.770 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.770 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.770 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_insecure": { 2024-05-15T14:12:11.770 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_insecure", 2024-05-15T14:12:11.770 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.771 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.771 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.771 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:11.771 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.771 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.771 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.771 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-15T14:12:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_password": { 2024-05-15T14:12:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_password", 2024-05-15T14:12:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-15T14:12:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.774 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.774 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_url": { 2024-05-15T14:12:11.774 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_url", 2024-05-15T14:12:11.774 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.774 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.774 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.775 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:11.775 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.775 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.775 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.775 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-15T14:12:11.775 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.775 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.776 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.776 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_username": { 2024-05-15T14:12:11.776 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_username", 2024-05-15T14:12:11.776 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.776 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.776 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.777 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:11.777 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.777 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.777 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.777 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-15T14:12:11.777 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.777 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.777 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.778 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.778 INFO:teuthology.orchestra.run.smithi033.stdout: "secure_monitoring_stack": { 2024-05-15T14:12:11.778 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "secure_monitoring_stack", 2024-05-15T14:12:11.778 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.778 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.778 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.778 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:11.778 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-15T14:12:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: "service_discovery_port": { 2024-05-15T14:12:11.780 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "service_discovery_port", 2024-05-15T14:12:11.780 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:11.780 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.780 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.780 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8765", 2024-05-15T14:12:11.780 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.780 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.781 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.781 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "cephadm service discovery port", 2024-05-15T14:12:11.781 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.781 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.781 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.781 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:11.781 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:11.782 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:11.782 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:11.782 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.782 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:11.782 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.782 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.782 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.783 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.783 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.783 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.783 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.783 INFO:teuthology.orchestra.run.smithi033.stdout: "ssh_config_file": { 2024-05-15T14:12:11.783 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssh_config_file", 2024-05-15T14:12:11.783 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-15T14:12:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.785 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.785 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.785 INFO:teuthology.orchestra.run.smithi033.stdout: "use_agent": { 2024-05-15T14:12:11.785 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_agent", 2024-05-15T14:12:11.785 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.786 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.786 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.786 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:11.786 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.786 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.786 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.786 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-15T14:12:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: "use_repo_digest": { 2024-05-15T14:12:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_repo_digest", 2024-05-15T14:12:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-15T14:12:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.789 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.789 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_failed_host_check": { 2024-05-15T14:12:11.789 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_failed_host_check", 2024-05-15T14:12:11.789 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.789 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.789 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.789 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:11.790 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.790 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.790 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.790 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if the host check fails", 2024-05-15T14:12:11.790 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.790 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.791 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.791 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_daemons": { 2024-05-15T14:12:11.791 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_daemons", 2024-05-15T14:12:11.791 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.791 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.791 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.791 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:11.792 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.792 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.792 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.792 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-15T14:12:11.792 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.792 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.792 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.793 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_hosts": { 2024-05-15T14:12:11.793 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_hosts", 2024-05-15T14:12:11.793 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.793 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.793 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.793 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:11.793 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.794 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.794 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.794 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-15T14:12:11.794 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.794 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.794 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.794 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:11.794 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:11.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.795 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:11.795 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crash", 2024-05-15T14:12:11.795 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:11.795 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:11.795 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:11.795 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:11.796 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:11.796 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.796 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.796 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.796 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:11.796 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.796 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.796 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.798 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.798 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.798 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.798 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:11.798 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:11.798 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.800 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.800 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.800 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:11.800 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:11.800 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.800 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.801 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.801 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:11.801 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.801 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.801 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:11.801 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:11.801 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:11.801 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:11.802 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:11.802 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:11.802 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:11.802 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:11.802 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.802 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.802 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.803 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.803 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:11.803 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:11.803 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.803 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.803 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: "retain_interval": { 2024-05-15T14:12:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retain_interval", 2024-05-15T14:12:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "31536000", 2024-05-15T14:12:11.806 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.806 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.806 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.806 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-15T14:12:11.806 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.806 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.806 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.807 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:11.807 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:11.807 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:11.807 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:11.807 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.807 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:11.808 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.808 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.808 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.808 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.808 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.808 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.808 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_recent_interval": { 2024-05-15T14:12:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_recent_interval", 2024-05-15T14:12:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1209600", 2024-05-15T14:12:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-15T14:12:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dashboard", 2024-05-15T14:12:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-15T14:12:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-15T14:12:11.812 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:11.812 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.812 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.812 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-15T14:12:11.812 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.812 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.812 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.813 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.813 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.813 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.813 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.813 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_HOST": { 2024-05-15T14:12:11.813 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-15T14:12:11.813 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.814 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.814 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.814 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:11.814 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.814 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.814 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.814 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.815 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.815 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.815 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.815 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-15T14:12:11.815 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-15T14:12:11.815 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.816 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.816 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.816 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:11.816 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.816 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.816 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.816 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.817 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.817 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.817 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.817 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_ENABLED": { 2024-05-15T14:12:11.817 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_ENABLED", 2024-05-15T14:12:11.817 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.818 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.818 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.818 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:11.818 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.818 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.818 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.818 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.819 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.819 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.819 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.819 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-15T14:12:11.819 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-15T14:12:11.819 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.819 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.820 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.820 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:11.820 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.820 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.820 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.820 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.820 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.820 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.821 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.821 INFO:teuthology.orchestra.run.smithi033.stdout: "ENABLE_BROWSABLE_API": { 2024-05-15T14:12:11.821 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-15T14:12:11.821 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.821 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.821 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.821 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:11.822 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.822 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.822 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.822 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.822 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.822 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.822 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-15T14:12:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-15T14:12:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.825 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-15T14:12:11.825 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-15T14:12:11.825 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.825 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.825 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.825 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:11.825 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.825 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.826 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.826 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.826 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.826 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.826 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.826 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-15T14:12:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-15T14:12:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-15T14:12:11.829 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-15T14:12:11.829 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.829 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.829 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.829 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:11.829 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.829 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.829 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.830 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.830 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.830 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.830 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.830 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-15T14:12:11.830 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-15T14:12:11.831 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.831 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.831 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.831 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:11.831 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.831 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.831 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.831 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-15T14:12:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-15T14:12:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.833 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.833 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:11.833 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.833 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.833 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.833 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.833 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.833 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.834 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.834 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.834 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-15T14:12:11.834 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-15T14:12:11.834 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.834 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.834 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.835 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:11.835 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.835 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.835 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.835 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.835 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.835 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.835 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-15T14:12:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-15T14:12:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.837 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.837 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.837 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.837 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.837 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.837 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.837 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.838 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_PASSWORD": { 2024-05-15T14:12:11.838 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-15T14:12:11.838 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.838 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.838 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.838 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-05-15T14:12:11.838 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.838 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-15T14:12:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-15T14:12:11.840 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.840 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.840 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.840 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:11.840 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.840 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_URL": { 2024-05-15T14:12:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_URL", 2024-05-15T14:12:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_USERNAME": { 2024-05-15T14:12:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-15T14:12:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.844 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.844 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.844 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-05-15T14:12:11.844 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.844 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.844 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.844 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.845 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.845 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.845 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.845 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-15T14:12:11.845 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-15T14:12:11.845 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.845 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-15T14:12:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-15T14:12:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-15T14:12:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-15T14:12:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.851 INFO:teuthology.orchestra.run.smithi033.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-15T14:12:11.851 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-15T14:12:11.851 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.851 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.851 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.851 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:11.851 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.851 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.852 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.852 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.852 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.852 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.852 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.852 INFO:teuthology.orchestra.run.smithi033.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-15T14:12:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-15T14:12:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "[]", 2024-05-15T14:12:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: "MULTICLUSTER_CONFIG": { 2024-05-15T14:12:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-15T14:12:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "{}", 2024-05-15T14:12:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.856 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.856 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.856 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_HOST": { 2024-05-15T14:12:11.856 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-15T14:12:11.856 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.856 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.857 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.857 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:11.857 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.857 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.857 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.857 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.857 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.857 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.858 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.858 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-15T14:12:11.858 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-15T14:12:11.858 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.858 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.858 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-15T14:12:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-15T14:12:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.862 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-15T14:12:11.862 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-15T14:12:11.862 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.862 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.862 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.862 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:11.862 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.863 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.863 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.863 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.863 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.863 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.863 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.864 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-15T14:12:11.864 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-15T14:12:11.864 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.864 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.864 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.864 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:11.864 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.865 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.865 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.865 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.865 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.865 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.865 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.866 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-15T14:12:11.866 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-15T14:12:11.866 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.866 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.866 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.866 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:11.866 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.866 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-15T14:12:11.868 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-15T14:12:11.868 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.868 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.868 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.868 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:11.868 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.868 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.868 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.869 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.869 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.869 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.869 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.869 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-15T14:12:11.869 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-15T14:12:11.869 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.870 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.870 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.870 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:11.870 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.870 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.870 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.870 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.871 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.871 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.871 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.871 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-15T14:12:11.871 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-15T14:12:11.871 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.871 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.873 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.873 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_ENABLED": { 2024-05-15T14:12:11.873 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-15T14:12:11.873 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.873 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.873 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-15T14:12:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-15T14:12:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-15T14:12:11.876 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.876 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.876 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.876 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.876 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.876 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.876 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-15T14:12:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-15T14:12:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-15T14:12:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.878 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.878 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.878 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.878 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.878 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.878 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.878 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-15T14:12:11.879 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-15T14:12:11.879 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:11.879 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.879 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.879 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8", 2024-05-15T14:12:11.879 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.879 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.879 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.880 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.880 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.880 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.880 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.880 INFO:teuthology.orchestra.run.smithi033.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-15T14:12:11.880 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-15T14:12:11.880 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:11.881 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.881 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.881 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "45", 2024-05-15T14:12:11.881 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.881 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.881 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.881 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.882 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.882 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.882 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.882 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ACCESS_KEY": { 2024-05-15T14:12:11.882 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-15T14:12:11.882 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.882 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.883 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.883 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:11.883 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.883 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.883 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.883 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.883 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.883 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.884 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.884 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-15T14:12:11.884 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-15T14:12:11.884 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.884 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.884 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.885 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-05-15T14:12:11.885 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.885 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.885 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.885 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.885 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.885 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.886 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.886 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SECRET_KEY": { 2024-05-15T14:12:11.886 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-15T14:12:11.886 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.886 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.886 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.886 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:11.887 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.887 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.887 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.887 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.887 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.887 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.887 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.888 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SSL_VERIFY": { 2024-05-15T14:12:11.888 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-15T14:12:11.888 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.888 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.888 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.888 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.890 INFO:teuthology.orchestra.run.smithi033.stdout: "UNSAFE_TLS_v1_2": { 2024-05-15T14:12:11.890 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-15T14:12:11.890 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.890 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.890 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.890 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:11.890 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.890 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.891 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.891 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.891 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.891 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.891 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.891 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-15T14:12:11.891 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-15T14:12:11.892 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:11.892 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.892 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.892 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:11.892 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.892 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.892 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.893 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.893 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.893 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.893 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.893 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-15T14:12:11.893 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-15T14:12:11.893 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:11.894 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.894 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.894 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-15T14:12:11.894 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.894 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.894 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.894 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.895 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.895 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.895 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.895 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-15T14:12:11.895 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-15T14:12:11.895 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:11.895 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.896 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.896 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-15T14:12:11.896 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.896 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.896 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.896 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.896 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.896 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: "cross_origin_url": { 2024-05-15T14:12:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cross_origin_url", 2024-05-15T14:12:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:11.898 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.898 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.898 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.898 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.898 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.898 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.898 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: "crt_file": { 2024-05-15T14:12:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crt_file", 2024-05-15T14:12:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.900 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.900 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.900 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.900 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.900 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.900 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.900 INFO:teuthology.orchestra.run.smithi033.stdout: "debug": { 2024-05-15T14:12:11.901 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "debug", 2024-05-15T14:12:11.901 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.901 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.901 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.901 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:11.901 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.901 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.901 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.902 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable/disable debug options", 2024-05-15T14:12:11.902 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.902 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.902 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.902 INFO:teuthology.orchestra.run.smithi033.stdout: "jwt_token_ttl": { 2024-05-15T14:12:11.902 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "jwt_token_ttl", 2024-05-15T14:12:11.903 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:11.903 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.903 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.903 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "28800", 2024-05-15T14:12:11.903 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.903 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.903 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.903 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.904 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.904 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.904 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.904 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-05-15T14:12:11.904 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-05-15T14:12:11.904 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.904 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.905 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.905 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:11.905 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.905 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.905 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.905 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.905 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.906 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.906 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.906 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:11.906 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:11.906 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.906 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.906 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.907 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:11.907 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.907 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.907 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:11.907 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:11.907 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:11.907 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:11.908 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:11.908 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:11.908 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:11.908 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:11.908 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.908 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.908 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.908 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.909 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:11.909 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:11.909 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.909 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.909 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.909 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:11.910 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.910 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.910 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.910 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.910 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.910 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.910 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.911 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:11.911 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:11.911 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.911 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.911 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.911 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:11.911 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.912 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.912 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:11.912 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:11.912 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:11.912 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:11.912 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:11.912 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:11.912 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:11.913 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:11.913 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.913 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.913 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.913 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.913 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:11.914 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:11.914 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.914 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.914 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.914 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:11.914 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.914 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.914 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.915 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.915 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.915 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.915 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.915 INFO:teuthology.orchestra.run.smithi033.stdout: "motd": { 2024-05-15T14:12:11.915 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "motd", 2024-05-15T14:12:11.916 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.916 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.916 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.916 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:11.916 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.916 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.916 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.916 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "The message of the day", 2024-05-15T14:12:11.917 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.917 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.917 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.917 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect_resolve_ip_addr": { 2024-05-15T14:12:11.917 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "redirect_resolve_ip_addr", 2024-05-15T14:12:11.917 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.917 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.918 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.918 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:11.918 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.918 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.918 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.918 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.918 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.919 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.919 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.919 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-05-15T14:12:11.919 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-05-15T14:12:11.919 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.919 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.920 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.920 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-05-15T14:12:11.920 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.920 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.920 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.920 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.920 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.921 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.921 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.921 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-05-15T14:12:11.921 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-05-15T14:12:11.921 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:11.921 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.922 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.922 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8080", 2024-05-15T14:12:11.922 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.922 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.922 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.922 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.922 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.922 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.923 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.923 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:11.923 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:11.923 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:11.923 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:11.923 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.923 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:11.924 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.924 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.924 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.924 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.924 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.924 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.924 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.925 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-05-15T14:12:11.925 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-05-15T14:12:11.925 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.925 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.925 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.925 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:11.925 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.926 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.926 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.926 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.926 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.926 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.926 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.926 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl_server_port": { 2024-05-15T14:12:11.927 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl_server_port", 2024-05-15T14:12:11.927 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:11.927 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.927 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.927 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8443", 2024-05-15T14:12:11.927 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.927 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.927 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.928 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.928 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.928 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.928 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.928 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-05-15T14:12:11.928 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-05-15T14:12:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "redirect", 2024-05-15T14:12:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:11.930 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect" 2024-05-15T14:12:11.930 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:11.930 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.930 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.930 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.930 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.930 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-05-15T14:12:11.931 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-05-15T14:12:11.931 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:11.931 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.931 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.931 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-05-15T14:12:11.931 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-05-15T14:12:11.931 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-05-15T14:12:11.931 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.932 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.932 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.932 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.932 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.932 INFO:teuthology.orchestra.run.smithi033.stdout: "url_prefix": { 2024-05-15T14:12:11.932 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url_prefix", 2024-05-15T14:12:11.933 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.933 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.933 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.933 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:11.933 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.933 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.933 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.934 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.934 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.934 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.934 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.934 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:11.934 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:11.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:11.935 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "devicehealth", 2024-05-15T14:12:11.935 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:11.935 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:11.935 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:11.935 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_monitoring": { 2024-05-15T14:12:11.935 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_monitoring", 2024-05-15T14:12:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.937 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "monitor device health metrics", 2024-05-15T14:12:11.937 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.937 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.937 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.937 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:11.937 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:11.938 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.938 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.938 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.938 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:11.938 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.938 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.938 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:11.938 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:11.939 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:11.939 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:11.939 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:11.939 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:11.939 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:11.939 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:11.940 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.940 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.940 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.940 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.940 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:11.940 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:11.940 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.941 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.941 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.941 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:11.941 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.941 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.941 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.941 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.942 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.942 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.942 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.942 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:11.942 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:11.942 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.942 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.943 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.943 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:11.943 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.943 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.943 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:11.943 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:11.943 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:11.944 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:11.944 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:11.944 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:11.944 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:11.944 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:11.944 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.944 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.944 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.945 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.945 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:11.945 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:11.945 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.945 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.945 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.945 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:11.946 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.946 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.946 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.946 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.946 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.946 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.946 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: "mark_out_threshold": { 2024-05-15T14:12:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mark_out_threshold", 2024-05-15T14:12:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2419200", 2024-05-15T14:12:11.948 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.948 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.948 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.948 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-15T14:12:11.948 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.948 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.948 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.949 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_name": { 2024-05-15T14:12:11.949 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_name", 2024-05-15T14:12:11.949 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.949 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.949 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.949 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "device_health_metrics", 2024-05-15T14:12:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-15T14:12:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.951 INFO:teuthology.orchestra.run.smithi033.stdout: "retention_period": { 2024-05-15T14:12:11.951 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retention_period", 2024-05-15T14:12:11.951 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:11.951 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.951 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.951 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15552000", 2024-05-15T14:12:11.952 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.952 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.952 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.952 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain device health metrics", 2024-05-15T14:12:11.952 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.952 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_frequency": { 2024-05-15T14:12:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_frequency", 2024-05-15T14:12:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-05-15T14:12:11.954 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.954 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.954 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.954 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-15T14:12:11.954 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.954 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.954 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.955 INFO:teuthology.orchestra.run.smithi033.stdout: "self_heal": { 2024-05-15T14:12:11.955 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "self_heal", 2024-05-15T14:12:11.955 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.955 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.955 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.955 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:11.956 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.956 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.956 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.956 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-15T14:12:11.956 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.956 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.956 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.957 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-15T14:12:11.957 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-15T14:12:11.957 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:11.957 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.957 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.957 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-15T14:12:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and check device health", 2024-05-15T14:12:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.959 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:11.959 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:11.959 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:11.959 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:11.959 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.959 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:11.959 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.960 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.960 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.960 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.960 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.960 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.960 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.960 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_threshold": { 2024-05-15T14:12:11.961 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_threshold", 2024-05-15T14:12:11.961 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:11.961 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.961 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.961 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7257600", 2024-05-15T14:12:11.961 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.961 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.961 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.962 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-15T14:12:11.962 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.962 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.962 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.962 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:11.962 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:11.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.963 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:11.963 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "diskprediction_local", 2024-05-15T14:12:11.963 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:11.963 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:11.963 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:11.963 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:11.963 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:11.963 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.964 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.964 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.964 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:11.964 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.964 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.964 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:11.964 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:11.965 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:11.965 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:11.965 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:11.965 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:11.965 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:11.965 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:11.965 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.965 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.966 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.966 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.966 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:11.966 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:11.966 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.966 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.968 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.968 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:11.968 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:11.968 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.968 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.968 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.969 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:11.969 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.969 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.969 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:11.969 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:11.969 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:11.969 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:11.969 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:11.970 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:11.970 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:11.970 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:11.970 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.970 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.970 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.970 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.971 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:11.971 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:11.971 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.971 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.971 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.971 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:11.972 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.972 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.972 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.972 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.972 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.972 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.972 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.973 INFO:teuthology.orchestra.run.smithi033.stdout: "predict_interval": { 2024-05-15T14:12:11.973 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predict_interval", 2024-05-15T14:12:11.973 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.973 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.973 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.973 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-05-15T14:12:11.973 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.974 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.974 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.974 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.974 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.974 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.974 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.975 INFO:teuthology.orchestra.run.smithi033.stdout: "predictor_model": { 2024-05-15T14:12:11.975 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predictor_model", 2024-05-15T14:12:11.975 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.975 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.975 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.975 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "prophetstor", 2024-05-15T14:12:11.976 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.976 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.976 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.976 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.976 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.976 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.977 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.977 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-15T14:12:11.977 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-15T14:12:11.977 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.977 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.977 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.978 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-15T14:12:11.978 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.978 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.978 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.978 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.978 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.979 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.979 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.979 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:11.979 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:11.979 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:11.979 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:11.980 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.980 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:11.980 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.980 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.980 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.980 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.980 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.981 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.981 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.981 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:11.981 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:11.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.981 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:11.981 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "influx", 2024-05-15T14:12:11.981 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": false, 2024-05-15T14:12:11.982 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "influxdb python module not found", 2024-05-15T14:12:11.982 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:11.982 INFO:teuthology.orchestra.run.smithi033.stdout: "batch_size": { 2024-05-15T14:12:11.982 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "batch_size", 2024-05-15T14:12:11.982 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:11.982 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.982 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.983 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5000", 2024-05-15T14:12:11.983 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.983 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.983 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.983 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-15T14:12:11.983 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.983 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.984 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.984 INFO:teuthology.orchestra.run.smithi033.stdout: "database": { 2024-05-15T14:12:11.984 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "database", 2024-05-15T14:12:11.984 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.984 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.984 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.985 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "ceph", 2024-05-15T14:12:11.985 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.985 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.985 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.985 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-15T14:12:11.985 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.985 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.985 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.986 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": { 2024-05-15T14:12:11.986 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hostname", 2024-05-15T14:12:11.986 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.986 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.986 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.986 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:11.987 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.987 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.987 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.987 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server hostname", 2024-05-15T14:12:11.987 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.987 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.987 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.988 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-15T14:12:11.988 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-15T14:12:11.988 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:11.988 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.988 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:11.988 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "30", 2024-05-15T14:12:11.988 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "5", 2024-05-15T14:12:11.989 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.989 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.989 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-15T14:12:11.989 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.989 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.989 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.990 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:11.990 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:11.990 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.990 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.990 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.990 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:11.991 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.991 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.991 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:11.991 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:11.991 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:11.991 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:11.991 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:11.992 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:11.992 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:11.992 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:11.992 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.992 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.992 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.992 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.993 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:11.993 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:11.993 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.993 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.993 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.993 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:11.993 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.994 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.994 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.994 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.994 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.994 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.994 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.994 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:11.995 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:11.995 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.995 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.995 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.995 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:11.995 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.995 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.996 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:11.996 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:11.996 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:11.996 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:11.996 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:11.996 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:11.996 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:11.996 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:11.997 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.997 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.997 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.997 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.997 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:11.997 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:11.997 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:11.998 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:11.998 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:11.998 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:11.998 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:11.998 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:11.998 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:11.998 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:11.999 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:11.999 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:11.999 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:11.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:11.999 INFO:teuthology.orchestra.run.smithi033.stdout: "password": { 2024-05-15T14:12:11.999 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "password", 2024-05-15T14:12:11.999 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:11.999 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.000 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.000 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.000 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.000 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.000 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.000 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "password of InfluxDB server user", 2024-05-15T14:12:12.000 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.001 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.001 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.001 INFO:teuthology.orchestra.run.smithi033.stdout: "port": { 2024-05-15T14:12:12.001 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "port", 2024-05-15T14:12:12.001 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.001 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.002 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.002 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8086", 2024-05-15T14:12:12.002 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.002 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.002 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.002 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server port", 2024-05-15T14:12:12.002 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.003 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.003 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.003 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.003 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.003 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.003 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.004 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.004 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.004 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.004 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.004 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.004 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.004 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.005 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.005 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.005 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-05-15T14:12:12.005 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-05-15T14:12:12.005 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.006 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.006 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.006 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "false", 2024-05-15T14:12:12.006 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.006 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.006 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.006 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-15T14:12:12.007 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.007 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.007 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.007 INFO:teuthology.orchestra.run.smithi033.stdout: "threads": { 2024-05-15T14:12:12.007 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threads", 2024-05-15T14:12:12.007 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.007 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.008 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.008 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-15T14:12:12.008 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-15T14:12:12.008 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "32", 2024-05-15T14:12:12.008 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.008 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-15T14:12:12.008 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: "username": { 2024-05-15T14:12:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "username", 2024-05-15T14:12:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.010 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.010 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.010 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.010 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.010 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "username of InfluxDB server user", 2024-05-15T14:12:12.010 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.010 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.011 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.011 INFO:teuthology.orchestra.run.smithi033.stdout: "verify_ssl": { 2024-05-15T14:12:12.011 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "verify_ssl", 2024-05-15T14:12:12.011 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.011 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.011 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.012 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "true", 2024-05-15T14:12:12.012 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.012 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.012 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.012 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-15T14:12:12.012 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.012 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.012 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.013 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.013 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.013 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.013 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "insights", 2024-05-15T14:12:12.013 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.013 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.013 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.015 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.015 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.015 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.015 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.015 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.015 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.015 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.016 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.016 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.016 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.016 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.016 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.016 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.018 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.018 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.018 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.018 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.018 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.018 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.019 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.019 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.019 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.019 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.019 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.019 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.019 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.019 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.021 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.021 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.021 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.021 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.021 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.021 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.022 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.022 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.022 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.022 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.022 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.022 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.022 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.023 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.023 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.023 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.023 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.023 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.023 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.024 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.024 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.024 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.024 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.024 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.024 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.024 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.024 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.024 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.025 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.025 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.025 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "iostat", 2024-05-15T14:12:12.025 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.025 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.026 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.026 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.026 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.026 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.026 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.026 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.026 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.026 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.027 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.027 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.027 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.027 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.027 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.027 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.027 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.028 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.028 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.028 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.028 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.028 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.028 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.028 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.029 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.029 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.029 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.029 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.029 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.029 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.029 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.029 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.030 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.030 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.030 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.030 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.030 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.030 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.031 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.031 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.031 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.031 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.031 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.031 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.031 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.031 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.032 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.032 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.032 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.032 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.032 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.032 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.032 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.032 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.033 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.033 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.033 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.033 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.033 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.033 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.034 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.034 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.034 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.034 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.034 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.034 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.034 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.035 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.035 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.035 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.035 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.035 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.035 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.035 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.036 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.036 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.036 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.036 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.036 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.036 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.036 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.037 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.037 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.037 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.037 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "k8sevents", 2024-05-15T14:12:12.037 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.037 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.038 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.038 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_event_retention_days": { 2024-05-15T14:12:12.038 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ceph_event_retention_days", 2024-05-15T14:12:12.038 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.038 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.038 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.038 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7", 2024-05-15T14:12:12.038 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.039 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.039 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.039 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-15T14:12:12.039 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.039 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.039 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.040 INFO:teuthology.orchestra.run.smithi033.stdout: "config_check_secs": { 2024-05-15T14:12:12.040 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_check_secs", 2024-05-15T14:12:12.040 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.040 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.040 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.040 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-15T14:12:12.040 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "10", 2024-05-15T14:12:12.040 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.041 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.041 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-15T14:12:12.041 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.041 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.041 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.041 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.042 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.042 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.042 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.042 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.042 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.042 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.042 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.043 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.043 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.043 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.043 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.043 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.043 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.043 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.044 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.044 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.044 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.044 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.044 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.044 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.045 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.045 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.045 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.045 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.045 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.045 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.045 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.045 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.046 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.046 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.046 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.046 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.046 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.046 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.047 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.047 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.047 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.047 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.047 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.047 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.047 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.050 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.050 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.050 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.050 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.050 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.050 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.050 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.050 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.051 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.051 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.051 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.051 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.051 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.051 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.053 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.053 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.053 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.053 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.053 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "localpool", 2024-05-15T14:12:12.053 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.054 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.054 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.054 INFO:teuthology.orchestra.run.smithi033.stdout: "failure_domain": { 2024-05-15T14:12:12.054 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "failure_domain", 2024-05-15T14:12:12.054 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.054 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.054 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.055 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "host", 2024-05-15T14:12:12.055 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.055 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.055 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.055 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "failure domain for any created local pool", 2024-05-15T14:12:12.055 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-15T14:12:12.055 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.055 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.056 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.056 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.056 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.056 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.056 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.056 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.057 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.057 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.057 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.057 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.057 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.057 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.057 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.059 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.059 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.059 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.059 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.059 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.059 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.059 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.060 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.060 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.060 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.060 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.060 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.060 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.060 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.061 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.061 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.061 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.061 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.061 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.061 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.061 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.061 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.062 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.062 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.062 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.062 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.062 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.062 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.062 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.064 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.064 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.064 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.064 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.064 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.064 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.064 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.064 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.065 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.065 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.065 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.065 INFO:teuthology.orchestra.run.smithi033.stdout: "min_size": { 2024-05-15T14:12:12.065 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_size", 2024-05-15T14:12:12.065 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default min_size for any created local pool", 2024-05-15T14:12:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-15T14:12:12.067 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.067 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.067 INFO:teuthology.orchestra.run.smithi033.stdout: "num_rep": { 2024-05-15T14:12:12.067 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "num_rep", 2024-05-15T14:12:12.067 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3", 2024-05-15T14:12:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default replica count for any created local pool", 2024-05-15T14:12:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.069 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.069 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.069 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num": { 2024-05-15T14:12:12.069 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_num", 2024-05-15T14:12:12.069 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.069 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.069 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.070 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "128", 2024-05-15T14:12:12.070 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.070 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.070 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.070 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default pg_num for any created local pool", 2024-05-15T14:12:12.070 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.070 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.071 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.071 INFO:teuthology.orchestra.run.smithi033.stdout: "prefix": { 2024-05-15T14:12:12.071 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prefix", 2024-05-15T14:12:12.071 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.071 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.071 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.071 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.072 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.072 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.072 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.072 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name prefix for any created local pool", 2024-05-15T14:12:12.072 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.072 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.072 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.073 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.073 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.073 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.073 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.073 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.073 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.073 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.074 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.074 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.074 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.074 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.074 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.074 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.074 INFO:teuthology.orchestra.run.smithi033.stdout: "subtree": { 2024-05-15T14:12:12.075 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "subtree", 2024-05-15T14:12:12.075 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.075 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.075 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.075 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "rack", 2024-05-15T14:12:12.075 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.075 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.076 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.076 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-15T14:12:12.076 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-15T14:12:12.076 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.076 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.076 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.077 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.077 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mds_autoscaler", 2024-05-15T14:12:12.077 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.077 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.077 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.078 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.078 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.078 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.078 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.078 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.078 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.079 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.079 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.079 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.079 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.079 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.079 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.079 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.080 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.080 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.080 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.080 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.080 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.080 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.082 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.082 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.082 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.082 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.082 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.082 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.082 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.083 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.083 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.083 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.083 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.083 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.083 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.084 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.084 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.084 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.084 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.084 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.084 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.084 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.084 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.085 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.085 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.085 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.085 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.085 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.085 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.085 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.086 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.086 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.086 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.086 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.086 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.086 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.086 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.087 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.087 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.087 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.087 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.087 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.087 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.087 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.088 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.088 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.088 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.088 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.088 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.088 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.088 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.088 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.089 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.089 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.089 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.089 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.089 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.089 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.090 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirroring", 2024-05-15T14:12:12.090 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.090 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.090 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.090 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.090 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.090 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.090 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.091 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.091 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.091 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.091 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.091 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.091 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.091 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.092 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.092 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.092 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.092 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.092 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.092 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.092 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.092 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.093 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.093 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.093 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.093 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.093 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.093 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.094 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.094 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.094 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.094 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.094 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.094 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.094 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.095 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.095 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.095 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.095 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.095 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.095 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.095 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.097 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.097 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.097 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.097 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.097 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.097 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.097 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.097 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.098 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.098 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.098 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.098 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.098 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.098 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.098 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.099 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.099 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.099 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.099 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.099 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.099 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.099 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.100 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.100 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.100 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.100 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.100 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.100 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.100 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.100 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.101 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.101 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.101 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.101 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.101 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.101 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.101 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.102 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "nfs", 2024-05-15T14:12:12.102 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.102 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.102 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.102 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.102 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.102 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.106 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.106 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.106 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.106 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.106 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.106 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.106 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.106 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.108 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.108 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.108 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.108 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.108 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.108 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.108 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.109 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.109 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.109 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.109 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.109 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.109 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.109 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.111 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.111 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.111 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.111 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.111 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.111 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.112 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.112 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.112 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.112 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.112 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.112 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.112 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.112 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.113 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.113 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.113 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.113 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.113 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.113 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.113 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.114 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-05-15T14:12:12.114 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.114 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.114 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.114 INFO:teuthology.orchestra.run.smithi033.stdout: "fail_fs": { 2024-05-15T14:12:12.114 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "fail_fs", 2024-05-15T14:12:12.114 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.115 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.115 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.115 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.115 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.115 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.115 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.115 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-15T14:12:12.115 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.116 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.116 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.116 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.116 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.116 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.116 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.116 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.116 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.117 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.117 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.117 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.117 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.117 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.117 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.117 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.118 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.118 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.118 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.118 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.118 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.118 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.118 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.118 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.119 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.119 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.119 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.119 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.119 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.119 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.120 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.120 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.120 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.120 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.120 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.120 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.120 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.120 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.121 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.121 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.121 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.121 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.121 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.121 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.121 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.122 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.122 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.122 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.122 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.122 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.122 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.122 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.123 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.123 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.123 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.123 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.123 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.123 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.123 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.124 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.124 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.124 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.124 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.124 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.124 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.124 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.124 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.125 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.125 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.125 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.125 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.125 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.125 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.125 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.126 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator": { 2024-05-15T14:12:12.126 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-05-15T14:12:12.126 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.126 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.126 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.126 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.126 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.127 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.127 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.127 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm", 2024-05-15T14:12:12.127 INFO:teuthology.orchestra.run.smithi033.stdout: "rook", 2024-05-15T14:12:12.127 INFO:teuthology.orchestra.run.smithi033.stdout: "test_orchestrator" 2024-05-15T14:12:12.127 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.127 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Orchestrator backend", 2024-05-15T14:12:12.128 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.128 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.128 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.128 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.128 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.128 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.129 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.129 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.129 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.129 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.129 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.129 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.129 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.130 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.130 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.130 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.130 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.130 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.130 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.130 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.131 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.131 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_perf_query", 2024-05-15T14:12:12.131 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.131 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.131 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.131 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.131 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.132 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.132 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.132 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.132 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.132 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.132 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.133 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.133 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.133 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.133 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.133 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.133 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.133 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.133 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.134 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.134 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.134 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.134 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.134 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.134 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.135 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.135 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.135 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.135 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.135 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.135 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.135 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.136 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.136 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.136 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.136 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.136 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.136 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.136 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.137 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.137 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.137 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.137 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.137 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.137 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.137 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.138 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.138 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.138 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.138 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.138 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.138 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.138 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.138 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.139 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.139 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.139 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.139 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.139 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.139 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.139 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.140 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.140 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.140 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.140 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.140 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.140 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.140 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_support", 2024-05-15T14:12:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.144 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.144 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.144 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.144 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.144 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.144 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.144 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.145 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.145 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.145 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.145 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.145 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.145 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.145 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.145 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.146 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.146 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.146 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.146 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.146 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.146 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.147 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.147 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.147 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.147 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.147 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.147 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.147 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.150 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.150 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.150 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.150 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.150 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.150 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.150 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.150 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.151 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.151 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.151 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.151 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.151 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.151 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.152 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.152 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.152 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.152 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.152 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.152 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.152 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.152 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.153 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.153 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.153 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.153 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.153 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.153 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.154 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.154 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.154 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.154 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.154 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.154 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.154 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.154 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_autoscaler", 2024-05-15T14:12:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.156 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.156 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.156 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.156 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.156 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.156 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.156 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.157 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.157 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.157 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.157 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.157 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.157 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.157 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.157 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.158 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.158 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.158 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.158 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.158 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.158 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.158 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.159 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.159 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.159 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.159 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.159 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.159 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.159 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.160 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.160 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.160 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.160 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.160 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.160 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.160 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.161 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.161 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.161 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.161 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.161 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.161 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.161 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.162 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.162 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.162 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.162 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.162 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.162 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.162 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.162 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.163 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.163 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.163 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.163 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.163 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.163 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.163 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.164 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.164 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.164 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.164 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.164 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.164 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.164 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.165 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-15T14:12:12.165 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-15T14:12:12.165 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:12.165 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.165 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.165 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-15T14:12:12.165 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.166 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.166 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.166 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.166 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.166 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.166 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.167 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.167 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.167 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.167 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.167 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.167 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.167 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.167 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.168 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.168 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.168 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.168 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.168 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.168 INFO:teuthology.orchestra.run.smithi033.stdout: "threshold": { 2024-05-15T14:12:12.168 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threshold", 2024-05-15T14:12:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-15T14:12:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-05-15T14:12:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1.0", 2024-05-15T14:12:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "scaling threshold", 2024-05-15T14:12:12.170 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-15T14:12:12.170 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.170 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.170 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.170 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.170 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.171 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "progress", 2024-05-15T14:12:12.171 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.171 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.171 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.171 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_pg_recovery_event": { 2024-05-15T14:12:12.171 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_pg_recovery_event", 2024-05-15T14:12:12.171 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.171 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.172 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.172 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.172 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.172 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.172 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.172 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-15T14:12:12.172 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.173 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.173 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.173 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-05-15T14:12:12.173 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-05-15T14:12:12.173 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.173 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.173 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.174 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.174 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.174 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.174 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.174 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.174 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.174 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.177 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.177 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.177 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.177 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.177 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.177 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.177 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.178 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.178 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.178 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.178 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.178 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.178 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.178 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.180 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.180 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.180 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.180 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.180 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.180 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.180 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.181 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.181 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.181 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.181 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.181 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.181 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.181 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.181 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.182 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.182 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.182 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.182 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.182 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.182 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.183 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.183 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.183 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.183 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.183 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.183 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.183 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: "max_completed_events": { 2024-05-15T14:12:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_completed_events", 2024-05-15T14:12:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "50", 2024-05-15T14:12:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "number of past completed events to remember", 2024-05-15T14:12:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.186 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.186 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-15T14:12:12.186 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-15T14:12:12.186 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:12.186 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.186 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.187 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-15T14:12:12.187 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.187 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.187 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.187 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long the module is going to sleep", 2024-05-15T14:12:12.187 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.187 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.187 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.188 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.188 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.188 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.188 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.188 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.188 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.189 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.189 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.189 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.189 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.189 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.189 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.189 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.190 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.190 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.190 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.190 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus", 2024-05-15T14:12:12.190 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.190 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.190 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.191 INFO:teuthology.orchestra.run.smithi033.stdout: "cache": { 2024-05-15T14:12:12.191 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cache", 2024-05-15T14:12:12.191 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.191 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.191 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.191 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.191 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.192 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.192 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.192 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.192 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.192 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.192 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.192 INFO:teuthology.orchestra.run.smithi033.stdout: "exclude_perf_counters": { 2024-05-15T14:12:12.193 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "exclude_perf_counters", 2024-05-15T14:12:12.193 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.193 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.193 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.193 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.193 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.193 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.194 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.194 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-15T14:12:12.194 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-15T14:12:12.194 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.194 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.194 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.195 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.195 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.195 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.195 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.195 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.195 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.195 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.195 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.196 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.196 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.196 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.196 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.197 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.197 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.197 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.197 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.197 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.197 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.197 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.198 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.198 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.198 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.198 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.198 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.198 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.199 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.199 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.199 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.199 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.199 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.199 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.199 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.200 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.200 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.200 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.200 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.200 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.200 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.200 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.200 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.201 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.201 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.201 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.201 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.201 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.201 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.201 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools": { 2024-05-15T14:12:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools", 2024-05-15T14:12:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.205 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.205 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.205 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.205 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.205 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.205 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.205 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.205 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.206 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.206 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.206 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-15T14:12:12.206 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-15T14:12:12.206 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.206 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "300", 2024-05-15T14:12:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.208 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.208 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_interval": { 2024-05-15T14:12:12.208 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_interval", 2024-05-15T14:12:12.208 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-15T14:12:12.208 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.208 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.208 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15.0", 2024-05-15T14:12:12.209 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.209 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.209 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.209 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.209 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.209 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.209 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.210 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-05-15T14:12:12.210 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-05-15T14:12:12.210 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.210 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.210 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.210 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-05-15T14:12:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-15T14:12:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.212 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-05-15T14:12:12.212 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-05-15T14:12:12.212 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.212 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.212 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.212 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "9283", 2024-05-15T14:12:12.212 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-15T14:12:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.214 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.214 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.214 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.214 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.214 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.214 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.214 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.214 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.215 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.215 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.215 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.215 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.215 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.215 INFO:teuthology.orchestra.run.smithi033.stdout: "stale_cache_strategy": { 2024-05-15T14:12:12.216 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stale_cache_strategy", 2024-05-15T14:12:12.216 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.216 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.216 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.216 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "log", 2024-05-15T14:12:12.216 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.216 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.216 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.217 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.217 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.217 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.217 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.217 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-05-15T14:12:12.217 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-05-15T14:12:12.217 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.218 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.218 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.218 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "default", 2024-05-15T14:12:12.218 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.218 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.218 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.218 INFO:teuthology.orchestra.run.smithi033.stdout: "default", 2024-05-15T14:12:12.219 INFO:teuthology.orchestra.run.smithi033.stdout: "error" 2024-05-15T14:12:12.219 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.219 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.219 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.219 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.219 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.220 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-05-15T14:12:12.220 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-05-15T14:12:12.220 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.220 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.220 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.220 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-05-15T14:12:12.220 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-05-15T14:12:12.221 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-05-15T14:12:12.221 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.221 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.221 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.221 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.221 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.221 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.221 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.222 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.222 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_support", 2024-05-15T14:12:12.222 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.222 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.222 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.223 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.223 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.223 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.223 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.223 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.223 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.223 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.224 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.224 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.224 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.224 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.224 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.224 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.224 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.224 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.225 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.225 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.225 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.225 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.225 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.225 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.225 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.226 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.226 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.226 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.226 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.226 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.226 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.226 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.227 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.227 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.227 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.227 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.227 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.227 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.227 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.229 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.229 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.229 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.229 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.229 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.229 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.229 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.229 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.230 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.230 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.230 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.230 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.230 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.230 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.231 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.231 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.231 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.231 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.231 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.231 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.231 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.231 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.232 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.232 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_snap_create": { 2024-05-15T14:12:12.232 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_snap_create", 2024-05-15T14:12:12.232 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.232 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.232 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.232 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-15T14:12:12.233 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.233 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.233 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.233 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.233 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.233 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.233 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.234 INFO:teuthology.orchestra.run.smithi033.stdout: "mirror_snapshot_schedule": { 2024-05-15T14:12:12.234 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirror_snapshot_schedule", 2024-05-15T14:12:12.234 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.234 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.234 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.234 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.234 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.235 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.235 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.235 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.235 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.235 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.235 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.235 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.236 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.236 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.236 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.236 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.236 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.236 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.236 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.236 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.237 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.237 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.237 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.237 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.237 INFO:teuthology.orchestra.run.smithi033.stdout: "trash_purge_schedule": { 2024-05-15T14:12:12.237 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "trash_purge_schedule", 2024-05-15T14:12:12.238 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.238 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.238 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.238 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.238 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.238 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.238 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.239 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.239 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.239 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.239 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.239 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.239 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.240 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "restful", 2024-05-15T14:12:12.240 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.240 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.240 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.240 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_auth": { 2024-05-15T14:12:12.240 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_auth", 2024-05-15T14:12:12.240 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.240 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.241 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.241 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.241 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.241 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.241 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.241 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.241 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.242 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.242 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.242 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-05-15T14:12:12.242 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-05-15T14:12:12.242 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.242 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.243 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.243 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.243 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.243 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.243 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.243 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.243 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.243 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.244 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.244 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.244 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.244 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.244 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.244 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.245 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.245 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.245 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.245 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.245 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.245 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.245 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.245 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.246 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.246 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.246 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.246 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.246 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.246 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.246 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.247 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.247 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.247 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.247 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.247 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.247 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.247 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.248 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.248 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.248 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.248 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.248 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.248 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.249 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.249 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.249 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.249 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.249 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.249 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.249 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.249 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.250 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.250 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.250 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.250 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.250 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.250 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.250 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.251 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.251 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.251 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.251 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.251 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.251 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.251 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.252 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.252 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.252 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.252 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.252 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.252 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.252 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.252 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.253 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.253 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.253 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.253 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-05-15T14:12:12.253 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-05-15T14:12:12.253 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.253 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.254 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.254 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.254 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.254 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.254 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.254 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.254 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.255 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.255 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.255 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-05-15T14:12:12.255 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-05-15T14:12:12.255 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.255 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.255 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.256 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.256 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.256 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.256 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.256 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.256 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.256 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.256 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.257 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.257 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.257 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.257 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.257 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.257 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.257 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.258 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.258 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.258 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.258 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.258 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.258 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.258 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rgw", 2024-05-15T14:12:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.261 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.261 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.261 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.261 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.261 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.261 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.261 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.261 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.262 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.262 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.262 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.262 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.262 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.262 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.263 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.263 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.263 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.263 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.263 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.263 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.263 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.263 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.264 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.264 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.264 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.264 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.264 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.264 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.264 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.265 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.265 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.265 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.265 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.265 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.265 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.265 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.266 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.266 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.266 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.266 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.266 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.266 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.266 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.266 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.267 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.267 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.267 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.267 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.267 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.267 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.267 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.268 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.268 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.268 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.268 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.268 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.268 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.268 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.269 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.269 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.269 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.269 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.269 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.269 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.269 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.270 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.270 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.270 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.270 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.270 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.270 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.270 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.270 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.271 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.271 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.271 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rook", 2024-05-15T14:12:12.271 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.271 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.271 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.272 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.272 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.272 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.272 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.272 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.272 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.272 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.272 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.273 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.273 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.273 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.273 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.273 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.273 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.273 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.274 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.274 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.274 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.274 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.274 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.274 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.275 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.275 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.275 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.275 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.275 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.275 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.275 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.275 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.276 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.276 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.276 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.276 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.276 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.276 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.277 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.277 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.277 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.277 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.277 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.277 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.277 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.277 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.278 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.278 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.278 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.278 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.278 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.278 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.278 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.278 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.279 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.279 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.279 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.279 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.279 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.279 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.279 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.280 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.280 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.280 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.280 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.280 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.280 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.280 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.281 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.281 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.281 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.281 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.281 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.281 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.281 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.282 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.282 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.282 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.282 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.282 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.282 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.282 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: "storage_class": { 2024-05-15T14:12:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "storage_class", 2024-05-15T14:12:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "local", 2024-05-15T14:12:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-15T14:12:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.285 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.285 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "selftest", 2024-05-15T14:12:12.285 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.285 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.285 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.285 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.286 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.286 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.286 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.286 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.286 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.286 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.286 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.288 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.288 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.288 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.288 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.288 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.288 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.288 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.289 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.289 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.289 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.289 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.289 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.289 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.289 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.289 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.290 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.290 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.290 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.290 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.290 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.290 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.291 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.291 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.291 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.291 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.291 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.291 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.291 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.293 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.293 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.293 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.293 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.293 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.293 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.295 INFO:teuthology.orchestra.run.smithi033.stdout: "roption1": { 2024-05-15T14:12:12.295 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption1", 2024-05-15T14:12:12.295 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.295 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.295 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.295 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.296 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.296 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.296 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.296 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.296 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.296 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.296 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.297 INFO:teuthology.orchestra.run.smithi033.stdout: "roption2": { 2024-05-15T14:12:12.297 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption2", 2024-05-15T14:12:12.297 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.297 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.297 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.297 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "xyz", 2024-05-15T14:12:12.297 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.298 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.298 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.298 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.298 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.298 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.298 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.299 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption1": { 2024-05-15T14:12:12.299 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption1", 2024-05-15T14:12:12.299 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.299 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.299 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.299 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.299 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.300 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.300 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.300 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.300 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.300 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.300 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.301 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption2": { 2024-05-15T14:12:12.301 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption2", 2024-05-15T14:12:12.301 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.301 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.301 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.301 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.301 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.302 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.302 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.302 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.302 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.302 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.302 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.302 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption3": { 2024-05-15T14:12:12.303 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption3", 2024-05-15T14:12:12.303 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-15T14:12:12.303 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.303 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.303 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.303 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.303 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption4": { 2024-05-15T14:12:12.305 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption4", 2024-05-15T14:12:12.305 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.305 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.305 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.305 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.305 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.305 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.306 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.306 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.306 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.306 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.306 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.306 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption5": { 2024-05-15T14:12:12.306 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption5", 2024-05-15T14:12:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption6": { 2024-05-15T14:12:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption6", 2024-05-15T14:12:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.310 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.310 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.310 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption7": { 2024-05-15T14:12:12.310 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption7", 2024-05-15T14:12:12.310 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.310 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-15T14:12:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "42", 2024-05-15T14:12:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.312 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.312 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.312 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.312 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.312 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.312 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.313 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.313 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.313 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.313 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.313 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.313 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.313 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: "testkey": { 2024-05-15T14:12:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testkey", 2024-05-15T14:12:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: "testlkey": { 2024-05-15T14:12:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testlkey", 2024-05-15T14:12:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.317 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.317 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.317 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.317 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.317 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.317 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.317 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.318 INFO:teuthology.orchestra.run.smithi033.stdout: "testnewline": { 2024-05-15T14:12:12.318 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testnewline", 2024-05-15T14:12:12.318 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.318 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.318 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.318 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.319 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.319 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.319 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.319 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.319 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.319 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.319 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.319 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.320 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.320 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.320 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smb", 2024-05-15T14:12:12.320 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.320 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.320 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.325 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.325 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.325 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.325 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.325 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.325 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.326 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.326 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.326 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.326 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.326 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.326 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.328 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.328 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.328 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.328 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.328 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.328 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.328 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.329 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.329 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.329 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.329 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.329 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.329 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.329 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.330 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.330 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.330 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.330 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.330 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.330 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.330 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.331 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.331 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.331 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.331 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.331 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.331 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.331 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.332 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.332 INFO:teuthology.orchestra.run.smithi033.stdout: "update_orchestration": { 2024-05-15T14:12:12.332 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "update_orchestration", 2024-05-15T14:12:12.332 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.332 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.332 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.332 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.333 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.333 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.333 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.333 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-15T14:12:12.333 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.333 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.333 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.334 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.334 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.334 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.334 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snap_schedule", 2024-05-15T14:12:12.334 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.334 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.334 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.335 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_m_granularity": { 2024-05-15T14:12:12.335 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_m_granularity", 2024-05-15T14:12:12.335 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.335 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.335 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.335 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.335 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.336 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.336 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.336 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow minute scheduled snapshots", 2024-05-15T14:12:12.336 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.336 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.336 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.336 INFO:teuthology.orchestra.run.smithi033.stdout: "dump_on_update": { 2024-05-15T14:12:12.337 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dump_on_update", 2024-05-15T14:12:12.337 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.337 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.337 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.337 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.337 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.337 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.337 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.338 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "dump database to debug log on update", 2024-05-15T14:12:12.338 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.338 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.338 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.338 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.338 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.339 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.339 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.339 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.339 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.339 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.339 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.339 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.339 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.340 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.340 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.340 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.340 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.340 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.340 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.340 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.341 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.341 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.341 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.341 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.341 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.341 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.341 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.341 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.342 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.342 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.342 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.342 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.342 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.342 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.342 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.344 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.344 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.344 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.344 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.344 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.344 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.344 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.345 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.345 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.345 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.345 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.345 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.345 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.345 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.345 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.346 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.346 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.346 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.346 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.346 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.346 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.346 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.347 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.347 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.347 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.347 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.347 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.347 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.347 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.350 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.350 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.350 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stats", 2024-05-15T14:12:12.350 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.350 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.350 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.351 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.351 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.351 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.351 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.351 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.351 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.351 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.352 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.352 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.352 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.352 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.352 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.352 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.352 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.353 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.353 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.353 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.353 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.353 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.353 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.353 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.354 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.354 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.354 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.354 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.354 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.354 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.354 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.355 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.355 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.355 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.355 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.355 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.355 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.355 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.355 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.359 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.359 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.359 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.359 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.359 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.359 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.359 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.360 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.360 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.360 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.360 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.360 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.360 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.360 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.362 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.362 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.362 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.362 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.362 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "status", 2024-05-15T14:12:12.362 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.363 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.363 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.363 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.363 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.363 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.363 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.363 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.363 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.364 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.364 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.364 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.364 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.364 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.364 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.364 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.364 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.365 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.365 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.365 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.365 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.365 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.365 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.365 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.366 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.366 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.366 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.366 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.366 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.366 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.366 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.366 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.367 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.367 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.367 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.367 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.367 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.367 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.367 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.368 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.368 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.368 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.368 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.368 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.368 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.368 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.368 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.369 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.369 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.369 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.369 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.369 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.369 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.369 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.370 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.370 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.370 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.370 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.370 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.370 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.370 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.370 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.371 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.371 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.371 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.371 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.371 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.371 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.371 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.371 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.372 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.372 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.372 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.372 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.372 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.372 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.372 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.372 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.373 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.373 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.373 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.373 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.373 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.373 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.373 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.373 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.374 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.374 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.374 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.374 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telegraf", 2024-05-15T14:12:12.374 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.374 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.374 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.375 INFO:teuthology.orchestra.run.smithi033.stdout: "address": { 2024-05-15T14:12:12.375 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "address", 2024-05-15T14:12:12.375 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.375 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.375 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.375 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-15T14:12:12.375 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.375 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.376 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.376 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.376 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.376 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.376 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.376 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.377 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-15T14:12:12.377 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-15T14:12:12.377 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:12.377 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.377 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.377 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15", 2024-05-15T14:12:12.377 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.377 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.378 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.378 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.378 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.378 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.378 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.380 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.380 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.380 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.380 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.380 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.380 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.381 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.381 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.381 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.381 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.381 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.381 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.382 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.382 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.382 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.382 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.382 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.382 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.382 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.383 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.383 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.383 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.383 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.383 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.383 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.383 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.383 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.384 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.384 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.384 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.384 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.384 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.384 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.385 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.385 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.385 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.385 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.385 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.385 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.385 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.386 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.386 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.386 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.386 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.386 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.386 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.386 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.387 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.387 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.387 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.387 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.387 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.387 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.387 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.389 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.389 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.389 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.389 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.389 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.389 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.389 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.390 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.390 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.390 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.390 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.390 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.390 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.390 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telemetry", 2024-05-15T14:12:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_basic": { 2024-05-15T14:12:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_basic", 2024-05-15T14:12:12.392 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.392 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.392 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.392 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.392 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.392 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.393 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.393 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-15T14:12:12.393 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.393 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.393 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.393 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_crash": { 2024-05-15T14:12:12.394 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_crash", 2024-05-15T14:12:12.394 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.394 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.394 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.394 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.394 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-15T14:12:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_device": { 2024-05-15T14:12:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_device", 2024-05-15T14:12:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.397 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.397 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-15T14:12:12.397 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.397 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.397 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.397 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_ident": { 2024-05-15T14:12:12.398 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_ident", 2024-05-15T14:12:12.398 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.398 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.398 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.398 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.398 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.398 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.399 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.399 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-15T14:12:12.399 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.399 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.399 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.399 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_perf": { 2024-05-15T14:12:12.400 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_perf", 2024-05-15T14:12:12.400 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.400 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.400 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.400 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.400 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.400 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.401 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.401 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-15T14:12:12.401 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.401 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.401 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.401 INFO:teuthology.orchestra.run.smithi033.stdout: "contact": { 2024-05-15T14:12:12.402 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "contact", 2024-05-15T14:12:12.402 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.402 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.402 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.402 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.402 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.402 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.403 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.403 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.403 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.403 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.403 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.403 INFO:teuthology.orchestra.run.smithi033.stdout: "description": { 2024-05-15T14:12:12.404 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "description", 2024-05-15T14:12:12.404 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.404 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.404 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.404 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.404 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.404 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.405 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.405 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.405 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.405 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.405 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.406 INFO:teuthology.orchestra.run.smithi033.stdout: "device_url": { 2024-05-15T14:12:12.406 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_url", 2024-05-15T14:12:12.406 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.406 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.406 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.406 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-15T14:12:12.407 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.407 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.407 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.407 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.407 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.407 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.407 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-05-15T14:12:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-05-15T14:12:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.409 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.409 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.409 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.409 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.409 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.409 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.410 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-15T14:12:12.410 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-15T14:12:12.410 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.410 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.410 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.410 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "24", 2024-05-15T14:12:12.410 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "8", 2024-05-15T14:12:12.411 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.411 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.411 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.411 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.411 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.411 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.411 INFO:teuthology.orchestra.run.smithi033.stdout: "last_opt_revision": { 2024-05-15T14:12:12.412 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "last_opt_revision", 2024-05-15T14:12:12.412 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.412 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.412 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.412 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1", 2024-05-15T14:12:12.412 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.412 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.413 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.413 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.413 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.413 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.413 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.413 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard": { 2024-05-15T14:12:12.413 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard", 2024-05-15T14:12:12.414 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.414 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.414 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.414 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.414 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.414 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.414 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.415 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.415 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.415 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.415 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.415 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard_description": { 2024-05-15T14:12:12.415 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard_description", 2024-05-15T14:12:12.416 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.416 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.416 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.416 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.416 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.416 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.416 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.416 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.417 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.417 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.417 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.417 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.417 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.417 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.418 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.418 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.418 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.418 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.418 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.418 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.418 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.419 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.419 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.419 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.419 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.419 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.419 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.419 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.419 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.420 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.420 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.420 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.420 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.420 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.420 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.421 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.421 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.421 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.421 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.421 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.421 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.421 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.422 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.422 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.422 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.422 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.422 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.422 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.422 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.423 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.423 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.423 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.423 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.423 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.423 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.423 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.424 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.424 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.424 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.424 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.424 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.424 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.424 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.425 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.425 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.425 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.425 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.425 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.425 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.425 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.426 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.426 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.426 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.426 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.426 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.426 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.426 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.427 INFO:teuthology.orchestra.run.smithi033.stdout: "organization": { 2024-05-15T14:12:12.427 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "organization", 2024-05-15T14:12:12.427 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.427 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.427 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.427 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.427 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.428 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.428 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.428 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.428 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.428 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.428 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.428 INFO:teuthology.orchestra.run.smithi033.stdout: "proxy": { 2024-05-15T14:12:12.429 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "proxy", 2024-05-15T14:12:12.429 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.429 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.429 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.429 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.429 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.429 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.429 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.430 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.430 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.430 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.430 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.430 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.430 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.431 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.431 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.431 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.431 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.431 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.431 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.431 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.432 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.432 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.432 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.432 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.432 INFO:teuthology.orchestra.run.smithi033.stdout: "url": { 2024-05-15T14:12:12.432 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url", 2024-05-15T14:12:12.432 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.433 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.433 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.433 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-15T14:12:12.433 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.433 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.433 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.433 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.433 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.434 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.434 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.434 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.434 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.434 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "test_orchestrator", 2024-05-15T14:12:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.436 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.436 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.436 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.436 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.436 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.436 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.436 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.437 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.437 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.437 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.437 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.437 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.437 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.437 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.438 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.438 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.438 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.438 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.438 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.438 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.439 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.439 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.439 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.439 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.439 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.439 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.439 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.440 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.440 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.440 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.440 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.440 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.440 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.441 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.441 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.441 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.441 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.441 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.441 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.441 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.442 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.442 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.442 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.442 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.442 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.442 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.443 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.443 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.443 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.443 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.443 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.443 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.445 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.445 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.445 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.445 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.445 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.445 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.446 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.446 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.446 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.446 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.446 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.446 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.446 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.447 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.447 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.447 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.447 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.447 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.447 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "volumes", 2024-05-15T14:12:12.447 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.448 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.448 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.448 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.448 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.448 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.448 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.448 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.448 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.449 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.449 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.449 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.449 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.449 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.449 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.449 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.449 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.450 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.450 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.450 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.450 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.450 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.450 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.451 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.451 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.451 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.451 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.451 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.451 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.451 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.452 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.452 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.452 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.452 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.452 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.452 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.452 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.453 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.453 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.453 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.453 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.453 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.453 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.453 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.453 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_clones": { 2024-05-15T14:12:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_clones", 2024-05-15T14:12:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4", 2024-05-15T14:12:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-15T14:12:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: "periodic_async_work": { 2024-05-15T14:12:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "periodic_async_work", 2024-05-15T14:12:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Periodically check for async work", 2024-05-15T14:12:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_delay": { 2024-05-15T14:12:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_delay", 2024-05-15T14:12:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-15T14:12:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_no_wait": { 2024-05-15T14:12:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_no_wait", 2024-05-15T14:12:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-15T14:12:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix", 2024-05-15T14:12:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "discovery_interval": { 2024-05-15T14:12:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "discovery_interval", 2024-05-15T14:12:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-15T14:12:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "100", 2024-05-15T14:12:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "identifier": { 2024-05-15T14:12:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "identifier", 2024-05-15T14:12:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-15T14:12:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-15T14:12:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-15T14:12:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_host": { 2024-05-15T14:12:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_host", 2024-05-15T14:12:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.485 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.485 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.485 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.485 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.485 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.485 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.485 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.486 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.486 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_port": { 2024-05-15T14:12:12.486 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_port", 2024-05-15T14:12:12.486 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.486 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.486 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.486 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10051", 2024-05-15T14:12:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.488 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_sender": { 2024-05-15T14:12:12.488 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_sender", 2024-05-15T14:12:12.488 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.488 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.488 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.488 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-15T14:12:12.488 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.488 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.489 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.489 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.489 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.489 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.489 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.489 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.489 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.489 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-15T14:12:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: "gid": 24856, 2024-05-15T14:12:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "z", 2024-05-15T14:12:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_features": 4540701547738038271, 2024-05-15T14:12:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: "available_modules": [ 2024-05-15T14:12:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "alerts", 2024-05-15T14:12:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-15T14:12:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-15T14:12:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-15T14:12:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How frequently to reexamine health status", 2024-05-15T14:12:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.495 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.495 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.495 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.495 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.495 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.495 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.495 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.495 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_destination": { 2024-05-15T14:12:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_destination", 2024-05-15T14:12:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email address to send alerts to", 2024-05-15T14:12:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_from_name": { 2024-05-15T14:12:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_from_name", 2024-05-15T14:12:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "Ceph", 2024-05-15T14:12:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email From: name", 2024-05-15T14:12:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_host": { 2024-05-15T14:12:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_host", 2024-05-15T14:12:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP server", 2024-05-15T14:12:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.508 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_password": { 2024-05-15T14:12:12.508 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_password", 2024-05-15T14:12:12.508 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.508 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.508 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.508 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.508 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.509 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.509 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.509 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Password to authenticate with", 2024-05-15T14:12:12.509 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.509 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.509 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.509 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_port": { 2024-05-15T14:12:12.510 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_port", 2024-05-15T14:12:12.510 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.510 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.510 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.510 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "465", 2024-05-15T14:12:12.510 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.510 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.511 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.511 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP port", 2024-05-15T14:12:12.511 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.511 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.511 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.511 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.511 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_sender": { 2024-05-15T14:12:12.511 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_sender", 2024-05-15T14:12:12.512 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.512 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.512 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.512 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.512 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.512 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.512 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.512 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP envelope sender", 2024-05-15T14:12:12.513 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.513 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.513 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.513 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_ssl": { 2024-05-15T14:12:12.513 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_ssl", 2024-05-15T14:12:12.513 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.514 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.514 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.514 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.514 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.514 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.514 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.514 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-15T14:12:12.515 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.515 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.515 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.515 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_user": { 2024-05-15T14:12:12.515 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_user", 2024-05-15T14:12:12.515 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.515 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.516 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.516 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.516 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.516 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.516 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.516 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "User to authenticate as", 2024-05-15T14:12:12.516 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.516 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.517 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.517 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.517 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.517 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.517 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.517 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.517 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.518 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.518 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.518 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.518 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.518 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.518 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.518 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.519 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.519 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.519 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.519 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "balancer", 2024-05-15T14:12:12.519 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.519 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.519 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.520 INFO:teuthology.orchestra.run.smithi033.stdout: "active": { 2024-05-15T14:12:12.520 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "active", 2024-05-15T14:12:12.520 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.520 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.520 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.520 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.520 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.520 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.521 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.521 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically balance PGs across cluster", 2024-05-15T14:12:12.521 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.521 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.521 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.521 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_time": { 2024-05-15T14:12:12.521 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_time", 2024-05-15T14:12:12.522 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.522 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.522 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.522 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0000", 2024-05-15T14:12:12.522 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.522 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.522 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.522 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "beginning time of day to automatically balance", 2024-05-15T14:12:12.523 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-15T14:12:12.523 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.523 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.523 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_weekday": { 2024-05-15T14:12:12.523 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_weekday", 2024-05-15T14:12:12.523 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-15T14:12:12.524 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.524 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.524 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.524 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-05-15T14:12:12.524 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-05-15T14:12:12.524 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.524 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-15T14:12:12.524 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-15T14:12:12.525 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.525 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.525 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_max_iterations": { 2024-05-15T14:12:12.525 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_max_iterations", 2024-05-15T14:12:12.525 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-15T14:12:12.525 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.525 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.526 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "25", 2024-05-15T14:12:12.526 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-15T14:12:12.526 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "250", 2024-05-15T14:12:12.526 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.526 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-15T14:12:12.526 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.526 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.527 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.527 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_metrics": { 2024-05-15T14:12:12.527 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_metrics", 2024-05-15T14:12:12.527 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.527 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.527 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.527 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "pgs,objects,bytes", 2024-05-15T14:12:12.528 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.528 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.528 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.528 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-15T14:12:12.528 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-15T14:12:12.528 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.528 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.529 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_step": { 2024-05-15T14:12:12.529 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_step", 2024-05-15T14:12:12.529 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-15T14:12:12.529 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.529 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.529 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.5", 2024-05-15T14:12:12.530 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0.001", 2024-05-15T14:12:12.530 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "0.999", 2024-05-15T14:12:12.530 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.530 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "aggressiveness of optimization", 2024-05-15T14:12:12.530 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-15T14:12:12.530 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.530 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.531 INFO:teuthology.orchestra.run.smithi033.stdout: "end_time": { 2024-05-15T14:12:12.531 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_time", 2024-05-15T14:12:12.531 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.531 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.531 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.531 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2359", 2024-05-15T14:12:12.532 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.532 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.532 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.532 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ending time of day to automatically balance", 2024-05-15T14:12:12.532 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-15T14:12:12.532 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.532 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.533 INFO:teuthology.orchestra.run.smithi033.stdout: "end_weekday": { 2024-05-15T14:12:12.533 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_weekday", 2024-05-15T14:12:12.533 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-15T14:12:12.533 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.533 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.533 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.534 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-05-15T14:12:12.534 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-05-15T14:12:12.534 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.534 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-15T14:12:12.534 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-15T14:12:12.534 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.534 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.535 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.535 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.535 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.535 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.535 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.535 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.535 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.536 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.536 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.536 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.536 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.536 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.536 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.536 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.537 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.537 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.537 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.537 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.537 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.537 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.537 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.538 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.538 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.538 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.538 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.538 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.538 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.538 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.539 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.539 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.539 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.539 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.539 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.539 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.540 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.540 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.540 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.540 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.540 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.540 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.540 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.541 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.541 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.541 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.541 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.541 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.541 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.541 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.542 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.542 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.542 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.542 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.542 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.542 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.542 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.544 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.544 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.544 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.544 INFO:teuthology.orchestra.run.smithi033.stdout: "min_score": { 2024-05-15T14:12:12.544 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_score", 2024-05-15T14:12:12.544 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-15T14:12:12.545 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.545 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.545 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.545 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.545 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.545 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.545 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-15T14:12:12.545 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.546 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.546 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.546 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-05-15T14:12:12.546 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-05-15T14:12:12.546 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.546 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "upmap", 2024-05-15T14:12:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-compat", 2024-05-15T14:12:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: "none", 2024-05-15T14:12:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: "read", 2024-05-15T14:12:12.548 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap", 2024-05-15T14:12:12.548 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap-read" 2024-05-15T14:12:12.548 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.548 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Balancer mode", 2024-05-15T14:12:12.548 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.548 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.548 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.549 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_ids": { 2024-05-15T14:12:12.549 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_ids", 2024-05-15T14:12:12.549 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.549 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.549 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.549 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.549 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.550 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.550 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.550 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-15T14:12:12.550 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.550 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.550 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.551 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-15T14:12:12.551 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-15T14:12:12.551 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:12.551 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.551 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.551 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-15T14:12:12.551 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.551 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.552 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.552 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-15T14:12:12.552 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.552 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.552 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.552 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.552 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.553 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.553 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.553 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.553 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.553 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.553 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.553 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.554 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.554 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.554 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.554 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.554 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_deviation": { 2024-05-15T14:12:12.554 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_deviation", 2024-05-15T14:12:12.554 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.555 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.555 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.555 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-15T14:12:12.555 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-15T14:12:12.555 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.555 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.555 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-15T14:12:12.555 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-15T14:12:12.556 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.556 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.556 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_optimizations": { 2024-05-15T14:12:12.556 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_optimizations", 2024-05-15T14:12:12.556 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-15T14:12:12.557 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.557 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.557 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-15T14:12:12.557 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.557 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.557 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.557 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-15T14:12:12.557 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.558 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.558 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.558 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.558 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.558 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm", 2024-05-15T14:12:12.559 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.559 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.559 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.559 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_down_multiplier": { 2024-05-15T14:12:12.559 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_down_multiplier", 2024-05-15T14:12:12.559 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-15T14:12:12.559 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.559 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.561 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-05-15T14:12:12.561 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.561 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.562 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.562 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-15T14:12:12.562 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.562 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.562 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.562 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_refresh_rate": { 2024-05-15T14:12:12.563 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_refresh_rate", 2024-05-15T14:12:12.563 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:12.563 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.563 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.563 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "20", 2024-05-15T14:12:12.563 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.563 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.563 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.563 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-15T14:12:12.564 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.564 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.564 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.564 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_starting_port": { 2024-05-15T14:12:12.564 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_starting_port", 2024-05-15T14:12:12.564 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.564 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.565 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.565 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4721", 2024-05-15T14:12:12.565 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.565 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.565 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.565 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-15T14:12:12.565 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.565 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.565 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.566 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_ptrace": { 2024-05-15T14:12:12.566 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_ptrace", 2024-05-15T14:12:12.566 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.566 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.566 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.566 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.566 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.567 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.567 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.567 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-15T14:12:12.567 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-15T14:12:12.567 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.567 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.567 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_interval": { 2024-05-15T14:12:12.568 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_interval", 2024-05-15T14:12:12.568 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:12.568 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.568 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.568 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-15T14:12:12.568 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.568 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.568 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.568 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-15T14:12:12.569 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.569 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.569 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.569 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_memory_target_ratio": { 2024-05-15T14:12:12.569 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_memory_target_ratio", 2024-05-15T14:12:12.569 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-15T14:12:12.569 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.569 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.570 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.7", 2024-05-15T14:12:12.570 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.570 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.570 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.570 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-15T14:12:12.570 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.570 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.570 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.571 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm_log_destination": { 2024-05-15T14:12:12.571 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm_log_destination", 2024-05-15T14:12:12.571 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.571 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.571 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.571 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.571 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.572 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.572 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.572 INFO:teuthology.orchestra.run.smithi033.stdout: "file", 2024-05-15T14:12:12.572 INFO:teuthology.orchestra.run.smithi033.stdout: "file,syslog", 2024-05-15T14:12:12.572 INFO:teuthology.orchestra.run.smithi033.stdout: "syslog" 2024-05-15T14:12:12.572 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.572 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-15T14:12:12.572 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: "cgroups_split": { 2024-05-15T14:12:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cgroups_split", 2024-05-15T14:12:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-15T14:12:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: "config_checks_enabled": { 2024-05-15T14:12:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_checks_enabled", 2024-05-15T14:12:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-15T14:12:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: "config_dashboard": { 2024-05-15T14:12:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_dashboard", 2024-05-15T14:12:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-15T14:12:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.578 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.578 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_alertmanager": { 2024-05-15T14:12:12.578 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_alertmanager", 2024-05-15T14:12:12.578 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.578 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.578 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.579 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-15T14:12:12.579 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.579 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.579 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.579 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-15T14:12:12.579 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.579 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.579 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.580 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_base": { 2024-05-15T14:12:12.580 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_base", 2024-05-15T14:12:12.580 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.580 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.580 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.580 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-15T14:12:12.580 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.580 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.581 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.581 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Container image name, without the tag", 2024-05-15T14:12:12.581 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.581 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.581 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.581 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_elasticsearch": { 2024-05-15T14:12:12.581 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_elasticsearch", 2024-05-15T14:12:12.582 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.582 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.582 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.582 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-15T14:12:12.582 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.582 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.582 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.582 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "elasticsearch container image", 2024-05-15T14:12:12.583 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.583 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.583 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.583 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_grafana": { 2024-05-15T14:12:12.583 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_grafana", 2024-05-15T14:12:12.583 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.583 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.583 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.584 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-15T14:12:12.584 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.584 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.584 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.584 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-15T14:12:12.584 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.584 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.584 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.585 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_haproxy": { 2024-05-15T14:12:12.585 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_haproxy", 2024-05-15T14:12:12.585 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.585 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.585 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.585 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-15T14:12:12.585 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.585 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.586 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.586 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "HAproxy container image", 2024-05-15T14:12:12.586 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.586 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.586 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.586 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_agent": { 2024-05-15T14:12:12.587 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_agent", 2024-05-15T14:12:12.587 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.587 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.587 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.587 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-15T14:12:12.587 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.587 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.587 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.587 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger agent container image", 2024-05-15T14:12:12.588 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.588 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.588 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.588 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_collector": { 2024-05-15T14:12:12.588 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_collector", 2024-05-15T14:12:12.588 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.588 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.588 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.589 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-15T14:12:12.589 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.589 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.589 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.589 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger collector container image", 2024-05-15T14:12:12.589 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.589 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.589 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.590 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_query": { 2024-05-15T14:12:12.590 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_query", 2024-05-15T14:12:12.590 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.590 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.590 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.590 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-15T14:12:12.590 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.590 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.591 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.591 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger query container image", 2024-05-15T14:12:12.591 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.591 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.591 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.591 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_keepalived": { 2024-05-15T14:12:12.591 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_keepalived", 2024-05-15T14:12:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-15T14:12:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Keepalived container image", 2024-05-15T14:12:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_loki": { 2024-05-15T14:12:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_loki", 2024-05-15T14:12:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-15T14:12:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Loki container image", 2024-05-15T14:12:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_node_exporter": { 2024-05-15T14:12:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_node_exporter", 2024-05-15T14:12:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-15T14:12:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-15T14:12:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_nvmeof": { 2024-05-15T14:12:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_nvmeof", 2024-05-15T14:12:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-15T14:12:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Nvme-of container image", 2024-05-15T14:12:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_prometheus": { 2024-05-15T14:12:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_prometheus", 2024-05-15T14:12:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-15T14:12:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-15T14:12:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_promtail": { 2024-05-15T14:12:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_promtail", 2024-05-15T14:12:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-15T14:12:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Promtail container image", 2024-05-15T14:12:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_samba": { 2024-05-15T14:12:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_samba", 2024-05-15T14:12:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-15T14:12:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Samba/SMB container image", 2024-05-15T14:12:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_snmp_gateway": { 2024-05-15T14:12:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_snmp_gateway", 2024-05-15T14:12:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-15T14:12:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SNMP Gateway container image", 2024-05-15T14:12:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "container_init": { 2024-05-15T14:12:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_init", 2024-05-15T14:12:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Run podman/docker with `--init`", 2024-05-15T14:12:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "daemon_cache_timeout": { 2024-05-15T14:12:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "daemon_cache_timeout", 2024-05-15T14:12:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-15T14:12:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-15T14:12:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "default_cephadm_command_timeout": { 2024-05-15T14:12:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_cephadm_command_timeout", 2024-05-15T14:12:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "900", 2024-05-15T14:12:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-15T14:12:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "default_registry": { 2024-05-15T14:12:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_registry", 2024-05-15T14:12:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io", 2024-05-15T14:12:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-15T14:12:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "device_cache_timeout": { 2024-05-15T14:12:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_cache_timeout", 2024-05-15T14:12:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1800", 2024-05-15T14:12:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache device inventory", 2024-05-15T14:12:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.613 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.613 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.613 INFO:teuthology.orchestra.run.smithi033.stdout: "device_enhanced_scan": { 2024-05-15T14:12:12.613 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_enhanced_scan", 2024-05-15T14:12:12.613 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.613 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.613 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-15T14:12:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "facts_cache_timeout": { 2024-05-15T14:12:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "facts_cache_timeout", 2024-05-15T14:12:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-15T14:12:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache host facts data", 2024-05-15T14:12:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "grafana_dashboards_path": { 2024-05-15T14:12:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "grafana_dashboards_path", 2024-05-15T14:12:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-15T14:12:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-15T14:12:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "host_check_interval": { 2024-05-15T14:12:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "host_check_interval", 2024-05-15T14:12:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-15T14:12:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to perform a host check", 2024-05-15T14:12:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "hw_monitoring": { 2024-05-15T14:12:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hw_monitoring", 2024-05-15T14:12:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-15T14:12:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "inventory_list_all": { 2024-05-15T14:12:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "inventory_list_all", 2024-05-15T14:12:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-15T14:12:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "log_refresh_metadata": { 2024-05-15T14:12:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_refresh_metadata", 2024-05-15T14:12:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-15T14:12:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-15T14:12:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-15T14:12:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-15T14:12:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-15T14:12:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-15T14:12:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-15T14:12:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "*", 2024-05-15T14:12:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-15T14:12:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "max_count_per_host": { 2024-05-15T14:12:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_count_per_host", 2024-05-15T14:12:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-15T14:12:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of daemons per service per host", 2024-05-15T14:12:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "max_osd_draining_count": { 2024-05-15T14:12:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_osd_draining_count", 2024-05-15T14:12:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-15T14:12:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-15T14:12:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: "migration_current": { 2024-05-15T14:12:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "migration_current", 2024-05-15T14:12:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "internal - do not modify", 2024-05-15T14:12:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-05-15T14:12:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-05-15T14:12:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "root", 2024-05-15T14:12:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm-package", 2024-05-15T14:12:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "root" 2024-05-15T14:12:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "mode for remote execution of cephadm", 2024-05-15T14:12:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "oob_default_addr": { 2024-05-15T14:12:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "oob_default_addr", 2024-05-15T14:12:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "169.254.1.1", 2024-05-15T14:12:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-15T14:12:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "prometheus_alerts_path": { 2024-05-15T14:12:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus_alerts_path", 2024-05-15T14:12:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-15T14:12:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-15T14:12:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_insecure": { 2024-05-15T14:12:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_insecure", 2024-05-15T14:12:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-15T14:12:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_password": { 2024-05-15T14:12:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_password", 2024-05-15T14:12:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-15T14:12:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_url": { 2024-05-15T14:12:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_url", 2024-05-15T14:12:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-15T14:12:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_username": { 2024-05-15T14:12:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_username", 2024-05-15T14:12:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-15T14:12:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "secure_monitoring_stack": { 2024-05-15T14:12:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "secure_monitoring_stack", 2024-05-15T14:12:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-15T14:12:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "service_discovery_port": { 2024-05-15T14:12:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "service_discovery_port", 2024-05-15T14:12:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8765", 2024-05-15T14:12:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "cephadm service discovery port", 2024-05-15T14:12:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "ssh_config_file": { 2024-05-15T14:12:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssh_config_file", 2024-05-15T14:12:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-15T14:12:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "use_agent": { 2024-05-15T14:12:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_agent", 2024-05-15T14:12:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-15T14:12:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: "use_repo_digest": { 2024-05-15T14:12:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_repo_digest", 2024-05-15T14:12:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-15T14:12:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_failed_host_check": { 2024-05-15T14:12:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_failed_host_check", 2024-05-15T14:12:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if the host check fails", 2024-05-15T14:12:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_daemons": { 2024-05-15T14:12:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_daemons", 2024-05-15T14:12:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-15T14:12:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_hosts": { 2024-05-15T14:12:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_hosts", 2024-05-15T14:12:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-15T14:12:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crash", 2024-05-15T14:12:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "retain_interval": { 2024-05-15T14:12:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retain_interval", 2024-05-15T14:12:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "31536000", 2024-05-15T14:12:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-15T14:12:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_recent_interval": { 2024-05-15T14:12:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_recent_interval", 2024-05-15T14:12:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1209600", 2024-05-15T14:12:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-15T14:12:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dashboard", 2024-05-15T14:12:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-15T14:12:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-15T14:12:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-15T14:12:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_HOST": { 2024-05-15T14:12:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-15T14:12:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-15T14:12:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-15T14:12:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_ENABLED": { 2024-05-15T14:12:12.690 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_ENABLED", 2024-05-15T14:12:12.690 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.690 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.690 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.690 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.690 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.690 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.690 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.690 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-15T14:12:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-15T14:12:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.692 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.692 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.692 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.692 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.692 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.692 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.692 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.692 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.693 INFO:teuthology.orchestra.run.smithi033.stdout: "ENABLE_BROWSABLE_API": { 2024-05-15T14:12:12.693 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-15T14:12:12.693 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.693 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.693 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.693 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.693 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.693 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.694 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.694 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.694 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.694 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.694 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.694 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-15T14:12:12.694 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-15T14:12:12.695 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.695 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.695 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.695 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.695 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.695 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.695 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.695 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.695 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.696 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.696 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.696 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-15T14:12:12.696 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-15T14:12:12.696 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.696 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.696 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.697 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.697 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.697 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.697 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.697 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.697 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.697 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.697 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.698 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-15T14:12:12.698 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-15T14:12:12.698 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.698 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.698 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.698 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.698 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.698 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.699 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.699 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.699 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.699 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.699 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.699 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-15T14:12:12.699 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-15T14:12:12.700 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.700 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.700 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.700 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.700 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.700 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.700 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.700 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.701 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.701 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.701 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.701 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-15T14:12:12.701 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-15T14:12:12.701 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.701 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.701 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.702 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.702 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.702 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.702 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.702 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.702 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.702 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.702 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-15T14:12:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-15T14:12:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-15T14:12:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-15T14:12:12.705 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.705 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.705 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.705 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.705 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.705 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.705 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.705 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-15T14:12:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-15T14:12:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.708 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.708 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_PASSWORD": { 2024-05-15T14:12:12.708 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-15T14:12:12.708 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.708 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.708 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.708 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-05-15T14:12:12.708 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.709 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.709 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.709 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.709 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.709 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.709 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.709 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-15T14:12:12.710 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-15T14:12:12.710 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.710 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.710 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.710 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.710 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.710 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.710 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.711 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.711 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.711 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.711 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.711 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_URL": { 2024-05-15T14:12:12.711 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_URL", 2024-05-15T14:12:12.711 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.711 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.713 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_USERNAME": { 2024-05-15T14:12:12.713 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-15T14:12:12.713 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.713 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.713 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.713 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-05-15T14:12:12.713 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.714 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.714 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.714 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.714 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.714 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.714 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.714 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-15T14:12:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-15T14:12:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.716 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.716 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.716 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.716 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-15T14:12:12.716 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-15T14:12:12.716 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.716 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.717 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.717 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.717 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.717 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.717 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.717 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.717 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.717 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.718 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.718 INFO:teuthology.orchestra.run.smithi033.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-15T14:12:12.718 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-15T14:12:12.718 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.718 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.718 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.718 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-15T14:12:12.720 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-15T14:12:12.720 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.720 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.720 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.720 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.720 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.720 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.720 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.720 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.721 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.721 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.721 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.721 INFO:teuthology.orchestra.run.smithi033.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-15T14:12:12.721 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-15T14:12:12.721 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.721 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.722 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.722 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "[]", 2024-05-15T14:12:12.722 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.722 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.722 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.722 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.722 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.722 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.723 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.723 INFO:teuthology.orchestra.run.smithi033.stdout: "MULTICLUSTER_CONFIG": { 2024-05-15T14:12:12.723 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-15T14:12:12.723 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.723 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.723 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.723 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "{}", 2024-05-15T14:12:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_HOST": { 2024-05-15T14:12:12.725 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-15T14:12:12.725 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.725 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.725 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.725 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.725 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.725 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.725 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.725 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.726 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.726 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.726 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.726 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-15T14:12:12.726 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-15T14:12:12.726 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.726 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.727 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.727 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.727 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.727 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.727 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.727 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.727 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.727 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.728 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.728 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-15T14:12:12.728 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-15T14:12:12.728 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.728 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.728 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.728 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.729 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.729 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.729 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.729 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.729 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.729 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.729 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.730 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-15T14:12:12.730 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-15T14:12:12.730 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.730 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.730 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.730 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.730 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.730 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.731 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.731 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.731 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.731 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.731 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.731 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-15T14:12:12.731 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-15T14:12:12.731 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.732 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.732 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.732 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.732 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.732 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.732 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.732 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.732 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.732 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.733 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.733 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-15T14:12:12.733 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-15T14:12:12.733 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.733 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.733 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.733 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.734 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.734 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.734 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.734 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.734 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.734 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.734 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.735 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-15T14:12:12.735 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-15T14:12:12.735 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.735 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.735 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.735 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.735 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.735 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.735 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.736 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.736 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.736 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.736 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.736 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-15T14:12:12.736 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-15T14:12:12.736 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.737 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.737 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.737 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.737 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.737 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.737 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.737 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.737 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.737 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.738 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.738 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-15T14:12:12.738 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-15T14:12:12.738 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.738 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.738 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.738 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.739 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.739 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.739 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.739 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.739 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.739 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.739 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.740 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_ENABLED": { 2024-05-15T14:12:12.740 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-15T14:12:12.740 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.740 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.740 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.740 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.740 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.740 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.740 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.741 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.741 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.741 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.741 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.741 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-15T14:12:12.741 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-15T14:12:12.741 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.742 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.742 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.742 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-15T14:12:12.742 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.742 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.742 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.742 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.742 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.743 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.743 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.743 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.743 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-15T14:12:12.743 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-15T14:12:12.743 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.743 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.743 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.744 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-15T14:12:12.744 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.744 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.744 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.744 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.744 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.744 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.744 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.744 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.745 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-15T14:12:12.745 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-15T14:12:12.745 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.745 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.745 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.745 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8", 2024-05-15T14:12:12.745 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.745 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.745 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.746 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.746 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.746 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.746 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.746 INFO:teuthology.orchestra.run.smithi033.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-15T14:12:12.746 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-15T14:12:12.746 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.747 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.747 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.747 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "45", 2024-05-15T14:12:12.747 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.747 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.747 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.747 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.747 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.747 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.748 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.748 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ACCESS_KEY": { 2024-05-15T14:12:12.748 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-15T14:12:12.748 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.748 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.748 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.748 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.749 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.749 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.749 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.749 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.749 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.749 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.749 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.750 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-15T14:12:12.750 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-15T14:12:12.750 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.750 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.750 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.750 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-05-15T14:12:12.750 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.750 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.751 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.751 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.751 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.751 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.751 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.751 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SECRET_KEY": { 2024-05-15T14:12:12.751 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-15T14:12:12.751 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.752 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.752 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.752 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.752 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.752 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.752 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.752 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.752 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.753 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.753 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.753 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SSL_VERIFY": { 2024-05-15T14:12:12.753 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-15T14:12:12.753 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.753 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.753 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.754 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.754 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.754 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.754 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.754 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.754 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.754 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.754 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.755 INFO:teuthology.orchestra.run.smithi033.stdout: "UNSAFE_TLS_v1_2": { 2024-05-15T14:12:12.755 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-15T14:12:12.755 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.755 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.755 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.755 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.755 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.755 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.756 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.756 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.756 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.756 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.756 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.756 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-15T14:12:12.756 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-15T14:12:12.757 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.757 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.757 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.757 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.757 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.757 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.757 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.757 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.757 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.758 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.758 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.758 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-15T14:12:12.758 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-15T14:12:12.758 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.758 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.758 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.759 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-15T14:12:12.759 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.759 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.759 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.759 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.759 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.759 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.759 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.759 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.760 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-15T14:12:12.760 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-15T14:12:12.760 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.760 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.760 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.760 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-15T14:12:12.760 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.760 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.760 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.761 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.761 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.761 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.761 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.761 INFO:teuthology.orchestra.run.smithi033.stdout: "cross_origin_url": { 2024-05-15T14:12:12.761 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cross_origin_url", 2024-05-15T14:12:12.761 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.762 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.762 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.762 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.762 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.762 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.762 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.762 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.762 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.762 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.763 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.763 INFO:teuthology.orchestra.run.smithi033.stdout: "crt_file": { 2024-05-15T14:12:12.763 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crt_file", 2024-05-15T14:12:12.763 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.763 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.763 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.763 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.763 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.764 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.764 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.764 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.764 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.764 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.764 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.764 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.764 INFO:teuthology.orchestra.run.smithi033.stdout: "debug": { 2024-05-15T14:12:12.765 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "debug", 2024-05-15T14:12:12.765 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.765 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.765 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.765 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.765 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.765 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.765 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.766 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable/disable debug options", 2024-05-15T14:12:12.766 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.766 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.766 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.766 INFO:teuthology.orchestra.run.smithi033.stdout: "jwt_token_ttl": { 2024-05-15T14:12:12.766 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "jwt_token_ttl", 2024-05-15T14:12:12.766 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.767 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.767 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.767 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "28800", 2024-05-15T14:12:12.767 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.767 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.767 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.767 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.767 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.767 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.768 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.768 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-05-15T14:12:12.768 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-05-15T14:12:12.768 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.768 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.768 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.768 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.769 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.769 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.769 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.769 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.769 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.769 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.769 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.769 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.770 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.770 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.770 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.770 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.770 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.770 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.770 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.770 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.771 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.771 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.771 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.771 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.771 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.771 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.771 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.771 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.772 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.772 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.772 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.772 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.772 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.772 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.772 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.772 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.773 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.773 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.773 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.773 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.773 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.773 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.773 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.773 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.774 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.774 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.774 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.774 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.774 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.774 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.774 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.775 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.775 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.775 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.775 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.775 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.775 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.775 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.775 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.775 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.776 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.776 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.776 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.776 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.776 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.776 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.776 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.776 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.777 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.777 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.777 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.777 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.777 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.777 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.777 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.777 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.778 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.778 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.778 INFO:teuthology.orchestra.run.smithi033.stdout: "motd": { 2024-05-15T14:12:12.778 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "motd", 2024-05-15T14:12:12.778 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.778 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.778 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.778 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.779 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.779 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.779 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.779 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "The message of the day", 2024-05-15T14:12:12.779 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.779 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.779 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.779 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect_resolve_ip_addr": { 2024-05-15T14:12:12.780 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "redirect_resolve_ip_addr", 2024-05-15T14:12:12.780 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.780 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.780 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.780 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.780 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.780 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.780 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.781 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.781 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.781 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.781 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.781 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-05-15T14:12:12.781 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-05-15T14:12:12.781 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.781 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.782 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.782 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-05-15T14:12:12.782 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.782 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.782 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.782 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.782 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.782 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.783 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.783 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-05-15T14:12:12.783 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-05-15T14:12:12.783 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.783 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.783 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.783 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8080", 2024-05-15T14:12:12.784 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.784 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.784 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.784 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.784 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.784 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.784 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.784 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.785 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.785 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.785 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.785 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.785 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.785 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.785 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.785 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.786 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.786 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.786 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.786 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.786 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.786 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-05-15T14:12:12.786 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-05-15T14:12:12.786 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.787 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.787 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.787 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.787 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.787 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.787 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.787 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.787 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl_server_port": { 2024-05-15T14:12:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl_server_port", 2024-05-15T14:12:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.789 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8443", 2024-05-15T14:12:12.789 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.789 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.789 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.789 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.789 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.789 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.789 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.790 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-05-15T14:12:12.790 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-05-15T14:12:12.790 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.790 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.790 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.790 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "redirect", 2024-05-15T14:12:12.790 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.790 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect" 2024-05-15T14:12:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-05-15T14:12:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-05-15T14:12:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-05-15T14:12:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-05-15T14:12:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-05-15T14:12:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: "url_prefix": { 2024-05-15T14:12:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url_prefix", 2024-05-15T14:12:12.793 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.794 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.794 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.794 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.794 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.794 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.794 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.794 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "devicehealth", 2024-05-15T14:12:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_monitoring": { 2024-05-15T14:12:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_monitoring", 2024-05-15T14:12:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.796 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.797 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.797 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.797 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "monitor device health metrics", 2024-05-15T14:12:12.797 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.797 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.797 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.797 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.797 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.800 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.800 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.800 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.800 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.800 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.800 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.800 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.803 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.803 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.803 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.803 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.803 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.803 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.803 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.803 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.804 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.805 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.805 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.805 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.805 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.805 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.805 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.805 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.805 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.806 INFO:teuthology.orchestra.run.smithi033.stdout: "mark_out_threshold": { 2024-05-15T14:12:12.806 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mark_out_threshold", 2024-05-15T14:12:12.806 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:12.806 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.806 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.806 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2419200", 2024-05-15T14:12:12.806 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.807 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.807 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.807 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-15T14:12:12.807 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.807 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.807 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.807 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_name": { 2024-05-15T14:12:12.808 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_name", 2024-05-15T14:12:12.808 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.808 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.808 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.808 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "device_health_metrics", 2024-05-15T14:12:12.808 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.808 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.808 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.808 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-15T14:12:12.809 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.809 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.809 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.809 INFO:teuthology.orchestra.run.smithi033.stdout: "retention_period": { 2024-05-15T14:12:12.809 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retention_period", 2024-05-15T14:12:12.809 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:12.809 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.810 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.810 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15552000", 2024-05-15T14:12:12.810 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.810 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.810 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.810 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain device health metrics", 2024-05-15T14:12:12.810 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.810 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_frequency": { 2024-05-15T14:12:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_frequency", 2024-05-15T14:12:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-05-15T14:12:12.811 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.812 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.812 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.812 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-15T14:12:12.812 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.812 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.812 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.812 INFO:teuthology.orchestra.run.smithi033.stdout: "self_heal": { 2024-05-15T14:12:12.813 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "self_heal", 2024-05-15T14:12:12.813 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.813 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.813 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.813 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:12.813 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.813 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.813 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.813 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-15T14:12:12.814 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.814 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.814 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.814 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-15T14:12:12.814 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-15T14:12:12.814 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:12.814 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.815 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.815 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-15T14:12:12.815 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.815 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.815 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.815 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and check device health", 2024-05-15T14:12:12.815 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.815 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.816 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.816 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.816 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.816 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.816 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.817 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.818 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_threshold": { 2024-05-15T14:12:12.818 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_threshold", 2024-05-15T14:12:12.818 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:12.818 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.818 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.818 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7257600", 2024-05-15T14:12:12.819 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.819 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.819 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.819 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-15T14:12:12.819 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.819 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.819 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.819 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.819 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.820 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.820 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "diskprediction_local", 2024-05-15T14:12:12.820 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.820 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.820 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.820 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.820 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.821 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.821 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.821 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.821 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.821 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.821 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.821 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.821 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.822 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.822 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.822 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.822 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.822 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.822 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.822 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.822 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.823 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.823 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.823 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.823 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.823 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.823 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.823 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.823 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.824 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.824 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.824 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.824 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.824 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.824 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.824 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.825 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.825 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.825 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.825 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.825 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.825 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.825 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.825 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.825 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.826 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.826 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.826 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.826 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.826 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.826 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.826 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.826 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.827 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.827 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.827 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.827 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.827 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.827 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.827 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.827 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.828 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.828 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.828 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.828 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.828 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.828 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.828 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.828 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.828 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.829 INFO:teuthology.orchestra.run.smithi033.stdout: "predict_interval": { 2024-05-15T14:12:12.829 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predict_interval", 2024-05-15T14:12:12.829 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.829 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.829 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.829 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-05-15T14:12:12.829 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.830 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.830 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.830 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.830 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.830 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.830 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.830 INFO:teuthology.orchestra.run.smithi033.stdout: "predictor_model": { 2024-05-15T14:12:12.831 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predictor_model", 2024-05-15T14:12:12.831 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.831 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.831 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.831 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "prophetstor", 2024-05-15T14:12:12.831 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.831 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.831 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.831 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.832 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.832 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.832 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.832 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-15T14:12:12.832 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-15T14:12:12.832 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.832 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.832 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.833 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-15T14:12:12.833 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.833 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.833 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.833 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.833 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.833 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.833 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.834 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.834 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.834 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.834 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.834 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.834 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.834 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.834 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.834 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.835 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.835 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.835 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.835 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.835 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.835 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.835 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.836 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.836 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "influx", 2024-05-15T14:12:12.836 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": false, 2024-05-15T14:12:12.836 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "influxdb python module not found", 2024-05-15T14:12:12.836 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.836 INFO:teuthology.orchestra.run.smithi033.stdout: "batch_size": { 2024-05-15T14:12:12.836 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "batch_size", 2024-05-15T14:12:12.836 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.836 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.837 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.837 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5000", 2024-05-15T14:12:12.837 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.837 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.837 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.837 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-15T14:12:12.837 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.837 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.838 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.838 INFO:teuthology.orchestra.run.smithi033.stdout: "database": { 2024-05-15T14:12:12.838 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "database", 2024-05-15T14:12:12.838 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.838 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.838 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.838 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "ceph", 2024-05-15T14:12:12.839 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.839 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.839 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.839 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-15T14:12:12.839 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.839 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.839 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.840 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": { 2024-05-15T14:12:12.840 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hostname", 2024-05-15T14:12:12.840 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.840 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.840 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.840 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.840 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.840 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.841 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.841 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server hostname", 2024-05-15T14:12:12.841 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.841 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.841 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.841 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-15T14:12:12.841 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-15T14:12:12.841 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:12.842 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.842 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.842 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "30", 2024-05-15T14:12:12.842 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "5", 2024-05-15T14:12:12.842 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.842 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.842 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-15T14:12:12.843 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.843 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.843 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.843 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.843 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.843 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.843 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.843 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.843 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.844 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.844 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.844 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.844 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.844 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.844 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.844 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.844 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.845 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.845 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.845 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.845 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.845 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.845 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.845 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.845 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.846 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.846 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.846 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.846 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.846 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.846 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.846 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.846 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.847 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.847 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.847 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.847 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.847 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.847 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.847 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.847 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.848 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.848 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.848 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.848 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.848 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.848 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.848 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.848 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.849 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.849 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.849 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.849 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.849 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.849 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.849 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.849 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.850 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.850 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.850 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.850 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.850 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.850 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.850 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.850 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.851 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.851 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.851 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.851 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.851 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.851 INFO:teuthology.orchestra.run.smithi033.stdout: "password": { 2024-05-15T14:12:12.851 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "password", 2024-05-15T14:12:12.852 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.852 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.852 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.852 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.852 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.852 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.852 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.852 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "password of InfluxDB server user", 2024-05-15T14:12:12.852 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.853 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.853 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.853 INFO:teuthology.orchestra.run.smithi033.stdout: "port": { 2024-05-15T14:12:12.853 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "port", 2024-05-15T14:12:12.853 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.853 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.853 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.854 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8086", 2024-05-15T14:12:12.854 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.854 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.854 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.854 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server port", 2024-05-15T14:12:12.854 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.854 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.854 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.855 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.855 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.855 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.855 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.855 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.855 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.855 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.856 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.856 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.856 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.856 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.856 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.856 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.856 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-05-15T14:12:12.857 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-05-15T14:12:12.857 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.857 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.857 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.857 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "false", 2024-05-15T14:12:12.857 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.857 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.857 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.857 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-15T14:12:12.858 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.858 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.858 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.858 INFO:teuthology.orchestra.run.smithi033.stdout: "threads": { 2024-05-15T14:12:12.858 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threads", 2024-05-15T14:12:12.858 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.858 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.859 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.859 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-15T14:12:12.859 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-15T14:12:12.859 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "32", 2024-05-15T14:12:12.859 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.859 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-15T14:12:12.859 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.859 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.859 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.860 INFO:teuthology.orchestra.run.smithi033.stdout: "username": { 2024-05-15T14:12:12.860 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "username", 2024-05-15T14:12:12.860 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.860 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.860 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.860 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.860 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.861 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.861 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.861 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "username of InfluxDB server user", 2024-05-15T14:12:12.861 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.861 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.861 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.861 INFO:teuthology.orchestra.run.smithi033.stdout: "verify_ssl": { 2024-05-15T14:12:12.862 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "verify_ssl", 2024-05-15T14:12:12.862 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.862 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.862 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.862 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "true", 2024-05-15T14:12:12.862 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.862 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.862 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.862 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-15T14:12:12.863 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.863 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.863 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.863 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.863 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.863 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.863 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "insights", 2024-05-15T14:12:12.864 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.864 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.864 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.864 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.864 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.864 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.864 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.864 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.864 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.865 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.865 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.865 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.865 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.865 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.865 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.865 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.865 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.865 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.866 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.866 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.866 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.866 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.866 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.866 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.867 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.867 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.867 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.867 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.867 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.867 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.867 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.867 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.867 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.869 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.869 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.869 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.869 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.869 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.869 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.869 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.869 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.869 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.870 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.870 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.870 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.870 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.870 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.870 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.870 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.871 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.871 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.871 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.871 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.871 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.871 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.871 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.871 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.872 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.872 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.872 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.872 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.872 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.872 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.872 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.873 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.873 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.873 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.873 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.873 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.873 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.873 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.873 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.873 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.874 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.874 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.874 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.874 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "iostat", 2024-05-15T14:12:12.874 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.874 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.875 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.875 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.875 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.875 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.875 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.875 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.875 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.875 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.875 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.876 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.876 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.876 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.876 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.876 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.876 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.876 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.876 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.876 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.877 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.877 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.877 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.877 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.877 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.877 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.877 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.878 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.878 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.878 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.878 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.878 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.878 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.878 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.878 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.878 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.879 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.879 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.879 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.879 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.879 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.879 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.879 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.880 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.880 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.880 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.880 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.880 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.880 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.880 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.880 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.881 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.881 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.881 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.881 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.881 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.881 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.881 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.881 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.882 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.882 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.882 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.882 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.882 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.882 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.882 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.882 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.883 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.883 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.883 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.883 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.883 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.883 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.883 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.884 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.884 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.884 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.884 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.884 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.884 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.884 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.884 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.885 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.885 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.885 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.885 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "k8sevents", 2024-05-15T14:12:12.885 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.885 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.885 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.885 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_event_retention_days": { 2024-05-15T14:12:12.886 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ceph_event_retention_days", 2024-05-15T14:12:12.886 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.886 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.886 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.886 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7", 2024-05-15T14:12:12.886 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.886 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.886 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.887 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-15T14:12:12.887 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.887 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.887 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.887 INFO:teuthology.orchestra.run.smithi033.stdout: "config_check_secs": { 2024-05-15T14:12:12.887 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_check_secs", 2024-05-15T14:12:12.887 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.888 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.888 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.888 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-15T14:12:12.888 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "10", 2024-05-15T14:12:12.888 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.888 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.888 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-15T14:12:12.888 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.888 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.889 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.889 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.889 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.889 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.889 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.889 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.889 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.890 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.890 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.890 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.890 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.890 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.890 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.890 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.890 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.891 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.891 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.891 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.891 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.891 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.891 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.891 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.891 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.892 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.892 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.892 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.892 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.892 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.892 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.892 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.892 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.892 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.893 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.893 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.893 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.893 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.893 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.893 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.893 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.893 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.894 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.894 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.894 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.894 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.894 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.894 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.894 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.894 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.895 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.895 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.895 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.895 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.895 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.895 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.895 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.895 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.896 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.896 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.896 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.896 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.896 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.896 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.896 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.896 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.897 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.897 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.897 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.897 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.897 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.897 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.897 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.898 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.898 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.898 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.898 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.898 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.898 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.898 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.898 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.898 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.899 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.899 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.899 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.899 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "localpool", 2024-05-15T14:12:12.899 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.899 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.899 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.899 INFO:teuthology.orchestra.run.smithi033.stdout: "failure_domain": { 2024-05-15T14:12:12.900 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "failure_domain", 2024-05-15T14:12:12.900 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.900 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.900 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.900 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "host", 2024-05-15T14:12:12.900 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.900 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.900 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.901 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "failure domain for any created local pool", 2024-05-15T14:12:12.901 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-15T14:12:12.901 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.901 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.901 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.901 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.901 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.901 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.902 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.902 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.902 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.902 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.902 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.902 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.902 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.902 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.903 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.903 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.903 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.903 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.903 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.903 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.903 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.903 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.904 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.904 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.904 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.904 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.904 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.904 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.904 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.904 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.905 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.905 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.905 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.905 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.905 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.905 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.905 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.906 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.906 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.906 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.906 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.906 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.906 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.906 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.906 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.907 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.907 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.907 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.907 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.907 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.907 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.907 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.907 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.908 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.908 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.908 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.908 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.908 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.908 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.908 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.908 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.909 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.909 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.909 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.909 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.909 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.909 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.909 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.909 INFO:teuthology.orchestra.run.smithi033.stdout: "min_size": { 2024-05-15T14:12:12.910 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_size", 2024-05-15T14:12:12.910 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.910 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.910 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.910 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.910 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.910 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.911 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.911 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default min_size for any created local pool", 2024-05-15T14:12:12.911 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-15T14:12:12.911 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.911 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.911 INFO:teuthology.orchestra.run.smithi033.stdout: "num_rep": { 2024-05-15T14:12:12.911 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "num_rep", 2024-05-15T14:12:12.911 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.912 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.912 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.912 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3", 2024-05-15T14:12:12.912 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.912 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.912 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.912 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default replica count for any created local pool", 2024-05-15T14:12:12.912 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.913 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.913 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.913 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num": { 2024-05-15T14:12:12.913 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_num", 2024-05-15T14:12:12.913 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.913 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.913 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.913 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "128", 2024-05-15T14:12:12.914 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.914 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.914 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.914 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default pg_num for any created local pool", 2024-05-15T14:12:12.914 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.914 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.914 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.915 INFO:teuthology.orchestra.run.smithi033.stdout: "prefix": { 2024-05-15T14:12:12.915 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prefix", 2024-05-15T14:12:12.915 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.915 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.915 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.915 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.915 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.915 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.916 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.916 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name prefix for any created local pool", 2024-05-15T14:12:12.916 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.916 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.916 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.916 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.916 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.916 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.917 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.917 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.917 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.917 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.917 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.917 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.917 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.917 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.918 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.918 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.918 INFO:teuthology.orchestra.run.smithi033.stdout: "subtree": { 2024-05-15T14:12:12.918 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "subtree", 2024-05-15T14:12:12.918 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.918 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.918 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.919 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "rack", 2024-05-15T14:12:12.919 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.919 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.919 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.919 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-15T14:12:12.919 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-15T14:12:12.919 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.919 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.920 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.920 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.920 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.920 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mds_autoscaler", 2024-05-15T14:12:12.920 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.920 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.920 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.920 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.921 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.921 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.921 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.921 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.921 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.921 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.921 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.921 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.922 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.922 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.922 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.922 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.922 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.922 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.922 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.922 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.923 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.923 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.923 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.923 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.923 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.923 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.923 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.924 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.924 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.924 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.924 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.924 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.924 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.924 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.924 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.924 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.925 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.925 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.925 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.925 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.925 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.925 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.925 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.926 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.926 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.926 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.926 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.926 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.926 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.926 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.926 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.927 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.927 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.927 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.927 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.927 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.927 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.927 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.928 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.928 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.928 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.928 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.928 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.928 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.928 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.928 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.928 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.929 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.929 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.929 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.929 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.929 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.929 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.929 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.930 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.930 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.930 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.930 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.930 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.930 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.930 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.930 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.931 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.931 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.931 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.931 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirroring", 2024-05-15T14:12:12.931 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.931 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.931 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.932 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.932 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.932 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.932 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.932 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.932 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.932 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.932 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.933 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.933 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.933 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.933 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.933 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.933 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.933 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.933 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.933 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.934 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.934 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.934 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.934 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.934 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.934 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.934 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.935 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.935 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.935 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.935 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.935 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.935 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.935 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.935 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.935 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.936 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.936 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.936 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.936 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.936 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.936 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.936 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.937 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.937 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.937 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.937 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.937 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.937 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.937 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.937 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.937 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.938 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.938 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.938 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.938 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.938 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.938 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.938 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.939 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.939 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.939 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.939 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.939 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.939 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.939 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.939 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.940 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.940 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.940 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.940 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.940 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.940 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.940 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.940 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.941 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.941 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.941 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.941 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.941 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.941 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.941 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.941 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.942 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.942 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "nfs", 2024-05-15T14:12:12.942 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.942 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.942 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.942 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.943 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.943 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.943 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.943 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.943 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.943 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.943 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.943 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.943 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.944 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.944 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.944 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.944 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.944 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.944 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.944 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.944 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.945 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.945 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.945 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.945 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.945 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.945 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.945 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.945 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.946 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.946 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.946 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.946 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.946 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.946 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.946 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.947 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.947 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.947 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.947 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.947 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.947 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.947 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.947 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.948 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.948 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.948 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.948 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.948 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.948 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.948 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.948 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.948 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.949 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.949 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.949 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.949 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.949 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.949 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.949 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.950 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.950 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.950 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.950 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.950 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.950 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.950 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.950 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.950 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.951 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.951 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.951 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.951 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.951 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.951 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.951 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.952 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.952 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.952 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.952 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.952 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.952 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.952 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.952 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.953 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.953 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-05-15T14:12:12.953 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.953 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.953 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.953 INFO:teuthology.orchestra.run.smithi033.stdout: "fail_fs": { 2024-05-15T14:12:12.953 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "fail_fs", 2024-05-15T14:12:12.953 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.954 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.954 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.954 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.954 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.954 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.954 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.954 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-15T14:12:12.954 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.955 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.955 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.955 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.955 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.955 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.955 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.955 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.956 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.956 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.956 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.956 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.956 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.956 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.956 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.956 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.956 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.957 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.957 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.957 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.957 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.957 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.957 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.957 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.958 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.958 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.958 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.958 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.958 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.958 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.958 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.958 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.959 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.959 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.959 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.959 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.959 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.959 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.959 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.959 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.960 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.960 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.960 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.960 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.960 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.960 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.960 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.960 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.961 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.961 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.961 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.961 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.961 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.961 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.961 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.961 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.962 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.962 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.962 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.962 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.962 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.962 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.962 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.962 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.963 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.963 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.963 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.963 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.963 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.963 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator": { 2024-05-15T14:12:12.963 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-05-15T14:12:12.964 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.964 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.964 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.964 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.964 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.964 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.964 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.964 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm", 2024-05-15T14:12:12.964 INFO:teuthology.orchestra.run.smithi033.stdout: "rook", 2024-05-15T14:12:12.965 INFO:teuthology.orchestra.run.smithi033.stdout: "test_orchestrator" 2024-05-15T14:12:12.965 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.965 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Orchestrator backend", 2024-05-15T14:12:12.965 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.965 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.965 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.965 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.966 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.966 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.966 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.966 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.966 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.966 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.966 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.966 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.967 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.967 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.967 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.967 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.967 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.967 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.967 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_perf_query", 2024-05-15T14:12:12.968 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.968 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.968 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.968 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.968 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.968 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.968 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.968 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.968 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.969 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.969 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.969 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.969 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.969 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.970 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.970 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.970 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.970 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.970 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.970 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.970 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.970 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.971 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.971 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.971 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.971 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.971 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.971 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.971 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.971 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.972 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.972 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.972 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.972 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.972 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.972 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.972 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.972 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.973 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.973 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.973 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.973 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.973 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.973 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.973 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.973 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.974 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.974 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.974 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.974 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.974 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.974 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.974 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.974 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.975 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.975 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.975 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.975 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.975 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.975 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.975 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.975 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.976 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.976 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.976 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.976 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.976 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.976 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.976 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.977 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.977 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.977 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.977 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.977 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.977 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.977 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.977 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.978 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.978 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.978 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.978 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.978 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.978 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.978 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.979 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_support", 2024-05-15T14:12:12.979 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.979 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.979 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.979 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.979 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.979 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.980 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.980 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.980 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.980 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.980 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.980 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.980 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.980 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.980 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.981 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.981 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.981 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.981 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.981 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.981 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.981 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.981 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.982 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.982 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.982 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.982 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.982 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.982 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.982 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.982 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.983 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.983 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.983 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.983 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.983 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.983 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.983 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.984 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.984 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.984 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.984 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.984 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.984 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.984 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.984 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.985 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.985 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.985 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.985 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.985 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.985 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.985 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.985 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.986 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.986 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.986 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.986 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.986 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.986 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.986 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.986 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.987 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.987 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.987 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.987 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.987 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.987 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.987 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.988 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:12.988 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:12.988 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:12.988 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:12.988 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.988 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:12.988 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.988 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.988 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.989 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.989 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.989 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.989 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.989 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.989 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:12.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.989 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:12.990 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_autoscaler", 2024-05-15T14:12:12.990 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:12.990 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:12.990 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:12.990 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:12.990 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:12.990 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.990 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.991 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.991 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:12.991 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.991 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.991 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.991 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.991 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.991 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.991 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.992 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.992 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.992 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.992 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.992 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.992 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.992 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.992 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:12.993 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:12.993 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.993 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.993 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.993 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.993 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.993 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.993 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.993 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.994 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.994 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.994 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.994 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:12.994 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:12.994 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:12.994 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.995 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.995 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:12.995 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.995 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.995 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:12.995 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:12.995 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:12.995 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:12.996 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:12.996 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:12.996 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:12.996 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:12.996 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.996 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.996 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.996 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.997 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:12.997 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:12.997 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:12.997 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.997 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:12.997 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:12.997 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.997 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.997 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.998 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.998 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:12.998 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:12.998 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:12.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:12.998 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-15T14:12:12.998 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-15T14:12:12.998 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:12.999 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:12.999 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:12.999 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-15T14:12:12.999 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:12.999 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:12.999 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:12.999 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:12.999 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.000 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.000 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.000 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.000 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.000 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.000 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.000 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.001 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.001 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.001 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.001 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.001 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.001 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.001 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.001 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.002 INFO:teuthology.orchestra.run.smithi033.stdout: "threshold": { 2024-05-15T14:12:13.002 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threshold", 2024-05-15T14:12:13.002 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-15T14:12:13.002 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.002 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.002 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-05-15T14:12:13.002 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1.0", 2024-05-15T14:12:13.003 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.003 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.003 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "scaling threshold", 2024-05-15T14:12:13.003 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-15T14:12:13.003 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.003 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.003 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.003 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.004 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.004 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "progress", 2024-05-15T14:12:13.004 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.004 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.004 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.004 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_pg_recovery_event": { 2024-05-15T14:12:13.004 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_pg_recovery_event", 2024-05-15T14:12:13.004 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.005 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.005 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.005 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.005 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.005 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.005 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.005 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-15T14:12:13.006 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.006 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.006 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.006 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-05-15T14:12:13.006 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-05-15T14:12:13.006 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.006 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.007 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.007 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.007 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.007 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.007 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.007 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.007 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.007 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.007 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.008 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.008 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.008 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.008 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.008 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.008 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.008 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.009 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.009 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.009 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.009 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.009 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.009 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.009 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.009 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.009 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.010 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.010 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.010 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.010 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.010 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.010 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.010 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.011 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.011 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.011 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.011 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.011 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.011 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.011 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.011 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.012 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.012 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.012 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.012 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.012 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.012 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.012 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.012 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.013 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.013 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.013 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.013 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.013 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.013 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.013 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.013 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.014 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.014 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.014 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.014 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.014 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.014 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.014 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.015 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.015 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.015 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.015 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.015 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.015 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.015 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.015 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.015 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.016 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.016 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.016 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.016 INFO:teuthology.orchestra.run.smithi033.stdout: "max_completed_events": { 2024-05-15T14:12:13.016 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_completed_events", 2024-05-15T14:12:13.016 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.016 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.017 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.017 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "50", 2024-05-15T14:12:13.017 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.017 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.017 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.017 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "number of past completed events to remember", 2024-05-15T14:12:13.017 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.017 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.017 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.018 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-15T14:12:13.018 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-15T14:12:13.018 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:13.018 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.018 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.018 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-15T14:12:13.018 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.019 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.019 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.019 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long the module is going to sleep", 2024-05-15T14:12:13.019 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.019 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.019 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.019 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.020 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.020 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.020 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.020 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.020 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.020 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.020 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.020 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.021 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.021 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.021 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.021 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.021 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.021 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.021 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus", 2024-05-15T14:12:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: "cache": { 2024-05-15T14:12:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cache", 2024-05-15T14:12:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.023 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.023 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.023 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.023 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.023 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.023 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.023 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.023 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.024 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.024 INFO:teuthology.orchestra.run.smithi033.stdout: "exclude_perf_counters": { 2024-05-15T14:12:13.024 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "exclude_perf_counters", 2024-05-15T14:12:13.024 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.024 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.024 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.024 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.024 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.025 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.025 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.025 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-15T14:12:13.025 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-15T14:12:13.025 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.025 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.025 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.026 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.026 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.026 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.026 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.026 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.026 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.026 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.026 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.026 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.027 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.027 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.027 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.027 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.027 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.027 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.027 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.027 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.028 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.028 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.028 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.028 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.028 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.028 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.028 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.029 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.029 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.029 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.029 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.029 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.029 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.029 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.029 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.030 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.030 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.030 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.030 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.030 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.030 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.030 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.030 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.031 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.031 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.031 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.031 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.031 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.031 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.031 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.031 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.031 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.032 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.032 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.032 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.032 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.032 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.032 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.032 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.033 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.033 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.033 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.033 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.033 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.033 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.033 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.033 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.033 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.034 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools": { 2024-05-15T14:12:13.034 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools", 2024-05-15T14:12:13.034 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.034 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.034 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.034 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.034 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.035 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.035 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.035 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.035 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.035 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.035 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.035 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-15T14:12:13.035 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-15T14:12:13.036 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.036 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.036 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.036 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "300", 2024-05-15T14:12:13.036 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.036 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.036 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.036 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.036 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.037 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.037 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.037 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_interval": { 2024-05-15T14:12:13.037 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_interval", 2024-05-15T14:12:13.037 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-15T14:12:13.037 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.037 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.038 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15.0", 2024-05-15T14:12:13.038 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.038 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.038 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.038 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.038 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.038 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.038 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.039 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-05-15T14:12:13.039 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-05-15T14:12:13.039 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.039 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.039 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.039 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-05-15T14:12:13.039 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.040 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.040 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.040 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-15T14:12:13.040 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.040 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.040 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.040 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-05-15T14:12:13.040 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-05-15T14:12:13.041 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.041 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.041 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.041 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "9283", 2024-05-15T14:12:13.041 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.041 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.041 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.041 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-15T14:12:13.041 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.042 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.042 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.042 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.042 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.042 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.042 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.042 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.043 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.043 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.043 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.043 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.043 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.043 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.043 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.043 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.044 INFO:teuthology.orchestra.run.smithi033.stdout: "stale_cache_strategy": { 2024-05-15T14:12:13.044 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stale_cache_strategy", 2024-05-15T14:12:13.044 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.044 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.044 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.044 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "log", 2024-05-15T14:12:13.044 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.044 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.045 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.045 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.045 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.045 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.045 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.045 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-05-15T14:12:13.045 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-05-15T14:12:13.046 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.046 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.046 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.046 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "default", 2024-05-15T14:12:13.046 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.046 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.046 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.046 INFO:teuthology.orchestra.run.smithi033.stdout: "default", 2024-05-15T14:12:13.047 INFO:teuthology.orchestra.run.smithi033.stdout: "error" 2024-05-15T14:12:13.047 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.047 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.047 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.047 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.047 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.047 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-05-15T14:12:13.048 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-05-15T14:12:13.048 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.048 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.048 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.048 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-05-15T14:12:13.048 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-05-15T14:12:13.048 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-05-15T14:12:13.048 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.049 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.049 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.049 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.049 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.049 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.049 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.049 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.049 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_support", 2024-05-15T14:12:13.050 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.050 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.050 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.050 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.050 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.050 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.050 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.050 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.051 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.051 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.051 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.051 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.051 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.051 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.051 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.051 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.052 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.052 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.052 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.052 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.052 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.052 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.052 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.052 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.053 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.053 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.053 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.053 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.053 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.053 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.053 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.053 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.054 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.054 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.054 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.054 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.054 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.054 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.054 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.055 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.055 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.055 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.055 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.055 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.055 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.055 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.055 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.055 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.056 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.056 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.056 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.056 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.056 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.056 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.056 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.056 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.057 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.057 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.057 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.057 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.057 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.057 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.057 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.057 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.058 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.058 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.058 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.058 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.058 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.058 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_snap_create": { 2024-05-15T14:12:13.058 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_snap_create", 2024-05-15T14:12:13.059 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.059 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.059 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.059 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-15T14:12:13.059 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.059 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.059 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.059 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.059 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.060 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.060 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.060 INFO:teuthology.orchestra.run.smithi033.stdout: "mirror_snapshot_schedule": { 2024-05-15T14:12:13.060 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirror_snapshot_schedule", 2024-05-15T14:12:13.060 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.060 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.060 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.060 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.061 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.061 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.061 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.061 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.061 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.061 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.061 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.062 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.062 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.062 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.062 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.062 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.062 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.062 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.062 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.063 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.063 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.063 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.063 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.063 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.063 INFO:teuthology.orchestra.run.smithi033.stdout: "trash_purge_schedule": { 2024-05-15T14:12:13.063 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "trash_purge_schedule", 2024-05-15T14:12:13.063 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.064 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.064 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.064 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.064 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.064 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.064 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.064 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.064 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.064 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.065 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.065 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.065 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.065 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.065 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "restful", 2024-05-15T14:12:13.065 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.065 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.066 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.066 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_auth": { 2024-05-15T14:12:13.066 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_auth", 2024-05-15T14:12:13.066 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.066 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.066 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.066 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.066 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.067 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.067 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.067 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.067 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.067 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.067 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.067 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-05-15T14:12:13.067 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-05-15T14:12:13.068 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.068 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.068 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.068 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.068 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.068 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.068 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.068 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.069 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.069 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.069 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.069 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.069 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.069 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.069 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.070 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.070 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.070 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.070 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.070 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.070 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.070 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.070 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.070 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.071 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.071 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.071 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.071 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.071 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.071 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.071 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.072 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.072 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.072 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.072 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.072 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.072 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.072 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.072 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.072 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.073 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.073 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.073 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.073 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.073 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.073 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.073 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.073 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.074 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.074 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.074 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.074 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.074 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.074 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.074 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.074 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.075 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.075 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.075 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.075 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.075 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.075 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.075 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.075 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.076 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.076 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.076 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.076 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.076 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.076 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.076 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.077 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.077 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.077 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.077 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.077 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.077 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.077 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-05-15T14:12:13.077 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-05-15T14:12:13.078 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.078 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.078 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.078 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.078 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.078 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.078 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.078 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.079 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.079 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.079 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.079 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-05-15T14:12:13.079 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-05-15T14:12:13.079 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.079 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.079 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.080 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.080 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.080 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.080 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.080 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.080 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.080 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.080 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.081 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.081 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.081 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.081 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.081 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.081 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.081 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.081 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.082 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.082 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.082 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.082 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.082 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.082 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.082 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.083 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rgw", 2024-05-15T14:12:13.083 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.083 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.083 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.083 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.083 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.083 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.083 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.084 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.084 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.084 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.084 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.084 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.084 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.084 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.084 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.085 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.085 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.085 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.085 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.085 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.085 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.085 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.085 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.086 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.086 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.086 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.086 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.086 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.086 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.086 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.087 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.087 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.087 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.087 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.087 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.087 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.087 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.087 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.088 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.088 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.088 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.088 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.088 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.088 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.088 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.088 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.089 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.089 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.089 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.089 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.089 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.089 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.089 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.089 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.089 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.090 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.090 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.090 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.090 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.090 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.090 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.090 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rook", 2024-05-15T14:12:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.100 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.100 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.100 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.100 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.100 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.100 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.100 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.100 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.101 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.101 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.101 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.101 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.101 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.101 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.101 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.101 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.102 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.102 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.102 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.102 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.102 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.102 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.102 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.102 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.103 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.103 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.103 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.103 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.103 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.103 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.103 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.103 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.103 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.104 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.104 INFO:teuthology.orchestra.run.smithi033.stdout: "storage_class": { 2024-05-15T14:12:13.104 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "storage_class", 2024-05-15T14:12:13.104 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.104 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.104 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.104 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "local", 2024-05-15T14:12:13.105 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.105 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.105 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.105 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-15T14:12:13.105 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.105 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.105 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.105 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.106 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.106 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.106 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "selftest", 2024-05-15T14:12:13.106 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.106 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.106 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.106 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.107 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.107 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.107 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.107 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.107 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.107 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.107 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.107 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.107 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.108 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.108 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.108 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.108 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.108 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.108 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.108 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.108 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.109 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.109 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.109 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.109 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.109 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.109 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.109 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.109 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.110 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.110 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.110 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.110 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.110 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.110 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.110 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.110 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.111 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.111 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.111 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.111 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.111 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.111 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.111 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.111 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.112 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.112 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.112 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.112 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.112 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.112 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.112 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.112 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.112 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.113 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.113 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.113 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.113 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.113 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.113 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.113 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.114 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.114 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.114 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.114 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.114 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.114 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.114 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.114 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.115 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.115 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.115 INFO:teuthology.orchestra.run.smithi033.stdout: "roption1": { 2024-05-15T14:12:13.115 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption1", 2024-05-15T14:12:13.115 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.115 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.115 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.115 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.116 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.116 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.116 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.116 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.116 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.116 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.116 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.116 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.116 INFO:teuthology.orchestra.run.smithi033.stdout: "roption2": { 2024-05-15T14:12:13.117 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption2", 2024-05-15T14:12:13.117 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.117 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.117 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.117 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "xyz", 2024-05-15T14:12:13.117 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.117 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.117 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.118 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.118 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.118 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.118 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.118 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.118 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption1": { 2024-05-15T14:12:13.118 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption1", 2024-05-15T14:12:13.118 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.119 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.119 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.119 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.119 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.119 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.119 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.119 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.119 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.120 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.120 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.120 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.120 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption2": { 2024-05-15T14:12:13.120 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption2", 2024-05-15T14:12:13.120 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.120 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.120 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.121 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.121 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.121 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.121 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.121 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.121 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.121 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.121 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.121 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.122 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption3": { 2024-05-15T14:12:13.122 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption3", 2024-05-15T14:12:13.122 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-15T14:12:13.122 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.122 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.122 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.122 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.122 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.123 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.123 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.123 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.123 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.123 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.123 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.123 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption4": { 2024-05-15T14:12:13.123 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption4", 2024-05-15T14:12:13.124 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.124 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.124 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.124 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.124 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.124 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.124 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.124 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.125 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.125 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.125 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.125 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.125 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption5": { 2024-05-15T14:12:13.125 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption5", 2024-05-15T14:12:13.125 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.125 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.125 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.126 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.126 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.126 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.126 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.126 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.126 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.126 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.126 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.127 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption6": { 2024-05-15T14:12:13.127 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption6", 2024-05-15T14:12:13.127 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.127 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.127 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.127 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.127 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.128 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.128 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.128 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.128 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.128 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.128 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.128 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.128 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption7": { 2024-05-15T14:12:13.128 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption7", 2024-05-15T14:12:13.129 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.129 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.129 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.129 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.129 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-15T14:12:13.129 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "42", 2024-05-15T14:12:13.129 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.129 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.130 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.130 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.130 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.130 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.130 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.130 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.130 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.130 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.130 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.131 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.131 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.131 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.131 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.131 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.131 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.131 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.131 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.132 INFO:teuthology.orchestra.run.smithi033.stdout: "testkey": { 2024-05-15T14:12:13.132 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testkey", 2024-05-15T14:12:13.132 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.132 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.132 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.132 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.132 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.132 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.133 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.133 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.133 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.133 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.133 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.133 INFO:teuthology.orchestra.run.smithi033.stdout: "testlkey": { 2024-05-15T14:12:13.133 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testlkey", 2024-05-15T14:12:13.133 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.134 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.134 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.134 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.134 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.134 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.134 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.134 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.134 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.135 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.135 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.135 INFO:teuthology.orchestra.run.smithi033.stdout: "testnewline": { 2024-05-15T14:12:13.135 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testnewline", 2024-05-15T14:12:13.135 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.135 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.135 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.136 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.136 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.136 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.136 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.136 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.136 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.136 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.136 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.137 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.137 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.137 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.137 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smb", 2024-05-15T14:12:13.137 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.137 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.137 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.137 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.138 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.138 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.138 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.138 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.138 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.138 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.138 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.138 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.139 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.139 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.139 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.139 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.139 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.139 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.139 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.139 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.140 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.140 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.140 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.140 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.140 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.140 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.140 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.140 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.141 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.141 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.141 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.141 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.141 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.141 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.141 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.141 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.142 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.142 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.142 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.142 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.142 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.142 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.142 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.142 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.143 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.143 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.143 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.143 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.143 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.143 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.143 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.143 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.143 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.144 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.144 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.144 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.144 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.144 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.144 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.144 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.145 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.145 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.145 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.145 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.145 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.145 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.145 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.145 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.145 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.146 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.146 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.146 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.146 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.146 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.146 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.146 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.147 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.147 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.147 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.147 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.147 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.147 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.147 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.148 INFO:teuthology.orchestra.run.smithi033.stdout: "update_orchestration": { 2024-05-15T14:12:13.148 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "update_orchestration", 2024-05-15T14:12:13.148 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.148 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.148 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.148 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.148 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.148 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.149 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.149 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-15T14:12:13.149 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.149 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.149 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.149 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.149 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.149 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.150 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snap_schedule", 2024-05-15T14:12:13.150 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.150 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.150 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.150 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_m_granularity": { 2024-05-15T14:12:13.150 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_m_granularity", 2024-05-15T14:12:13.150 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.150 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.151 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.151 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.151 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.151 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.151 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.151 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow minute scheduled snapshots", 2024-05-15T14:12:13.151 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.151 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.152 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.152 INFO:teuthology.orchestra.run.smithi033.stdout: "dump_on_update": { 2024-05-15T14:12:13.152 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dump_on_update", 2024-05-15T14:12:13.152 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.152 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.152 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.152 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.153 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.153 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.153 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.153 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "dump database to debug log on update", 2024-05-15T14:12:13.153 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.153 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.153 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.153 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.154 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.154 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.154 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.154 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.154 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.154 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.154 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.154 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.155 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.155 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.155 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.155 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.155 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.155 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.155 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.155 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.156 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.156 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.156 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.156 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.156 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.156 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.156 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.156 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.157 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.157 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.157 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.157 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.157 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.157 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.157 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.157 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.158 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.158 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.158 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.158 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.158 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.158 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.158 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.158 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.159 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.159 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.159 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.159 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.159 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.159 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.159 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.159 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.160 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.160 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.160 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.160 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.160 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.160 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.160 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.160 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.161 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.161 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.161 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.161 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.161 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.161 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.161 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.161 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.162 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.162 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.162 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.162 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.162 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.162 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.162 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.163 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.163 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.163 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.163 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.163 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.163 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.163 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.163 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.164 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.164 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stats", 2024-05-15T14:12:13.164 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.164 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.164 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.164 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.164 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.165 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.165 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.165 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.165 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.165 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.165 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.165 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.165 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.166 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.166 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.166 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.166 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.166 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.166 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.166 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.166 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.167 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.167 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.167 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.167 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.167 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.167 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.167 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.167 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.168 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.168 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.168 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.168 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.168 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.168 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.168 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.169 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.169 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.169 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.169 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.169 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.169 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.169 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.169 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.169 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.170 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.170 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.170 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.170 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.170 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.170 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.170 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.170 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.171 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.171 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.171 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.171 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.171 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.171 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.171 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.172 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.173 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.173 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.173 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.173 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.174 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.174 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.174 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.174 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.174 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.174 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.174 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.175 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.175 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.175 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.175 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.175 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.175 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.175 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.175 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.175 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.176 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.176 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.176 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.176 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.176 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "status", 2024-05-15T14:12:13.176 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.176 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.177 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.177 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.177 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.177 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.177 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.177 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.177 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.177 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.177 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.178 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.178 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.178 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.178 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.178 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.178 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.178 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.178 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.179 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.179 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.179 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.179 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.179 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.179 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.179 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.179 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.180 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.180 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.180 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.180 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.180 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.180 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.180 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.180 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.181 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.181 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.181 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.181 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.181 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.181 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.181 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.181 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.182 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.182 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.182 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.182 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.182 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.182 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.182 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.182 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.183 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.183 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.183 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.183 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.183 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.183 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.183 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.184 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.184 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.184 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.184 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.184 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.184 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.184 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.184 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.184 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.185 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.185 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.185 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.185 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.185 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.185 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.185 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.186 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.186 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.186 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.186 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.186 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.186 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.186 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.187 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.187 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.187 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.187 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.187 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telegraf", 2024-05-15T14:12:13.187 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.187 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.187 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.188 INFO:teuthology.orchestra.run.smithi033.stdout: "address": { 2024-05-15T14:12:13.188 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "address", 2024-05-15T14:12:13.188 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.188 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.188 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.188 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-15T14:12:13.188 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.188 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.189 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.189 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.189 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.189 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.189 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.189 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-15T14:12:13.189 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-15T14:12:13.189 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:13.190 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.190 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.190 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15", 2024-05-15T14:12:13.190 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.190 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.190 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.190 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.190 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.191 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.191 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.191 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.191 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.191 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.191 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.191 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.192 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.192 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.192 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.192 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.192 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.192 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.192 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.192 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.193 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.193 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.193 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.193 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.193 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.193 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.193 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.193 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.194 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.194 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.194 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.194 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.194 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.194 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.194 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.194 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.195 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.195 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.195 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.195 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.195 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.195 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.195 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.195 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.196 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.196 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.196 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.196 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.196 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.196 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.196 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.196 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.197 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.197 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.197 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.197 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.197 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.197 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.197 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.197 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.198 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.198 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.198 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.198 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.198 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.198 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.198 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.198 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.199 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.199 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.199 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.199 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.199 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.199 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.199 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.199 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.200 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.200 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.200 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.200 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.200 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.200 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.200 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.200 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.201 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.201 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.201 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.201 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.201 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.201 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telemetry", 2024-05-15T14:12:13.201 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.201 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.202 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.202 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_basic": { 2024-05-15T14:12:13.202 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_basic", 2024-05-15T14:12:13.202 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.202 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.202 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.202 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.202 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.203 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.203 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.203 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-15T14:12:13.203 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.203 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.203 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.203 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_crash": { 2024-05-15T14:12:13.203 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_crash", 2024-05-15T14:12:13.204 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.204 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.204 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.204 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.204 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.204 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.204 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.204 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-15T14:12:13.204 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.205 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.205 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.205 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_device": { 2024-05-15T14:12:13.205 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_device", 2024-05-15T14:12:13.205 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.205 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.205 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.206 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.206 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.206 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.206 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.206 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-15T14:12:13.206 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.206 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.206 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.207 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_ident": { 2024-05-15T14:12:13.207 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_ident", 2024-05-15T14:12:13.207 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.207 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.207 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.207 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.207 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.207 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.208 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.208 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-15T14:12:13.208 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.208 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.208 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.208 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_perf": { 2024-05-15T14:12:13.208 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_perf", 2024-05-15T14:12:13.209 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.209 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.209 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.209 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.209 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.209 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.209 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.209 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-15T14:12:13.209 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.210 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.210 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.210 INFO:teuthology.orchestra.run.smithi033.stdout: "contact": { 2024-05-15T14:12:13.210 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "contact", 2024-05-15T14:12:13.210 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.210 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.210 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.211 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.211 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.211 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.211 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.211 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.211 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.211 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.211 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.212 INFO:teuthology.orchestra.run.smithi033.stdout: "description": { 2024-05-15T14:12:13.212 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "description", 2024-05-15T14:12:13.212 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.212 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.212 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.212 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.212 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.212 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.213 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.213 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.213 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.213 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.213 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.213 INFO:teuthology.orchestra.run.smithi033.stdout: "device_url": { 2024-05-15T14:12:13.213 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_url", 2024-05-15T14:12:13.214 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.214 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.214 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.214 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-15T14:12:13.214 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.214 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.214 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.214 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.215 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.215 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.215 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.215 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-05-15T14:12:13.215 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-05-15T14:12:13.215 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.215 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.216 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.216 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.216 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.216 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.216 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.216 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.216 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.216 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.216 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.217 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-15T14:12:13.217 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-15T14:12:13.217 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.217 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.217 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.217 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "24", 2024-05-15T14:12:13.217 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "8", 2024-05-15T14:12:13.218 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.218 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.218 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.218 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.218 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.218 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.218 INFO:teuthology.orchestra.run.smithi033.stdout: "last_opt_revision": { 2024-05-15T14:12:13.219 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "last_opt_revision", 2024-05-15T14:12:13.219 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.219 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.219 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.219 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1", 2024-05-15T14:12:13.219 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.219 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.219 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.220 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.220 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.220 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.220 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.220 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard": { 2024-05-15T14:12:13.220 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard", 2024-05-15T14:12:13.220 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.220 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.221 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.221 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.221 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.221 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.221 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.221 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.221 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.221 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.222 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.222 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard_description": { 2024-05-15T14:12:13.222 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard_description", 2024-05-15T14:12:13.222 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.222 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.222 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.222 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.223 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.223 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.223 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.223 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.223 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.223 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.223 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.224 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.224 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.224 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.224 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.224 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.224 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.224 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.224 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.224 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.225 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.225 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.225 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.225 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.225 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.225 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.225 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.225 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.226 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.226 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.226 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.226 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.226 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.226 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.226 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.227 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.227 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.227 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.227 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.227 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.227 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.227 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.227 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.228 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.228 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.228 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.228 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.228 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.228 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.228 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.228 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.229 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.229 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.229 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.229 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.229 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.229 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.229 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.229 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.230 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.230 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.230 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.230 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.230 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.230 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.230 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.231 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.231 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.231 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.231 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.231 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.231 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.231 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.231 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.232 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.232 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.232 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.232 INFO:teuthology.orchestra.run.smithi033.stdout: "organization": { 2024-05-15T14:12:13.232 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "organization", 2024-05-15T14:12:13.232 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.232 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.233 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.233 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.233 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.233 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.233 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.233 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.233 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.233 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.233 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.234 INFO:teuthology.orchestra.run.smithi033.stdout: "proxy": { 2024-05-15T14:12:13.234 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "proxy", 2024-05-15T14:12:13.234 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.234 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.234 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.234 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.234 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.235 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.235 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.235 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.235 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.235 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.235 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.235 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.236 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.236 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.236 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.236 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.236 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.236 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.236 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.236 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.236 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.237 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.237 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.237 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.237 INFO:teuthology.orchestra.run.smithi033.stdout: "url": { 2024-05-15T14:12:13.237 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url", 2024-05-15T14:12:13.237 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.237 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.238 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.238 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-15T14:12:13.238 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.238 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.238 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.238 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.238 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.238 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.239 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.239 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.239 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.239 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.239 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "test_orchestrator", 2024-05-15T14:12:13.239 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.239 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.240 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.240 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.240 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.240 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.240 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.240 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.240 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.240 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.241 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.241 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.241 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.241 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.241 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.241 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.241 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.241 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.241 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.242 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.242 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.242 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.242 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.242 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.242 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.242 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.243 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.243 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.243 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.243 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.243 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.243 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.243 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.243 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.243 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.244 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.244 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.244 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.244 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.244 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.244 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.244 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.245 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.245 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.245 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.245 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.245 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.245 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.245 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.245 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.245 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.246 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.246 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.246 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.246 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.246 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.246 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.246 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.247 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.247 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.247 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.247 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.247 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.247 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.247 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.247 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.248 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.248 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.248 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.248 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.248 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.248 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.248 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.248 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.249 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.249 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.249 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.249 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.249 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.249 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.249 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.249 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.249 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.250 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.250 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.250 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "volumes", 2024-05-15T14:12:13.250 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.250 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.250 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.250 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.251 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.251 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.251 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.251 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.251 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.251 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.251 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.251 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.251 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.252 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.252 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.252 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.252 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.252 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.252 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.252 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.252 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.253 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.253 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.253 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.253 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.253 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.253 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.253 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.253 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.254 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.254 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.254 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.254 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.254 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.254 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.254 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.255 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.255 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.255 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.255 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.255 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.255 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.255 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.256 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.256 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.256 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.256 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.256 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.256 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.256 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.256 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.256 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.257 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.257 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.257 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.257 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.257 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.257 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.257 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.257 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.258 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.258 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.258 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.258 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.258 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.258 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.258 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.258 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.259 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.259 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_clones": { 2024-05-15T14:12:13.259 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_clones", 2024-05-15T14:12:13.259 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.259 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.259 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.259 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4", 2024-05-15T14:12:13.260 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.260 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.260 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.260 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-15T14:12:13.260 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.260 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.260 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.260 INFO:teuthology.orchestra.run.smithi033.stdout: "periodic_async_work": { 2024-05-15T14:12:13.261 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "periodic_async_work", 2024-05-15T14:12:13.261 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.261 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.261 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.261 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.261 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.261 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.261 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.261 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Periodically check for async work", 2024-05-15T14:12:13.262 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.262 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.262 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.262 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_delay": { 2024-05-15T14:12:13.262 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_delay", 2024-05-15T14:12:13.262 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.262 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.263 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.263 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.263 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.263 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.263 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.263 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-15T14:12:13.263 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.263 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.264 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.264 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_no_wait": { 2024-05-15T14:12:13.264 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_no_wait", 2024-05-15T14:12:13.264 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.264 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.264 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.264 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.264 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.265 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.265 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.265 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-15T14:12:13.265 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.265 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.265 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.265 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.266 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.266 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.266 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.266 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.266 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.266 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.266 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.266 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.267 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.267 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.267 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.267 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.267 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.267 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.267 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.268 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix", 2024-05-15T14:12:13.268 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.268 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.268 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.268 INFO:teuthology.orchestra.run.smithi033.stdout: "discovery_interval": { 2024-05-15T14:12:13.268 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "discovery_interval", 2024-05-15T14:12:13.268 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-15T14:12:13.268 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.268 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.269 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "100", 2024-05-15T14:12:13.269 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.269 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.269 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.269 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.269 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.269 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.269 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.270 INFO:teuthology.orchestra.run.smithi033.stdout: "identifier": { 2024-05-15T14:12:13.270 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "identifier", 2024-05-15T14:12:13.270 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.270 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.270 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.270 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.270 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.270 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.271 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.271 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.271 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.271 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.271 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.271 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-15T14:12:13.271 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-15T14:12:13.272 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:13.272 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.272 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.272 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-15T14:12:13.272 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.272 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.272 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.272 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.272 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.273 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.273 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.273 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.273 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.273 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.273 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.273 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.273 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.274 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.274 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.274 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.274 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.274 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.274 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.274 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.275 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.275 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.275 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.275 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.275 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.275 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.275 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.275 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.276 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.276 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.276 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.276 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.276 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.276 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.276 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.276 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.276 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.277 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.277 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.277 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.277 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.277 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.277 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.278 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.278 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.278 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.278 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.278 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.278 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.278 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.278 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.279 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.279 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.279 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.279 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.279 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.279 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.279 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.279 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.280 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.280 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.280 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.280 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.280 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.280 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.280 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.280 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.281 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.281 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.281 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.281 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.281 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.281 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.281 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.281 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.282 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.282 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.282 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.282 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.282 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.282 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.282 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.282 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.283 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.283 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.283 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.283 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_host": { 2024-05-15T14:12:13.283 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_host", 2024-05-15T14:12:13.283 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.283 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.284 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.284 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.284 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.284 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.284 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.284 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.284 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.284 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.285 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.285 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_port": { 2024-05-15T14:12:13.285 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_port", 2024-05-15T14:12:13.285 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.285 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.285 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.285 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10051", 2024-05-15T14:12:13.285 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.286 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.286 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.286 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.286 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.286 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.286 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.286 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_sender": { 2024-05-15T14:12:13.287 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_sender", 2024-05-15T14:12:13.287 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.287 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.287 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.287 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-15T14:12:13.287 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.287 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.287 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.288 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.288 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.288 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.288 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.288 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.288 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.288 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.288 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-15T14:12:13.288 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.289 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.289 INFO:teuthology.orchestra.run.smithi033.stdout: "modules": [ 2024-05-15T14:12:13.289 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm", 2024-05-15T14:12:13.289 INFO:teuthology.orchestra.run.smithi033.stdout: "dashboard", 2024-05-15T14:12:13.289 INFO:teuthology.orchestra.run.smithi033.stdout: "iostat", 2024-05-15T14:12:13.289 INFO:teuthology.orchestra.run.smithi033.stdout: "nfs", 2024-05-15T14:12:13.289 INFO:teuthology.orchestra.run.smithi033.stdout: "restful", 2024-05-15T14:12:13.289 INFO:teuthology.orchestra.run.smithi033.stdout: "snap_schedule" 2024-05-15T14:12:13.290 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.290 INFO:teuthology.orchestra.run.smithi033.stdout: "available_modules": [ 2024-05-15T14:12:13.290 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.290 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "alerts", 2024-05-15T14:12:13.290 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.290 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.290 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.290 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-15T14:12:13.291 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-15T14:12:13.291 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:13.291 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.291 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.291 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-15T14:12:13.291 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.291 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.291 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.291 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How frequently to reexamine health status", 2024-05-15T14:12:13.292 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.292 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.292 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.292 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.292 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.292 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.292 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.292 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.293 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.293 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.293 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.293 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.293 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.293 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.293 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.293 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.294 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.294 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.294 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.294 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.294 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.294 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.294 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.295 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.295 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.295 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.295 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.295 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.295 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.295 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.295 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.295 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.296 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.296 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.296 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.296 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.296 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.296 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.296 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.297 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.297 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.297 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.297 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.297 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.297 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.297 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.297 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.297 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.298 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.298 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.298 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.298 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.298 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.298 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.298 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.298 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.299 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.299 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.299 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.299 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.299 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.299 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.299 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.299 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.300 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.300 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.300 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.300 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.300 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.300 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_destination": { 2024-05-15T14:12:13.300 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_destination", 2024-05-15T14:12:13.301 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.301 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.301 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.301 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.301 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.301 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.301 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.301 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email address to send alerts to", 2024-05-15T14:12:13.301 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.302 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.302 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.302 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_from_name": { 2024-05-15T14:12:13.302 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_from_name", 2024-05-15T14:12:13.302 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.302 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.303 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.303 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "Ceph", 2024-05-15T14:12:13.303 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.303 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.303 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.303 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email From: name", 2024-05-15T14:12:13.303 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.303 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.303 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.304 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_host": { 2024-05-15T14:12:13.304 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_host", 2024-05-15T14:12:13.304 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.304 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.304 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.304 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.304 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.305 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.305 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.305 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP server", 2024-05-15T14:12:13.305 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.305 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.305 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.305 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_password": { 2024-05-15T14:12:13.306 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_password", 2024-05-15T14:12:13.306 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.306 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.306 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.306 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.306 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.306 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.306 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.306 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Password to authenticate with", 2024-05-15T14:12:13.307 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.307 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.307 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.307 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_port": { 2024-05-15T14:12:13.307 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_port", 2024-05-15T14:12:13.307 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.307 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.308 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.308 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "465", 2024-05-15T14:12:13.308 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.308 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.308 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.308 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP port", 2024-05-15T14:12:13.308 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.308 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.309 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.309 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_sender": { 2024-05-15T14:12:13.309 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_sender", 2024-05-15T14:12:13.309 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.309 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.309 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.309 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.310 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.310 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.310 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.310 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP envelope sender", 2024-05-15T14:12:13.310 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.310 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.310 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.311 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_ssl": { 2024-05-15T14:12:13.311 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_ssl", 2024-05-15T14:12:13.311 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.311 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.311 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.311 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.311 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.311 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.312 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.312 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-15T14:12:13.312 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.312 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.312 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.312 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_user": { 2024-05-15T14:12:13.312 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_user", 2024-05-15T14:12:13.313 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.313 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.313 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.313 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.313 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.313 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.313 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.313 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "User to authenticate as", 2024-05-15T14:12:13.313 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.314 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.314 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.314 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.314 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.314 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.314 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.315 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.315 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.315 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.315 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.315 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.315 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.315 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.315 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.316 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.316 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.316 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.316 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.316 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "balancer", 2024-05-15T14:12:13.316 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.316 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.316 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.317 INFO:teuthology.orchestra.run.smithi033.stdout: "active": { 2024-05-15T14:12:13.317 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "active", 2024-05-15T14:12:13.317 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.317 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.317 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.317 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.317 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.317 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.318 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.318 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically balance PGs across cluster", 2024-05-15T14:12:13.318 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.318 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.318 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.318 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_time": { 2024-05-15T14:12:13.318 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_time", 2024-05-15T14:12:13.319 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.319 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.319 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.319 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0000", 2024-05-15T14:12:13.319 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.319 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.319 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.319 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "beginning time of day to automatically balance", 2024-05-15T14:12:13.319 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-15T14:12:13.320 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.320 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.320 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_weekday": { 2024-05-15T14:12:13.320 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_weekday", 2024-05-15T14:12:13.320 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-15T14:12:13.320 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.320 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.321 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.321 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-05-15T14:12:13.321 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-05-15T14:12:13.321 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.321 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-15T14:12:13.321 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-15T14:12:13.321 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.321 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.322 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_max_iterations": { 2024-05-15T14:12:13.322 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_max_iterations", 2024-05-15T14:12:13.322 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-15T14:12:13.322 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.322 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.322 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "25", 2024-05-15T14:12:13.322 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-15T14:12:13.323 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "250", 2024-05-15T14:12:13.323 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.323 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-15T14:12:13.323 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.323 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.323 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.323 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_metrics": { 2024-05-15T14:12:13.323 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_metrics", 2024-05-15T14:12:13.324 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.324 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.324 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.324 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "pgs,objects,bytes", 2024-05-15T14:12:13.324 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.324 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.324 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.324 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-15T14:12:13.325 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-15T14:12:13.325 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.325 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.325 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_step": { 2024-05-15T14:12:13.325 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_step", 2024-05-15T14:12:13.325 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-15T14:12:13.325 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.325 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.326 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.5", 2024-05-15T14:12:13.326 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0.001", 2024-05-15T14:12:13.326 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "0.999", 2024-05-15T14:12:13.326 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.326 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "aggressiveness of optimization", 2024-05-15T14:12:13.326 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-15T14:12:13.326 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.326 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.327 INFO:teuthology.orchestra.run.smithi033.stdout: "end_time": { 2024-05-15T14:12:13.327 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_time", 2024-05-15T14:12:13.327 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.327 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.327 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.327 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2359", 2024-05-15T14:12:13.327 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.327 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.328 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.328 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ending time of day to automatically balance", 2024-05-15T14:12:13.328 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-15T14:12:13.328 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.328 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.328 INFO:teuthology.orchestra.run.smithi033.stdout: "end_weekday": { 2024-05-15T14:12:13.328 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_weekday", 2024-05-15T14:12:13.329 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-15T14:12:13.329 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.329 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.329 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.329 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-05-15T14:12:13.329 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-05-15T14:12:13.329 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.329 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-15T14:12:13.330 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-15T14:12:13.330 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.330 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.330 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.330 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.330 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.330 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.331 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.331 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.331 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.331 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.331 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.331 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.331 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.331 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.331 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.332 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.332 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.332 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.332 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.332 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.332 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.332 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.333 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.333 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.333 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.333 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.333 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.333 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.333 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.333 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.333 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.334 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.334 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.334 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.334 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.334 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.334 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.334 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.335 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.335 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.335 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.335 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.335 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.335 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.335 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.335 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.336 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.336 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.336 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.336 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.336 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.336 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.336 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.336 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.336 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.337 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.337 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.337 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.337 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.337 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.337 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.337 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.337 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.337 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.338 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.338 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.338 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.338 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.338 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.338 INFO:teuthology.orchestra.run.smithi033.stdout: "min_score": { 2024-05-15T14:12:13.338 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_score", 2024-05-15T14:12:13.339 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-15T14:12:13.339 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.339 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.339 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.339 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.339 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.339 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.339 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-15T14:12:13.340 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.340 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.340 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.340 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.340 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-05-15T14:12:13.340 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-05-15T14:12:13.340 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.340 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.340 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.341 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "upmap", 2024-05-15T14:12:13.341 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.341 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.341 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.341 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-compat", 2024-05-15T14:12:13.341 INFO:teuthology.orchestra.run.smithi033.stdout: "none", 2024-05-15T14:12:13.341 INFO:teuthology.orchestra.run.smithi033.stdout: "read", 2024-05-15T14:12:13.341 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap", 2024-05-15T14:12:13.341 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap-read" 2024-05-15T14:12:13.342 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.342 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Balancer mode", 2024-05-15T14:12:13.342 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.342 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.342 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.342 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.342 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_ids": { 2024-05-15T14:12:13.342 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_ids", 2024-05-15T14:12:13.343 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.343 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.343 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.343 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.343 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.343 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.343 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.343 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-15T14:12:13.344 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.344 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.344 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.344 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.344 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-15T14:12:13.344 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-15T14:12:13.344 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:13.344 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.344 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.345 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-15T14:12:13.345 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.345 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.345 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.345 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-15T14:12:13.345 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.345 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.345 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.346 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.346 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.346 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.346 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.346 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.346 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.346 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.346 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.347 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.347 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.347 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.347 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.347 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.347 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.347 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_deviation": { 2024-05-15T14:12:13.348 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_deviation", 2024-05-15T14:12:13.348 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.348 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.348 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.348 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-15T14:12:13.348 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-15T14:12:13.348 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.348 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.348 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-15T14:12:13.349 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-15T14:12:13.349 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.349 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.349 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_optimizations": { 2024-05-15T14:12:13.349 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_optimizations", 2024-05-15T14:12:13.349 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-15T14:12:13.349 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.349 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.350 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-15T14:12:13.350 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.350 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.350 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.350 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-15T14:12:13.350 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.350 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.350 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.351 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.351 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.351 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.351 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm", 2024-05-15T14:12:13.351 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.351 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.351 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.352 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_down_multiplier": { 2024-05-15T14:12:13.352 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_down_multiplier", 2024-05-15T14:12:13.352 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-15T14:12:13.352 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.352 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.352 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-05-15T14:12:13.352 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.352 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.353 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.353 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-15T14:12:13.353 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.353 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.353 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.353 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.353 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_refresh_rate": { 2024-05-15T14:12:13.353 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_refresh_rate", 2024-05-15T14:12:13.354 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:13.354 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.354 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.354 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "20", 2024-05-15T14:12:13.354 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.354 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.354 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.354 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-15T14:12:13.355 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.355 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.355 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.355 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.355 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_starting_port": { 2024-05-15T14:12:13.355 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_starting_port", 2024-05-15T14:12:13.355 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.355 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.355 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.356 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4721", 2024-05-15T14:12:13.356 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.356 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.356 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.356 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-15T14:12:13.356 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.356 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.356 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.357 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_ptrace": { 2024-05-15T14:12:13.357 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_ptrace", 2024-05-15T14:12:13.357 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.357 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.357 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.357 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.357 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.358 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.358 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.358 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-15T14:12:13.358 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-15T14:12:13.358 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.358 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.358 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.358 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_interval": { 2024-05-15T14:12:13.359 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_interval", 2024-05-15T14:12:13.359 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:13.359 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.359 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.359 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-15T14:12:13.359 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.359 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.359 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.360 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-15T14:12:13.360 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.360 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.360 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.360 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_memory_target_ratio": { 2024-05-15T14:12:13.360 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_memory_target_ratio", 2024-05-15T14:12:13.360 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-15T14:12:13.360 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.361 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.361 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.7", 2024-05-15T14:12:13.361 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.361 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.361 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.361 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-15T14:12:13.361 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.361 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.362 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.362 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm_log_destination": { 2024-05-15T14:12:13.362 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm_log_destination", 2024-05-15T14:12:13.362 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.362 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.362 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.362 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.362 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.363 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.363 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.363 INFO:teuthology.orchestra.run.smithi033.stdout: "file", 2024-05-15T14:12:13.363 INFO:teuthology.orchestra.run.smithi033.stdout: "file,syslog", 2024-05-15T14:12:13.363 INFO:teuthology.orchestra.run.smithi033.stdout: "syslog" 2024-05-15T14:12:13.363 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.363 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-15T14:12:13.363 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.364 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.364 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.364 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.364 INFO:teuthology.orchestra.run.smithi033.stdout: "cgroups_split": { 2024-05-15T14:12:13.364 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cgroups_split", 2024-05-15T14:12:13.364 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.364 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.364 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.365 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.365 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.365 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.365 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.365 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-15T14:12:13.365 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.365 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.365 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.366 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.366 INFO:teuthology.orchestra.run.smithi033.stdout: "config_checks_enabled": { 2024-05-15T14:12:13.366 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_checks_enabled", 2024-05-15T14:12:13.366 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.366 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.366 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.366 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.366 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.367 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.367 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.367 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-15T14:12:13.367 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.367 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.367 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.367 INFO:teuthology.orchestra.run.smithi033.stdout: "config_dashboard": { 2024-05-15T14:12:13.367 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_dashboard", 2024-05-15T14:12:13.368 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.368 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.368 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.368 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.368 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.368 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.368 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.368 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-15T14:12:13.369 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.369 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.369 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.369 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.369 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_alertmanager": { 2024-05-15T14:12:13.369 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_alertmanager", 2024-05-15T14:12:13.369 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.369 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.370 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.370 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-15T14:12:13.370 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.370 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.370 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.370 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-15T14:12:13.370 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.370 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.371 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.371 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.371 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_base": { 2024-05-15T14:12:13.371 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_base", 2024-05-15T14:12:13.371 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.371 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.371 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.371 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-15T14:12:13.371 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.372 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.372 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.372 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Container image name, without the tag", 2024-05-15T14:12:13.372 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.372 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.372 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.372 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.372 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_elasticsearch": { 2024-05-15T14:12:13.372 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_elasticsearch", 2024-05-15T14:12:13.373 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.373 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.373 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.373 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-15T14:12:13.373 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.373 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.373 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.373 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "elasticsearch container image", 2024-05-15T14:12:13.374 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.374 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.374 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.374 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.374 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_grafana": { 2024-05-15T14:12:13.374 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_grafana", 2024-05-15T14:12:13.374 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.374 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.374 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.375 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-15T14:12:13.375 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.375 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.375 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.375 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-15T14:12:13.375 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.375 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.375 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.376 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.376 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_haproxy": { 2024-05-15T14:12:13.376 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_haproxy", 2024-05-15T14:12:13.376 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.376 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.376 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.376 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-15T14:12:13.376 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.376 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.377 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.377 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "HAproxy container image", 2024-05-15T14:12:13.377 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.377 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.377 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.377 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.377 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_agent": { 2024-05-15T14:12:13.377 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_agent", 2024-05-15T14:12:13.378 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.378 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.378 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.378 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-15T14:12:13.378 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.378 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.378 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.378 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger agent container image", 2024-05-15T14:12:13.378 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.379 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.379 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.379 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.379 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_collector": { 2024-05-15T14:12:13.379 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_collector", 2024-05-15T14:12:13.379 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.379 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.379 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.380 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-15T14:12:13.380 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.380 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.380 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.380 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger collector container image", 2024-05-15T14:12:13.380 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.380 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.380 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.380 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.381 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_query": { 2024-05-15T14:12:13.381 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_query", 2024-05-15T14:12:13.381 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.381 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.381 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.381 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-15T14:12:13.381 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.381 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.382 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.382 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger query container image", 2024-05-15T14:12:13.382 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.382 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.382 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.382 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.382 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_keepalived": { 2024-05-15T14:12:13.382 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_keepalived", 2024-05-15T14:12:13.383 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.383 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.383 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.383 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-15T14:12:13.383 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.383 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.383 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.383 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Keepalived container image", 2024-05-15T14:12:13.383 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.384 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.384 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.384 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_loki": { 2024-05-15T14:12:13.384 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_loki", 2024-05-15T14:12:13.384 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.384 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.384 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.385 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-15T14:12:13.385 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.385 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.385 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.385 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Loki container image", 2024-05-15T14:12:13.385 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.385 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.385 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.386 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_node_exporter": { 2024-05-15T14:12:13.386 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_node_exporter", 2024-05-15T14:12:13.386 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.386 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.386 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.386 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-15T14:12:13.386 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.386 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.387 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.387 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-15T14:12:13.387 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.387 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.387 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.387 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_nvmeof": { 2024-05-15T14:12:13.387 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_nvmeof", 2024-05-15T14:12:13.387 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.388 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.388 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.388 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-15T14:12:13.388 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.388 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.388 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.388 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Nvme-of container image", 2024-05-15T14:12:13.388 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.389 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.389 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.389 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.389 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_prometheus": { 2024-05-15T14:12:13.389 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_prometheus", 2024-05-15T14:12:13.389 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.389 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.389 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.389 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-15T14:12:13.390 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.390 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.390 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.390 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-15T14:12:13.390 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.390 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.390 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.391 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_promtail": { 2024-05-15T14:12:13.391 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_promtail", 2024-05-15T14:12:13.391 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.391 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.391 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.391 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-15T14:12:13.391 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.392 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.392 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.392 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Promtail container image", 2024-05-15T14:12:13.392 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.392 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.392 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.392 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_samba": { 2024-05-15T14:12:13.392 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_samba", 2024-05-15T14:12:13.393 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.393 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.393 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.393 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-15T14:12:13.393 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.393 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.393 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.393 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Samba/SMB container image", 2024-05-15T14:12:13.394 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.394 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.394 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.394 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_snmp_gateway": { 2024-05-15T14:12:13.394 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_snmp_gateway", 2024-05-15T14:12:13.394 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.394 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.395 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.395 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-15T14:12:13.395 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.395 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.395 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.395 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SNMP Gateway container image", 2024-05-15T14:12:13.395 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.395 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.395 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.396 INFO:teuthology.orchestra.run.smithi033.stdout: "container_init": { 2024-05-15T14:12:13.396 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_init", 2024-05-15T14:12:13.396 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.396 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.396 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.396 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.396 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.397 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.397 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.397 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Run podman/docker with `--init`", 2024-05-15T14:12:13.397 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.397 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.397 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.397 INFO:teuthology.orchestra.run.smithi033.stdout: "daemon_cache_timeout": { 2024-05-15T14:12:13.397 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "daemon_cache_timeout", 2024-05-15T14:12:13.398 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:13.398 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.398 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.398 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-15T14:12:13.398 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.398 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.398 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.398 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-15T14:12:13.399 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.399 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.399 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.399 INFO:teuthology.orchestra.run.smithi033.stdout: "default_cephadm_command_timeout": { 2024-05-15T14:12:13.399 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_cephadm_command_timeout", 2024-05-15T14:12:13.399 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.399 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.399 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.400 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "900", 2024-05-15T14:12:13.400 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.400 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.400 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.400 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-15T14:12:13.400 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.400 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.400 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.401 INFO:teuthology.orchestra.run.smithi033.stdout: "default_registry": { 2024-05-15T14:12:13.401 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_registry", 2024-05-15T14:12:13.401 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.401 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.401 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.401 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io", 2024-05-15T14:12:13.401 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.401 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.402 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.402 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-15T14:12:13.402 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.402 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.402 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.402 INFO:teuthology.orchestra.run.smithi033.stdout: "device_cache_timeout": { 2024-05-15T14:12:13.402 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_cache_timeout", 2024-05-15T14:12:13.403 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:13.403 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.403 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.403 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1800", 2024-05-15T14:12:13.403 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.403 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.403 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.403 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache device inventory", 2024-05-15T14:12:13.403 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.404 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.404 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.404 INFO:teuthology.orchestra.run.smithi033.stdout: "device_enhanced_scan": { 2024-05-15T14:12:13.404 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_enhanced_scan", 2024-05-15T14:12:13.404 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.404 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.404 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.405 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.405 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.405 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.405 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.405 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-15T14:12:13.405 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.405 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.405 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.406 INFO:teuthology.orchestra.run.smithi033.stdout: "facts_cache_timeout": { 2024-05-15T14:12:13.406 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "facts_cache_timeout", 2024-05-15T14:12:13.406 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:13.406 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.406 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.406 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-15T14:12:13.406 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.407 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.407 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.407 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache host facts data", 2024-05-15T14:12:13.407 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.407 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.407 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.407 INFO:teuthology.orchestra.run.smithi033.stdout: "grafana_dashboards_path": { 2024-05-15T14:12:13.407 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "grafana_dashboards_path", 2024-05-15T14:12:13.408 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.408 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.408 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.408 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-15T14:12:13.408 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.408 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.408 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.408 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-15T14:12:13.409 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.409 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.409 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.409 INFO:teuthology.orchestra.run.smithi033.stdout: "host_check_interval": { 2024-05-15T14:12:13.409 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "host_check_interval", 2024-05-15T14:12:13.409 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:13.409 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.409 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.410 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-15T14:12:13.410 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.410 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.410 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.410 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to perform a host check", 2024-05-15T14:12:13.410 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.410 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.410 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.411 INFO:teuthology.orchestra.run.smithi033.stdout: "hw_monitoring": { 2024-05-15T14:12:13.411 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hw_monitoring", 2024-05-15T14:12:13.411 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.411 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.411 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.411 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.411 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.411 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.412 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.412 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-15T14:12:13.412 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.412 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.412 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.412 INFO:teuthology.orchestra.run.smithi033.stdout: "inventory_list_all": { 2024-05-15T14:12:13.412 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "inventory_list_all", 2024-05-15T14:12:13.413 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.413 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.413 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.413 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.413 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.413 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.413 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.413 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-15T14:12:13.413 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.414 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.414 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.414 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.414 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.414 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.414 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.414 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.415 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.415 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.415 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.415 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.415 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.415 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.415 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.415 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.416 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.416 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.416 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.416 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.416 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.416 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.416 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.416 INFO:teuthology.orchestra.run.smithi033.stdout: "log_refresh_metadata": { 2024-05-15T14:12:13.417 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_refresh_metadata", 2024-05-15T14:12:13.417 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.417 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.417 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.417 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.417 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.417 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.417 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.418 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-15T14:12:13.418 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.418 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.418 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.418 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.418 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.418 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.419 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.419 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.419 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.419 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.419 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.419 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.419 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-15T14:12:13.419 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.419 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.420 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.420 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.420 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.420 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.420 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.420 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.420 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.420 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.421 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.421 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.421 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.421 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.421 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.421 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.421 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.421 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.422 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.422 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.422 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.422 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.422 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.422 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.422 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.423 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.423 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.423 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.423 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.423 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.423 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.423 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.423 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.424 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.424 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.424 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.424 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-15T14:12:13.424 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-15T14:12:13.424 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.424 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.425 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.425 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.425 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.425 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.425 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.425 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-15T14:12:13.425 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.425 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.425 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.426 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-15T14:12:13.426 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-15T14:12:13.426 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.426 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.426 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.426 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "*", 2024-05-15T14:12:13.426 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.427 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.427 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.427 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-15T14:12:13.427 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.427 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.427 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.427 INFO:teuthology.orchestra.run.smithi033.stdout: "max_count_per_host": { 2024-05-15T14:12:13.428 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_count_per_host", 2024-05-15T14:12:13.428 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.428 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.428 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.428 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-15T14:12:13.428 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.428 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.428 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.429 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of daemons per service per host", 2024-05-15T14:12:13.429 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.429 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.429 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.429 INFO:teuthology.orchestra.run.smithi033.stdout: "max_osd_draining_count": { 2024-05-15T14:12:13.429 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_osd_draining_count", 2024-05-15T14:12:13.430 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.430 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.430 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.430 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-15T14:12:13.430 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.430 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.430 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.431 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-15T14:12:13.431 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.431 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.431 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.431 INFO:teuthology.orchestra.run.smithi033.stdout: "migration_current": { 2024-05-15T14:12:13.431 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "migration_current", 2024-05-15T14:12:13.431 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.432 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.432 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.432 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.432 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.432 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.432 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.432 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "internal - do not modify", 2024-05-15T14:12:13.432 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.432 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.433 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.433 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.433 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-05-15T14:12:13.433 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-05-15T14:12:13.433 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.433 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.433 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.433 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "root", 2024-05-15T14:12:13.434 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.434 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.434 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.434 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm-package", 2024-05-15T14:12:13.434 INFO:teuthology.orchestra.run.smithi033.stdout: "root" 2024-05-15T14:12:13.434 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.434 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "mode for remote execution of cephadm", 2024-05-15T14:12:13.434 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.435 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.435 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.435 INFO:teuthology.orchestra.run.smithi033.stdout: "oob_default_addr": { 2024-05-15T14:12:13.435 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "oob_default_addr", 2024-05-15T14:12:13.435 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.435 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.435 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.436 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "169.254.1.1", 2024-05-15T14:12:13.436 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.436 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.436 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.436 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-15T14:12:13.436 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.436 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.436 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.437 INFO:teuthology.orchestra.run.smithi033.stdout: "prometheus_alerts_path": { 2024-05-15T14:12:13.437 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus_alerts_path", 2024-05-15T14:12:13.437 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.437 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.437 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.437 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-15T14:12:13.437 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.437 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.438 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.438 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-15T14:12:13.438 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.438 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.438 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.438 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_insecure": { 2024-05-15T14:12:13.438 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_insecure", 2024-05-15T14:12:13.439 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.439 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.439 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.439 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.439 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.439 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.439 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.439 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-15T14:12:13.439 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.440 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.440 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.440 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_password": { 2024-05-15T14:12:13.440 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_password", 2024-05-15T14:12:13.440 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.440 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.440 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.441 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.441 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.441 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.441 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.441 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-15T14:12:13.441 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.441 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.441 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.442 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_url": { 2024-05-15T14:12:13.442 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_url", 2024-05-15T14:12:13.442 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.442 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.442 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.442 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.442 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.442 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.443 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.443 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-15T14:12:13.443 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.443 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.443 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.443 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_username": { 2024-05-15T14:12:13.443 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_username", 2024-05-15T14:12:13.444 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.444 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.444 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.444 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.444 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.444 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.444 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.444 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-15T14:12:13.445 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.445 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.445 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.445 INFO:teuthology.orchestra.run.smithi033.stdout: "secure_monitoring_stack": { 2024-05-15T14:12:13.445 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "secure_monitoring_stack", 2024-05-15T14:12:13.445 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.445 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.445 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.446 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.446 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.446 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.446 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.446 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-15T14:12:13.446 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.446 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.446 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.447 INFO:teuthology.orchestra.run.smithi033.stdout: "service_discovery_port": { 2024-05-15T14:12:13.447 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "service_discovery_port", 2024-05-15T14:12:13.447 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.447 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.447 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.447 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8765", 2024-05-15T14:12:13.447 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.447 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.448 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.448 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "cephadm service discovery port", 2024-05-15T14:12:13.448 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.448 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.448 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.448 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.448 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.449 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.449 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.449 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.449 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.449 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.449 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.449 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.449 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.450 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.450 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.450 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.450 INFO:teuthology.orchestra.run.smithi033.stdout: "ssh_config_file": { 2024-05-15T14:12:13.450 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssh_config_file", 2024-05-15T14:12:13.450 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.450 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.451 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.451 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.451 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.451 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.451 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.451 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-15T14:12:13.451 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.451 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.451 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.452 INFO:teuthology.orchestra.run.smithi033.stdout: "use_agent": { 2024-05-15T14:12:13.452 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_agent", 2024-05-15T14:12:13.452 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.452 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.452 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.452 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.452 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.453 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.453 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.453 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-15T14:12:13.453 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.453 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.453 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.453 INFO:teuthology.orchestra.run.smithi033.stdout: "use_repo_digest": { 2024-05-15T14:12:13.454 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_repo_digest", 2024-05-15T14:12:13.454 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.454 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.454 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.454 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.454 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.454 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.454 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.455 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-15T14:12:13.455 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.455 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.455 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.455 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_failed_host_check": { 2024-05-15T14:12:13.455 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_failed_host_check", 2024-05-15T14:12:13.455 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.456 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.456 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.456 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.456 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.456 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.456 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.456 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if the host check fails", 2024-05-15T14:12:13.456 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.456 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.457 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.457 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_daemons": { 2024-05-15T14:12:13.457 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_daemons", 2024-05-15T14:12:13.457 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.457 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.457 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.457 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.458 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.458 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.458 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.458 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-15T14:12:13.458 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.458 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.458 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.459 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_hosts": { 2024-05-15T14:12:13.459 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_hosts", 2024-05-15T14:12:13.459 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.459 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.459 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.459 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.459 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.459 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.459 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.460 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-15T14:12:13.460 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.460 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.460 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.460 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.460 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.461 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crash", 2024-05-15T14:12:13.461 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.461 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.461 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.461 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.461 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.461 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.461 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.461 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.462 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.462 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.462 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.462 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.462 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.462 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.462 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.462 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.463 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.463 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.463 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.463 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.463 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.463 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.463 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.463 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.464 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.464 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.464 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.464 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.464 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.464 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.464 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.464 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.465 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.465 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.465 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.465 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.465 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.465 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.465 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.465 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.466 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.466 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.466 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.466 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.466 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.466 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.466 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.466 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.467 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.467 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.467 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.467 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.467 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.467 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.467 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.467 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.468 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.468 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.468 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.468 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.468 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.468 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.468 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.468 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.468 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.469 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.469 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.469 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.469 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.469 INFO:teuthology.orchestra.run.smithi033.stdout: "retain_interval": { 2024-05-15T14:12:13.469 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retain_interval", 2024-05-15T14:12:13.469 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:13.470 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.470 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.470 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "31536000", 2024-05-15T14:12:13.470 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.470 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.470 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.470 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-15T14:12:13.470 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.470 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.471 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.471 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.471 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.471 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.471 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.471 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.471 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.471 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.472 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.472 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.472 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.472 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.472 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.472 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.472 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_recent_interval": { 2024-05-15T14:12:13.473 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_recent_interval", 2024-05-15T14:12:13.473 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:13.473 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.473 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.473 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1209600", 2024-05-15T14:12:13.473 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.473 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.473 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.473 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-15T14:12:13.474 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.474 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.474 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.474 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.474 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.474 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.474 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dashboard", 2024-05-15T14:12:13.475 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.475 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.475 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.475 INFO:teuthology.orchestra.run.smithi033.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-15T14:12:13.475 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-15T14:12:13.475 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.475 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.475 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.475 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-15T14:12:13.476 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.476 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.476 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.476 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.476 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.476 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.476 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.476 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_HOST": { 2024-05-15T14:12:13.477 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-15T14:12:13.477 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.477 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.477 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.477 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.477 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.477 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.477 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.478 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.478 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.478 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.478 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.478 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-15T14:12:13.478 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-15T14:12:13.478 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.478 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.479 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.479 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.479 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.479 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.479 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.479 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.479 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.479 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.480 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.480 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_ENABLED": { 2024-05-15T14:12:13.480 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_ENABLED", 2024-05-15T14:12:13.480 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.480 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.480 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.480 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.480 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.481 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.481 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.481 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.481 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.481 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.481 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.481 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-15T14:12:13.482 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-15T14:12:13.482 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.482 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.482 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.482 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.482 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.482 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.482 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.482 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.483 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.483 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.483 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.483 INFO:teuthology.orchestra.run.smithi033.stdout: "ENABLE_BROWSABLE_API": { 2024-05-15T14:12:13.483 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-15T14:12:13.483 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.483 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.484 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.484 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.484 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.484 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.484 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.484 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.484 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.484 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.484 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.485 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-15T14:12:13.485 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-15T14:12:13.485 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.485 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.485 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.485 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.485 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.485 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.486 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.486 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.486 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.486 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.486 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.486 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-15T14:12:13.486 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-15T14:12:13.487 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.487 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.487 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.487 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.487 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.487 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.487 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.487 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.488 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.488 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.488 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.488 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-15T14:12:13.488 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-15T14:12:13.488 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.488 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.488 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.489 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.489 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.489 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.489 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.489 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.489 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.489 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.489 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.490 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-15T14:12:13.490 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-15T14:12:13.490 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.490 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.490 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.490 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.490 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.490 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.491 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.491 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.491 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.491 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.491 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.491 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-15T14:12:13.491 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-15T14:12:13.491 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.492 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.492 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.492 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.492 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.492 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.492 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.492 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.492 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.492 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.493 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.493 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-15T14:12:13.493 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-15T14:12:13.493 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.493 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.493 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.493 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.494 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.494 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.494 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.494 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.494 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.494 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.494 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.494 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-15T14:12:13.495 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-15T14:12:13.495 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.495 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.495 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.495 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.495 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.495 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.495 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.496 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.496 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.496 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.496 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.496 INFO:teuthology.orchestra.run.smithi033.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-15T14:12:13.496 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-15T14:12:13.496 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.496 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.497 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.497 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.497 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.497 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.497 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.497 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.497 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.497 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.497 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.498 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_PASSWORD": { 2024-05-15T14:12:13.498 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-15T14:12:13.498 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.498 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.498 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.498 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-05-15T14:12:13.498 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.499 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.499 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.499 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.499 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.499 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.499 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.499 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-15T14:12:13.499 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-15T14:12:13.500 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.500 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.500 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.500 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.500 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.500 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.500 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.500 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.501 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.501 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.501 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.501 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_URL": { 2024-05-15T14:12:13.501 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_URL", 2024-05-15T14:12:13.501 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.501 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.501 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.502 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.502 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.502 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.502 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.502 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.502 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.502 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.502 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.503 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_USERNAME": { 2024-05-15T14:12:13.503 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-15T14:12:13.503 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.503 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.503 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.503 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-05-15T14:12:13.503 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.504 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.504 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.504 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.504 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.504 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.504 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.504 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-15T14:12:13.504 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-15T14:12:13.505 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.505 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.505 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.505 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.505 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.505 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.505 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.505 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.506 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.506 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.506 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.506 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-15T14:12:13.506 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-15T14:12:13.506 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.506 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.507 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.507 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.507 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.507 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.507 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.507 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.507 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.507 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.508 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.508 INFO:teuthology.orchestra.run.smithi033.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-15T14:12:13.508 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-15T14:12:13.508 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.508 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.508 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.508 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.509 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.509 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.509 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.509 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.509 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.509 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.509 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.509 INFO:teuthology.orchestra.run.smithi033.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-15T14:12:13.510 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-15T14:12:13.510 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.510 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.510 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.510 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.510 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.510 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.510 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.511 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.511 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.511 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.511 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.511 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.511 INFO:teuthology.orchestra.run.smithi033.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-15T14:12:13.511 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-15T14:12:13.511 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.511 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.512 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.512 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "[]", 2024-05-15T14:12:13.512 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.512 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.512 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.512 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.512 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.512 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.512 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.513 INFO:teuthology.orchestra.run.smithi033.stdout: "MULTICLUSTER_CONFIG": { 2024-05-15T14:12:13.513 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-15T14:12:13.513 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.513 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.513 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.513 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "{}", 2024-05-15T14:12:13.513 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.514 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.514 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.514 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.514 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.514 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.514 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.514 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.514 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_HOST": { 2024-05-15T14:12:13.515 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-15T14:12:13.515 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.515 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.515 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.515 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.515 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.515 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.515 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.515 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.516 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.516 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.516 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.516 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-15T14:12:13.516 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-15T14:12:13.516 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.516 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.517 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.517 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.517 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.517 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.517 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.517 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.517 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.517 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.517 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.518 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-15T14:12:13.518 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-15T14:12:13.518 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.518 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.518 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.518 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.518 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.519 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.519 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.519 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.519 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.519 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.519 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.519 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-15T14:12:13.520 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-15T14:12:13.520 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.520 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.520 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.520 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.520 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.520 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.520 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.520 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.521 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.521 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.521 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.521 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-15T14:12:13.521 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-15T14:12:13.521 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.521 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.522 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.522 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.522 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.522 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.522 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.522 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.522 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.522 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.523 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.523 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-15T14:12:13.523 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-15T14:12:13.523 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.523 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.523 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.523 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.523 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.524 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.524 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.524 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.524 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.524 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.524 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.525 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-15T14:12:13.525 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-15T14:12:13.525 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.525 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.525 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.525 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.525 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.525 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.526 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.526 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.526 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.526 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.526 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.526 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-15T14:12:13.526 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-15T14:12:13.526 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.527 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.527 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.527 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.527 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.527 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.527 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.527 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.527 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.528 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.528 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.528 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-15T14:12:13.528 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-15T14:12:13.528 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.528 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.528 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.528 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.529 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.529 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.529 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.529 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.529 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.529 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.529 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.530 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_ENABLED": { 2024-05-15T14:12:13.530 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-15T14:12:13.530 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.530 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.530 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.530 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.530 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.530 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.531 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.531 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.531 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.531 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.531 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.531 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-15T14:12:13.531 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-15T14:12:13.531 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.532 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.532 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.532 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-15T14:12:13.532 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.532 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.532 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.532 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.532 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.533 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.533 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.533 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-15T14:12:13.533 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-15T14:12:13.533 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.533 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.533 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.533 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-15T14:12:13.534 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.534 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.534 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.534 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.534 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.534 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.534 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.535 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-15T14:12:13.535 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-15T14:12:13.535 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.535 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.535 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.535 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8", 2024-05-15T14:12:13.535 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.535 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.536 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.536 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.536 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.536 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.536 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.536 INFO:teuthology.orchestra.run.smithi033.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-15T14:12:13.536 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-15T14:12:13.536 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.537 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.537 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.537 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "45", 2024-05-15T14:12:13.537 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.537 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.537 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.537 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.537 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.538 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.538 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.538 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ACCESS_KEY": { 2024-05-15T14:12:13.538 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-15T14:12:13.538 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.538 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.538 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.539 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.539 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.539 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.539 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.539 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.539 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.539 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.539 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.540 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-15T14:12:13.540 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-15T14:12:13.540 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.540 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.540 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.540 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-05-15T14:12:13.540 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.540 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.541 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.541 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.541 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.541 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.541 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.541 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SECRET_KEY": { 2024-05-15T14:12:13.541 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-15T14:12:13.542 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.542 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.542 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.542 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.542 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.542 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.542 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.542 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.543 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.543 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.543 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.543 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SSL_VERIFY": { 2024-05-15T14:12:13.543 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-15T14:12:13.543 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.543 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.544 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.544 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.544 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.544 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.544 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.544 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.544 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.544 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.544 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.545 INFO:teuthology.orchestra.run.smithi033.stdout: "UNSAFE_TLS_v1_2": { 2024-05-15T14:12:13.545 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-15T14:12:13.545 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.545 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.545 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.545 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.546 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.546 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.546 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.546 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.546 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.546 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.546 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.546 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-15T14:12:13.547 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-15T14:12:13.547 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.547 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.547 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.547 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.547 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.547 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.547 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.548 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.548 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.548 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.548 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.548 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-15T14:12:13.548 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-15T14:12:13.548 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.548 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.549 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.549 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-15T14:12:13.549 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.549 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.549 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.549 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.549 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.549 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.550 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.550 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-15T14:12:13.550 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-15T14:12:13.550 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.550 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.550 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.550 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-15T14:12:13.551 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.551 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.551 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.551 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.551 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.551 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.551 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.551 INFO:teuthology.orchestra.run.smithi033.stdout: "cross_origin_url": { 2024-05-15T14:12:13.552 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cross_origin_url", 2024-05-15T14:12:13.552 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.552 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.552 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.552 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.552 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.552 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.552 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.553 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.553 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.553 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.553 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.553 INFO:teuthology.orchestra.run.smithi033.stdout: "crt_file": { 2024-05-15T14:12:13.553 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crt_file", 2024-05-15T14:12:13.553 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.553 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.554 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.554 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.554 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.554 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.554 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.554 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.554 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.554 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: "debug": { 2024-05-15T14:12:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "debug", 2024-05-15T14:12:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable/disable debug options", 2024-05-15T14:12:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: "jwt_token_ttl": { 2024-05-15T14:12:13.557 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "jwt_token_ttl", 2024-05-15T14:12:13.557 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.557 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.557 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.557 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "28800", 2024-05-15T14:12:13.557 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.557 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.557 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.557 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.558 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.558 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.558 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.558 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-05-15T14:12:13.558 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-05-15T14:12:13.558 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.558 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.559 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.559 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.559 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.559 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.559 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.559 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.559 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.559 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.559 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.560 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.560 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.560 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.560 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.560 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.560 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.560 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.561 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.561 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.561 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.561 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.561 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.561 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.561 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.561 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.561 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.562 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.562 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.562 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.562 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.562 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.562 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.563 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.563 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.563 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.563 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.563 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.563 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.563 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.563 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.563 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.564 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.564 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.564 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.564 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.564 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.564 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.564 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.566 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.566 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.566 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.566 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.566 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.566 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.566 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.568 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.568 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.568 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.568 INFO:teuthology.orchestra.run.smithi033.stdout: "motd": { 2024-05-15T14:12:13.568 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "motd", 2024-05-15T14:12:13.568 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.568 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.569 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.569 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.569 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.569 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.569 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.569 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "The message of the day", 2024-05-15T14:12:13.569 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.569 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.569 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.570 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect_resolve_ip_addr": { 2024-05-15T14:12:13.570 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "redirect_resolve_ip_addr", 2024-05-15T14:12:13.570 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.570 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.570 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.570 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.570 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.571 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.571 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.571 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.571 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.571 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.571 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.571 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-05-15T14:12:13.572 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-05-15T14:12:13.572 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.572 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.572 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.572 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-05-15T14:12:13.572 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.572 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.572 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.572 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-05-15T14:12:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-05-15T14:12:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.574 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.574 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8080", 2024-05-15T14:12:13.574 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.574 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.574 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.574 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.574 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.574 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.575 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.575 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.575 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.575 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.575 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.575 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.575 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.575 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-05-15T14:12:13.577 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-05-15T14:12:13.577 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.577 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.577 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.577 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.577 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.577 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.577 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.577 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.578 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.578 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.578 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.578 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl_server_port": { 2024-05-15T14:12:13.578 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl_server_port", 2024-05-15T14:12:13.578 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.578 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.579 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.579 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8443", 2024-05-15T14:12:13.579 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.579 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.579 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.579 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.579 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.579 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.580 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.580 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-05-15T14:12:13.580 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-05-15T14:12:13.580 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.580 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.580 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.580 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "redirect", 2024-05-15T14:12:13.580 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect" 2024-05-15T14:12:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-05-15T14:12:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-05-15T14:12:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-05-15T14:12:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-05-15T14:12:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-05-15T14:12:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: "url_prefix": { 2024-05-15T14:12:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url_prefix", 2024-05-15T14:12:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.585 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.585 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.585 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.585 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.585 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.585 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.585 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "devicehealth", 2024-05-15T14:12:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_monitoring": { 2024-05-15T14:12:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_monitoring", 2024-05-15T14:12:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "monitor device health metrics", 2024-05-15T14:12:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.589 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.589 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.589 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.589 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.589 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.589 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.589 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.589 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.589 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.591 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.591 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.591 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.591 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.591 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.591 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.591 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.591 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.593 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.593 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.593 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.593 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.593 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.593 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.593 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.593 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.594 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.594 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.594 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.594 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.594 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.594 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.594 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.595 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.595 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.595 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.595 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.595 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.595 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.595 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.595 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.595 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.596 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.596 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.596 INFO:teuthology.orchestra.run.smithi033.stdout: "mark_out_threshold": { 2024-05-15T14:12:13.596 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mark_out_threshold", 2024-05-15T14:12:13.596 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:13.596 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.596 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.597 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2419200", 2024-05-15T14:12:13.597 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.597 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.597 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.597 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-15T14:12:13.597 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.597 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.597 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_name": { 2024-05-15T14:12:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_name", 2024-05-15T14:12:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "device_health_metrics", 2024-05-15T14:12:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-15T14:12:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: "retention_period": { 2024-05-15T14:12:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retention_period", 2024-05-15T14:12:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15552000", 2024-05-15T14:12:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain device health metrics", 2024-05-15T14:12:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_frequency": { 2024-05-15T14:12:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_frequency", 2024-05-15T14:12:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.602 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-05-15T14:12:13.602 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.602 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.602 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.602 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-15T14:12:13.602 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.602 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.602 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.603 INFO:teuthology.orchestra.run.smithi033.stdout: "self_heal": { 2024-05-15T14:12:13.603 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "self_heal", 2024-05-15T14:12:13.603 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.603 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.603 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.603 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.603 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.603 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.604 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.604 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-15T14:12:13.604 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.604 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.604 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.604 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-15T14:12:13.604 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-15T14:12:13.605 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:13.605 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.605 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.605 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-15T14:12:13.605 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.605 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.605 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.605 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and check device health", 2024-05-15T14:12:13.605 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.606 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.606 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.606 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.606 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.606 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.606 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.606 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.607 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.607 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.607 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.607 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.607 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.607 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.607 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.607 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.608 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_threshold": { 2024-05-15T14:12:13.608 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_threshold", 2024-05-15T14:12:13.608 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:13.608 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.608 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.608 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7257600", 2024-05-15T14:12:13.608 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.608 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.609 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.609 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-15T14:12:13.609 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.609 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.609 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.609 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.609 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.610 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.610 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "diskprediction_local", 2024-05-15T14:12:13.610 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.610 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.610 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.610 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.610 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.610 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.611 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.611 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.611 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.611 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.611 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.611 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.611 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.611 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.611 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.612 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.612 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.612 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.612 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.612 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.612 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.612 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.612 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.613 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.613 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.613 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.613 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.613 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.613 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.613 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.613 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.614 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.614 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.614 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.614 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.614 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.614 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.614 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.614 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.615 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.615 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.615 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.615 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.615 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.615 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.615 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.615 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.617 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.617 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.617 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.617 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.617 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.617 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.617 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.618 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.618 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.618 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.618 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.618 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.618 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.618 INFO:teuthology.orchestra.run.smithi033.stdout: "predict_interval": { 2024-05-15T14:12:13.619 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predict_interval", 2024-05-15T14:12:13.619 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.619 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.619 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.619 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-05-15T14:12:13.619 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.619 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.619 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.620 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.620 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.620 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.620 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.620 INFO:teuthology.orchestra.run.smithi033.stdout: "predictor_model": { 2024-05-15T14:12:13.620 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predictor_model", 2024-05-15T14:12:13.620 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.620 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.621 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.621 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "prophetstor", 2024-05-15T14:12:13.621 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.621 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.621 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.621 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.621 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.621 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.622 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.622 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-15T14:12:13.622 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-15T14:12:13.622 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.622 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.622 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.622 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-15T14:12:13.623 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.623 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.623 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.623 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.623 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.623 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.623 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.624 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.624 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.624 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.624 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.624 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.624 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.624 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.624 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.624 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.625 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.625 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.625 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.625 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.625 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.625 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.625 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.626 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "influx", 2024-05-15T14:12:13.626 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": false, 2024-05-15T14:12:13.626 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "influxdb python module not found", 2024-05-15T14:12:13.626 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.626 INFO:teuthology.orchestra.run.smithi033.stdout: "batch_size": { 2024-05-15T14:12:13.626 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "batch_size", 2024-05-15T14:12:13.626 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.626 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.627 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.627 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5000", 2024-05-15T14:12:13.627 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.627 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.627 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.627 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-15T14:12:13.627 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.627 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.627 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.628 INFO:teuthology.orchestra.run.smithi033.stdout: "database": { 2024-05-15T14:12:13.628 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "database", 2024-05-15T14:12:13.628 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.628 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.628 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.628 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "ceph", 2024-05-15T14:12:13.628 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.629 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.629 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.629 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-15T14:12:13.629 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.629 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.629 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.629 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": { 2024-05-15T14:12:13.629 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hostname", 2024-05-15T14:12:13.630 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.630 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.630 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.630 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.630 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.630 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.630 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.631 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server hostname", 2024-05-15T14:12:13.631 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.631 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.631 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.631 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-15T14:12:13.631 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-15T14:12:13.631 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:13.632 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.632 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.632 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "30", 2024-05-15T14:12:13.632 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "5", 2024-05-15T14:12:13.632 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.632 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.632 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-15T14:12:13.632 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.632 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.633 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.633 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.633 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.633 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.633 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.633 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.633 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.634 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.634 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.634 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.634 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.634 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.634 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.634 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.634 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.635 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.635 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.635 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.635 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.635 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.635 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.635 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.636 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.636 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.636 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.636 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.636 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.636 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.636 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.636 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.636 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.637 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.637 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.637 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.637 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.637 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.637 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.637 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.637 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.638 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.638 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.638 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.638 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.638 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.638 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.638 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.638 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.639 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.639 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.639 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.639 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.639 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.639 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.639 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.640 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.640 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.640 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.640 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.640 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.640 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.640 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.640 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.640 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.641 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.641 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.641 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.641 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.641 INFO:teuthology.orchestra.run.smithi033.stdout: "password": { 2024-05-15T14:12:13.641 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "password", 2024-05-15T14:12:13.641 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.642 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.642 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.642 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.642 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.642 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.642 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.642 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "password of InfluxDB server user", 2024-05-15T14:12:13.642 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.642 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.643 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.643 INFO:teuthology.orchestra.run.smithi033.stdout: "port": { 2024-05-15T14:12:13.643 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "port", 2024-05-15T14:12:13.643 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.643 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.643 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.643 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8086", 2024-05-15T14:12:13.644 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.644 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.644 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.644 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server port", 2024-05-15T14:12:13.644 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.644 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.644 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.645 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.645 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.645 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.645 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.645 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.645 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.645 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.645 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.646 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.646 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.646 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.646 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.646 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.646 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-05-15T14:12:13.646 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-05-15T14:12:13.647 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.647 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.647 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.647 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "false", 2024-05-15T14:12:13.647 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.647 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.647 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.647 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-15T14:12:13.647 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.648 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.648 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.648 INFO:teuthology.orchestra.run.smithi033.stdout: "threads": { 2024-05-15T14:12:13.648 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threads", 2024-05-15T14:12:13.648 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.648 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.648 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.649 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-15T14:12:13.649 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-15T14:12:13.649 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "32", 2024-05-15T14:12:13.649 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.649 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-15T14:12:13.649 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.649 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.649 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.650 INFO:teuthology.orchestra.run.smithi033.stdout: "username": { 2024-05-15T14:12:13.650 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "username", 2024-05-15T14:12:13.650 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.650 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.650 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.650 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.650 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.650 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.651 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.651 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "username of InfluxDB server user", 2024-05-15T14:12:13.651 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.651 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.651 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.651 INFO:teuthology.orchestra.run.smithi033.stdout: "verify_ssl": { 2024-05-15T14:12:13.651 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "verify_ssl", 2024-05-15T14:12:13.652 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.652 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.652 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.652 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "true", 2024-05-15T14:12:13.652 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.652 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.652 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.652 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-15T14:12:13.652 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.653 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.653 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.653 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.653 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.653 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.653 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "insights", 2024-05-15T14:12:13.653 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.654 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.654 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.654 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.654 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.654 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.654 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.654 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.654 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.654 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.655 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.655 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.655 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.655 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.655 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.655 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.655 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.655 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.655 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.656 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.656 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.656 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.656 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.656 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.656 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.656 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.657 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.657 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.657 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.657 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.657 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.657 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.657 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.657 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.657 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.658 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.658 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.658 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.658 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.658 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.658 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.658 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.659 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.659 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.659 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.659 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.659 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.659 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.659 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.659 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.659 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.660 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.660 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.660 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.660 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.660 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.660 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.660 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.661 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.661 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.661 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.661 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.661 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.661 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.661 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.661 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.661 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.662 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.662 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.662 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.662 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.662 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.662 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.662 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.663 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.663 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.663 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.663 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.663 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.663 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.663 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.663 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.663 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.664 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.664 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.664 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "iostat", 2024-05-15T14:12:13.664 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.664 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.664 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.664 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.664 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.665 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.665 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.665 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.665 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.665 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.665 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.665 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.665 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.666 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.666 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.666 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.666 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.666 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.666 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.666 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.666 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.666 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.667 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.667 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.667 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.667 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.667 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.667 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.667 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.668 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.668 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.668 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.668 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.668 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.668 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.668 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.669 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.669 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.669 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.669 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.669 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.669 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.669 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.669 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.669 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.670 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.670 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.670 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.670 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.670 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.670 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.670 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.670 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.671 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.671 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.671 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.671 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.671 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.671 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.671 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.672 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.672 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.672 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.672 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.672 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.672 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.672 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.672 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.673 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.673 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.673 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.673 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.673 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.673 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.673 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.673 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.674 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.674 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.674 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.674 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.674 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.674 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.674 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.674 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.675 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.675 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "k8sevents", 2024-05-15T14:12:13.675 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.675 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.675 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.675 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_event_retention_days": { 2024-05-15T14:12:13.675 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ceph_event_retention_days", 2024-05-15T14:12:13.676 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.676 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.676 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.676 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7", 2024-05-15T14:12:13.676 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.676 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.676 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.676 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-15T14:12:13.676 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.677 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.677 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.677 INFO:teuthology.orchestra.run.smithi033.stdout: "config_check_secs": { 2024-05-15T14:12:13.677 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_check_secs", 2024-05-15T14:12:13.677 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.677 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.677 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.678 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-15T14:12:13.678 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "10", 2024-05-15T14:12:13.678 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.678 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.678 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-15T14:12:13.678 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.678 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.678 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.679 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.679 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.679 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.679 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.679 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.679 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.679 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.679 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.680 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.680 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.680 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.680 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.680 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.680 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.680 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.680 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.680 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.681 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.681 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.681 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.681 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.681 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.681 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.681 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.682 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.682 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.682 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.682 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.682 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.682 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.682 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.682 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.684 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.684 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.684 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.684 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.684 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.684 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.684 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.684 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.685 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.685 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.685 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.685 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.685 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.685 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.685 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.685 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.686 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.686 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.686 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.686 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.686 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.686 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.686 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.686 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.687 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.687 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.687 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.687 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.687 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.687 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.687 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.688 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.688 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.688 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.688 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.688 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.688 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.688 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.688 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.688 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.689 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.689 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.689 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "localpool", 2024-05-15T14:12:13.689 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.689 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.689 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.689 INFO:teuthology.orchestra.run.smithi033.stdout: "failure_domain": { 2024-05-15T14:12:13.690 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "failure_domain", 2024-05-15T14:12:13.690 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.690 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.690 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.690 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "host", 2024-05-15T14:12:13.690 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.690 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.690 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.691 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "failure domain for any created local pool", 2024-05-15T14:12:13.691 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-15T14:12:13.691 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.691 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.691 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.691 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.691 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.691 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.692 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.692 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.692 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.692 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.692 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.692 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.692 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.692 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.692 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.693 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.693 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.693 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.693 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.693 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.693 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.693 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.694 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.694 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.694 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.694 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.694 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.694 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.694 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.694 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.695 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.695 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.695 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.695 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.695 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.695 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.695 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.696 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.696 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.696 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.696 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.696 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.696 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.696 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.696 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.696 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.697 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.697 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.697 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.697 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.697 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.697 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.697 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.697 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.698 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.698 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.698 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.698 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.698 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.698 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.698 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.698 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.699 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.699 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.699 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.699 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.699 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.699 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.699 INFO:teuthology.orchestra.run.smithi033.stdout: "min_size": { 2024-05-15T14:12:13.699 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_size", 2024-05-15T14:12:13.700 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.700 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.700 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.700 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.700 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.700 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.700 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.700 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default min_size for any created local pool", 2024-05-15T14:12:13.701 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-15T14:12:13.701 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.701 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.701 INFO:teuthology.orchestra.run.smithi033.stdout: "num_rep": { 2024-05-15T14:12:13.701 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "num_rep", 2024-05-15T14:12:13.701 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.701 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.701 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.702 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3", 2024-05-15T14:12:13.702 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.702 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.702 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.702 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default replica count for any created local pool", 2024-05-15T14:12:13.702 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.702 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.702 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.703 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num": { 2024-05-15T14:12:13.703 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_num", 2024-05-15T14:12:13.703 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.703 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.703 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.703 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "128", 2024-05-15T14:12:13.703 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.703 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.704 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.704 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default pg_num for any created local pool", 2024-05-15T14:12:13.704 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.704 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.704 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.704 INFO:teuthology.orchestra.run.smithi033.stdout: "prefix": { 2024-05-15T14:12:13.704 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prefix", 2024-05-15T14:12:13.705 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.705 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.705 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.705 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.705 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.705 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.705 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.705 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name prefix for any created local pool", 2024-05-15T14:12:13.705 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.706 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.706 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.706 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.706 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.706 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.706 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.706 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.707 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.707 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.707 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.707 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.707 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.707 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.707 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.707 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.708 INFO:teuthology.orchestra.run.smithi033.stdout: "subtree": { 2024-05-15T14:12:13.708 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "subtree", 2024-05-15T14:12:13.708 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.708 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.708 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.708 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "rack", 2024-05-15T14:12:13.708 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.708 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.709 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.709 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-15T14:12:13.709 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-15T14:12:13.709 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.709 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.709 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.709 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.710 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.710 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mds_autoscaler", 2024-05-15T14:12:13.710 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.710 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.710 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.710 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.710 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.710 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.711 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.711 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.711 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.711 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.711 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.711 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.711 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.711 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.711 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.712 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.712 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.712 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.712 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.712 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.712 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.712 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.712 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.713 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.713 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.713 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.713 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.713 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.713 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.713 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.713 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.714 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.714 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.714 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.714 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.714 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.714 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.714 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.715 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.715 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.715 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.715 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.715 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.715 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.715 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.715 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.716 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.716 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.716 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.716 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.716 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.716 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.716 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.716 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.716 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.717 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.717 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.717 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.717 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.717 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.717 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.717 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.718 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.718 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.718 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.718 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.718 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.718 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.718 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.718 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.719 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.719 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.719 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.719 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.719 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.719 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.719 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.719 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.720 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.720 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.720 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.720 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.720 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.720 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.720 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.720 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirroring", 2024-05-15T14:12:13.721 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.721 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.721 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.721 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.721 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.721 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.721 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.721 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.722 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.722 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.722 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.722 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.722 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.722 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.722 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.722 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.723 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.723 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.723 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.723 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.723 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.723 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.723 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.723 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.724 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.724 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.724 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.724 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.724 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.724 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.724 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.724 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.725 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.725 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.725 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.725 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.725 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.725 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.725 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.725 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.726 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.726 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.726 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.726 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.726 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.726 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.726 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.726 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.727 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.727 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.727 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.727 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.727 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.727 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.727 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.727 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.728 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.728 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.728 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.728 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.728 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.728 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.728 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.728 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.729 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.729 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.729 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.729 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.729 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.729 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.729 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.730 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.730 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.730 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.730 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.730 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.730 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.730 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.730 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.730 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.731 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.731 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.731 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.731 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.731 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.731 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "nfs", 2024-05-15T14:12:13.731 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.732 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.732 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.732 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.732 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.732 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.732 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.732 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.732 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.733 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.733 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.733 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.733 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.733 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.733 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.733 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.733 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.733 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.734 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.734 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.734 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.734 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.734 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.734 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.734 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.735 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.735 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.735 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.735 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.735 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.735 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.735 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.735 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.736 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.736 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.736 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.736 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.736 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.736 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.736 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.736 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.737 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.737 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.737 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.737 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.737 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.737 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.737 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.737 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.738 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.738 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.738 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.738 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.738 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.738 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.738 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.738 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.739 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.739 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.739 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.739 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.739 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.739 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.739 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.740 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.740 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.740 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.740 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.740 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.740 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.740 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.740 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.741 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.741 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.741 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.741 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.741 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.741 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.741 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.741 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.742 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.742 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.742 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.742 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.742 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.742 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-05-15T14:12:13.742 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.742 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.743 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.743 INFO:teuthology.orchestra.run.smithi033.stdout: "fail_fs": { 2024-05-15T14:12:13.743 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "fail_fs", 2024-05-15T14:12:13.743 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.743 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.743 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.743 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.743 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.744 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.744 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.744 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-15T14:12:13.744 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.744 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.744 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.744 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.744 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.745 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.745 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.745 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.745 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.745 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.745 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.745 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.745 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.745 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.746 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.746 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.746 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.746 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.746 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.746 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.746 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.746 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.747 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.747 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.747 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.747 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.747 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.747 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.747 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.747 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.748 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.748 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.748 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.748 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.748 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.748 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.748 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.748 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.749 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.749 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.749 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.749 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.749 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.749 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.749 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.749 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.749 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.750 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.750 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.750 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.750 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.750 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.750 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.750 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.750 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.751 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.751 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.751 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.751 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.751 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.751 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.751 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.751 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.752 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.752 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.752 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.752 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.752 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.752 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.752 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.753 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator": { 2024-05-15T14:12:13.753 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-05-15T14:12:13.753 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.753 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.753 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.753 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.753 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.753 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.753 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.754 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm", 2024-05-15T14:12:13.754 INFO:teuthology.orchestra.run.smithi033.stdout: "rook", 2024-05-15T14:12:13.754 INFO:teuthology.orchestra.run.smithi033.stdout: "test_orchestrator" 2024-05-15T14:12:13.754 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.754 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Orchestrator backend", 2024-05-15T14:12:13.754 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.754 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.754 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.755 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.755 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.755 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.755 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.755 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.755 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.755 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.755 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.756 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.756 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.756 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.756 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.756 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.756 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.756 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.757 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.757 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_perf_query", 2024-05-15T14:12:13.757 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.757 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.757 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.757 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.757 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.757 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.758 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.758 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.758 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.758 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.758 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.758 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.758 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.758 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.759 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.759 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.759 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.759 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.759 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.759 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.759 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.759 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.759 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.760 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.760 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.760 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.760 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.760 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.760 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.760 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.760 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.761 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.761 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.761 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.761 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.761 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.761 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.761 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.761 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.762 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.762 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.762 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.762 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.762 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.762 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.762 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.762 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.763 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.763 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.763 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.763 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.763 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.763 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.763 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.763 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.764 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.764 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.764 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.764 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.764 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.764 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.764 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.764 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.764 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.765 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.765 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.765 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.765 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.765 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.765 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.765 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.765 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.766 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.766 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.766 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.766 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.766 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.766 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.766 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.766 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.767 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.767 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.767 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.767 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.767 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.767 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.767 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.767 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.768 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_support", 2024-05-15T14:12:13.768 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.768 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.768 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.768 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.768 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.768 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.768 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.768 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.769 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.769 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.769 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.769 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.769 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.769 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.769 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.769 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.769 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.770 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.770 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.770 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.770 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.770 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.770 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.770 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.770 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.771 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.771 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.771 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.771 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.771 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.771 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.771 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.771 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.771 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.772 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.772 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.772 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.772 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.772 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.772 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.772 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.773 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.773 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.773 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.773 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.773 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.773 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.773 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.773 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.773 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.774 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.774 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.774 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.774 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.774 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.774 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.774 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.775 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.775 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.775 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.775 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.775 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.775 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.775 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.775 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.775 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.776 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.776 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.776 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.776 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.776 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.776 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.776 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.776 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.777 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.777 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.777 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.777 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.777 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.777 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.777 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.777 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.778 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.778 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.778 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.778 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.778 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.778 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_autoscaler", 2024-05-15T14:12:13.778 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.778 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.779 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.779 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.779 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.779 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.779 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.779 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.779 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.779 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.779 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.780 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.780 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.780 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.780 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.780 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.780 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.780 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.780 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.781 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.781 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.781 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.781 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.781 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.781 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.781 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.781 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.782 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.782 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.782 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.782 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.782 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.782 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.782 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.782 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.783 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.783 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.783 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.783 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.783 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.783 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.783 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.784 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.784 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.784 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.784 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.784 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.784 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.784 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.784 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.785 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.785 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.785 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.785 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.785 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.785 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.785 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.785 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.786 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.786 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.786 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.786 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.786 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.786 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.786 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.786 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.787 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.787 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.787 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.787 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-15T14:12:13.787 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-15T14:12:13.787 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:13.787 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.788 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.788 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-15T14:12:13.788 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.788 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.788 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.788 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.788 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.788 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.788 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.789 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.789 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.789 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.789 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.789 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.789 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.789 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.790 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.790 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.790 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.790 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.790 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.790 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.790 INFO:teuthology.orchestra.run.smithi033.stdout: "threshold": { 2024-05-15T14:12:13.791 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threshold", 2024-05-15T14:12:13.791 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-15T14:12:13.791 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.791 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.791 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-05-15T14:12:13.791 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1.0", 2024-05-15T14:12:13.791 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.791 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.792 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "scaling threshold", 2024-05-15T14:12:13.792 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-15T14:12:13.792 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.792 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.792 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.792 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.792 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.792 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "progress", 2024-05-15T14:12:13.793 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.793 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.793 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.793 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_pg_recovery_event": { 2024-05-15T14:12:13.793 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_pg_recovery_event", 2024-05-15T14:12:13.793 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.793 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.793 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.794 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.794 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.794 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.794 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.794 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-15T14:12:13.794 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.794 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.794 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.795 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-05-15T14:12:13.795 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-05-15T14:12:13.795 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.795 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.795 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.795 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.795 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.795 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.796 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.796 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.796 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.796 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.796 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.796 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.796 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.796 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.797 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.797 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.797 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.797 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.797 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.797 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.797 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.797 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.797 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.798 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.798 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.798 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.798 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.798 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.798 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.798 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.798 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.799 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.799 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.799 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.799 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.799 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.799 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.799 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.799 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.800 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.800 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.800 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.800 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.800 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.800 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.800 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.801 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.801 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.801 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.801 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.801 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.801 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.801 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.801 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.802 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.802 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.802 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.802 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.802 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.802 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.802 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.802 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.802 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.803 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.803 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.803 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.803 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.803 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.803 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.803 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.804 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.804 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.804 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.804 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.804 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.804 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.804 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.804 INFO:teuthology.orchestra.run.smithi033.stdout: "max_completed_events": { 2024-05-15T14:12:13.805 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_completed_events", 2024-05-15T14:12:13.805 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.805 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.805 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.805 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "50", 2024-05-15T14:12:13.805 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.805 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.805 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.806 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "number of past completed events to remember", 2024-05-15T14:12:13.806 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.806 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.806 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.806 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-15T14:12:13.806 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-15T14:12:13.806 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:13.806 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.807 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.807 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-15T14:12:13.807 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.807 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.807 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.807 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long the module is going to sleep", 2024-05-15T14:12:13.807 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.807 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.808 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.808 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.808 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.808 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.808 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.808 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.808 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.808 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.809 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.809 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.809 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.809 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.809 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.809 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.809 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.809 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.810 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.810 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus", 2024-05-15T14:12:13.810 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.810 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.810 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.810 INFO:teuthology.orchestra.run.smithi033.stdout: "cache": { 2024-05-15T14:12:13.810 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cache", 2024-05-15T14:12:13.810 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.811 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.811 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.811 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.811 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.811 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.811 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.811 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.811 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.811 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.812 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.812 INFO:teuthology.orchestra.run.smithi033.stdout: "exclude_perf_counters": { 2024-05-15T14:12:13.812 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "exclude_perf_counters", 2024-05-15T14:12:13.812 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.812 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.812 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.812 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.812 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.813 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.813 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.813 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-15T14:12:13.813 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-15T14:12:13.813 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.813 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.813 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.814 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.814 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.814 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.814 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.814 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.814 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.814 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.814 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.815 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.815 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.815 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.815 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.815 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.815 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.815 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.815 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.815 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.816 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.816 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.816 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.816 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.816 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.816 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.816 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.817 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.817 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.817 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.817 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.817 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.817 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.817 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.817 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.818 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.818 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.818 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.818 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.818 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.818 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.818 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.818 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.819 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.819 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.819 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.819 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.819 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.819 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.819 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.819 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.819 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.820 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.820 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.820 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.820 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.820 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.820 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.820 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.821 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.821 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.821 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.821 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.821 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.821 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.821 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.821 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.821 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.822 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools": { 2024-05-15T14:12:13.822 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools", 2024-05-15T14:12:13.822 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.822 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.822 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.822 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.822 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.823 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.823 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.823 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.823 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.823 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.823 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.823 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-15T14:12:13.823 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-15T14:12:13.824 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.824 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.824 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.824 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "300", 2024-05-15T14:12:13.824 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.824 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.824 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.824 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.824 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.825 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.825 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.825 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_interval": { 2024-05-15T14:12:13.825 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_interval", 2024-05-15T14:12:13.825 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-15T14:12:13.825 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.825 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.826 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15.0", 2024-05-15T14:12:13.826 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.826 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.826 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.826 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.826 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.826 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.826 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.827 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-05-15T14:12:13.827 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-05-15T14:12:13.827 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.827 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.827 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.827 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-05-15T14:12:13.827 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.827 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.827 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.828 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-15T14:12:13.828 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.828 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.828 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.828 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.828 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-05-15T14:12:13.828 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-05-15T14:12:13.828 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.829 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.829 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.829 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "9283", 2024-05-15T14:12:13.829 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.829 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.829 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.829 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-15T14:12:13.829 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.829 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.830 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.830 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.830 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.830 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.830 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.830 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.830 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.831 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.831 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.831 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.831 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.831 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.831 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.831 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.831 INFO:teuthology.orchestra.run.smithi033.stdout: "stale_cache_strategy": { 2024-05-15T14:12:13.832 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stale_cache_strategy", 2024-05-15T14:12:13.832 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.832 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.832 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.832 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "log", 2024-05-15T14:12:13.832 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.832 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.832 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.832 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.833 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.833 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.833 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.833 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.833 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-05-15T14:12:13.833 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-05-15T14:12:13.833 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.833 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.833 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.834 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "default", 2024-05-15T14:12:13.834 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.834 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.834 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.834 INFO:teuthology.orchestra.run.smithi033.stdout: "default", 2024-05-15T14:12:13.834 INFO:teuthology.orchestra.run.smithi033.stdout: "error" 2024-05-15T14:12:13.834 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.834 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.835 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.835 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.835 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.835 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-05-15T14:12:13.835 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-05-15T14:12:13.835 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.835 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.835 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.836 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-05-15T14:12:13.836 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-05-15T14:12:13.836 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-05-15T14:12:13.836 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.836 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.836 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.836 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.836 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.837 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.837 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.837 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.837 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.837 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_support", 2024-05-15T14:12:13.837 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.837 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.837 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.837 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.838 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.838 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.838 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.838 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.838 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.838 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.838 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.838 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.839 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.839 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.839 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.839 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.839 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.839 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.839 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.839 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.839 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.840 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.840 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.840 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.840 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.840 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.840 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.840 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.841 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.841 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.841 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.841 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.841 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.841 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.841 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.841 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.842 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.842 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.842 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.842 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.842 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.842 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.842 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.842 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.843 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.843 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.843 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.843 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.843 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.843 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.843 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.843 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.843 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.844 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.844 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.844 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.844 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.844 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.844 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.844 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.844 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.845 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.845 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.845 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.845 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.845 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.845 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.845 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.845 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.846 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_snap_create": { 2024-05-15T14:12:13.846 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_snap_create", 2024-05-15T14:12:13.846 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.846 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.846 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.846 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-15T14:12:13.846 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.846 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.847 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.847 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.847 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.847 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.847 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.847 INFO:teuthology.orchestra.run.smithi033.stdout: "mirror_snapshot_schedule": { 2024-05-15T14:12:13.847 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirror_snapshot_schedule", 2024-05-15T14:12:13.847 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.848 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.848 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.848 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.848 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.848 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.848 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.848 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.848 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.849 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.849 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.849 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.849 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.849 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.849 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.849 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.849 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.850 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.850 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.850 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.850 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.850 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.850 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.850 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.851 INFO:teuthology.orchestra.run.smithi033.stdout: "trash_purge_schedule": { 2024-05-15T14:12:13.851 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "trash_purge_schedule", 2024-05-15T14:12:13.851 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.851 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.851 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.851 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.851 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.851 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.851 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.852 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.852 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.852 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.852 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.852 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.852 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.852 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.852 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "restful", 2024-05-15T14:12:13.853 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.853 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.853 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.853 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_auth": { 2024-05-15T14:12:13.853 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_auth", 2024-05-15T14:12:13.853 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.853 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.853 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.854 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.854 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.854 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.854 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.854 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.854 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.854 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.854 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.855 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-05-15T14:12:13.855 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-05-15T14:12:13.855 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.855 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.855 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.855 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.855 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.855 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.856 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.856 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.856 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.856 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.856 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.856 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.856 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.856 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.857 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.857 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.857 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.857 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.857 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.857 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.857 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.857 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.857 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.858 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.858 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.858 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.858 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.858 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.858 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.858 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.858 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.859 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.859 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.859 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.859 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.859 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.859 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.859 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.859 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.860 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.860 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.860 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.860 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.860 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.860 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.860 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.860 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.861 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.861 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.861 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.861 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.861 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.861 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.861 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.861 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.862 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.862 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.862 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.862 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.862 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.862 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.862 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.862 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.863 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.863 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.863 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.863 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.863 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.863 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.863 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.863 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.864 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.864 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.864 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.864 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.864 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.864 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.864 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-05-15T14:12:13.865 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-05-15T14:12:13.865 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.865 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.865 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.865 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.865 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.865 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.865 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.865 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.866 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.866 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.866 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.866 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-05-15T14:12:13.866 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-05-15T14:12:13.866 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.866 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.867 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.867 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.867 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.867 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.867 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.867 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.867 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.867 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.867 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.868 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.868 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.868 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.868 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.868 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.868 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.868 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.869 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.869 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.869 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.869 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.869 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.869 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.869 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.869 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.870 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.870 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rgw", 2024-05-15T14:12:13.870 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.870 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.870 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.870 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.870 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.870 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.871 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.871 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.871 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.871 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.871 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.871 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.871 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.871 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.871 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.872 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.872 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.872 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.872 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.872 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.872 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.872 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.872 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.873 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.873 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.873 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.873 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.873 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.873 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.873 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.873 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.874 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.874 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.874 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.874 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.874 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.874 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.874 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.875 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.875 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.875 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.875 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.875 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.875 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.875 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.875 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.875 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.876 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.876 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.876 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.876 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.876 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.876 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.876 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.876 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.876 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.877 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.877 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.877 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.877 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.877 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.877 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.877 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.878 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.878 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.878 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.878 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.878 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.878 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.878 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.879 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.879 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.879 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.879 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.879 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.879 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.879 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.879 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.879 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.880 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.880 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.880 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.880 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.880 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.880 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.880 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rook", 2024-05-15T14:12:13.880 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.881 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.881 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.881 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.881 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.881 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.881 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.881 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.881 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.881 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.882 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.882 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.882 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.882 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.882 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.882 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.882 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.882 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.883 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.883 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.883 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.883 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.883 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.883 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.883 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.883 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.884 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.884 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.884 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.884 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.884 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.884 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.884 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.884 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.885 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.885 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.885 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.885 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.885 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.885 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.885 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.885 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.886 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.886 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.886 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.886 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.886 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.886 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.886 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.886 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.887 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.887 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.887 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.887 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.887 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.887 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.887 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.887 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.888 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.888 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.888 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.888 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.888 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.888 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.888 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.888 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.889 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.889 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.889 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.889 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.889 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.889 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.889 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.889 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.890 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.890 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.890 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.890 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.890 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.890 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.890 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.890 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.891 INFO:teuthology.orchestra.run.smithi033.stdout: "storage_class": { 2024-05-15T14:12:13.891 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "storage_class", 2024-05-15T14:12:13.891 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.891 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.891 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.891 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "local", 2024-05-15T14:12:13.891 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.891 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.892 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.892 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-15T14:12:13.892 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.892 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.892 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.892 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.892 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.892 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.893 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "selftest", 2024-05-15T14:12:13.893 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.893 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.893 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.893 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.893 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.893 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.893 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.893 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.894 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.894 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.894 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.894 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.894 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.894 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.894 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.894 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.895 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.895 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.895 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.895 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.895 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.895 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.895 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.895 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.896 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.896 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.896 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.896 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.896 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.896 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.896 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.896 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.897 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.897 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.897 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.897 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.897 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.897 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.897 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.897 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.898 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.898 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.898 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.898 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.898 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.898 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.898 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.898 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.899 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.899 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.899 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.899 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.899 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.899 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.899 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.899 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.900 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.900 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.900 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.900 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.900 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.900 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.900 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.900 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.900 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.901 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.901 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.901 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.901 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.901 INFO:teuthology.orchestra.run.smithi033.stdout: "roption1": { 2024-05-15T14:12:13.901 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption1", 2024-05-15T14:12:13.901 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.902 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.902 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.902 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.902 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.902 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.902 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.902 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.902 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.902 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.903 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.903 INFO:teuthology.orchestra.run.smithi033.stdout: "roption2": { 2024-05-15T14:12:13.903 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption2", 2024-05-15T14:12:13.903 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.903 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.903 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.903 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "xyz", 2024-05-15T14:12:13.904 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.904 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.904 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.904 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.904 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.904 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.904 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.904 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption1": { 2024-05-15T14:12:13.905 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption1", 2024-05-15T14:12:13.905 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.905 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.905 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.905 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.905 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.905 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.905 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.905 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.906 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.906 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.906 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.906 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption2": { 2024-05-15T14:12:13.906 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption2", 2024-05-15T14:12:13.906 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.906 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.907 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.907 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.907 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.907 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.907 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.907 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.907 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.907 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.908 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.908 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption3": { 2024-05-15T14:12:13.908 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption3", 2024-05-15T14:12:13.908 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-15T14:12:13.908 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.908 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.908 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.908 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.909 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.909 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.909 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.909 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.909 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.909 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.909 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption4": { 2024-05-15T14:12:13.910 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption4", 2024-05-15T14:12:13.910 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.910 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.910 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.910 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.910 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.910 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.910 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.910 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.911 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.911 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.911 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.911 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption5": { 2024-05-15T14:12:13.911 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption5", 2024-05-15T14:12:13.911 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.911 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.912 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.912 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.912 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.912 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.912 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.912 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.912 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.912 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.912 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.913 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption6": { 2024-05-15T14:12:13.913 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption6", 2024-05-15T14:12:13.913 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.913 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.913 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.913 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.913 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.913 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.914 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.914 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.914 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.914 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.914 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.914 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption7": { 2024-05-15T14:12:13.914 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption7", 2024-05-15T14:12:13.915 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.915 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.915 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.915 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.915 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-15T14:12:13.915 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "42", 2024-05-15T14:12:13.915 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.915 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.916 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.916 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.916 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.916 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.916 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.916 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.916 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.916 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.917 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.917 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.917 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.917 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.917 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.917 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.917 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.917 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.918 INFO:teuthology.orchestra.run.smithi033.stdout: "testkey": { 2024-05-15T14:12:13.918 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testkey", 2024-05-15T14:12:13.918 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.918 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.918 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.918 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.918 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.918 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.919 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.919 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.919 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.919 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.919 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.919 INFO:teuthology.orchestra.run.smithi033.stdout: "testlkey": { 2024-05-15T14:12:13.919 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testlkey", 2024-05-15T14:12:13.920 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.920 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.920 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.920 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.920 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.920 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.920 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.920 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.921 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.921 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.921 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.921 INFO:teuthology.orchestra.run.smithi033.stdout: "testnewline": { 2024-05-15T14:12:13.921 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testnewline", 2024-05-15T14:12:13.921 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.921 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.921 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.922 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.922 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.922 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.922 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.922 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.922 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.922 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.922 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.923 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.923 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.923 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smb", 2024-05-15T14:12:13.923 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.923 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.923 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.923 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.924 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.924 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.924 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.924 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.924 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.924 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.924 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.924 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.925 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.925 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.925 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.925 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.925 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.925 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.925 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.925 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.925 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.926 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.926 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.926 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.926 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.926 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.926 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.926 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.927 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.927 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.927 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.927 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.927 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.927 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.927 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.927 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.928 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.928 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.928 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.928 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.928 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.928 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.928 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.928 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.929 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.929 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.929 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.929 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.929 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.929 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.929 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.929 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.930 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.930 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.930 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.930 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.930 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.930 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.930 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.930 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.931 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.931 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.931 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.931 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.931 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.931 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.931 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.931 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.932 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.932 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.932 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.932 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.932 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.932 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.932 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.932 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.933 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.933 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.933 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.933 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.933 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.933 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.933 INFO:teuthology.orchestra.run.smithi033.stdout: "update_orchestration": { 2024-05-15T14:12:13.934 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "update_orchestration", 2024-05-15T14:12:13.934 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.934 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.934 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.934 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.934 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.934 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.934 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.934 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-15T14:12:13.935 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.935 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.935 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.935 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.935 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.935 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snap_schedule", 2024-05-15T14:12:13.936 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.936 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.936 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.936 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_m_granularity": { 2024-05-15T14:12:13.936 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_m_granularity", 2024-05-15T14:12:13.936 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.936 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.936 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.936 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.937 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.937 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.937 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.937 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow minute scheduled snapshots", 2024-05-15T14:12:13.937 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.937 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.937 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.938 INFO:teuthology.orchestra.run.smithi033.stdout: "dump_on_update": { 2024-05-15T14:12:13.938 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dump_on_update", 2024-05-15T14:12:13.938 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.938 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.938 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.938 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.938 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.938 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.939 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.939 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "dump database to debug log on update", 2024-05-15T14:12:13.939 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.939 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.939 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.939 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.939 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.939 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.940 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.940 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.940 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.940 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.940 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.940 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.940 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.940 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.941 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.941 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.941 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.941 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.941 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.941 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.941 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.941 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.941 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.942 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.942 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.942 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.942 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.942 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.942 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.942 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.943 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.943 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.943 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.943 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.943 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.943 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.943 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.943 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.944 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.944 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.944 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.944 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.944 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.944 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.944 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.944 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.945 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.945 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.945 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.945 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.945 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.945 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.945 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.945 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.945 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.946 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.946 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.946 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.946 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.946 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.946 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.946 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.947 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.947 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.947 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.947 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.947 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.947 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.947 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.948 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.948 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.948 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.948 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.948 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.948 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.948 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.948 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.948 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.949 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.949 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.949 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.949 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.949 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.949 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.950 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stats", 2024-05-15T14:12:13.950 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.950 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.950 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.950 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.950 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.950 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.950 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.950 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.951 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.951 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.951 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.951 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.951 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.951 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.951 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.951 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.952 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.952 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.952 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.952 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.952 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.952 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.952 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.953 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.953 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.953 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.953 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.953 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.953 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.953 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.953 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.953 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.954 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.954 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.954 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.954 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.954 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.954 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.954 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.955 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.955 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.955 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.955 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.955 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.955 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.955 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.955 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.955 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.956 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.956 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.956 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.956 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.956 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.956 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.956 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.956 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.957 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.957 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.957 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.957 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.957 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.957 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.957 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.957 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.958 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.958 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.958 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.958 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.958 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.958 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.958 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.959 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.959 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.959 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.959 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.959 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.959 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.959 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.959 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.959 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.960 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.960 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.960 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.960 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.960 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "status", 2024-05-15T14:12:13.960 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.961 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.961 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.961 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.961 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.961 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.961 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.961 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.961 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.961 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.962 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.962 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.962 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.962 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.962 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.962 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.962 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.962 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.963 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.963 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.963 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.963 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.963 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.963 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.963 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.964 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.964 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.964 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.964 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.964 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.964 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.964 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.964 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.964 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.965 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.965 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.965 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.965 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.965 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.965 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.965 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.965 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.966 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.966 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.966 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.966 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.966 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.966 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.966 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.966 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.967 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.967 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.967 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.967 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.967 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.967 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.967 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.968 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.968 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.968 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.968 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.968 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.968 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.968 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.968 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.968 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.969 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.969 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.969 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.969 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.969 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.969 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.969 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.970 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.970 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.970 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.970 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.970 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.970 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.970 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.970 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.971 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.971 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.971 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.971 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.971 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telegraf", 2024-05-15T14:12:13.971 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.971 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.971 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.972 INFO:teuthology.orchestra.run.smithi033.stdout: "address": { 2024-05-15T14:12:13.972 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "address", 2024-05-15T14:12:13.972 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.972 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.972 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.972 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-15T14:12:13.972 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.972 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.972 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.973 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.973 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.973 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.973 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.973 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-15T14:12:13.973 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-15T14:12:13.973 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:13.974 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.974 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.974 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15", 2024-05-15T14:12:13.974 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.974 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.974 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.974 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.974 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.975 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.975 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.975 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:13.975 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:13.975 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.975 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.975 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.975 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.976 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.976 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.976 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.976 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.976 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.976 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.976 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.976 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.977 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.977 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.977 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.977 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.977 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.977 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.977 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:13.977 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:13.978 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.978 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.978 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.978 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.978 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.978 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.978 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.978 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.979 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.979 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.979 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.979 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:13.979 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:13.979 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.979 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.979 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.980 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:13.980 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.980 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.980 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:13.980 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:13.980 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:13.980 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:13.980 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:13.981 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:13.981 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:13.981 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:13.981 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.981 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.981 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.981 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.982 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:13.982 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:13.982 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.982 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.982 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.982 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.982 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.982 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.982 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.983 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.983 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.983 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.983 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.983 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:13.983 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:13.983 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:13.984 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:13.984 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:13.984 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:13.984 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.984 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.984 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.984 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.984 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.984 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.985 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.985 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.985 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:13.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:13.985 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telemetry", 2024-05-15T14:12:13.985 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:13.985 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:13.986 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:13.986 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_basic": { 2024-05-15T14:12:13.986 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_basic", 2024-05-15T14:12:13.986 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.986 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.986 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.986 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.986 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.986 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.987 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.987 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-15T14:12:13.987 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.987 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.987 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.987 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_crash": { 2024-05-15T14:12:13.987 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_crash", 2024-05-15T14:12:13.988 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.988 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.988 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.988 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.988 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.988 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.988 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.988 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-15T14:12:13.988 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.989 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.989 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.989 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_device": { 2024-05-15T14:12:13.989 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_device", 2024-05-15T14:12:13.989 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.989 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.989 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.990 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:13.990 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.990 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.990 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.990 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-15T14:12:13.990 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.990 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.990 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.991 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_ident": { 2024-05-15T14:12:13.991 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_ident", 2024-05-15T14:12:13.991 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.991 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.991 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.991 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.991 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.991 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.992 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.992 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-15T14:12:13.992 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.992 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.992 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.992 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_perf": { 2024-05-15T14:12:13.992 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_perf", 2024-05-15T14:12:13.993 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.993 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.993 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.993 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:13.993 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.993 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.993 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.993 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-15T14:12:13.993 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.994 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.994 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.994 INFO:teuthology.orchestra.run.smithi033.stdout: "contact": { 2024-05-15T14:12:13.994 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "contact", 2024-05-15T14:12:13.994 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.994 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.994 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.995 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.995 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.995 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.995 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.995 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.995 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.995 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.995 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.996 INFO:teuthology.orchestra.run.smithi033.stdout: "description": { 2024-05-15T14:12:13.996 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "description", 2024-05-15T14:12:13.996 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.996 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.996 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.996 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:13.996 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.996 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.997 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.997 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.997 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.997 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.997 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.997 INFO:teuthology.orchestra.run.smithi033.stdout: "device_url": { 2024-05-15T14:12:13.997 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_url", 2024-05-15T14:12:13.998 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:13.998 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.998 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:13.998 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-15T14:12:13.998 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:13.998 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:13.998 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:13.998 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:13.999 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:13.999 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:13.999 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:13.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:13.999 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-05-15T14:12:13.999 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-05-15T14:12:13.999 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:13.999 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:13.999 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:14.000 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:14.000 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.000 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.000 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:14.000 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:14.000 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.000 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.000 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.001 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-15T14:12:14.001 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-15T14:12:14.001 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:14.001 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:14.001 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:14.001 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "24", 2024-05-15T14:12:14.001 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "8", 2024-05-15T14:12:14.001 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.002 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:14.002 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:14.002 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.002 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.002 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.002 INFO:teuthology.orchestra.run.smithi033.stdout: "last_opt_revision": { 2024-05-15T14:12:14.002 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "last_opt_revision", 2024-05-15T14:12:14.002 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:14.003 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:14.003 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:14.003 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1", 2024-05-15T14:12:14.003 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.003 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.003 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:14.003 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:14.003 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.004 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.004 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.004 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard": { 2024-05-15T14:12:14.004 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard", 2024-05-15T14:12:14.004 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:14.004 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:14.004 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:14.004 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:14.005 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.005 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.005 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:14.005 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:14.005 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.005 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.005 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.006 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard_description": { 2024-05-15T14:12:14.006 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard_description", 2024-05-15T14:12:14.006 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:14.006 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:14.006 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:14.006 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:14.006 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.006 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.006 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:14.007 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:14.007 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.007 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.007 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.007 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:14.007 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:14.007 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:14.008 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:14.008 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:14.008 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:14.008 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.008 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.008 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:14.008 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:14.008 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:14.008 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:14.009 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:14.009 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:14.009 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:14.009 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:14.009 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:14.009 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.009 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.009 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.010 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:14.010 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:14.010 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:14.010 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:14.010 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:14.010 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:14.010 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.011 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.011 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:14.011 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:14.011 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.011 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.011 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.011 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:14.011 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:14.012 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:14.012 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:14.012 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:14.012 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:14.012 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.012 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.012 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:14.012 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:14.013 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:14.013 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:14.013 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:14.013 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:14.013 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:14.013 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:14.013 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:14.013 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.013 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.014 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.014 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:14.014 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:14.014 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:14.014 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:14.014 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:14.014 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:14.015 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.015 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.015 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:14.015 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:14.015 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.015 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.015 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.016 INFO:teuthology.orchestra.run.smithi033.stdout: "organization": { 2024-05-15T14:12:14.016 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "organization", 2024-05-15T14:12:14.016 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:14.016 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:14.016 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:14.016 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:14.016 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.016 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.016 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:14.017 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:14.017 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.017 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.017 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.017 INFO:teuthology.orchestra.run.smithi033.stdout: "proxy": { 2024-05-15T14:12:14.017 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "proxy", 2024-05-15T14:12:14.017 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:14.018 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:14.018 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:14.018 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:14.018 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.018 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.018 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:14.018 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:14.018 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.018 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.019 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.019 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:14.019 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:14.019 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:14.019 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:14.019 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:14.019 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:14.020 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.020 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.020 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:14.020 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:14.020 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.020 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.020 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.020 INFO:teuthology.orchestra.run.smithi033.stdout: "url": { 2024-05-15T14:12:14.021 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url", 2024-05-15T14:12:14.021 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:14.021 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:14.021 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:14.021 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-15T14:12:14.021 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.021 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.021 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:14.021 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:14.022 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.022 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.022 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.022 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:14.022 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:14.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.022 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:14.022 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "test_orchestrator", 2024-05-15T14:12:14.023 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:14.023 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:14.023 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:14.023 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:14.023 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:14.023 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:14.023 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:14.023 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:14.023 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:14.024 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.024 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.024 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:14.024 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:14.024 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:14.024 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:14.024 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:14.024 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:14.025 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:14.025 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:14.025 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:14.025 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.025 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.025 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.025 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:14.025 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:14.026 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:14.026 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:14.026 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:14.026 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:14.026 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.026 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.026 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:14.026 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:14.027 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.027 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.027 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.027 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:14.027 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:14.027 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:14.027 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:14.027 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:14.028 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:14.028 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.028 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.028 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:14.028 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:14.028 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:14.028 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:14.028 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:14.028 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:14.029 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:14.029 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:14.029 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:14.029 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.029 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.029 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.029 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:14.030 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:14.030 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:14.030 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:14.030 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:14.030 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:14.030 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.030 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.030 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:14.031 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:14.031 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.031 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.031 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.031 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:14.031 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:14.031 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:14.031 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:14.032 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:14.032 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:14.032 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.032 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.032 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:14.032 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:14.032 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.032 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.032 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.033 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:14.033 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:14.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:14.033 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "volumes", 2024-05-15T14:12:14.033 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:14.033 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:14.033 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:14.033 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:14.034 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:14.034 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:14.034 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:14.034 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:14.034 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:14.034 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.034 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.034 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:14.035 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:14.035 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:14.035 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:14.035 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:14.035 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:14.035 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:14.035 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:14.035 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:14.036 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.036 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.036 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.036 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:14.036 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:14.036 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:14.036 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:14.036 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:14.037 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:14.037 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.037 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.037 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:14.037 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:14.037 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.037 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.037 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.038 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:14.038 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:14.038 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:14.038 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:14.038 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:14.038 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:14.038 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.038 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.039 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:14.039 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:14.039 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:14.039 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:14.039 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:14.039 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:14.039 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:14.039 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:14.039 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:14.040 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.040 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.040 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.040 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:14.040 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:14.040 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:14.040 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:14.041 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:14.041 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:14.041 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.041 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.041 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:14.041 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:14.041 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.041 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.041 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.042 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_clones": { 2024-05-15T14:12:14.042 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_clones", 2024-05-15T14:12:14.042 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:14.042 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:14.042 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:14.042 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4", 2024-05-15T14:12:14.042 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.043 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.043 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:14.043 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-15T14:12:14.043 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.043 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.043 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.043 INFO:teuthology.orchestra.run.smithi033.stdout: "periodic_async_work": { 2024-05-15T14:12:14.043 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "periodic_async_work", 2024-05-15T14:12:14.044 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:14.044 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:14.044 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:14.044 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:14.044 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.044 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.044 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:14.044 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Periodically check for async work", 2024-05-15T14:12:14.045 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.045 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.045 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.045 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_delay": { 2024-05-15T14:12:14.045 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_delay", 2024-05-15T14:12:14.045 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:14.045 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:14.045 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:14.046 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:14.046 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.046 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.046 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:14.046 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-15T14:12:14.046 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.046 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.046 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.047 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_no_wait": { 2024-05-15T14:12:14.047 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_no_wait", 2024-05-15T14:12:14.047 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:14.047 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:14.047 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:14.047 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-15T14:12:14.047 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.048 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.048 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:14.048 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-15T14:12:14.048 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.048 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.048 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.048 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:14.048 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:14.049 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:14.049 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:14.049 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:14.049 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:14.049 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.049 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.049 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:14.049 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:14.050 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.050 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.050 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.050 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:14.050 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:14.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:14.050 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix", 2024-05-15T14:12:14.050 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-15T14:12:14.051 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-15T14:12:14.051 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-15T14:12:14.051 INFO:teuthology.orchestra.run.smithi033.stdout: "discovery_interval": { 2024-05-15T14:12:14.051 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "discovery_interval", 2024-05-15T14:12:14.051 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-15T14:12:14.051 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:14.051 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:14.051 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "100", 2024-05-15T14:12:14.052 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.052 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.052 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:14.052 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:14.052 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.052 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.052 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.052 INFO:teuthology.orchestra.run.smithi033.stdout: "identifier": { 2024-05-15T14:12:14.053 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "identifier", 2024-05-15T14:12:14.053 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:14.053 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:14.053 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:14.053 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:14.053 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.053 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.053 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:14.054 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:14.054 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.054 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.054 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.054 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-15T14:12:14.054 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-15T14:12:14.054 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-15T14:12:14.054 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:14.055 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:14.055 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-15T14:12:14.055 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.055 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.055 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:14.055 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:14.055 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.055 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.056 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.056 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-15T14:12:14.056 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-15T14:12:14.056 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:14.056 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:14.056 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:14.056 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:14.056 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.057 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.057 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:14.057 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:14.057 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:14.057 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:14.057 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:14.057 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:14.057 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:14.058 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:14.058 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:14.058 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.058 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.058 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.058 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-15T14:12:14.058 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-15T14:12:14.059 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:14.059 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:14.059 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:14.059 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:14.059 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.059 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.059 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:14.059 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:14.059 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.060 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.060 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.060 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-15T14:12:14.060 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-15T14:12:14.060 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:14.060 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:14.060 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:14.061 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-15T14:12:14.061 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.061 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.061 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-15T14:12:14.061 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-15T14:12:14.061 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-15T14:12:14.061 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-15T14:12:14.061 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-15T14:12:14.061 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-15T14:12:14.062 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-15T14:12:14.062 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:14.062 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:14.062 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.062 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.062 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.062 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-15T14:12:14.063 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-15T14:12:14.063 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-15T14:12:14.063 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:14.063 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:14.063 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-15T14:12:14.063 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.063 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.063 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:14.063 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:14.064 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.064 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.064 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.064 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-05-15T14:12:14.064 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-05-15T14:12:14.064 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:14.064 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-05-15T14:12:14.065 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-15T14:12:14.065 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-15T14:12:14.065 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.065 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.065 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:14.065 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:14.065 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.065 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.065 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.066 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_host": { 2024-05-15T14:12:14.066 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_host", 2024-05-15T14:12:14.066 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:14.066 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:14.066 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:14.066 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-15T14:12:14.066 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.067 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.067 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:14.067 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:14.067 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.067 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.067 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.067 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_port": { 2024-05-15T14:12:14.067 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_port", 2024-05-15T14:12:14.068 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-15T14:12:14.068 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:14.068 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:14.068 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10051", 2024-05-15T14:12:14.068 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.068 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.068 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:14.068 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:14.069 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.069 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.069 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.069 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_sender": { 2024-05-15T14:12:14.069 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_sender", 2024-05-15T14:12:14.069 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-15T14:12:14.069 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-15T14:12:14.069 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-15T14:12:14.070 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-15T14:12:14.070 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-15T14:12:14.070 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-15T14:12:14.070 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-15T14:12:14.070 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-15T14:12:14.070 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-15T14:12:14.070 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-15T14:12:14.070 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-15T14:12:14.070 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:14.071 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:14.071 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:14.071 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:14.071 INFO:teuthology.orchestra.run.smithi033.stdout: "services": { 2024-05-15T14:12:14.071 INFO:teuthology.orchestra.run.smithi033.stdout: "dashboard": "https://172.21.15.33:8443/" 2024-05-15T14:12:14.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.071 INFO:teuthology.orchestra.run.smithi033.stdout: "always_on_modules": { 2024-05-15T14:12:14.071 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus": [ 2024-05-15T14:12:14.072 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-05-15T14:12:14.072 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-05-15T14:12:14.072 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-05-15T14:12:14.072 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-05-15T14:12:14.072 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-05-15T14:12:14.072 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-05-15T14:12:14.072 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-05-15T14:12:14.072 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-05-15T14:12:14.072 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-05-15T14:12:14.073 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-05-15T14:12:14.073 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:14.073 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific": [ 2024-05-15T14:12:14.073 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-05-15T14:12:14.073 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-05-15T14:12:14.073 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-05-15T14:12:14.073 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-05-15T14:12:14.073 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-05-15T14:12:14.073 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-05-15T14:12:14.074 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-05-15T14:12:14.074 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-05-15T14:12:14.074 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-05-15T14:12:14.074 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-05-15T14:12:14.074 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:14.074 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy": [ 2024-05-15T14:12:14.074 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-05-15T14:12:14.074 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-05-15T14:12:14.075 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-05-15T14:12:14.075 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-05-15T14:12:14.075 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-05-15T14:12:14.075 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-05-15T14:12:14.075 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-05-15T14:12:14.075 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-05-15T14:12:14.075 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-05-15T14:12:14.075 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-05-15T14:12:14.076 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:14.076 INFO:teuthology.orchestra.run.smithi033.stdout: "reef": [ 2024-05-15T14:12:14.076 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-05-15T14:12:14.076 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-05-15T14:12:14.076 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-05-15T14:12:14.076 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-05-15T14:12:14.076 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-05-15T14:12:14.076 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-05-15T14:12:14.076 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-05-15T14:12:14.077 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-05-15T14:12:14.077 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-05-15T14:12:14.077 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-05-15T14:12:14.077 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-15T14:12:14.077 INFO:teuthology.orchestra.run.smithi033.stdout: "squid": [ 2024-05-15T14:12:14.077 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-05-15T14:12:14.077 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-05-15T14:12:14.077 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-05-15T14:12:14.077 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-05-15T14:12:14.078 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-05-15T14:12:14.078 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-05-15T14:12:14.078 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-05-15T14:12:14.078 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-05-15T14:12:14.078 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-05-15T14:12:14.078 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-05-15T14:12:14.078 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-15T14:12:14.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.079 INFO:teuthology.orchestra.run.smithi033.stdout: "last_failure_osd_epoch": 83, 2024-05-15T14:12:14.079 INFO:teuthology.orchestra.run.smithi033.stdout: "active_clients": [ 2024-05-15T14:12:14.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:14.079 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "devicehealth", 2024-05-15T14:12:14.079 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-15T14:12:14.079 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:14.079 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-15T14:12:14.079 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-05-15T14:12:14.079 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 3898733961 2024-05-15T14:12:14.080 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:14.080 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-15T14:12:14.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.080 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:14.080 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "libcephsqlite", 2024-05-15T14:12:14.080 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-15T14:12:14.080 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:14.080 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-15T14:12:14.080 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-05-15T14:12:14.081 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 2403336279 2024-05-15T14:12:14.081 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:14.081 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-15T14:12:14.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:14.081 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_support", 2024-05-15T14:12:14.081 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-15T14:12:14.081 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:14.081 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-15T14:12:14.082 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-05-15T14:12:14.082 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 1737024829 2024-05-15T14:12:14.082 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:14.082 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-15T14:12:14.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.082 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:14.082 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snap_schedule", 2024-05-15T14:12:14.082 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-15T14:12:14.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:14.083 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-15T14:12:14.083 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-05-15T14:12:14.083 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 982493193 2024-05-15T14:12:14.083 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:14.083 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-15T14:12:14.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.083 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:14.083 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snap_schedule", 2024-05-15T14:12:14.084 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-15T14:12:14.084 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:14.084 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-15T14:12:14.084 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-05-15T14:12:14.084 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 32989564 2024-05-15T14:12:14.084 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:14.084 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-15T14:12:14.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:14.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:14.085 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "volumes", 2024-05-15T14:12:14.085 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-15T14:12:14.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-15T14:12:14.085 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-15T14:12:14.085 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-05-15T14:12:14.085 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 1981215961 2024-05-15T14:12:14.085 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:14.085 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-15T14:12:14.086 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:14.086 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-15T14:12:14.086 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T14:12:14.087 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:11 smithi081 ceph-mon[36434]: from='client.16821 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T14:12:14.087 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:11 smithi081 ceph-mon[36434]: osdmap e98: 12 total, 12 up, 12 in 2024-05-15T14:12:14.087 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:11 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/1084167140' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-15T14:12:14.088 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:11 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/1134281336' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-15T14:12:14.088 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:11 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/2114073653' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-15T14:12:14.088 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:12 smithi081 ceph-mon[36434]: pgmap v373: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 985 KiB/s wr, 73 op/s 2024-05-15T14:12:14.088 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:12 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/3261472208' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-15T14:12:14.088 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:11 smithi052 ceph-mon[36190]: from='client.16821 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T14:12:14.088 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:11 smithi052 ceph-mon[36190]: osdmap e98: 12 total, 12 up, 12 in 2024-05-15T14:12:14.088 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:11 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/1084167140' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-15T14:12:14.089 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:11 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/1134281336' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-15T14:12:14.089 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:11 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/2114073653' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-15T14:12:14.089 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:12 smithi052 ceph-mon[36190]: pgmap v373: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 985 KiB/s wr, 73 op/s 2024-05-15T14:12:14.089 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:12 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/3261472208' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-15T14:12:14.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.597+0000 7f00ee304640 1 -- 172.21.15.33:0/3261472208 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f00d00b2960 msgr2=0x7f00d00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:14.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.597+0000 7f00ee304640 1 --2- 172.21.15.33:0/3261472208 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f00d00b2960 0x7f00d00b4e20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f00dc000c00 tx=0x7f00dc031400 comp rx=0 tx=0).stop 2024-05-15T14:12:14.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.597+0000 7f00ee304640 1 -- 172.21.15.33:0/3261472208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00e8101ba0 msgr2=0x7f00e81981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:14.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.597+0000 7f00ee304640 1 --2- 172.21.15.33:0/3261472208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00e8101ba0 0x7f00e81981f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f00d800d950 tx=0x7f00d800de10 comp rx=0 tx=0).stop 2024-05-15T14:12:14.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.597+0000 7f00ee304640 1 -- 172.21.15.33:0/3261472208 shutdown_connections 2024-05-15T14:12:14.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.597+0000 7f00ee304640 1 --2- 172.21.15.33:0/3261472208 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f00d00b2960 0x7f00d00b4e20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:14.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.597+0000 7f00ee304640 1 --2- 172.21.15.33:0/3261472208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f00e8106950 0x7f00e819cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:14.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.597+0000 7f00ee304640 1 --2- 172.21.15.33:0/3261472208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f00e8102560 0x7f00e8198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:14.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.597+0000 7f00ee304640 1 --2- 172.21.15.33:0/3261472208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00e8101ba0 0x7f00e81981f0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:14.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.597+0000 7f00ee304640 1 -- 172.21.15.33:0/3261472208 >> 172.21.15.33:0/3261472208 conn(0x7f00e80787a0 msgr2=0x7f00e80ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:14.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.597+0000 7f00ee304640 1 -- 172.21.15.33:0/3261472208 shutdown_connections 2024-05-15T14:12:14.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:11.597+0000 7f00ee304640 1 -- 172.21.15.33:0/3261472208 wait complete. 2024-05-15T14:12:14.098 DEBUG:tasks.check_counter:Getting stats from a 2024-05-15T14:12:14.099 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-15T14:12:14.099 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:11 smithi033 ceph-mon[31650]: from='client.16821 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-15T14:12:14.099 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:11 smithi033 ceph-mon[31650]: osdmap e98: 12 total, 12 up, 12 in 2024-05-15T14:12:14.100 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:11 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/1084167140' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-15T14:12:14.100 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:11 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/1134281336' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-15T14:12:14.100 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:11 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/2114073653' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-15T14:12:14.100 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:12 smithi033 ceph-mon[31650]: pgmap v373: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 985 KiB/s wr, 73 op/s 2024-05-15T14:12:14.100 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:12 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/3261472208' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-15T14:12:14.364 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:12:14.750 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:14 smithi033 ceph-mon[31650]: pgmap v374: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 902 KiB/s wr, 75 op/s 2024-05-15T14:12:14.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:14 smithi052 ceph-mon[36190]: pgmap v374: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 902 KiB/s wr, 75 op/s 2024-05-15T14:12:14.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:14 smithi081 ceph-mon[36434]: pgmap v374: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 902 KiB/s wr, 75 op/s 2024-05-15T14:12:16.036 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T14:12:16.036 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-05-15T14:12:16.036 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 260, 2024-05-15T14:12:16.037 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 247, 2024-05-15T14:12:16.037 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 1723712, 2024-05-15T14:12:16.037 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 151248, 2024-05-15T14:12:16.037 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 4, 2024-05-15T14:12:16.037 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 0, 2024-05-15T14:12:16.037 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.047883352, 2024-05-15T14:12:16.037 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.013907786, 2024-05-15T14:12:16.037 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.038060057, 2024-05-15T14:12:16.038 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.004606542, 2024-05-15T14:12:16.038 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:16.038 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 247, 2024-05-15T14:12:16.038 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.011162466, 2024-05-15T14:12:16.038 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000045192 2024-05-15T14:12:16.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:16.038 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:16.038 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-15T14:12:16.039 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-15T14:12:16.039 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-15T14:12:16.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:16.039 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 1723712, 2024-05-15T14:12:16.039 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 151248 2024-05-15T14:12:16.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:16.039 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-05-15T14:12:16.039 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 2, 2024-05-15T14:12:16.039 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 187, 2024-05-15T14:12:16.040 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 320, 2024-05-15T14:12:16.040 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 190304, 2024-05-15T14:12:16.040 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 5, 2024-05-15T14:12:16.040 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 3, 2024-05-15T14:12:16.040 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.014812492, 2024-05-15T14:12:16.040 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.010145975, 2024-05-15T14:12:16.040 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.001793516, 2024-05-15T14:12:16.040 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-15T14:12:16.040 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:16.041 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 184, 2024-05-15T14:12:16.041 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.018482666, 2024-05-15T14:12:16.041 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000100449 2024-05-15T14:12:16.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:16.041 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:16.041 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-15T14:12:16.041 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-15T14:12:16.041 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-15T14:12:16.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:16.042 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-15T14:12:16.042 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 190304 2024-05-15T14:12:16.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:16.042 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-05-15T14:12:16.042 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 0, 2024-05-15T14:12:16.042 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 2, 2024-05-15T14:12:16.042 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 0, 2024-05-15T14:12:16.042 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 0, 2024-05-15T14:12:16.042 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 2, 2024-05-15T14:12:16.043 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 1, 2024-05-15T14:12:16.043 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.000496626, 2024-05-15T14:12:16.043 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.000000000, 2024-05-15T14:12:16.043 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.000413808, 2024-05-15T14:12:16.043 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-15T14:12:16.043 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:16.043 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-15T14:12:16.043 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-15T14:12:16.044 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-15T14:12:16.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:16.044 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:16.044 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-15T14:12:16.044 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-15T14:12:16.044 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-15T14:12:16.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:16.044 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-15T14:12:16.044 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 0 2024-05-15T14:12:16.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:16.045 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-05-15T14:12:16.045 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 0, 2024-05-15T14:12:16.045 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-05-15T14:12:16.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:16.045 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-05-15T14:12:16.045 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-05-15T14:12:16.045 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-05-15T14:12:16.045 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-05-15T14:12:16.046 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-05-15T14:12:16.046 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-05-15T14:12:16.046 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-05-15T14:12:16.046 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-15T14:12:16.046 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-05-15T14:12:16.046 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-15T14:12:16.046 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-05-15T14:12:16.046 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-05-15T14:12:16.046 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-05-15T14:12:16.047 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-15T14:12:16.047 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-05-15T14:12:16.047 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-05-15T14:12:16.047 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-05-15T14:12:16.047 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-05-15T14:12:16.047 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-05-15T14:12:16.048 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-15T14:12:16.048 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-05-15T14:12:16.048 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-15T14:12:16.048 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-05-15T14:12:16.048 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-05-15T14:12:16.048 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-05-15T14:12:16.048 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-05-15T14:12:16.048 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-05-15T14:12:16.048 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-15T14:12:16.049 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-05-15T14:12:16.049 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-05-15T14:12:16.049 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-05-15T14:12:16.049 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-05-15T14:12:16.049 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-05-15T14:12:16.049 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-05-15T14:12:16.049 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-05-15T14:12:16.049 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-05-15T14:12:16.050 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-05-15T14:12:16.050 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 148731, 2024-05-15T14:12:16.050 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 31, 2024-05-15T14:12:16.050 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-05-15T14:12:16.050 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-05-15T14:12:16.050 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-05-15T14:12:16.050 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-05-15T14:12:16.050 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-05-15T14:12:16.051 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-05-15T14:12:16.051 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-05-15T14:12:16.051 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-05-15T14:12:16.051 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 0, 2024-05-15T14:12:16.051 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 0, 2024-05-15T14:12:16.051 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-05-15T14:12:16.051 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-05-15T14:12:16.051 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-05-15T14:12:16.051 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-05-15T14:12:16.052 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 928, 2024-05-15T14:12:16.052 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 2, 2024-05-15T14:12:16.052 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-05-15T14:12:16.052 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-05-15T14:12:16.052 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-05-15T14:12:16.052 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-05-15T14:12:16.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:16.053 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-15T14:12:16.053 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-05-15T14:12:16.053 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-05-15T14:12:16.053 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-05-15T14:12:16.053 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 258, 2024-05-15T14:12:16.053 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 1684590, 2024-05-15T14:12:16.053 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:16.053 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 258, 2024-05-15T14:12:16.053 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-05-15T14:12:16.054 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-05-15T14:12:16.054 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 258, 2024-05-15T14:12:16.054 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 1684590, 2024-05-15T14:12:16.054 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-05-15T14:12:16.054 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-15T14:12:16.054 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-15T14:12:16.054 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-15T14:12:16.054 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:16.054 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:16.055 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T14:12:16.692 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-05-15T14:12:16.692 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-05-15T14:12:16.693 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-05-15T14:12:16.693 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-05-15T14:12:16.693 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-05-15T14:12:16.693 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-05-15T14:12:16.693 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-05-15T14:12:16.693 DEBUG:tasks.check_counter:Getting stats from d 2024-05-15T14:12:16.694 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-15T14:12:16.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:16 smithi052 ceph-mon[36190]: pgmap v375: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 805 KiB/s wr, 67 op/s 2024-05-15T14:12:16.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:16 smithi081 ceph-mon[36434]: pgmap v375: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 805 KiB/s wr, 67 op/s 2024-05-15T14:12:16.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:16 smithi033 ceph-mon[31650]: pgmap v375: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 805 KiB/s wr, 67 op/s 2024-05-15T14:12:16.956 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:12:18.514 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T14:12:18.514 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-05-15T14:12:18.514 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 0, 2024-05-15T14:12:18.514 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 2, 2024-05-15T14:12:18.514 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 0, 2024-05-15T14:12:18.514 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 0, 2024-05-15T14:12:18.515 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 4, 2024-05-15T14:12:18.515 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 0, 2024-05-15T14:12:18.515 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.001455374, 2024-05-15T14:12:18.515 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.000000000, 2024-05-15T14:12:18.515 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.000377981, 2024-05-15T14:12:18.515 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-15T14:12:18.515 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:18.515 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-15T14:12:18.516 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-15T14:12:18.516 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-15T14:12:18.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:18.516 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:18.516 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-15T14:12:18.516 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-15T14:12:18.516 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-15T14:12:18.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:18.516 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-15T14:12:18.517 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 0 2024-05-15T14:12:18.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:18.517 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-05-15T14:12:18.517 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 266, 2024-05-15T14:12:18.517 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 253, 2024-05-15T14:12:18.517 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 1725344, 2024-05-15T14:12:18.518 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 155088, 2024-05-15T14:12:18.518 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 2, 2024-05-15T14:12:18.518 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 1, 2024-05-15T14:12:18.518 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.050209544, 2024-05-15T14:12:18.518 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.015191249, 2024-05-15T14:12:18.518 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.041205531, 2024-05-15T14:12:18.518 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.004659003, 2024-05-15T14:12:18.518 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:18.519 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 253, 2024-05-15T14:12:18.519 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.011410003, 2024-05-15T14:12:18.519 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000045098 2024-05-15T14:12:18.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:18.519 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:18.519 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-15T14:12:18.519 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-15T14:12:18.519 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-15T14:12:18.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:18.520 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 1725344, 2024-05-15T14:12:18.520 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 155088 2024-05-15T14:12:18.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:18.520 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-05-15T14:12:18.520 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 2, 2024-05-15T14:12:18.520 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 192, 2024-05-15T14:12:18.520 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 320, 2024-05-15T14:12:18.520 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 191184, 2024-05-15T14:12:18.520 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 5, 2024-05-15T14:12:18.521 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 3, 2024-05-15T14:12:18.521 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.015012452, 2024-05-15T14:12:18.521 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.010361221, 2024-05-15T14:12:18.521 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.001957790, 2024-05-15T14:12:18.521 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-15T14:12:18.521 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:18.521 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 189, 2024-05-15T14:12:18.521 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.011589404, 2024-05-15T14:12:18.522 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000061319 2024-05-15T14:12:18.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:18.522 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:18.522 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-15T14:12:18.522 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-15T14:12:18.522 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-15T14:12:18.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:18.522 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-15T14:12:18.522 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 191184 2024-05-15T14:12:18.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:18.523 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-05-15T14:12:18.523 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 0, 2024-05-15T14:12:18.523 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-05-15T14:12:18.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:18.523 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-05-15T14:12:18.523 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-05-15T14:12:18.523 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-05-15T14:12:18.523 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-05-15T14:12:18.524 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-05-15T14:12:18.524 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-05-15T14:12:18.524 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-05-15T14:12:18.524 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-15T14:12:18.524 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-05-15T14:12:18.524 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-15T14:12:18.525 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-05-15T14:12:18.525 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-05-15T14:12:18.525 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-05-15T14:12:18.525 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-15T14:12:18.525 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-05-15T14:12:18.525 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-05-15T14:12:18.525 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-05-15T14:12:18.525 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-05-15T14:12:18.525 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-05-15T14:12:18.526 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-15T14:12:18.526 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-05-15T14:12:18.526 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-15T14:12:18.526 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-05-15T14:12:18.526 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-05-15T14:12:18.526 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-05-15T14:12:18.526 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-05-15T14:12:18.526 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-05-15T14:12:18.527 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-15T14:12:18.527 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-05-15T14:12:18.527 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-05-15T14:12:18.527 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-05-15T14:12:18.527 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-05-15T14:12:18.527 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-05-15T14:12:18.527 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-05-15T14:12:18.527 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-05-15T14:12:18.528 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-05-15T14:12:18.528 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-05-15T14:12:18.528 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 148748, 2024-05-15T14:12:18.528 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 32, 2024-05-15T14:12:18.528 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-05-15T14:12:18.528 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-05-15T14:12:18.528 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-05-15T14:12:18.528 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-05-15T14:12:18.528 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-05-15T14:12:18.529 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-05-15T14:12:18.530 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-05-15T14:12:18.530 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-05-15T14:12:18.530 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 0, 2024-05-15T14:12:18.530 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 0, 2024-05-15T14:12:18.530 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-05-15T14:12:18.531 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-05-15T14:12:18.531 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-05-15T14:12:18.531 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-05-15T14:12:18.531 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 928, 2024-05-15T14:12:18.531 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 2, 2024-05-15T14:12:18.531 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-05-15T14:12:18.531 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-05-15T14:12:18.531 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-05-15T14:12:18.531 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-05-15T14:12:18.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:18.532 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-15T14:12:18.532 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-05-15T14:12:18.532 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-05-15T14:12:18.532 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-05-15T14:12:18.532 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 264, 2024-05-15T14:12:18.532 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 1685370, 2024-05-15T14:12:18.533 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:18.533 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 264, 2024-05-15T14:12:18.533 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-05-15T14:12:18.533 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-05-15T14:12:18.533 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 264, 2024-05-15T14:12:18.533 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 1685370, 2024-05-15T14:12:18.533 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-05-15T14:12:18.533 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-15T14:12:18.533 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-15T14:12:18.534 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-15T14:12:18.534 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:18.534 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:18.534 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T14:12:18.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:18 smithi052 ceph-mon[36190]: pgmap v376: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 KiB/s wr, 14 op/s 2024-05-15T14:12:18.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:18 smithi081 ceph-mon[36434]: pgmap v376: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 KiB/s wr, 14 op/s 2024-05-15T14:12:18.878 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:18 smithi033 ceph-mon[31650]: pgmap v376: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 KiB/s wr, 14 op/s 2024-05-15T14:12:19.161 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-15T14:12:19.161 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-15T14:12:19.162 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-05-15T14:12:19.162 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-05-15T14:12:19.162 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-05-15T14:12:19.162 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-05-15T14:12:19.163 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-05-15T14:12:19.163 DEBUG:tasks.check_counter:Getting stats from g 2024-05-15T14:12:19.163 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-15T14:12:19.417 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:12:20.766 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:20 smithi033 ceph-mon[31650]: pgmap v377: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 KiB/s wr, 14 op/s 2024-05-15T14:12:20.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:20 smithi052 ceph-mon[36190]: pgmap v377: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 KiB/s wr, 14 op/s 2024-05-15T14:12:20.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:20 smithi081 ceph-mon[36434]: pgmap v377: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 KiB/s wr, 14 op/s 2024-05-15T14:12:21.026 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T14:12:21.026 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-05-15T14:12:21.026 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 0, 2024-05-15T14:12:21.027 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 2, 2024-05-15T14:12:21.027 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 0, 2024-05-15T14:12:21.027 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 0, 2024-05-15T14:12:21.027 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 4, 2024-05-15T14:12:21.027 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 0, 2024-05-15T14:12:21.027 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.001458224, 2024-05-15T14:12:21.027 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.000000000, 2024-05-15T14:12:21.028 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.000340863, 2024-05-15T14:12:21.028 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-15T14:12:21.028 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:21.028 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-15T14:12:21.028 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-15T14:12:21.028 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-15T14:12:21.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:21.028 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:21.029 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-15T14:12:21.029 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-15T14:12:21.029 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-15T14:12:21.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:21.029 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-15T14:12:21.029 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 0 2024-05-15T14:12:21.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:21.029 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-05-15T14:12:21.030 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 6, 2024-05-15T14:12:21.030 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 191, 2024-05-15T14:12:21.030 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 3312, 2024-05-15T14:12:21.030 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 191104, 2024-05-15T14:12:21.030 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 5, 2024-05-15T14:12:21.030 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 3, 2024-05-15T14:12:21.030 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.014755632, 2024-05-15T14:12:21.030 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.009826464, 2024-05-15T14:12:21.030 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.002338477, 2024-05-15T14:12:21.031 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-15T14:12:21.031 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:21.031 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 189, 2024-05-15T14:12:21.031 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.020469167, 2024-05-15T14:12:21.031 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000108302 2024-05-15T14:12:21.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:21.031 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:21.031 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-15T14:12:21.032 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-15T14:12:21.032 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-15T14:12:21.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:21.032 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-05-15T14:12:21.032 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 191104 2024-05-15T14:12:21.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:21.032 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-05-15T14:12:21.032 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 258, 2024-05-15T14:12:21.033 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 247, 2024-05-15T14:12:21.033 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 1721264, 2024-05-15T14:12:21.033 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 152080, 2024-05-15T14:12:21.033 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 2, 2024-05-15T14:12:21.033 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 1, 2024-05-15T14:12:21.033 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.048726724, 2024-05-15T14:12:21.033 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.015399130, 2024-05-15T14:12:21.033 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.038111915, 2024-05-15T14:12:21.034 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.004618825, 2024-05-15T14:12:21.034 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:21.034 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 246, 2024-05-15T14:12:21.034 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.010623279, 2024-05-15T14:12:21.034 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000043184 2024-05-15T14:12:21.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:21.034 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:21.034 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-15T14:12:21.034 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-15T14:12:21.035 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-15T14:12:21.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:21.035 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 1721264, 2024-05-15T14:12:21.035 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 152080 2024-05-15T14:12:21.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:21.035 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-05-15T14:12:21.035 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 0, 2024-05-15T14:12:21.035 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-05-15T14:12:21.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:21.036 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-05-15T14:12:21.036 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-05-15T14:12:21.036 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-05-15T14:12:21.036 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-05-15T14:12:21.036 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-05-15T14:12:21.036 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-05-15T14:12:21.036 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-05-15T14:12:21.036 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-15T14:12:21.037 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-05-15T14:12:21.037 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-15T14:12:21.037 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-05-15T14:12:21.037 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-05-15T14:12:21.037 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-05-15T14:12:21.037 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-15T14:12:21.037 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-05-15T14:12:21.037 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-05-15T14:12:21.038 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-05-15T14:12:21.038 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-05-15T14:12:21.038 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-05-15T14:12:21.038 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-15T14:12:21.038 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-05-15T14:12:21.039 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-15T14:12:21.039 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-05-15T14:12:21.039 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-05-15T14:12:21.039 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-05-15T14:12:21.039 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-05-15T14:12:21.039 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-05-15T14:12:21.039 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-15T14:12:21.039 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-05-15T14:12:21.040 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-05-15T14:12:21.040 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-05-15T14:12:21.040 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-05-15T14:12:21.040 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-05-15T14:12:21.040 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-05-15T14:12:21.040 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-05-15T14:12:21.040 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-05-15T14:12:21.040 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-05-15T14:12:21.041 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 148754, 2024-05-15T14:12:21.041 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 31, 2024-05-15T14:12:21.041 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-05-15T14:12:21.041 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-05-15T14:12:21.041 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-05-15T14:12:21.041 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-05-15T14:12:21.041 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-05-15T14:12:21.041 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-05-15T14:12:21.042 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-05-15T14:12:21.042 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-05-15T14:12:21.042 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 0, 2024-05-15T14:12:21.042 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 0, 2024-05-15T14:12:21.042 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-05-15T14:12:21.042 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-05-15T14:12:21.042 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-05-15T14:12:21.042 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-05-15T14:12:21.043 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 928, 2024-05-15T14:12:21.043 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 2, 2024-05-15T14:12:21.043 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-05-15T14:12:21.043 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-05-15T14:12:21.043 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-05-15T14:12:21.043 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-05-15T14:12:21.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:21.043 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-15T14:12:21.044 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-05-15T14:12:21.044 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-05-15T14:12:21.044 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-05-15T14:12:21.044 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 260, 2024-05-15T14:12:21.044 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 1684850, 2024-05-15T14:12:21.044 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:21.044 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 260, 2024-05-15T14:12:21.044 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-05-15T14:12:21.044 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-05-15T14:12:21.045 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 260, 2024-05-15T14:12:21.045 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 1684850, 2024-05-15T14:12:21.045 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-05-15T14:12:21.045 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-15T14:12:21.045 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-15T14:12:21.045 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-15T14:12:21.045 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:21.045 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:21.046 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T14:12:21.650 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-05-15T14:12:21.650 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-05-15T14:12:21.650 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-05-15T14:12:21.650 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-05-15T14:12:21.650 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-05-15T14:12:21.651 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-05-15T14:12:21.651 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-05-15T14:12:21.651 DEBUG:tasks.check_counter:Getting stats from j 2024-05-15T14:12:21.651 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-15T14:12:21.904 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:12:22.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:22 smithi033 ceph-mon[31650]: pgmap v378: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-05-15T14:12:22.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:22 smithi052 ceph-mon[36190]: pgmap v378: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-05-15T14:12:22.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:22 smithi081 ceph-mon[36434]: pgmap v378: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-05-15T14:12:23.672 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-15T14:12:23.672 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-05-15T14:12:23.672 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 265, 2024-05-15T14:12:23.672 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 254, 2024-05-15T14:12:23.672 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 1723168, 2024-05-15T14:12:23.673 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 156560, 2024-05-15T14:12:23.673 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 4, 2024-05-15T14:12:23.673 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 0, 2024-05-15T14:12:23.673 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.048116173, 2024-05-15T14:12:23.673 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.014810125, 2024-05-15T14:12:23.673 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.038460236, 2024-05-15T14:12:23.673 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.004789621, 2024-05-15T14:12:23.673 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:23.674 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 253, 2024-05-15T14:12:23.674 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.011070458, 2024-05-15T14:12:23.674 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000043756 2024-05-15T14:12:23.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:23.674 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:23.674 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-15T14:12:23.674 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-15T14:12:23.674 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-15T14:12:23.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:23.675 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 1723168, 2024-05-15T14:12:23.675 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 156560 2024-05-15T14:12:23.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:23.675 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-05-15T14:12:23.675 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 2, 2024-05-15T14:12:23.675 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 194, 2024-05-15T14:12:23.675 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 320, 2024-05-15T14:12:23.675 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 191536, 2024-05-15T14:12:23.676 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 5, 2024-05-15T14:12:23.676 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 3, 2024-05-15T14:12:23.676 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.015516314, 2024-05-15T14:12:23.676 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.010626838, 2024-05-15T14:12:23.676 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.002216669, 2024-05-15T14:12:23.676 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-15T14:12:23.676 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:23.676 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 191, 2024-05-15T14:12:23.676 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.011462265, 2024-05-15T14:12:23.677 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000060011 2024-05-15T14:12:23.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:23.677 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:23.677 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-15T14:12:23.677 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-15T14:12:23.677 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-15T14:12:23.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:23.677 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-15T14:12:23.678 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 191536 2024-05-15T14:12:23.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:23.678 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-05-15T14:12:23.678 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 4, 2024-05-15T14:12:23.678 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 4, 2024-05-15T14:12:23.678 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 2992, 2024-05-15T14:12:23.678 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 448, 2024-05-15T14:12:23.678 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 2, 2024-05-15T14:12:23.678 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 1, 2024-05-15T14:12:23.679 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.000705206, 2024-05-15T14:12:23.679 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.000040492, 2024-05-15T14:12:23.679 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.000631413, 2024-05-15T14:12:23.679 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-15T14:12:23.679 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:23.679 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 3, 2024-05-15T14:12:23.679 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000837770, 2024-05-15T14:12:23.679 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000279256 2024-05-15T14:12:23.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:23.680 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:23.680 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-15T14:12:23.680 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-15T14:12:23.680 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-15T14:12:23.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:23.680 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-15T14:12:23.680 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 448 2024-05-15T14:12:23.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:23.681 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-05-15T14:12:23.681 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 0, 2024-05-15T14:12:23.681 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-05-15T14:12:23.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:23.681 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-05-15T14:12:23.681 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-05-15T14:12:23.681 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-05-15T14:12:23.681 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-05-15T14:12:23.682 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-05-15T14:12:23.682 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-05-15T14:12:23.682 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-05-15T14:12:23.682 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-15T14:12:23.682 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-05-15T14:12:23.682 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-15T14:12:23.682 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-05-15T14:12:23.682 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-05-15T14:12:23.683 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-05-15T14:12:23.683 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-15T14:12:23.683 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-05-15T14:12:23.683 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-05-15T14:12:23.683 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-05-15T14:12:23.683 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-05-15T14:12:23.683 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-05-15T14:12:23.683 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-15T14:12:23.683 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-05-15T14:12:23.684 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-15T14:12:23.684 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-05-15T14:12:23.684 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-05-15T14:12:23.684 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-05-15T14:12:23.684 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-05-15T14:12:23.684 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-05-15T14:12:23.684 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-15T14:12:23.684 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-05-15T14:12:23.684 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-05-15T14:12:23.685 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-05-15T14:12:23.685 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-05-15T14:12:23.685 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-05-15T14:12:23.685 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-05-15T14:12:23.685 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-05-15T14:12:23.685 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-05-15T14:12:23.685 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-05-15T14:12:23.685 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 148731, 2024-05-15T14:12:23.686 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 31, 2024-05-15T14:12:23.686 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-05-15T14:12:23.686 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-05-15T14:12:23.686 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-05-15T14:12:23.686 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-05-15T14:12:23.686 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-05-15T14:12:23.686 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-05-15T14:12:23.687 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-05-15T14:12:23.687 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-05-15T14:12:23.687 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 0, 2024-05-15T14:12:23.687 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 0, 2024-05-15T14:12:23.687 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-05-15T14:12:23.687 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-05-15T14:12:23.687 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-05-15T14:12:23.687 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-05-15T14:12:23.687 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 928, 2024-05-15T14:12:23.688 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 2, 2024-05-15T14:12:23.688 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-05-15T14:12:23.688 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-05-15T14:12:23.688 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-05-15T14:12:23.688 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-05-15T14:12:23.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-15T14:12:23.688 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-15T14:12:23.688 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-05-15T14:12:23.688 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-05-15T14:12:23.689 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-05-15T14:12:23.689 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 267, 2024-05-15T14:12:23.689 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 1685760, 2024-05-15T14:12:23.689 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:23.689 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 267, 2024-05-15T14:12:23.689 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-05-15T14:12:23.689 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-05-15T14:12:23.689 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 267, 2024-05-15T14:12:23.689 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 1685760, 2024-05-15T14:12:23.690 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-05-15T14:12:23.690 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-15T14:12:23.690 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-15T14:12:23.690 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-15T14:12:23.690 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:23.690 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-15T14:12:23.690 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-15T14:12:24.297 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-15T14:12:24.297 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-15T14:12:24.297 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-05-15T14:12:24.297 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-05-15T14:12:24.297 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-05-15T14:12:24.297 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-05-15T14:12:24.298 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-05-15T14:12:24.298 DEBUG:tasks.check_counter:Getting stats from b 2024-05-15T14:12:24.298 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-15T14:12:24.554 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.b/config 2024-05-15T14:12:24.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:24 smithi052 ceph-mon[36190]: pgmap v379: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 37 KiB/s wr, 17 op/s 2024-05-15T14:12:24.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:24 smithi081 ceph-mon[36434]: pgmap v379: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 37 KiB/s wr, 17 op/s 2024-05-15T14:12:24.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:24 smithi033 ceph-mon[31650]: pgmap v379: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 37 KiB/s wr, 17 op/s 2024-05-15T14:12:25.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:25 smithi052 ceph-mon[36190]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:12:25.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:25 smithi081 ceph-mon[36434]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:12:25.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:25 smithi033 ceph-mon[31650]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:12:26.484 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-15T14:12:26.770 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-0": { 2024-05-15T14:12:26.770 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 131604, 2024-05-15T14:12:26.770 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 104178, 2024-05-15T14:12:26.770 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 38382774, 2024-05-15T14:12:26.770 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 64895626, 2024-05-15T14:12:26.770 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 10, 2024-05-15T14:12:26.770 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 4, 2024-05-15T14:12:26.771 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 6.145467162, 2024-05-15T14:12:26.771 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 2.316301755, 2024-05-15T14:12:26.771 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 4.452672753, 2024-05-15T14:12:26.771 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.315589062, 2024-05-15T14:12:26.771 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:26.771 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 104178, 2024-05-15T14:12:26.771 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 1.784871294, 2024-05-15T14:12:26.771 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000017132 2024-05-15T14:12:26.772 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.772 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:26.772 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 212732, 2024-05-15T14:12:26.772 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.027785738, 2024-05-15T14:12:26.772 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000130 2024-05-15T14:12:26.772 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.772 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-15T14:12:26.772 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 0 2024-05-15T14:12:26.773 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.773 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-1": { 2024-05-15T14:12:26.773 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 10111, 2024-05-15T14:12:26.773 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 10030, 2024-05-15T14:12:26.773 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 4322143, 2024-05-15T14:12:26.773 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 41021885, 2024-05-15T14:12:26.773 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 9, 2024-05-15T14:12:26.773 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 7, 2024-05-15T14:12:26.774 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.728734771, 2024-05-15T14:12:26.774 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.222787674, 2024-05-15T14:12:26.774 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.238035694, 2024-05-15T14:12:26.774 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.237058257, 2024-05-15T14:12:26.774 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:26.774 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 10031, 2024-05-15T14:12:26.774 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.307120927, 2024-05-15T14:12:26.774 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000030617 2024-05-15T14:12:26.775 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.775 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:26.775 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 1828, 2024-05-15T14:12:26.775 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000291793, 2024-05-15T14:12:26.775 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000159 2024-05-15T14:12:26.775 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.775 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 1833056, 2024-05-15T14:12:26.775 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 167328 2024-05-15T14:12:26.775 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.776 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-2": { 2024-05-15T14:12:26.776 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 91953, 2024-05-15T14:12:26.776 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 143364, 2024-05-15T14:12:26.776 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 29890331, 2024-05-15T14:12:26.776 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 124308665, 2024-05-15T14:12:26.776 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 18, 2024-05-15T14:12:26.776 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 9, 2024-05-15T14:12:26.776 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 5.323348628, 2024-05-15T14:12:26.777 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 2.263446193, 2024-05-15T14:12:26.777 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 3.220515865, 2024-05-15T14:12:26.777 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.228434184, 2024-05-15T14:12:26.777 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:26.777 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 143358, 2024-05-15T14:12:26.777 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 2.795964134, 2024-05-15T14:12:26.777 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000019503 2024-05-15T14:12:26.777 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.778 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:26.778 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 201136, 2024-05-15T14:12:26.778 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.030699399, 2024-05-15T14:12:26.778 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000152 2024-05-15T14:12:26.778 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.778 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 5168, 2024-05-15T14:12:26.778 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 497232 2024-05-15T14:12:26.778 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.779 INFO:teuthology.orchestra.run.smithi052.stdout: "cct": { 2024-05-15T14:12:26.779 INFO:teuthology.orchestra.run.smithi052.stdout: "total_workers": 1, 2024-05-15T14:12:26.779 INFO:teuthology.orchestra.run.smithi052.stdout: "unhealthy_workers": 0 2024-05-15T14:12:26.779 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.779 INFO:teuthology.orchestra.run.smithi052.stdout: "finisher-MDSRank": { 2024-05-15T14:12:26.779 INFO:teuthology.orchestra.run.smithi052.stdout: "queue_len": 0, 2024-05-15T14:12:26.779 INFO:teuthology.orchestra.run.smithi052.stdout: "complete_latency": { 2024-05-15T14:12:26.779 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 31324, 2024-05-15T14:12:26.780 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 16.883975600, 2024-05-15T14:12:26.780 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000539010 2024-05-15T14:12:26.780 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-15T14:12:26.780 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.780 INFO:teuthology.orchestra.run.smithi052.stdout: "finisher-PurgeQueue": { 2024-05-15T14:12:26.780 INFO:teuthology.orchestra.run.smithi052.stdout: "queue_len": 0, 2024-05-15T14:12:26.780 INFO:teuthology.orchestra.run.smithi052.stdout: "complete_latency": { 2024-05-15T14:12:26.780 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 17344, 2024-05-15T14:12:26.781 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 1.178115197, 2024-05-15T14:12:26.781 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000067926 2024-05-15T14:12:26.781 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-15T14:12:26.781 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.781 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": { 2024-05-15T14:12:26.781 INFO:teuthology.orchestra.run.smithi052.stdout: "request": 64910, 2024-05-15T14:12:26.781 INFO:teuthology.orchestra.run.smithi052.stdout: "reply": 58228, 2024-05-15T14:12:26.781 INFO:teuthology.orchestra.run.smithi052.stdout: "reply_latency": { 2024-05-15T14:12:26.782 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 58228, 2024-05-15T14:12:26.782 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 72.512547017, 2024-05-15T14:12:26.782 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.001245320 2024-05-15T14:12:26.782 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.782 INFO:teuthology.orchestra.run.smithi052.stdout: "slow_reply": 0, 2024-05-15T14:12:26.782 INFO:teuthology.orchestra.run.smithi052.stdout: "forward": 6682, 2024-05-15T14:12:26.782 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_fetch_complete": 0, 2024-05-15T14:12:26.782 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_fetch_keys": 0, 2024-05-15T14:12:26.782 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_commit": 11, 2024-05-15T14:12:26.783 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_split": 0, 2024-05-15T14:12:26.783 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_merge": 0, 2024-05-15T14:12:26.783 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes": 370, 2024-05-15T14:12:26.783 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_top": 23, 2024-05-15T14:12:26.783 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_bottom": 347, 2024-05-15T14:12:26.783 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_pin_tail": 0, 2024-05-15T14:12:26.783 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_pinned": 336, 2024-05-15T14:12:26.783 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_expired": 0, 2024-05-15T14:12:26.784 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_with_caps": 29, 2024-05-15T14:12:26.784 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": 29, 2024-05-15T14:12:26.784 INFO:teuthology.orchestra.run.smithi052.stdout: "subtrees": 6, 2024-05-15T14:12:26.784 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse": 128169, 2024-05-15T14:12:26.784 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_hit": 57575, 2024-05-15T14:12:26.784 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_forward": 6682, 2024-05-15T14:12:26.784 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_discover": 25, 2024-05-15T14:12:26.785 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_dir_fetch": 0, 2024-05-15T14:12:26.785 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_remote_ino": 0, 2024-05-15T14:12:26.785 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_lock": 1433, 2024-05-15T14:12:26.785 INFO:teuthology.orchestra.run.smithi052.stdout: "load_cent": 14701, 2024-05-15T14:12:26.785 INFO:teuthology.orchestra.run.smithi052.stdout: "q": 1, 2024-05-15T14:12:26.785 INFO:teuthology.orchestra.run.smithi052.stdout: "exported": 0, 2024-05-15T14:12:26.785 INFO:teuthology.orchestra.run.smithi052.stdout: "exported_inodes": 0, 2024-05-15T14:12:26.786 INFO:teuthology.orchestra.run.smithi052.stdout: "imported": 4, 2024-05-15T14:12:26.786 INFO:teuthology.orchestra.run.smithi052.stdout: "imported_inodes": 119, 2024-05-15T14:12:26.786 INFO:teuthology.orchestra.run.smithi052.stdout: "openino_dir_fetch": 0, 2024-05-15T14:12:26.786 INFO:teuthology.orchestra.run.smithi052.stdout: "openino_backtrace_fetch": 0, 2024-05-15T14:12:26.786 INFO:teuthology.orchestra.run.smithi052.stdout: "openino_peer_discover": 0, 2024-05-15T14:12:26.786 INFO:teuthology.orchestra.run.smithi052.stdout: "root_rfiles": 67, 2024-05-15T14:12:26.786 INFO:teuthology.orchestra.run.smithi052.stdout: "root_rbytes": 21137952, 2024-05-15T14:12:26.786 INFO:teuthology.orchestra.run.smithi052.stdout: "root_rsnaps": 6, 2024-05-15T14:12:26.787 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_backtrace_fetch": 0, 2024-05-15T14:12:26.787 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_set_tag": 0, 2024-05-15T14:12:26.787 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_backtrace_repaired": 0, 2024-05-15T14:12:26.787 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_inotable_repaired": 0, 2024-05-15T14:12:26.787 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_dir_inodes": 0, 2024-05-15T14:12:26.787 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_dir_base_inodes": 0, 2024-05-15T14:12:26.788 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_dirfrag_rstats": 0, 2024-05-15T14:12:26.788 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_file_inodes": 0, 2024-05-15T14:12:26.788 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_inode_file_caps": 0, 2024-05-15T14:12:26.788 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_revoke": 773, 2024-05-15T14:12:26.788 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_grant": 35662, 2024-05-15T14:12:26.788 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_trunc": 0, 2024-05-15T14:12:26.788 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-15T14:12:26.788 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-15T14:12:26.788 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_caps": 10493, 2024-05-15T14:12:26.789 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_caps_dirty": 6131, 2024-05-15T14:12:26.789 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_cap_release": 1725, 2024-05-15T14:12:26.789 INFO:teuthology.orchestra.run.smithi052.stdout: "process_request_cap_release": 56893 2024-05-15T14:12:26.789 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.789 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_cache": { 2024-05-15T14:12:26.789 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_update": 212, 2024-05-15T14:12:26.790 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_update_receipt": 531, 2024-05-15T14:12:26.790 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_try_discover": 272, 2024-05-15T14:12:26.790 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_send_discover": 533, 2024-05-15T14:12:26.790 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_handle_discover": 254, 2024-05-15T14:12:26.790 INFO:teuthology.orchestra.run.smithi052.stdout: "num_strays": 0, 2024-05-15T14:12:26.790 INFO:teuthology.orchestra.run.smithi052.stdout: "num_strays_delayed": 0, 2024-05-15T14:12:26.790 INFO:teuthology.orchestra.run.smithi052.stdout: "num_strays_enqueuing": 0, 2024-05-15T14:12:26.790 INFO:teuthology.orchestra.run.smithi052.stdout: "strays_created": 10187, 2024-05-15T14:12:26.790 INFO:teuthology.orchestra.run.smithi052.stdout: "strays_enqueued": 10187, 2024-05-15T14:12:26.791 INFO:teuthology.orchestra.run.smithi052.stdout: "strays_reintegrated": 0, 2024-05-15T14:12:26.791 INFO:teuthology.orchestra.run.smithi052.stdout: "strays_migrated": 0, 2024-05-15T14:12:26.791 INFO:teuthology.orchestra.run.smithi052.stdout: "num_recovering_processing": 0, 2024-05-15T14:12:26.791 INFO:teuthology.orchestra.run.smithi052.stdout: "num_recovering_enqueued": 0, 2024-05-15T14:12:26.791 INFO:teuthology.orchestra.run.smithi052.stdout: "num_recovering_prioritized": 0, 2024-05-15T14:12:26.791 INFO:teuthology.orchestra.run.smithi052.stdout: "recovery_started": 0, 2024-05-15T14:12:26.791 INFO:teuthology.orchestra.run.smithi052.stdout: "recovery_completed": 0, 2024-05-15T14:12:26.792 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_quiesce_path": 10, 2024-05-15T14:12:26.792 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_quiesce_inode": 349, 2024-05-15T14:12:26.792 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_enqueue_scrub": 0, 2024-05-15T14:12:26.792 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_exportdir": 1, 2024-05-15T14:12:26.792 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_flush": 0, 2024-05-15T14:12:26.792 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_fragmentdir": 0, 2024-05-15T14:12:26.792 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_fragstats": 0, 2024-05-15T14:12:26.792 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_inodestats": 0 2024-05-15T14:12:26.793 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.793 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_log": { 2024-05-15T14:12:26.793 INFO:teuthology.orchestra.run.smithi052.stdout: "evlrg": 0, 2024-05-15T14:12:26.793 INFO:teuthology.orchestra.run.smithi052.stdout: "evadd": 53155, 2024-05-15T14:12:26.793 INFO:teuthology.orchestra.run.smithi052.stdout: "evex": 0, 2024-05-15T14:12:26.793 INFO:teuthology.orchestra.run.smithi052.stdout: "evtrm": 0, 2024-05-15T14:12:26.793 INFO:teuthology.orchestra.run.smithi052.stdout: "ev": 53155, 2024-05-15T14:12:26.793 INFO:teuthology.orchestra.run.smithi052.stdout: "evexg": 0, 2024-05-15T14:12:26.794 INFO:teuthology.orchestra.run.smithi052.stdout: "evexd": 0, 2024-05-15T14:12:26.794 INFO:teuthology.orchestra.run.smithi052.stdout: "segadd": 67, 2024-05-15T14:12:26.794 INFO:teuthology.orchestra.run.smithi052.stdout: "segex": 0, 2024-05-15T14:12:26.794 INFO:teuthology.orchestra.run.smithi052.stdout: "segtrm": 0, 2024-05-15T14:12:26.794 INFO:teuthology.orchestra.run.smithi052.stdout: "seg": 67, 2024-05-15T14:12:26.794 INFO:teuthology.orchestra.run.smithi052.stdout: "segmjr": 6, 2024-05-15T14:12:26.794 INFO:teuthology.orchestra.run.smithi052.stdout: "segexg": 0, 2024-05-15T14:12:26.794 INFO:teuthology.orchestra.run.smithi052.stdout: "segexd": 0, 2024-05-15T14:12:26.795 INFO:teuthology.orchestra.run.smithi052.stdout: "expos": 4194304, 2024-05-15T14:12:26.795 INFO:teuthology.orchestra.run.smithi052.stdout: "wrpos": 98457218, 2024-05-15T14:12:26.795 INFO:teuthology.orchestra.run.smithi052.stdout: "rdpos": 0, 2024-05-15T14:12:26.795 INFO:teuthology.orchestra.run.smithi052.stdout: "jlat": { 2024-05-15T14:12:26.795 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 15807, 2024-05-15T14:12:26.795 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 28.770276878, 2024-05-15T14:12:26.795 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.001820097 2024-05-15T14:12:26.795 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.795 INFO:teuthology.orchestra.run.smithi052.stdout: "replayed": 0 2024-05-15T14:12:26.796 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.796 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_mem": { 2024-05-15T14:12:26.796 INFO:teuthology.orchestra.run.smithi052.stdout: "ino": 307, 2024-05-15T14:12:26.796 INFO:teuthology.orchestra.run.smithi052.stdout: "ino+": 10453, 2024-05-15T14:12:26.796 INFO:teuthology.orchestra.run.smithi052.stdout: "ino-": 10146, 2024-05-15T14:12:26.796 INFO:teuthology.orchestra.run.smithi052.stdout: "dir": 34, 2024-05-15T14:12:26.796 INFO:teuthology.orchestra.run.smithi052.stdout: "dir+": 290, 2024-05-15T14:12:26.796 INFO:teuthology.orchestra.run.smithi052.stdout: "dir-": 256, 2024-05-15T14:12:26.797 INFO:teuthology.orchestra.run.smithi052.stdout: "dn": 370, 2024-05-15T14:12:26.797 INFO:teuthology.orchestra.run.smithi052.stdout: "dn+": 14270, 2024-05-15T14:12:26.797 INFO:teuthology.orchestra.run.smithi052.stdout: "dn-": 13900, 2024-05-15T14:12:26.797 INFO:teuthology.orchestra.run.smithi052.stdout: "cap": 29, 2024-05-15T14:12:26.797 INFO:teuthology.orchestra.run.smithi052.stdout: "cap+": 10584, 2024-05-15T14:12:26.797 INFO:teuthology.orchestra.run.smithi052.stdout: "cap-": 10555, 2024-05-15T14:12:26.797 INFO:teuthology.orchestra.run.smithi052.stdout: "rss": 67384, 2024-05-15T14:12:26.797 INFO:teuthology.orchestra.run.smithi052.stdout: "heap": 223608 2024-05-15T14:12:26.797 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.798 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_server": { 2024-05-15T14:12:26.798 INFO:teuthology.orchestra.run.smithi052.stdout: "dispatch_client_request": 109200, 2024-05-15T14:12:26.798 INFO:teuthology.orchestra.run.smithi052.stdout: "dispatch_server_request": 0, 2024-05-15T14:12:26.798 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_request": 64910, 2024-05-15T14:12:26.798 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_session": 1029, 2024-05-15T14:12:26.798 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_peer_request": 12176, 2024-05-15T14:12:26.798 INFO:teuthology.orchestra.run.smithi052.stdout: "req_create_latency": { 2024-05-15T14:12:26.798 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 10187, 2024-05-15T14:12:26.799 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 19.999281934, 2024-05-15T14:12:26.799 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.001963216 2024-05-15T14:12:26.799 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.799 INFO:teuthology.orchestra.run.smithi052.stdout: "req_getattr_latency": { 2024-05-15T14:12:26.799 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 325, 2024-05-15T14:12:26.799 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.221951547, 2024-05-15T14:12:26.799 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000682927 2024-05-15T14:12:26.799 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.800 INFO:teuthology.orchestra.run.smithi052.stdout: "req_getfilelock_latency": { 2024-05-15T14:12:26.800 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:26.800 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:26.800 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:26.800 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.800 INFO:teuthology.orchestra.run.smithi052.stdout: "req_link_latency": { 2024-05-15T14:12:26.800 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:26.800 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:26.801 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:26.801 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.801 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookup_latency": { 2024-05-15T14:12:26.801 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 13726, 2024-05-15T14:12:26.801 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 6.391321245, 2024-05-15T14:12:26.801 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000465636 2024-05-15T14:12:26.801 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.801 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookuphash_latency": { 2024-05-15T14:12:26.802 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:26.802 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:26.802 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:26.802 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.802 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookupino_latency": { 2024-05-15T14:12:26.802 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 1, 2024-05-15T14:12:26.802 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000171848, 2024-05-15T14:12:26.802 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000171848 2024-05-15T14:12:26.803 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.803 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookupname_latency": { 2024-05-15T14:12:26.803 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:26.803 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:26.803 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:26.803 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.803 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookupparent_latency": { 2024-05-15T14:12:26.803 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:26.803 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:26.804 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:26.804 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.804 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookupsnap_latency": { 2024-05-15T14:12:26.804 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:26.804 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:26.804 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:26.805 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.805 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lssnap_latency": { 2024-05-15T14:12:26.806 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:26.806 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:26.806 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:26.806 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.806 INFO:teuthology.orchestra.run.smithi052.stdout: "req_mkdir_latency": { 2024-05-15T14:12:26.806 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:26.806 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:26.806 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:26.807 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.807 INFO:teuthology.orchestra.run.smithi052.stdout: "req_mknod_latency": { 2024-05-15T14:12:26.807 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:26.807 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:26.807 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:26.808 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.808 INFO:teuthology.orchestra.run.smithi052.stdout: "req_mksnap_latency": { 2024-05-15T14:12:26.808 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:26.808 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:26.808 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:26.808 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.808 INFO:teuthology.orchestra.run.smithi052.stdout: "req_open_latency": { 2024-05-15T14:12:26.808 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:26.809 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:26.809 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:26.809 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.809 INFO:teuthology.orchestra.run.smithi052.stdout: "req_readdir_latency": { 2024-05-15T14:12:26.809 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 22361, 2024-05-15T14:12:26.810 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 9.833596489, 2024-05-15T14:12:26.810 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000439765 2024-05-15T14:12:26.810 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.810 INFO:teuthology.orchestra.run.smithi052.stdout: "req_rename_latency": { 2024-05-15T14:12:26.810 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 1433, 2024-05-15T14:12:26.810 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 2.914599057, 2024-05-15T14:12:26.810 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.002033914 2024-05-15T14:12:26.810 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.811 INFO:teuthology.orchestra.run.smithi052.stdout: "req_renamesnap_latency": { 2024-05-15T14:12:26.811 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:26.811 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:26.811 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:26.811 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.811 INFO:teuthology.orchestra.run.smithi052.stdout: "req_snapdiff_latency": { 2024-05-15T14:12:26.812 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:26.812 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:26.812 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:26.812 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.812 INFO:teuthology.orchestra.run.smithi052.stdout: "req_rmdir_latency": { 2024-05-15T14:12:26.812 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:26.812 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:26.812 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:26.813 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.813 INFO:teuthology.orchestra.run.smithi052.stdout: "req_rmsnap_latency": { 2024-05-15T14:12:26.813 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:26.813 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:26.813 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:26.814 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.814 INFO:teuthology.orchestra.run.smithi052.stdout: "req_rmxattr_latency": { 2024-05-15T14:12:26.814 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:26.814 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:26.814 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:26.814 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.814 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setattr_latency": { 2024-05-15T14:12:26.815 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 8, 2024-05-15T14:12:26.815 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.030155629, 2024-05-15T14:12:26.815 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.003769453 2024-05-15T14:12:26.815 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.815 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setdirlayout_latency": { 2024-05-15T14:12:26.815 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:26.816 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:26.816 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:26.816 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.816 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setfilelock_latency": { 2024-05-15T14:12:26.816 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:26.816 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:26.816 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:26.816 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.817 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setlayout_latency": { 2024-05-15T14:12:26.817 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:26.817 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:26.817 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:26.817 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.817 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setxattr_latency": { 2024-05-15T14:12:26.817 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:26.817 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:26.818 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:26.818 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.818 INFO:teuthology.orchestra.run.smithi052.stdout: "req_symlink_latency": { 2024-05-15T14:12:26.818 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:26.818 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:26.818 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:26.818 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.818 INFO:teuthology.orchestra.run.smithi052.stdout: "req_unlink_latency": { 2024-05-15T14:12:26.819 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 10187, 2024-05-15T14:12:26.819 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 33.121469268, 2024-05-15T14:12:26.819 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.003251346 2024-05-15T14:12:26.819 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.819 INFO:teuthology.orchestra.run.smithi052.stdout: "cap_revoke_eviction": 0, 2024-05-15T14:12:26.819 INFO:teuthology.orchestra.run.smithi052.stdout: "cap_acquisition_throttle": 0, 2024-05-15T14:12:26.819 INFO:teuthology.orchestra.run.smithi052.stdout: "req_getvxattr_latency": { 2024-05-15T14:12:26.819 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:26.819 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:26.820 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:26.820 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-15T14:12:26.820 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.820 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_sessions": { 2024-05-15T14:12:26.820 INFO:teuthology.orchestra.run.smithi052.stdout: "session_count": 2, 2024-05-15T14:12:26.820 INFO:teuthology.orchestra.run.smithi052.stdout: "session_add": 4, 2024-05-15T14:12:26.821 INFO:teuthology.orchestra.run.smithi052.stdout: "session_remove": 2, 2024-05-15T14:12:26.821 INFO:teuthology.orchestra.run.smithi052.stdout: "sessions_open": 2, 2024-05-15T14:12:26.821 INFO:teuthology.orchestra.run.smithi052.stdout: "sessions_stale": 0, 2024-05-15T14:12:26.821 INFO:teuthology.orchestra.run.smithi052.stdout: "total_load": 7401, 2024-05-15T14:12:26.821 INFO:teuthology.orchestra.run.smithi052.stdout: "average_load": 3700, 2024-05-15T14:12:26.821 INFO:teuthology.orchestra.run.smithi052.stdout: "avg_session_uptime": 691, 2024-05-15T14:12:26.821 INFO:teuthology.orchestra.run.smithi052.stdout: "mdthresh_evicted": 0 2024-05-15T14:12:26.821 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.822 INFO:teuthology.orchestra.run.smithi052.stdout: "mempool": { 2024-05-15T14:12:26.822 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_bytes": 0, 2024-05-15T14:12:26.822 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_items": 0, 2024-05-15T14:12:26.822 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_bytes": 0, 2024-05-15T14:12:26.822 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_items": 0, 2024-05-15T14:12:26.823 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_bytes": 0, 2024-05-15T14:12:26.823 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_items": 0, 2024-05-15T14:12:26.823 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-15T14:12:26.823 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_items": 0, 2024-05-15T14:12:26.823 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-15T14:12:26.823 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_items": 0, 2024-05-15T14:12:26.823 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_bytes": 0, 2024-05-15T14:12:26.824 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_items": 0, 2024-05-15T14:12:26.824 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-15T14:12:26.824 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_items": 0, 2024-05-15T14:12:26.824 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_bytes": 0, 2024-05-15T14:12:26.824 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_items": 0, 2024-05-15T14:12:26.824 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_bytes": 0, 2024-05-15T14:12:26.824 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_items": 0, 2024-05-15T14:12:26.824 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-15T14:12:26.825 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_items": 0, 2024-05-15T14:12:26.825 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-15T14:12:26.825 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_items": 0, 2024-05-15T14:12:26.825 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_bytes": 0, 2024-05-15T14:12:26.825 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_items": 0, 2024-05-15T14:12:26.825 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_bytes": 0, 2024-05-15T14:12:26.825 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_items": 0, 2024-05-15T14:12:26.826 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-15T14:12:26.826 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_items": 0, 2024-05-15T14:12:26.826 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_bytes": 0, 2024-05-15T14:12:26.826 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_items": 0, 2024-05-15T14:12:26.826 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_bytes": 0, 2024-05-15T14:12:26.826 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_items": 0, 2024-05-15T14:12:26.826 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_bytes": 0, 2024-05-15T14:12:26.826 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_items": 0, 2024-05-15T14:12:26.827 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_bytes": 0, 2024-05-15T14:12:26.827 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_items": 0, 2024-05-15T14:12:26.827 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_bytes": 1207230, 2024-05-15T14:12:26.827 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_items": 450, 2024-05-15T14:12:26.827 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_bytes": 0, 2024-05-15T14:12:26.827 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_items": 0, 2024-05-15T14:12:26.827 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_bytes": 0, 2024-05-15T14:12:26.828 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_items": 0, 2024-05-15T14:12:26.828 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_bytes": 0, 2024-05-15T14:12:26.828 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_items": 0, 2024-05-15T14:12:26.828 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_bytes": 0, 2024-05-15T14:12:26.828 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_items": 0, 2024-05-15T14:12:26.828 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_bytes": 8128, 2024-05-15T14:12:26.828 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_items": 169, 2024-05-15T14:12:26.828 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_bytes": 0, 2024-05-15T14:12:26.829 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_items": 0, 2024-05-15T14:12:26.829 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_bytes": 0, 2024-05-15T14:12:26.829 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_items": 0, 2024-05-15T14:12:26.829 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_bytes": 1206581, 2024-05-15T14:12:26.829 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_items": 17664, 2024-05-15T14:12:26.829 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_bytes": 0, 2024-05-15T14:12:26.829 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_items": 0, 2024-05-15T14:12:26.829 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_bytes": 0, 2024-05-15T14:12:26.829 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_items": 0 2024-05-15T14:12:26.830 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.830 INFO:teuthology.orchestra.run.smithi052.stdout: "objecter": { 2024-05-15T14:12:26.830 INFO:teuthology.orchestra.run.smithi052.stdout: "op_active": 0, 2024-05-15T14:12:26.830 INFO:teuthology.orchestra.run.smithi052.stdout: "op_laggy": 0, 2024-05-15T14:12:26.830 INFO:teuthology.orchestra.run.smithi052.stdout: "op_send": 27722, 2024-05-15T14:12:26.830 INFO:teuthology.orchestra.run.smithi052.stdout: "op_send_bytes": 96205421, 2024-05-15T14:12:26.830 INFO:teuthology.orchestra.run.smithi052.stdout: "op_resend": 28, 2024-05-15T14:12:26.830 INFO:teuthology.orchestra.run.smithi052.stdout: "op_reply": 27722, 2024-05-15T14:12:26.831 INFO:teuthology.orchestra.run.smithi052.stdout: "op_latency": { 2024-05-15T14:12:26.831 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 27722, 2024-05-15T14:12:26.831 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 62.452200655, 2024-05-15T14:12:26.831 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.002252802 2024-05-15T14:12:26.831 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.831 INFO:teuthology.orchestra.run.smithi052.stdout: "op_inflight": 0, 2024-05-15T14:12:26.831 INFO:teuthology.orchestra.run.smithi052.stdout: "oplen_avg": { 2024-05-15T14:12:26.831 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 27722, 2024-05-15T14:12:26.832 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 28362 2024-05-15T14:12:26.832 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.832 INFO:teuthology.orchestra.run.smithi052.stdout: "op": 27722, 2024-05-15T14:12:26.832 INFO:teuthology.orchestra.run.smithi052.stdout: "op_r": 506, 2024-05-15T14:12:26.832 INFO:teuthology.orchestra.run.smithi052.stdout: "op_w": 27216, 2024-05-15T14:12:26.832 INFO:teuthology.orchestra.run.smithi052.stdout: "op_rmw": 0, 2024-05-15T14:12:26.832 INFO:teuthology.orchestra.run.smithi052.stdout: "op_pg": 0, 2024-05-15T14:12:26.832 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_stat": 0, 2024-05-15T14:12:26.832 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_create": 10, 2024-05-15T14:12:26.833 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_read": 506, 2024-05-15T14:12:26.833 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_write": 16313, 2024-05-15T14:12:26.833 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_writefull": 279, 2024-05-15T14:12:26.833 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_writesame": 0, 2024-05-15T14:12:26.833 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_append": 0, 2024-05-15T14:12:26.833 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_zero": 0, 2024-05-15T14:12:26.833 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_truncate": 0, 2024-05-15T14:12:26.833 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_delete": 10221, 2024-05-15T14:12:26.834 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_mapext": 0, 2024-05-15T14:12:26.834 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_sparse_read": 0, 2024-05-15T14:12:26.834 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_clonerange": 0, 2024-05-15T14:12:26.834 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_getxattr": 0, 2024-05-15T14:12:26.834 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_setxattr": 20, 2024-05-15T14:12:26.834 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_cmpxattr": 0, 2024-05-15T14:12:26.834 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_rmxattr": 0, 2024-05-15T14:12:26.834 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_resetxattrs": 0, 2024-05-15T14:12:26.835 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_call": 0, 2024-05-15T14:12:26.835 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_watch": 0, 2024-05-15T14:12:26.835 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_notify": 0, 2024-05-15T14:12:26.835 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_src_cmpxattr": 0, 2024-05-15T14:12:26.835 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_pgls": 0, 2024-05-15T14:12:26.835 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_pgls_filter": 0, 2024-05-15T14:12:26.835 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_other": 0, 2024-05-15T14:12:26.835 INFO:teuthology.orchestra.run.smithi052.stdout: "linger_active": 0, 2024-05-15T14:12:26.835 INFO:teuthology.orchestra.run.smithi052.stdout: "linger_send": 0, 2024-05-15T14:12:26.836 INFO:teuthology.orchestra.run.smithi052.stdout: "linger_resend": 0, 2024-05-15T14:12:26.836 INFO:teuthology.orchestra.run.smithi052.stdout: "linger_ping": 0, 2024-05-15T14:12:26.836 INFO:teuthology.orchestra.run.smithi052.stdout: "poolop_active": 0, 2024-05-15T14:12:26.836 INFO:teuthology.orchestra.run.smithi052.stdout: "poolop_send": 0, 2024-05-15T14:12:26.836 INFO:teuthology.orchestra.run.smithi052.stdout: "poolop_resend": 0, 2024-05-15T14:12:26.836 INFO:teuthology.orchestra.run.smithi052.stdout: "poolstat_active": 0, 2024-05-15T14:12:26.836 INFO:teuthology.orchestra.run.smithi052.stdout: "poolstat_send": 0, 2024-05-15T14:12:26.836 INFO:teuthology.orchestra.run.smithi052.stdout: "poolstat_resend": 0, 2024-05-15T14:12:26.837 INFO:teuthology.orchestra.run.smithi052.stdout: "statfs_active": 0, 2024-05-15T14:12:26.837 INFO:teuthology.orchestra.run.smithi052.stdout: "statfs_send": 0, 2024-05-15T14:12:26.837 INFO:teuthology.orchestra.run.smithi052.stdout: "statfs_resend": 0, 2024-05-15T14:12:26.837 INFO:teuthology.orchestra.run.smithi052.stdout: "command_active": 0, 2024-05-15T14:12:26.837 INFO:teuthology.orchestra.run.smithi052.stdout: "command_send": 0, 2024-05-15T14:12:26.837 INFO:teuthology.orchestra.run.smithi052.stdout: "command_resend": 0, 2024-05-15T14:12:26.837 INFO:teuthology.orchestra.run.smithi052.stdout: "map_epoch": 98, 2024-05-15T14:12:26.837 INFO:teuthology.orchestra.run.smithi052.stdout: "map_full": 0, 2024-05-15T14:12:26.837 INFO:teuthology.orchestra.run.smithi052.stdout: "map_inc": 18, 2024-05-15T14:12:26.838 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_sessions": 12, 2024-05-15T14:12:26.838 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_session_open": 12, 2024-05-15T14:12:26.838 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_session_close": 0, 2024-05-15T14:12:26.838 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_laggy": 0, 2024-05-15T14:12:26.838 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_wr": 722, 2024-05-15T14:12:26.838 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_rd": 0, 2024-05-15T14:12:26.838 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_del": 291 2024-05-15T14:12:26.838 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.839 INFO:teuthology.orchestra.run.smithi052.stdout: "oft": { 2024-05-15T14:12:26.839 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_total_objs": 1, 2024-05-15T14:12:26.839 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_total_kv_pairs": 0, 2024-05-15T14:12:26.839 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_total_updates": 0, 2024-05-15T14:12:26.839 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_total_removes": 0 2024-05-15T14:12:26.839 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.839 INFO:teuthology.orchestra.run.smithi052.stdout: "osd-slow-ops": { 2024-05-15T14:12:26.839 INFO:teuthology.orchestra.run.smithi052.stdout: "slow_ops_count": 312 2024-05-15T14:12:26.839 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.840 INFO:teuthology.orchestra.run.smithi052.stdout: "purge_queue": { 2024-05-15T14:12:26.840 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executing_ops": 0, 2024-05-15T14:12:26.840 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executing_ops_high_water": 10, 2024-05-15T14:12:26.840 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executing": 0, 2024-05-15T14:12:26.840 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executing_high_water": 10, 2024-05-15T14:12:26.840 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executed_ops": 10187, 2024-05-15T14:12:26.840 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executed": 10187, 2024-05-15T14:12:26.840 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_item_in_journal": 0 2024-05-15T14:12:26.841 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.841 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-15T14:12:26.841 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-05-15T14:12:26.841 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 104857600, 2024-05-15T14:12:26.841 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-05-15T14:12:26.841 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 233664, 2024-05-15T14:12:26.841 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 51545256, 2024-05-15T14:12:26.841 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:26.842 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 233664, 2024-05-15T14:12:26.842 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-05-15T14:12:26.842 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-05-15T14:12:26.842 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 233664, 2024-05-15T14:12:26.842 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 51545256, 2024-05-15T14:12:26.842 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-05-15T14:12:26.842 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:26.842 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:26.843 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:26.843 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-15T14:12:26.843 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.843 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-objecter_bytes": { 2024-05-15T14:12:26.843 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-05-15T14:12:26.843 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 104857600, 2024-05-15T14:12:26.843 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-05-15T14:12:26.844 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 0, 2024-05-15T14:12:26.844 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 0, 2024-05-15T14:12:26.844 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:26.844 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 0, 2024-05-15T14:12:26.844 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 27722, 2024-05-15T14:12:26.844 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 97235332, 2024-05-15T14:12:26.845 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 27722, 2024-05-15T14:12:26.845 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 97235332, 2024-05-15T14:12:26.845 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-05-15T14:12:26.845 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:26.845 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:26.845 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:26.845 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-15T14:12:26.845 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.846 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-objecter_ops": { 2024-05-15T14:12:26.846 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-05-15T14:12:26.846 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 1024, 2024-05-15T14:12:26.846 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-05-15T14:12:26.846 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 0, 2024-05-15T14:12:26.846 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 0, 2024-05-15T14:12:26.846 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:26.846 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 0, 2024-05-15T14:12:26.847 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 27722, 2024-05-15T14:12:26.847 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 27722, 2024-05-15T14:12:26.847 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 27722, 2024-05-15T14:12:26.847 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 27722, 2024-05-15T14:12:26.847 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-05-15T14:12:26.847 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:26.847 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:26.847 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:26.848 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-15T14:12:26.848 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.848 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-write_buf_throttle": { 2024-05-15T14:12:26.848 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-05-15T14:12:26.848 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 3758096384, 2024-05-15T14:12:26.848 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-05-15T14:12:26.848 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 10187, 2024-05-15T14:12:26.848 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 1029911, 2024-05-15T14:12:26.849 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:26.849 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 10187, 2024-05-15T14:12:26.849 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-05-15T14:12:26.849 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-05-15T14:12:26.849 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 506, 2024-05-15T14:12:26.849 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 1029911, 2024-05-15T14:12:26.849 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-05-15T14:12:26.849 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:26.850 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:26.850 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:26.850 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-15T14:12:26.850 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:26.850 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-write_buf_throttle-0x55c4557a4460": { 2024-05-15T14:12:26.850 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 25133, 2024-05-15T14:12:26.850 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 3758096384, 2024-05-15T14:12:26.850 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-05-15T14:12:26.851 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 53155, 2024-05-15T14:12:26.851 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 94262914, 2024-05-15T14:12:26.851 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:26.851 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 53155, 2024-05-15T14:12:26.851 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-05-15T14:12:26.851 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-05-15T14:12:26.851 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 15807, 2024-05-15T14:12:26.851 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 94237781, 2024-05-15T14:12:26.852 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-05-15T14:12:26.852 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:26.852 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:26.852 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:26.852 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-15T14:12:26.852 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-15T14:12:26.852 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-15T14:12:26.853 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:26 smithi052 ceph-mon[36190]: pgmap v380: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-05-15T14:12:26.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:26 smithi081 ceph-mon[36434]: pgmap v380: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-05-15T14:12:26.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:26 smithi033 ceph-mon[31650]: pgmap v380: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-05-15T14:12:27.179 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-05-15T14:12:27.179 INFO:tasks.check_counter:Daemon mds.b mds.imported=4 2024-05-15T14:12:27.179 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=212 2024-05-15T14:12:27.179 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=531 2024-05-15T14:12:27.179 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-05-15T14:12:27.179 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=0 2024-05-15T14:12:27.180 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=0 2024-05-15T14:12:27.180 DEBUG:tasks.check_counter:Getting stats from e 2024-05-15T14:12:27.180 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-15T14:12:27.429 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.b/config 2024-05-15T14:12:27.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:27 smithi052 ceph-mon[36190]: pgmap v381: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 870 KiB/s wr, 39 op/s 2024-05-15T14:12:27.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:27 smithi033 ceph-mon[31650]: pgmap v381: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 870 KiB/s wr, 39 op/s 2024-05-15T14:12:28.107 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:27 smithi081 ceph-mon[36434]: pgmap v381: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 870 KiB/s wr, 39 op/s 2024-05-15T14:12:29.220 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-15T14:12:29.221 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-0": { 2024-05-15T14:12:29.221 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 264, 2024-05-15T14:12:29.221 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 253, 2024-05-15T14:12:29.221 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 1722896, 2024-05-15T14:12:29.221 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 155920, 2024-05-15T14:12:29.221 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 4, 2024-05-15T14:12:29.221 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 0, 2024-05-15T14:12:29.222 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.054651425, 2024-05-15T14:12:29.222 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.015227035, 2024-05-15T14:12:29.222 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.042495743, 2024-05-15T14:12:29.222 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.006007000, 2024-05-15T14:12:29.222 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:29.222 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 252, 2024-05-15T14:12:29.222 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.012139432, 2024-05-15T14:12:29.222 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000048172 2024-05-15T14:12:29.223 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:29.223 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:29.223 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:29.223 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:29.223 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:29.223 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:29.223 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 1722896, 2024-05-15T14:12:29.223 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 155920 2024-05-15T14:12:29.224 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:29.224 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-1": { 2024-05-15T14:12:29.224 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 2, 2024-05-15T14:12:29.224 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 193, 2024-05-15T14:12:29.224 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 320, 2024-05-15T14:12:29.224 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 191360, 2024-05-15T14:12:29.224 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 5, 2024-05-15T14:12:29.224 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 3, 2024-05-15T14:12:29.224 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.012617757, 2024-05-15T14:12:29.225 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.007618715, 2024-05-15T14:12:29.225 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.001717445, 2024-05-15T14:12:29.225 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-15T14:12:29.225 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:29.225 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 190, 2024-05-15T14:12:29.225 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.021118657, 2024-05-15T14:12:29.225 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000111150 2024-05-15T14:12:29.225 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:29.225 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:29.226 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:29.226 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:29.226 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:29.226 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:29.226 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-15T14:12:29.226 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 191360 2024-05-15T14:12:29.226 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:29.226 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-2": { 2024-05-15T14:12:29.227 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 4, 2024-05-15T14:12:29.227 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 4, 2024-05-15T14:12:29.227 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 2992, 2024-05-15T14:12:29.227 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 448, 2024-05-15T14:12:29.227 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 2, 2024-05-15T14:12:29.227 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 1, 2024-05-15T14:12:29.227 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.000590834, 2024-05-15T14:12:29.227 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.000049852, 2024-05-15T14:12:29.228 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.000512406, 2024-05-15T14:12:29.228 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-15T14:12:29.228 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:29.228 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 3, 2024-05-15T14:12:29.228 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000681940, 2024-05-15T14:12:29.228 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000227313 2024-05-15T14:12:29.228 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:29.228 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:29.228 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:29.229 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:29.229 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:29.229 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:29.229 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-15T14:12:29.229 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 448 2024-05-15T14:12:29.229 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:29.229 INFO:teuthology.orchestra.run.smithi052.stdout: "cct": { 2024-05-15T14:12:29.229 INFO:teuthology.orchestra.run.smithi052.stdout: "total_workers": 0, 2024-05-15T14:12:29.230 INFO:teuthology.orchestra.run.smithi052.stdout: "unhealthy_workers": 0 2024-05-15T14:12:29.230 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:29.230 INFO:teuthology.orchestra.run.smithi052.stdout: "mempool": { 2024-05-15T14:12:29.230 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_bytes": 0, 2024-05-15T14:12:29.230 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_items": 0, 2024-05-15T14:12:29.230 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_bytes": 0, 2024-05-15T14:12:29.230 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_items": 0, 2024-05-15T14:12:29.230 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_bytes": 0, 2024-05-15T14:12:29.230 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_items": 0, 2024-05-15T14:12:29.231 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-15T14:12:29.231 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_items": 0, 2024-05-15T14:12:29.231 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-15T14:12:29.231 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_items": 0, 2024-05-15T14:12:29.231 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_bytes": 0, 2024-05-15T14:12:29.231 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_items": 0, 2024-05-15T14:12:29.231 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-15T14:12:29.231 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_items": 0, 2024-05-15T14:12:29.232 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_bytes": 0, 2024-05-15T14:12:29.232 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_items": 0, 2024-05-15T14:12:29.232 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_bytes": 0, 2024-05-15T14:12:29.232 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_items": 0, 2024-05-15T14:12:29.232 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-15T14:12:29.232 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_items": 0, 2024-05-15T14:12:29.232 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-15T14:12:29.232 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_items": 0, 2024-05-15T14:12:29.232 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_bytes": 0, 2024-05-15T14:12:29.233 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_items": 0, 2024-05-15T14:12:29.233 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_bytes": 0, 2024-05-15T14:12:29.233 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_items": 0, 2024-05-15T14:12:29.233 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-15T14:12:29.233 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_items": 0, 2024-05-15T14:12:29.233 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_bytes": 0, 2024-05-15T14:12:29.233 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_items": 0, 2024-05-15T14:12:29.233 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_bytes": 0, 2024-05-15T14:12:29.233 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_items": 0, 2024-05-15T14:12:29.234 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_bytes": 0, 2024-05-15T14:12:29.234 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_items": 0, 2024-05-15T14:12:29.234 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_bytes": 0, 2024-05-15T14:12:29.234 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_items": 0, 2024-05-15T14:12:29.234 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_bytes": 148704, 2024-05-15T14:12:29.234 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_items": 29, 2024-05-15T14:12:29.234 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_bytes": 0, 2024-05-15T14:12:29.234 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_items": 0, 2024-05-15T14:12:29.235 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_bytes": 0, 2024-05-15T14:12:29.235 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_items": 0, 2024-05-15T14:12:29.235 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_bytes": 0, 2024-05-15T14:12:29.235 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_items": 0, 2024-05-15T14:12:29.235 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_bytes": 0, 2024-05-15T14:12:29.235 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_items": 0, 2024-05-15T14:12:29.235 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_bytes": 0, 2024-05-15T14:12:29.235 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_items": 0, 2024-05-15T14:12:29.235 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_bytes": 0, 2024-05-15T14:12:29.236 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_items": 0, 2024-05-15T14:12:29.236 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_bytes": 0, 2024-05-15T14:12:29.236 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_items": 0, 2024-05-15T14:12:29.236 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_bytes": 928, 2024-05-15T14:12:29.236 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_items": 2, 2024-05-15T14:12:29.236 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_bytes": 0, 2024-05-15T14:12:29.236 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_items": 0, 2024-05-15T14:12:29.236 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_bytes": 0, 2024-05-15T14:12:29.237 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_items": 0 2024-05-15T14:12:29.237 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:29.237 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-15T14:12:29.237 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-05-15T14:12:29.237 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 104857600, 2024-05-15T14:12:29.237 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-05-15T14:12:29.237 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 266, 2024-05-15T14:12:29.237 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 1685630, 2024-05-15T14:12:29.237 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:29.238 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 266, 2024-05-15T14:12:29.238 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-05-15T14:12:29.238 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-05-15T14:12:29.238 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 266, 2024-05-15T14:12:29.238 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 1685630, 2024-05-15T14:12:29.238 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-05-15T14:12:29.238 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:29.238 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:29.239 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:29.239 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-15T14:12:29.239 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-15T14:12:29.239 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-15T14:12:30.076 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-05-15T14:12:30.076 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-05-15T14:12:30.076 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.e 2024-05-15T14:12:30.077 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.e 2024-05-15T14:12:30.077 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-05-15T14:12:30.077 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-05-15T14:12:30.077 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-05-15T14:12:30.077 DEBUG:tasks.check_counter:Getting stats from h 2024-05-15T14:12:30.077 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-15T14:12:30.332 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.b/config 2024-05-15T14:12:30.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:30 smithi033 ceph-mon[31650]: pgmap v382: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 860 KiB/s wr, 35 op/s 2024-05-15T14:12:30.729 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:30 smithi052 ceph-mon[36190]: pgmap v382: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 860 KiB/s wr, 35 op/s 2024-05-15T14:12:30.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:30 smithi081 ceph-mon[36434]: pgmap v382: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 860 KiB/s wr, 35 op/s 2024-05-15T14:12:32.096 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-15T14:12:32.097 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-0": { 2024-05-15T14:12:32.097 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 2176, 2024-05-15T14:12:32.097 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 2178, 2024-05-15T14:12:32.097 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 25792734, 2024-05-15T14:12:32.097 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 691975, 2024-05-15T14:12:32.097 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 7, 2024-05-15T14:12:32.098 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 3, 2024-05-15T14:12:32.098 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.185402597, 2024-05-15T14:12:32.098 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.056513984, 2024-05-15T14:12:32.098 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.058248914, 2024-05-15T14:12:32.098 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.054842880, 2024-05-15T14:12:32.098 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:32.099 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 2176, 2024-05-15T14:12:32.099 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.068154270, 2024-05-15T14:12:32.099 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000031320 2024-05-15T14:12:32.099 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.099 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:32.099 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.099 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.099 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.099 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.100 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-15T14:12:32.100 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 0 2024-05-15T14:12:32.100 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.100 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-1": { 2024-05-15T14:12:32.100 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 607, 2024-05-15T14:12:32.100 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 797, 2024-05-15T14:12:32.100 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 20753885, 2024-05-15T14:12:32.101 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 689844, 2024-05-15T14:12:32.101 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 8, 2024-05-15T14:12:32.101 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 6, 2024-05-15T14:12:32.101 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.064705028, 2024-05-15T14:12:32.101 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.020411487, 2024-05-15T14:12:32.101 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.024646049, 2024-05-15T14:12:32.101 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.013887830, 2024-05-15T14:12:32.101 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:32.102 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 794, 2024-05-15T14:12:32.102 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.032912798, 2024-05-15T14:12:32.102 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000041451 2024-05-15T14:12:32.102 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.102 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:32.102 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.103 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.103 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.103 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.103 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-15T14:12:32.103 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 497392 2024-05-15T14:12:32.103 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.103 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-2": { 2024-05-15T14:12:32.103 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 491, 2024-05-15T14:12:32.103 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 460, 2024-05-15T14:12:32.104 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 10262803, 2024-05-15T14:12:32.104 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 217922, 2024-05-15T14:12:32.104 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 5, 2024-05-15T14:12:32.104 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 4, 2024-05-15T14:12:32.104 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.068230761, 2024-05-15T14:12:32.104 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.018906552, 2024-05-15T14:12:32.104 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.049427399, 2024-05-15T14:12:32.104 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.010162936, 2024-05-15T14:12:32.104 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:32.105 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 460, 2024-05-15T14:12:32.105 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.018822081, 2024-05-15T14:12:32.105 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000040917 2024-05-15T14:12:32.105 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.105 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:32.105 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.105 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.105 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.105 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.106 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 1826208, 2024-05-15T14:12:32.106 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 159728 2024-05-15T14:12:32.106 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.106 INFO:teuthology.orchestra.run.smithi052.stdout: "cct": { 2024-05-15T14:12:32.106 INFO:teuthology.orchestra.run.smithi052.stdout: "total_workers": 1, 2024-05-15T14:12:32.106 INFO:teuthology.orchestra.run.smithi052.stdout: "unhealthy_workers": 0 2024-05-15T14:12:32.106 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.107 INFO:teuthology.orchestra.run.smithi052.stdout: "finisher-MDSRank": { 2024-05-15T14:12:32.107 INFO:teuthology.orchestra.run.smithi052.stdout: "queue_len": 0, 2024-05-15T14:12:32.107 INFO:teuthology.orchestra.run.smithi052.stdout: "complete_latency": { 2024-05-15T14:12:32.107 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 5073, 2024-05-15T14:12:32.107 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.111917947, 2024-05-15T14:12:32.107 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000022061 2024-05-15T14:12:32.107 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-15T14:12:32.107 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.108 INFO:teuthology.orchestra.run.smithi052.stdout: "finisher-PurgeQueue": { 2024-05-15T14:12:32.108 INFO:teuthology.orchestra.run.smithi052.stdout: "queue_len": 0, 2024-05-15T14:12:32.108 INFO:teuthology.orchestra.run.smithi052.stdout: "complete_latency": { 2024-05-15T14:12:32.108 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.108 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.108 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.108 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-15T14:12:32.108 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.109 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": { 2024-05-15T14:12:32.109 INFO:teuthology.orchestra.run.smithi052.stdout: "request": 0, 2024-05-15T14:12:32.109 INFO:teuthology.orchestra.run.smithi052.stdout: "reply": 0, 2024-05-15T14:12:32.109 INFO:teuthology.orchestra.run.smithi052.stdout: "reply_latency": { 2024-05-15T14:12:32.109 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.109 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.109 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.109 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.110 INFO:teuthology.orchestra.run.smithi052.stdout: "slow_reply": 0, 2024-05-15T14:12:32.110 INFO:teuthology.orchestra.run.smithi052.stdout: "forward": 0, 2024-05-15T14:12:32.110 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_fetch_complete": 0, 2024-05-15T14:12:32.110 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_fetch_keys": 0, 2024-05-15T14:12:32.110 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_commit": 0, 2024-05-15T14:12:32.110 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_split": 0, 2024-05-15T14:12:32.110 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_merge": 0, 2024-05-15T14:12:32.110 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes": 3997, 2024-05-15T14:12:32.111 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_top": 99, 2024-05-15T14:12:32.111 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_bottom": 3898, 2024-05-15T14:12:32.111 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_pin_tail": 0, 2024-05-15T14:12:32.111 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_pinned": 3856, 2024-05-15T14:12:32.111 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_expired": 0, 2024-05-15T14:12:32.111 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_with_caps": 0, 2024-05-15T14:12:32.111 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": 0, 2024-05-15T14:12:32.111 INFO:teuthology.orchestra.run.smithi052.stdout: "subtrees": 7, 2024-05-15T14:12:32.111 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse": 0, 2024-05-15T14:12:32.112 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_hit": 0, 2024-05-15T14:12:32.112 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_forward": 0, 2024-05-15T14:12:32.112 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_discover": 0, 2024-05-15T14:12:32.112 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_dir_fetch": 0, 2024-05-15T14:12:32.112 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_remote_ino": 0, 2024-05-15T14:12:32.112 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_lock": 0, 2024-05-15T14:12:32.112 INFO:teuthology.orchestra.run.smithi052.stdout: "load_cent": 0, 2024-05-15T14:12:32.112 INFO:teuthology.orchestra.run.smithi052.stdout: "q": 0, 2024-05-15T14:12:32.113 INFO:teuthology.orchestra.run.smithi052.stdout: "exported": 0, 2024-05-15T14:12:32.113 INFO:teuthology.orchestra.run.smithi052.stdout: "exported_inodes": 0, 2024-05-15T14:12:32.113 INFO:teuthology.orchestra.run.smithi052.stdout: "imported": 0, 2024-05-15T14:12:32.113 INFO:teuthology.orchestra.run.smithi052.stdout: "imported_inodes": 0, 2024-05-15T14:12:32.113 INFO:teuthology.orchestra.run.smithi052.stdout: "openino_dir_fetch": 0, 2024-05-15T14:12:32.113 INFO:teuthology.orchestra.run.smithi052.stdout: "openino_backtrace_fetch": 0, 2024-05-15T14:12:32.113 INFO:teuthology.orchestra.run.smithi052.stdout: "openino_peer_discover": 0, 2024-05-15T14:12:32.113 INFO:teuthology.orchestra.run.smithi052.stdout: "root_rfiles": 0, 2024-05-15T14:12:32.113 INFO:teuthology.orchestra.run.smithi052.stdout: "root_rbytes": 0, 2024-05-15T14:12:32.114 INFO:teuthology.orchestra.run.smithi052.stdout: "root_rsnaps": 0, 2024-05-15T14:12:32.114 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_backtrace_fetch": 0, 2024-05-15T14:12:32.114 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_set_tag": 0, 2024-05-15T14:12:32.114 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_backtrace_repaired": 0, 2024-05-15T14:12:32.114 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_inotable_repaired": 0, 2024-05-15T14:12:32.114 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_dir_inodes": 0, 2024-05-15T14:12:32.114 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_dir_base_inodes": 0, 2024-05-15T14:12:32.114 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_dirfrag_rstats": 0, 2024-05-15T14:12:32.115 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_file_inodes": 0, 2024-05-15T14:12:32.115 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_inode_file_caps": 0, 2024-05-15T14:12:32.115 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_revoke": 0, 2024-05-15T14:12:32.115 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_grant": 0, 2024-05-15T14:12:32.115 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_trunc": 0, 2024-05-15T14:12:32.115 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-15T14:12:32.116 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-15T14:12:32.116 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_caps": 0, 2024-05-15T14:12:32.116 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_caps_dirty": 0, 2024-05-15T14:12:32.116 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_cap_release": 0, 2024-05-15T14:12:32.116 INFO:teuthology.orchestra.run.smithi052.stdout: "process_request_cap_release": 0 2024-05-15T14:12:32.116 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.116 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_cache": { 2024-05-15T14:12:32.116 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_update": 0, 2024-05-15T14:12:32.116 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_update_receipt": 0, 2024-05-15T14:12:32.117 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_try_discover": 0, 2024-05-15T14:12:32.117 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_send_discover": 0, 2024-05-15T14:12:32.117 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_handle_discover": 0, 2024-05-15T14:12:32.117 INFO:teuthology.orchestra.run.smithi052.stdout: "num_strays": 0, 2024-05-15T14:12:32.117 INFO:teuthology.orchestra.run.smithi052.stdout: "num_strays_delayed": 0, 2024-05-15T14:12:32.117 INFO:teuthology.orchestra.run.smithi052.stdout: "num_strays_enqueuing": 0, 2024-05-15T14:12:32.118 INFO:teuthology.orchestra.run.smithi052.stdout: "strays_created": 0, 2024-05-15T14:12:32.118 INFO:teuthology.orchestra.run.smithi052.stdout: "strays_enqueued": 0, 2024-05-15T14:12:32.118 INFO:teuthology.orchestra.run.smithi052.stdout: "strays_reintegrated": 0, 2024-05-15T14:12:32.118 INFO:teuthology.orchestra.run.smithi052.stdout: "strays_migrated": 0, 2024-05-15T14:12:32.118 INFO:teuthology.orchestra.run.smithi052.stdout: "num_recovering_processing": 0, 2024-05-15T14:12:32.118 INFO:teuthology.orchestra.run.smithi052.stdout: "num_recovering_enqueued": 0, 2024-05-15T14:12:32.118 INFO:teuthology.orchestra.run.smithi052.stdout: "num_recovering_prioritized": 0, 2024-05-15T14:12:32.118 INFO:teuthology.orchestra.run.smithi052.stdout: "recovery_started": 0, 2024-05-15T14:12:32.119 INFO:teuthology.orchestra.run.smithi052.stdout: "recovery_completed": 0, 2024-05-15T14:12:32.119 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_quiesce_path": 0, 2024-05-15T14:12:32.119 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_quiesce_inode": 0, 2024-05-15T14:12:32.119 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_enqueue_scrub": 0, 2024-05-15T14:12:32.119 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_exportdir": 0, 2024-05-15T14:12:32.119 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_flush": 0, 2024-05-15T14:12:32.119 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_fragmentdir": 0, 2024-05-15T14:12:32.119 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_fragstats": 0, 2024-05-15T14:12:32.120 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_inodestats": 0 2024-05-15T14:12:32.120 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.120 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_log": { 2024-05-15T14:12:32.120 INFO:teuthology.orchestra.run.smithi052.stdout: "evlrg": 0, 2024-05-15T14:12:32.120 INFO:teuthology.orchestra.run.smithi052.stdout: "evadd": 0, 2024-05-15T14:12:32.120 INFO:teuthology.orchestra.run.smithi052.stdout: "evex": 0, 2024-05-15T14:12:32.120 INFO:teuthology.orchestra.run.smithi052.stdout: "evtrm": 0, 2024-05-15T14:12:32.120 INFO:teuthology.orchestra.run.smithi052.stdout: "ev": 32457, 2024-05-15T14:12:32.120 INFO:teuthology.orchestra.run.smithi052.stdout: "evexg": 0, 2024-05-15T14:12:32.121 INFO:teuthology.orchestra.run.smithi052.stdout: "evexd": 0, 2024-05-15T14:12:32.121 INFO:teuthology.orchestra.run.smithi052.stdout: "segadd": 0, 2024-05-15T14:12:32.121 INFO:teuthology.orchestra.run.smithi052.stdout: "segex": 0, 2024-05-15T14:12:32.121 INFO:teuthology.orchestra.run.smithi052.stdout: "segtrm": 0, 2024-05-15T14:12:32.121 INFO:teuthology.orchestra.run.smithi052.stdout: "seg": 39, 2024-05-15T14:12:32.121 INFO:teuthology.orchestra.run.smithi052.stdout: "segmjr": 4, 2024-05-15T14:12:32.121 INFO:teuthology.orchestra.run.smithi052.stdout: "segexg": 0, 2024-05-15T14:12:32.122 INFO:teuthology.orchestra.run.smithi052.stdout: "segexd": 0, 2024-05-15T14:12:32.122 INFO:teuthology.orchestra.run.smithi052.stdout: "expos": 4194304, 2024-05-15T14:12:32.122 INFO:teuthology.orchestra.run.smithi052.stdout: "wrpos": 58347791, 2024-05-15T14:12:32.122 INFO:teuthology.orchestra.run.smithi052.stdout: "rdpos": 58309062, 2024-05-15T14:12:32.122 INFO:teuthology.orchestra.run.smithi052.stdout: "jlat": { 2024-05-15T14:12:32.122 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.122 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.123 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.123 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.123 INFO:teuthology.orchestra.run.smithi052.stdout: "replayed": 32457 2024-05-15T14:12:32.123 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.123 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_mem": { 2024-05-15T14:12:32.123 INFO:teuthology.orchestra.run.smithi052.stdout: "ino": 265, 2024-05-15T14:12:32.124 INFO:teuthology.orchestra.run.smithi052.stdout: "ino+": 4094, 2024-05-15T14:12:32.124 INFO:teuthology.orchestra.run.smithi052.stdout: "ino-": 3829, 2024-05-15T14:12:32.124 INFO:teuthology.orchestra.run.smithi052.stdout: "dir": 26, 2024-05-15T14:12:32.124 INFO:teuthology.orchestra.run.smithi052.stdout: "dir+": 46, 2024-05-15T14:12:32.124 INFO:teuthology.orchestra.run.smithi052.stdout: "dir-": 20, 2024-05-15T14:12:32.124 INFO:teuthology.orchestra.run.smithi052.stdout: "dn": 3997, 2024-05-15T14:12:32.124 INFO:teuthology.orchestra.run.smithi052.stdout: "dn+": 4187, 2024-05-15T14:12:32.124 INFO:teuthology.orchestra.run.smithi052.stdout: "dn-": 190, 2024-05-15T14:12:32.124 INFO:teuthology.orchestra.run.smithi052.stdout: "cap": 0, 2024-05-15T14:12:32.125 INFO:teuthology.orchestra.run.smithi052.stdout: "cap+": 0, 2024-05-15T14:12:32.125 INFO:teuthology.orchestra.run.smithi052.stdout: "cap-": 0, 2024-05-15T14:12:32.125 INFO:teuthology.orchestra.run.smithi052.stdout: "rss": 56716, 2024-05-15T14:12:32.125 INFO:teuthology.orchestra.run.smithi052.stdout: "heap": 215416 2024-05-15T14:12:32.125 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.125 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_server": { 2024-05-15T14:12:32.126 INFO:teuthology.orchestra.run.smithi052.stdout: "dispatch_client_request": 0, 2024-05-15T14:12:32.126 INFO:teuthology.orchestra.run.smithi052.stdout: "dispatch_server_request": 0, 2024-05-15T14:12:32.126 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_request": 0, 2024-05-15T14:12:32.126 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_session": 0, 2024-05-15T14:12:32.126 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_peer_request": 0, 2024-05-15T14:12:32.126 INFO:teuthology.orchestra.run.smithi052.stdout: "req_create_latency": { 2024-05-15T14:12:32.126 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.127 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.127 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.127 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.127 INFO:teuthology.orchestra.run.smithi052.stdout: "req_getattr_latency": { 2024-05-15T14:12:32.127 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.127 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.127 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.128 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.128 INFO:teuthology.orchestra.run.smithi052.stdout: "req_getfilelock_latency": { 2024-05-15T14:12:32.128 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.128 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.128 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.128 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.128 INFO:teuthology.orchestra.run.smithi052.stdout: "req_link_latency": { 2024-05-15T14:12:32.128 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.129 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.129 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.129 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.129 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookup_latency": { 2024-05-15T14:12:32.129 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.129 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.129 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.129 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.130 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookuphash_latency": { 2024-05-15T14:12:32.130 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.130 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.130 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.130 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.130 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookupino_latency": { 2024-05-15T14:12:32.130 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.130 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.131 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.131 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.131 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookupname_latency": { 2024-05-15T14:12:32.131 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.131 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.131 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.131 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.131 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookupparent_latency": { 2024-05-15T14:12:32.131 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.132 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.132 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.132 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.132 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookupsnap_latency": { 2024-05-15T14:12:32.132 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.132 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.132 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.133 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.133 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lssnap_latency": { 2024-05-15T14:12:32.133 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.133 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.133 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.133 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.133 INFO:teuthology.orchestra.run.smithi052.stdout: "req_mkdir_latency": { 2024-05-15T14:12:32.134 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.134 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.134 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.135 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.135 INFO:teuthology.orchestra.run.smithi052.stdout: "req_mknod_latency": { 2024-05-15T14:12:32.135 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.135 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.135 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.135 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.135 INFO:teuthology.orchestra.run.smithi052.stdout: "req_mksnap_latency": { 2024-05-15T14:12:32.135 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.136 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.136 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.136 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.136 INFO:teuthology.orchestra.run.smithi052.stdout: "req_open_latency": { 2024-05-15T14:12:32.136 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.136 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.136 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.137 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.137 INFO:teuthology.orchestra.run.smithi052.stdout: "req_readdir_latency": { 2024-05-15T14:12:32.137 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.137 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.137 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.137 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.137 INFO:teuthology.orchestra.run.smithi052.stdout: "req_rename_latency": { 2024-05-15T14:12:32.138 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.138 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.138 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.138 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.138 INFO:teuthology.orchestra.run.smithi052.stdout: "req_renamesnap_latency": { 2024-05-15T14:12:32.138 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.138 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.138 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.139 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.139 INFO:teuthology.orchestra.run.smithi052.stdout: "req_snapdiff_latency": { 2024-05-15T14:12:32.139 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.139 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.139 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.139 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.139 INFO:teuthology.orchestra.run.smithi052.stdout: "req_rmdir_latency": { 2024-05-15T14:12:32.140 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.140 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.140 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.140 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.140 INFO:teuthology.orchestra.run.smithi052.stdout: "req_rmsnap_latency": { 2024-05-15T14:12:32.140 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.140 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.140 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.141 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.141 INFO:teuthology.orchestra.run.smithi052.stdout: "req_rmxattr_latency": { 2024-05-15T14:12:32.141 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.141 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.141 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.141 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.141 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setattr_latency": { 2024-05-15T14:12:32.141 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.141 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.142 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.142 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.142 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setdirlayout_latency": { 2024-05-15T14:12:32.142 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.142 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.142 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.142 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.143 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setfilelock_latency": { 2024-05-15T14:12:32.143 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.143 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.143 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.143 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.143 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setlayout_latency": { 2024-05-15T14:12:32.143 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.143 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.144 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.144 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.144 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setxattr_latency": { 2024-05-15T14:12:32.144 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.144 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.144 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.144 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.144 INFO:teuthology.orchestra.run.smithi052.stdout: "req_symlink_latency": { 2024-05-15T14:12:32.145 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.145 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.145 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.145 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.145 INFO:teuthology.orchestra.run.smithi052.stdout: "req_unlink_latency": { 2024-05-15T14:12:32.145 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.145 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.145 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.146 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.146 INFO:teuthology.orchestra.run.smithi052.stdout: "cap_revoke_eviction": 0, 2024-05-15T14:12:32.146 INFO:teuthology.orchestra.run.smithi052.stdout: "cap_acquisition_throttle": 0, 2024-05-15T14:12:32.146 INFO:teuthology.orchestra.run.smithi052.stdout: "req_getvxattr_latency": { 2024-05-15T14:12:32.146 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.146 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.146 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.147 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-15T14:12:32.147 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.147 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_sessions": { 2024-05-15T14:12:32.147 INFO:teuthology.orchestra.run.smithi052.stdout: "session_count": 2, 2024-05-15T14:12:32.147 INFO:teuthology.orchestra.run.smithi052.stdout: "session_add": 4, 2024-05-15T14:12:32.147 INFO:teuthology.orchestra.run.smithi052.stdout: "session_remove": 2, 2024-05-15T14:12:32.147 INFO:teuthology.orchestra.run.smithi052.stdout: "sessions_open": 3, 2024-05-15T14:12:32.147 INFO:teuthology.orchestra.run.smithi052.stdout: "sessions_stale": 0, 2024-05-15T14:12:32.148 INFO:teuthology.orchestra.run.smithi052.stdout: "total_load": 0, 2024-05-15T14:12:32.148 INFO:teuthology.orchestra.run.smithi052.stdout: "average_load": 0, 2024-05-15T14:12:32.148 INFO:teuthology.orchestra.run.smithi052.stdout: "avg_session_uptime": 2827, 2024-05-15T14:12:32.148 INFO:teuthology.orchestra.run.smithi052.stdout: "mdthresh_evicted": 0 2024-05-15T14:12:32.148 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.148 INFO:teuthology.orchestra.run.smithi052.stdout: "mempool": { 2024-05-15T14:12:32.148 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_bytes": 0, 2024-05-15T14:12:32.148 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_items": 0, 2024-05-15T14:12:32.148 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_bytes": 0, 2024-05-15T14:12:32.149 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_items": 0, 2024-05-15T14:12:32.149 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_bytes": 0, 2024-05-15T14:12:32.149 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_items": 0, 2024-05-15T14:12:32.149 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-15T14:12:32.149 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_items": 0, 2024-05-15T14:12:32.149 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-15T14:12:32.149 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_items": 0, 2024-05-15T14:12:32.149 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_bytes": 0, 2024-05-15T14:12:32.149 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_items": 0, 2024-05-15T14:12:32.150 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-15T14:12:32.150 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_items": 0, 2024-05-15T14:12:32.150 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_bytes": 0, 2024-05-15T14:12:32.150 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_items": 0, 2024-05-15T14:12:32.150 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_bytes": 0, 2024-05-15T14:12:32.150 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_items": 0, 2024-05-15T14:12:32.150 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-15T14:12:32.150 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_items": 0, 2024-05-15T14:12:32.151 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-15T14:12:32.151 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_items": 0, 2024-05-15T14:12:32.151 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_bytes": 0, 2024-05-15T14:12:32.151 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_items": 0, 2024-05-15T14:12:32.151 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_bytes": 0, 2024-05-15T14:12:32.151 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_items": 0, 2024-05-15T14:12:32.152 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-15T14:12:32.152 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_items": 0, 2024-05-15T14:12:32.152 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_bytes": 0, 2024-05-15T14:12:32.152 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_items": 0, 2024-05-15T14:12:32.152 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_bytes": 0, 2024-05-15T14:12:32.152 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_items": 0, 2024-05-15T14:12:32.152 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_bytes": 0, 2024-05-15T14:12:32.153 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_items": 0, 2024-05-15T14:12:32.153 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_bytes": 0, 2024-05-15T14:12:32.153 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_items": 0, 2024-05-15T14:12:32.153 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_bytes": 234628, 2024-05-15T14:12:32.153 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_items": 338, 2024-05-15T14:12:32.153 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_bytes": 0, 2024-05-15T14:12:32.153 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_items": 0, 2024-05-15T14:12:32.153 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_bytes": 0, 2024-05-15T14:12:32.154 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_items": 0, 2024-05-15T14:12:32.154 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_bytes": 0, 2024-05-15T14:12:32.154 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_items": 0, 2024-05-15T14:12:32.154 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_bytes": 0, 2024-05-15T14:12:32.154 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_items": 0, 2024-05-15T14:12:32.154 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_bytes": 8128, 2024-05-15T14:12:32.154 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_items": 169, 2024-05-15T14:12:32.155 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_bytes": 0, 2024-05-15T14:12:32.155 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_items": 0, 2024-05-15T14:12:32.155 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_bytes": 0, 2024-05-15T14:12:32.155 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_items": 0, 2024-05-15T14:12:32.155 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_bytes": 3533614, 2024-05-15T14:12:32.155 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_items": 21876, 2024-05-15T14:12:32.155 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_bytes": 0, 2024-05-15T14:12:32.155 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_items": 0, 2024-05-15T14:12:32.156 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_bytes": 0, 2024-05-15T14:12:32.156 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_items": 0 2024-05-15T14:12:32.156 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.156 INFO:teuthology.orchestra.run.smithi052.stdout: "objecter": { 2024-05-15T14:12:32.156 INFO:teuthology.orchestra.run.smithi052.stdout: "op_active": 0, 2024-05-15T14:12:32.156 INFO:teuthology.orchestra.run.smithi052.stdout: "op_laggy": 0, 2024-05-15T14:12:32.156 INFO:teuthology.orchestra.run.smithi052.stdout: "op_send": 2964, 2024-05-15T14:12:32.157 INFO:teuthology.orchestra.run.smithi052.stdout: "op_send_bytes": 21, 2024-05-15T14:12:32.157 INFO:teuthology.orchestra.run.smithi052.stdout: "op_resend": 3, 2024-05-15T14:12:32.157 INFO:teuthology.orchestra.run.smithi052.stdout: "op_reply": 2964, 2024-05-15T14:12:32.157 INFO:teuthology.orchestra.run.smithi052.stdout: "op_latency": { 2024-05-15T14:12:32.157 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 2964, 2024-05-15T14:12:32.157 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 1.468981295, 2024-05-15T14:12:32.158 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000495607 2024-05-15T14:12:32.158 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.158 INFO:teuthology.orchestra.run.smithi052.stdout: "op_inflight": 0, 2024-05-15T14:12:32.158 INFO:teuthology.orchestra.run.smithi052.stdout: "oplen_avg": { 2024-05-15T14:12:32.158 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 2964, 2024-05-15T14:12:32.158 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 2965 2024-05-15T14:12:32.159 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.159 INFO:teuthology.orchestra.run.smithi052.stdout: "op": 2964, 2024-05-15T14:12:32.159 INFO:teuthology.orchestra.run.smithi052.stdout: "op_r": 2964, 2024-05-15T14:12:32.159 INFO:teuthology.orchestra.run.smithi052.stdout: "op_w": 0, 2024-05-15T14:12:32.159 INFO:teuthology.orchestra.run.smithi052.stdout: "op_rmw": 0, 2024-05-15T14:12:32.159 INFO:teuthology.orchestra.run.smithi052.stdout: "op_pg": 0, 2024-05-15T14:12:32.159 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_stat": 1698, 2024-05-15T14:12:32.159 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_create": 0, 2024-05-15T14:12:32.160 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_read": 1264, 2024-05-15T14:12:32.160 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_write": 0, 2024-05-15T14:12:32.160 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_writefull": 0, 2024-05-15T14:12:32.160 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_writesame": 0, 2024-05-15T14:12:32.160 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_append": 0, 2024-05-15T14:12:32.160 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_zero": 0, 2024-05-15T14:12:32.161 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_truncate": 0, 2024-05-15T14:12:32.161 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_delete": 0, 2024-05-15T14:12:32.161 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_mapext": 0, 2024-05-15T14:12:32.161 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_sparse_read": 0, 2024-05-15T14:12:32.161 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_clonerange": 0, 2024-05-15T14:12:32.161 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_getxattr": 1, 2024-05-15T14:12:32.161 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_setxattr": 0, 2024-05-15T14:12:32.162 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_cmpxattr": 0, 2024-05-15T14:12:32.162 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_rmxattr": 0, 2024-05-15T14:12:32.162 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_resetxattrs": 0, 2024-05-15T14:12:32.162 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_call": 0, 2024-05-15T14:12:32.162 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_watch": 0, 2024-05-15T14:12:32.162 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_notify": 0, 2024-05-15T14:12:32.163 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_src_cmpxattr": 0, 2024-05-15T14:12:32.163 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_pgls": 0, 2024-05-15T14:12:32.163 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_pgls_filter": 0, 2024-05-15T14:12:32.163 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_other": 0, 2024-05-15T14:12:32.163 INFO:teuthology.orchestra.run.smithi052.stdout: "linger_active": 0, 2024-05-15T14:12:32.163 INFO:teuthology.orchestra.run.smithi052.stdout: "linger_send": 0, 2024-05-15T14:12:32.163 INFO:teuthology.orchestra.run.smithi052.stdout: "linger_resend": 0, 2024-05-15T14:12:32.164 INFO:teuthology.orchestra.run.smithi052.stdout: "linger_ping": 0, 2024-05-15T14:12:32.164 INFO:teuthology.orchestra.run.smithi052.stdout: "poolop_active": 0, 2024-05-15T14:12:32.164 INFO:teuthology.orchestra.run.smithi052.stdout: "poolop_send": 0, 2024-05-15T14:12:32.164 INFO:teuthology.orchestra.run.smithi052.stdout: "poolop_resend": 0, 2024-05-15T14:12:32.164 INFO:teuthology.orchestra.run.smithi052.stdout: "poolstat_active": 0, 2024-05-15T14:12:32.164 INFO:teuthology.orchestra.run.smithi052.stdout: "poolstat_send": 0, 2024-05-15T14:12:32.164 INFO:teuthology.orchestra.run.smithi052.stdout: "poolstat_resend": 0, 2024-05-15T14:12:32.164 INFO:teuthology.orchestra.run.smithi052.stdout: "statfs_active": 0, 2024-05-15T14:12:32.165 INFO:teuthology.orchestra.run.smithi052.stdout: "statfs_send": 0, 2024-05-15T14:12:32.165 INFO:teuthology.orchestra.run.smithi052.stdout: "statfs_resend": 0, 2024-05-15T14:12:32.165 INFO:teuthology.orchestra.run.smithi052.stdout: "command_active": 0, 2024-05-15T14:12:32.165 INFO:teuthology.orchestra.run.smithi052.stdout: "command_send": 0, 2024-05-15T14:12:32.165 INFO:teuthology.orchestra.run.smithi052.stdout: "command_resend": 0, 2024-05-15T14:12:32.165 INFO:teuthology.orchestra.run.smithi052.stdout: "map_epoch": 98, 2024-05-15T14:12:32.165 INFO:teuthology.orchestra.run.smithi052.stdout: "map_full": 0, 2024-05-15T14:12:32.166 INFO:teuthology.orchestra.run.smithi052.stdout: "map_inc": 18, 2024-05-15T14:12:32.166 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_sessions": 9, 2024-05-15T14:12:32.166 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_session_open": 9, 2024-05-15T14:12:32.166 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_session_close": 0, 2024-05-15T14:12:32.166 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_laggy": 0, 2024-05-15T14:12:32.166 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_wr": 0, 2024-05-15T14:12:32.166 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_rd": 2, 2024-05-15T14:12:32.166 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_del": 0 2024-05-15T14:12:32.167 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.167 INFO:teuthology.orchestra.run.smithi052.stdout: "oft": { 2024-05-15T14:12:32.167 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_total_objs": 0, 2024-05-15T14:12:32.167 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_total_kv_pairs": 0, 2024-05-15T14:12:32.167 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_total_updates": 0, 2024-05-15T14:12:32.167 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_total_removes": 0 2024-05-15T14:12:32.167 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.168 INFO:teuthology.orchestra.run.smithi052.stdout: "osd-slow-ops": { 2024-05-15T14:12:32.168 INFO:teuthology.orchestra.run.smithi052.stdout: "slow_ops_count": 0 2024-05-15T14:12:32.168 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.168 INFO:teuthology.orchestra.run.smithi052.stdout: "purge_queue": { 2024-05-15T14:12:32.168 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executing_ops": 0, 2024-05-15T14:12:32.168 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executing_ops_high_water": 0, 2024-05-15T14:12:32.168 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executing": 0, 2024-05-15T14:12:32.168 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executing_high_water": 0, 2024-05-15T14:12:32.168 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executed_ops": 0, 2024-05-15T14:12:32.169 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executed": 0, 2024-05-15T14:12:32.169 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_item_in_journal": 0 2024-05-15T14:12:32.169 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.169 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-15T14:12:32.169 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-05-15T14:12:32.169 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 104857600, 2024-05-15T14:12:32.170 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-05-15T14:12:32.170 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 3270, 2024-05-15T14:12:32.170 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 56496654, 2024-05-15T14:12:32.170 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:32.170 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 3270, 2024-05-15T14:12:32.170 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-05-15T14:12:32.170 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-05-15T14:12:32.171 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 3270, 2024-05-15T14:12:32.171 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 56496654, 2024-05-15T14:12:32.171 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-05-15T14:12:32.171 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.171 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.171 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.171 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-15T14:12:32.171 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.171 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-objecter_bytes": { 2024-05-15T14:12:32.172 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-05-15T14:12:32.172 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 104857600, 2024-05-15T14:12:32.172 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-05-15T14:12:32.172 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 0, 2024-05-15T14:12:32.172 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 0, 2024-05-15T14:12:32.172 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:32.172 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 0, 2024-05-15T14:12:32.173 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 2964, 2024-05-15T14:12:32.173 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 54156250, 2024-05-15T14:12:32.173 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 2964, 2024-05-15T14:12:32.173 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 54156250, 2024-05-15T14:12:32.173 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-05-15T14:12:32.173 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.173 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.173 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.174 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-15T14:12:32.174 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.174 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-objecter_ops": { 2024-05-15T14:12:32.174 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-05-15T14:12:32.174 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 1024, 2024-05-15T14:12:32.174 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-05-15T14:12:32.174 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 0, 2024-05-15T14:12:32.175 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 0, 2024-05-15T14:12:32.175 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:32.175 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 0, 2024-05-15T14:12:32.175 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 2964, 2024-05-15T14:12:32.175 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 2964, 2024-05-15T14:12:32.175 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 2964, 2024-05-15T14:12:32.175 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 2964, 2024-05-15T14:12:32.175 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-05-15T14:12:32.176 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.176 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.176 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.176 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-15T14:12:32.176 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.176 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-write_buf_throttle": { 2024-05-15T14:12:32.176 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-05-15T14:12:32.177 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 3758096384, 2024-05-15T14:12:32.177 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-05-15T14:12:32.177 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 0, 2024-05-15T14:12:32.177 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 0, 2024-05-15T14:12:32.177 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:32.177 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 0, 2024-05-15T14:12:32.178 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-05-15T14:12:32.178 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-05-15T14:12:32.178 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 0, 2024-05-15T14:12:32.178 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 0, 2024-05-15T14:12:32.178 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-05-15T14:12:32.178 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.178 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.179 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.179 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-15T14:12:32.179 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:32.179 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-write_buf_throttle-0x55f11d6186e0": { 2024-05-15T14:12:32.179 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-05-15T14:12:32.179 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 3758096384, 2024-05-15T14:12:32.180 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-05-15T14:12:32.180 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 0, 2024-05-15T14:12:32.180 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 0, 2024-05-15T14:12:32.180 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:32.180 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 0, 2024-05-15T14:12:32.180 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-05-15T14:12:32.180 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-05-15T14:12:32.180 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 0, 2024-05-15T14:12:32.181 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 0, 2024-05-15T14:12:32.181 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-05-15T14:12:32.181 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:32.181 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:32.181 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:32.181 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-15T14:12:32.181 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-15T14:12:32.181 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-15T14:12:32.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:32 smithi033 ceph-mon[31650]: pgmap v383: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 860 KiB/s wr, 35 op/s 2024-05-15T14:12:32.737 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:32 smithi052 ceph-mon[36190]: pgmap v383: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 860 KiB/s wr, 35 op/s 2024-05-15T14:12:32.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:32 smithi081 ceph-mon[36434]: pgmap v383: 97 pgs: 97 active+clean; 202 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 860 KiB/s wr, 35 op/s 2024-05-15T14:12:32.903 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-05-15T14:12:32.903 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-05-15T14:12:32.903 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-05-15T14:12:32.903 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-05-15T14:12:32.903 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-05-15T14:12:32.903 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-05-15T14:12:32.903 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-05-15T14:12:32.904 DEBUG:tasks.check_counter:Getting stats from k 2024-05-15T14:12:32.904 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-15T14:12:33.151 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.b/config 2024-05-15T14:12:34.532 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:34 smithi052 ceph-mon[36190]: pgmap v384: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 882 KiB/s wr, 40 op/s 2024-05-15T14:12:34.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:34 smithi033 ceph-mon[31650]: pgmap v384: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 882 KiB/s wr, 40 op/s 2024-05-15T14:12:34.817 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-05-15T14:12:34.817 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-0": { 2024-05-15T14:12:34.817 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 0, 2024-05-15T14:12:34.817 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 2, 2024-05-15T14:12:34.817 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 0, 2024-05-15T14:12:34.817 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 0, 2024-05-15T14:12:34.818 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 4, 2024-05-15T14:12:34.818 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 0, 2024-05-15T14:12:34.818 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.001345262, 2024-05-15T14:12:34.818 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.000000000, 2024-05-15T14:12:34.818 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.000374428, 2024-05-15T14:12:34.818 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-15T14:12:34.818 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:34.818 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:34.819 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:34.819 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:34.819 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:34.819 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:34.819 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:34.819 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:34.819 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:34.819 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:34.820 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-15T14:12:34.820 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 0 2024-05-15T14:12:34.820 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:34.820 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-1": { 2024-05-15T14:12:34.820 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 2, 2024-05-15T14:12:34.820 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 197, 2024-05-15T14:12:34.820 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 320, 2024-05-15T14:12:34.820 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 192064, 2024-05-15T14:12:34.821 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 5, 2024-05-15T14:12:34.821 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 3, 2024-05-15T14:12:34.821 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.012817499, 2024-05-15T14:12:34.821 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.008636238, 2024-05-15T14:12:34.821 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.001564922, 2024-05-15T14:12:34.821 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-15T14:12:34.821 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:34.821 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 194, 2024-05-15T14:12:34.821 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.018992693, 2024-05-15T14:12:34.822 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000097900 2024-05-15T14:12:34.822 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:34.822 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:34.822 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:34.822 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:34.822 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:34.822 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:34.822 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-15T14:12:34.823 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 192064 2024-05-15T14:12:34.823 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:34.823 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-2": { 2024-05-15T14:12:34.823 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 272, 2024-05-15T14:12:34.823 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 259, 2024-05-15T14:12:34.823 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 1726976, 2024-05-15T14:12:34.823 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 158928, 2024-05-15T14:12:34.823 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 2, 2024-05-15T14:12:34.823 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 1, 2024-05-15T14:12:34.824 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.054247403, 2024-05-15T14:12:34.824 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.014838594, 2024-05-15T14:12:34.824 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.044558390, 2024-05-15T14:12:34.824 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.006121445, 2024-05-15T14:12:34.824 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:34.824 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 259, 2024-05-15T14:12:34.824 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.013596526, 2024-05-15T14:12:34.824 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000052496 2024-05-15T14:12:34.825 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:34.825 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:34.825 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:34.825 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:34.825 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:34.825 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:34.825 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 1726976, 2024-05-15T14:12:34.825 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 158928 2024-05-15T14:12:34.825 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:34.826 INFO:teuthology.orchestra.run.smithi052.stdout: "cct": { 2024-05-15T14:12:34.826 INFO:teuthology.orchestra.run.smithi052.stdout: "total_workers": 0, 2024-05-15T14:12:34.826 INFO:teuthology.orchestra.run.smithi052.stdout: "unhealthy_workers": 0 2024-05-15T14:12:34.826 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:34.826 INFO:teuthology.orchestra.run.smithi052.stdout: "mempool": { 2024-05-15T14:12:34.826 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_bytes": 0, 2024-05-15T14:12:34.826 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_items": 0, 2024-05-15T14:12:34.826 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_bytes": 0, 2024-05-15T14:12:34.827 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_items": 0, 2024-05-15T14:12:34.827 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_bytes": 0, 2024-05-15T14:12:34.827 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_items": 0, 2024-05-15T14:12:34.827 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-15T14:12:34.827 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_items": 0, 2024-05-15T14:12:34.827 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-15T14:12:34.827 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_items": 0, 2024-05-15T14:12:34.827 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_bytes": 0, 2024-05-15T14:12:34.827 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_items": 0, 2024-05-15T14:12:34.828 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-15T14:12:34.828 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_items": 0, 2024-05-15T14:12:34.828 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_bytes": 0, 2024-05-15T14:12:34.828 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_items": 0, 2024-05-15T14:12:34.828 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_bytes": 0, 2024-05-15T14:12:34.828 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_items": 0, 2024-05-15T14:12:34.828 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-15T14:12:34.828 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_items": 0, 2024-05-15T14:12:34.829 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-15T14:12:34.829 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_items": 0, 2024-05-15T14:12:34.829 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_bytes": 0, 2024-05-15T14:12:34.829 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_items": 0, 2024-05-15T14:12:34.829 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_bytes": 0, 2024-05-15T14:12:34.829 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_items": 0, 2024-05-15T14:12:34.829 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-15T14:12:34.829 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_items": 0, 2024-05-15T14:12:34.830 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_bytes": 0, 2024-05-15T14:12:34.830 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_items": 0, 2024-05-15T14:12:34.830 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_bytes": 0, 2024-05-15T14:12:34.830 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_items": 0, 2024-05-15T14:12:34.830 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_bytes": 0, 2024-05-15T14:12:34.830 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_items": 0, 2024-05-15T14:12:34.830 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_bytes": 0, 2024-05-15T14:12:34.830 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_items": 0, 2024-05-15T14:12:34.830 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_bytes": 148721, 2024-05-15T14:12:34.831 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_items": 30, 2024-05-15T14:12:34.831 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_bytes": 0, 2024-05-15T14:12:34.831 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_items": 0, 2024-05-15T14:12:34.831 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_bytes": 0, 2024-05-15T14:12:34.831 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_items": 0, 2024-05-15T14:12:34.831 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_bytes": 0, 2024-05-15T14:12:34.831 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_items": 0, 2024-05-15T14:12:34.831 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_bytes": 0, 2024-05-15T14:12:34.832 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_items": 0, 2024-05-15T14:12:34.832 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_bytes": 0, 2024-05-15T14:12:34.832 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_items": 0, 2024-05-15T14:12:34.832 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_bytes": 0, 2024-05-15T14:12:34.832 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_items": 0, 2024-05-15T14:12:34.832 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_bytes": 0, 2024-05-15T14:12:34.832 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_items": 0, 2024-05-15T14:12:34.832 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_bytes": 928, 2024-05-15T14:12:34.832 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_items": 2, 2024-05-15T14:12:34.833 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_bytes": 0, 2024-05-15T14:12:34.833 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_items": 0, 2024-05-15T14:12:34.833 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_bytes": 0, 2024-05-15T14:12:34.833 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_items": 0 2024-05-15T14:12:34.833 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-05-15T14:12:34.833 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-15T14:12:34.833 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-05-15T14:12:34.833 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 104857600, 2024-05-15T14:12:34.833 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-05-15T14:12:34.834 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 270, 2024-05-15T14:12:34.834 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 1686150, 2024-05-15T14:12:34.834 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:34.834 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 270, 2024-05-15T14:12:34.834 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-05-15T14:12:34.834 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-05-15T14:12:34.834 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 270, 2024-05-15T14:12:34.834 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 1686150, 2024-05-15T14:12:34.835 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-05-15T14:12:34.835 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-05-15T14:12:34.835 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-05-15T14:12:34.835 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-05-15T14:12:34.835 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-15T14:12:34.835 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-05-15T14:12:34.835 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-05-15T14:12:34.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:34 smithi081 ceph-mon[36434]: pgmap v384: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 882 KiB/s wr, 40 op/s 2024-05-15T14:12:35.534 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:35 smithi052 ceph-mon[36190]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:12:35.565 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-05-15T14:12:35.565 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-05-15T14:12:35.565 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-05-15T14:12:35.565 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-05-15T14:12:35.565 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-05-15T14:12:35.566 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-05-15T14:12:35.566 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-05-15T14:12:35.566 DEBUG:tasks.check_counter:Getting stats from c 2024-05-15T14:12:35.566 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-15T14:12:35.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:35 smithi033 ceph-mon[31650]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:12:35.813 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.c/config 2024-05-15T14:12:35.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:35 smithi081 ceph-mon[36434]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:12:36.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:36 smithi081 ceph-mon[36434]: pgmap v385: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 870 KiB/s wr, 34 op/s 2024-05-15T14:12:36.696 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:36 smithi033 ceph-mon[31650]: pgmap v385: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 870 KiB/s wr, 34 op/s 2024-05-15T14:12:36.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:36 smithi052 ceph-mon[36190]: pgmap v385: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 870 KiB/s wr, 34 op/s 2024-05-15T14:12:37.681 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-05-15T14:12:37.682 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-0": { 2024-05-15T14:12:37.682 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 5894, 2024-05-15T14:12:37.682 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 6046, 2024-05-15T14:12:37.682 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 1490095, 2024-05-15T14:12:37.682 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 27505834, 2024-05-15T14:12:37.683 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 19, 2024-05-15T14:12:37.683 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 6, 2024-05-15T14:12:37.683 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.412055743, 2024-05-15T14:12:37.683 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.156766561, 2024-05-15T14:12:37.683 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.092983228, 2024-05-15T14:12:37.683 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.132808518, 2024-05-15T14:12:37.683 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:37.683 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 6042, 2024-05-15T14:12:37.684 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.182597414, 2024-05-15T14:12:37.684 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000030221 2024-05-15T14:12:37.684 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.684 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:37.684 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 23, 2024-05-15T14:12:37.684 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000003498, 2024-05-15T14:12:37.684 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000152 2024-05-15T14:12:37.684 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.685 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 5664, 2024-05-15T14:12:37.685 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 356496 2024-05-15T14:12:37.685 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.685 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-1": { 2024-05-15T14:12:37.685 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 185526, 2024-05-15T14:12:37.685 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 164020, 2024-05-15T14:12:37.685 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 56328809, 2024-05-15T14:12:37.685 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 129620705, 2024-05-15T14:12:37.686 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 11, 2024-05-15T14:12:37.686 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 7, 2024-05-15T14:12:37.686 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 8.257491243, 2024-05-15T14:12:37.686 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 3.629116970, 2024-05-15T14:12:37.686 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 6.121592606, 2024-05-15T14:12:37.686 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.064804721, 2024-05-15T14:12:37.686 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:37.687 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 164020, 2024-05-15T14:12:37.687 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 2.464493906, 2024-05-15T14:12:37.687 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000015025 2024-05-15T14:12:37.687 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.687 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:37.687 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 333191, 2024-05-15T14:12:37.687 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.042620201, 2024-05-15T14:12:37.687 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000127 2024-05-15T14:12:37.688 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.688 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 1888, 2024-05-15T14:12:37.688 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 153776 2024-05-15T14:12:37.688 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.688 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-2": { 2024-05-15T14:12:37.688 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 4856, 2024-05-15T14:12:37.688 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 4888, 2024-05-15T14:12:37.688 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 3466579, 2024-05-15T14:12:37.688 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 18778754, 2024-05-15T14:12:37.689 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 7, 2024-05-15T14:12:37.689 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 6, 2024-05-15T14:12:37.689 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.374618159, 2024-05-15T14:12:37.689 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.136743484, 2024-05-15T14:12:37.689 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.140992411, 2024-05-15T14:12:37.689 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.099536304, 2024-05-15T14:12:37.689 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:37.689 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 4887, 2024-05-15T14:12:37.690 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.141115837, 2024-05-15T14:12:37.690 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000028875 2024-05-15T14:12:37.690 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.690 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:37.690 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 1795, 2024-05-15T14:12:37.690 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000254534, 2024-05-15T14:12:37.690 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000141 2024-05-15T14:12:37.690 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.691 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 1842096, 2024-05-15T14:12:37.691 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 165696 2024-05-15T14:12:37.691 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.691 INFO:teuthology.orchestra.run.smithi081.stdout: "cct": { 2024-05-15T14:12:37.691 INFO:teuthology.orchestra.run.smithi081.stdout: "total_workers": 1, 2024-05-15T14:12:37.691 INFO:teuthology.orchestra.run.smithi081.stdout: "unhealthy_workers": 0 2024-05-15T14:12:37.691 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.691 INFO:teuthology.orchestra.run.smithi081.stdout: "finisher-MDSRank": { 2024-05-15T14:12:37.692 INFO:teuthology.orchestra.run.smithi081.stdout: "queue_len": 0, 2024-05-15T14:12:37.692 INFO:teuthology.orchestra.run.smithi081.stdout: "complete_latency": { 2024-05-15T14:12:37.692 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 14416, 2024-05-15T14:12:37.692 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 8.135313428, 2024-05-15T14:12:37.692 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000564325 2024-05-15T14:12:37.692 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:37.692 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.692 INFO:teuthology.orchestra.run.smithi081.stdout: "finisher-PurgeQueue": { 2024-05-15T14:12:37.693 INFO:teuthology.orchestra.run.smithi081.stdout: "queue_len": 0, 2024-05-15T14:12:37.693 INFO:teuthology.orchestra.run.smithi081.stdout: "complete_latency": { 2024-05-15T14:12:37.693 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 6512, 2024-05-15T14:12:37.693 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.367309034, 2024-05-15T14:12:37.693 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000056404 2024-05-15T14:12:37.693 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:37.693 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.693 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2024-05-15T14:12:37.693 INFO:teuthology.orchestra.run.smithi081.stdout: "request": 46938, 2024-05-15T14:12:37.694 INFO:teuthology.orchestra.run.smithi081.stdout: "reply": 39540, 2024-05-15T14:12:37.694 INFO:teuthology.orchestra.run.smithi081.stdout: "reply_latency": { 2024-05-15T14:12:37.694 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 39540, 2024-05-15T14:12:37.694 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 74.065669171, 2024-05-15T14:12:37.694 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.001873183 2024-05-15T14:12:37.694 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.694 INFO:teuthology.orchestra.run.smithi081.stdout: "slow_reply": 0, 2024-05-15T14:12:37.695 INFO:teuthology.orchestra.run.smithi081.stdout: "forward": 7398, 2024-05-15T14:12:37.695 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_fetch_complete": 1, 2024-05-15T14:12:37.695 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_fetch_keys": 0, 2024-05-15T14:12:37.695 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_commit": 19, 2024-05-15T14:12:37.695 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_split": 1, 2024-05-15T14:12:37.695 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_merge": 0, 2024-05-15T14:12:37.695 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes": 334, 2024-05-15T14:12:37.695 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_top": 91, 2024-05-15T14:12:37.695 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_bottom": 232, 2024-05-15T14:12:37.696 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_pin_tail": 11, 2024-05-15T14:12:37.696 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_pinned": 167, 2024-05-15T14:12:37.696 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_expired": 185, 2024-05-15T14:12:37.696 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_with_caps": 21, 2024-05-15T14:12:37.696 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": 21, 2024-05-15T14:12:37.696 INFO:teuthology.orchestra.run.smithi081.stdout: "subtrees": 8, 2024-05-15T14:12:37.696 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse": 75895, 2024-05-15T14:12:37.696 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_hit": 38920, 2024-05-15T14:12:37.697 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_forward": 7382, 2024-05-15T14:12:37.697 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_discover": 57, 2024-05-15T14:12:37.697 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_dir_fetch": 1, 2024-05-15T14:12:37.697 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_remote_ino": 0, 2024-05-15T14:12:37.697 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_lock": 648, 2024-05-15T14:12:37.697 INFO:teuthology.orchestra.run.smithi081.stdout: "load_cent": 9497, 2024-05-15T14:12:37.697 INFO:teuthology.orchestra.run.smithi081.stdout: "q": 0, 2024-05-15T14:12:37.697 INFO:teuthology.orchestra.run.smithi081.stdout: "exported": 9, 2024-05-15T14:12:37.697 INFO:teuthology.orchestra.run.smithi081.stdout: "exported_inodes": 179, 2024-05-15T14:12:37.698 INFO:teuthology.orchestra.run.smithi081.stdout: "imported": 7, 2024-05-15T14:12:37.698 INFO:teuthology.orchestra.run.smithi081.stdout: "imported_inodes": 280, 2024-05-15T14:12:37.698 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_dir_fetch": 0, 2024-05-15T14:12:37.698 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_backtrace_fetch": 0, 2024-05-15T14:12:37.698 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_peer_discover": 0, 2024-05-15T14:12:37.698 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rfiles": 67, 2024-05-15T14:12:37.698 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rbytes": 21176629, 2024-05-15T14:12:37.699 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rsnaps": 6, 2024-05-15T14:12:37.699 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_backtrace_fetch": 0, 2024-05-15T14:12:37.699 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_set_tag": 0, 2024-05-15T14:12:37.699 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_backtrace_repaired": 0, 2024-05-15T14:12:37.699 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_inotable_repaired": 0, 2024-05-15T14:12:37.699 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dir_inodes": 0, 2024-05-15T14:12:37.699 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dir_base_inodes": 0, 2024-05-15T14:12:37.699 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dirfrag_rstats": 0, 2024-05-15T14:12:37.699 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_file_inodes": 0, 2024-05-15T14:12:37.700 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_inode_file_caps": 7, 2024-05-15T14:12:37.700 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_revoke": 4396, 2024-05-15T14:12:37.700 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_grant": 21224, 2024-05-15T14:12:37.700 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_trunc": 0, 2024-05-15T14:12:37.700 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-15T14:12:37.700 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-15T14:12:37.700 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_caps": 13762, 2024-05-15T14:12:37.700 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_caps_dirty": 4236, 2024-05-15T14:12:37.700 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_cap_release": 2903, 2024-05-15T14:12:37.701 INFO:teuthology.orchestra.run.smithi081.stdout: "process_request_cap_release": 37135 2024-05-15T14:12:37.701 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.701 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_cache": { 2024-05-15T14:12:37.701 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_update": 236, 2024-05-15T14:12:37.701 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_update_receipt": 475, 2024-05-15T14:12:37.701 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_try_discover": 247, 2024-05-15T14:12:37.701 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_send_discover": 530, 2024-05-15T14:12:37.701 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_handle_discover": 261, 2024-05-15T14:12:37.701 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays": 0, 2024-05-15T14:12:37.702 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays_delayed": 0, 2024-05-15T14:12:37.702 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays_enqueuing": 0, 2024-05-15T14:12:37.702 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_created": 3689, 2024-05-15T14:12:37.702 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_enqueued": 3689, 2024-05-15T14:12:37.702 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_reintegrated": 0, 2024-05-15T14:12:37.702 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_migrated": 0, 2024-05-15T14:12:37.702 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_processing": 0, 2024-05-15T14:12:37.702 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_enqueued": 0, 2024-05-15T14:12:37.703 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_prioritized": 0, 2024-05-15T14:12:37.703 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_started": 0, 2024-05-15T14:12:37.703 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_completed": 0, 2024-05-15T14:12:37.703 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_quiesce_path": 10, 2024-05-15T14:12:37.703 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_quiesce_inode": 348, 2024-05-15T14:12:37.703 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_enqueue_scrub": 0, 2024-05-15T14:12:37.703 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_exportdir": 19, 2024-05-15T14:12:37.703 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_flush": 0, 2024-05-15T14:12:37.704 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_fragmentdir": 1, 2024-05-15T14:12:37.704 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_fragstats": 0, 2024-05-15T14:12:37.704 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_inodestats": 0 2024-05-15T14:12:37.704 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.704 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_log": { 2024-05-15T14:12:37.704 INFO:teuthology.orchestra.run.smithi081.stdout: "evlrg": 0, 2024-05-15T14:12:37.704 INFO:teuthology.orchestra.run.smithi081.stdout: "evadd": 32459, 2024-05-15T14:12:37.704 INFO:teuthology.orchestra.run.smithi081.stdout: "evex": 0, 2024-05-15T14:12:37.704 INFO:teuthology.orchestra.run.smithi081.stdout: "evtrm": 0, 2024-05-15T14:12:37.705 INFO:teuthology.orchestra.run.smithi081.stdout: "ev": 32459, 2024-05-15T14:12:37.705 INFO:teuthology.orchestra.run.smithi081.stdout: "evexg": 0, 2024-05-15T14:12:37.705 INFO:teuthology.orchestra.run.smithi081.stdout: "evexd": 0, 2024-05-15T14:12:37.705 INFO:teuthology.orchestra.run.smithi081.stdout: "segadd": 39, 2024-05-15T14:12:37.705 INFO:teuthology.orchestra.run.smithi081.stdout: "segex": 0, 2024-05-15T14:12:37.705 INFO:teuthology.orchestra.run.smithi081.stdout: "segtrm": 0, 2024-05-15T14:12:37.705 INFO:teuthology.orchestra.run.smithi081.stdout: "seg": 39, 2024-05-15T14:12:37.705 INFO:teuthology.orchestra.run.smithi081.stdout: "segmjr": 4, 2024-05-15T14:12:37.706 INFO:teuthology.orchestra.run.smithi081.stdout: "segexg": 0, 2024-05-15T14:12:37.706 INFO:teuthology.orchestra.run.smithi081.stdout: "segexd": 0, 2024-05-15T14:12:37.706 INFO:teuthology.orchestra.run.smithi081.stdout: "expos": 4194304, 2024-05-15T14:12:37.706 INFO:teuthology.orchestra.run.smithi081.stdout: "wrpos": 58349552, 2024-05-15T14:12:37.706 INFO:teuthology.orchestra.run.smithi081.stdout: "rdpos": 0, 2024-05-15T14:12:37.706 INFO:teuthology.orchestra.run.smithi081.stdout: "jlat": { 2024-05-15T14:12:37.706 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 7199, 2024-05-15T14:12:37.706 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 15.465496779, 2024-05-15T14:12:37.707 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.002148284 2024-05-15T14:12:37.707 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.707 INFO:teuthology.orchestra.run.smithi081.stdout: "replayed": 0 2024-05-15T14:12:37.707 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.707 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_mem": { 2024-05-15T14:12:37.707 INFO:teuthology.orchestra.run.smithi081.stdout: "ino": 288, 2024-05-15T14:12:37.707 INFO:teuthology.orchestra.run.smithi081.stdout: "ino+": 4130, 2024-05-15T14:12:37.707 INFO:teuthology.orchestra.run.smithi081.stdout: "ino-": 3842, 2024-05-15T14:12:37.707 INFO:teuthology.orchestra.run.smithi081.stdout: "dir": 37, 2024-05-15T14:12:37.708 INFO:teuthology.orchestra.run.smithi081.stdout: "dir+": 292, 2024-05-15T14:12:37.708 INFO:teuthology.orchestra.run.smithi081.stdout: "dir-": 255, 2024-05-15T14:12:37.708 INFO:teuthology.orchestra.run.smithi081.stdout: "dn": 334, 2024-05-15T14:12:37.708 INFO:teuthology.orchestra.run.smithi081.stdout: "dn+": 7718, 2024-05-15T14:12:37.708 INFO:teuthology.orchestra.run.smithi081.stdout: "dn-": 7384, 2024-05-15T14:12:37.708 INFO:teuthology.orchestra.run.smithi081.stdout: "cap": 21, 2024-05-15T14:12:37.708 INFO:teuthology.orchestra.run.smithi081.stdout: "cap+": 4679, 2024-05-15T14:12:37.708 INFO:teuthology.orchestra.run.smithi081.stdout: "cap-": 4658, 2024-05-15T14:12:37.708 INFO:teuthology.orchestra.run.smithi081.stdout: "rss": 68552, 2024-05-15T14:12:37.709 INFO:teuthology.orchestra.run.smithi081.stdout: "heap": 223608 2024-05-15T14:12:37.709 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.709 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_server": { 2024-05-15T14:12:37.709 INFO:teuthology.orchestra.run.smithi081.stdout: "dispatch_client_request": 66815, 2024-05-15T14:12:37.709 INFO:teuthology.orchestra.run.smithi081.stdout: "dispatch_server_request": 3, 2024-05-15T14:12:37.709 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_request": 46938, 2024-05-15T14:12:37.709 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_session": 757, 2024-05-15T14:12:37.709 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_peer_request": 5493, 2024-05-15T14:12:37.710 INFO:teuthology.orchestra.run.smithi081.stdout: "req_create_latency": { 2024-05-15T14:12:37.710 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 3665, 2024-05-15T14:12:37.710 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 7.210635986, 2024-05-15T14:12:37.710 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.001967431 2024-05-15T14:12:37.710 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.710 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getattr_latency": { 2024-05-15T14:12:37.710 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 147, 2024-05-15T14:12:37.710 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.278262626, 2024-05-15T14:12:37.711 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.001892943 2024-05-15T14:12:37.711 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.711 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getfilelock_latency": { 2024-05-15T14:12:37.711 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:37.711 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:37.711 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:37.711 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.711 INFO:teuthology.orchestra.run.smithi081.stdout: "req_link_latency": { 2024-05-15T14:12:37.712 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:37.712 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:37.712 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:37.712 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.712 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookup_latency": { 2024-05-15T14:12:37.712 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 7048, 2024-05-15T14:12:37.712 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 3.127586635, 2024-05-15T14:12:37.712 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000443755 2024-05-15T14:12:37.712 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.713 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookuphash_latency": { 2024-05-15T14:12:37.713 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:37.713 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:37.713 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:37.713 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.713 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupino_latency": { 2024-05-15T14:12:37.713 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 1, 2024-05-15T14:12:37.713 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000140456, 2024-05-15T14:12:37.713 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000140456 2024-05-15T14:12:37.714 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.714 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupname_latency": { 2024-05-15T14:12:37.714 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:37.714 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:37.714 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:37.718 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.718 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupparent_latency": { 2024-05-15T14:12:37.718 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:37.718 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:37.718 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:37.718 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.718 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupsnap_latency": { 2024-05-15T14:12:37.719 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:37.719 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:37.719 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:37.719 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.720 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lssnap_latency": { 2024-05-15T14:12:37.720 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:37.720 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:37.720 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:37.720 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.720 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mkdir_latency": { 2024-05-15T14:12:37.721 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:37.721 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:37.721 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:37.721 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.721 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mknod_latency": { 2024-05-15T14:12:37.721 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:37.721 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:37.721 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:37.722 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.722 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mksnap_latency": { 2024-05-15T14:12:37.722 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:37.722 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:37.722 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:37.722 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.722 INFO:teuthology.orchestra.run.smithi081.stdout: "req_open_latency": { 2024-05-15T14:12:37.722 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 1, 2024-05-15T14:12:37.723 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000588338, 2024-05-15T14:12:37.723 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000588338 2024-05-15T14:12:37.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.723 INFO:teuthology.orchestra.run.smithi081.stdout: "req_readdir_latency": { 2024-05-15T14:12:37.723 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 24093, 2024-05-15T14:12:37.723 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 10.654757724, 2024-05-15T14:12:37.723 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000442234 2024-05-15T14:12:37.723 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.724 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rename_latency": { 2024-05-15T14:12:37.724 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 648, 2024-05-15T14:12:37.724 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 1.263193987, 2024-05-15T14:12:37.724 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.001949373 2024-05-15T14:12:37.724 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.724 INFO:teuthology.orchestra.run.smithi081.stdout: "req_renamesnap_latency": { 2024-05-15T14:12:37.724 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:37.724 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:37.725 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:37.725 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.725 INFO:teuthology.orchestra.run.smithi081.stdout: "req_snapdiff_latency": { 2024-05-15T14:12:37.725 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:37.725 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:37.725 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:37.725 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.725 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmdir_latency": { 2024-05-15T14:12:37.726 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:37.726 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:37.726 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:37.726 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.726 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmsnap_latency": { 2024-05-15T14:12:37.726 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:37.726 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:37.726 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:37.727 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.727 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmxattr_latency": { 2024-05-15T14:12:37.727 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:37.727 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:37.727 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:37.727 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.727 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setattr_latency": { 2024-05-15T14:12:37.727 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 52, 2024-05-15T14:12:37.728 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.098872098, 2024-05-15T14:12:37.728 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.001901386 2024-05-15T14:12:37.728 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.728 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setdirlayout_latency": { 2024-05-15T14:12:37.728 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:37.728 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:37.728 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:37.728 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.729 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setfilelock_latency": { 2024-05-15T14:12:37.729 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 196, 2024-05-15T14:12:37.729 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.091532154, 2024-05-15T14:12:37.729 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000467000 2024-05-15T14:12:37.729 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.729 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setlayout_latency": { 2024-05-15T14:12:37.729 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:37.729 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:37.730 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:37.730 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.730 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setxattr_latency": { 2024-05-15T14:12:37.730 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:37.730 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:37.730 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:37.730 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.730 INFO:teuthology.orchestra.run.smithi081.stdout: "req_symlink_latency": { 2024-05-15T14:12:37.731 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:37.731 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:37.731 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:37.731 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.731 INFO:teuthology.orchestra.run.smithi081.stdout: "req_unlink_latency": { 2024-05-15T14:12:37.731 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 3689, 2024-05-15T14:12:37.731 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 51.340099167, 2024-05-15T14:12:37.731 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.013917077 2024-05-15T14:12:37.732 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.732 INFO:teuthology.orchestra.run.smithi081.stdout: "cap_revoke_eviction": 0, 2024-05-15T14:12:37.732 INFO:teuthology.orchestra.run.smithi081.stdout: "cap_acquisition_throttle": 0, 2024-05-15T14:12:37.732 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getvxattr_latency": { 2024-05-15T14:12:37.732 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:37.732 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:37.732 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:37.732 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:37.732 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.733 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_sessions": { 2024-05-15T14:12:37.733 INFO:teuthology.orchestra.run.smithi081.stdout: "session_count": 2, 2024-05-15T14:12:37.733 INFO:teuthology.orchestra.run.smithi081.stdout: "session_add": 4, 2024-05-15T14:12:37.733 INFO:teuthology.orchestra.run.smithi081.stdout: "session_remove": 2, 2024-05-15T14:12:37.733 INFO:teuthology.orchestra.run.smithi081.stdout: "sessions_open": 2, 2024-05-15T14:12:37.733 INFO:teuthology.orchestra.run.smithi081.stdout: "sessions_stale": 0, 2024-05-15T14:12:37.733 INFO:teuthology.orchestra.run.smithi081.stdout: "total_load": 5594, 2024-05-15T14:12:37.733 INFO:teuthology.orchestra.run.smithi081.stdout: "average_load": 2797, 2024-05-15T14:12:37.733 INFO:teuthology.orchestra.run.smithi081.stdout: "avg_session_uptime": 714, 2024-05-15T14:12:37.734 INFO:teuthology.orchestra.run.smithi081.stdout: "mdthresh_evicted": 0 2024-05-15T14:12:37.734 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.734 INFO:teuthology.orchestra.run.smithi081.stdout: "mempool": { 2024-05-15T14:12:37.734 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_bytes": 0, 2024-05-15T14:12:37.734 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_items": 0, 2024-05-15T14:12:37.734 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_bytes": 0, 2024-05-15T14:12:37.734 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_items": 0, 2024-05-15T14:12:37.734 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_bytes": 0, 2024-05-15T14:12:37.735 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_items": 0, 2024-05-15T14:12:37.735 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-15T14:12:37.735 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_items": 0, 2024-05-15T14:12:37.735 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-15T14:12:37.735 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_items": 0, 2024-05-15T14:12:37.735 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_bytes": 0, 2024-05-15T14:12:37.735 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_items": 0, 2024-05-15T14:12:37.736 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-15T14:12:37.736 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_items": 0, 2024-05-15T14:12:37.736 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_bytes": 0, 2024-05-15T14:12:37.736 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_items": 0, 2024-05-15T14:12:37.736 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_bytes": 0, 2024-05-15T14:12:37.736 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_items": 0, 2024-05-15T14:12:37.737 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-15T14:12:37.737 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_items": 0, 2024-05-15T14:12:37.737 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-15T14:12:37.737 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_items": 0, 2024-05-15T14:12:37.737 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_bytes": 0, 2024-05-15T14:12:37.737 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_items": 0, 2024-05-15T14:12:37.737 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_bytes": 0, 2024-05-15T14:12:37.738 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_items": 0, 2024-05-15T14:12:37.738 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-15T14:12:37.738 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_items": 0, 2024-05-15T14:12:37.738 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_bytes": 0, 2024-05-15T14:12:37.738 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_items": 0, 2024-05-15T14:12:37.738 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_bytes": 0, 2024-05-15T14:12:37.738 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_items": 0, 2024-05-15T14:12:37.739 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_bytes": 0, 2024-05-15T14:12:37.739 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_items": 0, 2024-05-15T14:12:37.739 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_bytes": 0, 2024-05-15T14:12:37.739 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_items": 0, 2024-05-15T14:12:37.739 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_bytes": 314192, 2024-05-15T14:12:37.739 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_items": 370, 2024-05-15T14:12:37.739 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_bytes": 0, 2024-05-15T14:12:37.739 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_items": 0, 2024-05-15T14:12:37.740 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_bytes": 0, 2024-05-15T14:12:37.740 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_items": 0, 2024-05-15T14:12:37.740 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_bytes": 0, 2024-05-15T14:12:37.740 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_items": 0, 2024-05-15T14:12:37.740 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_bytes": 0, 2024-05-15T14:12:37.740 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_items": 0, 2024-05-15T14:12:37.740 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_bytes": 8128, 2024-05-15T14:12:37.740 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_items": 169, 2024-05-15T14:12:37.741 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_bytes": 0, 2024-05-15T14:12:37.741 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_items": 0, 2024-05-15T14:12:37.741 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_bytes": 0, 2024-05-15T14:12:37.741 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_items": 0, 2024-05-15T14:12:37.741 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_bytes": 1057370, 2024-05-15T14:12:37.741 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_items": 8894, 2024-05-15T14:12:37.741 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_bytes": 0, 2024-05-15T14:12:37.741 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_items": 0, 2024-05-15T14:12:37.742 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_bytes": 0, 2024-05-15T14:12:37.742 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_items": 0 2024-05-15T14:12:37.742 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.742 INFO:teuthology.orchestra.run.smithi081.stdout: "objecter": { 2024-05-15T14:12:37.742 INFO:teuthology.orchestra.run.smithi081.stdout: "op_active": 0, 2024-05-15T14:12:37.742 INFO:teuthology.orchestra.run.smithi081.stdout: "op_laggy": 0, 2024-05-15T14:12:37.742 INFO:teuthology.orchestra.run.smithi081.stdout: "op_send": 12025, 2024-05-15T14:12:37.743 INFO:teuthology.orchestra.run.smithi081.stdout: "op_send_bytes": 55093647, 2024-05-15T14:12:37.743 INFO:teuthology.orchestra.run.smithi081.stdout: "op_resend": 33, 2024-05-15T14:12:37.743 INFO:teuthology.orchestra.run.smithi081.stdout: "op_reply": 12025, 2024-05-15T14:12:37.743 INFO:teuthology.orchestra.run.smithi081.stdout: "op_latency": { 2024-05-15T14:12:37.743 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 12025, 2024-05-15T14:12:37.743 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 31.716439920, 2024-05-15T14:12:37.743 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.002637541 2024-05-15T14:12:37.743 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.744 INFO:teuthology.orchestra.run.smithi081.stdout: "op_inflight": 0, 2024-05-15T14:12:37.744 INFO:teuthology.orchestra.run.smithi081.stdout: "oplen_avg": { 2024-05-15T14:12:37.744 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 12025, 2024-05-15T14:12:37.744 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 12562 2024-05-15T14:12:37.744 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.744 INFO:teuthology.orchestra.run.smithi081.stdout: "op": 12025, 2024-05-15T14:12:37.744 INFO:teuthology.orchestra.run.smithi081.stdout: "op_r": 267, 2024-05-15T14:12:37.745 INFO:teuthology.orchestra.run.smithi081.stdout: "op_w": 11758, 2024-05-15T14:12:37.745 INFO:teuthology.orchestra.run.smithi081.stdout: "op_rmw": 0, 2024-05-15T14:12:37.745 INFO:teuthology.orchestra.run.smithi081.stdout: "op_pg": 0, 2024-05-15T14:12:37.745 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_stat": 0, 2024-05-15T14:12:37.745 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_create": 10, 2024-05-15T14:12:37.745 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_read": 266, 2024-05-15T14:12:37.745 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_write": 7465, 2024-05-15T14:12:37.745 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_writefull": 180, 2024-05-15T14:12:37.745 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_writesame": 0, 2024-05-15T14:12:37.746 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_append": 0, 2024-05-15T14:12:37.746 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_zero": 0, 2024-05-15T14:12:37.746 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_truncate": 1, 2024-05-15T14:12:37.746 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_delete": 3713, 2024-05-15T14:12:37.746 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_mapext": 0, 2024-05-15T14:12:37.746 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_sparse_read": 0, 2024-05-15T14:12:37.746 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_clonerange": 0, 2024-05-15T14:12:37.746 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_getxattr": 1, 2024-05-15T14:12:37.747 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_setxattr": 20, 2024-05-15T14:12:37.747 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_cmpxattr": 0, 2024-05-15T14:12:37.747 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_rmxattr": 0, 2024-05-15T14:12:37.747 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_resetxattrs": 0, 2024-05-15T14:12:37.747 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_call": 0, 2024-05-15T14:12:37.747 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_watch": 0, 2024-05-15T14:12:37.747 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_notify": 0, 2024-05-15T14:12:37.747 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_src_cmpxattr": 0, 2024-05-15T14:12:37.748 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_pgls": 0, 2024-05-15T14:12:37.748 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_pgls_filter": 0, 2024-05-15T14:12:37.748 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_other": 0, 2024-05-15T14:12:37.748 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_active": 0, 2024-05-15T14:12:37.748 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_send": 0, 2024-05-15T14:12:37.748 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_resend": 0, 2024-05-15T14:12:37.748 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_ping": 0, 2024-05-15T14:12:37.748 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_active": 0, 2024-05-15T14:12:37.748 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_send": 0, 2024-05-15T14:12:37.749 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_resend": 0, 2024-05-15T14:12:37.749 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_active": 0, 2024-05-15T14:12:37.749 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_send": 0, 2024-05-15T14:12:37.749 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_resend": 0, 2024-05-15T14:12:37.749 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_active": 0, 2024-05-15T14:12:37.749 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_send": 0, 2024-05-15T14:12:37.749 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_resend": 0, 2024-05-15T14:12:37.749 INFO:teuthology.orchestra.run.smithi081.stdout: "command_active": 0, 2024-05-15T14:12:37.750 INFO:teuthology.orchestra.run.smithi081.stdout: "command_send": 0, 2024-05-15T14:12:37.750 INFO:teuthology.orchestra.run.smithi081.stdout: "command_resend": 0, 2024-05-15T14:12:37.750 INFO:teuthology.orchestra.run.smithi081.stdout: "map_epoch": 98, 2024-05-15T14:12:37.750 INFO:teuthology.orchestra.run.smithi081.stdout: "map_full": 0, 2024-05-15T14:12:37.750 INFO:teuthology.orchestra.run.smithi081.stdout: "map_inc": 18, 2024-05-15T14:12:37.750 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_sessions": 12, 2024-05-15T14:12:37.750 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_session_open": 12, 2024-05-15T14:12:37.751 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_session_close": 0, 2024-05-15T14:12:37.751 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_laggy": 0, 2024-05-15T14:12:37.751 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_wr": 714, 2024-05-15T14:12:37.751 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_rd": 2, 2024-05-15T14:12:37.751 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_del": 190 2024-05-15T14:12:37.751 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.751 INFO:teuthology.orchestra.run.smithi081.stdout: "oft": { 2024-05-15T14:12:37.751 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_objs": 1, 2024-05-15T14:12:37.752 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_kv_pairs": 0, 2024-05-15T14:12:37.752 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_updates": 0, 2024-05-15T14:12:37.752 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_removes": 0 2024-05-15T14:12:37.752 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.752 INFO:teuthology.orchestra.run.smithi081.stdout: "osd-slow-ops": { 2024-05-15T14:12:37.752 INFO:teuthology.orchestra.run.smithi081.stdout: "slow_ops_count": 315 2024-05-15T14:12:37.752 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.752 INFO:teuthology.orchestra.run.smithi081.stdout: "purge_queue": { 2024-05-15T14:12:37.753 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_ops": 0, 2024-05-15T14:12:37.753 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_ops_high_water": 10, 2024-05-15T14:12:37.753 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing": 0, 2024-05-15T14:12:37.753 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_high_water": 10, 2024-05-15T14:12:37.753 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executed_ops": 3689, 2024-05-15T14:12:37.753 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executed": 3689, 2024-05-15T14:12:37.753 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_item_in_journal": 0 2024-05-15T14:12:37.753 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.754 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-15T14:12:37.754 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-05-15T14:12:37.754 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2024-05-15T14:12:37.754 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-05-15T14:12:37.754 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 196272, 2024-05-15T14:12:37.754 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 43599997, 2024-05-15T14:12:37.754 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:37.755 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 196272, 2024-05-15T14:12:37.755 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-05-15T14:12:37.755 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-05-15T14:12:37.755 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 196272, 2024-05-15T14:12:37.755 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 43599997, 2024-05-15T14:12:37.755 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-05-15T14:12:37.755 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:37.755 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:37.756 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:37.756 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:37.756 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.756 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-objecter_bytes": { 2024-05-15T14:12:37.756 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-05-15T14:12:37.756 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2024-05-15T14:12:37.756 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-05-15T14:12:37.757 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-05-15T14:12:37.757 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-05-15T14:12:37.757 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:37.757 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-05-15T14:12:37.757 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 12025, 2024-05-15T14:12:37.757 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 55467348, 2024-05-15T14:12:37.757 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 12025, 2024-05-15T14:12:37.757 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 55467348, 2024-05-15T14:12:37.758 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-05-15T14:12:37.758 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:37.758 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:37.758 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:37.758 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:37.758 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.758 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-objecter_ops": { 2024-05-15T14:12:37.758 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-05-15T14:12:37.759 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 1024, 2024-05-15T14:12:37.759 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-05-15T14:12:37.759 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-05-15T14:12:37.759 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-05-15T14:12:37.759 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:37.759 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-05-15T14:12:37.759 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 12025, 2024-05-15T14:12:37.760 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 12025, 2024-05-15T14:12:37.760 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 12025, 2024-05-15T14:12:37.760 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 12025, 2024-05-15T14:12:37.760 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-05-15T14:12:37.760 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:37.760 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:37.760 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:37.761 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:37.761 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.761 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-write_buf_throttle": { 2024-05-15T14:12:37.761 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-05-15T14:12:37.761 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 3758096384, 2024-05-15T14:12:37.761 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-05-15T14:12:37.761 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 3689, 2024-05-15T14:12:37.761 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 373717, 2024-05-15T14:12:37.762 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:37.762 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 3689, 2024-05-15T14:12:37.762 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-05-15T14:12:37.762 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-05-15T14:12:37.762 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 266, 2024-05-15T14:12:37.762 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 373717, 2024-05-15T14:12:37.762 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-05-15T14:12:37.763 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:37.763 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:37.763 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:37.763 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:37.763 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:37.763 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-write_buf_throttle-0x55bd633a0460": { 2024-05-15T14:12:37.763 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-05-15T14:12:37.764 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 3758096384, 2024-05-15T14:12:37.764 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-05-15T14:12:37.764 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 32459, 2024-05-15T14:12:37.764 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 54155248, 2024-05-15T14:12:37.764 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:37.764 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 32459, 2024-05-15T14:12:37.764 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-05-15T14:12:37.765 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-05-15T14:12:37.765 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 7199, 2024-05-15T14:12:37.765 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 54155248, 2024-05-15T14:12:37.765 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-05-15T14:12:37.765 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:37.765 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:37.765 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:37.765 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:37.766 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:37.766 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-05-15T14:12:38.511 INFO:tasks.check_counter:Daemon mds.c mds.exported=9 2024-05-15T14:12:38.511 INFO:tasks.check_counter:Daemon mds.c mds.imported=7 2024-05-15T14:12:38.511 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=236 2024-05-15T14:12:38.512 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=475 2024-05-15T14:12:38.512 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-05-15T14:12:38.512 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-05-15T14:12:38.512 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-05-15T14:12:38.512 DEBUG:tasks.check_counter:Getting stats from f 2024-05-15T14:12:38.512 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-15T14:12:38.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:38 smithi033 ceph-mon[31650]: pgmap v386: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 880 KiB/s wr, 39 op/s 2024-05-15T14:12:38.759 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.c/config 2024-05-15T14:12:38.797 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:38 smithi081 ceph-mon[36434]: pgmap v386: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 880 KiB/s wr, 39 op/s 2024-05-15T14:12:38.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:38 smithi052 ceph-mon[36190]: pgmap v386: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 880 KiB/s wr, 39 op/s 2024-05-15T14:12:40.483 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-05-15T14:12:40.483 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-0": { 2024-05-15T14:12:40.483 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 73, 2024-05-15T14:12:40.483 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 73, 2024-05-15T14:12:40.484 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 2624207, 2024-05-15T14:12:40.484 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 22405, 2024-05-15T14:12:40.484 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 7, 2024-05-15T14:12:40.484 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 3, 2024-05-15T14:12:40.484 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.009379568, 2024-05-15T14:12:40.484 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.002424895, 2024-05-15T14:12:40.485 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.003935189, 2024-05-15T14:12:40.485 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.001665280, 2024-05-15T14:12:40.485 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:40.485 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 72, 2024-05-15T14:12:40.485 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.005363996, 2024-05-15T14:12:40.485 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000074499 2024-05-15T14:12:40.485 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.486 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:40.486 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.486 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.486 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.486 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.486 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-15T14:12:40.486 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 448 2024-05-15T14:12:40.487 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.487 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-1": { 2024-05-15T14:12:40.487 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 241, 2024-05-15T14:12:40.487 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 236, 2024-05-15T14:12:40.487 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 12802068, 2024-05-15T14:12:40.487 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 82993, 2024-05-15T14:12:40.487 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 4, 2024-05-15T14:12:40.487 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 3, 2024-05-15T14:12:40.488 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.029116119, 2024-05-15T14:12:40.488 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.007854723, 2024-05-15T14:12:40.488 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.013629300, 2024-05-15T14:12:40.488 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.006789931, 2024-05-15T14:12:40.488 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:40.488 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 235, 2024-05-15T14:12:40.488 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.010839242, 2024-05-15T14:12:40.489 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000046124 2024-05-15T14:12:40.489 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.489 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:40.489 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.489 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.489 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.489 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.489 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 196112, 2024-05-15T14:12:40.490 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 22512 2024-05-15T14:12:40.490 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.490 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-2": { 2024-05-15T14:12:40.490 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 180, 2024-05-15T14:12:40.490 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 206, 2024-05-15T14:12:40.490 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 21017707, 2024-05-15T14:12:40.490 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 188347, 2024-05-15T14:12:40.491 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 5, 2024-05-15T14:12:40.491 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 4, 2024-05-15T14:12:40.491 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.024283593, 2024-05-15T14:12:40.491 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.005467015, 2024-05-15T14:12:40.491 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.013285505, 2024-05-15T14:12:40.491 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.004105642, 2024-05-15T14:12:40.491 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:40.492 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 204, 2024-05-15T14:12:40.492 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.006926465, 2024-05-15T14:12:40.492 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000033953 2024-05-15T14:12:40.492 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.492 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:40.492 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.492 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.493 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.493 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.493 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-15T14:12:40.493 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 131424 2024-05-15T14:12:40.493 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.493 INFO:teuthology.orchestra.run.smithi081.stdout: "cct": { 2024-05-15T14:12:40.493 INFO:teuthology.orchestra.run.smithi081.stdout: "total_workers": 1, 2024-05-15T14:12:40.494 INFO:teuthology.orchestra.run.smithi081.stdout: "unhealthy_workers": 0 2024-05-15T14:12:40.494 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.494 INFO:teuthology.orchestra.run.smithi081.stdout: "finisher-MDSRank": { 2024-05-15T14:12:40.494 INFO:teuthology.orchestra.run.smithi081.stdout: "queue_len": 0, 2024-05-15T14:12:40.494 INFO:teuthology.orchestra.run.smithi081.stdout: "complete_latency": { 2024-05-15T14:12:40.494 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 722, 2024-05-15T14:12:40.494 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.024834779, 2024-05-15T14:12:40.494 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000034397 2024-05-15T14:12:40.495 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:40.495 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.495 INFO:teuthology.orchestra.run.smithi081.stdout: "finisher-PurgeQueue": { 2024-05-15T14:12:40.495 INFO:teuthology.orchestra.run.smithi081.stdout: "queue_len": 0, 2024-05-15T14:12:40.495 INFO:teuthology.orchestra.run.smithi081.stdout: "complete_latency": { 2024-05-15T14:12:40.495 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.495 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.496 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.496 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:40.496 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.496 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2024-05-15T14:12:40.496 INFO:teuthology.orchestra.run.smithi081.stdout: "request": 0, 2024-05-15T14:12:40.496 INFO:teuthology.orchestra.run.smithi081.stdout: "reply": 0, 2024-05-15T14:12:40.496 INFO:teuthology.orchestra.run.smithi081.stdout: "reply_latency": { 2024-05-15T14:12:40.497 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.497 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.497 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.497 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.497 INFO:teuthology.orchestra.run.smithi081.stdout: "slow_reply": 0, 2024-05-15T14:12:40.497 INFO:teuthology.orchestra.run.smithi081.stdout: "forward": 0, 2024-05-15T14:12:40.497 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_fetch_complete": 0, 2024-05-15T14:12:40.498 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_fetch_keys": 0, 2024-05-15T14:12:40.498 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_commit": 0, 2024-05-15T14:12:40.498 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_split": 0, 2024-05-15T14:12:40.498 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_merge": 0, 2024-05-15T14:12:40.498 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes": 572, 2024-05-15T14:12:40.498 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_top": 315, 2024-05-15T14:12:40.498 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_bottom": 257, 2024-05-15T14:12:40.499 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_pin_tail": 0, 2024-05-15T14:12:40.499 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_pinned": 122, 2024-05-15T14:12:40.499 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_expired": 0, 2024-05-15T14:12:40.499 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_with_caps": 0, 2024-05-15T14:12:40.499 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": 0, 2024-05-15T14:12:40.499 INFO:teuthology.orchestra.run.smithi081.stdout: "subtrees": 17, 2024-05-15T14:12:40.499 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse": 0, 2024-05-15T14:12:40.499 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_hit": 0, 2024-05-15T14:12:40.500 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_forward": 0, 2024-05-15T14:12:40.500 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_discover": 0, 2024-05-15T14:12:40.500 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_dir_fetch": 0, 2024-05-15T14:12:40.500 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_remote_ino": 0, 2024-05-15T14:12:40.500 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_lock": 0, 2024-05-15T14:12:40.500 INFO:teuthology.orchestra.run.smithi081.stdout: "load_cent": 0, 2024-05-15T14:12:40.500 INFO:teuthology.orchestra.run.smithi081.stdout: "q": 0, 2024-05-15T14:12:40.501 INFO:teuthology.orchestra.run.smithi081.stdout: "exported": 0, 2024-05-15T14:12:40.501 INFO:teuthology.orchestra.run.smithi081.stdout: "exported_inodes": 0, 2024-05-15T14:12:40.501 INFO:teuthology.orchestra.run.smithi081.stdout: "imported": 0, 2024-05-15T14:12:40.501 INFO:teuthology.orchestra.run.smithi081.stdout: "imported_inodes": 0, 2024-05-15T14:12:40.501 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_dir_fetch": 0, 2024-05-15T14:12:40.501 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_backtrace_fetch": 0, 2024-05-15T14:12:40.501 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_peer_discover": 0, 2024-05-15T14:12:40.502 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rfiles": 67, 2024-05-15T14:12:40.502 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rbytes": 21176629, 2024-05-15T14:12:40.502 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rsnaps": 6, 2024-05-15T14:12:40.502 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_backtrace_fetch": 0, 2024-05-15T14:12:40.502 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_set_tag": 0, 2024-05-15T14:12:40.502 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_backtrace_repaired": 0, 2024-05-15T14:12:40.502 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_inotable_repaired": 0, 2024-05-15T14:12:40.502 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dir_inodes": 0, 2024-05-15T14:12:40.503 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dir_base_inodes": 0, 2024-05-15T14:12:40.503 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dirfrag_rstats": 0, 2024-05-15T14:12:40.503 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_file_inodes": 0, 2024-05-15T14:12:40.503 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_inode_file_caps": 0, 2024-05-15T14:12:40.503 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_revoke": 0, 2024-05-15T14:12:40.503 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_grant": 0, 2024-05-15T14:12:40.504 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_trunc": 0, 2024-05-15T14:12:40.504 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-15T14:12:40.504 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-15T14:12:40.504 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_caps": 0, 2024-05-15T14:12:40.504 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_caps_dirty": 0, 2024-05-15T14:12:40.504 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_cap_release": 0, 2024-05-15T14:12:40.504 INFO:teuthology.orchestra.run.smithi081.stdout: "process_request_cap_release": 0 2024-05-15T14:12:40.505 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.505 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_cache": { 2024-05-15T14:12:40.505 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_update": 0, 2024-05-15T14:12:40.505 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_update_receipt": 0, 2024-05-15T14:12:40.505 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_try_discover": 0, 2024-05-15T14:12:40.505 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_send_discover": 0, 2024-05-15T14:12:40.505 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_handle_discover": 0, 2024-05-15T14:12:40.505 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays": 0, 2024-05-15T14:12:40.506 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays_delayed": 0, 2024-05-15T14:12:40.506 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays_enqueuing": 0, 2024-05-15T14:12:40.506 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_created": 0, 2024-05-15T14:12:40.506 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_enqueued": 0, 2024-05-15T14:12:40.506 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_reintegrated": 0, 2024-05-15T14:12:40.506 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_migrated": 0, 2024-05-15T14:12:40.506 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_processing": 0, 2024-05-15T14:12:40.507 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_enqueued": 0, 2024-05-15T14:12:40.507 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_prioritized": 0, 2024-05-15T14:12:40.507 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_started": 0, 2024-05-15T14:12:40.507 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_completed": 0, 2024-05-15T14:12:40.507 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_quiesce_path": 0, 2024-05-15T14:12:40.507 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_quiesce_inode": 0, 2024-05-15T14:12:40.507 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_enqueue_scrub": 0, 2024-05-15T14:12:40.508 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_exportdir": 0, 2024-05-15T14:12:40.508 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_flush": 0, 2024-05-15T14:12:40.508 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_fragmentdir": 0, 2024-05-15T14:12:40.508 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_fragstats": 0, 2024-05-15T14:12:40.508 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_inodestats": 0 2024-05-15T14:12:40.508 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.508 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_log": { 2024-05-15T14:12:40.508 INFO:teuthology.orchestra.run.smithi081.stdout: "evlrg": 0, 2024-05-15T14:12:40.509 INFO:teuthology.orchestra.run.smithi081.stdout: "evadd": 0, 2024-05-15T14:12:40.509 INFO:teuthology.orchestra.run.smithi081.stdout: "evex": 0, 2024-05-15T14:12:40.509 INFO:teuthology.orchestra.run.smithi081.stdout: "evtrm": 0, 2024-05-15T14:12:40.509 INFO:teuthology.orchestra.run.smithi081.stdout: "ev": 5251, 2024-05-15T14:12:40.509 INFO:teuthology.orchestra.run.smithi081.stdout: "evexg": 0, 2024-05-15T14:12:40.509 INFO:teuthology.orchestra.run.smithi081.stdout: "evexd": 0, 2024-05-15T14:12:40.509 INFO:teuthology.orchestra.run.smithi081.stdout: "segadd": 0, 2024-05-15T14:12:40.510 INFO:teuthology.orchestra.run.smithi081.stdout: "segex": 0, 2024-05-15T14:12:40.510 INFO:teuthology.orchestra.run.smithi081.stdout: "segtrm": 0, 2024-05-15T14:12:40.510 INFO:teuthology.orchestra.run.smithi081.stdout: "seg": 2, 2024-05-15T14:12:40.510 INFO:teuthology.orchestra.run.smithi081.stdout: "segmjr": 7, 2024-05-15T14:12:40.510 INFO:teuthology.orchestra.run.smithi081.stdout: "segexg": 0, 2024-05-15T14:12:40.510 INFO:teuthology.orchestra.run.smithi081.stdout: "segexd": 0, 2024-05-15T14:12:40.510 INFO:teuthology.orchestra.run.smithi081.stdout: "expos": 278373050, 2024-05-15T14:12:40.510 INFO:teuthology.orchestra.run.smithi081.stdout: "wrpos": 279427022, 2024-05-15T14:12:40.511 INFO:teuthology.orchestra.run.smithi081.stdout: "rdpos": 279418226, 2024-05-15T14:12:40.511 INFO:teuthology.orchestra.run.smithi081.stdout: "jlat": { 2024-05-15T14:12:40.511 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.511 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.511 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.511 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.511 INFO:teuthology.orchestra.run.smithi081.stdout: "replayed": 5251 2024-05-15T14:12:40.512 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.512 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_mem": { 2024-05-15T14:12:40.512 INFO:teuthology.orchestra.run.smithi081.stdout: "ino": 84, 2024-05-15T14:12:40.512 INFO:teuthology.orchestra.run.smithi081.stdout: "ino+": 556, 2024-05-15T14:12:40.512 INFO:teuthology.orchestra.run.smithi081.stdout: "ino-": 472, 2024-05-15T14:12:40.512 INFO:teuthology.orchestra.run.smithi081.stdout: "dir": 44, 2024-05-15T14:12:40.512 INFO:teuthology.orchestra.run.smithi081.stdout: "dir+": 45, 2024-05-15T14:12:40.512 INFO:teuthology.orchestra.run.smithi081.stdout: "dir-": 1, 2024-05-15T14:12:40.513 INFO:teuthology.orchestra.run.smithi081.stdout: "dn": 572, 2024-05-15T14:12:40.513 INFO:teuthology.orchestra.run.smithi081.stdout: "dn+": 583, 2024-05-15T14:12:40.513 INFO:teuthology.orchestra.run.smithi081.stdout: "dn-": 11, 2024-05-15T14:12:40.513 INFO:teuthology.orchestra.run.smithi081.stdout: "cap": 0, 2024-05-15T14:12:40.513 INFO:teuthology.orchestra.run.smithi081.stdout: "cap+": 0, 2024-05-15T14:12:40.513 INFO:teuthology.orchestra.run.smithi081.stdout: "cap-": 0, 2024-05-15T14:12:40.513 INFO:teuthology.orchestra.run.smithi081.stdout: "rss": 51524, 2024-05-15T14:12:40.513 INFO:teuthology.orchestra.run.smithi081.stdout: "heap": 215392 2024-05-15T14:12:40.514 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.514 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_server": { 2024-05-15T14:12:40.514 INFO:teuthology.orchestra.run.smithi081.stdout: "dispatch_client_request": 0, 2024-05-15T14:12:40.514 INFO:teuthology.orchestra.run.smithi081.stdout: "dispatch_server_request": 0, 2024-05-15T14:12:40.514 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_request": 0, 2024-05-15T14:12:40.514 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_session": 0, 2024-05-15T14:12:40.514 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_peer_request": 0, 2024-05-15T14:12:40.515 INFO:teuthology.orchestra.run.smithi081.stdout: "req_create_latency": { 2024-05-15T14:12:40.515 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.515 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.515 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.515 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.515 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getattr_latency": { 2024-05-15T14:12:40.515 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.515 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.516 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.516 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.516 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getfilelock_latency": { 2024-05-15T14:12:40.516 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.516 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.516 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.516 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.517 INFO:teuthology.orchestra.run.smithi081.stdout: "req_link_latency": { 2024-05-15T14:12:40.517 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.517 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.517 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.517 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.517 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookup_latency": { 2024-05-15T14:12:40.517 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.517 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.518 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.518 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.518 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookuphash_latency": { 2024-05-15T14:12:40.518 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.518 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.518 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.518 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.519 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupino_latency": { 2024-05-15T14:12:40.519 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.519 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.519 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.519 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.519 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupname_latency": { 2024-05-15T14:12:40.519 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.519 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.520 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.520 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.520 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupparent_latency": { 2024-05-15T14:12:40.520 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.520 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.520 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.520 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.520 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupsnap_latency": { 2024-05-15T14:12:40.521 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.521 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.521 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.521 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.521 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lssnap_latency": { 2024-05-15T14:12:40.521 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.521 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.522 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.522 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.522 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mkdir_latency": { 2024-05-15T14:12:40.523 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.523 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.523 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.523 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.523 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mknod_latency": { 2024-05-15T14:12:40.523 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.524 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.524 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.524 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.524 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mksnap_latency": { 2024-05-15T14:12:40.524 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.524 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.524 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.524 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.525 INFO:teuthology.orchestra.run.smithi081.stdout: "req_open_latency": { 2024-05-15T14:12:40.525 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.525 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.525 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.525 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.525 INFO:teuthology.orchestra.run.smithi081.stdout: "req_readdir_latency": { 2024-05-15T14:12:40.525 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.525 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.526 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.526 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.526 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rename_latency": { 2024-05-15T14:12:40.526 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.526 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.526 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.527 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.527 INFO:teuthology.orchestra.run.smithi081.stdout: "req_renamesnap_latency": { 2024-05-15T14:12:40.527 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.527 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.527 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.527 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.527 INFO:teuthology.orchestra.run.smithi081.stdout: "req_snapdiff_latency": { 2024-05-15T14:12:40.527 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.528 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.528 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.528 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.528 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmdir_latency": { 2024-05-15T14:12:40.528 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.528 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.528 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.528 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.529 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmsnap_latency": { 2024-05-15T14:12:40.529 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.529 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.529 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.529 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.529 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmxattr_latency": { 2024-05-15T14:12:40.529 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.529 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.530 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.530 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.530 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setattr_latency": { 2024-05-15T14:12:40.530 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.530 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.530 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.530 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.531 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setdirlayout_latency": { 2024-05-15T14:12:40.531 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.531 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.531 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.531 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.531 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setfilelock_latency": { 2024-05-15T14:12:40.531 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.531 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.532 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.532 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.532 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setlayout_latency": { 2024-05-15T14:12:40.532 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.532 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.532 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.532 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.533 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setxattr_latency": { 2024-05-15T14:12:40.533 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.533 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.533 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.533 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.533 INFO:teuthology.orchestra.run.smithi081.stdout: "req_symlink_latency": { 2024-05-15T14:12:40.533 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.533 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.534 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.534 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.534 INFO:teuthology.orchestra.run.smithi081.stdout: "req_unlink_latency": { 2024-05-15T14:12:40.534 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.534 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.534 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.534 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.535 INFO:teuthology.orchestra.run.smithi081.stdout: "cap_revoke_eviction": 0, 2024-05-15T14:12:40.535 INFO:teuthology.orchestra.run.smithi081.stdout: "cap_acquisition_throttle": 0, 2024-05-15T14:12:40.535 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getvxattr_latency": { 2024-05-15T14:12:40.535 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.535 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.535 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.535 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:40.535 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.536 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_sessions": { 2024-05-15T14:12:40.536 INFO:teuthology.orchestra.run.smithi081.stdout: "session_count": 2, 2024-05-15T14:12:40.536 INFO:teuthology.orchestra.run.smithi081.stdout: "session_add": 2, 2024-05-15T14:12:40.536 INFO:teuthology.orchestra.run.smithi081.stdout: "session_remove": 0, 2024-05-15T14:12:40.536 INFO:teuthology.orchestra.run.smithi081.stdout: "sessions_open": 0, 2024-05-15T14:12:40.536 INFO:teuthology.orchestra.run.smithi081.stdout: "sessions_stale": 0, 2024-05-15T14:12:40.536 INFO:teuthology.orchestra.run.smithi081.stdout: "total_load": 0, 2024-05-15T14:12:40.536 INFO:teuthology.orchestra.run.smithi081.stdout: "average_load": 0, 2024-05-15T14:12:40.537 INFO:teuthology.orchestra.run.smithi081.stdout: "avg_session_uptime": 2129, 2024-05-15T14:12:40.537 INFO:teuthology.orchestra.run.smithi081.stdout: "mdthresh_evicted": 0 2024-05-15T14:12:40.537 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.537 INFO:teuthology.orchestra.run.smithi081.stdout: "mempool": { 2024-05-15T14:12:40.537 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_bytes": 0, 2024-05-15T14:12:40.537 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_items": 0, 2024-05-15T14:12:40.537 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_bytes": 0, 2024-05-15T14:12:40.538 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_items": 0, 2024-05-15T14:12:40.538 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_bytes": 0, 2024-05-15T14:12:40.538 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_items": 0, 2024-05-15T14:12:40.538 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-15T14:12:40.538 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_items": 0, 2024-05-15T14:12:40.538 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-15T14:12:40.538 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_items": 0, 2024-05-15T14:12:40.539 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_bytes": 0, 2024-05-15T14:12:40.539 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_items": 0, 2024-05-15T14:12:40.539 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-15T14:12:40.539 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_items": 0, 2024-05-15T14:12:40.539 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_bytes": 0, 2024-05-15T14:12:40.539 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_items": 0, 2024-05-15T14:12:40.539 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_bytes": 0, 2024-05-15T14:12:40.540 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_items": 0, 2024-05-15T14:12:40.540 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-15T14:12:40.540 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_items": 0, 2024-05-15T14:12:40.540 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-15T14:12:40.540 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_items": 0, 2024-05-15T14:12:40.540 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_bytes": 0, 2024-05-15T14:12:40.540 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_items": 0, 2024-05-15T14:12:40.541 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_bytes": 0, 2024-05-15T14:12:40.541 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_items": 0, 2024-05-15T14:12:40.541 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-15T14:12:40.541 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_items": 0, 2024-05-15T14:12:40.541 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_bytes": 0, 2024-05-15T14:12:40.541 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_items": 0, 2024-05-15T14:12:40.541 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_bytes": 0, 2024-05-15T14:12:40.542 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_items": 0, 2024-05-15T14:12:40.542 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_bytes": 0, 2024-05-15T14:12:40.542 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_items": 0, 2024-05-15T14:12:40.542 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_bytes": 0, 2024-05-15T14:12:40.542 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_items": 0, 2024-05-15T14:12:40.542 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_bytes": 210186, 2024-05-15T14:12:40.542 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_items": 204, 2024-05-15T14:12:40.543 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_bytes": 0, 2024-05-15T14:12:40.543 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_items": 0, 2024-05-15T14:12:40.543 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_bytes": 0, 2024-05-15T14:12:40.543 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_items": 0, 2024-05-15T14:12:40.543 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_bytes": 0, 2024-05-15T14:12:40.543 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_items": 0, 2024-05-15T14:12:40.543 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_bytes": 0, 2024-05-15T14:12:40.543 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_items": 0, 2024-05-15T14:12:40.544 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_bytes": 8128, 2024-05-15T14:12:40.544 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_items": 169, 2024-05-15T14:12:40.544 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_bytes": 0, 2024-05-15T14:12:40.544 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_items": 0, 2024-05-15T14:12:40.544 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_bytes": 0, 2024-05-15T14:12:40.544 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_items": 0, 2024-05-15T14:12:40.544 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_bytes": 753404, 2024-05-15T14:12:40.545 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_items": 5786, 2024-05-15T14:12:40.545 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_bytes": 0, 2024-05-15T14:12:40.545 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_items": 0, 2024-05-15T14:12:40.545 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_bytes": 0, 2024-05-15T14:12:40.545 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_items": 0 2024-05-15T14:12:40.545 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.545 INFO:teuthology.orchestra.run.smithi081.stdout: "objecter": { 2024-05-15T14:12:40.545 INFO:teuthology.orchestra.run.smithi081.stdout: "op_active": 0, 2024-05-15T14:12:40.546 INFO:teuthology.orchestra.run.smithi081.stdout: "op_laggy": 0, 2024-05-15T14:12:40.546 INFO:teuthology.orchestra.run.smithi081.stdout: "op_send": 438, 2024-05-15T14:12:40.546 INFO:teuthology.orchestra.run.smithi081.stdout: "op_send_bytes": 26, 2024-05-15T14:12:40.546 INFO:teuthology.orchestra.run.smithi081.stdout: "op_resend": 4, 2024-05-15T14:12:40.546 INFO:teuthology.orchestra.run.smithi081.stdout: "op_reply": 438, 2024-05-15T14:12:40.546 INFO:teuthology.orchestra.run.smithi081.stdout: "op_latency": { 2024-05-15T14:12:40.546 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 438, 2024-05-15T14:12:40.547 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.312994476, 2024-05-15T14:12:40.547 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000714599 2024-05-15T14:12:40.547 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.547 INFO:teuthology.orchestra.run.smithi081.stdout: "op_inflight": 0, 2024-05-15T14:12:40.547 INFO:teuthology.orchestra.run.smithi081.stdout: "oplen_avg": { 2024-05-15T14:12:40.547 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 438, 2024-05-15T14:12:40.547 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 439 2024-05-15T14:12:40.548 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.548 INFO:teuthology.orchestra.run.smithi081.stdout: "op": 438, 2024-05-15T14:12:40.548 INFO:teuthology.orchestra.run.smithi081.stdout: "op_r": 438, 2024-05-15T14:12:40.548 INFO:teuthology.orchestra.run.smithi081.stdout: "op_w": 0, 2024-05-15T14:12:40.548 INFO:teuthology.orchestra.run.smithi081.stdout: "op_rmw": 0, 2024-05-15T14:12:40.548 INFO:teuthology.orchestra.run.smithi081.stdout: "op_pg": 0, 2024-05-15T14:12:40.548 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_stat": 241, 2024-05-15T14:12:40.549 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_create": 0, 2024-05-15T14:12:40.549 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_read": 194, 2024-05-15T14:12:40.549 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_write": 0, 2024-05-15T14:12:40.549 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_writefull": 0, 2024-05-15T14:12:40.549 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_writesame": 0, 2024-05-15T14:12:40.549 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_append": 0, 2024-05-15T14:12:40.549 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_zero": 0, 2024-05-15T14:12:40.550 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_truncate": 0, 2024-05-15T14:12:40.550 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_delete": 0, 2024-05-15T14:12:40.550 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_mapext": 0, 2024-05-15T14:12:40.550 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_sparse_read": 0, 2024-05-15T14:12:40.550 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_clonerange": 0, 2024-05-15T14:12:40.550 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_getxattr": 2, 2024-05-15T14:12:40.550 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_setxattr": 0, 2024-05-15T14:12:40.551 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_cmpxattr": 0, 2024-05-15T14:12:40.551 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_rmxattr": 0, 2024-05-15T14:12:40.551 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_resetxattrs": 0, 2024-05-15T14:12:40.551 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_call": 0, 2024-05-15T14:12:40.551 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_watch": 0, 2024-05-15T14:12:40.551 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_notify": 0, 2024-05-15T14:12:40.551 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_src_cmpxattr": 0, 2024-05-15T14:12:40.551 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_pgls": 0, 2024-05-15T14:12:40.552 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_pgls_filter": 0, 2024-05-15T14:12:40.552 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_other": 0, 2024-05-15T14:12:40.552 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_active": 0, 2024-05-15T14:12:40.552 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_send": 0, 2024-05-15T14:12:40.552 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_resend": 0, 2024-05-15T14:12:40.552 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_ping": 0, 2024-05-15T14:12:40.552 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_active": 0, 2024-05-15T14:12:40.553 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_send": 0, 2024-05-15T14:12:40.553 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_resend": 0, 2024-05-15T14:12:40.553 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_active": 0, 2024-05-15T14:12:40.553 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_send": 0, 2024-05-15T14:12:40.553 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_resend": 0, 2024-05-15T14:12:40.553 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_active": 0, 2024-05-15T14:12:40.553 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_send": 0, 2024-05-15T14:12:40.553 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_resend": 0, 2024-05-15T14:12:40.554 INFO:teuthology.orchestra.run.smithi081.stdout: "command_active": 0, 2024-05-15T14:12:40.554 INFO:teuthology.orchestra.run.smithi081.stdout: "command_send": 0, 2024-05-15T14:12:40.554 INFO:teuthology.orchestra.run.smithi081.stdout: "command_resend": 0, 2024-05-15T14:12:40.554 INFO:teuthology.orchestra.run.smithi081.stdout: "map_epoch": 98, 2024-05-15T14:12:40.554 INFO:teuthology.orchestra.run.smithi081.stdout: "map_full": 0, 2024-05-15T14:12:40.554 INFO:teuthology.orchestra.run.smithi081.stdout: "map_inc": 4, 2024-05-15T14:12:40.554 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_sessions": 7, 2024-05-15T14:12:40.555 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_session_open": 7, 2024-05-15T14:12:40.555 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_session_close": 0, 2024-05-15T14:12:40.555 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_laggy": 0, 2024-05-15T14:12:40.555 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_wr": 0, 2024-05-15T14:12:40.555 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_rd": 2, 2024-05-15T14:12:40.555 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_del": 0 2024-05-15T14:12:40.555 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.556 INFO:teuthology.orchestra.run.smithi081.stdout: "oft": { 2024-05-15T14:12:40.556 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_objs": 0, 2024-05-15T14:12:40.556 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_kv_pairs": 0, 2024-05-15T14:12:40.556 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_updates": 0, 2024-05-15T14:12:40.556 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_removes": 0 2024-05-15T14:12:40.556 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.556 INFO:teuthology.orchestra.run.smithi081.stdout: "osd-slow-ops": { 2024-05-15T14:12:40.557 INFO:teuthology.orchestra.run.smithi081.stdout: "slow_ops_count": 0 2024-05-15T14:12:40.557 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.557 INFO:teuthology.orchestra.run.smithi081.stdout: "purge_queue": { 2024-05-15T14:12:40.557 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_ops": 0, 2024-05-15T14:12:40.557 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_ops_high_water": 0, 2024-05-15T14:12:40.557 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing": 0, 2024-05-15T14:12:40.557 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_high_water": 0, 2024-05-15T14:12:40.558 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executed_ops": 0, 2024-05-15T14:12:40.558 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executed": 0, 2024-05-15T14:12:40.558 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_item_in_journal": 0 2024-05-15T14:12:40.558 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.558 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-15T14:12:40.558 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-05-15T14:12:40.558 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2024-05-15T14:12:40.558 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-05-15T14:12:40.559 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 490, 2024-05-15T14:12:40.559 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 36394321, 2024-05-15T14:12:40.559 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:40.559 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 490, 2024-05-15T14:12:40.559 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-05-15T14:12:40.559 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-05-15T14:12:40.560 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 490, 2024-05-15T14:12:40.560 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 36394321, 2024-05-15T14:12:40.560 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-05-15T14:12:40.560 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.560 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.560 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.560 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:40.560 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.561 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-objecter_bytes": { 2024-05-15T14:12:40.561 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-05-15T14:12:40.561 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2024-05-15T14:12:40.561 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-05-15T14:12:40.561 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-05-15T14:12:40.561 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-05-15T14:12:40.562 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:40.562 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-05-15T14:12:40.562 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 438, 2024-05-15T14:12:40.562 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 36117135, 2024-05-15T14:12:40.562 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 438, 2024-05-15T14:12:40.562 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 36117135, 2024-05-15T14:12:40.562 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-05-15T14:12:40.562 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.563 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.563 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.563 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:40.563 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.563 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-objecter_ops": { 2024-05-15T14:12:40.563 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-05-15T14:12:40.563 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 1024, 2024-05-15T14:12:40.564 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-05-15T14:12:40.564 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-05-15T14:12:40.564 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-05-15T14:12:40.564 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:40.564 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-05-15T14:12:40.564 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 438, 2024-05-15T14:12:40.564 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 438, 2024-05-15T14:12:40.565 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 438, 2024-05-15T14:12:40.565 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 438, 2024-05-15T14:12:40.565 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-05-15T14:12:40.565 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.565 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.565 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.565 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:40.566 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.566 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-write_buf_throttle": { 2024-05-15T14:12:40.566 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-05-15T14:12:40.566 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 3758096384, 2024-05-15T14:12:40.566 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-05-15T14:12:40.566 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-05-15T14:12:40.566 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-05-15T14:12:40.567 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:40.567 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-05-15T14:12:40.567 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-05-15T14:12:40.567 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-05-15T14:12:40.567 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 0, 2024-05-15T14:12:40.567 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 0, 2024-05-15T14:12:40.567 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-05-15T14:12:40.567 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.568 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.568 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.568 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:40.568 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:40.568 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-write_buf_throttle-0x55a3ae0986e0": { 2024-05-15T14:12:40.568 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-05-15T14:12:40.569 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 3758096384, 2024-05-15T14:12:40.569 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-05-15T14:12:40.569 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-05-15T14:12:40.569 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-05-15T14:12:40.569 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:40.569 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-05-15T14:12:40.569 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-05-15T14:12:40.569 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-05-15T14:12:40.570 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 0, 2024-05-15T14:12:40.570 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 0, 2024-05-15T14:12:40.570 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-05-15T14:12:40.570 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:40.570 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:40.570 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:40.570 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:40.571 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:40.571 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-05-15T14:12:40.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:40 smithi081 ceph-mon[36434]: pgmap v387: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 KiB/s wr, 9 op/s 2024-05-15T14:12:40.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:40 smithi033 ceph-mon[31650]: pgmap v387: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 KiB/s wr, 9 op/s 2024-05-15T14:12:40.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:40 smithi052 ceph-mon[36190]: pgmap v387: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 KiB/s wr, 9 op/s 2024-05-15T14:12:41.289 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-15T14:12:41.289 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-15T14:12:41.289 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-05-15T14:12:41.289 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-05-15T14:12:41.289 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-05-15T14:12:41.289 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-05-15T14:12:41.289 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-05-15T14:12:41.289 DEBUG:tasks.check_counter:Getting stats from i 2024-05-15T14:12:41.290 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-15T14:12:41.529 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.c/config 2024-05-15T14:12:42.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:42 smithi033 ceph-mon[31650]: pgmap v388: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 KiB/s wr, 9 op/s 2024-05-15T14:12:42.701 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:42 smithi081 ceph-mon[36434]: pgmap v388: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 KiB/s wr, 9 op/s 2024-05-15T14:12:42.842 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:42 smithi052 ceph-mon[36190]: pgmap v388: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 KiB/s wr, 9 op/s 2024-05-15T14:12:43.256 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-05-15T14:12:43.256 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-0": { 2024-05-15T14:12:43.256 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 118764, 2024-05-15T14:12:43.257 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 146130, 2024-05-15T14:12:43.257 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 27322080, 2024-05-15T14:12:43.257 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 199715498, 2024-05-15T14:12:43.257 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 153, 2024-05-15T14:12:43.257 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 6, 2024-05-15T14:12:43.257 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 7.049723436, 2024-05-15T14:12:43.257 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 2.920292884, 2024-05-15T14:12:43.257 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 4.026971000, 2024-05-15T14:12:43.257 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.597001419, 2024-05-15T14:12:43.258 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:43.258 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 146129, 2024-05-15T14:12:43.258 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 5.679999870, 2024-05-15T14:12:43.258 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000038869 2024-05-15T14:12:43.258 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.258 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:43.258 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 213639, 2024-05-15T14:12:43.258 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.032590737, 2024-05-15T14:12:43.259 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000152 2024-05-15T14:12:43.259 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.259 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 46928, 2024-05-15T14:12:43.259 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 4144 2024-05-15T14:12:43.259 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.259 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-1": { 2024-05-15T14:12:43.259 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 16302, 2024-05-15T14:12:43.259 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 16269, 2024-05-15T14:12:43.260 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 7050736, 2024-05-15T14:12:43.260 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 84236081, 2024-05-15T14:12:43.260 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 17, 2024-05-15T14:12:43.260 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 5, 2024-05-15T14:12:43.260 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 1.250889182, 2024-05-15T14:12:43.260 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.511814370, 2024-05-15T14:12:43.260 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.287622504, 2024-05-15T14:12:43.260 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.380601245, 2024-05-15T14:12:43.260 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:43.261 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 16267, 2024-05-15T14:12:43.261 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 5.539911643, 2024-05-15T14:12:43.261 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000340561 2024-05-15T14:12:43.261 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.261 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:43.261 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 300, 2024-05-15T14:12:43.261 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000065157, 2024-05-15T14:12:43.261 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000217 2024-05-15T14:12:43.261 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.262 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 1948304, 2024-05-15T14:12:43.262 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 560640 2024-05-15T14:12:43.262 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.262 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-2": { 2024-05-15T14:12:43.262 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 226900, 2024-05-15T14:12:43.262 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 306379, 2024-05-15T14:12:43.262 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 64024367, 2024-05-15T14:12:43.262 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 220204340, 2024-05-15T14:12:43.263 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 16, 2024-05-15T14:12:43.263 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 7, 2024-05-15T14:12:43.263 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 12.219232126, 2024-05-15T14:12:43.263 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 5.588461168, 2024-05-15T14:12:43.263 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 7.457439414, 2024-05-15T14:12:43.263 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.503222027, 2024-05-15T14:12:43.263 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:43.263 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 306377, 2024-05-15T14:12:43.264 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 8.376346386, 2024-05-15T14:12:43.264 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000027339 2024-05-15T14:12:43.264 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.264 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:43.264 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 481539, 2024-05-15T14:12:43.264 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.064468025, 2024-05-15T14:12:43.264 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000133 2024-05-15T14:12:43.264 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.265 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 375120, 2024-05-15T14:12:43.265 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 265488 2024-05-15T14:12:43.265 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.265 INFO:teuthology.orchestra.run.smithi081.stdout: "cct": { 2024-05-15T14:12:43.265 INFO:teuthology.orchestra.run.smithi081.stdout: "total_workers": 1, 2024-05-15T14:12:43.265 INFO:teuthology.orchestra.run.smithi081.stdout: "unhealthy_workers": 0 2024-05-15T14:12:43.265 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.265 INFO:teuthology.orchestra.run.smithi081.stdout: "finisher-MDSRank": { 2024-05-15T14:12:43.266 INFO:teuthology.orchestra.run.smithi081.stdout: "queue_len": 0, 2024-05-15T14:12:43.266 INFO:teuthology.orchestra.run.smithi081.stdout: "complete_latency": { 2024-05-15T14:12:43.266 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 67335, 2024-05-15T14:12:43.266 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 40.499218809, 2024-05-15T14:12:43.266 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000601458 2024-05-15T14:12:43.266 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:43.266 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.266 INFO:teuthology.orchestra.run.smithi081.stdout: "finisher-PurgeQueue": { 2024-05-15T14:12:43.266 INFO:teuthology.orchestra.run.smithi081.stdout: "queue_len": 0, 2024-05-15T14:12:43.267 INFO:teuthology.orchestra.run.smithi081.stdout: "complete_latency": { 2024-05-15T14:12:43.267 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 16065, 2024-05-15T14:12:43.267 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 1.221188308, 2024-05-15T14:12:43.267 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000076015 2024-05-15T14:12:43.267 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:43.267 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.267 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2024-05-15T14:12:43.267 INFO:teuthology.orchestra.run.smithi081.stdout: "request": 98930, 2024-05-15T14:12:43.267 INFO:teuthology.orchestra.run.smithi081.stdout: "reply": 90547, 2024-05-15T14:12:43.268 INFO:teuthology.orchestra.run.smithi081.stdout: "reply_latency": { 2024-05-15T14:12:43.268 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 90547, 2024-05-15T14:12:43.268 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 329.854683319, 2024-05-15T14:12:43.268 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.003642911 2024-05-15T14:12:43.268 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.268 INFO:teuthology.orchestra.run.smithi081.stdout: "slow_reply": 0, 2024-05-15T14:12:43.268 INFO:teuthology.orchestra.run.smithi081.stdout: "forward": 8383, 2024-05-15T14:12:43.268 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_fetch_complete": 7, 2024-05-15T14:12:43.269 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_fetch_keys": 0, 2024-05-15T14:12:43.269 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_commit": 1108, 2024-05-15T14:12:43.269 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_split": 3, 2024-05-15T14:12:43.269 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_merge": 3, 2024-05-15T14:12:43.269 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes": 328, 2024-05-15T14:12:43.269 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_top": 138, 2024-05-15T14:12:43.269 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_bottom": 174, 2024-05-15T14:12:43.269 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_pin_tail": 16, 2024-05-15T14:12:43.269 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_pinned": 130, 2024-05-15T14:12:43.269 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_expired": 572, 2024-05-15T14:12:43.270 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_with_caps": 29, 2024-05-15T14:12:43.270 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": 29, 2024-05-15T14:12:43.270 INFO:teuthology.orchestra.run.smithi081.stdout: "subtrees": 18, 2024-05-15T14:12:43.270 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse": 157574, 2024-05-15T14:12:43.270 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_hit": 106868, 2024-05-15T14:12:43.270 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_forward": 8368, 2024-05-15T14:12:43.270 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_discover": 23, 2024-05-15T14:12:43.270 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_dir_fetch": 5, 2024-05-15T14:12:43.270 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_remote_ino": 0, 2024-05-15T14:12:43.271 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_lock": 795, 2024-05-15T14:12:43.271 INFO:teuthology.orchestra.run.smithi081.stdout: "load_cent": 13715, 2024-05-15T14:12:43.271 INFO:teuthology.orchestra.run.smithi081.stdout: "q": 12, 2024-05-15T14:12:43.271 INFO:teuthology.orchestra.run.smithi081.stdout: "exported": 10, 2024-05-15T14:12:43.271 INFO:teuthology.orchestra.run.smithi081.stdout: "exported_inodes": 345, 2024-05-15T14:12:43.271 INFO:teuthology.orchestra.run.smithi081.stdout: "imported": 10, 2024-05-15T14:12:43.271 INFO:teuthology.orchestra.run.smithi081.stdout: "imported_inodes": 180, 2024-05-15T14:12:43.271 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_dir_fetch": 0, 2024-05-15T14:12:43.271 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_backtrace_fetch": 0, 2024-05-15T14:12:43.272 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_peer_discover": 0, 2024-05-15T14:12:43.272 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rfiles": 67, 2024-05-15T14:12:43.272 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rbytes": 21176629, 2024-05-15T14:12:43.272 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rsnaps": 6, 2024-05-15T14:12:43.272 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_backtrace_fetch": 0, 2024-05-15T14:12:43.272 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_set_tag": 0, 2024-05-15T14:12:43.272 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_backtrace_repaired": 0, 2024-05-15T14:12:43.272 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_inotable_repaired": 0, 2024-05-15T14:12:43.272 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dir_inodes": 0, 2024-05-15T14:12:43.273 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dir_base_inodes": 0, 2024-05-15T14:12:43.273 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dirfrag_rstats": 0, 2024-05-15T14:12:43.273 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_file_inodes": 0, 2024-05-15T14:12:43.273 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_inode_file_caps": 22, 2024-05-15T14:12:43.273 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_revoke": 19056, 2024-05-15T14:12:43.273 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_grant": 29784, 2024-05-15T14:12:43.273 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_trunc": 0, 2024-05-15T14:12:43.273 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-15T14:12:43.273 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-15T14:12:43.273 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_caps": 33525, 2024-05-15T14:12:43.274 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_caps_dirty": 8066, 2024-05-15T14:12:43.274 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_cap_release": 4701, 2024-05-15T14:12:43.274 INFO:teuthology.orchestra.run.smithi081.stdout: "process_request_cap_release": 52523 2024-05-15T14:12:43.274 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.274 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_cache": { 2024-05-15T14:12:43.274 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_update": 39, 2024-05-15T14:12:43.274 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_update_receipt": 0, 2024-05-15T14:12:43.274 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_try_discover": 0, 2024-05-15T14:12:43.274 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_send_discover": 23, 2024-05-15T14:12:43.275 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_handle_discover": 571, 2024-05-15T14:12:43.275 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays": 0, 2024-05-15T14:12:43.275 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays_delayed": 0, 2024-05-15T14:12:43.275 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays_enqueuing": 0, 2024-05-15T14:12:43.275 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_created": 9269, 2024-05-15T14:12:43.275 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_enqueued": 9269, 2024-05-15T14:12:43.275 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_reintegrated": 0, 2024-05-15T14:12:43.275 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_migrated": 0, 2024-05-15T14:12:43.275 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_processing": 0, 2024-05-15T14:12:43.276 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_enqueued": 0, 2024-05-15T14:12:43.276 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_prioritized": 0, 2024-05-15T14:12:43.276 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_started": 0, 2024-05-15T14:12:43.276 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_completed": 0, 2024-05-15T14:12:43.276 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_quiesce_path": 10, 2024-05-15T14:12:43.276 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_quiesce_inode": 407, 2024-05-15T14:12:43.276 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_enqueue_scrub": 0, 2024-05-15T14:12:43.276 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_exportdir": 10, 2024-05-15T14:12:43.276 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_flush": 0, 2024-05-15T14:12:43.277 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_fragmentdir": 6, 2024-05-15T14:12:43.277 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_fragstats": 0, 2024-05-15T14:12:43.277 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_inodestats": 0 2024-05-15T14:12:43.277 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.277 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_log": { 2024-05-15T14:12:43.277 INFO:teuthology.orchestra.run.smithi081.stdout: "evlrg": 0, 2024-05-15T14:12:43.277 INFO:teuthology.orchestra.run.smithi081.stdout: "evadd": 63609, 2024-05-15T14:12:43.277 INFO:teuthology.orchestra.run.smithi081.stdout: "evex": 63397, 2024-05-15T14:12:43.277 INFO:teuthology.orchestra.run.smithi081.stdout: "evtrm": 63397, 2024-05-15T14:12:43.277 INFO:teuthology.orchestra.run.smithi081.stdout: "ev": 212, 2024-05-15T14:12:43.278 INFO:teuthology.orchestra.run.smithi081.stdout: "evexg": 0, 2024-05-15T14:12:43.278 INFO:teuthology.orchestra.run.smithi081.stdout: "evexd": 837, 2024-05-15T14:12:43.278 INFO:teuthology.orchestra.run.smithi081.stdout: "segadd": 135, 2024-05-15T14:12:43.278 INFO:teuthology.orchestra.run.smithi081.stdout: "segex": 134, 2024-05-15T14:12:43.278 INFO:teuthology.orchestra.run.smithi081.stdout: "segtrm": 134, 2024-05-15T14:12:43.278 INFO:teuthology.orchestra.run.smithi081.stdout: "seg": 1, 2024-05-15T14:12:43.278 INFO:teuthology.orchestra.run.smithi081.stdout: "segmjr": 1, 2024-05-15T14:12:43.278 INFO:teuthology.orchestra.run.smithi081.stdout: "segexg": 0, 2024-05-15T14:12:43.279 INFO:teuthology.orchestra.run.smithi081.stdout: "segexd": 2, 2024-05-15T14:12:43.279 INFO:teuthology.orchestra.run.smithi081.stdout: "expos": 278373050, 2024-05-15T14:12:43.279 INFO:teuthology.orchestra.run.smithi081.stdout: "wrpos": 279427022, 2024-05-15T14:12:43.279 INFO:teuthology.orchestra.run.smithi081.stdout: "rdpos": 0, 2024-05-15T14:12:43.279 INFO:teuthology.orchestra.run.smithi081.stdout: "jlat": { 2024-05-15T14:12:43.279 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 33711, 2024-05-15T14:12:43.279 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 69.740268211, 2024-05-15T14:12:43.279 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.002068768 2024-05-15T14:12:43.279 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.279 INFO:teuthology.orchestra.run.smithi081.stdout: "replayed": 0 2024-05-15T14:12:43.280 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.280 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_mem": { 2024-05-15T14:12:43.280 INFO:teuthology.orchestra.run.smithi081.stdout: "ino": 241, 2024-05-15T14:12:43.280 INFO:teuthology.orchestra.run.smithi081.stdout: "ino+": 9900, 2024-05-15T14:12:43.280 INFO:teuthology.orchestra.run.smithi081.stdout: "ino-": 9659, 2024-05-15T14:12:43.280 INFO:teuthology.orchestra.run.smithi081.stdout: "dir": 52, 2024-05-15T14:12:43.280 INFO:teuthology.orchestra.run.smithi081.stdout: "dir+": 86, 2024-05-15T14:12:43.280 INFO:teuthology.orchestra.run.smithi081.stdout: "dir-": 34, 2024-05-15T14:12:43.280 INFO:teuthology.orchestra.run.smithi081.stdout: "dn": 328, 2024-05-15T14:12:43.281 INFO:teuthology.orchestra.run.smithi081.stdout: "dn+": 19368, 2024-05-15T14:12:43.281 INFO:teuthology.orchestra.run.smithi081.stdout: "dn-": 19040, 2024-05-15T14:12:43.281 INFO:teuthology.orchestra.run.smithi081.stdout: "cap": 29, 2024-05-15T14:12:43.281 INFO:teuthology.orchestra.run.smithi081.stdout: "cap+": 9838, 2024-05-15T14:12:43.281 INFO:teuthology.orchestra.run.smithi081.stdout: "cap-": 9809, 2024-05-15T14:12:43.281 INFO:teuthology.orchestra.run.smithi081.stdout: "rss": 84388, 2024-05-15T14:12:43.281 INFO:teuthology.orchestra.run.smithi081.stdout: "heap": 231800 2024-05-15T14:12:43.281 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.281 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_server": { 2024-05-15T14:12:43.282 INFO:teuthology.orchestra.run.smithi081.stdout: "dispatch_client_request": 140647, 2024-05-15T14:12:43.282 INFO:teuthology.orchestra.run.smithi081.stdout: "dispatch_server_request": 17664, 2024-05-15T14:12:43.282 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_request": 98930, 2024-05-15T14:12:43.282 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_session": 1560, 2024-05-15T14:12:43.282 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_peer_request": 35259, 2024-05-15T14:12:43.282 INFO:teuthology.orchestra.run.smithi081.stdout: "req_create_latency": { 2024-05-15T14:12:43.282 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 9360, 2024-05-15T14:12:43.282 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 111.291115185, 2024-05-15T14:12:43.282 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.011890076 2024-05-15T14:12:43.283 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.283 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getattr_latency": { 2024-05-15T14:12:43.283 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 19610, 2024-05-15T14:12:43.283 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 101.180059837, 2024-05-15T14:12:43.283 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.005159615 2024-05-15T14:12:43.283 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.283 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getfilelock_latency": { 2024-05-15T14:12:43.283 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:43.284 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:43.284 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:43.284 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.284 INFO:teuthology.orchestra.run.smithi081.stdout: "req_link_latency": { 2024-05-15T14:12:43.284 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:43.284 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:43.284 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:43.284 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.284 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookup_latency": { 2024-05-15T14:12:43.285 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 21908, 2024-05-15T14:12:43.285 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 57.844557704, 2024-05-15T14:12:43.285 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.002640339 2024-05-15T14:12:43.285 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.285 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookuphash_latency": { 2024-05-15T14:12:43.285 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:43.285 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:43.285 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:43.285 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.286 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupino_latency": { 2024-05-15T14:12:43.286 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:43.286 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:43.286 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:43.286 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.286 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupname_latency": { 2024-05-15T14:12:43.286 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:43.286 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:43.287 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:43.287 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.287 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupparent_latency": { 2024-05-15T14:12:43.287 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:43.287 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:43.287 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:43.287 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.287 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupsnap_latency": { 2024-05-15T14:12:43.287 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:43.288 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:43.288 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:43.288 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.288 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lssnap_latency": { 2024-05-15T14:12:43.288 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 12, 2024-05-15T14:12:43.289 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.002845080, 2024-05-15T14:12:43.289 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000237090 2024-05-15T14:12:43.289 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.289 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mkdir_latency": { 2024-05-15T14:12:43.289 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 20, 2024-05-15T14:12:43.289 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.024674158, 2024-05-15T14:12:43.289 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.001233707 2024-05-15T14:12:43.289 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.289 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mknod_latency": { 2024-05-15T14:12:43.290 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:43.290 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:43.290 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:43.290 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.290 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mksnap_latency": { 2024-05-15T14:12:43.290 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 12, 2024-05-15T14:12:43.290 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 2.989232648, 2024-05-15T14:12:43.290 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.249102720 2024-05-15T14:12:43.290 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.291 INFO:teuthology.orchestra.run.smithi081.stdout: "req_open_latency": { 2024-05-15T14:12:43.291 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 8, 2024-05-15T14:12:43.291 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.003589340, 2024-05-15T14:12:43.291 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000448667 2024-05-15T14:12:43.291 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.291 INFO:teuthology.orchestra.run.smithi081.stdout: "req_readdir_latency": { 2024-05-15T14:12:43.291 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 27018, 2024-05-15T14:12:43.291 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 34.919941969, 2024-05-15T14:12:43.292 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.001292469 2024-05-15T14:12:43.292 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.292 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rename_latency": { 2024-05-15T14:12:43.292 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 2768, 2024-05-15T14:12:43.292 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 4.736262948, 2024-05-15T14:12:43.292 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.001711077 2024-05-15T14:12:43.293 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.293 INFO:teuthology.orchestra.run.smithi081.stdout: "req_renamesnap_latency": { 2024-05-15T14:12:43.293 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:43.293 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:43.293 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:43.293 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.293 INFO:teuthology.orchestra.run.smithi081.stdout: "req_snapdiff_latency": { 2024-05-15T14:12:43.294 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:43.294 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:43.294 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:43.294 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.294 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmdir_latency": { 2024-05-15T14:12:43.294 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:43.294 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:43.294 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:43.295 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.295 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmsnap_latency": { 2024-05-15T14:12:43.295 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 6, 2024-05-15T14:12:43.295 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 3.362302009, 2024-05-15T14:12:43.295 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.560383668 2024-05-15T14:12:43.295 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.295 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmxattr_latency": { 2024-05-15T14:12:43.295 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:43.296 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:43.296 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:43.296 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.296 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setattr_latency": { 2024-05-15T14:12:43.296 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 1, 2024-05-15T14:12:43.296 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000858873, 2024-05-15T14:12:43.296 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000858873 2024-05-15T14:12:43.296 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.297 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setdirlayout_latency": { 2024-05-15T14:12:43.297 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:43.297 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:43.297 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:43.297 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.297 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setfilelock_latency": { 2024-05-15T14:12:43.297 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 548, 2024-05-15T14:12:43.297 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.264873247, 2024-05-15T14:12:43.298 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000483345 2024-05-15T14:12:43.298 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.298 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setlayout_latency": { 2024-05-15T14:12:43.298 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:43.298 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:43.298 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:43.298 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.298 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setxattr_latency": { 2024-05-15T14:12:43.298 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 7, 2024-05-15T14:12:43.299 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.021544856, 2024-05-15T14:12:43.299 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.003077836 2024-05-15T14:12:43.299 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.299 INFO:teuthology.orchestra.run.smithi081.stdout: "req_symlink_latency": { 2024-05-15T14:12:43.299 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:43.299 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:43.299 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:43.299 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.300 INFO:teuthology.orchestra.run.smithi081.stdout: "req_unlink_latency": { 2024-05-15T14:12:43.300 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 9269, 2024-05-15T14:12:43.300 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 13.212825465, 2024-05-15T14:12:43.300 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.001425485 2024-05-15T14:12:43.300 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.300 INFO:teuthology.orchestra.run.smithi081.stdout: "cap_revoke_eviction": 0, 2024-05-15T14:12:43.300 INFO:teuthology.orchestra.run.smithi081.stdout: "cap_acquisition_throttle": 0, 2024-05-15T14:12:43.300 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getvxattr_latency": { 2024-05-15T14:12:43.301 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:43.301 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:43.301 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:43.301 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:43.301 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.301 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_sessions": { 2024-05-15T14:12:43.301 INFO:teuthology.orchestra.run.smithi081.stdout: "session_count": 2, 2024-05-15T14:12:43.301 INFO:teuthology.orchestra.run.smithi081.stdout: "session_add": 5, 2024-05-15T14:12:43.302 INFO:teuthology.orchestra.run.smithi081.stdout: "session_remove": 3, 2024-05-15T14:12:43.302 INFO:teuthology.orchestra.run.smithi081.stdout: "sessions_open": 2, 2024-05-15T14:12:43.302 INFO:teuthology.orchestra.run.smithi081.stdout: "sessions_stale": 0, 2024-05-15T14:12:43.302 INFO:teuthology.orchestra.run.smithi081.stdout: "total_load": 9578, 2024-05-15T14:12:43.302 INFO:teuthology.orchestra.run.smithi081.stdout: "average_load": 4789, 2024-05-15T14:12:43.302 INFO:teuthology.orchestra.run.smithi081.stdout: "avg_session_uptime": 719, 2024-05-15T14:12:43.302 INFO:teuthology.orchestra.run.smithi081.stdout: "mdthresh_evicted": 0 2024-05-15T14:12:43.302 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.303 INFO:teuthology.orchestra.run.smithi081.stdout: "mempool": { 2024-05-15T14:12:43.303 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_bytes": 360, 2024-05-15T14:12:43.303 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_items": 360, 2024-05-15T14:12:43.303 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_bytes": 0, 2024-05-15T14:12:43.303 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_items": 0, 2024-05-15T14:12:43.303 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_bytes": 0, 2024-05-15T14:12:43.303 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_items": 0, 2024-05-15T14:12:43.303 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-15T14:12:43.303 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_items": 0, 2024-05-15T14:12:43.304 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-15T14:12:43.304 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_items": 0, 2024-05-15T14:12:43.304 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_bytes": 0, 2024-05-15T14:12:43.304 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_items": 0, 2024-05-15T14:12:43.304 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-15T14:12:43.304 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_items": 0, 2024-05-15T14:12:43.304 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_bytes": 0, 2024-05-15T14:12:43.304 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_items": 0, 2024-05-15T14:12:43.304 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_bytes": 0, 2024-05-15T14:12:43.305 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_items": 0, 2024-05-15T14:12:43.305 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-15T14:12:43.305 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_items": 0, 2024-05-15T14:12:43.305 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-15T14:12:43.305 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_items": 0, 2024-05-15T14:12:43.305 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_bytes": 0, 2024-05-15T14:12:43.305 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_items": 0, 2024-05-15T14:12:43.305 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_bytes": 0, 2024-05-15T14:12:43.306 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_items": 0, 2024-05-15T14:12:43.306 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-15T14:12:43.306 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_items": 0, 2024-05-15T14:12:43.306 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_bytes": 0, 2024-05-15T14:12:43.306 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_items": 0, 2024-05-15T14:12:43.306 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_bytes": 0, 2024-05-15T14:12:43.306 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_items": 0, 2024-05-15T14:12:43.306 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_bytes": 0, 2024-05-15T14:12:43.306 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_items": 0, 2024-05-15T14:12:43.307 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_bytes": 0, 2024-05-15T14:12:43.307 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_items": 0, 2024-05-15T14:12:43.307 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_bytes": 317436, 2024-05-15T14:12:43.307 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_items": 258, 2024-05-15T14:12:43.307 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_bytes": 88, 2024-05-15T14:12:43.307 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_items": 1, 2024-05-15T14:12:43.307 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_bytes": 0, 2024-05-15T14:12:43.307 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_items": 0, 2024-05-15T14:12:43.308 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_bytes": 0, 2024-05-15T14:12:43.308 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_items": 0, 2024-05-15T14:12:43.308 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_bytes": 0, 2024-05-15T14:12:43.308 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_items": 0, 2024-05-15T14:12:43.308 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_bytes": 8128, 2024-05-15T14:12:43.308 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_items": 169, 2024-05-15T14:12:43.308 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_bytes": 0, 2024-05-15T14:12:43.308 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_items": 0, 2024-05-15T14:12:43.308 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_bytes": 0, 2024-05-15T14:12:43.309 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_items": 0, 2024-05-15T14:12:43.309 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_bytes": 1006731, 2024-05-15T14:12:43.309 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_items": 8885, 2024-05-15T14:12:43.309 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_bytes": 0, 2024-05-15T14:12:43.309 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_items": 0, 2024-05-15T14:12:43.309 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_bytes": 0, 2024-05-15T14:12:43.309 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_items": 0 2024-05-15T14:12:43.309 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.309 INFO:teuthology.orchestra.run.smithi081.stdout: "objecter": { 2024-05-15T14:12:43.310 INFO:teuthology.orchestra.run.smithi081.stdout: "op_active": 0, 2024-05-15T14:12:43.310 INFO:teuthology.orchestra.run.smithi081.stdout: "op_laggy": 0, 2024-05-15T14:12:43.310 INFO:teuthology.orchestra.run.smithi081.stdout: "op_send": 52287, 2024-05-15T14:12:43.310 INFO:teuthology.orchestra.run.smithi081.stdout: "op_send_bytes": 286075217, 2024-05-15T14:12:43.310 INFO:teuthology.orchestra.run.smithi081.stdout: "op_resend": 37, 2024-05-15T14:12:43.310 INFO:teuthology.orchestra.run.smithi081.stdout: "op_reply": 52287, 2024-05-15T14:12:43.310 INFO:teuthology.orchestra.run.smithi081.stdout: "op_latency": { 2024-05-15T14:12:43.310 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 52287, 2024-05-15T14:12:43.310 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 280.286049032, 2024-05-15T14:12:43.311 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.005360530 2024-05-15T14:12:43.311 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.311 INFO:teuthology.orchestra.run.smithi081.stdout: "op_inflight": 0, 2024-05-15T14:12:43.311 INFO:teuthology.orchestra.run.smithi081.stdout: "oplen_avg": { 2024-05-15T14:12:43.311 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 52287, 2024-05-15T14:12:43.311 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 66905 2024-05-15T14:12:43.311 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.311 INFO:teuthology.orchestra.run.smithi081.stdout: "op": 52287, 2024-05-15T14:12:43.311 INFO:teuthology.orchestra.run.smithi081.stdout: "op_r": 397, 2024-05-15T14:12:43.311 INFO:teuthology.orchestra.run.smithi081.stdout: "op_w": 51890, 2024-05-15T14:12:43.312 INFO:teuthology.orchestra.run.smithi081.stdout: "op_rmw": 0, 2024-05-15T14:12:43.312 INFO:teuthology.orchestra.run.smithi081.stdout: "op_pg": 0, 2024-05-15T14:12:43.312 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_stat": 1049, 2024-05-15T14:12:43.312 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_create": 5795, 2024-05-15T14:12:43.312 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_read": 390, 2024-05-15T14:12:43.312 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_write": 34101, 2024-05-15T14:12:43.312 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_writefull": 426, 2024-05-15T14:12:43.312 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_writesame": 0, 2024-05-15T14:12:43.312 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_append": 0, 2024-05-15T14:12:43.313 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_zero": 0, 2024-05-15T14:12:43.313 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_truncate": 3, 2024-05-15T14:12:43.313 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_delete": 9935, 2024-05-15T14:12:43.313 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_mapext": 0, 2024-05-15T14:12:43.313 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_sparse_read": 0, 2024-05-15T14:12:43.313 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_clonerange": 0, 2024-05-15T14:12:43.313 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_getxattr": 5, 2024-05-15T14:12:43.313 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_setxattr": 11590, 2024-05-15T14:12:43.313 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_cmpxattr": 0, 2024-05-15T14:12:43.314 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_rmxattr": 0, 2024-05-15T14:12:43.314 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_resetxattrs": 0, 2024-05-15T14:12:43.314 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_call": 0, 2024-05-15T14:12:43.314 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_watch": 0, 2024-05-15T14:12:43.314 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_notify": 0, 2024-05-15T14:12:43.314 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_src_cmpxattr": 0, 2024-05-15T14:12:43.314 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_pgls": 0, 2024-05-15T14:12:43.314 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_pgls_filter": 0, 2024-05-15T14:12:43.314 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_other": 0, 2024-05-15T14:12:43.314 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_active": 0, 2024-05-15T14:12:43.315 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_send": 0, 2024-05-15T14:12:43.315 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_resend": 0, 2024-05-15T14:12:43.315 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_ping": 0, 2024-05-15T14:12:43.315 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_active": 0, 2024-05-15T14:12:43.315 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_send": 0, 2024-05-15T14:12:43.315 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_resend": 0, 2024-05-15T14:12:43.315 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_active": 0, 2024-05-15T14:12:43.315 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_send": 0, 2024-05-15T14:12:43.315 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_resend": 0, 2024-05-15T14:12:43.315 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_active": 0, 2024-05-15T14:12:43.316 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_send": 0, 2024-05-15T14:12:43.316 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_resend": 0, 2024-05-15T14:12:43.316 INFO:teuthology.orchestra.run.smithi081.stdout: "command_active": 0, 2024-05-15T14:12:43.316 INFO:teuthology.orchestra.run.smithi081.stdout: "command_send": 0, 2024-05-15T14:12:43.316 INFO:teuthology.orchestra.run.smithi081.stdout: "command_resend": 0, 2024-05-15T14:12:43.316 INFO:teuthology.orchestra.run.smithi081.stdout: "map_epoch": 98, 2024-05-15T14:12:43.316 INFO:teuthology.orchestra.run.smithi081.stdout: "map_full": 0, 2024-05-15T14:12:43.316 INFO:teuthology.orchestra.run.smithi081.stdout: "map_inc": 18, 2024-05-15T14:12:43.317 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_sessions": 12, 2024-05-15T14:12:43.317 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_session_open": 12, 2024-05-15T14:12:43.317 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_session_close": 0, 2024-05-15T14:12:43.317 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_laggy": 0, 2024-05-15T14:12:43.317 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_wr": 2964, 2024-05-15T14:12:43.317 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_rd": 14, 2024-05-15T14:12:43.317 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_del": 633 2024-05-15T14:12:43.317 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.318 INFO:teuthology.orchestra.run.smithi081.stdout: "oft": { 2024-05-15T14:12:43.318 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_objs": 1, 2024-05-15T14:12:43.318 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_kv_pairs": 0, 2024-05-15T14:12:43.318 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_updates": 0, 2024-05-15T14:12:43.318 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_removes": 0 2024-05-15T14:12:43.318 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.318 INFO:teuthology.orchestra.run.smithi081.stdout: "osd-slow-ops": { 2024-05-15T14:12:43.318 INFO:teuthology.orchestra.run.smithi081.stdout: "slow_ops_count": 375 2024-05-15T14:12:43.318 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.319 INFO:teuthology.orchestra.run.smithi081.stdout: "purge_queue": { 2024-05-15T14:12:43.319 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_ops": 0, 2024-05-15T14:12:43.319 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_ops_high_water": 10, 2024-05-15T14:12:43.319 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing": 0, 2024-05-15T14:12:43.319 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_high_water": 10, 2024-05-15T14:12:43.319 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executed_ops": 9269, 2024-05-15T14:12:43.319 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executed": 9269, 2024-05-15T14:12:43.319 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_item_in_journal": 0 2024-05-15T14:12:43.319 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.320 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-15T14:12:43.320 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-05-15T14:12:43.320 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2024-05-15T14:12:43.320 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-05-15T14:12:43.320 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 361962, 2024-05-15T14:12:43.320 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 65745349, 2024-05-15T14:12:43.320 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:43.320 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 361962, 2024-05-15T14:12:43.320 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-05-15T14:12:43.320 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-05-15T14:12:43.321 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 361962, 2024-05-15T14:12:43.321 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 65745349, 2024-05-15T14:12:43.321 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-05-15T14:12:43.321 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:43.321 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:43.321 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:43.321 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:43.321 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.322 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-objecter_bytes": { 2024-05-15T14:12:43.322 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-05-15T14:12:43.322 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2024-05-15T14:12:43.322 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-05-15T14:12:43.322 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-05-15T14:12:43.322 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-05-15T14:12:43.322 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:43.323 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-05-15T14:12:43.323 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 52287, 2024-05-15T14:12:43.323 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 287012554, 2024-05-15T14:12:43.323 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 52287, 2024-05-15T14:12:43.323 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 287012554, 2024-05-15T14:12:43.323 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-05-15T14:12:43.323 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:43.323 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:43.324 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:43.324 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:43.324 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.324 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-objecter_ops": { 2024-05-15T14:12:43.324 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-05-15T14:12:43.324 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 1024, 2024-05-15T14:12:43.324 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-05-15T14:12:43.324 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-05-15T14:12:43.324 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-05-15T14:12:43.325 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:43.325 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-05-15T14:12:43.325 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 52287, 2024-05-15T14:12:43.325 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 52287, 2024-05-15T14:12:43.325 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 52287, 2024-05-15T14:12:43.325 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 52287, 2024-05-15T14:12:43.325 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-05-15T14:12:43.325 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:43.326 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:43.326 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:43.326 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:43.326 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.326 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-write_buf_throttle": { 2024-05-15T14:12:43.326 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-05-15T14:12:43.326 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 3758096384, 2024-05-15T14:12:43.326 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-05-15T14:12:43.327 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 9269, 2024-05-15T14:12:43.327 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 937449, 2024-05-15T14:12:43.327 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:43.327 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 9269, 2024-05-15T14:12:43.327 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-05-15T14:12:43.327 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-05-15T14:12:43.327 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 390, 2024-05-15T14:12:43.327 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 937449, 2024-05-15T14:12:43.327 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-05-15T14:12:43.327 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:43.328 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:43.328 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:43.328 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:43.328 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:43.328 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-write_buf_throttle-0x564b9a550460": { 2024-05-15T14:12:43.328 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-05-15T14:12:43.328 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 3758096384, 2024-05-15T14:12:43.328 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-05-15T14:12:43.329 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 63609, 2024-05-15T14:12:43.329 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 275232718, 2024-05-15T14:12:43.329 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:43.329 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 63609, 2024-05-15T14:12:43.329 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-05-15T14:12:43.329 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-05-15T14:12:43.329 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 33711, 2024-05-15T14:12:43.329 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 275232718, 2024-05-15T14:12:43.329 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-05-15T14:12:43.330 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:43.330 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:43.330 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:43.330 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:43.330 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:43.330 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-05-15T14:12:43.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:43 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:12:43.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:43 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:12:43.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:43 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-15T14:12:44.045 INFO:tasks.check_counter:Daemon mds.i mds.exported=10 2024-05-15T14:12:44.045 INFO:tasks.check_counter:Daemon mds.i mds.imported=10 2024-05-15T14:12:44.046 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=39 2024-05-15T14:12:44.046 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=0 2024-05-15T14:12:44.046 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-05-15T14:12:44.046 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=12 2024-05-15T14:12:44.046 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=6 2024-05-15T14:12:44.046 DEBUG:tasks.check_counter:Getting stats from l 2024-05-15T14:12:44.046 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-15T14:12:44.280 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.c/config 2024-05-15T14:12:44.672 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:44 smithi081 ceph-mon[36434]: pgmap v389: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 43 KiB/s wr, 14 op/s 2024-05-15T14:12:44.672 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:44 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:12:44.672 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:44 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:12:44.672 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:44 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:12:44.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:44 smithi033 ceph-mon[31650]: pgmap v389: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 43 KiB/s wr, 14 op/s 2024-05-15T14:12:44.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:44 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:12:44.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:44 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:12:44.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:44 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:12:44.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:44 smithi052 ceph-mon[36190]: pgmap v389: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 43 KiB/s wr, 14 op/s 2024-05-15T14:12:44.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:44 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-15T14:12:44.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:44 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-15T14:12:44.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:44 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' 2024-05-15T14:12:45.749 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:45 smithi081 ceph-mon[36434]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:12:45.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:45 smithi052 ceph-mon[36190]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:12:45.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:45 smithi033 ceph-mon[31650]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 3 up:standby-replay 6 up:standby 2024-05-15T14:12:45.992 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-05-15T14:12:45.992 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-0": { 2024-05-15T14:12:45.993 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 835, 2024-05-15T14:12:45.993 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 804, 2024-05-15T14:12:45.993 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 24990517, 2024-05-15T14:12:45.993 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 330406, 2024-05-15T14:12:45.993 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 6, 2024-05-15T14:12:45.993 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 2, 2024-05-15T14:12:45.993 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.104084730, 2024-05-15T14:12:45.994 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.027386012, 2024-05-15T14:12:45.994 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.065676943, 2024-05-15T14:12:45.994 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.018722345, 2024-05-15T14:12:45.994 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:45.994 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 803, 2024-05-15T14:12:45.994 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.031569802, 2024-05-15T14:12:45.994 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000039314 2024-05-15T14:12:45.994 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:45.995 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:45.995 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:45.995 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:45.995 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:45.995 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:45.995 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 1834384, 2024-05-15T14:12:45.995 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 165040 2024-05-15T14:12:45.995 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:45.995 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-1": { 2024-05-15T14:12:45.996 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 741, 2024-05-15T14:12:45.996 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 938, 2024-05-15T14:12:45.996 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 33740568, 2024-05-15T14:12:45.996 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 736863, 2024-05-15T14:12:45.996 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 8, 2024-05-15T14:12:45.996 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 6, 2024-05-15T14:12:45.996 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.083513573, 2024-05-15T14:12:45.996 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.025228767, 2024-05-15T14:12:45.996 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.031185555, 2024-05-15T14:12:45.997 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.019677973, 2024-05-15T14:12:45.997 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:45.997 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 935, 2024-05-15T14:12:45.997 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.041026075, 2024-05-15T14:12:45.997 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000043878 2024-05-15T14:12:45.997 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:45.997 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:45.997 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:45.997 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:45.998 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:45.998 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:45.998 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-15T14:12:45.998 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 501856 2024-05-15T14:12:45.998 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:45.998 INFO:teuthology.orchestra.run.smithi081.stdout: "AsyncMessenger::Worker-2": { 2024-05-15T14:12:45.998 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_messages": 1769, 2024-05-15T14:12:45.998 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages": 1769, 2024-05-15T14:12:45.999 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_bytes": 38276868, 2024-05-15T14:12:45.999 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_bytes": 561776, 2024-05-15T14:12:45.999 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_created_connections": 6, 2024-05-15T14:12:45.999 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_active_connections": 5, 2024-05-15T14:12:45.999 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_total_time": 0.171597902, 2024-05-15T14:12:45.999 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_send_time": 0.047402761, 2024-05-15T14:12:45.999 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_recv_time": 0.060557318, 2024-05-15T14:12:46.000 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_running_fast_dispatch_time": 0.049097156, 2024-05-15T14:12:46.000 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_messages_queue_lat": { 2024-05-15T14:12:46.000 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 1768, 2024-05-15T14:12:46.000 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.062825179, 2024-05-15T14:12:46.000 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000035534 2024-05-15T14:12:46.000 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.000 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_handle_ack_lat": { 2024-05-15T14:12:46.000 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.001 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.001 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.001 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.001 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-15T14:12:46.001 INFO:teuthology.orchestra.run.smithi081.stdout: "msgr_send_encrypted_bytes": 448 2024-05-15T14:12:46.001 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.001 INFO:teuthology.orchestra.run.smithi081.stdout: "cct": { 2024-05-15T14:12:46.001 INFO:teuthology.orchestra.run.smithi081.stdout: "total_workers": 1, 2024-05-15T14:12:46.001 INFO:teuthology.orchestra.run.smithi081.stdout: "unhealthy_workers": 0 2024-05-15T14:12:46.002 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.002 INFO:teuthology.orchestra.run.smithi081.stdout: "finisher-MDSRank": { 2024-05-15T14:12:46.002 INFO:teuthology.orchestra.run.smithi081.stdout: "queue_len": 0, 2024-05-15T14:12:46.002 INFO:teuthology.orchestra.run.smithi081.stdout: "complete_latency": { 2024-05-15T14:12:46.002 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 5147, 2024-05-15T14:12:46.002 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.121126400, 2024-05-15T14:12:46.002 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000023533 2024-05-15T14:12:46.002 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:46.003 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.003 INFO:teuthology.orchestra.run.smithi081.stdout: "finisher-PurgeQueue": { 2024-05-15T14:12:46.003 INFO:teuthology.orchestra.run.smithi081.stdout: "queue_len": 0, 2024-05-15T14:12:46.003 INFO:teuthology.orchestra.run.smithi081.stdout: "complete_latency": { 2024-05-15T14:12:46.003 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.003 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.003 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.003 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:46.004 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.004 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": { 2024-05-15T14:12:46.004 INFO:teuthology.orchestra.run.smithi081.stdout: "request": 0, 2024-05-15T14:12:46.004 INFO:teuthology.orchestra.run.smithi081.stdout: "reply": 0, 2024-05-15T14:12:46.004 INFO:teuthology.orchestra.run.smithi081.stdout: "reply_latency": { 2024-05-15T14:12:46.004 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.004 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.004 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.005 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.005 INFO:teuthology.orchestra.run.smithi081.stdout: "slow_reply": 0, 2024-05-15T14:12:46.005 INFO:teuthology.orchestra.run.smithi081.stdout: "forward": 0, 2024-05-15T14:12:46.005 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_fetch_complete": 0, 2024-05-15T14:12:46.005 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_fetch_keys": 0, 2024-05-15T14:12:46.005 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_commit": 0, 2024-05-15T14:12:46.005 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_split": 0, 2024-05-15T14:12:46.005 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_merge": 0, 2024-05-15T14:12:46.005 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes": 10473, 2024-05-15T14:12:46.006 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_top": 23, 2024-05-15T14:12:46.006 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_bottom": 10450, 2024-05-15T14:12:46.006 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_pin_tail": 0, 2024-05-15T14:12:46.006 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_pinned": 10439, 2024-05-15T14:12:46.006 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_expired": 0, 2024-05-15T14:12:46.006 INFO:teuthology.orchestra.run.smithi081.stdout: "inodes_with_caps": 0, 2024-05-15T14:12:46.006 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": 0, 2024-05-15T14:12:46.006 INFO:teuthology.orchestra.run.smithi081.stdout: "subtrees": 11, 2024-05-15T14:12:46.007 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse": 0, 2024-05-15T14:12:46.007 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_hit": 0, 2024-05-15T14:12:46.007 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_forward": 0, 2024-05-15T14:12:46.007 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_discover": 0, 2024-05-15T14:12:46.007 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_dir_fetch": 0, 2024-05-15T14:12:46.007 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_remote_ino": 0, 2024-05-15T14:12:46.007 INFO:teuthology.orchestra.run.smithi081.stdout: "traverse_lock": 0, 2024-05-15T14:12:46.008 INFO:teuthology.orchestra.run.smithi081.stdout: "load_cent": 0, 2024-05-15T14:12:46.008 INFO:teuthology.orchestra.run.smithi081.stdout: "q": 0, 2024-05-15T14:12:46.008 INFO:teuthology.orchestra.run.smithi081.stdout: "exported": 0, 2024-05-15T14:12:46.008 INFO:teuthology.orchestra.run.smithi081.stdout: "exported_inodes": 0, 2024-05-15T14:12:46.008 INFO:teuthology.orchestra.run.smithi081.stdout: "imported": 0, 2024-05-15T14:12:46.008 INFO:teuthology.orchestra.run.smithi081.stdout: "imported_inodes": 0, 2024-05-15T14:12:46.008 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_dir_fetch": 0, 2024-05-15T14:12:46.008 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_backtrace_fetch": 0, 2024-05-15T14:12:46.009 INFO:teuthology.orchestra.run.smithi081.stdout: "openino_peer_discover": 0, 2024-05-15T14:12:46.009 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rfiles": 0, 2024-05-15T14:12:46.009 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rbytes": 0, 2024-05-15T14:12:46.009 INFO:teuthology.orchestra.run.smithi081.stdout: "root_rsnaps": 0, 2024-05-15T14:12:46.009 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_backtrace_fetch": 0, 2024-05-15T14:12:46.009 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_set_tag": 0, 2024-05-15T14:12:46.009 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_backtrace_repaired": 0, 2024-05-15T14:12:46.009 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_inotable_repaired": 0, 2024-05-15T14:12:46.009 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dir_inodes": 0, 2024-05-15T14:12:46.010 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dir_base_inodes": 0, 2024-05-15T14:12:46.010 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_dirfrag_rstats": 0, 2024-05-15T14:12:46.010 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_file_inodes": 0, 2024-05-15T14:12:46.010 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_inode_file_caps": 0, 2024-05-15T14:12:46.010 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_revoke": 0, 2024-05-15T14:12:46.010 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_grant": 0, 2024-05-15T14:12:46.010 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_trunc": 0, 2024-05-15T14:12:46.010 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-15T14:12:46.011 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-15T14:12:46.011 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_caps": 0, 2024-05-15T14:12:46.011 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_caps_dirty": 0, 2024-05-15T14:12:46.011 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_cap_release": 0, 2024-05-15T14:12:46.011 INFO:teuthology.orchestra.run.smithi081.stdout: "process_request_cap_release": 0 2024-05-15T14:12:46.011 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.011 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_cache": { 2024-05-15T14:12:46.011 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_update": 0, 2024-05-15T14:12:46.011 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_update_receipt": 0, 2024-05-15T14:12:46.012 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_try_discover": 0, 2024-05-15T14:12:46.012 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_send_discover": 0, 2024-05-15T14:12:46.012 INFO:teuthology.orchestra.run.smithi081.stdout: "dir_handle_discover": 0, 2024-05-15T14:12:46.012 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays": 0, 2024-05-15T14:12:46.012 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays_delayed": 0, 2024-05-15T14:12:46.012 INFO:teuthology.orchestra.run.smithi081.stdout: "num_strays_enqueuing": 0, 2024-05-15T14:12:46.012 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_created": 0, 2024-05-15T14:12:46.012 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_enqueued": 0, 2024-05-15T14:12:46.013 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_reintegrated": 0, 2024-05-15T14:12:46.013 INFO:teuthology.orchestra.run.smithi081.stdout: "strays_migrated": 0, 2024-05-15T14:12:46.013 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_processing": 0, 2024-05-15T14:12:46.013 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_enqueued": 0, 2024-05-15T14:12:46.013 INFO:teuthology.orchestra.run.smithi081.stdout: "num_recovering_prioritized": 0, 2024-05-15T14:12:46.013 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_started": 0, 2024-05-15T14:12:46.013 INFO:teuthology.orchestra.run.smithi081.stdout: "recovery_completed": 0, 2024-05-15T14:12:46.013 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_quiesce_path": 0, 2024-05-15T14:12:46.013 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_quiesce_inode": 0, 2024-05-15T14:12:46.014 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_enqueue_scrub": 0, 2024-05-15T14:12:46.014 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_exportdir": 0, 2024-05-15T14:12:46.014 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_flush": 0, 2024-05-15T14:12:46.014 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_fragmentdir": 0, 2024-05-15T14:12:46.014 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_fragstats": 0, 2024-05-15T14:12:46.014 INFO:teuthology.orchestra.run.smithi081.stdout: "ireq_inodestats": 0 2024-05-15T14:12:46.014 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.014 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_log": { 2024-05-15T14:12:46.014 INFO:teuthology.orchestra.run.smithi081.stdout: "evlrg": 0, 2024-05-15T14:12:46.015 INFO:teuthology.orchestra.run.smithi081.stdout: "evadd": 0, 2024-05-15T14:12:46.015 INFO:teuthology.orchestra.run.smithi081.stdout: "evex": 0, 2024-05-15T14:12:46.015 INFO:teuthology.orchestra.run.smithi081.stdout: "evtrm": 0, 2024-05-15T14:12:46.015 INFO:teuthology.orchestra.run.smithi081.stdout: "ev": 53182, 2024-05-15T14:12:46.015 INFO:teuthology.orchestra.run.smithi081.stdout: "evexg": 0, 2024-05-15T14:12:46.015 INFO:teuthology.orchestra.run.smithi081.stdout: "evexd": 0, 2024-05-15T14:12:46.015 INFO:teuthology.orchestra.run.smithi081.stdout: "segadd": 0, 2024-05-15T14:12:46.015 INFO:teuthology.orchestra.run.smithi081.stdout: "segex": 0, 2024-05-15T14:12:46.016 INFO:teuthology.orchestra.run.smithi081.stdout: "segtrm": 0, 2024-05-15T14:12:46.016 INFO:teuthology.orchestra.run.smithi081.stdout: "seg": 67, 2024-05-15T14:12:46.016 INFO:teuthology.orchestra.run.smithi081.stdout: "segmjr": 6, 2024-05-15T14:12:46.016 INFO:teuthology.orchestra.run.smithi081.stdout: "segexg": 0, 2024-05-15T14:12:46.016 INFO:teuthology.orchestra.run.smithi081.stdout: "segexd": 0, 2024-05-15T14:12:46.016 INFO:teuthology.orchestra.run.smithi081.stdout: "expos": 4194304, 2024-05-15T14:12:46.016 INFO:teuthology.orchestra.run.smithi081.stdout: "wrpos": 98520921, 2024-05-15T14:12:46.016 INFO:teuthology.orchestra.run.smithi081.stdout: "rdpos": 98520347, 2024-05-15T14:12:46.016 INFO:teuthology.orchestra.run.smithi081.stdout: "jlat": { 2024-05-15T14:12:46.016 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.017 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.017 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.017 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.017 INFO:teuthology.orchestra.run.smithi081.stdout: "replayed": 53182 2024-05-15T14:12:46.017 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.017 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_mem": { 2024-05-15T14:12:46.017 INFO:teuthology.orchestra.run.smithi081.stdout: "ino": 230, 2024-05-15T14:12:46.017 INFO:teuthology.orchestra.run.smithi081.stdout: "ino+": 10458, 2024-05-15T14:12:46.017 INFO:teuthology.orchestra.run.smithi081.stdout: "ino-": 10228, 2024-05-15T14:12:46.018 INFO:teuthology.orchestra.run.smithi081.stdout: "dir": 30, 2024-05-15T14:12:46.018 INFO:teuthology.orchestra.run.smithi081.stdout: "dir+": 33, 2024-05-15T14:12:46.018 INFO:teuthology.orchestra.run.smithi081.stdout: "dir-": 3, 2024-05-15T14:12:46.018 INFO:teuthology.orchestra.run.smithi081.stdout: "dn": 10473, 2024-05-15T14:12:46.018 INFO:teuthology.orchestra.run.smithi081.stdout: "dn+": 10529, 2024-05-15T14:12:46.018 INFO:teuthology.orchestra.run.smithi081.stdout: "dn-": 56, 2024-05-15T14:12:46.018 INFO:teuthology.orchestra.run.smithi081.stdout: "cap": 0, 2024-05-15T14:12:46.018 INFO:teuthology.orchestra.run.smithi081.stdout: "cap+": 0, 2024-05-15T14:12:46.018 INFO:teuthology.orchestra.run.smithi081.stdout: "cap-": 0, 2024-05-15T14:12:46.019 INFO:teuthology.orchestra.run.smithi081.stdout: "rss": 61776, 2024-05-15T14:12:46.019 INFO:teuthology.orchestra.run.smithi081.stdout: "heap": 215416 2024-05-15T14:12:46.019 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.019 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_server": { 2024-05-15T14:12:46.019 INFO:teuthology.orchestra.run.smithi081.stdout: "dispatch_client_request": 0, 2024-05-15T14:12:46.019 INFO:teuthology.orchestra.run.smithi081.stdout: "dispatch_server_request": 0, 2024-05-15T14:12:46.019 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_request": 0, 2024-05-15T14:12:46.019 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_client_session": 0, 2024-05-15T14:12:46.019 INFO:teuthology.orchestra.run.smithi081.stdout: "handle_peer_request": 0, 2024-05-15T14:12:46.020 INFO:teuthology.orchestra.run.smithi081.stdout: "req_create_latency": { 2024-05-15T14:12:46.020 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.020 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.020 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.020 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.020 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getattr_latency": { 2024-05-15T14:12:46.020 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.020 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.020 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.021 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.021 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getfilelock_latency": { 2024-05-15T14:12:46.021 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.021 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.021 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.021 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.021 INFO:teuthology.orchestra.run.smithi081.stdout: "req_link_latency": { 2024-05-15T14:12:46.021 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.022 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.022 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.022 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.022 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookup_latency": { 2024-05-15T14:12:46.022 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.022 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.022 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.022 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.023 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookuphash_latency": { 2024-05-15T14:12:46.023 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.023 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.023 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.023 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.023 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupino_latency": { 2024-05-15T14:12:46.023 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.023 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.023 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.023 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.024 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupname_latency": { 2024-05-15T14:12:46.024 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.024 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.024 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.024 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.024 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupparent_latency": { 2024-05-15T14:12:46.024 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.024 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.024 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.024 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.025 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lookupsnap_latency": { 2024-05-15T14:12:46.025 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.025 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.025 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.025 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.025 INFO:teuthology.orchestra.run.smithi081.stdout: "req_lssnap_latency": { 2024-05-15T14:12:46.025 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.025 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.025 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.026 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.026 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mkdir_latency": { 2024-05-15T14:12:46.026 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.026 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.027 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.027 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.027 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mknod_latency": { 2024-05-15T14:12:46.027 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.027 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.027 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.027 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.027 INFO:teuthology.orchestra.run.smithi081.stdout: "req_mksnap_latency": { 2024-05-15T14:12:46.028 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.028 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.028 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.028 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.028 INFO:teuthology.orchestra.run.smithi081.stdout: "req_open_latency": { 2024-05-15T14:12:46.028 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.028 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.028 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.029 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.029 INFO:teuthology.orchestra.run.smithi081.stdout: "req_readdir_latency": { 2024-05-15T14:12:46.029 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.029 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.029 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.029 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.029 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rename_latency": { 2024-05-15T14:12:46.029 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.030 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.030 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.030 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.030 INFO:teuthology.orchestra.run.smithi081.stdout: "req_renamesnap_latency": { 2024-05-15T14:12:46.030 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.030 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.030 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.030 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.030 INFO:teuthology.orchestra.run.smithi081.stdout: "req_snapdiff_latency": { 2024-05-15T14:12:46.031 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.031 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.031 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.031 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.031 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmdir_latency": { 2024-05-15T14:12:46.031 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.031 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.031 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.032 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.032 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmsnap_latency": { 2024-05-15T14:12:46.032 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.032 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.032 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.032 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.032 INFO:teuthology.orchestra.run.smithi081.stdout: "req_rmxattr_latency": { 2024-05-15T14:12:46.032 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.033 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.033 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.033 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.033 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setattr_latency": { 2024-05-15T14:12:46.033 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.033 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.033 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.033 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.033 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setdirlayout_latency": { 2024-05-15T14:12:46.034 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.034 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.034 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.034 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.034 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setfilelock_latency": { 2024-05-15T14:12:46.034 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.034 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.035 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.035 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.035 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setlayout_latency": { 2024-05-15T14:12:46.035 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.035 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.035 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.035 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.035 INFO:teuthology.orchestra.run.smithi081.stdout: "req_setxattr_latency": { 2024-05-15T14:12:46.035 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.036 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.036 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.036 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.036 INFO:teuthology.orchestra.run.smithi081.stdout: "req_symlink_latency": { 2024-05-15T14:12:46.036 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.036 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.036 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.036 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.037 INFO:teuthology.orchestra.run.smithi081.stdout: "req_unlink_latency": { 2024-05-15T14:12:46.037 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.037 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.037 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.037 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.037 INFO:teuthology.orchestra.run.smithi081.stdout: "cap_revoke_eviction": 0, 2024-05-15T14:12:46.037 INFO:teuthology.orchestra.run.smithi081.stdout: "cap_acquisition_throttle": 0, 2024-05-15T14:12:46.037 INFO:teuthology.orchestra.run.smithi081.stdout: "req_getvxattr_latency": { 2024-05-15T14:12:46.037 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.038 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.038 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.038 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:46.038 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.038 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_sessions": { 2024-05-15T14:12:46.038 INFO:teuthology.orchestra.run.smithi081.stdout: "session_count": 2, 2024-05-15T14:12:46.038 INFO:teuthology.orchestra.run.smithi081.stdout: "session_add": 4, 2024-05-15T14:12:46.038 INFO:teuthology.orchestra.run.smithi081.stdout: "session_remove": 2, 2024-05-15T14:12:46.039 INFO:teuthology.orchestra.run.smithi081.stdout: "sessions_open": 3, 2024-05-15T14:12:46.039 INFO:teuthology.orchestra.run.smithi081.stdout: "sessions_stale": 0, 2024-05-15T14:12:46.039 INFO:teuthology.orchestra.run.smithi081.stdout: "total_load": 0, 2024-05-15T14:12:46.039 INFO:teuthology.orchestra.run.smithi081.stdout: "average_load": 0, 2024-05-15T14:12:46.039 INFO:teuthology.orchestra.run.smithi081.stdout: "avg_session_uptime": 2830, 2024-05-15T14:12:46.039 INFO:teuthology.orchestra.run.smithi081.stdout: "mdthresh_evicted": 0 2024-05-15T14:12:46.039 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.039 INFO:teuthology.orchestra.run.smithi081.stdout: "mempool": { 2024-05-15T14:12:46.039 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_bytes": 0, 2024-05-15T14:12:46.040 INFO:teuthology.orchestra.run.smithi081.stdout: "bloom_filter_items": 0, 2024-05-15T14:12:46.040 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_bytes": 0, 2024-05-15T14:12:46.040 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_alloc_items": 0, 2024-05-15T14:12:46.040 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_bytes": 0, 2024-05-15T14:12:46.040 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_data_items": 0, 2024-05-15T14:12:46.040 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-15T14:12:46.040 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_onode_items": 0, 2024-05-15T14:12:46.040 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-15T14:12:46.040 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_meta_items": 0, 2024-05-15T14:12:46.041 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_bytes": 0, 2024-05-15T14:12:46.041 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_other_items": 0, 2024-05-15T14:12:46.041 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-15T14:12:46.041 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_cache_buffer_items": 0, 2024-05-15T14:12:46.041 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_bytes": 0, 2024-05-15T14:12:46.041 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_extent_items": 0, 2024-05-15T14:12:46.041 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_bytes": 0, 2024-05-15T14:12:46.041 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_blob_items": 0, 2024-05-15T14:12:46.041 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-15T14:12:46.042 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_shared_blob_items": 0, 2024-05-15T14:12:46.042 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-15T14:12:46.042 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_inline_bl_items": 0, 2024-05-15T14:12:46.042 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_bytes": 0, 2024-05-15T14:12:46.042 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_fsck_items": 0, 2024-05-15T14:12:46.042 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_bytes": 0, 2024-05-15T14:12:46.042 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_txc_items": 0, 2024-05-15T14:12:46.042 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-15T14:12:46.043 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_deferred_items": 0, 2024-05-15T14:12:46.043 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_bytes": 0, 2024-05-15T14:12:46.043 INFO:teuthology.orchestra.run.smithi081.stdout: "bluestore_writing_items": 0, 2024-05-15T14:12:46.043 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_bytes": 0, 2024-05-15T14:12:46.043 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_items": 0, 2024-05-15T14:12:46.043 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_bytes": 0, 2024-05-15T14:12:46.043 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_reader_items": 0, 2024-05-15T14:12:46.043 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_bytes": 0, 2024-05-15T14:12:46.043 INFO:teuthology.orchestra.run.smithi081.stdout: "bluefs_file_writer_items": 0, 2024-05-15T14:12:46.044 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_bytes": 232011, 2024-05-15T14:12:46.044 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_anon_items": 287, 2024-05-15T14:12:46.044 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_bytes": 0, 2024-05-15T14:12:46.044 INFO:teuthology.orchestra.run.smithi081.stdout: "buffer_meta_items": 0, 2024-05-15T14:12:46.044 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_bytes": 0, 2024-05-15T14:12:46.044 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_items": 0, 2024-05-15T14:12:46.044 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_bytes": 0, 2024-05-15T14:12:46.044 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_mapbl_items": 0, 2024-05-15T14:12:46.045 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_bytes": 0, 2024-05-15T14:12:46.045 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_pglog_items": 0, 2024-05-15T14:12:46.045 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_bytes": 8128, 2024-05-15T14:12:46.045 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_items": 169, 2024-05-15T14:12:46.045 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_bytes": 0, 2024-05-15T14:12:46.045 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_mapping_items": 0, 2024-05-15T14:12:46.045 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_bytes": 0, 2024-05-15T14:12:46.045 INFO:teuthology.orchestra.run.smithi081.stdout: "pgmap_items": 0, 2024-05-15T14:12:46.045 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_bytes": 8022564, 2024-05-15T14:12:46.046 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_co_items": 47064, 2024-05-15T14:12:46.046 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_bytes": 0, 2024-05-15T14:12:46.046 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_1_items": 0, 2024-05-15T14:12:46.046 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_bytes": 0, 2024-05-15T14:12:46.046 INFO:teuthology.orchestra.run.smithi081.stdout: "unittest_2_items": 0 2024-05-15T14:12:46.046 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.046 INFO:teuthology.orchestra.run.smithi081.stdout: "objecter": { 2024-05-15T14:12:46.046 INFO:teuthology.orchestra.run.smithi081.stdout: "op_active": 0, 2024-05-15T14:12:46.047 INFO:teuthology.orchestra.run.smithi081.stdout: "op_laggy": 0, 2024-05-15T14:12:46.047 INFO:teuthology.orchestra.run.smithi081.stdout: "op_send": 3024, 2024-05-15T14:12:46.047 INFO:teuthology.orchestra.run.smithi081.stdout: "op_send_bytes": 21, 2024-05-15T14:12:46.047 INFO:teuthology.orchestra.run.smithi081.stdout: "op_resend": 3, 2024-05-15T14:12:46.047 INFO:teuthology.orchestra.run.smithi081.stdout: "op_reply": 3024, 2024-05-15T14:12:46.047 INFO:teuthology.orchestra.run.smithi081.stdout: "op_latency": { 2024-05-15T14:12:46.047 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 3024, 2024-05-15T14:12:46.047 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 1.796968304, 2024-05-15T14:12:46.047 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000594235 2024-05-15T14:12:46.048 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.048 INFO:teuthology.orchestra.run.smithi081.stdout: "op_inflight": 0, 2024-05-15T14:12:46.048 INFO:teuthology.orchestra.run.smithi081.stdout: "oplen_avg": { 2024-05-15T14:12:46.048 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 3024, 2024-05-15T14:12:46.048 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 3025 2024-05-15T14:12:46.048 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.048 INFO:teuthology.orchestra.run.smithi081.stdout: "op": 3024, 2024-05-15T14:12:46.048 INFO:teuthology.orchestra.run.smithi081.stdout: "op_r": 3024, 2024-05-15T14:12:46.049 INFO:teuthology.orchestra.run.smithi081.stdout: "op_w": 0, 2024-05-15T14:12:46.049 INFO:teuthology.orchestra.run.smithi081.stdout: "op_rmw": 0, 2024-05-15T14:12:46.049 INFO:teuthology.orchestra.run.smithi081.stdout: "op_pg": 0, 2024-05-15T14:12:46.049 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_stat": 1720, 2024-05-15T14:12:46.049 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_create": 0, 2024-05-15T14:12:46.049 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_read": 1302, 2024-05-15T14:12:46.049 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_write": 0, 2024-05-15T14:12:46.049 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_writefull": 0, 2024-05-15T14:12:46.049 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_writesame": 0, 2024-05-15T14:12:46.050 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_append": 0, 2024-05-15T14:12:46.050 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_zero": 0, 2024-05-15T14:12:46.050 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_truncate": 0, 2024-05-15T14:12:46.050 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_delete": 0, 2024-05-15T14:12:46.050 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_mapext": 0, 2024-05-15T14:12:46.050 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_sparse_read": 0, 2024-05-15T14:12:46.050 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_clonerange": 0, 2024-05-15T14:12:46.050 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_getxattr": 1, 2024-05-15T14:12:46.051 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_setxattr": 0, 2024-05-15T14:12:46.051 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_cmpxattr": 0, 2024-05-15T14:12:46.051 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_rmxattr": 0, 2024-05-15T14:12:46.051 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_resetxattrs": 0, 2024-05-15T14:12:46.051 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_call": 0, 2024-05-15T14:12:46.051 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_watch": 0, 2024-05-15T14:12:46.051 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_notify": 0, 2024-05-15T14:12:46.051 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_src_cmpxattr": 0, 2024-05-15T14:12:46.051 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_pgls": 0, 2024-05-15T14:12:46.052 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_pgls_filter": 0, 2024-05-15T14:12:46.052 INFO:teuthology.orchestra.run.smithi081.stdout: "osdop_other": 0, 2024-05-15T14:12:46.052 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_active": 0, 2024-05-15T14:12:46.052 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_send": 0, 2024-05-15T14:12:46.052 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_resend": 0, 2024-05-15T14:12:46.052 INFO:teuthology.orchestra.run.smithi081.stdout: "linger_ping": 0, 2024-05-15T14:12:46.052 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_active": 0, 2024-05-15T14:12:46.052 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_send": 0, 2024-05-15T14:12:46.052 INFO:teuthology.orchestra.run.smithi081.stdout: "poolop_resend": 0, 2024-05-15T14:12:46.053 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_active": 0, 2024-05-15T14:12:46.053 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_send": 0, 2024-05-15T14:12:46.053 INFO:teuthology.orchestra.run.smithi081.stdout: "poolstat_resend": 0, 2024-05-15T14:12:46.053 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_active": 0, 2024-05-15T14:12:46.053 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_send": 0, 2024-05-15T14:12:46.053 INFO:teuthology.orchestra.run.smithi081.stdout: "statfs_resend": 0, 2024-05-15T14:12:46.053 INFO:teuthology.orchestra.run.smithi081.stdout: "command_active": 0, 2024-05-15T14:12:46.053 INFO:teuthology.orchestra.run.smithi081.stdout: "command_send": 0, 2024-05-15T14:12:46.053 INFO:teuthology.orchestra.run.smithi081.stdout: "command_resend": 0, 2024-05-15T14:12:46.054 INFO:teuthology.orchestra.run.smithi081.stdout: "map_epoch": 98, 2024-05-15T14:12:46.054 INFO:teuthology.orchestra.run.smithi081.stdout: "map_full": 0, 2024-05-15T14:12:46.054 INFO:teuthology.orchestra.run.smithi081.stdout: "map_inc": 18, 2024-05-15T14:12:46.054 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_sessions": 9, 2024-05-15T14:12:46.054 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_session_open": 9, 2024-05-15T14:12:46.054 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_session_close": 0, 2024-05-15T14:12:46.054 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_laggy": 0, 2024-05-15T14:12:46.054 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_wr": 0, 2024-05-15T14:12:46.054 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_rd": 2, 2024-05-15T14:12:46.054 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_del": 0 2024-05-15T14:12:46.055 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.055 INFO:teuthology.orchestra.run.smithi081.stdout: "oft": { 2024-05-15T14:12:46.055 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_objs": 0, 2024-05-15T14:12:46.055 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_kv_pairs": 0, 2024-05-15T14:12:46.055 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_updates": 0, 2024-05-15T14:12:46.055 INFO:teuthology.orchestra.run.smithi081.stdout: "omap_total_removes": 0 2024-05-15T14:12:46.055 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.055 INFO:teuthology.orchestra.run.smithi081.stdout: "osd-slow-ops": { 2024-05-15T14:12:46.055 INFO:teuthology.orchestra.run.smithi081.stdout: "slow_ops_count": 0 2024-05-15T14:12:46.056 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.056 INFO:teuthology.orchestra.run.smithi081.stdout: "purge_queue": { 2024-05-15T14:12:46.056 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_ops": 0, 2024-05-15T14:12:46.056 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_ops_high_water": 0, 2024-05-15T14:12:46.056 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing": 0, 2024-05-15T14:12:46.056 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executing_high_water": 0, 2024-05-15T14:12:46.056 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executed_ops": 0, 2024-05-15T14:12:46.056 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_executed": 0, 2024-05-15T14:12:46.056 INFO:teuthology.orchestra.run.smithi081.stdout: "pq_item_in_journal": 0 2024-05-15T14:12:46.056 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.057 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-15T14:12:46.057 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-05-15T14:12:46.057 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2024-05-15T14:12:46.057 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-05-15T14:12:46.057 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 3341, 2024-05-15T14:12:46.057 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 96688233, 2024-05-15T14:12:46.057 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:46.057 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 3341, 2024-05-15T14:12:46.057 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-05-15T14:12:46.057 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-05-15T14:12:46.058 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 3341, 2024-05-15T14:12:46.058 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 96688233, 2024-05-15T14:12:46.058 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-05-15T14:12:46.058 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.058 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.058 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.058 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:46.058 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.058 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-objecter_bytes": { 2024-05-15T14:12:46.059 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-05-15T14:12:46.059 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 104857600, 2024-05-15T14:12:46.059 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-05-15T14:12:46.059 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-05-15T14:12:46.059 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-05-15T14:12:46.059 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:46.059 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-05-15T14:12:46.059 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 3024, 2024-05-15T14:12:46.059 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 94326727, 2024-05-15T14:12:46.060 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 3024, 2024-05-15T14:12:46.060 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 94326727, 2024-05-15T14:12:46.060 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-05-15T14:12:46.060 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.060 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.060 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.060 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:46.060 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.060 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-objecter_ops": { 2024-05-15T14:12:46.060 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-05-15T14:12:46.061 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 1024, 2024-05-15T14:12:46.061 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-05-15T14:12:46.061 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-05-15T14:12:46.061 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-05-15T14:12:46.061 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:46.061 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-05-15T14:12:46.061 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 3024, 2024-05-15T14:12:46.061 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 3024, 2024-05-15T14:12:46.061 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 3024, 2024-05-15T14:12:46.061 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 3024, 2024-05-15T14:12:46.062 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-05-15T14:12:46.062 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.062 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.062 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.062 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:46.062 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.062 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-write_buf_throttle": { 2024-05-15T14:12:46.062 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-05-15T14:12:46.063 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 3758096384, 2024-05-15T14:12:46.063 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-05-15T14:12:46.063 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-05-15T14:12:46.063 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-05-15T14:12:46.063 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:46.063 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-05-15T14:12:46.063 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-05-15T14:12:46.063 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-05-15T14:12:46.063 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 0, 2024-05-15T14:12:46.064 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 0, 2024-05-15T14:12:46.064 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-05-15T14:12:46.064 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.064 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.064 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.064 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:46.064 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2024-05-15T14:12:46.064 INFO:teuthology.orchestra.run.smithi081.stdout: "throttle-write_buf_throttle-0x55fd3ea406e0": { 2024-05-15T14:12:46.065 INFO:teuthology.orchestra.run.smithi081.stdout: "val": 0, 2024-05-15T14:12:46.065 INFO:teuthology.orchestra.run.smithi081.stdout: "max": 3758096384, 2024-05-15T14:12:46.065 INFO:teuthology.orchestra.run.smithi081.stdout: "get_started": 0, 2024-05-15T14:12:46.065 INFO:teuthology.orchestra.run.smithi081.stdout: "get": 0, 2024-05-15T14:12:46.065 INFO:teuthology.orchestra.run.smithi081.stdout: "get_sum": 0, 2024-05-15T14:12:46.065 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_fail": 0, 2024-05-15T14:12:46.065 INFO:teuthology.orchestra.run.smithi081.stdout: "get_or_fail_success": 0, 2024-05-15T14:12:46.065 INFO:teuthology.orchestra.run.smithi081.stdout: "take": 0, 2024-05-15T14:12:46.065 INFO:teuthology.orchestra.run.smithi081.stdout: "take_sum": 0, 2024-05-15T14:12:46.066 INFO:teuthology.orchestra.run.smithi081.stdout: "put": 0, 2024-05-15T14:12:46.066 INFO:teuthology.orchestra.run.smithi081.stdout: "put_sum": 0, 2024-05-15T14:12:46.066 INFO:teuthology.orchestra.run.smithi081.stdout: "wait": { 2024-05-15T14:12:46.066 INFO:teuthology.orchestra.run.smithi081.stdout: "avgcount": 0, 2024-05-15T14:12:46.066 INFO:teuthology.orchestra.run.smithi081.stdout: "sum": 0.000000000, 2024-05-15T14:12:46.066 INFO:teuthology.orchestra.run.smithi081.stdout: "avgtime": 0.000000000 2024-05-15T14:12:46.066 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:46.066 INFO:teuthology.orchestra.run.smithi081.stdout: } 2024-05-15T14:12:46.066 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-05-15T14:12:46.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:46 smithi052 ceph-mon[36190]: pgmap v390: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-05-15T14:12:46.853 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:46 smithi081 ceph-mon[36434]: pgmap v390: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-05-15T14:12:46.882 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-05-15T14:12:46.882 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-05-15T14:12:46.882 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-05-15T14:12:46.882 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-05-15T14:12:46.883 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-05-15T14:12:46.883 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-05-15T14:12:46.883 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-05-15T14:12:46.883 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-15T14:12:46.894 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-15T14:12:46.894 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-15T14:12:46.894 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-15T14:12:46.916 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-15T14:12:46.916 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-15T14:12:46.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:46 smithi033 ceph-mon[31650]: pgmap v390: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-05-15T14:12:46.961 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph osd blocklist ls 2024-05-15T14:12:47.258 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:12:48.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:48 smithi033 ceph-mon[31650]: pgmap v391: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-05-15T14:12:48.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:48 smithi052 ceph-mon[36190]: pgmap v391: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-05-15T14:12:48.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:48 smithi081 ceph-mon[36434]: pgmap v391: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-05-15T14:12:48.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.903+0000 7f826db06700 1 -- 172.21.15.33:0/3104372390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82681084b0 msgr2=0x7f8268108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:48.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.903+0000 7f826db06700 1 --2- 172.21.15.33:0/3104372390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82681084b0 0x7f8268108890 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8258009990 tx=0x7f8258009ca0 comp rx=0 tx=0).stop 2024-05-15T14:12:48.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.904+0000 7f826db06700 1 -- 172.21.15.33:0/3104372390 shutdown_connections 2024-05-15T14:12:48.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.904+0000 7f826db06700 1 --2- 172.21.15.33:0/3104372390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8268102e30 0x7f826810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:48.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.904+0000 7f826db06700 1 --2- 172.21.15.33:0/3104372390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82681084b0 0x7f8268108890 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:48.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.904+0000 7f826db06700 1 --2- 172.21.15.33:0/3104372390 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8268102470 0x7f82681028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:48.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.904+0000 7f826db06700 1 -- 172.21.15.33:0/3104372390 >> 172.21.15.33:0/3104372390 conn(0x7f82680793f0 msgr2=0x7f8268079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:48.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.904+0000 7f826db06700 1 -- 172.21.15.33:0/3104372390 shutdown_connections 2024-05-15T14:12:48.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.904+0000 7f826db06700 1 -- 172.21.15.33:0/3104372390 wait complete. 2024-05-15T14:12:48.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.904+0000 7f826db06700 1 Processor -- start 2024-05-15T14:12:48.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.904+0000 7f826db06700 1 -- start start 2024-05-15T14:12:48.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.905+0000 7f826db06700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8268102470 0x7f8268104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:48.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.905+0000 7f826db06700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8268102e30 0x7f82681045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:48.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.905+0000 7f826db06700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82681084b0 0x7f8268104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:48.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.905+0000 7f826db06700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f82681125e0 con 0x7f8268102e30 2024-05-15T14:12:48.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.905+0000 7f826db06700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8268112460 con 0x7f82681084b0 2024-05-15T14:12:48.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.905+0000 7f826db06700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f8268112760 con 0x7f8268102470 2024-05-15T14:12:48.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.905+0000 7f8266ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8268102e30 0x7f82681045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:48.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.905+0000 7f8266ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8268102e30 0x7f82681045c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46480/0 (socket says 172.21.15.33:46480) 2024-05-15T14:12:48.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.905+0000 7f8266ffd700 1 -- 172.21.15.33:0/336953656 learned_addr learned my addr 172.21.15.33:0/336953656 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:12:48.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.905+0000 7f82677fe700 1 --2- 172.21.15.33:0/336953656 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8268102470 0x7f8268104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:48.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.905+0000 7f8267fff700 1 --2- 172.21.15.33:0/336953656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82681084b0 0x7f8268104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:48.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.905+0000 7f8266ffd700 1 -- 172.21.15.33:0/336953656 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8268102470 msgr2=0x7f8268104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:48.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.905+0000 7f8266ffd700 1 --2- 172.21.15.33:0/336953656 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8268102470 0x7f8268104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:48.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.905+0000 7f8266ffd700 1 -- 172.21.15.33:0/336953656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82681084b0 msgr2=0x7f8268104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:48.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.905+0000 7f8266ffd700 1 --2- 172.21.15.33:0/336953656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82681084b0 0x7f8268104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:48.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.905+0000 7f8266ffd700 1 -- 172.21.15.33:0/336953656 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82681a8140 con 0x7f8268102e30 2024-05-15T14:12:48.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.905+0000 7f8266ffd700 1 --2- 172.21.15.33:0/336953656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8268102e30 0x7f82681045c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f825c00d980 tx=0x7f825c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:48.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.906+0000 7f8264ff9700 1 -- 172.21.15.33:0/336953656 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c017070 con 0x7f8268102e30 2024-05-15T14:12:48.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.906+0000 7f8264ff9700 1 -- 172.21.15.33:0/336953656 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f825c005020 con 0x7f8268102e30 2024-05-15T14:12:48.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.906+0000 7f826db06700 1 -- 172.21.15.33:0/336953656 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82681a8430 con 0x7f8268102e30 2024-05-15T14:12:48.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.906+0000 7f8264ff9700 1 -- 172.21.15.33:0/336953656 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c009b50 con 0x7f8268102e30 2024-05-15T14:12:48.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.906+0000 7f826db06700 1 -- 172.21.15.33:0/336953656 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82681a8920 con 0x7f8268102e30 2024-05-15T14:12:48.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.907+0000 7f826db06700 1 -- 172.21.15.33:0/336953656 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f826806a230 con 0x7f8268102e30 2024-05-15T14:12:48.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.908+0000 7f8264ff9700 1 -- 172.21.15.33:0/336953656 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f825c0051d0 con 0x7f8268102e30 2024-05-15T14:12:48.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.909+0000 7f8264ff9700 1 --2- 172.21.15.33:0/336953656 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f82500b29f0 0x7f82500b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:48.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.909+0000 7f82677fe700 1 --2- 172.21.15.33:0/336953656 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f82500b29f0 0x7f82500b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:48.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.909+0000 7f8264ff9700 1 -- 172.21.15.33:0/336953656 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8542+0+0 (secure 0 0 0) 0x7f825c0e29b0 con 0x7f8268102e30 2024-05-15T14:12:48.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.909+0000 7f82677fe700 1 --2- 172.21.15.33:0/336953656 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f82500b29f0 0x7f82500b4eb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f825800b5c0 tx=0x7f8258005960 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:48.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:48.912+0000 7f8264ff9700 1 -- 172.21.15.33:0/336953656 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f825c020020 con 0x7f8268102e30 2024-05-15T14:12:49.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:49.141+0000 7f826db06700 1 -- 172.21.15.33:0/336953656 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f826802e560 con 0x7f8268102e30 2024-05-15T14:12:49.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:49.141+0000 7f8264ff9700 1 -- 172.21.15.33:0/336953656 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v98) ==== 81+0+945 (secure 0 0 0) 0x7f825c0aa380 con 0x7f8268102e30 2024-05-15T14:12:49.144 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1564238110 2024-05-16T14:00:21.157290+0000 2024-05-15T14:12:49.144 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/1797827152 2024-05-16T14:00:21.157290+0000 2024-05-15T14:12:49.144 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/436121571 2024-05-16T13:50:04.881656+0000 2024-05-15T14:12:49.144 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2647407310 2024-05-16T14:00:21.157290+0000 2024-05-15T14:12:49.144 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2326878092 2024-05-16T14:00:21.157290+0000 2024-05-15T14:12:49.144 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/2680375465 2024-05-16T13:50:04.881656+0000 2024-05-15T14:12:49.144 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2259371407 2024-05-16T14:00:21.157290+0000 2024-05-15T14:12:49.144 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/2680375465 2024-05-16T13:50:04.881656+0000 2024-05-15T14:12:49.145 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1022787179 2024-05-16T13:49:37.564643+0000 2024-05-15T14:12:49.145 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3720711369 2024-05-16T13:50:04.881656+0000 2024-05-15T14:12:49.145 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1282466838 2024-05-16T13:49:37.564643+0000 2024-05-15T14:12:49.145 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3022248969 2024-05-16T13:49:37.564643+0000 2024-05-15T14:12:49.145 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/3303495364 2024-05-16T13:49:37.564643+0000 2024-05-15T14:12:49.145 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/1797827152 2024-05-16T14:00:21.157290+0000 2024-05-15T14:12:49.145 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/3303495364 2024-05-16T13:49:37.564643+0000 2024-05-15T14:12:49.145 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/4282812692 2024-05-16T13:50:04.881656+0000 2024-05-15T14:12:49.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:49.143+0000 7f826db06700 1 -- 172.21.15.33:0/336953656 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f82500b29f0 msgr2=0x7f82500b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:49.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:49.143+0000 7f826db06700 1 --2- 172.21.15.33:0/336953656 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f82500b29f0 0x7f82500b4eb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f825800b5c0 tx=0x7f8258005960 comp rx=0 tx=0).stop 2024-05-15T14:12:49.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:49.143+0000 7f826db06700 1 -- 172.21.15.33:0/336953656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8268102e30 msgr2=0x7f82681045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:49.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:49.143+0000 7f826db06700 1 --2- 172.21.15.33:0/336953656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8268102e30 0x7f82681045c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f825c00d980 tx=0x7f825c00dc90 comp rx=0 tx=0).stop 2024-05-15T14:12:49.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:49.144+0000 7f826db06700 1 -- 172.21.15.33:0/336953656 shutdown_connections 2024-05-15T14:12:49.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:49.144+0000 7f826db06700 1 --2- 172.21.15.33:0/336953656 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f82500b29f0 0x7f82500b4eb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:49.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:49.144+0000 7f826db06700 1 --2- 172.21.15.33:0/336953656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f82681084b0 0x7f8268104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:49.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:49.144+0000 7f826db06700 1 --2- 172.21.15.33:0/336953656 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8268102470 0x7f8268104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:49.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:49.144+0000 7f826db06700 1 --2- 172.21.15.33:0/336953656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8268102e30 0x7f82681045c0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:49.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:49.144+0000 7f826db06700 1 -- 172.21.15.33:0/336953656 >> 172.21.15.33:0/336953656 conn(0x7f82680793f0 msgr2=0x7f826806bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:49.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:49.144+0000 7f826db06700 1 -- 172.21.15.33:0/336953656 shutdown_connections 2024-05-15T14:12:49.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:49.144+0000 7f826db06700 1 -- 172.21.15.33:0/336953656 wait complete. 2024-05-15T14:12:49.148 INFO:teuthology.orchestra.run.smithi033.stderr:listed 16 entries 2024-05-15T14:12:49.607 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:49 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/336953656' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-15T14:12:49.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:49 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/336953656' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-15T14:12:49.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:49 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/336953656' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-15T14:12:49.980 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-15T14:12:49.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T14:12:49.980 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-15T14:12:50.070 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2024-05-15T14:12:50.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T14:12:50.071 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-15T14:12:50.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T14:12:50.130 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-05-15T14:12:50.167 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-15T14:12:50.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T14:12:50.167 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-15T14:12:50.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-15T14:12:50.253 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link delete ceph-brx 2024-05-15T14:12:50.318 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-15T14:12:50.329 INFO:tasks.cephadm:Teardown begin 2024-05-15T14:12:50.329 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-15T14:12:50.329 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T14:12:50.548 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:50 smithi033 ceph-mon[31650]: pgmap v392: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 8 op/s 2024-05-15T14:12:50.587 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:12:50.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:50 smithi052 ceph-mon[36190]: pgmap v392: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 8 op/s 2024-05-15T14:12:50.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:50 smithi081 ceph-mon[36434]: pgmap v392: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 8 op/s 2024-05-15T14:12:52.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.185+0000 7f0f2096c700 1 -- 172.21.15.33:0/2544547748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f1c1088f0 msgr2=0x7f0f1c110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:52.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.185+0000 7f0f2096c700 1 --2- 172.21.15.33:0/2544547748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f1c1088f0 0x7f0f1c110470 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f0f1400b0d0 tx=0x7f0f1400b3e0 comp rx=0 tx=0).stop 2024-05-15T14:12:52.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.185+0000 7f0f2096c700 1 -- 172.21.15.33:0/2544547748 shutdown_connections 2024-05-15T14:12:52.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.185+0000 7f0f2096c700 1 --2- 172.21.15.33:0/2544547748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f1c1088f0 0x7f0f1c110470 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:52.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.185+0000 7f0f2096c700 1 --2- 172.21.15.33:0/2544547748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f1c1037f0 0x7f0f1c103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:52.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.185+0000 7f0f2096c700 1 --2- 172.21.15.33:0/2544547748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f1c1041a0 0x7f0f1c1081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:52.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.185+0000 7f0f2096c700 1 -- 172.21.15.33:0/2544547748 >> 172.21.15.33:0/2544547748 conn(0x7f0f1c07a6a0 msgr2=0x7f0f1c07aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:52.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.185+0000 7f0f2096c700 1 -- 172.21.15.33:0/2544547748 shutdown_connections 2024-05-15T14:12:52.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.185+0000 7f0f2096c700 1 -- 172.21.15.33:0/2544547748 wait complete. 2024-05-15T14:12:52.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.186+0000 7f0f2096c700 1 Processor -- start 2024-05-15T14:12:52.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.186+0000 7f0f2096c700 1 -- start start 2024-05-15T14:12:52.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.186+0000 7f0f2096c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f1c1037f0 0x7f0f1c19bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:52.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.186+0000 7f0f2096c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f1c1041a0 0x7f0f1c19c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:52.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.186+0000 7f0f2096c700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f1c1088f0 0x7f0f1c1a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:52.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.186+0000 7f0f2096c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1c113730 con 0x7f0f1c1037f0 2024-05-15T14:12:52.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.186+0000 7f0f2096c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1c1135b0 con 0x7f0f1c1041a0 2024-05-15T14:12:52.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.186+0000 7f0f2096c700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1c1138b0 con 0x7f0f1c1088f0 2024-05-15T14:12:52.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.186+0000 7f0f1a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f1c1037f0 0x7f0f1c19bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:52.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.186+0000 7f0f1a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f1c1037f0 0x7f0f1c19bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46502/0 (socket says 172.21.15.33:46502) 2024-05-15T14:12:52.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.186+0000 7f0f1a59c700 1 -- 172.21.15.33:0/2984383163 learned_addr learned my addr 172.21.15.33:0/2984383163 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:12:52.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.186+0000 7f0f1ad9d700 1 --2- 172.21.15.33:0/2984383163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f1c1088f0 0x7f0f1c1a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:52.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.186+0000 7f0f19d9b700 1 --2- 172.21.15.33:0/2984383163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f1c1041a0 0x7f0f1c19c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:52.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.187+0000 7f0f1a59c700 1 -- 172.21.15.33:0/2984383163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f1c1088f0 msgr2=0x7f0f1c1a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:52.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.187+0000 7f0f1a59c700 1 --2- 172.21.15.33:0/2984383163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f1c1088f0 0x7f0f1c1a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:52.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.187+0000 7f0f1a59c700 1 -- 172.21.15.33:0/2984383163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f1c1041a0 msgr2=0x7f0f1c19c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:52.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.187+0000 7f0f1a59c700 1 --2- 172.21.15.33:0/2984383163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f1c1041a0 0x7f0f1c19c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:52.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.187+0000 7f0f1a59c700 1 -- 172.21.15.33:0/2984383163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f1c1a0d40 con 0x7f0f1c1037f0 2024-05-15T14:12:52.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.187+0000 7f0f19d9b700 1 --2- 172.21.15.33:0/2984383163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f1c1041a0 0x7f0f1c19c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:12:52.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.187+0000 7f0f1a59c700 1 --2- 172.21.15.33:0/2984383163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f1c1037f0 0x7f0f1c19bd30 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c00c920 tx=0x7f0f0c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:52.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.187+0000 7f0f0b7fe700 1 -- 172.21.15.33:0/2984383163 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c007b00 con 0x7f0f1c1037f0 2024-05-15T14:12:52.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.187+0000 7f0f0b7fe700 1 -- 172.21.15.33:0/2984383163 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f0c004ce0 con 0x7f0f1c1037f0 2024-05-15T14:12:52.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.187+0000 7f0f2096c700 1 -- 172.21.15.33:0/2984383163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f1c1a1030 con 0x7f0f1c1037f0 2024-05-15T14:12:52.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.187+0000 7f0f0b7fe700 1 -- 172.21.15.33:0/2984383163 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c005550 con 0x7f0f1c1037f0 2024-05-15T14:12:52.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.187+0000 7f0f2096c700 1 -- 172.21.15.33:0/2984383163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f1c1a88b0 con 0x7f0f1c1037f0 2024-05-15T14:12:52.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.188+0000 7f0f2096c700 1 -- 172.21.15.33:0/2984383163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f1c06a230 con 0x7f0f1c1037f0 2024-05-15T14:12:52.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.190+0000 7f0f0b7fe700 1 -- 172.21.15.33:0/2984383163 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0f0c007ca0 con 0x7f0f1c1037f0 2024-05-15T14:12:52.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.191+0000 7f0f0b7fe700 1 --2- 172.21.15.33:0/2984383163 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0f040b29f0 0x7f0f040b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:52.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.191+0000 7f0f19d9b700 1 --2- 172.21.15.33:0/2984383163 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0f040b29f0 0x7f0f040b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:52.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.191+0000 7f0f0b7fe700 1 -- 172.21.15.33:0/2984383163 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8542+0+0 (secure 0 0 0) 0x7f0f0c0e2650 con 0x7f0f1c1037f0 2024-05-15T14:12:52.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.191+0000 7f0f19d9b700 1 --2- 172.21.15.33:0/2984383163 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0f040b29f0 0x7f0f040b4eb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c103440 tx=0x7f0f10008040 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:52.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.194+0000 7f0f0b7fe700 1 -- 172.21.15.33:0/2984383163 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0f0c014030 con 0x7f0f1c1037f0 2024-05-15T14:12:52.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.465+0000 7f0f2096c700 1 -- 172.21.15.33:0/2984383163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f1c19d090 con 0x7f0f1c1037f0 2024-05-15T14:12:52.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.466+0000 7f0f0b7fe700 1 -- 172.21.15.33:0/2984383163 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11862 (secure 0 0 0) 0x7f0f1c19d090 con 0x7f0f1c1037f0 2024-05-15T14:12:52.468 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T14:12:52.468 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2030117633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2030117633},{"type":"v1","addr":"172.21.15.33:6839","nonce":2030117633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1918979061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1918979061},{"type":"v1","addr":"172.21.15.33:6841","nonce":1918979061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T13:57:19.056417+0000","modified":"2024-05-15T14:12:43.692091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24463,"mds_2":24461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/1629697841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1629697841},{"type":"v1","addr":"172.21.15.52:6837","nonce":1629697841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.52:6839/2696044471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2696044471},{"type":"v1","addr":"172.21.15.52:6839","nonce":2696044471}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.81:6839/1554395940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":1554395940},{"type":"v1","addr":"172.21.15.81:6839","nonce":1554395940}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26206":{"gid":26206,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/3674797952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3674797952},{"type":"v1","addr":"172.21.15.81:6835","nonce":3674797952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24463,24478,24461]},"id":1}]} 2024-05-15T14:12:52.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.468+0000 7f0f2096c700 1 -- 172.21.15.33:0/2984383163 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0f040b29f0 msgr2=0x7f0f040b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:52.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.468+0000 7f0f2096c700 1 --2- 172.21.15.33:0/2984383163 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0f040b29f0 0x7f0f040b4eb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c103440 tx=0x7f0f10008040 comp rx=0 tx=0).stop 2024-05-15T14:12:52.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.468+0000 7f0f2096c700 1 -- 172.21.15.33:0/2984383163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f1c1037f0 msgr2=0x7f0f1c19bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:52.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.468+0000 7f0f2096c700 1 --2- 172.21.15.33:0/2984383163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f1c1037f0 0x7f0f1c19bd30 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c00c920 tx=0x7f0f0c00cc30 comp rx=0 tx=0).stop 2024-05-15T14:12:52.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.468+0000 7f0f2096c700 1 -- 172.21.15.33:0/2984383163 shutdown_connections 2024-05-15T14:12:52.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.468+0000 7f0f2096c700 1 --2- 172.21.15.33:0/2984383163 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f0f040b29f0 0x7f0f040b4eb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:52.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.468+0000 7f0f2096c700 1 --2- 172.21.15.33:0/2984383163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0f1c1088f0 0x7f0f1c1a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:52.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.468+0000 7f0f2096c700 1 --2- 172.21.15.33:0/2984383163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f1c1037f0 0x7f0f1c19bd30 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:52.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.468+0000 7f0f2096c700 1 --2- 172.21.15.33:0/2984383163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0f1c1041a0 0x7f0f1c19c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:52.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.468+0000 7f0f2096c700 1 -- 172.21.15.33:0/2984383163 >> 172.21.15.33:0/2984383163 conn(0x7f0f1c07a6a0 msgr2=0x7f0f1c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:52.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.469+0000 7f0f2096c700 1 -- 172.21.15.33:0/2984383163 shutdown_connections 2024-05-15T14:12:52.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:52.469+0000 7f0f2096c700 1 -- 172.21.15.33:0/2984383163 wait complete. 2024-05-15T14:12:52.472 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 42 2024-05-15T14:12:52.759 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:52 smithi033 ceph-mon[31650]: pgmap v393: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 8 op/s 2024-05-15T14:12:52.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:52 smithi052 ceph-mon[36190]: pgmap v393: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 8 op/s 2024-05-15T14:12:52.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:52 smithi081 ceph-mon[36434]: pgmap v393: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 8 op/s 2024-05-15T14:12:53.069 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-15T14:12:53.326 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:12:53.798 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:53 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/2984383163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:12:53.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:53 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/2984383163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:12:53.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:53 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/2984383163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:12:54.605 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:54 smithi033 ceph-mon[31650]: pgmap v394: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-05-15T14:12:54.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:54 smithi052 ceph-mon[36190]: pgmap v394: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-05-15T14:12:54.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.844+0000 7fe5ed9e7700 1 -- 172.21.15.33:0/3530785204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5e81095b0 msgr2=0x7fe5e8109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:54.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.844+0000 7fe5ed9e7700 1 --2- 172.21.15.33:0/3530785204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5e81095b0 0x7fe5e8109990 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80098f0 tx=0x7fe5d8009c00 comp rx=0 tx=0).stop 2024-05-15T14:12:54.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.844+0000 7fe5ed9e7700 1 -- 172.21.15.33:0/3530785204 shutdown_connections 2024-05-15T14:12:54.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.844+0000 7fe5ed9e7700 1 --2- 172.21.15.33:0/3530785204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe5e806bbe0 0x7fe5e81103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:54.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.844+0000 7fe5ed9e7700 1 --2- 172.21.15.33:0/3530785204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5e81095b0 0x7fe5e8109990 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:54.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.844+0000 7fe5ed9e7700 1 --2- 172.21.15.33:0/3530785204 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5e806b220 0x7fe5e806b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:54.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.844+0000 7fe5ed9e7700 1 -- 172.21.15.33:0/3530785204 >> 172.21.15.33:0/3530785204 conn(0x7fe5e80793f0 msgr2=0x7fe5e8079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:54.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.844+0000 7fe5ed9e7700 1 -- 172.21.15.33:0/3530785204 shutdown_connections 2024-05-15T14:12:54.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.844+0000 7fe5ed9e7700 1 -- 172.21.15.33:0/3530785204 wait complete. 2024-05-15T14:12:54.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.844+0000 7fe5ed9e7700 1 Processor -- start 2024-05-15T14:12:54.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.845+0000 7fe5ed9e7700 1 -- start start 2024-05-15T14:12:54.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.845+0000 7fe5ed9e7700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5e806b220 0x7fe5e8102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:54.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.845+0000 7fe5ed9e7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe5e806bbe0 0x7fe5e8102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:54.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.845+0000 7fe5ed9e7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5e81095b0 0x7fe5e8102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:54.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.845+0000 7fe5ed9e7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e8113730 con 0x7fe5e81095b0 2024-05-15T14:12:54.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.845+0000 7fe5ed9e7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e81135b0 con 0x7fe5e806bbe0 2024-05-15T14:12:54.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.845+0000 7fe5ed9e7700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e81138b0 con 0x7fe5e806b220 2024-05-15T14:12:54.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.845+0000 7fe5e77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5e81095b0 0x7fe5e8102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:54.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.845+0000 7fe5e77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5e81095b0 0x7fe5e8102f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51862/0 (socket says 172.21.15.33:51862) 2024-05-15T14:12:54.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.845+0000 7fe5e77fe700 1 -- 172.21.15.33:0/1390398674 learned_addr learned my addr 172.21.15.33:0/1390398674 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:12:54.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.845+0000 7fe5e67fc700 1 --2- 172.21.15.33:0/1390398674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe5e806bbe0 0x7fe5e8102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:54.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.845+0000 7fe5e6ffd700 1 --2- 172.21.15.33:0/1390398674 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5e806b220 0x7fe5e8102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:54.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.845+0000 7fe5e77fe700 1 -- 172.21.15.33:0/1390398674 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5e806b220 msgr2=0x7fe5e8102510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:54.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.845+0000 7fe5e77fe700 1 --2- 172.21.15.33:0/1390398674 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5e806b220 0x7fe5e8102510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:54.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.845+0000 7fe5e77fe700 1 -- 172.21.15.33:0/1390398674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe5e806bbe0 msgr2=0x7fe5e8102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:54.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.845+0000 7fe5e77fe700 1 --2- 172.21.15.33:0/1390398674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe5e806bbe0 0x7fe5e8102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:54.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.845+0000 7fe5e77fe700 1 -- 172.21.15.33:0/1390398674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e81a8230 con 0x7fe5e81095b0 2024-05-15T14:12:54.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.845+0000 7fe5e67fc700 1 --2- 172.21.15.33:0/1390398674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe5e806bbe0 0x7fe5e8102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-15T14:12:54.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.846+0000 7fe5e77fe700 1 --2- 172.21.15.33:0/1390398674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5e81095b0 0x7fe5e8102f90 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe5e000e3c0 tx=0x7fe5e000e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:54.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.846+0000 7fe5ec9e5700 1 -- 172.21.15.33:0/1390398674 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e00090d0 con 0x7fe5e81095b0 2024-05-15T14:12:54.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.846+0000 7fe5ed9e7700 1 -- 172.21.15.33:0/1390398674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5e81a8520 con 0x7fe5e81095b0 2024-05-15T14:12:54.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.846+0000 7fe5ec9e5700 1 -- 172.21.15.33:0/1390398674 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5e0009270 con 0x7fe5e81095b0 2024-05-15T14:12:54.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.846+0000 7fe5ec9e5700 1 -- 172.21.15.33:0/1390398674 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0007df0 con 0x7fe5e81095b0 2024-05-15T14:12:54.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.846+0000 7fe5ed9e7700 1 -- 172.21.15.33:0/1390398674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5e81a8a60 con 0x7fe5e81095b0 2024-05-15T14:12:54.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.847+0000 7fe5ed9e7700 1 -- 172.21.15.33:0/1390398674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5e810ba90 con 0x7fe5e81095b0 2024-05-15T14:12:54.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.848+0000 7fe5ec9e5700 1 -- 172.21.15.33:0/1390398674 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe5e0012050 con 0x7fe5e81095b0 2024-05-15T14:12:54.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.849+0000 7fe5ec9e5700 1 --2- 172.21.15.33:0/1390398674 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fe5d00b29f0 0x7fe5d00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:54.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.849+0000 7fe5e6ffd700 1 --2- 172.21.15.33:0/1390398674 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fe5d00b29f0 0x7fe5d00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:54.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.849+0000 7fe5ec9e5700 1 -- 172.21.15.33:0/1390398674 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8542+0+0 (secure 0 0 0) 0x7fe5e00e2050 con 0x7fe5e81095b0 2024-05-15T14:12:54.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.850+0000 7fe5e6ffd700 1 --2- 172.21.15.33:0/1390398674 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fe5d00b29f0 0x7fe5d00b4eb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80098c0 tx=0x7fe5d8019040 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:54.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:54.853+0000 7fe5ec9e5700 1 -- 172.21.15.33:0/1390398674 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe5e0022060 con 0x7fe5e81095b0 2024-05-15T14:12:54.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:54 smithi081 ceph-mon[36434]: pgmap v394: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-05-15T14:12:55.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:55.119+0000 7fe5ed9e7700 1 -- 172.21.15.33:0/1390398674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe5e810b470 con 0x7fe5e81095b0 2024-05-15T14:12:55.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:55.119+0000 7fe5ec9e5700 1 -- 172.21.15.33:0/1390398674 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11862 (secure 0 0 0) 0x7fe5e0010040 con 0x7fe5e81095b0 2024-05-15T14:12:55.122 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T14:12:55.122 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/3975341851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":3975341851},{"type":"v1","addr":"172.21.15.33:6835","nonce":3975341851}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3558077615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3558077615},{"type":"v1","addr":"172.21.15.33:6837","nonce":3558077615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2030117633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2030117633},{"type":"v1","addr":"172.21.15.33:6839","nonce":2030117633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1918979061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1918979061},{"type":"v1","addr":"172.21.15.33:6841","nonce":1918979061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6833/2083190067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":2083190067},{"type":"v1","addr":"172.21.15.52:6833","nonce":2083190067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3191964924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3191964924},{"type":"v1","addr":"172.21.15.52:6835","nonce":3191964924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-15T13:57:19.056417+0000","modified":"2024-05-15T14:12:43.692091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24463,"mds_2":24461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24424":{"gid":24424,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6833/1872861431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":1872861431},{"type":"v1","addr":"172.21.15.81:6833","nonce":1872861431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6837/1629697841","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":1629697841},{"type":"v1","addr":"172.21.15.52:6837","nonce":1629697841}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"b","rank":2,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.52:6839/2696044471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2696044471},{"type":"v1","addr":"172.21.15.52:6839","nonce":2696044471}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.81:6837/744100239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":744100239},{"type":"v1","addr":"172.21.15.81:6837","nonce":744100239}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.81:6839/1554395940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6838","nonce":1554395940},{"type":"v1","addr":"172.21.15.81:6839","nonce":1554395940}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26206":{"gid":26206,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.81:6835/3674797952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3674797952},{"type":"v1","addr":"172.21.15.81:6835","nonce":3674797952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24463,24478,24461]},"id":1}]} 2024-05-15T14:12:55.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:55.122+0000 7fe5ed9e7700 1 -- 172.21.15.33:0/1390398674 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fe5d00b29f0 msgr2=0x7fe5d00b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:55.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:55.122+0000 7fe5ed9e7700 1 --2- 172.21.15.33:0/1390398674 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fe5d00b29f0 0x7fe5d00b4eb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80098c0 tx=0x7fe5d8019040 comp rx=0 tx=0).stop 2024-05-15T14:12:55.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:55.122+0000 7fe5ed9e7700 1 -- 172.21.15.33:0/1390398674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5e81095b0 msgr2=0x7fe5e8102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:55.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:55.122+0000 7fe5ed9e7700 1 --2- 172.21.15.33:0/1390398674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5e81095b0 0x7fe5e8102f90 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe5e000e3c0 tx=0x7fe5e000e6d0 comp rx=0 tx=0).stop 2024-05-15T14:12:55.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:55.122+0000 7fe5ed9e7700 1 -- 172.21.15.33:0/1390398674 shutdown_connections 2024-05-15T14:12:55.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:55.122+0000 7fe5ed9e7700 1 --2- 172.21.15.33:0/1390398674 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fe5d00b29f0 0x7fe5d00b4eb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:55.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:55.122+0000 7fe5ed9e7700 1 --2- 172.21.15.33:0/1390398674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5e81095b0 0x7fe5e8102f90 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:55.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:55.122+0000 7fe5ed9e7700 1 --2- 172.21.15.33:0/1390398674 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5e806b220 0x7fe5e8102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:55.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:55.122+0000 7fe5ed9e7700 1 --2- 172.21.15.33:0/1390398674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe5e806bbe0 0x7fe5e8102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:55.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:55.122+0000 7fe5ed9e7700 1 -- 172.21.15.33:0/1390398674 >> 172.21.15.33:0/1390398674 conn(0x7fe5e80793f0 msgr2=0x7fe5e8101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:55.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:55.122+0000 7fe5ed9e7700 1 -- 172.21.15.33:0/1390398674 shutdown_connections 2024-05-15T14:12:55.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:55.122+0000 7fe5ed9e7700 1 -- 172.21.15.33:0/1390398674 wait complete. 2024-05-15T14:12:55.125 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 42 2024-05-15T14:12:55.827 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:55 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/1390398674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:12:55.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:55 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/1390398674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:12:55.857 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-15T14:12:55.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:55 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/1390398674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-15T14:12:56.111 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:12:56.538 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:56 smithi033 ceph-mon[31650]: pgmap v395: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 8 op/s 2024-05-15T14:12:56.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:56 smithi052 ceph-mon[36190]: pgmap v395: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 8 op/s 2024-05-15T14:12:56.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:56 smithi081 ceph-mon[36434]: pgmap v395: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 8 op/s 2024-05-15T14:12:57.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.703+0000 7f399b0c4700 1 -- 172.21.15.33:0/782969508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f399406bbe0 msgr2=0x7f39941103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:57.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.703+0000 7f399b0c4700 1 --2- 172.21.15.33:0/782969508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f399406bbe0 0x7f39941103d0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f398c009990 tx=0x7f398c009ca0 comp rx=0 tx=0).stop 2024-05-15T14:12:57.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.704+0000 7f399b0c4700 1 -- 172.21.15.33:0/782969508 shutdown_connections 2024-05-15T14:12:57.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.704+0000 7f399b0c4700 1 --2- 172.21.15.33:0/782969508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f399406bbe0 0x7f39941103d0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:57.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.704+0000 7f399b0c4700 1 --2- 172.21.15.33:0/782969508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f39941095b0 0x7f3994109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:57.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.704+0000 7f399b0c4700 1 --2- 172.21.15.33:0/782969508 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f399406b220 0x7f399406b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:57.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.704+0000 7f399b0c4700 1 -- 172.21.15.33:0/782969508 >> 172.21.15.33:0/782969508 conn(0x7f39940793f0 msgr2=0x7f3994079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:57.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.704+0000 7f399b0c4700 1 -- 172.21.15.33:0/782969508 shutdown_connections 2024-05-15T14:12:57.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.704+0000 7f399b0c4700 1 -- 172.21.15.33:0/782969508 wait complete. 2024-05-15T14:12:57.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.704+0000 7f399b0c4700 1 Processor -- start 2024-05-15T14:12:57.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.704+0000 7f399b0c4700 1 -- start start 2024-05-15T14:12:57.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.705+0000 7f399b0c4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f399406b220 0x7f3994102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:57.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.705+0000 7f399b0c4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f399406bbe0 0x7f3994102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:57.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.705+0000 7f399b0c4700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39941095b0 0x7f3994102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:57.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.705+0000 7f399b0c4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f39941136e0 con 0x7f399406b220 2024-05-15T14:12:57.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.705+0000 7f399b0c4700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3994113560 con 0x7f399406bbe0 2024-05-15T14:12:57.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.705+0000 7f399b0c4700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3994113860 con 0x7f39941095b0 2024-05-15T14:12:57.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.705+0000 7f3998e60700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f399406b220 0x7f3994102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:57.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.705+0000 7f3998e60700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f399406b220 0x7f3994102510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51880/0 (socket says 172.21.15.33:51880) 2024-05-15T14:12:57.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.705+0000 7f3998e60700 1 -- 172.21.15.33:0/3876580423 learned_addr learned my addr 172.21.15.33:0/3876580423 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:12:57.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.705+0000 7f3993fff700 1 --2- 172.21.15.33:0/3876580423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f399406bbe0 0x7f3994102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:57.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.705+0000 7f3999661700 1 --2- 172.21.15.33:0/3876580423 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39941095b0 0x7f3994102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:57.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.705+0000 7f3998e60700 1 -- 172.21.15.33:0/3876580423 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39941095b0 msgr2=0x7f3994102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:57.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.705+0000 7f3998e60700 1 --2- 172.21.15.33:0/3876580423 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39941095b0 0x7f3994102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:57.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.705+0000 7f3998e60700 1 -- 172.21.15.33:0/3876580423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f399406bbe0 msgr2=0x7f3994102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:57.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.705+0000 7f3998e60700 1 --2- 172.21.15.33:0/3876580423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f399406bbe0 0x7f3994102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:57.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.705+0000 7f3998e60700 1 -- 172.21.15.33:0/3876580423 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39941a8160 con 0x7f399406b220 2024-05-15T14:12:57.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.705+0000 7f3993fff700 1 --2- 172.21.15.33:0/3876580423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f399406bbe0 0x7f3994102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:12:57.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.705+0000 7f3998e60700 1 --2- 172.21.15.33:0/3876580423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f399406b220 0x7f3994102510 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f397c00d980 tx=0x7f397c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:57.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.705+0000 7f3991ffb700 1 -- 172.21.15.33:0/3876580423 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c016070 con 0x7f399406b220 2024-05-15T14:12:57.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.705+0000 7f3991ffb700 1 -- 172.21.15.33:0/3876580423 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f397c0044d0 con 0x7f399406b220 2024-05-15T14:12:57.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.705+0000 7f399b0c4700 1 -- 172.21.15.33:0/3876580423 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39941a8450 con 0x7f399406b220 2024-05-15T14:12:57.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.705+0000 7f3991ffb700 1 -- 172.21.15.33:0/3876580423 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c004d00 con 0x7f399406b220 2024-05-15T14:12:57.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.706+0000 7f399b0c4700 1 -- 172.21.15.33:0/3876580423 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39941a8990 con 0x7f399406b220 2024-05-15T14:12:57.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.707+0000 7f399b0c4700 1 -- 172.21.15.33:0/3876580423 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f399406a230 con 0x7f399406b220 2024-05-15T14:12:57.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.708+0000 7f3991ffb700 1 -- 172.21.15.33:0/3876580423 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f397c004020 con 0x7f399406b220 2024-05-15T14:12:57.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.709+0000 7f3991ffb700 1 --2- 172.21.15.33:0/3876580423 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f39800b29f0 0x7f39800b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:12:57.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.709+0000 7f3993fff700 1 --2- 172.21.15.33:0/3876580423 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f39800b29f0 0x7f39800b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:12:57.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.709+0000 7f3991ffb700 1 -- 172.21.15.33:0/3876580423 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8542+0+0 (secure 0 0 0) 0x7f397c0e2500 con 0x7f399406b220 2024-05-15T14:12:57.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.709+0000 7f3993fff700 1 --2- 172.21.15.33:0/3876580423 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f39800b29f0 0x7f39800b4eb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3984007fd0 tx=0x7f3984007810 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:12:57.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.713+0000 7f3991ffb700 1 -- 172.21.15.33:0/3876580423 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f397c01f020 con 0x7f399406b220 2024-05-15T14:12:57.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.931+0000 7f399b0c4700 1 -- 172.21.15.33:0/3876580423 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f399402e560 con 0x7f399406b220 2024-05-15T14:12:57.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.932+0000 7f3991ffb700 1 -- 172.21.15.33:0/3876580423 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+23061 (secure 0 0 0) 0x7f397c0a9e70 con 0x7f399406b220 2024-05-15T14:12:57.935 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-15T14:12:57.935 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":98,"fsid":"bca78ca4-12c1-11ef-bc98-c7b262605968","created":"2024-05-15T13:49:10.327613+0000","modified":"2024-05-15T14:12:10.551430+0000","last_up_change":"2024-05-15T13:56:16.097711+0000","last_in_change":"2024-05-15T13:55:56.722143+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-15T13:52:32.970844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-15T13:57:12.324802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-15T13:57:15.898406+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":19,"snap_epoch":97,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1b434111-810f-450a-b63d-c97080e9f8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6803","nonce":887409130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6805","nonce":887409130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6809","nonce":887409130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":887409130},{"type":"v1","addr":"172.21.15.33:6807","nonce":887409130}]},"public_addr":"172.21.15.33:6803/887409130","cluster_addr":"172.21.15.33:6805/887409130","heartbeat_back_addr":"172.21.15.33:6809/887409130","heartbeat_front_addr":"172.21.15.33:6807/887409130","state":["exists","up"]},{"osd":1,"uuid":"0b578d4e-a841-4b27-bc95-d6c3bae580fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6801","nonce":2794180003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6803","nonce":2794180003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6807","nonce":2794180003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":2794180003},{"type":"v1","addr":"172.21.15.52:6805","nonce":2794180003}]},"public_addr":"172.21.15.52:6801/2794180003","cluster_addr":"172.21.15.52:6803/2794180003","heartbeat_back_addr":"172.21.15.52:6807/2794180003","heartbeat_front_addr":"172.21.15.52:6805/2794180003","state":["exists","up"]},{"osd":2,"uuid":"0e9c9390-ed33-464b-8f39-bca75941e05e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6801","nonce":315610978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6803","nonce":315610978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6807","nonce":315610978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":315610978},{"type":"v1","addr":"172.21.15.81:6805","nonce":315610978}]},"public_addr":"172.21.15.81:6801/315610978","cluster_addr":"172.21.15.81:6803/315610978","heartbeat_back_addr":"172.21.15.81:6807/315610978","heartbeat_front_addr":"172.21.15.81:6805/315610978","state":["exists","up"]},{"osd":3,"uuid":"ecdb24c1-fb15-42a5-9bae-436bbae9512a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6811","nonce":818533501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6813","nonce":818533501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6817","nonce":818533501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":818533501},{"type":"v1","addr":"172.21.15.33:6815","nonce":818533501}]},"public_addr":"172.21.15.33:6811/818533501","cluster_addr":"172.21.15.33:6813/818533501","heartbeat_back_addr":"172.21.15.33:6817/818533501","heartbeat_front_addr":"172.21.15.33:6815/818533501","state":["exists","up"]},{"osd":4,"uuid":"706752a0-b17f-4563-a7a9-d5078bf0f3b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6809","nonce":3010133776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6811","nonce":3010133776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6815","nonce":3010133776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":3010133776},{"type":"v1","addr":"172.21.15.52:6813","nonce":3010133776}]},"public_addr":"172.21.15.52:6809/3010133776","cluster_addr":"172.21.15.52:6811/3010133776","heartbeat_back_addr":"172.21.15.52:6815/3010133776","heartbeat_front_addr":"172.21.15.52:6813/3010133776","state":["exists","up"]},{"osd":5,"uuid":"3d45ab97-f364-4903-974d-8acc961363d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6809","nonce":2132616297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6811","nonce":2132616297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6815","nonce":2132616297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":2132616297},{"type":"v1","addr":"172.21.15.81:6813","nonce":2132616297}]},"public_addr":"172.21.15.81:6809/2132616297","cluster_addr":"172.21.15.81:6811/2132616297","heartbeat_back_addr":"172.21.15.81:6815/2132616297","heartbeat_front_addr":"172.21.15.81:6813/2132616297","state":["exists","up"]},{"osd":6,"uuid":"3be524c5-dcf9-4742-b896-ea9a44a6b391","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6819","nonce":1260691801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6821","nonce":1260691801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6825","nonce":1260691801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":1260691801},{"type":"v1","addr":"172.21.15.33:6823","nonce":1260691801}]},"public_addr":"172.21.15.33:6819/1260691801","cluster_addr":"172.21.15.33:6821/1260691801","heartbeat_back_addr":"172.21.15.33:6825/1260691801","heartbeat_front_addr":"172.21.15.33:6823/1260691801","state":["exists","up"]},{"osd":7,"uuid":"bda2f920-4f2c-4eed-8127-83c1973676e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6817","nonce":1654595813}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6819","nonce":1654595813}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6823","nonce":1654595813}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":1654595813},{"type":"v1","addr":"172.21.15.52:6821","nonce":1654595813}]},"public_addr":"172.21.15.52:6817/1654595813","cluster_addr":"172.21.15.52:6819/1654595813","heartbeat_back_addr":"172.21.15.52:6823/1654595813","heartbeat_front_addr":"172.21.15.52:6821/1654595813","state":["exists","up"]},{"osd":8,"uuid":"550585b8-0df4-49ee-95c8-3d084744dbd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6817","nonce":3009661748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6819","nonce":3009661748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6823","nonce":3009661748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":3009661748},{"type":"v1","addr":"172.21.15.81:6821","nonce":3009661748}]},"public_addr":"172.21.15.81:6817/3009661748","cluster_addr":"172.21.15.81:6819/3009661748","heartbeat_back_addr":"172.21.15.81:6823/3009661748","heartbeat_front_addr":"172.21.15.81:6821/3009661748","state":["exists","up"]},{"osd":9,"uuid":"1d781846-26f3-4b23-b22c-c994a8509a71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6827","nonce":2843220858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6829","nonce":2843220858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6833","nonce":2843220858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":2843220858},{"type":"v1","addr":"172.21.15.33:6831","nonce":2843220858}]},"public_addr":"172.21.15.33:6827/2843220858","cluster_addr":"172.21.15.33:6829/2843220858","heartbeat_back_addr":"172.21.15.33:6833/2843220858","heartbeat_front_addr":"172.21.15.33:6831/2843220858","state":["exists","up"]},{"osd":10,"uuid":"af58c940-fdc4-418b-a638-7dc44438b34c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6825","nonce":2422349831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6827","nonce":2422349831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6831","nonce":2422349831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":2422349831},{"type":"v1","addr":"172.21.15.52:6829","nonce":2422349831}]},"public_addr":"172.21.15.52:6825/2422349831","cluster_addr":"172.21.15.52:6827/2422349831","heartbeat_back_addr":"172.21.15.52:6831/2422349831","heartbeat_front_addr":"172.21.15.52:6829/2422349831","state":["exists","up"]},{"osd":11,"uuid":"ab84aaea-2086-4842-8bf4-628842fce46a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6825","nonce":326313090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6827","nonce":326313090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6831","nonce":326313090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":326313090},{"type":"v1","addr":"172.21.15.81:6829","nonce":326313090}]},"public_addr":"172.21.15.81:6825/326313090","cluster_addr":"172.21.15.81:6827/326313090","heartbeat_back_addr":"172.21.15.81:6831/326313090","heartbeat_front_addr":"172.21.15.81:6829/326313090","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:51:41.493136+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:52:05.784773+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:52:29.769429+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:52:54.891620+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:53:19.412922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:53:43.921841+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:54:09.486732+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:54:34.064370+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:54:58.655304+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:55:24.548693+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:55:49.941191+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-15T13:56:14.919558+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1564238110":"2024-05-16T14:00:21.157290+0000","172.21.15.33:6800/1797827152":"2024-05-16T14:00:21.157290+0000","172.21.15.33:0/436121571":"2024-05-16T13:50:04.881656+0000","172.21.15.33:0/2647407310":"2024-05-16T14:00:21.157290+0000","172.21.15.33:0/2326878092":"2024-05-16T14:00:21.157290+0000","172.21.15.33:6801/2680375465":"2024-05-16T13:50:04.881656+0000","172.21.15.33:0/2259371407":"2024-05-16T14:00:21.157290+0000","172.21.15.33:6800/2680375465":"2024-05-16T13:50:04.881656+0000","172.21.15.33:0/1022787179":"2024-05-16T13:49:37.564643+0000","172.21.15.33:0/3720711369":"2024-05-16T13:50:04.881656+0000","172.21.15.33:0/1282466838":"2024-05-16T13:49:37.564643+0000","172.21.15.33:0/3022248969":"2024-05-16T13:49:37.564643+0000","172.21.15.33:6801/3303495364":"2024-05-16T13:49:37.564643+0000","172.21.15.33:6801/1797827152":"2024-05-16T14:00:21.157290+0000","172.21.15.33:6800/3303495364":"2024-05-16T13:49:37.564643+0000","172.21.15.33:0/4282812692":"2024-05-16T13:50:04.881656+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":19,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-15T14:12:57.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.935+0000 7f399b0c4700 1 -- 172.21.15.33:0/3876580423 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f39800b29f0 msgr2=0x7f39800b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:57.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.935+0000 7f399b0c4700 1 --2- 172.21.15.33:0/3876580423 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f39800b29f0 0x7f39800b4eb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3984007fd0 tx=0x7f3984007810 comp rx=0 tx=0).stop 2024-05-15T14:12:57.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.935+0000 7f399b0c4700 1 -- 172.21.15.33:0/3876580423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f399406b220 msgr2=0x7f3994102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:12:57.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.935+0000 7f399b0c4700 1 --2- 172.21.15.33:0/3876580423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f399406b220 0x7f3994102510 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f397c00d980 tx=0x7f397c00dd40 comp rx=0 tx=0).stop 2024-05-15T14:12:57.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.935+0000 7f399b0c4700 1 -- 172.21.15.33:0/3876580423 shutdown_connections 2024-05-15T14:12:57.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.935+0000 7f399b0c4700 1 --2- 172.21.15.33:0/3876580423 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f39800b29f0 0x7f39800b4eb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:57.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.935+0000 7f399b0c4700 1 --2- 172.21.15.33:0/3876580423 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39941095b0 0x7f3994102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:57.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.935+0000 7f399b0c4700 1 --2- 172.21.15.33:0/3876580423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f399406b220 0x7f3994102510 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:57.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.935+0000 7f399b0c4700 1 --2- 172.21.15.33:0/3876580423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f399406bbe0 0x7f3994102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:12:57.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.935+0000 7f399b0c4700 1 -- 172.21.15.33:0/3876580423 >> 172.21.15.33:0/3876580423 conn(0x7f39940793f0 msgr2=0x7f3994101140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:12:57.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.935+0000 7f399b0c4700 1 -- 172.21.15.33:0/3876580423 shutdown_connections 2024-05-15T14:12:57.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:12:57.935+0000 7f399b0c4700 1 -- 172.21.15.33:0/3876580423 wait complete. 2024-05-15T14:12:58.541 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs fail cephfs 2024-05-15T14:12:58.567 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:58 smithi033 ceph-mon[31650]: pgmap v396: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 11 op/s 2024-05-15T14:12:58.567 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:12:58 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/3876580423' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T14:12:58.796 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:12:58.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:58 smithi052 ceph-mon[36190]: pgmap v396: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 11 op/s 2024-05-15T14:12:58.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:12:58 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/3876580423' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T14:12:58.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:58 smithi081 ceph-mon[36434]: pgmap v396: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 11 op/s 2024-05-15T14:12:58.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:12:58 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/3876580423' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-15T14:13:00.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.368+0000 7fbf42054700 1 -- 172.21.15.33:0/3526840409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf3c06c6d0 msgr2=0x7fbf3c110400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:13:00.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.368+0000 7fbf42054700 1 --2- 172.21.15.33:0/3526840409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf3c06c6d0 0x7fbf3c110400 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fbf3400b0d0 tx=0x7fbf3400b3e0 comp rx=0 tx=0).stop 2024-05-15T14:13:00.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.368+0000 7fbf42054700 1 -- 172.21.15.33:0/3526840409 shutdown_connections 2024-05-15T14:13:00.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.368+0000 7fbf42054700 1 --2- 172.21.15.33:0/3526840409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf3c06c6d0 0x7fbf3c110400 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:00.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.368+0000 7fbf42054700 1 --2- 172.21.15.33:0/3526840409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf3c109630 0x7fbf3c109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:00.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.368+0000 7fbf42054700 1 --2- 172.21.15.33:0/3526840409 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf3c06bd10 0x7fbf3c06c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:00.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.368+0000 7fbf42054700 1 -- 172.21.15.33:0/3526840409 >> 172.21.15.33:0/3526840409 conn(0x7fbf3c0794e0 msgr2=0x7fbf3c0798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:13:00.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.368+0000 7fbf42054700 1 -- 172.21.15.33:0/3526840409 shutdown_connections 2024-05-15T14:13:00.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.368+0000 7fbf42054700 1 -- 172.21.15.33:0/3526840409 wait complete. 2024-05-15T14:13:00.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.369+0000 7fbf42054700 1 Processor -- start 2024-05-15T14:13:00.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.369+0000 7fbf42054700 1 -- start start 2024-05-15T14:13:00.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.369+0000 7fbf42054700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf3c06bd10 0x7fbf3c19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:13:00.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.369+0000 7fbf42054700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf3c06c6d0 0x7fbf3c19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:13:00.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.369+0000 7fbf42054700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf3c109630 0x7fbf3c199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:13:00.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.369+0000 7fbf42054700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c113750 con 0x7fbf3c06c6d0 2024-05-15T14:13:00.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.369+0000 7fbf42054700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c1135d0 con 0x7fbf3c06bd10 2024-05-15T14:13:00.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.369+0000 7fbf42054700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3c1138d0 con 0x7fbf3c109630 2024-05-15T14:13:00.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.369+0000 7fbf3affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf3c06c6d0 0x7fbf3c19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:13:00.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.369+0000 7fbf3affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf3c06c6d0 0x7fbf3c19fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51912/0 (socket says 172.21.15.33:51912) 2024-05-15T14:13:00.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.369+0000 7fbf3affd700 1 -- 172.21.15.33:0/1182160719 learned_addr learned my addr 172.21.15.33:0/1182160719 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:13:00.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.369+0000 7fbf3bfff700 1 --2- 172.21.15.33:0/1182160719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf3c109630 0x7fbf3c199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:13:00.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.369+0000 7fbf3b7fe700 1 --2- 172.21.15.33:0/1182160719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf3c06bd10 0x7fbf3c19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:13:00.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.369+0000 7fbf3affd700 1 -- 172.21.15.33:0/1182160719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf3c109630 msgr2=0x7fbf3c199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:13:00.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.370+0000 7fbf3affd700 1 --2- 172.21.15.33:0/1182160719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf3c109630 0x7fbf3c199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:00.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.370+0000 7fbf3affd700 1 -- 172.21.15.33:0/1182160719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf3c06bd10 msgr2=0x7fbf3c19f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:13:00.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.370+0000 7fbf3affd700 1 --2- 172.21.15.33:0/1182160719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf3c06bd10 0x7fbf3c19f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:00.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.370+0000 7fbf3affd700 1 -- 172.21.15.33:0/1182160719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf3c19a0b0 con 0x7fbf3c06c6d0 2024-05-15T14:13:00.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.370+0000 7fbf3b7fe700 1 --2- 172.21.15.33:0/1182160719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf3c06bd10 0x7fbf3c19f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:13:00.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.370+0000 7fbf3affd700 1 --2- 172.21.15.33:0/1182160719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf3c06c6d0 0x7fbf3c19fcd0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fbf3000d980 tx=0x7fbf3000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:13:00.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.370+0000 7fbf38ff9700 1 -- 172.21.15.33:0/1182160719 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf30017070 con 0x7fbf3c06c6d0 2024-05-15T14:13:00.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.370+0000 7fbf42054700 1 -- 172.21.15.33:0/1182160719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf3c19a3a0 con 0x7fbf3c06c6d0 2024-05-15T14:13:00.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.370+0000 7fbf38ff9700 1 -- 172.21.15.33:0/1182160719 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf30005020 con 0x7fbf3c06c6d0 2024-05-15T14:13:00.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.370+0000 7fbf38ff9700 1 -- 172.21.15.33:0/1182160719 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf30009b50 con 0x7fbf3c06c6d0 2024-05-15T14:13:00.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.370+0000 7fbf42054700 1 -- 172.21.15.33:0/1182160719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf3c1a6660 con 0x7fbf3c06c6d0 2024-05-15T14:13:00.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.371+0000 7fbf42054700 1 -- 172.21.15.33:0/1182160719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf3c06a230 con 0x7fbf3c06c6d0 2024-05-15T14:13:00.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.372+0000 7fbf38ff9700 1 -- 172.21.15.33:0/1182160719 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbf300051d0 con 0x7fbf3c06c6d0 2024-05-15T14:13:00.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.373+0000 7fbf38ff9700 1 --2- 172.21.15.33:0/1182160719 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbf240b29f0 0x7fbf240b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:13:00.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.373+0000 7fbf3b7fe700 1 --2- 172.21.15.33:0/1182160719 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbf240b29f0 0x7fbf240b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:13:00.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.373+0000 7fbf38ff9700 1 -- 172.21.15.33:0/1182160719 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8542+0+0 (secure 0 0 0) 0x7fbf300e2560 con 0x7fbf3c06c6d0 2024-05-15T14:13:00.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.374+0000 7fbf3b7fe700 1 --2- 172.21.15.33:0/1182160719 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbf240b29f0 0x7fbf240b4eb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c005fd0 tx=0x7fbf2c005f60 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:13:00.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.378+0000 7fbf38ff9700 1 -- 172.21.15.33:0/1182160719 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbf30020020 con 0x7fbf3c06c6d0 2024-05-15T14:13:00.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.689+0000 7fbf42054700 1 -- 172.21.15.33:0/1182160719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fbf3c02e560 con 0x7fbf3c06c6d0 2024-05-15T14:13:00.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.691+0000 7fbf38ff9700 1 -- 172.21.15.33:0/1182160719 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v43) ==== 161+0+0 (secure 0 0 0) 0x7fbf300a9f30 con 0x7fbf3c06c6d0 2024-05-15T14:13:00.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.695+0000 7fbf42054700 1 -- 172.21.15.33:0/1182160719 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbf240b29f0 msgr2=0x7fbf240b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:13:00.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.695+0000 7fbf42054700 1 --2- 172.21.15.33:0/1182160719 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbf240b29f0 0x7fbf240b4eb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c005fd0 tx=0x7fbf2c005f60 comp rx=0 tx=0).stop 2024-05-15T14:13:00.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.695+0000 7fbf42054700 1 -- 172.21.15.33:0/1182160719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf3c06c6d0 msgr2=0x7fbf3c19fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:13:00.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.695+0000 7fbf42054700 1 --2- 172.21.15.33:0/1182160719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf3c06c6d0 0x7fbf3c19fcd0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fbf3000d980 tx=0x7fbf3000dc90 comp rx=0 tx=0).stop 2024-05-15T14:13:00.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.695+0000 7fbf42054700 1 -- 172.21.15.33:0/1182160719 shutdown_connections 2024-05-15T14:13:00.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.695+0000 7fbf42054700 1 --2- 172.21.15.33:0/1182160719 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7fbf240b29f0 0x7fbf240b4eb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:00.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.695+0000 7fbf42054700 1 --2- 172.21.15.33:0/1182160719 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf3c109630 0x7fbf3c199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:00.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.695+0000 7fbf42054700 1 --2- 172.21.15.33:0/1182160719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf3c06bd10 0x7fbf3c19f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:00.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.695+0000 7fbf42054700 1 --2- 172.21.15.33:0/1182160719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf3c06c6d0 0x7fbf3c19fcd0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:00.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.695+0000 7fbf42054700 1 -- 172.21.15.33:0/1182160719 >> 172.21.15.33:0/1182160719 conn(0x7fbf3c0794e0 msgr2=0x7fbf3c100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:13:00.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.695+0000 7fbf42054700 1 -- 172.21.15.33:0/1182160719 shutdown_connections 2024-05-15T14:13:00.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:00.695+0000 7fbf42054700 1 -- 172.21.15.33:0/1182160719 wait complete. 2024-05-15T14:13:00.699 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-15T14:13:00.752 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:00 smithi081 ceph-mon[36434]: pgmap v397: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 7 op/s 2024-05-15T14:13:00.752 INFO:journalctl@ceph.mds.c.smithi081.stdout:May 15 14:13:00 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-c[66566]: ignoring --setuser ceph since I am not root 2024-05-15T14:13:00.752 INFO:journalctl@ceph.mds.c.smithi081.stdout:May 15 14:13:00 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-c[66566]: ignoring --setgroup ceph since I am not root 2024-05-15T14:13:00.753 INFO:journalctl@ceph.mds.c.smithi081.stdout:May 15 14:13:00 smithi081 ceph-mds[66570]: -- 172.21.15.81:0/820518631 <== mon.1 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55987c3e3a40 con 0x55987c3b6c00 2024-05-15T14:13:00.753 INFO:journalctl@ceph.mds.c.smithi081.stdout:May 15 14:13:00 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-c[66566]: starting mds.c at 2024-05-15T14:13:00.753 INFO:journalctl@ceph.mds.i.smithi081.stdout:May 15 14:13:00 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-i[66884]: ignoring --setuser ceph since I am not root 2024-05-15T14:13:00.753 INFO:journalctl@ceph.mds.i.smithi081.stdout:May 15 14:13:00 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-i[66884]: ignoring --setgroup ceph since I am not root 2024-05-15T14:13:00.753 INFO:journalctl@ceph.mds.i.smithi081.stdout:May 15 14:13:00 smithi081 ceph-mds[66888]: -- 172.21.15.81:0/852920870 <== mon.1 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d519655a40 con 0x55d519629000 2024-05-15T14:13:00.753 INFO:journalctl@ceph.mds.i.smithi081.stdout:May 15 14:13:00 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-i[66884]: starting mds.i at 2024-05-15T14:13:00.754 INFO:journalctl@ceph.mds.l.smithi081.stdout:May 15 14:13:00 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-l[65961]: ignoring --setuser ceph since I am not root 2024-05-15T14:13:00.754 INFO:journalctl@ceph.mds.l.smithi081.stdout:May 15 14:13:00 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-l[65961]: ignoring --setgroup ceph since I am not root 2024-05-15T14:13:00.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:00 smithi052 ceph-mon[36190]: pgmap v397: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 7 op/s 2024-05-15T14:13:00.841 INFO:journalctl@ceph.mds.b.smithi052.stdout:May 15 14:13:00 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-b[66920]: ignoring --setuser ceph since I am not root 2024-05-15T14:13:00.841 INFO:journalctl@ceph.mds.b.smithi052.stdout:May 15 14:13:00 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-b[66920]: ignoring --setgroup ceph since I am not root 2024-05-15T14:13:00.842 INFO:journalctl@ceph.mds.b.smithi052.stdout:May 15 14:13:00 smithi052 ceph-mds[66924]: -- 172.21.15.52:0/1636665608 <== mon.1 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5593ce9bda40 con 0x5593ce990c00 2024-05-15T14:13:00.842 INFO:journalctl@ceph.mds.b.smithi052.stdout:May 15 14:13:00 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-b[66920]: starting mds.b at 2024-05-15T14:13:00.842 INFO:journalctl@ceph.mds.h.smithi052.stdout:May 15 14:13:00 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-h[66613]: ignoring --setuser ceph since I am not root 2024-05-15T14:13:00.842 INFO:journalctl@ceph.mds.h.smithi052.stdout:May 15 14:13:00 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-h[66613]: ignoring --setgroup ceph since I am not root 2024-05-15T14:13:00.842 INFO:journalctl@ceph.mds.h.smithi052.stdout:May 15 14:13:00 smithi052 ceph-mds[66617]: -- 172.21.15.52:0/835812125 <== mon.1 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558cd97bba40 con 0x558cd978ec00 2024-05-15T14:13:00.842 INFO:journalctl@ceph.mds.h.smithi052.stdout:May 15 14:13:00 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-h[66613]: starting mds.h at 2024-05-15T14:13:00.843 INFO:journalctl@ceph.mds.h.smithi052.stdout:May 15 14:13:00 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-h[66613]: 2024-05-15T14:13:00.740+0000 7f2ed91d2700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6836/0: (98) Address already in use 2024-05-15T14:13:00.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:00 smithi033 ceph-mon[31650]: pgmap v397: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 7 op/s 2024-05-15T14:13:00.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:00 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mon-a[31646]: 2024-05-15T14:13:00.690+0000 7f6ae4ecf700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-15T14:13:01.107 INFO:journalctl@ceph.mds.l.smithi081.stdout:May 15 14:13:00 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-l[65961]: starting mds.l at 2024-05-15T14:13:01.108 INFO:journalctl@ceph.mds.f.smithi081.stdout:May 15 14:13:00 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-f[66264]: ignoring --setuser ceph since I am not root 2024-05-15T14:13:01.108 INFO:journalctl@ceph.mds.f.smithi081.stdout:May 15 14:13:00 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-f[66264]: ignoring --setgroup ceph since I am not root 2024-05-15T14:13:01.108 INFO:journalctl@ceph.mds.f.smithi081.stdout:May 15 14:13:00 smithi081 ceph-mds[66268]: -- 172.21.15.81:0/4198792144 <== mon.1 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5556720f5a40 con 0x5556720c8c00 2024-05-15T14:13:01.108 INFO:journalctl@ceph.mds.f.smithi081.stdout:May 15 14:13:00 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-f[66264]: starting mds.f at 2024-05-15T14:13:01.542 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-05-15T14:13:01.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:01 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/1182160719' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-15T14:13:01.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:01 smithi052 ceph-mon[36190]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-15T14:13:01.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:01 smithi052 ceph-mon[36190]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-15T14:13:01.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:01 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/1182160719' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-15T14:13:01.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:01 smithi052 ceph-mon[36190]: osdmap e99: 12 total, 12 up, 12 in 2024-05-15T14:13:01.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:01 smithi052 ceph-mon[36190]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-15T14:13:01.843 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:13:01.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:01 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/1182160719' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-15T14:13:01.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:01 smithi081 ceph-mon[36434]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-15T14:13:01.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:01 smithi081 ceph-mon[36434]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-15T14:13:01.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:01 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/1182160719' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-15T14:13:01.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:01 smithi081 ceph-mon[36434]: osdmap e99: 12 total, 12 up, 12 in 2024-05-15T14:13:01.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:01 smithi081 ceph-mon[36434]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-15T14:13:01.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:01 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/1182160719' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-15T14:13:01.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:01 smithi033 ceph-mon[31650]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-15T14:13:01.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:01 smithi033 ceph-mon[31650]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-15T14:13:01.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:01 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/1182160719' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-15T14:13:01.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:01 smithi033 ceph-mon[31650]: osdmap e99: 12 total, 12 up, 12 in 2024-05-15T14:13:01.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:01 smithi033 ceph-mon[31650]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-15T14:13:02.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:02 smithi033 ceph-mon[31650]: pgmap v399: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-05-15T14:13:02.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:02 smithi033 ceph-mon[31650]: mds.? [v2:172.21.15.81:6832/2158632188,v1:172.21.15.81:6833/2158632188] up:boot 2024-05-15T14:13:02.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:02 smithi033 ceph-mon[31650]: mds.? [v2:172.21.15.52:6837/3160040347,v1:172.21.15.52:6839/3160040347] up:boot 2024-05-15T14:13:02.506 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:02 smithi033 ceph-mon[31650]: mds.? [v2:172.21.15.52:6836/2325567952,v1:172.21.15.52:6838/2325567952] up:boot 2024-05-15T14:13:02.506 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:02 smithi033 ceph-mon[31650]: mds.? [v2:172.21.15.81:6834/1596299082,v1:172.21.15.81:6835/1596299082] up:boot 2024-05-15T14:13:02.506 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:02 smithi033 ceph-mon[31650]: mds.? [v2:172.21.15.81:6836/2723705181,v1:172.21.15.81:6837/2723705181] up:boot 2024-05-15T14:13:02.506 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:02 smithi033 ceph-mon[31650]: mds.? [v2:172.21.15.81:6838/1042465469,v1:172.21.15.81:6839/1042465469] up:boot 2024-05-15T14:13:02.506 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:02 smithi033 ceph-mon[31650]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-15T14:13:02.506 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:02 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-15T14:13:02.506 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:02 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-15T14:13:02.507 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:02 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-15T14:13:02.507 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:02 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-15T14:13:02.507 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:02 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-15T14:13:02.507 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:02 smithi033 ceph-mon[31650]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-15T14:13:02.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:02 smithi052 ceph-mon[36190]: pgmap v399: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-05-15T14:13:02.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:02 smithi052 ceph-mon[36190]: mds.? [v2:172.21.15.81:6832/2158632188,v1:172.21.15.81:6833/2158632188] up:boot 2024-05-15T14:13:02.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:02 smithi052 ceph-mon[36190]: mds.? [v2:172.21.15.52:6837/3160040347,v1:172.21.15.52:6839/3160040347] up:boot 2024-05-15T14:13:02.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:02 smithi052 ceph-mon[36190]: mds.? [v2:172.21.15.52:6836/2325567952,v1:172.21.15.52:6838/2325567952] up:boot 2024-05-15T14:13:02.842 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:02 smithi052 ceph-mon[36190]: mds.? [v2:172.21.15.81:6834/1596299082,v1:172.21.15.81:6835/1596299082] up:boot 2024-05-15T14:13:02.842 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:02 smithi052 ceph-mon[36190]: mds.? [v2:172.21.15.81:6836/2723705181,v1:172.21.15.81:6837/2723705181] up:boot 2024-05-15T14:13:02.842 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:02 smithi052 ceph-mon[36190]: mds.? [v2:172.21.15.81:6838/1042465469,v1:172.21.15.81:6839/1042465469] up:boot 2024-05-15T14:13:02.842 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:02 smithi052 ceph-mon[36190]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-15T14:13:02.842 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:02 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-15T14:13:02.842 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:02 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-15T14:13:02.842 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:02 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-15T14:13:02.843 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:02 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-15T14:13:02.843 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:02 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-15T14:13:02.843 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:02 smithi052 ceph-mon[36190]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-15T14:13:02.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:02 smithi081 ceph-mon[36434]: pgmap v399: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 9 op/s 2024-05-15T14:13:02.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:02 smithi081 ceph-mon[36434]: mds.? [v2:172.21.15.81:6832/2158632188,v1:172.21.15.81:6833/2158632188] up:boot 2024-05-15T14:13:02.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:02 smithi081 ceph-mon[36434]: mds.? [v2:172.21.15.52:6837/3160040347,v1:172.21.15.52:6839/3160040347] up:boot 2024-05-15T14:13:02.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:02 smithi081 ceph-mon[36434]: mds.? [v2:172.21.15.52:6836/2325567952,v1:172.21.15.52:6838/2325567952] up:boot 2024-05-15T14:13:02.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:02 smithi081 ceph-mon[36434]: mds.? [v2:172.21.15.81:6834/1596299082,v1:172.21.15.81:6835/1596299082] up:boot 2024-05-15T14:13:02.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:02 smithi081 ceph-mon[36434]: mds.? [v2:172.21.15.81:6836/2723705181,v1:172.21.15.81:6837/2723705181] up:boot 2024-05-15T14:13:02.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:02 smithi081 ceph-mon[36434]: mds.? [v2:172.21.15.81:6838/1042465469,v1:172.21.15.81:6839/1042465469] up:boot 2024-05-15T14:13:02.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:02 smithi081 ceph-mon[36434]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-15T14:13:02.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:02 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-15T14:13:02.859 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:02 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-15T14:13:02.859 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:02 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-15T14:13:02.859 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:02 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-15T14:13:02.859 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:02 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-15T14:13:02.859 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:02 smithi081 ceph-mon[36434]: from='mgr.15144 172.21.15.33:0/3156454774' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-15T14:13:03.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.423+0000 7f65c93a0700 1 -- 172.21.15.33:0/4163341502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65c41095b0 msgr2=0x7f65c4109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:13:03.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.423+0000 7f65c93a0700 1 --2- 172.21.15.33:0/4163341502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65c41095b0 0x7f65c4109990 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f65b4009990 tx=0x7f65b4009ca0 comp rx=0 tx=0).stop 2024-05-15T14:13:03.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.423+0000 7f65c93a0700 1 -- 172.21.15.33:0/4163341502 shutdown_connections 2024-05-15T14:13:03.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.423+0000 7f65c93a0700 1 --2- 172.21.15.33:0/4163341502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65c406bbe0 0x7f65c41103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:03.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.423+0000 7f65c93a0700 1 --2- 172.21.15.33:0/4163341502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65c41095b0 0x7f65c4109990 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:03.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.423+0000 7f65c93a0700 1 --2- 172.21.15.33:0/4163341502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65c406b220 0x7f65c406b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:03.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.423+0000 7f65c93a0700 1 -- 172.21.15.33:0/4163341502 >> 172.21.15.33:0/4163341502 conn(0x7f65c40793f0 msgr2=0x7f65c4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:13:03.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.423+0000 7f65c93a0700 1 -- 172.21.15.33:0/4163341502 shutdown_connections 2024-05-15T14:13:03.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.423+0000 7f65c93a0700 1 -- 172.21.15.33:0/4163341502 wait complete. 2024-05-15T14:13:03.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.423+0000 7f65c93a0700 1 Processor -- start 2024-05-15T14:13:03.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.424+0000 7f65c93a0700 1 -- start start 2024-05-15T14:13:03.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.424+0000 7f65c93a0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65c406b220 0x7f65c4102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:13:03.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.424+0000 7f65c93a0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65c406bbe0 0x7f65c4102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:13:03.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.424+0000 7f65c93a0700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65c41095b0 0x7f65c4102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:13:03.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.424+0000 7f65c93a0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f65c4113740 con 0x7f65c406b220 2024-05-15T14:13:03.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.424+0000 7f65c93a0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f65c41135c0 con 0x7f65c406bbe0 2024-05-15T14:13:03.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.424+0000 7f65c93a0700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f65c41138c0 con 0x7f65c41095b0 2024-05-15T14:13:03.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.424+0000 7f65c2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65c406b220 0x7f65c4102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:13:03.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.424+0000 7f65c2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65c406b220 0x7f65c4102510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59796/0 (socket says 172.21.15.33:59796) 2024-05-15T14:13:03.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.424+0000 7f65c2ffd700 1 -- 172.21.15.33:0/1058777397 learned_addr learned my addr 172.21.15.33:0/1058777397 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:13:03.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.424+0000 7f65c27fc700 1 --2- 172.21.15.33:0/1058777397 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65c406bbe0 0x7f65c4102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:13:03.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.424+0000 7f65c37fe700 1 --2- 172.21.15.33:0/1058777397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65c41095b0 0x7f65c4102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:13:03.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.424+0000 7f65c2ffd700 1 -- 172.21.15.33:0/1058777397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65c41095b0 msgr2=0x7f65c4102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:13:03.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.424+0000 7f65c2ffd700 1 --2- 172.21.15.33:0/1058777397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65c41095b0 0x7f65c4102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:03.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.424+0000 7f65c2ffd700 1 -- 172.21.15.33:0/1058777397 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65c406bbe0 msgr2=0x7f65c4102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:13:03.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.424+0000 7f65c2ffd700 1 --2- 172.21.15.33:0/1058777397 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65c406bbe0 0x7f65c4102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:03.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.424+0000 7f65c2ffd700 1 -- 172.21.15.33:0/1058777397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c41a8350 con 0x7f65c406b220 2024-05-15T14:13:03.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.425+0000 7f65c2ffd700 1 --2- 172.21.15.33:0/1058777397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65c406b220 0x7f65c4102510 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f65b4000c00 tx=0x7f65b40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:13:03.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.425+0000 7f65abfff700 1 -- 172.21.15.33:0/1058777397 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b4027070 con 0x7f65c406b220 2024-05-15T14:13:03.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.425+0000 7f65c93a0700 1 -- 172.21.15.33:0/1058777397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c41a85e0 con 0x7f65c406b220 2024-05-15T14:13:03.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.425+0000 7f65abfff700 1 -- 172.21.15.33:0/1058777397 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65b40040e0 con 0x7f65c406b220 2024-05-15T14:13:03.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.425+0000 7f65abfff700 1 -- 172.21.15.33:0/1058777397 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b400f460 con 0x7f65c406b220 2024-05-15T14:13:03.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.425+0000 7f65c93a0700 1 -- 172.21.15.33:0/1058777397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c41a8a90 con 0x7f65c406b220 2024-05-15T14:13:03.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.426+0000 7f65c93a0700 1 -- 172.21.15.33:0/1058777397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65c410ba90 con 0x7f65c406b220 2024-05-15T14:13:03.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.427+0000 7f65abfff700 1 -- 172.21.15.33:0/1058777397 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f65b402a070 con 0x7f65c406b220 2024-05-15T14:13:03.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.428+0000 7f65abfff700 1 --2- 172.21.15.33:0/1058777397 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f65ac0b29f0 0x7f65ac0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:13:03.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.428+0000 7f65c27fc700 1 --2- 172.21.15.33:0/1058777397 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f65ac0b29f0 0x7f65ac0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:13:03.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.428+0000 7f65abfff700 1 -- 172.21.15.33:0/1058777397 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8756+0+0 (secure 0 0 0) 0x7f65b40e3b40 con 0x7f65c406b220 2024-05-15T14:13:03.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.429+0000 7f65c27fc700 1 --2- 172.21.15.33:0/1058777397 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f65ac0b29f0 0x7f65ac0b4eb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f65c40799c0 tx=0x7f65b8009190 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:13:03.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.431+0000 7f65abfff700 1 -- 172.21.15.33:0/1058777397 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f65b4031070 con 0x7f65c406b220 2024-05-15T14:13:03.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:03.693+0000 7f65c93a0700 1 -- 172.21.15.33:0/1058777397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f65c406a230 con 0x7f65c406b220 2024-05-15T14:13:04.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:04.506+0000 7f65abfff700 1 -- 172.21.15.33:0/1058777397 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v45) ==== 104+0+0 (secure 0 0 0) 0x7f65b4013070 con 0x7f65c406b220 2024-05-15T14:13:04.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:04.509+0000 7f65c93a0700 1 -- 172.21.15.33:0/1058777397 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f65ac0b29f0 msgr2=0x7f65ac0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:13:04.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:04.509+0000 7f65c93a0700 1 --2- 172.21.15.33:0/1058777397 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f65ac0b29f0 0x7f65ac0b4eb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f65c40799c0 tx=0x7f65b8009190 comp rx=0 tx=0).stop 2024-05-15T14:13:04.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:04.509+0000 7f65c93a0700 1 -- 172.21.15.33:0/1058777397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65c406b220 msgr2=0x7f65c4102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:13:04.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:04.509+0000 7f65c93a0700 1 --2- 172.21.15.33:0/1058777397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65c406b220 0x7f65c4102510 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f65b4000c00 tx=0x7f65b40047b0 comp rx=0 tx=0).stop 2024-05-15T14:13:04.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:04.509+0000 7f65c93a0700 1 -- 172.21.15.33:0/1058777397 shutdown_connections 2024-05-15T14:13:04.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:04.509+0000 7f65c93a0700 1 --2- 172.21.15.33:0/1058777397 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f65ac0b29f0 0x7f65ac0b4eb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:04.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:04.509+0000 7f65c93a0700 1 --2- 172.21.15.33:0/1058777397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65c41095b0 0x7f65c4102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:04.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:04.509+0000 7f65c93a0700 1 --2- 172.21.15.33:0/1058777397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65c406b220 0x7f65c4102510 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:04.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:04.509+0000 7f65c93a0700 1 --2- 172.21.15.33:0/1058777397 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65c406bbe0 0x7f65c4102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:04.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:04.509+0000 7f65c93a0700 1 -- 172.21.15.33:0/1058777397 >> 172.21.15.33:0/1058777397 conn(0x7f65c40793f0 msgr2=0x7f65c4101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:13:04.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:04.509+0000 7f65c93a0700 1 -- 172.21.15.33:0/1058777397 shutdown_connections 2024-05-15T14:13:04.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:04.509+0000 7f65c93a0700 1 -- 172.21.15.33:0/1058777397 wait complete. 2024-05-15T14:13:04.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:04 smithi052 ceph-mon[36190]: pgmap v400: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 67 KiB/s wr, 26 op/s 2024-05-15T14:13:04.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:04 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/1058777397' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-15T14:13:04.842 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:04 smithi033 ceph-mon[31650]: pgmap v400: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 67 KiB/s wr, 26 op/s 2024-05-15T14:13:04.842 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:04 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/1058777397' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-15T14:13:04.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:04 smithi081 ceph-mon[36434]: pgmap v400: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 67 KiB/s wr, 26 op/s 2024-05-15T14:13:04.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:04 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/1058777397' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-15T14:13:05.295 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-15T14:13:05.555 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:13:05.807 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:05 smithi033 ceph-mon[31650]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-15T14:13:05.807 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:05 smithi033 ceph-mon[31650]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-15T14:13:05.807 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:05 smithi033 ceph-mon[31650]: Cluster is now healthy 2024-05-15T14:13:05.808 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:05 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/1058777397' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-15T14:13:05.808 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:05 smithi033 ceph-mon[31650]: fsmap 12 up:standby 2024-05-15T14:13:05.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:05 smithi052 ceph-mon[36190]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-15T14:13:05.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:05 smithi052 ceph-mon[36190]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-15T14:13:05.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:05 smithi052 ceph-mon[36190]: Cluster is now healthy 2024-05-15T14:13:05.842 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:05 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/1058777397' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-15T14:13:05.842 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:05 smithi052 ceph-mon[36190]: fsmap 12 up:standby 2024-05-15T14:13:05.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:05 smithi081 ceph-mon[36434]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-15T14:13:05.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:05 smithi081 ceph-mon[36434]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-15T14:13:05.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:05 smithi081 ceph-mon[36434]: Cluster is now healthy 2024-05-15T14:13:05.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:05 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/1058777397' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-15T14:13:05.858 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:05 smithi081 ceph-mon[36434]: fsmap 12 up:standby 2024-05-15T14:13:06.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:06 smithi033 ceph-mon[31650]: pgmap v401: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 67 KiB/s wr, 26 op/s 2024-05-15T14:13:06.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:06 smithi052 ceph-mon[36190]: pgmap v401: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 67 KiB/s wr, 26 op/s 2024-05-15T14:13:06.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:06 smithi081 ceph-mon[36434]: pgmap v401: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 67 KiB/s wr, 26 op/s 2024-05-15T14:13:07.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.020+0000 7f3015217700 1 -- 172.21.15.33:0/4168008009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f301006c630 msgr2=0x7f30101103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:13:07.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.020+0000 7f3015217700 1 --2- 172.21.15.33:0/4168008009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f301006c630 0x7f30101103d0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f300800b0a0 tx=0x7f300800b3b0 comp rx=0 tx=0).stop 2024-05-15T14:13:07.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.021+0000 7f3015217700 1 -- 172.21.15.33:0/4168008009 shutdown_connections 2024-05-15T14:13:07.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.021+0000 7f3015217700 1 --2- 172.21.15.33:0/4168008009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f301006c630 0x7f30101103d0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:07.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.021+0000 7f3015217700 1 --2- 172.21.15.33:0/4168008009 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30101095b0 0x7f3010109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:07.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.021+0000 7f3015217700 1 --2- 172.21.15.33:0/4168008009 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f301006bc70 0x7f301006c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:07.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.021+0000 7f3015217700 1 -- 172.21.15.33:0/4168008009 >> 172.21.15.33:0/4168008009 conn(0x7f30100793f0 msgr2=0x7f3010079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:13:07.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.021+0000 7f3015217700 1 -- 172.21.15.33:0/4168008009 shutdown_connections 2024-05-15T14:13:07.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.021+0000 7f3015217700 1 -- 172.21.15.33:0/4168008009 wait complete. 2024-05-15T14:13:07.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.021+0000 7f3015217700 1 Processor -- start 2024-05-15T14:13:07.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.022+0000 7f3015217700 1 -- start start 2024-05-15T14:13:07.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.022+0000 7f3015217700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f301006bc70 0x7f301019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:13:07.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.022+0000 7f3015217700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f301006c630 0x7f301019fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:13:07.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.022+0000 7f3015217700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30101095b0 0x7f3010199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:13:07.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.022+0000 7f3015217700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f30101136e0 con 0x7f30101095b0 2024-05-15T14:13:07.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.022+0000 7f3015217700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3010113560 con 0x7f301006c630 2024-05-15T14:13:07.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.022+0000 7f3015217700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f3010113860 con 0x7f301006bc70 2024-05-15T14:13:07.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.022+0000 7f300f59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30101095b0 0x7f3010199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:13:07.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.022+0000 7f300f59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30101095b0 0x7f3010199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59814/0 (socket says 172.21.15.33:59814) 2024-05-15T14:13:07.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.022+0000 7f300f59e700 1 -- 172.21.15.33:0/2615300342 learned_addr learned my addr 172.21.15.33:0/2615300342 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:13:07.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.022+0000 7f300ed9d700 1 --2- 172.21.15.33:0/2615300342 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f301006bc70 0x7f301019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:13:07.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.022+0000 7f300e59c700 1 --2- 172.21.15.33:0/2615300342 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f301006c630 0x7f301019fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:13:07.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.022+0000 7f300f59e700 1 -- 172.21.15.33:0/2615300342 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f301006bc70 msgr2=0x7f301019f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:13:07.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.022+0000 7f300f59e700 1 --2- 172.21.15.33:0/2615300342 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f301006bc70 0x7f301019f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:07.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.022+0000 7f300f59e700 1 -- 172.21.15.33:0/2615300342 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f301006c630 msgr2=0x7f301019fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:13:07.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.022+0000 7f300f59e700 1 --2- 172.21.15.33:0/2615300342 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f301006c630 0x7f301019fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:07.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.022+0000 7f300f59e700 1 -- 172.21.15.33:0/2615300342 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f301019a160 con 0x7f30101095b0 2024-05-15T14:13:07.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.022+0000 7f300e59c700 1 --2- 172.21.15.33:0/2615300342 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f301006c630 0x7f301019fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-15T14:13:07.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.023+0000 7f300f59e700 1 --2- 172.21.15.33:0/2615300342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30101095b0 0x7f3010199a00 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f300800b830 tx=0x7f3008004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:13:07.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.023+0000 7f2ff7fff700 1 -- 172.21.15.33:0/2615300342 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3008021070 con 0x7f30101095b0 2024-05-15T14:13:07.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.023+0000 7f2ff7fff700 1 -- 172.21.15.33:0/2615300342 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3008007e30 con 0x7f30101095b0 2024-05-15T14:13:07.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.023+0000 7f3015217700 1 -- 172.21.15.33:0/2615300342 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f301019a3f0 con 0x7f30101095b0 2024-05-15T14:13:07.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.023+0000 7f2ff7fff700 1 -- 172.21.15.33:0/2615300342 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30080047d0 con 0x7f30101095b0 2024-05-15T14:13:07.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.023+0000 7f3015217700 1 -- 172.21.15.33:0/2615300342 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30101a6710 con 0x7f30101095b0 2024-05-15T14:13:07.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.024+0000 7f3015217700 1 -- 172.21.15.33:0/2615300342 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f301010ba90 con 0x7f30101095b0 2024-05-15T14:13:07.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.026+0000 7f2ff7fff700 1 -- 172.21.15.33:0/2615300342 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3008007980 con 0x7f30101095b0 2024-05-15T14:13:07.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.027+0000 7f2ff7fff700 1 --2- 172.21.15.33:0/2615300342 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f2ff80b29f0 0x7f2ff80b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:13:07.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.027+0000 7f300ed9d700 1 --2- 172.21.15.33:0/2615300342 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f2ff80b29f0 0x7f2ff80b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:13:07.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.027+0000 7f2ff7fff700 1 -- 172.21.15.33:0/2615300342 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8756+0+0 (secure 0 0 0) 0x7f300800f020 con 0x7f30101095b0 2024-05-15T14:13:07.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.027+0000 7f300ed9d700 1 --2- 172.21.15.33:0/2615300342 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f2ff80b29f0 0x7f2ff80b4eb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f301019ab40 tx=0x7f300000b3c0 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:13:07.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.031+0000 7f2ff7fff700 1 -- 172.21.15.33:0/2615300342 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f300802a020 con 0x7f30101095b0 2024-05-15T14:13:07.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.245+0000 7f3015217700 1 -- 172.21.15.33:0/2615300342 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f301019af30 con 0x7f30101095b0 2024-05-15T14:13:07.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.523+0000 7f2ff7fff700 1 -- 172.21.15.33:0/2615300342 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v100) ==== 181+0+0 (secure 0 0 0) 0x7f300802e3e0 con 0x7f30101095b0 2024-05-15T14:13:07.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.527+0000 7f3015217700 1 -- 172.21.15.33:0/2615300342 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f2ff80b29f0 msgr2=0x7f2ff80b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:13:07.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.527+0000 7f3015217700 1 --2- 172.21.15.33:0/2615300342 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f2ff80b29f0 0x7f2ff80b4eb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f301019ab40 tx=0x7f300000b3c0 comp rx=0 tx=0).stop 2024-05-15T14:13:07.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.527+0000 7f3015217700 1 -- 172.21.15.33:0/2615300342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30101095b0 msgr2=0x7f3010199a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:13:07.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.527+0000 7f3015217700 1 --2- 172.21.15.33:0/2615300342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30101095b0 0x7f3010199a00 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f300800b830 tx=0x7f3008004ce0 comp rx=0 tx=0).stop 2024-05-15T14:13:07.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.527+0000 7f3015217700 1 -- 172.21.15.33:0/2615300342 shutdown_connections 2024-05-15T14:13:07.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.527+0000 7f3015217700 1 --2- 172.21.15.33:0/2615300342 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f2ff80b29f0 0x7f2ff80b4eb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:07.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.527+0000 7f3015217700 1 --2- 172.21.15.33:0/2615300342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30101095b0 0x7f3010199a00 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:07.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.527+0000 7f3015217700 1 --2- 172.21.15.33:0/2615300342 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f301006bc70 0x7f301019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:07.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.527+0000 7f3015217700 1 --2- 172.21.15.33:0/2615300342 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f301006c630 0x7f301019fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:07.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.527+0000 7f3015217700 1 -- 172.21.15.33:0/2615300342 >> 172.21.15.33:0/2615300342 conn(0x7f30100793f0 msgr2=0x7f30101012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:13:07.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.528+0000 7f3015217700 1 -- 172.21.15.33:0/2615300342 shutdown_connections 2024-05-15T14:13:07.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:07.528+0000 7f3015217700 1 -- 172.21.15.33:0/2615300342 wait complete. 2024-05-15T14:13:07.531 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2024-05-15T14:13:07.833 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:07 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/2615300342' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-15T14:13:07.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:07 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/2615300342' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-15T14:13:07.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:07 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/2615300342' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-15T14:13:08.222 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b shell --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-15T14:13:08.486 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/mon.a/config 2024-05-15T14:13:08.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:08 smithi052 ceph-mon[36190]: pgmap v402: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 63 KiB/s wr, 23 op/s 2024-05-15T14:13:08.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:08 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/2615300342' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-15T14:13:08.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:08 smithi052 ceph-mon[36190]: osdmap e100: 12 total, 12 up, 12 in 2024-05-15T14:13:08.854 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:08 smithi033 ceph-mon[31650]: pgmap v402: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 63 KiB/s wr, 23 op/s 2024-05-15T14:13:08.854 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:08 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/2615300342' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-15T14:13:08.854 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:08 smithi033 ceph-mon[31650]: osdmap e100: 12 total, 12 up, 12 in 2024-05-15T14:13:08.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:08 smithi081 ceph-mon[36434]: pgmap v402: 97 pgs: 97 active+clean; 203 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 63 KiB/s wr, 23 op/s 2024-05-15T14:13:08.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:08 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/2615300342' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-15T14:13:08.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:08 smithi081 ceph-mon[36434]: osdmap e100: 12 total, 12 up, 12 in 2024-05-15T14:13:10.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.072+0000 7f52d8188700 1 -- 172.21.15.33:0/2613722354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52d0102e00 msgr2=0x7f52d010f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:13:10.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.072+0000 7f52d8188700 1 --2- 172.21.15.33:0/2613722354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52d0102e00 0x7f52d010f2f0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f52c8009990 tx=0x7f52c8009ca0 comp rx=0 tx=0).stop 2024-05-15T14:13:10.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.072+0000 7f52d8188700 1 -- 172.21.15.33:0/2613722354 shutdown_connections 2024-05-15T14:13:10.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.072+0000 7f52d8188700 1 --2- 172.21.15.33:0/2613722354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52d0102e00 0x7f52d010f2f0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:10.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.072+0000 7f52d8188700 1 --2- 172.21.15.33:0/2613722354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52d0108520 0x7f52d0108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:10.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.072+0000 7f52d8188700 1 --2- 172.21.15.33:0/2613722354 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52d0102440 0x7f52d01028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:10.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.073+0000 7f52d8188700 1 -- 172.21.15.33:0/2613722354 >> 172.21.15.33:0/2613722354 conn(0x7f52d00793f0 msgr2=0x7f52d0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:13:10.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.073+0000 7f52d8188700 1 -- 172.21.15.33:0/2613722354 shutdown_connections 2024-05-15T14:13:10.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.073+0000 7f52d8188700 1 -- 172.21.15.33:0/2613722354 wait complete. 2024-05-15T14:13:10.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.073+0000 7f52d8188700 1 Processor -- start 2024-05-15T14:13:10.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.073+0000 7f52d8188700 1 -- start start 2024-05-15T14:13:10.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.074+0000 7f52d8188700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52d0102440 0x7f52d0104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:13:10.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.074+0000 7f52d8188700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52d0102e00 0x7f52d01045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:13:10.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.074+0000 7f52d8188700 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52d0108520 0x7f52d0104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:13:10.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.074+0000 7f52d8188700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f52d01125d0 con 0x7f52d0102e00 2024-05-15T14:13:10.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.074+0000 7f52d8188700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f52d0112450 con 0x7f52d0102440 2024-05-15T14:13:10.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.074+0000 7f52d8188700 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 -- 0x7f52d0112750 con 0x7f52d0108520 2024-05-15T14:13:10.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.074+0000 7f52d5723700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52d0102e00 0x7f52d01045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:13:10.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.074+0000 7f52d5723700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52d0102e00 0x7f52d01045c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59830/0 (socket says 172.21.15.33:59830) 2024-05-15T14:13:10.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.074+0000 7f52d5723700 1 -- 172.21.15.33:0/334394839 learned_addr learned my addr 172.21.15.33:0/334394839 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-15T14:13:10.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.074+0000 7f52d5f24700 1 --2- 172.21.15.33:0/334394839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52d0102440 0x7f52d0104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:13:10.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.074+0000 7f52d6725700 1 --2- 172.21.15.33:0/334394839 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52d0108520 0x7f52d0104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:13:10.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.074+0000 7f52d5723700 1 -- 172.21.15.33:0/334394839 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52d0108520 msgr2=0x7f52d0104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:13:10.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.074+0000 7f52d5723700 1 --2- 172.21.15.33:0/334394839 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52d0108520 0x7f52d0104b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:10.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.074+0000 7f52d5723700 1 -- 172.21.15.33:0/334394839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52d0102440 msgr2=0x7f52d0104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:13:10.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.074+0000 7f52d5723700 1 --2- 172.21.15.33:0/334394839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52d0102440 0x7f52d0104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:10.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.074+0000 7f52d5723700 1 -- 172.21.15.33:0/334394839 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d01a80c0 con 0x7f52d0102e00 2024-05-15T14:13:10.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.074+0000 7f52d5723700 1 --2- 172.21.15.33:0/334394839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52d0102e00 0x7f52d01045c0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f52c000d980 tx=0x7f52c000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:13:10.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.074+0000 7f52c6ffd700 1 -- 172.21.15.33:0/334394839 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c0016070 con 0x7f52d0102e00 2024-05-15T14:13:10.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.074+0000 7f52c6ffd700 1 -- 172.21.15.33:0/334394839 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52c0004830 con 0x7f52d0102e00 2024-05-15T14:13:10.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.074+0000 7f52d8188700 1 -- 172.21.15.33:0/334394839 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52d01a83b0 con 0x7f52d0102e00 2024-05-15T14:13:10.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.074+0000 7f52c6ffd700 1 -- 172.21.15.33:0/334394839 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c00054c0 con 0x7f52d0102e00 2024-05-15T14:13:10.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.075+0000 7f52d8188700 1 -- 172.21.15.33:0/334394839 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52d01a8940 con 0x7f52d0102e00 2024-05-15T14:13:10.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.075+0000 7f52d8188700 1 -- 172.21.15.33:0/334394839 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52d006a230 con 0x7f52d0102e00 2024-05-15T14:13:10.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.077+0000 7f52c6ffd700 1 -- 172.21.15.33:0/334394839 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f52c0004220 con 0x7f52d0102e00 2024-05-15T14:13:10.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.077+0000 7f52c6ffd700 1 --2- 172.21.15.33:0/334394839 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f52bc0b2a70 0x7f52bc0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-15T14:13:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.078+0000 7f52d5f24700 1 --2- 172.21.15.33:0/334394839 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f52bc0b2a70 0x7f52bc0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-15T14:13:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.078+0000 7f52c6ffd700 1 -- 172.21.15.33:0/334394839 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8202+0+0 (secure 0 0 0) 0x7f52c00e2760 con 0x7f52d0102e00 2024-05-15T14:13:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.078+0000 7f52d5f24700 1 --2- 172.21.15.33:0/334394839 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f52bc0b2a70 0x7f52bc0b4f30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f52cc00f540 tx=0x7f52cc00b040 comp rx=0 tx=0).ready entity=mgr.15144 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-15T14:13:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.081+0000 7f52c6ffd700 1 -- 172.21.15.33:0/334394839 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f52c001f020 con 0x7f52d0102e00 2024-05-15T14:13:10.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.294+0000 7f52d8188700 1 -- 172.21.15.33:0/334394839 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f52d002e560 con 0x7f52d0102e00 2024-05-15T14:13:10.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.532+0000 7f52c6ffd700 1 -- 172.21.15.33:0/334394839 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v101) ==== 169+0+0 (secure 0 0 0) 0x7f52c00aa280 con 0x7f52d0102e00 2024-05-15T14:13:10.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.534+0000 7f52d8188700 1 -- 172.21.15.33:0/334394839 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f52bc0b2a70 msgr2=0x7f52bc0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:13:10.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.534+0000 7f52d8188700 1 --2- 172.21.15.33:0/334394839 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f52bc0b2a70 0x7f52bc0b4f30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f52cc00f540 tx=0x7f52cc00b040 comp rx=0 tx=0).stop 2024-05-15T14:13:10.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.534+0000 7f52d8188700 1 -- 172.21.15.33:0/334394839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52d0102e00 msgr2=0x7f52d01045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-15T14:13:10.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.534+0000 7f52d8188700 1 --2- 172.21.15.33:0/334394839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52d0102e00 0x7f52d01045c0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f52c000d980 tx=0x7f52c000dc90 comp rx=0 tx=0).stop 2024-05-15T14:13:10.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.534+0000 7f52d8188700 1 -- 172.21.15.33:0/334394839 shutdown_connections 2024-05-15T14:13:10.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.534+0000 7f52d8188700 1 --2- 172.21.15.33:0/334394839 >> [v2:172.21.15.33:6800/1796050977,v1:172.21.15.33:6801/1796050977] conn(0x7f52bc0b2a70 0x7f52bc0b4f30 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:10.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.534+0000 7f52d8188700 1 --2- 172.21.15.33:0/334394839 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52d0108520 0x7f52d0104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:10.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.534+0000 7f52d8188700 1 --2- 172.21.15.33:0/334394839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52d0102440 0x7f52d0104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:10.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.534+0000 7f52d8188700 1 --2- 172.21.15.33:0/334394839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52d0102e00 0x7f52d01045c0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-15T14:13:10.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.534+0000 7f52d8188700 1 -- 172.21.15.33:0/334394839 >> 172.21.15.33:0/334394839 conn(0x7f52d00793f0 msgr2=0x7f52d006b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-15T14:13:10.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.534+0000 7f52d8188700 1 -- 172.21.15.33:0/334394839 shutdown_connections 2024-05-15T14:13:10.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-15T14:13:10.534+0000 7f52d8188700 1 -- 172.21.15.33:0/334394839 wait complete. 2024-05-15T14:13:10.539 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2024-05-15T14:13:10.555 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:10 smithi033 ceph-mon[31650]: pgmap v404: 65 pgs: 65 active+clean; 56 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T14:13:10.555 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:10 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/334394839' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-15T14:13:10.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:10 smithi052 ceph-mon[36190]: pgmap v404: 65 pgs: 65 active+clean; 56 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T14:13:10.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:10 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/334394839' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-15T14:13:10.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:10 smithi081 ceph-mon[36434]: pgmap v404: 65 pgs: 65 active+clean; 56 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-15T14:13:10.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:10 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/334394839' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-15T14:13:11.191 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-15T14:13:11.222 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-15T14:13:11.255 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-15T14:13:11.289 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-15T14:13:11.290 DEBUG:teuthology.orchestra.run.smithi033:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-15T14:13:11.312 DEBUG:teuthology.orchestra.run.smithi052:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-15T14:13:11.329 DEBUG:teuthology.orchestra.run.smithi081:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-15T14:13:11.346 INFO:tasks.cephadm:Stopping all daemons... 2024-05-15T14:13:11.347 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-15T14:13:11.347 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mon.a 2024-05-15T14:13:11.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:11 smithi033 systemd[1]: Stopping Ceph mon.a for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T14:13:11.794 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:11 smithi033 ceph-mon[31650]: from='client.? 172.21.15.33:0/334394839' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-15T14:13:11.794 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:11 smithi033 ceph-mon[31650]: osdmap e101: 12 total, 12 up, 12 in 2024-05-15T14:13:11.794 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:11 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mon-a[31646]: 2024-05-15T14:13:11.679+0000 7f6aea6da700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T14:13:11.794 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:11 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mon-a[31646]: 2024-05-15T14:13:11.679+0000 7f6aea6da700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-15T14:13:11.840 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:11 smithi052 ceph-mon[36190]: from='client.? 172.21.15.33:0/334394839' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-15T14:13:11.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:11 smithi052 ceph-mon[36190]: osdmap e101: 12 total, 12 up, 12 in 2024-05-15T14:13:11.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:11 smithi081 ceph-mon[36434]: from='client.? 172.21.15.33:0/334394839' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-15T14:13:11.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:11 smithi081 ceph-mon[36434]: osdmap e101: 12 total, 12 up, 12 in 2024-05-15T14:13:12.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:11 smithi033 podman[103301]: 2024-05-15 14:13:11.792389033 +0000 UTC m=+0.226127967 container died 40d78556a6be0d1051a52d5c5ad4f7b3dcb7b9fc62a70f200f75e93206213a0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-15T14:13:12.432 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:12 smithi033 podman[103301]: 2024-05-15 14:13:12.162556035 +0000 UTC m=+0.596294968 container cleanup 40d78556a6be0d1051a52d5c5ad4f7b3dcb7b9fc62a70f200f75e93206213a0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True) 2024-05-15T14:13:12.432 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 15 14:13:12 smithi033 bash[103301]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mon-a 2024-05-15T14:13:12.597 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mon.a.service' 2024-05-15T14:13:12.644 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T14:13:12.644 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-15T14:13:12.644 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-15T14:13:12.645 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mon.b 2024-05-15T14:13:12.964 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:12 smithi052 systemd[1]: Stopping Ceph mon.b for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T14:13:13.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:12 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mon-b[36185]: 2024-05-15T14:13:12.961+0000 7f99f1c9d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T14:13:13.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:12 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mon-b[36185]: 2024-05-15T14:13:12.961+0000 7f99f1c9d700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-15T14:13:13.343 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:13 smithi052 podman[73386]: 2024-05-15 14:13:13.155585487 +0000 UTC m=+0.299816642 container died 9ec142c69ed61ec3a0db7de69bc144c11a46dfc349b4e578d269f17587c9d651 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mon-b, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-15T14:13:13.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:13 smithi052 podman[73386]: 2024-05-15 14:13:13.53671711 +0000 UTC m=+0.680948257 container cleanup 9ec142c69ed61ec3a0db7de69bc144c11a46dfc349b4e578d269f17587c9d651 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0) 2024-05-15T14:13:13.841 INFO:journalctl@ceph.mon.b.smithi052.stdout:May 15 14:13:13 smithi052 bash[73386]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mon-b 2024-05-15T14:13:14.119 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mon.b.service' 2024-05-15T14:13:14.170 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T14:13:14.170 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-15T14:13:14.171 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-15T14:13:14.171 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mon.c 2024-05-15T14:13:14.508 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:14 smithi081 systemd[1]: Stopping Ceph mon.c for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T14:13:14.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:14 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mon-c[36408]: 2024-05-15T14:13:14.505+0000 7fabb8fc1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T14:13:14.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:14 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mon-c[36408]: 2024-05-15T14:13:14.505+0000 7fabb8fc1700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-15T14:13:14.857 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:14 smithi081 podman[73989]: 2024-05-15 14:13:14.689586101 +0000 UTC m=+0.326694183 container died c2480a95732991152574092afa4d1eddfad683ffee1ead5b052f88eb44e83403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mon-c, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-15T14:13:15.283 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:15 smithi081 podman[73989]: 2024-05-15 14:13:15.11436092 +0000 UTC m=+0.751468999 container cleanup c2480a95732991152574092afa4d1eddfad683ffee1ead5b052f88eb44e83403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mon-c, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, ceph=True) 2024-05-15T14:13:15.284 INFO:journalctl@ceph.mon.c.smithi081.stdout:May 15 14:13:15 smithi081 bash[73989]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mon-c 2024-05-15T14:13:15.493 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mon.c.service' 2024-05-15T14:13:15.540 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T14:13:15.541 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-15T14:13:15.541 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-15T14:13:15.541 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mgr.x 2024-05-15T14:13:15.861 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:13:15 smithi033 systemd[1]: Stopping Ceph mgr.x for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T14:13:16.179 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:13:15 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:13:15.858+0000 7f2a6772e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T14:13:16.179 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:13:15 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x[31876]: 2024-05-15T14:13:15.858+0000 7f2a6772e700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-05-15T14:13:16.179 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:13:15 smithi033 podman[103423]: 2024-05-15 14:13:15.89045265 +0000 UTC m=+0.141909611 container died cfcd834cab7f62715ead13e0c4ccc57ede5e0c0fdedf5a146a9e0a6e5413d350 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-15T14:13:16.441 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:13:16 smithi033 podman[103423]: 2024-05-15 14:13:16.259737305 +0000 UTC m=+0.511194273 container cleanup cfcd834cab7f62715ead13e0c4ccc57ede5e0c0fdedf5a146a9e0a6e5413d350 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9b63866860f0b71216b2c0a73aedd2e0a8a5c98b, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-15T14:13:16.441 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 15 14:13:16 smithi033 bash[103423]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-x 2024-05-15T14:13:16.844 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mgr.x.service' 2024-05-15T14:13:16.880 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T14:13:16.880 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-15T14:13:16.881 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-15T14:13:16.881 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mgr.y 2024-05-15T14:13:17.216 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 14:13:16 smithi052 systemd[1]: Stopping Ceph mgr.y for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T14:13:17.496 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 14:13:17 smithi052 podman[73508]: 2024-05-15 14:13:17.214966752 +0000 UTC m=+0.130353121 container died c0b639a4f8996095e6496e0a510008e7560a8122dcc085d15a03b304c1586b6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-15T14:13:17.841 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 14:13:17 smithi052 podman[73508]: 2024-05-15 14:13:17.577465607 +0000 UTC m=+0.492851976 container cleanup c0b639a4f8996095e6496e0a510008e7560a8122dcc085d15a03b304c1586b6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=) 2024-05-15T14:13:17.841 INFO:journalctl@ceph.mgr.y.smithi052.stdout:May 15 14:13:17 smithi052 bash[73508]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-y 2024-05-15T14:13:18.181 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mgr.y.service' 2024-05-15T14:13:18.217 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T14:13:18.218 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-15T14:13:18.218 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-15T14:13:18.218 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mgr.z 2024-05-15T14:13:18.551 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 14:13:18 smithi081 systemd[1]: Stopping Ceph mgr.z for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T14:13:18.821 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 14:13:18 smithi081 podman[74111]: 2024-05-15 14:13:18.549605213 +0000 UTC m=+0.143037415 container died 9f81e1ae812cab180772f94d1b95355371948ee107288c81f81e80b34c8a8a38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-15T14:13:19.107 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 14:13:18 smithi081 podman[74111]: 2024-05-15 14:13:18.894204986 +0000 UTC m=+0.487637211 container cleanup 9f81e1ae812cab180772f94d1b95355371948ee107288c81f81e80b34c8a8a38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-15T14:13:19.107 INFO:journalctl@ceph.mgr.z.smithi081.stdout:May 15 14:13:18 smithi081 bash[74111]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mgr-z 2024-05-15T14:13:19.332 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mgr.z.service' 2024-05-15T14:13:19.366 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T14:13:19.366 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-15T14:13:19.367 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-15T14:13:19.367 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.0 2024-05-15T14:13:19.682 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 15 14:13:19 smithi033 systemd[1]: Stopping Ceph osd.0 for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T14:13:19.941 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 15 14:13:19 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-0[43694]: 2024-05-15T14:13:19.679+0000 7f4f582d7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T14:13:19.941 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 15 14:13:19 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-0[43694]: 2024-05-15T14:13:19.679+0000 7f4f582d7700 -1 osd.0 101 *** Got signal Terminated *** 2024-05-15T14:13:19.941 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 15 14:13:19 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-0[43694]: 2024-05-15T14:13:19.679+0000 7f4f582d7700 -1 osd.0 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-15T14:13:25.383 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 15 14:13:25 smithi033 podman[103545]: 2024-05-15 14:13:25.034887091 +0000 UTC m=+5.469945326 container died 5adae181e337d0b9a81648be8a9a3cc2c2d0ec24344cd4c556739446d5a486fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-15T14:13:25.383 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 15 14:13:25 smithi033 podman[103545]: 2024-05-15 14:13:25.381719014 +0000 UTC m=+5.816777249 container cleanup 5adae181e337d0b9a81648be8a9a3cc2c2d0ec24344cd4c556739446d5a486fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-15T14:13:25.691 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 15 14:13:25 smithi033 bash[103545]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-0 2024-05-15T14:13:25.691 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 15 14:13:25 smithi033 podman[103560]: 2024-05-15 14:13:25.56536694 +0000 UTC m=+0.524331734 container remove 5adae181e337d0b9a81648be8a9a3cc2c2d0ec24344cd4c556739446d5a486fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, ceph=True) 2024-05-15T14:13:26.278 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 15 14:13:26 smithi033 podman[103652]: 2024-05-15 14:13:25.919839798 +0000 UTC m=+0.017920056 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-15T14:13:26.279 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 15 14:13:26 smithi033 podman[103652]: 2024-05-15 14:13:26.060953439 +0000 UTC m=+0.159033677 container create f2c1236372481822eef43e3f6fb6260f2624009a30dced8fee9dc54be6147d03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240514) 2024-05-15T14:13:26.598 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 15 14:13:26 smithi033 podman[103652]: 2024-05-15 14:13:26.277117604 +0000 UTC m=+0.375197853 container init f2c1236372481822eef43e3f6fb6260f2624009a30dced8fee9dc54be6147d03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T14:13:26.598 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 15 14:13:26 smithi033 podman[103652]: 2024-05-15 14:13:26.281213054 +0000 UTC m=+0.379293300 container start f2c1236372481822eef43e3f6fb6260f2624009a30dced8fee9dc54be6147d03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-15T14:13:26.598 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 15 14:13:26 smithi033 podman[103652]: 2024-05-15 14:13:26.385465396 +0000 UTC m=+0.483545645 container attach f2c1236372481822eef43e3f6fb6260f2624009a30dced8fee9dc54be6147d03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5) 2024-05-15T14:13:26.890 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 15 14:13:26 smithi033 podman[103652]: 2024-05-15 14:13:26.596813461 +0000 UTC m=+0.694893708 container died f2c1236372481822eef43e3f6fb6260f2624009a30dced8fee9dc54be6147d03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0) 2024-05-15T14:13:27.172 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.0.service' 2024-05-15T14:13:27.219 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T14:13:27.219 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-15T14:13:27.219 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-15T14:13:27.219 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.1 2024-05-15T14:13:27.591 INFO:journalctl@ceph.osd.1.smithi052.stdout:May 15 14:13:27 smithi052 systemd[1]: Stopping Ceph osd.1 for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T14:13:28.091 INFO:journalctl@ceph.osd.1.smithi052.stdout:May 15 14:13:27 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-1[40752]: 2024-05-15T14:13:27.665+0000 7f15c2dfa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T14:13:28.091 INFO:journalctl@ceph.osd.1.smithi052.stdout:May 15 14:13:27 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-1[40752]: 2024-05-15T14:13:27.665+0000 7f15c2dfa700 -1 osd.1 101 *** Got signal Terminated *** 2024-05-15T14:13:28.091 INFO:journalctl@ceph.osd.1.smithi052.stdout:May 15 14:13:27 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-1[40752]: 2024-05-15T14:13:27.665+0000 7f15c2dfa700 -1 osd.1 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-15T14:13:33.341 INFO:journalctl@ceph.osd.1.smithi052.stdout:May 15 14:13:33 smithi052 podman[73646]: 2024-05-15 14:13:33.050474215 +0000 UTC m=+5.626409927 container died 7da3988ca00cb295cdc8b5f4214698c43ea26074550ae81776d3e8b41cf20f3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-1, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-05-15T14:13:33.781 INFO:journalctl@ceph.osd.1.smithi052.stdout:May 15 14:13:33 smithi052 podman[73646]: 2024-05-15 14:13:33.516217708 +0000 UTC m=+6.092153461 container cleanup 7da3988ca00cb295cdc8b5f4214698c43ea26074550ae81776d3e8b41cf20f3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux ) 2024-05-15T14:13:33.781 INFO:journalctl@ceph.osd.1.smithi052.stdout:May 15 14:13:33 smithi052 bash[73646]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-1 2024-05-15T14:13:34.091 INFO:journalctl@ceph.osd.1.smithi052.stdout:May 15 14:13:33 smithi052 podman[73663]: 2024-05-15 14:13:33.779810736 +0000 UTC m=+0.730544833 container remove 7da3988ca00cb295cdc8b5f4214698c43ea26074550ae81776d3e8b41cf20f3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-1, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-15T14:13:34.407 INFO:journalctl@ceph.osd.1.smithi052.stdout:May 15 14:13:34 smithi052 podman[73754]: 2024-05-15 14:13:34.056686759 +0000 UTC m=+0.017938399 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-15T14:13:34.407 INFO:journalctl@ceph.osd.1.smithi052.stdout:May 15 14:13:34 smithi052 podman[73754]: 2024-05-15 14:13:34.194170644 +0000 UTC m=+0.155422287 container create db0cd8dd3e7539f10b6acb0298628f93999f3aec38852272d0e4b473c766b4f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T14:13:34.407 INFO:journalctl@ceph.osd.1.smithi052.stdout:May 15 14:13:34 smithi052 podman[73754]: 2024-05-15 14:13:34.40529009 +0000 UTC m=+0.366541717 container init db0cd8dd3e7539f10b6acb0298628f93999f3aec38852272d0e4b473c766b4f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-15T14:13:34.718 INFO:journalctl@ceph.osd.1.smithi052.stdout:May 15 14:13:34 smithi052 podman[73754]: 2024-05-15 14:13:34.40882586 +0000 UTC m=+0.370077492 container start db0cd8dd3e7539f10b6acb0298628f93999f3aec38852272d0e4b473c766b4f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-15T14:13:34.719 INFO:journalctl@ceph.osd.1.smithi052.stdout:May 15 14:13:34 smithi052 podman[73754]: 2024-05-15 14:13:34.480451172 +0000 UTC m=+0.441702817 container attach db0cd8dd3e7539f10b6acb0298628f93999f3aec38852272d0e4b473c766b4f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-15T14:13:35.062 INFO:journalctl@ceph.osd.1.smithi052.stdout:May 15 14:13:34 smithi052 podman[73754]: 2024-05-15 14:13:34.717494486 +0000 UTC m=+0.678746112 container died db0cd8dd3e7539f10b6acb0298628f93999f3aec38852272d0e4b473c766b4f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-15T14:13:35.344 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.1.service' 2024-05-15T14:13:35.387 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T14:13:35.387 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-15T14:13:35.388 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-15T14:13:35.388 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.2 2024-05-15T14:13:35.684 INFO:journalctl@ceph.osd.2.smithi081.stdout:May 15 14:13:35 smithi081 systemd[1]: Stopping Ceph osd.2 for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T14:13:36.107 INFO:journalctl@ceph.osd.2.smithi081.stdout:May 15 14:13:35 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-2[41560]: 2024-05-15T14:13:35.681+0000 7f89dd5f7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T14:13:36.107 INFO:journalctl@ceph.osd.2.smithi081.stdout:May 15 14:13:35 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-2[41560]: 2024-05-15T14:13:35.681+0000 7f89dd5f7700 -1 osd.2 101 *** Got signal Terminated *** 2024-05-15T14:13:36.107 INFO:journalctl@ceph.osd.2.smithi081.stdout:May 15 14:13:35 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-2[41560]: 2024-05-15T14:13:35.681+0000 7f89dd5f7700 -1 osd.2 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-15T14:13:41.476 INFO:journalctl@ceph.osd.2.smithi081.stdout:May 15 14:13:41 smithi081 podman[74246]: 2024-05-15 14:13:41.18867981 +0000 UTC m=+5.605489617 container died 8a6fc4cb89da898e8045279cfddc4fb78aac757eb222044042ed312d8207cbe6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-15T14:13:41.827 INFO:journalctl@ceph.osd.2.smithi081.stdout:May 15 14:13:41 smithi081 podman[74246]: 2024-05-15 14:13:41.565643312 +0000 UTC m=+5.982453120 container cleanup 8a6fc4cb89da898e8045279cfddc4fb78aac757eb222044042ed312d8207cbe6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-15T14:13:41.827 INFO:journalctl@ceph.osd.2.smithi081.stdout:May 15 14:13:41 smithi081 bash[74246]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-2 2024-05-15T14:13:42.107 INFO:journalctl@ceph.osd.2.smithi081.stdout:May 15 14:13:41 smithi081 podman[74262]: 2024-05-15 14:13:41.825623709 +0000 UTC m=+0.635806094 container remove 8a6fc4cb89da898e8045279cfddc4fb78aac757eb222044042ed312d8207cbe6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-2, org.label-schema.build-date=20240514, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-15T14:13:42.456 INFO:journalctl@ceph.osd.2.smithi081.stdout:May 15 14:13:42 smithi081 podman[74354]: 2024-05-15 14:13:42.08589388 +0000 UTC m=+0.017542011 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-15T14:13:42.456 INFO:journalctl@ceph.osd.2.smithi081.stdout:May 15 14:13:42 smithi081 podman[74354]: 2024-05-15 14:13:42.209791612 +0000 UTC m=+0.141439725 container create 6513913083c4c18a661e00d72826c6402cc5d747dc5bc6a4537af6ff6c85b574 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-15T14:13:42.750 INFO:journalctl@ceph.osd.2.smithi081.stdout:May 15 14:13:42 smithi081 podman[74354]: 2024-05-15 14:13:42.454568144 +0000 UTC m=+0.386216262 container init 6513913083c4c18a661e00d72826c6402cc5d747dc5bc6a4537af6ff6c85b574 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-15T14:13:42.751 INFO:journalctl@ceph.osd.2.smithi081.stdout:May 15 14:13:42 smithi081 podman[74354]: 2024-05-15 14:13:42.458430742 +0000 UTC m=+0.390078862 container start 6513913083c4c18a661e00d72826c6402cc5d747dc5bc6a4537af6ff6c85b574 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-15T14:13:42.751 INFO:journalctl@ceph.osd.2.smithi081.stdout:May 15 14:13:42 smithi081 podman[74354]: 2024-05-15 14:13:42.528918095 +0000 UTC m=+0.460566212 container attach 6513913083c4c18a661e00d72826c6402cc5d747dc5bc6a4537af6ff6c85b574 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-15T14:13:43.042 INFO:journalctl@ceph.osd.2.smithi081.stdout:May 15 14:13:42 smithi081 podman[74354]: 2024-05-15 14:13:42.74966124 +0000 UTC m=+0.681309353 container died 6513913083c4c18a661e00d72826c6402cc5d747dc5bc6a4537af6ff6c85b574 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-15T14:13:43.352 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.2.service' 2024-05-15T14:13:43.395 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T14:13:43.395 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-15T14:13:43.395 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-15T14:13:43.396 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.3 2024-05-15T14:13:43.691 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 15 14:13:43 smithi033 systemd[1]: Stopping Ceph osd.3 for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T14:13:44.191 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 15 14:13:43 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-3[50628]: 2024-05-15T14:13:43.722+0000 7f9c71691700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T14:13:44.191 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 15 14:13:43 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-3[50628]: 2024-05-15T14:13:43.722+0000 7f9c71691700 -1 osd.3 101 *** Got signal Terminated *** 2024-05-15T14:13:44.191 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 15 14:13:43 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-3[50628]: 2024-05-15T14:13:43.722+0000 7f9c71691700 -1 osd.3 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-15T14:13:45.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:13:45 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:13:45.510+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:13:46.691 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 15 14:13:46 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-3[50628]: 2024-05-15T14:13:46.386+0000 7f9c6dc85700 -1 osd.3 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.394010+0000 front 2024-05-15T14:13:24.394038+0000 (oldest deadline 2024-05-15T14:13:46.094230+0000) 2024-05-15T14:13:46.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:13:46 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:13:46.497+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:13:47.691 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 15 14:13:47 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-3[50628]: 2024-05-15T14:13:47.339+0000 7f9c6dc85700 -1 osd.3 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.394010+0000 front 2024-05-15T14:13:24.394038+0000 (oldest deadline 2024-05-15T14:13:46.094230+0000) 2024-05-15T14:13:47.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:13:47 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:13:47.515+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:13:48.691 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 15 14:13:48 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-3[50628]: 2024-05-15T14:13:48.293+0000 7f9c6dc85700 -1 osd.3 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.394010+0000 front 2024-05-15T14:13:24.394038+0000 (oldest deadline 2024-05-15T14:13:46.094230+0000) 2024-05-15T14:13:48.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:13:48 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:13:48.536+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:13:49.358 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:13:48 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5[48018]: 2024-05-15T14:13:48.925+0000 7f5ee1e28700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:19.585073+0000 front 2024-05-15T14:13:19.585099+0000 (oldest deadline 2024-05-15T14:13:48.385527+0000) 2024-05-15T14:13:49.441 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 15 14:13:49 smithi033 podman[103775]: 2024-05-15 14:13:49.171889712 +0000 UTC m=+5.571661405 container died f25bac3320a5772e825bbd2112110d93e2532206d3c95b4d78d97ee6168ab8e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-3, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-15T14:13:49.811 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 15 14:13:49 smithi033 podman[103775]: 2024-05-15 14:13:49.549522216 +0000 UTC m=+5.949293909 container cleanup f25bac3320a5772e825bbd2112110d93e2532206d3c95b4d78d97ee6168ab8e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-15T14:13:49.811 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 15 14:13:49 smithi033 bash[103775]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-3 2024-05-15T14:13:49.812 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:13:49 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:13:49.562+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:13:49.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:13:49 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:13:49.504+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:13:50.184 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 15 14:13:49 smithi033 podman[103790]: 2024-05-15 14:13:49.8093022 +0000 UTC m=+0.635914793 container remove f25bac3320a5772e825bbd2112110d93e2532206d3c95b4d78d97ee6168ab8e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-15T14:13:50.357 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:13:49 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5[48018]: 2024-05-15T14:13:49.940+0000 7f5ee1e28700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:19.585073+0000 front 2024-05-15T14:13:19.585099+0000 (oldest deadline 2024-05-15T14:13:48.385527+0000) 2024-05-15T14:13:50.441 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:13:50 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:13:50.298+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.305228+0000 front 2024-05-15T14:13:24.305185+0000 (oldest deadline 2024-05-15T14:13:50.205368+0000) 2024-05-15T14:13:50.442 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 15 14:13:50 smithi033 podman[103877]: 2024-05-15 14:13:50.085256379 +0000 UTC m=+0.017749983 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-15T14:13:50.442 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 15 14:13:50 smithi033 podman[103877]: 2024-05-15 14:13:50.184969046 +0000 UTC m=+0.117462636 container create 9a59678b221e7ccb12fcc6fa61be2c911ada6e504ce51fcc4725f77205a752cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2) 2024-05-15T14:13:50.442 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 15 14:13:50 smithi033 podman[103877]: 2024-05-15 14:13:50.395668168 +0000 UTC m=+0.328161760 container init 9a59678b221e7ccb12fcc6fa61be2c911ada6e504ce51fcc4725f77205a752cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-05-15T14:13:50.442 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 15 14:13:50 smithi033 podman[103877]: 2024-05-15 14:13:50.399458744 +0000 UTC m=+0.331952336 container start 9a59678b221e7ccb12fcc6fa61be2c911ada6e504ce51fcc4725f77205a752cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=) 2024-05-15T14:13:50.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:13:50 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:13:50.462+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:13:50.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:13:50 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:13:50.520+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:13:50.942 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 15 14:13:50 smithi033 podman[103877]: 2024-05-15 14:13:50.495861369 +0000 UTC m=+0.428354966 container attach 9a59678b221e7ccb12fcc6fa61be2c911ada6e504ce51fcc4725f77205a752cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-3-deactivate, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514) 2024-05-15T14:13:50.942 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 15 14:13:50 smithi033 podman[103877]: 2024-05-15 14:13:50.705844873 +0000 UTC m=+0.638338466 container died 9a59678b221e7ccb12fcc6fa61be2c911ada6e504ce51fcc4725f77205a752cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-15T14:13:51.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:13:50 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:13:50.615+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:13:51.299 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.3.service' 2024-05-15T14:13:51.340 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T14:13:51.341 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-15T14:13:51.341 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-15T14:13:51.341 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.4 2024-05-15T14:13:51.357 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:13:50 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5[48018]: 2024-05-15T14:13:50.897+0000 7f5ee1e28700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:19.585073+0000 front 2024-05-15T14:13:19.585099+0000 (oldest deadline 2024-05-15T14:13:48.385527+0000) 2024-05-15T14:13:51.546 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:13:51 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:13:51.334+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.305228+0000 front 2024-05-15T14:13:24.305185+0000 (oldest deadline 2024-05-15T14:13:50.205368+0000) 2024-05-15T14:13:51.673 INFO:journalctl@ceph.osd.4.smithi052.stdout:May 15 14:13:51 smithi052 systemd[1]: Stopping Ceph osd.4 for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T14:13:51.673 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:13:51 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:13:51.612+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:13:51.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:13:51 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:13:51.491+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:13:51.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:13:51 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:13:51.544+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:13:52.091 INFO:journalctl@ceph.osd.4.smithi052.stdout:May 15 14:13:51 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-4[47696]: 2024-05-15T14:13:51.670+0000 7f5441109700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T14:13:52.091 INFO:journalctl@ceph.osd.4.smithi052.stdout:May 15 14:13:51 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-4[47696]: 2024-05-15T14:13:51.670+0000 7f5441109700 -1 osd.4 101 *** Got signal Terminated *** 2024-05-15T14:13:52.091 INFO:journalctl@ceph.osd.4.smithi052.stdout:May 15 14:13:51 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-4[47696]: 2024-05-15T14:13:51.670+0000 7f5441109700 -1 osd.4 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-15T14:13:52.357 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:13:51 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5[48018]: 2024-05-15T14:13:51.870+0000 7f5ee1e28700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:19.585073+0000 front 2024-05-15T14:13:19.585099+0000 (oldest deadline 2024-05-15T14:13:48.385527+0000) 2024-05-15T14:13:52.691 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:13:52 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:13:52.350+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.305228+0000 front 2024-05-15T14:13:24.305185+0000 (oldest deadline 2024-05-15T14:13:50.205368+0000) 2024-05-15T14:13:52.691 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:13:52 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:13:52.589+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:13:52.829 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:13:52 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:13:52.504+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:13:53.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:13:52 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:13:52.636+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:13:53.107 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:13:52 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5[48018]: 2024-05-15T14:13:52.828+0000 7f5ee1e28700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:19.585073+0000 front 2024-05-15T14:13:19.585099+0000 (oldest deadline 2024-05-15T14:13:48.385527+0000) 2024-05-15T14:13:53.691 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:13:53 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:13:53.352+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.305228+0000 front 2024-05-15T14:13:24.305185+0000 (oldest deadline 2024-05-15T14:13:50.205368+0000) 2024-05-15T14:13:53.691 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:13:53 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:13:53.576+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:13:53.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:13:53 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:13:53.482+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:13:54.090 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:13:53 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:13:53.686+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:13:54.357 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:13:53 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5[48018]: 2024-05-15T14:13:53.867+0000 7f5ee1e28700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:19.585073+0000 front 2024-05-15T14:13:19.585099+0000 (oldest deadline 2024-05-15T14:13:48.385527+0000) 2024-05-15T14:13:54.358 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:13:53 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5[48018]: 2024-05-15T14:13:53.867+0000 7f5ee1e28700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.286827+0000 front 2024-05-15T14:13:31.286583+0000 (oldest deadline 2024-05-15T14:13:53.586554+0000) 2024-05-15T14:13:54.691 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:13:54 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:13:54.594+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:13:54.691 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:13:54 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:13:54.371+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.305228+0000 front 2024-05-15T14:13:24.305185+0000 (oldest deadline 2024-05-15T14:13:50.205368+0000) 2024-05-15T14:13:54.827 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:13:54 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:13:54.485+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:13:55.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:13:54 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:13:54.728+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:13:55.107 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:13:54 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5[48018]: 2024-05-15T14:13:54.826+0000 7f5ee1e28700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:19.585073+0000 front 2024-05-15T14:13:19.585099+0000 (oldest deadline 2024-05-15T14:13:48.385527+0000) 2024-05-15T14:13:55.107 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:13:54 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5[48018]: 2024-05-15T14:13:54.826+0000 7f5ee1e28700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.286827+0000 front 2024-05-15T14:13:31.286583+0000 (oldest deadline 2024-05-15T14:13:53.586554+0000) 2024-05-15T14:13:55.691 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:13:55 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:13:55.362+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.305228+0000 front 2024-05-15T14:13:24.305185+0000 (oldest deadline 2024-05-15T14:13:50.205368+0000) 2024-05-15T14:13:55.691 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:13:55 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:13:55.604+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:13:55.811 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:13:55 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:13:55.511+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:13:55.811 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:13:55 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:13:55.511+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:13:56.090 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:13:55 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:13:55.717+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:13:56.107 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:13:55 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5[48018]: 2024-05-15T14:13:55.808+0000 7f5ee1e28700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:19.585073+0000 front 2024-05-15T14:13:19.585099+0000 (oldest deadline 2024-05-15T14:13:48.385527+0000) 2024-05-15T14:13:56.107 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:13:55 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5[48018]: 2024-05-15T14:13:55.808+0000 7f5ee1e28700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.286827+0000 front 2024-05-15T14:13:31.286583+0000 (oldest deadline 2024-05-15T14:13:53.586554+0000) 2024-05-15T14:13:56.547 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:13:56 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:13:56.119+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:30.097646+0000 front 2024-05-15T14:13:30.097513+0000 (oldest deadline 2024-05-15T14:13:55.397501+0000) 2024-05-15T14:13:56.583 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:13:56 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:13:56.315+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.305228+0000 front 2024-05-15T14:13:24.305185+0000 (oldest deadline 2024-05-15T14:13:50.205368+0000) 2024-05-15T14:13:56.801 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:13:56 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:13:56.546+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:13:56.802 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:13:56 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:13:56.546+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:13:56.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:13:56 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:13:56.582+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:13:56.992 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:13:56 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:13:56.688+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:13:57.107 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:13:56 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5[48018]: 2024-05-15T14:13:56.800+0000 7f5ee1e28700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:19.585073+0000 front 2024-05-15T14:13:19.585099+0000 (oldest deadline 2024-05-15T14:13:48.385527+0000) 2024-05-15T14:13:57.107 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:13:56 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5[48018]: 2024-05-15T14:13:56.800+0000 7f5ee1e28700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.286827+0000 front 2024-05-15T14:13:31.286583+0000 (oldest deadline 2024-05-15T14:13:53.586554+0000) 2024-05-15T14:13:57.290 INFO:journalctl@ceph.osd.4.smithi052.stdout:May 15 14:13:57 smithi052 podman[73861]: 2024-05-15 14:13:57.022822523 +0000 UTC m=+5.485189410 container died a0a6fec869beec7f0451f720f5ccfab5232f74b42f9f44d9963a3a0d94616a06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5) 2024-05-15T14:13:57.563 INFO:journalctl@ceph.osd.4.smithi052.stdout:May 15 14:13:57 smithi052 podman[73861]: 2024-05-15 14:13:57.396438645 +0000 UTC m=+5.858805523 container cleanup a0a6fec869beec7f0451f720f5ccfab5232f74b42f9f44d9963a3a0d94616a06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-4, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True) 2024-05-15T14:13:57.563 INFO:journalctl@ceph.osd.4.smithi052.stdout:May 15 14:13:57 smithi052 bash[73861]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-4 2024-05-15T14:13:57.566 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:13:57 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:13:57.307+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.305228+0000 front 2024-05-15T14:13:24.305185+0000 (oldest deadline 2024-05-15T14:13:50.205368+0000) 2024-05-15T14:13:57.566 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:13:57 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:13:57.307+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.506547+0000 front 2024-05-15T14:13:32.506702+0000 (oldest deadline 2024-05-15T14:13:56.606631+0000) 2024-05-15T14:13:57.582 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:13:57 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:13:57.136+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:30.097646+0000 front 2024-05-15T14:13:30.097513+0000 (oldest deadline 2024-05-15T14:13:55.397501+0000) 2024-05-15T14:13:57.841 INFO:journalctl@ceph.osd.4.smithi052.stdout:May 15 14:13:57 smithi052 podman[73879]: 2024-05-15 14:13:57.647632958 +0000 UTC m=+0.616762584 container remove a0a6fec869beec7f0451f720f5ccfab5232f74b42f9f44d9963a3a0d94616a06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-4, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-15T14:13:57.841 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:13:57 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:13:57.559+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.554523+0000 front 2024-05-15T14:13:31.554496+0000 (oldest deadline 2024-05-15T14:13:56.854739+0000) 2024-05-15T14:13:57.842 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:13:57 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:13:57.708+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:13:57.842 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:13:57 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:13:57.708+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:13:57.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:13:57 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:13:57.581+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:13:57.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:13:57 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:13:57.581+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:13:57.858 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:13:57 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5[48018]: 2024-05-15T14:13:57.764+0000 7f5ee1e28700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:19.585073+0000 front 2024-05-15T14:13:19.585099+0000 (oldest deadline 2024-05-15T14:13:48.385527+0000) 2024-05-15T14:13:57.858 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:13:57 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5[48018]: 2024-05-15T14:13:57.764+0000 7f5ee1e28700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.286827+0000 front 2024-05-15T14:13:31.286583+0000 (oldest deadline 2024-05-15T14:13:53.586554+0000) 2024-05-15T14:13:57.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:13:57 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:13:57.564+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:13:58.309 INFO:journalctl@ceph.osd.4.smithi052.stdout:May 15 14:13:58 smithi052 podman[73976]: 2024-05-15 14:13:57.909781962 +0000 UTC m=+0.018188530 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-15T14:13:58.309 INFO:journalctl@ceph.osd.4.smithi052.stdout:May 15 14:13:58 smithi052 podman[73976]: 2024-05-15 14:13:58.043950647 +0000 UTC m=+0.152357201 container create ab8e485666e322b3c00ed01ba1e735ba4ba2c93d6f70414a3bb1e208143dd26f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-4-deactivate, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-15T14:13:58.309 INFO:journalctl@ceph.osd.4.smithi052.stdout:May 15 14:13:58 smithi052 podman[73976]: 2024-05-15 14:13:58.250974186 +0000 UTC m=+0.359380765 container init ab8e485666e322b3c00ed01ba1e735ba4ba2c93d6f70414a3bb1e208143dd26f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2) 2024-05-15T14:13:58.309 INFO:journalctl@ceph.osd.4.smithi052.stdout:May 15 14:13:58 smithi052 podman[73976]: 2024-05-15 14:13:58.254818292 +0000 UTC m=+0.363224864 container start ab8e485666e322b3c00ed01ba1e735ba4ba2c93d6f70414a3bb1e208143dd26f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866) 2024-05-15T14:13:58.554 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:13:58 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:13:58.150+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:30.097646+0000 front 2024-05-15T14:13:30.097513+0000 (oldest deadline 2024-05-15T14:13:55.397501+0000) 2024-05-15T14:13:58.563 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:13:58 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:13:58.519+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.554523+0000 front 2024-05-15T14:13:31.554496+0000 (oldest deadline 2024-05-15T14:13:56.854739+0000) 2024-05-15T14:13:58.563 INFO:journalctl@ceph.osd.4.smithi052.stdout:May 15 14:13:58 smithi052 podman[73976]: 2024-05-15 14:13:58.332715654 +0000 UTC m=+0.441122235 container attach ab8e485666e322b3c00ed01ba1e735ba4ba2c93d6f70414a3bb1e208143dd26f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD) 2024-05-15T14:13:58.588 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:13:58 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:13:58.289+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.305228+0000 front 2024-05-15T14:13:24.305185+0000 (oldest deadline 2024-05-15T14:13:50.205368+0000) 2024-05-15T14:13:58.588 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:13:58 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:13:58.289+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.506547+0000 front 2024-05-15T14:13:32.506702+0000 (oldest deadline 2024-05-15T14:13:56.606631+0000) 2024-05-15T14:13:58.841 INFO:journalctl@ceph.osd.4.smithi052.stdout:May 15 14:13:58 smithi052 podman[73976]: 2024-05-15 14:13:58.561947696 +0000 UTC m=+0.670354265 container died ab8e485666e322b3c00ed01ba1e735ba4ba2c93d6f70414a3bb1e208143dd26f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-15T14:13:58.841 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:13:58 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:13:58.708+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:13:58.841 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:13:58 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:13:58.708+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:13:58.857 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:13:58 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5[48018]: 2024-05-15T14:13:58.763+0000 7f5ee1e28700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:19.585073+0000 front 2024-05-15T14:13:19.585099+0000 (oldest deadline 2024-05-15T14:13:48.385527+0000) 2024-05-15T14:13:58.857 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:13:58 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5[48018]: 2024-05-15T14:13:58.763+0000 7f5ee1e28700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.286827+0000 front 2024-05-15T14:13:31.286583+0000 (oldest deadline 2024-05-15T14:13:53.586554+0000) 2024-05-15T14:13:58.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:13:58 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:13:58.552+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:13:58.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:13:58 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:13:58.552+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:13:58.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:13:58 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:13:58.586+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:13:58.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:13:58 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:13:58.586+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:13:59.129 INFO:journalctl@ceph.osd.4.smithi052.stdout:May 15 14:13:59 smithi052 podman[74003]: 2024-05-15 14:13:59.112585476 +0000 UTC m=+0.540375199 container remove ab8e485666e322b3c00ed01ba1e735ba4ba2c93d6f70414a3bb1e208143dd26f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-4-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-15T14:13:59.134 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.4.service' 2024-05-15T14:13:59.176 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T14:13:59.176 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-15T14:13:59.177 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-15T14:13:59.177 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.5 2024-05-15T14:13:59.462 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:13:59 smithi081 systemd[1]: Stopping Ceph osd.5 for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T14:13:59.463 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:13:59 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:13:59.153+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:30.097646+0000 front 2024-05-15T14:13:30.097513+0000 (oldest deadline 2024-05-15T14:13:55.397501+0000) 2024-05-15T14:13:59.572 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:13:59 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:13:59.306+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.305228+0000 front 2024-05-15T14:13:24.305185+0000 (oldest deadline 2024-05-15T14:13:50.205368+0000) 2024-05-15T14:13:59.572 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:13:59 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:13:59.306+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.506547+0000 front 2024-05-15T14:13:32.506702+0000 (oldest deadline 2024-05-15T14:13:56.606631+0000) 2024-05-15T14:13:59.779 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:13:59 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:13:59.518+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:13:59.780 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:13:59 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:13:59.518+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:13:59.780 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:13:59 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5[48018]: 2024-05-15T14:13:59.459+0000 7f5ee983c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T14:13:59.780 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:13:59 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5[48018]: 2024-05-15T14:13:59.459+0000 7f5ee983c700 -1 osd.5 101 *** Got signal Terminated *** 2024-05-15T14:13:59.780 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:13:59 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5[48018]: 2024-05-15T14:13:59.459+0000 7f5ee983c700 -1 osd.5 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-15T14:13:59.780 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:13:59 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5[48018]: 2024-05-15T14:13:59.777+0000 7f5ee1e28700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:19.585073+0000 front 2024-05-15T14:13:19.585099+0000 (oldest deadline 2024-05-15T14:13:48.385527+0000) 2024-05-15T14:13:59.781 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:13:59 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5[48018]: 2024-05-15T14:13:59.777+0000 7f5ee1e28700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.286827+0000 front 2024-05-15T14:13:31.286583+0000 (oldest deadline 2024-05-15T14:13:53.586554+0000) 2024-05-15T14:13:59.841 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:13:59 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:13:59.758+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:13:59.841 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:13:59 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:13:59.758+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:13:59.841 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:13:59 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:13:59.543+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.554523+0000 front 2024-05-15T14:13:31.554496+0000 (oldest deadline 2024-05-15T14:13:56.854739+0000) 2024-05-15T14:13:59.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:13:59 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:13:59.570+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:13:59.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:13:59 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:13:59.570+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:00.541 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:00 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:00.181+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:30.097646+0000 front 2024-05-15T14:13:30.097513+0000 (oldest deadline 2024-05-15T14:13:55.397501+0000) 2024-05-15T14:14:00.691 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:00 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:00.332+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.305228+0000 front 2024-05-15T14:13:24.305185+0000 (oldest deadline 2024-05-15T14:13:50.205368+0000) 2024-05-15T14:14:00.691 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:00 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:00.332+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.506547+0000 front 2024-05-15T14:13:32.506702+0000 (oldest deadline 2024-05-15T14:13:56.606631+0000) 2024-05-15T14:14:00.692 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:00 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:00.561+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:00.692 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:00 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:00.561+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:00.857 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:14:00 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5[48018]: 2024-05-15T14:14:00.734+0000 7f5ee1e28700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:19.585073+0000 front 2024-05-15T14:13:19.585099+0000 (oldest deadline 2024-05-15T14:13:48.385527+0000) 2024-05-15T14:14:00.857 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:14:00 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5[48018]: 2024-05-15T14:14:00.734+0000 7f5ee1e28700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.286827+0000 front 2024-05-15T14:13:31.286583+0000 (oldest deadline 2024-05-15T14:13:53.586554+0000) 2024-05-15T14:14:00.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:00 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:00.538+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:00.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:00 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:00.538+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:01.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:00 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:00.758+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:01.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:00 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:00.758+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:01.091 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:00 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:00.593+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.554523+0000 front 2024-05-15T14:13:31.554496+0000 (oldest deadline 2024-05-15T14:13:56.854739+0000) 2024-05-15T14:14:01.573 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:01 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:01.144+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:30.097646+0000 front 2024-05-15T14:13:30.097513+0000 (oldest deadline 2024-05-15T14:13:55.397501+0000) 2024-05-15T14:14:01.691 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:01 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:01.531+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:01.691 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:01 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:01.531+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:01.691 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:01 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:01.324+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.305228+0000 front 2024-05-15T14:13:24.305185+0000 (oldest deadline 2024-05-15T14:13:50.205368+0000) 2024-05-15T14:14:01.692 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:01 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:01.324+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.506547+0000 front 2024-05-15T14:13:32.506702+0000 (oldest deadline 2024-05-15T14:13:56.606631+0000) 2024-05-15T14:14:01.692 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:01 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:01.324+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:36.607339+0000 front 2024-05-15T14:13:36.607414+0000 (oldest deadline 2024-05-15T14:14:01.307079+0000) 2024-05-15T14:14:01.857 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:14:01 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5[48018]: 2024-05-15T14:14:01.753+0000 7f5ee1e28700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:19.585073+0000 front 2024-05-15T14:13:19.585099+0000 (oldest deadline 2024-05-15T14:13:48.385527+0000) 2024-05-15T14:14:01.857 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:14:01 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5[48018]: 2024-05-15T14:14:01.753+0000 7f5ee1e28700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.286827+0000 front 2024-05-15T14:13:31.286583+0000 (oldest deadline 2024-05-15T14:13:53.586554+0000) 2024-05-15T14:14:01.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:01 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:01.571+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:01.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:01 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:01.571+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:01.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:01 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:01.571+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:02.091 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:01 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:01.615+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.554523+0000 front 2024-05-15T14:13:31.554496+0000 (oldest deadline 2024-05-15T14:13:56.854739+0000) 2024-05-15T14:14:02.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:01 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:01.727+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:02.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:01 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:01.727+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:02.580 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:02 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:02.274+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.305228+0000 front 2024-05-15T14:13:24.305185+0000 (oldest deadline 2024-05-15T14:13:50.205368+0000) 2024-05-15T14:14:02.580 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:02 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:02.274+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.506547+0000 front 2024-05-15T14:13:32.506702+0000 (oldest deadline 2024-05-15T14:13:56.606631+0000) 2024-05-15T14:14:02.580 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:02 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:02.274+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:36.607339+0000 front 2024-05-15T14:13:36.607414+0000 (oldest deadline 2024-05-15T14:14:01.307079+0000) 2024-05-15T14:14:02.607 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:02 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:02.138+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:30.097646+0000 front 2024-05-15T14:13:30.097513+0000 (oldest deadline 2024-05-15T14:13:55.397501+0000) 2024-05-15T14:14:02.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:02 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:02.578+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:02.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:02 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:02.578+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:02.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:02 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:02.578+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:37.642201+0000 front 2024-05-15T14:13:37.642108+0000 (oldest deadline 2024-05-15T14:14:02.342241+0000) 2024-05-15T14:14:03.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:02 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:02.723+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:03.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:02 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:02.723+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:03.092 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:02 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:02.616+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.554523+0000 front 2024-05-15T14:13:31.554496+0000 (oldest deadline 2024-05-15T14:13:56.854739+0000) 2024-05-15T14:14:03.092 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:02 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:02.616+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:36.855393+0000 front 2024-05-15T14:13:36.855364+0000 (oldest deadline 2024-05-15T14:14:02.155628+0000) 2024-05-15T14:14:03.107 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:02 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:02.611+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:03.107 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:02 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:02.611+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:03.107 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:02 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:02.611+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:03.108 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:14:02 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5[48018]: 2024-05-15T14:14:02.762+0000 7f5ee1e28700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:19.585073+0000 front 2024-05-15T14:13:19.585099+0000 (oldest deadline 2024-05-15T14:13:48.385527+0000) 2024-05-15T14:14:03.108 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:14:02 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5[48018]: 2024-05-15T14:14:02.762+0000 7f5ee1e28700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.286827+0000 front 2024-05-15T14:13:31.286583+0000 (oldest deadline 2024-05-15T14:13:53.586554+0000) 2024-05-15T14:14:03.604 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:03 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:03.125+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:30.097646+0000 front 2024-05-15T14:13:30.097513+0000 (oldest deadline 2024-05-15T14:13:55.397501+0000) 2024-05-15T14:14:03.691 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:03 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:03.536+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:03.691 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:03 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:03.536+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:03.691 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:03 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:03.536+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:37.642201+0000 front 2024-05-15T14:13:37.642108+0000 (oldest deadline 2024-05-15T14:14:02.342241+0000) 2024-05-15T14:14:03.691 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:03 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:03.316+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.305228+0000 front 2024-05-15T14:13:24.305185+0000 (oldest deadline 2024-05-15T14:13:50.205368+0000) 2024-05-15T14:14:03.692 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:03 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:03.316+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.506547+0000 front 2024-05-15T14:13:32.506702+0000 (oldest deadline 2024-05-15T14:13:56.606631+0000) 2024-05-15T14:14:03.692 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:03 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:03.316+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:36.607339+0000 front 2024-05-15T14:13:36.607414+0000 (oldest deadline 2024-05-15T14:14:01.307079+0000) 2024-05-15T14:14:03.857 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:14:03 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5[48018]: 2024-05-15T14:14:03.747+0000 7f5ee1e28700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:19.585073+0000 front 2024-05-15T14:13:19.585099+0000 (oldest deadline 2024-05-15T14:13:48.385527+0000) 2024-05-15T14:14:03.858 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:14:03 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5[48018]: 2024-05-15T14:14:03.747+0000 7f5ee1e28700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.286827+0000 front 2024-05-15T14:13:31.286583+0000 (oldest deadline 2024-05-15T14:13:53.586554+0000) 2024-05-15T14:14:03.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:03 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:03.602+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:03.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:03 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:03.602+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:03.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:03 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:03.602+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:04.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:03 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:03.728+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:04.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:03 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:03.728+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:04.091 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:03 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:03.626+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.554523+0000 front 2024-05-15T14:13:31.554496+0000 (oldest deadline 2024-05-15T14:13:56.854739+0000) 2024-05-15T14:14:04.092 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:03 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:03.626+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:36.855393+0000 front 2024-05-15T14:13:36.855364+0000 (oldest deadline 2024-05-15T14:14:02.155628+0000) 2024-05-15T14:14:04.357 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:04 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:04.082+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:30.097646+0000 front 2024-05-15T14:13:30.097513+0000 (oldest deadline 2024-05-15T14:13:55.397501+0000) 2024-05-15T14:14:04.560 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:04 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:04.267+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.305228+0000 front 2024-05-15T14:13:24.305185+0000 (oldest deadline 2024-05-15T14:13:50.205368+0000) 2024-05-15T14:14:04.560 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:04 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:04.267+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.506547+0000 front 2024-05-15T14:13:32.506702+0000 (oldest deadline 2024-05-15T14:13:56.606631+0000) 2024-05-15T14:14:04.560 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:04 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:04.267+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:36.607339+0000 front 2024-05-15T14:13:36.607414+0000 (oldest deadline 2024-05-15T14:14:01.307079+0000) 2024-05-15T14:14:04.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:04 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:04.558+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:04.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:04 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:04.558+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:04.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:04 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:04.558+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:37.642201+0000 front 2024-05-15T14:13:37.642108+0000 (oldest deadline 2024-05-15T14:14:02.342241+0000) 2024-05-15T14:14:05.071 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:14:04 smithi081 podman[74473]: 2024-05-15 14:14:04.816436446 +0000 UTC m=+5.448743683 container died e3626aba0ece09279e74b9f6ff2bae205af90e8d4f21641905ff6a2e7a1ba23e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-15T14:14:05.071 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:04 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:04.649+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:05.071 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:04 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:04.649+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:05.071 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:04 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:04.649+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:05.090 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:04 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:04.774+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:05.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:04 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:04.774+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:05.091 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:04 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:04.602+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.554523+0000 front 2024-05-15T14:13:31.554496+0000 (oldest deadline 2024-05-15T14:13:56.854739+0000) 2024-05-15T14:14:05.091 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:04 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:04.602+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:36.855393+0000 front 2024-05-15T14:13:36.855364+0000 (oldest deadline 2024-05-15T14:14:02.155628+0000) 2024-05-15T14:14:05.358 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:14:05 smithi081 podman[74473]: 2024-05-15 14:14:05.177325878 +0000 UTC m=+5.809633109 container cleanup e3626aba0ece09279e74b9f6ff2bae205af90e8d4f21641905ff6a2e7a1ba23e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514) 2024-05-15T14:14:05.358 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:14:05 smithi081 bash[74473]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5 2024-05-15T14:14:05.358 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:05 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:05.068+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:30.097646+0000 front 2024-05-15T14:13:30.097513+0000 (oldest deadline 2024-05-15T14:13:55.397501+0000) 2024-05-15T14:14:05.358 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:05 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:05.068+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.599430+0000 front 2024-05-15T14:13:40.599455+0000 (oldest deadline 2024-05-15T14:14:04.699619+0000) 2024-05-15T14:14:05.521 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:05 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:05.227+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.305228+0000 front 2024-05-15T14:13:24.305185+0000 (oldest deadline 2024-05-15T14:13:50.205368+0000) 2024-05-15T14:14:05.521 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:05 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:05.227+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.506547+0000 front 2024-05-15T14:13:32.506702+0000 (oldest deadline 2024-05-15T14:13:56.606631+0000) 2024-05-15T14:14:05.521 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:05 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:05.227+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:36.607339+0000 front 2024-05-15T14:13:36.607414+0000 (oldest deadline 2024-05-15T14:14:01.307079+0000) 2024-05-15T14:14:05.761 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:14:05 smithi081 podman[74488]: 2024-05-15 14:14:05.445312905 +0000 UTC m=+0.621342328 container remove e3626aba0ece09279e74b9f6ff2bae205af90e8d4f21641905ff6a2e7a1ba23e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-15T14:14:05.761 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:05 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:05.656+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:05.762 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:05 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:05.656+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:05.762 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:05 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:05.656+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:05.841 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:05 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:05.749+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:05.842 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:05 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:05.750+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:05.842 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:05 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:05.750+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:05.842 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:05 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:05.580+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.554523+0000 front 2024-05-15T14:13:31.554496+0000 (oldest deadline 2024-05-15T14:13:56.854739+0000) 2024-05-15T14:14:05.842 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:05 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:05.580+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:36.855393+0000 front 2024-05-15T14:13:36.855364+0000 (oldest deadline 2024-05-15T14:14:02.155628+0000) 2024-05-15T14:14:05.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:05 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:05.519+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:05.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:05 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:05.519+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:05.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:05 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:05.519+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:37.642201+0000 front 2024-05-15T14:13:37.642108+0000 (oldest deadline 2024-05-15T14:14:02.342241+0000) 2024-05-15T14:14:06.033 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:14:05 smithi081 podman[74582]: 2024-05-15 14:14:05.662703704 +0000 UTC m=+0.017533278 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-15T14:14:06.034 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:14:05 smithi081 podman[74582]: 2024-05-15 14:14:05.79123021 +0000 UTC m=+0.146059771 container create a48453beb5273b25ecd4a36c9883aaf6621a6200855de1e50377819becb3018a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240514, io.buildah.version=1.33.5) 2024-05-15T14:14:06.034 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:14:06 smithi081 podman[74582]: 2024-05-15 14:14:06.032142809 +0000 UTC m=+0.386972370 container init a48453beb5273b25ecd4a36c9883aaf6621a6200855de1e50377819becb3018a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-15T14:14:06.334 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:06 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:06.101+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:30.097646+0000 front 2024-05-15T14:13:30.097513+0000 (oldest deadline 2024-05-15T14:13:55.397501+0000) 2024-05-15T14:14:06.334 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:06 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:06.101+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.599430+0000 front 2024-05-15T14:13:40.599455+0000 (oldest deadline 2024-05-15T14:14:04.699619+0000) 2024-05-15T14:14:06.334 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:14:06 smithi081 podman[74582]: 2024-05-15 14:14:06.035437261 +0000 UTC m=+0.390266822 container start a48453beb5273b25ecd4a36c9883aaf6621a6200855de1e50377819becb3018a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T14:14:06.335 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:14:06 smithi081 podman[74582]: 2024-05-15 14:14:06.129981061 +0000 UTC m=+0.484810623 container attach a48453beb5273b25ecd4a36c9883aaf6621a6200855de1e50377819becb3018a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-15T14:14:06.489 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:06 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:06.220+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.305228+0000 front 2024-05-15T14:13:24.305185+0000 (oldest deadline 2024-05-15T14:13:50.205368+0000) 2024-05-15T14:14:06.489 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:06 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:06.220+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.506547+0000 front 2024-05-15T14:13:32.506702+0000 (oldest deadline 2024-05-15T14:13:56.606631+0000) 2024-05-15T14:14:06.489 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:06 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:06.220+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:36.607339+0000 front 2024-05-15T14:13:36.607414+0000 (oldest deadline 2024-05-15T14:14:01.307079+0000) 2024-05-15T14:14:06.607 INFO:journalctl@ceph.osd.5.smithi081.stdout:May 15 14:14:06 smithi081 podman[74582]: 2024-05-15 14:14:06.333017569 +0000 UTC m=+0.687847140 container died a48453beb5273b25ecd4a36c9883aaf6621a6200855de1e50377819becb3018a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux ) 2024-05-15T14:14:06.896 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:06 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:06.679+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:06.896 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:06 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:06.679+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:06.896 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:06 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:06.679+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:06.915 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.5.service' 2024-05-15T14:14:06.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:06 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:06.487+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:06.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:06 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:06.487+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:06.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:06 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:06.487+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:37.642201+0000 front 2024-05-15T14:13:37.642108+0000 (oldest deadline 2024-05-15T14:14:02.342241+0000) 2024-05-15T14:14:06.956 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T14:14:06.956 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-15T14:14:06.956 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-15T14:14:06.957 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.6 2024-05-15T14:14:07.091 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:06 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:06.627+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.554523+0000 front 2024-05-15T14:13:31.554496+0000 (oldest deadline 2024-05-15T14:13:56.854739+0000) 2024-05-15T14:14:07.091 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:06 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:06.627+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:36.855393+0000 front 2024-05-15T14:13:36.855364+0000 (oldest deadline 2024-05-15T14:14:02.155628+0000) 2024-05-15T14:14:07.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:06 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:06.735+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:07.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:06 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:06.735+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:07.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:06 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:06.735+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:07.262 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:07 smithi033 systemd[1]: Stopping Ceph osd.6 for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T14:14:07.262 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:07 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:07.208+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.305228+0000 front 2024-05-15T14:13:24.305185+0000 (oldest deadline 2024-05-15T14:13:50.205368+0000) 2024-05-15T14:14:07.262 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:07 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:07.208+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.506547+0000 front 2024-05-15T14:13:32.506702+0000 (oldest deadline 2024-05-15T14:13:56.606631+0000) 2024-05-15T14:14:07.265 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:07 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:07.208+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:36.607339+0000 front 2024-05-15T14:13:36.607414+0000 (oldest deadline 2024-05-15T14:14:01.307079+0000) 2024-05-15T14:14:07.266 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:07 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:07.259+0000 7f52407c4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T14:14:07.266 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:07 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:07.259+0000 7f52407c4700 -1 osd.6 101 *** Got signal Terminated *** 2024-05-15T14:14:07.266 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:07 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:07.259+0000 7f52407c4700 -1 osd.6 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-15T14:14:07.357 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:07 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:07.087+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:30.097646+0000 front 2024-05-15T14:13:30.097513+0000 (oldest deadline 2024-05-15T14:13:55.397501+0000) 2024-05-15T14:14:07.357 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:07 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:07.087+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.599430+0000 front 2024-05-15T14:13:40.599455+0000 (oldest deadline 2024-05-15T14:14:04.699619+0000) 2024-05-15T14:14:07.691 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:07 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:07.472+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:07.691 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:07 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:07.472+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:07.691 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:07 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:07.472+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:37.642201+0000 front 2024-05-15T14:13:37.642108+0000 (oldest deadline 2024-05-15T14:14:02.342241+0000) 2024-05-15T14:14:08.091 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:07 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:07.668+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.554523+0000 front 2024-05-15T14:13:31.554496+0000 (oldest deadline 2024-05-15T14:13:56.854739+0000) 2024-05-15T14:14:08.091 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:07 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:07.668+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:36.855393+0000 front 2024-05-15T14:13:36.855364+0000 (oldest deadline 2024-05-15T14:14:02.155628+0000) 2024-05-15T14:14:08.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:07 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:07.707+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:08.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:07 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:07.707+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:08.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:07 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:07.707+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:08.092 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:07 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:07.665+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:08.092 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:07 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:07.665+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:08.092 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:07 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:07.665+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:08.357 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:08 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:08.089+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:30.097646+0000 front 2024-05-15T14:13:30.097513+0000 (oldest deadline 2024-05-15T14:13:55.397501+0000) 2024-05-15T14:14:08.357 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:08 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:08.089+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.599430+0000 front 2024-05-15T14:13:40.599455+0000 (oldest deadline 2024-05-15T14:14:04.699619+0000) 2024-05-15T14:14:08.498 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:08 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:08.201+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.305228+0000 front 2024-05-15T14:13:24.305185+0000 (oldest deadline 2024-05-15T14:13:50.205368+0000) 2024-05-15T14:14:08.499 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:08 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:08.201+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.506547+0000 front 2024-05-15T14:13:32.506702+0000 (oldest deadline 2024-05-15T14:13:56.606631+0000) 2024-05-15T14:14:08.499 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:08 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:08.201+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:36.607339+0000 front 2024-05-15T14:13:36.607414+0000 (oldest deadline 2024-05-15T14:14:01.307079+0000) 2024-05-15T14:14:08.796 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:08 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:08.496+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:08.797 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:08 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:08.496+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:08.797 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:08 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:08.496+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:37.642201+0000 front 2024-05-15T14:13:37.642108+0000 (oldest deadline 2024-05-15T14:14:02.342241+0000) 2024-05-15T14:14:09.091 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:08 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:08.665+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.554523+0000 front 2024-05-15T14:13:31.554496+0000 (oldest deadline 2024-05-15T14:13:56.854739+0000) 2024-05-15T14:14:09.091 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:08 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:08.665+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:36.855393+0000 front 2024-05-15T14:13:36.855364+0000 (oldest deadline 2024-05-15T14:14:02.155628+0000) 2024-05-15T14:14:09.092 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:08 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:08.727+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:09.092 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:08 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:08.727+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:09.092 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:08 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:08.727+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:09.103 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:08 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:08.703+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:09.104 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:08 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:08.703+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:09.104 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:08 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:08.703+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:09.357 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:09 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:09.101+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:30.097646+0000 front 2024-05-15T14:13:30.097513+0000 (oldest deadline 2024-05-15T14:13:55.397501+0000) 2024-05-15T14:14:09.357 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:09 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:09.101+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.599430+0000 front 2024-05-15T14:13:40.599455+0000 (oldest deadline 2024-05-15T14:14:04.699619+0000) 2024-05-15T14:14:09.441 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:09 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:09.179+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.305228+0000 front 2024-05-15T14:13:24.305185+0000 (oldest deadline 2024-05-15T14:13:50.205368+0000) 2024-05-15T14:14:09.441 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:09 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:09.179+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.506547+0000 front 2024-05-15T14:13:32.506702+0000 (oldest deadline 2024-05-15T14:13:56.606631+0000) 2024-05-15T14:14:09.442 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:09 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:09.179+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:36.607339+0000 front 2024-05-15T14:13:36.607414+0000 (oldest deadline 2024-05-15T14:14:01.307079+0000) 2024-05-15T14:14:09.779 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:09 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:09.469+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:09.780 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:09 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:09.469+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:09.780 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:09 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:09.469+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:37.642201+0000 front 2024-05-15T14:13:37.642108+0000 (oldest deadline 2024-05-15T14:14:02.342241+0000) 2024-05-15T14:14:10.076 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:09 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:09.753+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:10.077 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:09 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:09.753+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:10.077 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:09 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:09.753+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:10.091 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:09 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:09.635+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.554523+0000 front 2024-05-15T14:13:31.554496+0000 (oldest deadline 2024-05-15T14:13:56.854739+0000) 2024-05-15T14:14:10.091 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:09 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:09.635+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:36.855393+0000 front 2024-05-15T14:13:36.855364+0000 (oldest deadline 2024-05-15T14:14:02.155628+0000) 2024-05-15T14:14:10.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:09 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:09.774+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:10.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:09 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:09.774+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:10.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:09 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:09.774+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:10.357 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:10 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:10.074+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:30.097646+0000 front 2024-05-15T14:13:30.097513+0000 (oldest deadline 2024-05-15T14:13:55.397501+0000) 2024-05-15T14:14:10.357 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:10 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:10.074+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.599430+0000 front 2024-05-15T14:13:40.599455+0000 (oldest deadline 2024-05-15T14:14:04.699619+0000) 2024-05-15T14:14:10.441 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:10 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:10.169+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.305228+0000 front 2024-05-15T14:13:24.305185+0000 (oldest deadline 2024-05-15T14:13:50.205368+0000) 2024-05-15T14:14:10.441 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:10 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:10.169+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.506547+0000 front 2024-05-15T14:13:32.506702+0000 (oldest deadline 2024-05-15T14:13:56.606631+0000) 2024-05-15T14:14:10.442 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:10 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:10.169+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:36.607339+0000 front 2024-05-15T14:13:36.607414+0000 (oldest deadline 2024-05-15T14:14:01.307079+0000) 2024-05-15T14:14:10.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:10 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:10.503+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:10.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:10 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:10.503+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:10.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:10 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:10.503+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:37.642201+0000 front 2024-05-15T14:13:37.642108+0000 (oldest deadline 2024-05-15T14:14:02.342241+0000) 2024-05-15T14:14:11.068 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:10 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:10.793+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:11.069 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:10 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:10.793+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:11.069 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:10 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:10.793+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:11.091 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:10 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:10.674+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.554523+0000 front 2024-05-15T14:13:31.554496+0000 (oldest deadline 2024-05-15T14:13:56.854739+0000) 2024-05-15T14:14:11.091 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:10 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:10.674+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:36.855393+0000 front 2024-05-15T14:13:36.855364+0000 (oldest deadline 2024-05-15T14:14:02.155628+0000) 2024-05-15T14:14:11.091 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:10 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:10.674+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:47.358089+0000 front 2024-05-15T14:13:47.357842+0000 (oldest deadline 2024-05-15T14:14:09.658072+0000) 2024-05-15T14:14:11.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:10 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:10.759+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:11.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:10 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:10.759+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:11.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:10 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:10.759+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:11.357 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:11 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:11.066+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:30.097646+0000 front 2024-05-15T14:13:30.097513+0000 (oldest deadline 2024-05-15T14:13:55.397501+0000) 2024-05-15T14:14:11.357 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:11 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:11.066+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.599430+0000 front 2024-05-15T14:13:40.599455+0000 (oldest deadline 2024-05-15T14:14:04.699619+0000) 2024-05-15T14:14:11.357 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:11 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:11.066+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.101413+0000 front 2024-05-15T14:13:48.101351+0000 (oldest deadline 2024-05-15T14:14:10.401284+0000) 2024-05-15T14:14:11.493 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:11 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:11.196+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.305228+0000 front 2024-05-15T14:13:24.305185+0000 (oldest deadline 2024-05-15T14:13:50.205368+0000) 2024-05-15T14:14:11.493 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:11 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:11.196+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.506547+0000 front 2024-05-15T14:13:32.506702+0000 (oldest deadline 2024-05-15T14:13:56.606631+0000) 2024-05-15T14:14:11.493 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:11 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:11.196+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:36.607339+0000 front 2024-05-15T14:13:36.607414+0000 (oldest deadline 2024-05-15T14:14:01.307079+0000) 2024-05-15T14:14:11.494 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:11 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:11.196+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:46.508649+0000 front 2024-05-15T14:13:46.508578+0000 (oldest deadline 2024-05-15T14:14:10.608675+0000) 2024-05-15T14:14:11.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:11 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:11.492+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:11.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:11 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:11.492+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:11.942 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:11 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:11.492+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:37.642201+0000 front 2024-05-15T14:13:37.642108+0000 (oldest deadline 2024-05-15T14:14:02.342241+0000) 2024-05-15T14:14:12.066 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:11 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:11.759+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:12.066 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:11 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:11.759+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:12.066 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:11 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:11.759+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:12.091 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:11 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:11.689+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.554523+0000 front 2024-05-15T14:13:31.554496+0000 (oldest deadline 2024-05-15T14:13:56.854739+0000) 2024-05-15T14:14:12.091 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:11 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:11.689+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:36.855393+0000 front 2024-05-15T14:13:36.855364+0000 (oldest deadline 2024-05-15T14:14:02.155628+0000) 2024-05-15T14:14:12.091 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:11 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:11.689+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:47.358089+0000 front 2024-05-15T14:13:47.357842+0000 (oldest deadline 2024-05-15T14:14:09.658072+0000) 2024-05-15T14:14:12.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:11 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:11.804+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:12.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:11 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:11.804+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:12.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:11 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:11.804+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:12.357 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:12 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:12.063+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:30.097646+0000 front 2024-05-15T14:13:30.097513+0000 (oldest deadline 2024-05-15T14:13:55.397501+0000) 2024-05-15T14:14:12.358 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:12 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:12.063+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.599430+0000 front 2024-05-15T14:13:40.599455+0000 (oldest deadline 2024-05-15T14:14:04.699619+0000) 2024-05-15T14:14:12.358 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:12 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:12.063+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.101413+0000 front 2024-05-15T14:13:48.101351+0000 (oldest deadline 2024-05-15T14:14:10.401284+0000) 2024-05-15T14:14:12.459 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:12 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:12.206+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.305228+0000 front 2024-05-15T14:13:24.305185+0000 (oldest deadline 2024-05-15T14:13:50.205368+0000) 2024-05-15T14:14:12.459 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:12 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:12.206+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.506547+0000 front 2024-05-15T14:13:32.506702+0000 (oldest deadline 2024-05-15T14:13:56.606631+0000) 2024-05-15T14:14:12.459 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:12 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:12.206+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:36.607339+0000 front 2024-05-15T14:13:36.607414+0000 (oldest deadline 2024-05-15T14:14:01.307079+0000) 2024-05-15T14:14:12.460 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:12 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6[57527]: 2024-05-15T14:14:12.206+0000 7f5238db0700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:46.508649+0000 front 2024-05-15T14:13:46.508578+0000 (oldest deadline 2024-05-15T14:14:10.608675+0000) 2024-05-15T14:14:12.938 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:12 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:12.457+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:12.938 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:12 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:12.457+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:12.938 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:12 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:12.457+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:37.642201+0000 front 2024-05-15T14:13:37.642108+0000 (oldest deadline 2024-05-15T14:14:02.342241+0000) 2024-05-15T14:14:12.939 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:12 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:12.457+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:46.443309+0000 front 2024-05-15T14:13:46.443491+0000 (oldest deadline 2024-05-15T14:14:11.743363+0000) 2024-05-15T14:14:12.939 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:12 smithi033 podman[103983]: 2024-05-15 14:14:12.66397068 +0000 UTC m=+5.503758286 container died 3d29df46e4d589ab5df2e6176b1c81b17da6a399ce31ce88569a7cdd6e00a963 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-15T14:14:13.038 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:12 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:12.782+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:13.038 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:12 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:12.782+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:13.038 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:12 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:12.782+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:13.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:12 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:12.836+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:13.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:12 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:12.836+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:13.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:12 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:12.836+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:13.092 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:12 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:12.662+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.554523+0000 front 2024-05-15T14:13:31.554496+0000 (oldest deadline 2024-05-15T14:13:56.854739+0000) 2024-05-15T14:14:13.092 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:12 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:12.662+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:36.855393+0000 front 2024-05-15T14:13:36.855364+0000 (oldest deadline 2024-05-15T14:14:02.155628+0000) 2024-05-15T14:14:13.092 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:12 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:12.662+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:47.358089+0000 front 2024-05-15T14:13:47.357842+0000 (oldest deadline 2024-05-15T14:14:09.658072+0000) 2024-05-15T14:14:13.191 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:13 smithi033 podman[103983]: 2024-05-15 14:14:13.109568639 +0000 UTC m=+5.949356234 container cleanup 3d29df46e4d589ab5df2e6176b1c81b17da6a399ce31ce88569a7cdd6e00a963 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240514, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-15T14:14:13.191 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:13 smithi033 bash[103983]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6 2024-05-15T14:14:13.357 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:13 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:13.035+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:30.097646+0000 front 2024-05-15T14:13:30.097513+0000 (oldest deadline 2024-05-15T14:13:55.397501+0000) 2024-05-15T14:14:13.357 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:13 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:13.035+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.599430+0000 front 2024-05-15T14:13:40.599455+0000 (oldest deadline 2024-05-15T14:14:04.699619+0000) 2024-05-15T14:14:13.357 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:13 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:13.035+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.101413+0000 front 2024-05-15T14:13:48.101351+0000 (oldest deadline 2024-05-15T14:14:10.401284+0000) 2024-05-15T14:14:13.691 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:13 smithi033 podman[104010]: 2024-05-15 14:14:13.411000014 +0000 UTC m=+0.747824934 container remove 3d29df46e4d589ab5df2e6176b1c81b17da6a399ce31ce88569a7cdd6e00a963 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD) 2024-05-15T14:14:13.691 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:13 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:13.504+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:13.691 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:13 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:13.504+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:13.691 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:13 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:13.504+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:37.642201+0000 front 2024-05-15T14:13:37.642108+0000 (oldest deadline 2024-05-15T14:14:02.342241+0000) 2024-05-15T14:14:13.692 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:13 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:13.504+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:46.443309+0000 front 2024-05-15T14:13:46.443491+0000 (oldest deadline 2024-05-15T14:14:11.743363+0000) 2024-05-15T14:14:14.024 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:13 smithi033 podman[104103]: 2024-05-15 14:14:13.651314111 +0000 UTC m=+0.017538160 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-15T14:14:14.024 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:13 smithi033 podman[104103]: 2024-05-15 14:14:13.795616311 +0000 UTC m=+0.161840360 container create 7f74a0f7c72f5eaaa745cf680190d20d1bad3509885893f2a1a7a2d57e5ad5f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-05-15T14:14:14.024 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:13 smithi033 podman[104103]: 2024-05-15 14:14:13.9476815 +0000 UTC m=+0.313905549 container init 7f74a0f7c72f5eaaa745cf680190d20d1bad3509885893f2a1a7a2d57e5ad5f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-15T14:14:14.025 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:13 smithi033 podman[104103]: 2024-05-15 14:14:13.950976217 +0000 UTC m=+0.317200268 container start 7f74a0f7c72f5eaaa745cf680190d20d1bad3509885893f2a1a7a2d57e5ad5f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T14:14:14.025 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:14 smithi033 podman[104103]: 2024-05-15 14:14:14.022737244 +0000 UTC m=+0.388961320 container attach 7f74a0f7c72f5eaaa745cf680190d20d1bad3509885893f2a1a7a2d57e5ad5f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-15T14:14:14.068 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:13 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:13.773+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:14.068 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:13 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:13.773+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:14.069 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:13 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:13.773+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:14.091 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:13 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:13.655+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.554523+0000 front 2024-05-15T14:13:31.554496+0000 (oldest deadline 2024-05-15T14:13:56.854739+0000) 2024-05-15T14:14:14.091 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:13 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:13.655+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:36.855393+0000 front 2024-05-15T14:13:36.855364+0000 (oldest deadline 2024-05-15T14:14:02.155628+0000) 2024-05-15T14:14:14.091 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:13 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:13.655+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:47.358089+0000 front 2024-05-15T14:13:47.357842+0000 (oldest deadline 2024-05-15T14:14:09.658072+0000) 2024-05-15T14:14:14.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:13 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:13.848+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:14.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:13 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:13.848+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:14.092 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:13 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:13.848+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:14.092 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:13 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:13.848+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.671290+0000 front 2024-05-15T14:13:48.671390+0000 (oldest deadline 2024-05-15T14:14:13.371393+0000) 2024-05-15T14:14:14.357 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:14 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:14.066+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:30.097646+0000 front 2024-05-15T14:13:30.097513+0000 (oldest deadline 2024-05-15T14:13:55.397501+0000) 2024-05-15T14:14:14.357 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:14 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:14.066+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.599430+0000 front 2024-05-15T14:13:40.599455+0000 (oldest deadline 2024-05-15T14:14:04.699619+0000) 2024-05-15T14:14:14.358 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:14 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:14.066+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.101413+0000 front 2024-05-15T14:13:48.101351+0000 (oldest deadline 2024-05-15T14:14:10.401284+0000) 2024-05-15T14:14:14.441 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 15 14:14:14 smithi033 podman[104103]: 2024-05-15 14:14:14.259631381 +0000 UTC m=+0.625855444 container died 7f74a0f7c72f5eaaa745cf680190d20d1bad3509885893f2a1a7a2d57e5ad5f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-6-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, ceph=True) 2024-05-15T14:14:14.785 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:14 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:14.524+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:14.786 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:14 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:14.524+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:14.786 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:14 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:14.524+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:37.642201+0000 front 2024-05-15T14:13:37.642108+0000 (oldest deadline 2024-05-15T14:14:02.342241+0000) 2024-05-15T14:14:14.786 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:14 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:14.524+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:46.443309+0000 front 2024-05-15T14:13:46.443491+0000 (oldest deadline 2024-05-15T14:14:11.743363+0000) 2024-05-15T14:14:14.809 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.6.service' 2024-05-15T14:14:14.852 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T14:14:14.852 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-15T14:14:14.853 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-15T14:14:14.853 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.7 2024-05-15T14:14:14.896 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:14 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:14.640+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.554523+0000 front 2024-05-15T14:13:31.554496+0000 (oldest deadline 2024-05-15T14:13:56.854739+0000) 2024-05-15T14:14:14.896 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:14 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:14.640+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:36.855393+0000 front 2024-05-15T14:13:36.855364+0000 (oldest deadline 2024-05-15T14:14:02.155628+0000) 2024-05-15T14:14:14.896 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:14 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:14.640+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:47.358089+0000 front 2024-05-15T14:13:47.357842+0000 (oldest deadline 2024-05-15T14:14:09.658072+0000) 2024-05-15T14:14:14.897 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:14 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:14.854+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:14.897 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:14 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:14.854+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:14.897 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:14 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:14.854+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:14.897 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:14 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:14.854+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.671290+0000 front 2024-05-15T14:13:48.671390+0000 (oldest deadline 2024-05-15T14:14:13.371393+0000) 2024-05-15T14:14:15.107 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:14 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:14.746+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:15.107 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:14 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:14.746+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:15.107 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:14 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:14.746+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:15.341 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:14 smithi052 systemd[1]: Stopping Ceph osd.7 for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T14:14:15.341 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:15 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:15.138+0000 7f0767526700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T14:14:15.341 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:15 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:15.138+0000 7f0767526700 -1 osd.7 101 *** Got signal Terminated *** 2024-05-15T14:14:15.341 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:15 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:15.138+0000 7f0767526700 -1 osd.7 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-15T14:14:15.607 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:15 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:15.109+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:30.097646+0000 front 2024-05-15T14:13:30.097513+0000 (oldest deadline 2024-05-15T14:13:55.397501+0000) 2024-05-15T14:14:15.607 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:15 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:15.109+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.599430+0000 front 2024-05-15T14:13:40.599455+0000 (oldest deadline 2024-05-15T14:14:04.699619+0000) 2024-05-15T14:14:15.608 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:15 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:15.109+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.101413+0000 front 2024-05-15T14:13:48.101351+0000 (oldest deadline 2024-05-15T14:14:10.401284+0000) 2024-05-15T14:14:15.897 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:15 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:15.606+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.554523+0000 front 2024-05-15T14:13:31.554496+0000 (oldest deadline 2024-05-15T14:13:56.854739+0000) 2024-05-15T14:14:15.898 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:15 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:15.606+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:36.855393+0000 front 2024-05-15T14:13:36.855364+0000 (oldest deadline 2024-05-15T14:14:02.155628+0000) 2024-05-15T14:14:15.898 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:15 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:15.606+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:47.358089+0000 front 2024-05-15T14:13:47.357842+0000 (oldest deadline 2024-05-15T14:14:09.658072+0000) 2024-05-15T14:14:15.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:15 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:15.526+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:15.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:15 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:15.526+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:15.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:15 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:15.526+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:37.642201+0000 front 2024-05-15T14:13:37.642108+0000 (oldest deadline 2024-05-15T14:14:02.342241+0000) 2024-05-15T14:14:15.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:15 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:15.526+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:46.443309+0000 front 2024-05-15T14:13:46.443491+0000 (oldest deadline 2024-05-15T14:14:11.743363+0000) 2024-05-15T14:14:16.107 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:15 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:15.720+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:16.107 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:15 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:15.720+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:16.107 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:15 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:15.720+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:16.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:15 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:15.895+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:16.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:15 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:15.895+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:16.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:15 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:15.895+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:16.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:15 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:15.895+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.671290+0000 front 2024-05-15T14:13:48.671390+0000 (oldest deadline 2024-05-15T14:14:13.371393+0000) 2024-05-15T14:14:16.607 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:16 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:16.149+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:30.097646+0000 front 2024-05-15T14:13:30.097513+0000 (oldest deadline 2024-05-15T14:13:55.397501+0000) 2024-05-15T14:14:16.607 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:16 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:16.149+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.599430+0000 front 2024-05-15T14:13:40.599455+0000 (oldest deadline 2024-05-15T14:14:04.699619+0000) 2024-05-15T14:14:16.608 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:16 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:16.149+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.101413+0000 front 2024-05-15T14:13:48.101351+0000 (oldest deadline 2024-05-15T14:14:10.401284+0000) 2024-05-15T14:14:16.882 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:16 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:16.589+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.554523+0000 front 2024-05-15T14:13:31.554496+0000 (oldest deadline 2024-05-15T14:13:56.854739+0000) 2024-05-15T14:14:16.882 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:16 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:16.589+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:36.855393+0000 front 2024-05-15T14:13:36.855364+0000 (oldest deadline 2024-05-15T14:14:02.155628+0000) 2024-05-15T14:14:16.882 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:16 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:16.589+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:47.358089+0000 front 2024-05-15T14:13:47.357842+0000 (oldest deadline 2024-05-15T14:14:09.658072+0000) 2024-05-15T14:14:16.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:16 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:16.571+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:16.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:16 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:16.571+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:16.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:16 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:16.571+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:37.642201+0000 front 2024-05-15T14:13:37.642108+0000 (oldest deadline 2024-05-15T14:14:02.342241+0000) 2024-05-15T14:14:16.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:16 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:16.571+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:46.443309+0000 front 2024-05-15T14:13:46.443491+0000 (oldest deadline 2024-05-15T14:14:11.743363+0000) 2024-05-15T14:14:17.107 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:16 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:16.720+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:17.107 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:16 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:16.720+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:17.107 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:16 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:16.720+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:17.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:16 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:16.880+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:17.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:16 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:16.880+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:17.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:16 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:16.880+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:17.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:16 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:16.880+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.671290+0000 front 2024-05-15T14:13:48.671390+0000 (oldest deadline 2024-05-15T14:14:13.371393+0000) 2024-05-15T14:14:17.607 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:17 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:17.139+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:30.097646+0000 front 2024-05-15T14:13:30.097513+0000 (oldest deadline 2024-05-15T14:13:55.397501+0000) 2024-05-15T14:14:17.608 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:17 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:17.139+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.599430+0000 front 2024-05-15T14:13:40.599455+0000 (oldest deadline 2024-05-15T14:14:04.699619+0000) 2024-05-15T14:14:17.608 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:17 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:17.139+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.101413+0000 front 2024-05-15T14:13:48.101351+0000 (oldest deadline 2024-05-15T14:14:10.401284+0000) 2024-05-15T14:14:17.914 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:17 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:17.624+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.554523+0000 front 2024-05-15T14:13:31.554496+0000 (oldest deadline 2024-05-15T14:13:56.854739+0000) 2024-05-15T14:14:17.915 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:17 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:17.624+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:36.855393+0000 front 2024-05-15T14:13:36.855364+0000 (oldest deadline 2024-05-15T14:14:02.155628+0000) 2024-05-15T14:14:17.915 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:17 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:17.624+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:47.358089+0000 front 2024-05-15T14:13:47.357842+0000 (oldest deadline 2024-05-15T14:14:09.658072+0000) 2024-05-15T14:14:17.915 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:17 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:17.624+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:56.059783+0000 front 2024-05-15T14:13:56.059835+0000 (oldest deadline 2024-05-15T14:14:17.159941+0000) 2024-05-15T14:14:17.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:17 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:17.546+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:17.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:17 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:17.546+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:17.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:17 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:17.546+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:37.642201+0000 front 2024-05-15T14:13:37.642108+0000 (oldest deadline 2024-05-15T14:14:02.342241+0000) 2024-05-15T14:14:17.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:17 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:17.546+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:46.443309+0000 front 2024-05-15T14:13:46.443491+0000 (oldest deadline 2024-05-15T14:14:11.743363+0000) 2024-05-15T14:14:18.107 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:17 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:17.691+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:18.108 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:17 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:17.691+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:18.108 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:17 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:17.691+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:18.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:17 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:17.912+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:18.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:17 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:17.912+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:18.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:17 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:17.912+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:18.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:17 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:17.912+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.671290+0000 front 2024-05-15T14:13:48.671390+0000 (oldest deadline 2024-05-15T14:14:13.371393+0000) 2024-05-15T14:14:18.607 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:18 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:18.184+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:30.097646+0000 front 2024-05-15T14:13:30.097513+0000 (oldest deadline 2024-05-15T14:13:55.397501+0000) 2024-05-15T14:14:18.608 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:18 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:18.184+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.599430+0000 front 2024-05-15T14:13:40.599455+0000 (oldest deadline 2024-05-15T14:14:04.699619+0000) 2024-05-15T14:14:18.608 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:18 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:18.184+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.101413+0000 front 2024-05-15T14:13:48.101351+0000 (oldest deadline 2024-05-15T14:14:10.401284+0000) 2024-05-15T14:14:18.608 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:18 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:18.184+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:56.302327+0000 front 2024-05-15T14:13:56.302374+0000 (oldest deadline 2024-05-15T14:14:17.402501+0000) 2024-05-15T14:14:18.930 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:18 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:18.600+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.554523+0000 front 2024-05-15T14:13:31.554496+0000 (oldest deadline 2024-05-15T14:13:56.854739+0000) 2024-05-15T14:14:18.930 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:18 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:18.600+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:36.855393+0000 front 2024-05-15T14:13:36.855364+0000 (oldest deadline 2024-05-15T14:14:02.155628+0000) 2024-05-15T14:14:18.931 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:18 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:18.600+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:47.358089+0000 front 2024-05-15T14:13:47.357842+0000 (oldest deadline 2024-05-15T14:14:09.658072+0000) 2024-05-15T14:14:18.931 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:18 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:18.600+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:56.059783+0000 front 2024-05-15T14:13:56.059835+0000 (oldest deadline 2024-05-15T14:14:17.159941+0000) 2024-05-15T14:14:18.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:18 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:18.555+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:18.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:18 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:18.555+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:18.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:18 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:18.555+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:37.642201+0000 front 2024-05-15T14:13:37.642108+0000 (oldest deadline 2024-05-15T14:14:02.342241+0000) 2024-05-15T14:14:18.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:18 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:18.555+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:46.443309+0000 front 2024-05-15T14:13:46.443491+0000 (oldest deadline 2024-05-15T14:14:11.743363+0000) 2024-05-15T14:14:19.107 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:18 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:18.725+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:19.108 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:18 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:18.725+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:19.108 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:18 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:18.725+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:19.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:18 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:18.928+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:19.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:18 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:18.928+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:19.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:18 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:18.928+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:19.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:18 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:18.928+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.671290+0000 front 2024-05-15T14:13:48.671390+0000 (oldest deadline 2024-05-15T14:14:13.371393+0000) 2024-05-15T14:14:19.607 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:19 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:19.201+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:30.097646+0000 front 2024-05-15T14:13:30.097513+0000 (oldest deadline 2024-05-15T14:13:55.397501+0000) 2024-05-15T14:14:19.607 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:19 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:19.201+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.599430+0000 front 2024-05-15T14:13:40.599455+0000 (oldest deadline 2024-05-15T14:14:04.699619+0000) 2024-05-15T14:14:19.608 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:19 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:19.201+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.101413+0000 front 2024-05-15T14:13:48.101351+0000 (oldest deadline 2024-05-15T14:14:10.401284+0000) 2024-05-15T14:14:19.608 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:19 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:19.201+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:56.302327+0000 front 2024-05-15T14:13:56.302374+0000 (oldest deadline 2024-05-15T14:14:17.402501+0000) 2024-05-15T14:14:19.929 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:19 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:19.649+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.554523+0000 front 2024-05-15T14:13:31.554496+0000 (oldest deadline 2024-05-15T14:13:56.854739+0000) 2024-05-15T14:14:19.930 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:19 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:19.649+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:36.855393+0000 front 2024-05-15T14:13:36.855364+0000 (oldest deadline 2024-05-15T14:14:02.155628+0000) 2024-05-15T14:14:19.930 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:19 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:19.649+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:47.358089+0000 front 2024-05-15T14:13:47.357842+0000 (oldest deadline 2024-05-15T14:14:09.658072+0000) 2024-05-15T14:14:19.930 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:19 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7[54515]: 2024-05-15T14:14:19.649+0000 7f0763b1a700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:56.059783+0000 front 2024-05-15T14:13:56.059835+0000 (oldest deadline 2024-05-15T14:14:17.159941+0000) 2024-05-15T14:14:19.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:19 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:19.519+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:19.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:19 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:19.519+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:19.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:19 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:19.519+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:37.642201+0000 front 2024-05-15T14:13:37.642108+0000 (oldest deadline 2024-05-15T14:14:02.342241+0000) 2024-05-15T14:14:19.942 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:19 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:19.519+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:46.443309+0000 front 2024-05-15T14:13:46.443491+0000 (oldest deadline 2024-05-15T14:14:11.743363+0000) 2024-05-15T14:14:20.107 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:19 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:19.681+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:20.107 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:19 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:19.681+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:20.108 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:19 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:19.681+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:20.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:19 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:19.927+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:20.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:19 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:19.927+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:20.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:19 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:19.927+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:20.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:19 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:19.927+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.671290+0000 front 2024-05-15T14:13:48.671390+0000 (oldest deadline 2024-05-15T14:14:13.371393+0000) 2024-05-15T14:14:20.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:19 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:19.927+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:53.371760+0000 front 2024-05-15T14:13:53.371804+0000 (oldest deadline 2024-05-15T14:14:19.172547+0000) 2024-05-15T14:14:20.607 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:20 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:20.214+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:30.097646+0000 front 2024-05-15T14:13:30.097513+0000 (oldest deadline 2024-05-15T14:13:55.397501+0000) 2024-05-15T14:14:20.608 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:20 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:20.214+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.599430+0000 front 2024-05-15T14:13:40.599455+0000 (oldest deadline 2024-05-15T14:14:04.699619+0000) 2024-05-15T14:14:20.608 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:20 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:20.214+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.101413+0000 front 2024-05-15T14:13:48.101351+0000 (oldest deadline 2024-05-15T14:14:10.401284+0000) 2024-05-15T14:14:20.608 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:20 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:20.214+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:56.302327+0000 front 2024-05-15T14:13:56.302374+0000 (oldest deadline 2024-05-15T14:14:17.402501+0000) 2024-05-15T14:14:20.759 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:20 smithi052 podman[74083]: 2024-05-15 14:14:20.498802047 +0000 UTC m=+5.449513515 container died c0e3cc5f40b9bd161ed30eac755d6c94e4e08a54bbc07f0cdf6e9cfc02bffa94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T14:14:20.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:20 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:20.540+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:20.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:20 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:20.540+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:20.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:20 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:20.540+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:37.642201+0000 front 2024-05-15T14:13:37.642108+0000 (oldest deadline 2024-05-15T14:14:02.342241+0000) 2024-05-15T14:14:20.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:20 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:20.540+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:46.443309+0000 front 2024-05-15T14:13:46.443491+0000 (oldest deadline 2024-05-15T14:14:11.743363+0000) 2024-05-15T14:14:21.091 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:20 smithi052 podman[74083]: 2024-05-15 14:14:20.848083673 +0000 UTC m=+5.798795133 container cleanup c0e3cc5f40b9bd161ed30eac755d6c94e4e08a54bbc07f0cdf6e9cfc02bffa94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514) 2024-05-15T14:14:21.091 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:20 smithi052 bash[74083]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7 2024-05-15T14:14:21.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:20 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:20.970+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:21.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:20 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:20.970+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:21.092 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:20 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:20.970+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:21.092 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:20 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:20.970+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.671290+0000 front 2024-05-15T14:13:48.671390+0000 (oldest deadline 2024-05-15T14:14:13.371393+0000) 2024-05-15T14:14:21.092 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:20 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:20.970+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:53.371760+0000 front 2024-05-15T14:13:53.371804+0000 (oldest deadline 2024-05-15T14:14:19.172547+0000) 2024-05-15T14:14:21.107 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:20 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:20.634+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:21.107 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:20 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:20.634+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:21.107 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:20 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:20.634+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:21.591 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:21 smithi052 podman[74102]: 2024-05-15 14:14:21.291156602 +0000 UTC m=+0.787807372 container remove c0e3cc5f40b9bd161ed30eac755d6c94e4e08a54bbc07f0cdf6e9cfc02bffa94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-15T14:14:21.607 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:21 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:21.217+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:30.097646+0000 front 2024-05-15T14:13:30.097513+0000 (oldest deadline 2024-05-15T14:13:55.397501+0000) 2024-05-15T14:14:21.607 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:21 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:21.217+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.599430+0000 front 2024-05-15T14:13:40.599455+0000 (oldest deadline 2024-05-15T14:14:04.699619+0000) 2024-05-15T14:14:21.607 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:21 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:21.217+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.101413+0000 front 2024-05-15T14:13:48.101351+0000 (oldest deadline 2024-05-15T14:14:10.401284+0000) 2024-05-15T14:14:21.608 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:21 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:21.217+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:56.302327+0000 front 2024-05-15T14:13:56.302374+0000 (oldest deadline 2024-05-15T14:14:17.402501+0000) 2024-05-15T14:14:21.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:21 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:21.569+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:21.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:21 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:21.569+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:21.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:21 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:21.569+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:37.642201+0000 front 2024-05-15T14:13:37.642108+0000 (oldest deadline 2024-05-15T14:14:02.342241+0000) 2024-05-15T14:14:21.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:21 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:21.569+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:46.443309+0000 front 2024-05-15T14:13:46.443491+0000 (oldest deadline 2024-05-15T14:14:11.743363+0000) 2024-05-15T14:14:21.953 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:21 smithi052 podman[74189]: 2024-05-15 14:14:21.536861057 +0000 UTC m=+0.018025885 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-15T14:14:21.953 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:21 smithi052 podman[74189]: 2024-05-15 14:14:21.641977584 +0000 UTC m=+0.123142391 container create a13870e99a68670ad0ad973939b9b096b5280aeab05fcd50558b7f8e367c3a00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-15T14:14:21.953 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:21 smithi052 podman[74189]: 2024-05-15 14:14:21.852809322 +0000 UTC m=+0.333974130 container init a13870e99a68670ad0ad973939b9b096b5280aeab05fcd50558b7f8e367c3a00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514) 2024-05-15T14:14:21.953 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:21 smithi052 podman[74189]: 2024-05-15 14:14:21.855981748 +0000 UTC m=+0.337146563 container start a13870e99a68670ad0ad973939b9b096b5280aeab05fcd50558b7f8e367c3a00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS) 2024-05-15T14:14:22.107 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:21 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:21.659+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:22.107 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:21 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:21.659+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:22.108 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:21 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:21.659+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:22.108 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:21 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:21.659+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:55.816245+0000 front 2024-05-15T14:13:55.816144+0000 (oldest deadline 2024-05-15T14:14:21.016641+0000) 2024-05-15T14:14:22.341 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:21 smithi052 podman[74189]: 2024-05-15 14:14:21.969613237 +0000 UTC m=+0.450778047 container attach a13870e99a68670ad0ad973939b9b096b5280aeab05fcd50558b7f8e367c3a00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7-deactivate, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-15T14:14:22.341 INFO:journalctl@ceph.osd.7.smithi052.stdout:May 15 14:14:22 smithi052 podman[74189]: 2024-05-15 14:14:22.16033551 +0000 UTC m=+0.641500318 container died a13870e99a68670ad0ad973939b9b096b5280aeab05fcd50558b7f8e367c3a00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-15T14:14:22.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:21 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:21.950+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:22.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:21 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:21.950+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:22.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:21 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:21.950+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:22.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:21 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:21.950+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.671290+0000 front 2024-05-15T14:13:48.671390+0000 (oldest deadline 2024-05-15T14:14:13.371393+0000) 2024-05-15T14:14:22.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:21 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:21.950+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:53.371760+0000 front 2024-05-15T14:13:53.371804+0000 (oldest deadline 2024-05-15T14:14:19.172547+0000) 2024-05-15T14:14:22.607 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:22 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:22.206+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:30.097646+0000 front 2024-05-15T14:13:30.097513+0000 (oldest deadline 2024-05-15T14:13:55.397501+0000) 2024-05-15T14:14:22.608 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:22 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:22.206+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.599430+0000 front 2024-05-15T14:13:40.599455+0000 (oldest deadline 2024-05-15T14:14:04.699619+0000) 2024-05-15T14:14:22.608 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:22 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:22.206+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.101413+0000 front 2024-05-15T14:13:48.101351+0000 (oldest deadline 2024-05-15T14:14:10.401284+0000) 2024-05-15T14:14:22.608 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:22 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:22.206+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:56.302327+0000 front 2024-05-15T14:13:56.302374+0000 (oldest deadline 2024-05-15T14:14:17.402501+0000) 2024-05-15T14:14:22.752 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.7.service' 2024-05-15T14:14:22.831 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T14:14:22.831 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-15T14:14:22.832 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-15T14:14:22.832 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.8 2024-05-15T14:14:22.882 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:22 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:22.614+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:22.882 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:22 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:22.614+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:22.884 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:22 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:22.614+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:22.884 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:22 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:22.615+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:55.816245+0000 front 2024-05-15T14:13:55.816144+0000 (oldest deadline 2024-05-15T14:14:21.016641+0000) 2024-05-15T14:14:22.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:22 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:22.613+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:22.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:22 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:22.613+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:22.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:22 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:22.613+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:37.642201+0000 front 2024-05-15T14:13:37.642108+0000 (oldest deadline 2024-05-15T14:14:02.342241+0000) 2024-05-15T14:14:22.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:22 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:22.613+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:46.443309+0000 front 2024-05-15T14:13:46.443491+0000 (oldest deadline 2024-05-15T14:14:11.743363+0000) 2024-05-15T14:14:22.942 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:22 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:22.613+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:56.345295+0000 front 2024-05-15T14:13:56.345229+0000 (oldest deadline 2024-05-15T14:14:22.245016+0000) 2024-05-15T14:14:23.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:22 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:22.919+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:23.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:22 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:22.919+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:23.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:22 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:22.919+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:23.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:22 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:22.919+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.671290+0000 front 2024-05-15T14:13:48.671390+0000 (oldest deadline 2024-05-15T14:14:13.371393+0000) 2024-05-15T14:14:23.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:22 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:22.919+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:53.371760+0000 front 2024-05-15T14:13:53.371804+0000 (oldest deadline 2024-05-15T14:14:19.172547+0000) 2024-05-15T14:14:23.148 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:22 smithi081 systemd[1]: Stopping Ceph osd.8 for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T14:14:23.607 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:23 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:23.146+0000 7f0a642a1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T14:14:23.608 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:23 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:23.146+0000 7f0a642a1700 -1 osd.8 101 *** Got signal Terminated *** 2024-05-15T14:14:23.608 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:23 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:23.146+0000 7f0a642a1700 -1 osd.8 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-15T14:14:23.608 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:23 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:23.214+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:30.097646+0000 front 2024-05-15T14:13:30.097513+0000 (oldest deadline 2024-05-15T14:13:55.397501+0000) 2024-05-15T14:14:23.608 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:23 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:23.214+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.599430+0000 front 2024-05-15T14:13:40.599455+0000 (oldest deadline 2024-05-15T14:14:04.699619+0000) 2024-05-15T14:14:23.608 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:23 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:23.214+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.101413+0000 front 2024-05-15T14:13:48.101351+0000 (oldest deadline 2024-05-15T14:14:10.401284+0000) 2024-05-15T14:14:23.608 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:23 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:23.214+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:56.302327+0000 front 2024-05-15T14:13:56.302374+0000 (oldest deadline 2024-05-15T14:14:17.402501+0000) 2024-05-15T14:14:23.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:23 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:23.656+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:23.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:23 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:23.656+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:23.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:23 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:23.656+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:37.642201+0000 front 2024-05-15T14:13:37.642108+0000 (oldest deadline 2024-05-15T14:14:02.342241+0000) 2024-05-15T14:14:23.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:23 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:23.656+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:46.443309+0000 front 2024-05-15T14:13:46.443491+0000 (oldest deadline 2024-05-15T14:14:11.743363+0000) 2024-05-15T14:14:23.942 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:23 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:23.656+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:56.345295+0000 front 2024-05-15T14:13:56.345229+0000 (oldest deadline 2024-05-15T14:14:22.245016+0000) 2024-05-15T14:14:24.107 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:23 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:23.643+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:24.107 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:23 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:23.643+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:24.107 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:23 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:23.643+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:24.108 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:23 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:23.643+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:55.816245+0000 front 2024-05-15T14:13:55.816144+0000 (oldest deadline 2024-05-15T14:14:21.016641+0000) 2024-05-15T14:14:24.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:23 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:23.903+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:24.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:23 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:23.903+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:24.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:23 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:23.903+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:24.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:23 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:23.903+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.671290+0000 front 2024-05-15T14:13:48.671390+0000 (oldest deadline 2024-05-15T14:14:13.371393+0000) 2024-05-15T14:14:24.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:23 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:23.903+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:53.371760+0000 front 2024-05-15T14:13:53.371804+0000 (oldest deadline 2024-05-15T14:14:19.172547+0000) 2024-05-15T14:14:24.599 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:24 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:24.225+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:30.097646+0000 front 2024-05-15T14:13:30.097513+0000 (oldest deadline 2024-05-15T14:13:55.397501+0000) 2024-05-15T14:14:24.599 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:24 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:24.226+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.599430+0000 front 2024-05-15T14:13:40.599455+0000 (oldest deadline 2024-05-15T14:14:04.699619+0000) 2024-05-15T14:14:24.599 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:24 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:24.226+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.101413+0000 front 2024-05-15T14:13:48.101351+0000 (oldest deadline 2024-05-15T14:14:10.401284+0000) 2024-05-15T14:14:24.599 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:24 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:24.226+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:56.302327+0000 front 2024-05-15T14:13:56.302374+0000 (oldest deadline 2024-05-15T14:14:17.402501+0000) 2024-05-15T14:14:24.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:24 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:24.597+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:24.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:24 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:24.597+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:24.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:24 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:24.597+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:24.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:24 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:24.597+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:55.816245+0000 front 2024-05-15T14:13:55.816144+0000 (oldest deadline 2024-05-15T14:14:21.016641+0000) 2024-05-15T14:14:24.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:24 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:24.664+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:24.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:24 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:24.664+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:24.942 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:24 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:24.664+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:37.642201+0000 front 2024-05-15T14:13:37.642108+0000 (oldest deadline 2024-05-15T14:14:02.342241+0000) 2024-05-15T14:14:24.942 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:24 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:24.664+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:46.443309+0000 front 2024-05-15T14:13:46.443491+0000 (oldest deadline 2024-05-15T14:14:11.743363+0000) 2024-05-15T14:14:24.942 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:24 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:24.664+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:56.345295+0000 front 2024-05-15T14:13:56.345229+0000 (oldest deadline 2024-05-15T14:14:22.245016+0000) 2024-05-15T14:14:25.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:24 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:24.903+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:25.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:24 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:24.903+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:25.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:24 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:24.903+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:25.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:24 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:24.903+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.671290+0000 front 2024-05-15T14:13:48.671390+0000 (oldest deadline 2024-05-15T14:14:13.371393+0000) 2024-05-15T14:14:25.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:24 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:24.903+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:53.371760+0000 front 2024-05-15T14:13:53.371804+0000 (oldest deadline 2024-05-15T14:14:19.172547+0000) 2024-05-15T14:14:25.607 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:25 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:25.263+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:30.097646+0000 front 2024-05-15T14:13:30.097513+0000 (oldest deadline 2024-05-15T14:13:55.397501+0000) 2024-05-15T14:14:25.607 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:25 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:25.263+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.599430+0000 front 2024-05-15T14:13:40.599455+0000 (oldest deadline 2024-05-15T14:14:04.699619+0000) 2024-05-15T14:14:25.608 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:25 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:25.263+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.101413+0000 front 2024-05-15T14:13:48.101351+0000 (oldest deadline 2024-05-15T14:14:10.401284+0000) 2024-05-15T14:14:25.608 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:25 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:25.263+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:56.302327+0000 front 2024-05-15T14:13:56.302374+0000 (oldest deadline 2024-05-15T14:14:17.402501+0000) 2024-05-15T14:14:25.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:25 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:25.634+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:25.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:25 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:25.634+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:25.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:25 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:25.634+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:37.642201+0000 front 2024-05-15T14:13:37.642108+0000 (oldest deadline 2024-05-15T14:14:02.342241+0000) 2024-05-15T14:14:25.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:25 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:25.634+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:46.443309+0000 front 2024-05-15T14:13:46.443491+0000 (oldest deadline 2024-05-15T14:14:11.743363+0000) 2024-05-15T14:14:25.942 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:25 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:25.634+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:56.345295+0000 front 2024-05-15T14:13:56.345229+0000 (oldest deadline 2024-05-15T14:14:22.245016+0000) 2024-05-15T14:14:26.107 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:25 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:25.609+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:26.107 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:25 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:25.609+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:26.108 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:25 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:25.609+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:26.108 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:25 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:25.609+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:55.816245+0000 front 2024-05-15T14:13:55.816144+0000 (oldest deadline 2024-05-15T14:14:21.016641+0000) 2024-05-15T14:14:26.242 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:25 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:25.919+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:26.243 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:25 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:25.919+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:26.243 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:25 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:25.919+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:26.243 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:25 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:25.919+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.671290+0000 front 2024-05-15T14:13:48.671390+0000 (oldest deadline 2024-05-15T14:14:13.371393+0000) 2024-05-15T14:14:26.243 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:25 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:25.919+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:53.371760+0000 front 2024-05-15T14:13:53.371804+0000 (oldest deadline 2024-05-15T14:14:19.172547+0000) 2024-05-15T14:14:26.382 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:26 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:26.220+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:30.097646+0000 front 2024-05-15T14:13:30.097513+0000 (oldest deadline 2024-05-15T14:13:55.397501+0000) 2024-05-15T14:14:26.382 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:26 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:26.220+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.599430+0000 front 2024-05-15T14:13:40.599455+0000 (oldest deadline 2024-05-15T14:14:04.699619+0000) 2024-05-15T14:14:26.383 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:26 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:26.220+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.101413+0000 front 2024-05-15T14:13:48.101351+0000 (oldest deadline 2024-05-15T14:14:10.401284+0000) 2024-05-15T14:14:26.383 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:26 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:26.220+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:56.302327+0000 front 2024-05-15T14:13:56.302374+0000 (oldest deadline 2024-05-15T14:14:17.402501+0000) 2024-05-15T14:14:26.383 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:26 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:26.220+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:00.204065+0000 front 2024-05-15T14:14:00.204086+0000 (oldest deadline 2024-05-15T14:14:26.104135+0000) 2024-05-15T14:14:26.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:26 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:26.640+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:26.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:26 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:26.640+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:26.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:26 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:26.640+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:37.642201+0000 front 2024-05-15T14:13:37.642108+0000 (oldest deadline 2024-05-15T14:14:02.342241+0000) 2024-05-15T14:14:26.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:26 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:26.640+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:46.443309+0000 front 2024-05-15T14:13:46.443491+0000 (oldest deadline 2024-05-15T14:14:11.743363+0000) 2024-05-15T14:14:26.942 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:26 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:26.640+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:56.345295+0000 front 2024-05-15T14:13:56.345229+0000 (oldest deadline 2024-05-15T14:14:22.245016+0000) 2024-05-15T14:14:26.942 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:26 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:26.640+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:02.245625+0000 front 2024-05-15T14:14:02.245696+0000 (oldest deadline 2024-05-15T14:14:25.745479+0000) 2024-05-15T14:14:27.107 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:26 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:26.632+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:27.108 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:26 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:26.632+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:27.108 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:26 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:26.632+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:27.108 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:26 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:26.632+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:55.816245+0000 front 2024-05-15T14:13:55.816144+0000 (oldest deadline 2024-05-15T14:14:21.016641+0000) 2024-05-15T14:14:27.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:26 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:26.911+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:27.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:26 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:26.911+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:27.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:26 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:26.911+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:27.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:26 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:26.911+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.671290+0000 front 2024-05-15T14:13:48.671390+0000 (oldest deadline 2024-05-15T14:14:13.371393+0000) 2024-05-15T14:14:27.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:26 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:26.911+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:53.371760+0000 front 2024-05-15T14:13:53.371804+0000 (oldest deadline 2024-05-15T14:14:19.172547+0000) 2024-05-15T14:14:27.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:26 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:26.911+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:02.573887+0000 front 2024-05-15T14:14:02.573912+0000 (oldest deadline 2024-05-15T14:14:26.073839+0000) 2024-05-15T14:14:27.607 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:27 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:27.243+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:30.097646+0000 front 2024-05-15T14:13:30.097513+0000 (oldest deadline 2024-05-15T14:13:55.397501+0000) 2024-05-15T14:14:27.608 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:27 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:27.243+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.599430+0000 front 2024-05-15T14:13:40.599455+0000 (oldest deadline 2024-05-15T14:14:04.699619+0000) 2024-05-15T14:14:27.608 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:27 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:27.243+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.101413+0000 front 2024-05-15T14:13:48.101351+0000 (oldest deadline 2024-05-15T14:14:10.401284+0000) 2024-05-15T14:14:27.608 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:27 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:27.243+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:56.302327+0000 front 2024-05-15T14:13:56.302374+0000 (oldest deadline 2024-05-15T14:14:17.402501+0000) 2024-05-15T14:14:27.608 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:27 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8[54740]: 2024-05-15T14:14:27.243+0000 7f0a5c88d700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:00.204065+0000 front 2024-05-15T14:14:00.204086+0000 (oldest deadline 2024-05-15T14:14:26.104135+0000) 2024-05-15T14:14:27.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:27 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:27.685+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:27.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:27 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:27.685+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:27.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:27 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:27.685+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:37.642201+0000 front 2024-05-15T14:13:37.642108+0000 (oldest deadline 2024-05-15T14:14:02.342241+0000) 2024-05-15T14:14:27.942 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:27 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:27.685+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:46.443309+0000 front 2024-05-15T14:13:46.443491+0000 (oldest deadline 2024-05-15T14:14:11.743363+0000) 2024-05-15T14:14:27.942 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:27 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:27.685+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:56.345295+0000 front 2024-05-15T14:13:56.345229+0000 (oldest deadline 2024-05-15T14:14:22.245016+0000) 2024-05-15T14:14:27.942 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:27 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:27.685+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:02.245625+0000 front 2024-05-15T14:14:02.245696+0000 (oldest deadline 2024-05-15T14:14:25.745479+0000) 2024-05-15T14:14:28.107 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:27 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:27.642+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:28.108 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:27 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:27.642+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:28.108 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:27 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:27.642+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:28.108 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:27 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:27.642+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:55.816245+0000 front 2024-05-15T14:13:55.816144+0000 (oldest deadline 2024-05-15T14:14:21.016641+0000) 2024-05-15T14:14:28.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:27 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:27.892+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:28.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:27 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:27.892+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:28.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:27 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:27.892+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:28.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:27 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:27.892+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.671290+0000 front 2024-05-15T14:13:48.671390+0000 (oldest deadline 2024-05-15T14:14:13.371393+0000) 2024-05-15T14:14:28.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:27 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:27.892+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:53.371760+0000 front 2024-05-15T14:13:53.371804+0000 (oldest deadline 2024-05-15T14:14:19.172547+0000) 2024-05-15T14:14:28.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:27 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:27.892+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:02.573887+0000 front 2024-05-15T14:14:02.573912+0000 (oldest deadline 2024-05-15T14:14:26.073839+0000) 2024-05-15T14:14:28.850 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:28 smithi081 podman[74688]: 2024-05-15 14:14:28.56784704 +0000 UTC m=+5.539214719 container died 321f32b34f2bf315a48f203a9860e66ab071ec6a86705aa72022af4bfd21eba4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-05-15T14:14:28.851 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:28 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:28.603+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:28.851 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:28 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:28.603+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:28.851 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:28 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:28.603+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:28.851 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:28 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:28.603+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:55.816245+0000 front 2024-05-15T14:13:55.816144+0000 (oldest deadline 2024-05-15T14:14:21.016641+0000) 2024-05-15T14:14:28.851 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:28 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:28.603+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:04.318554+0000 front 2024-05-15T14:14:04.318616+0000 (oldest deadline 2024-05-15T14:14:27.818829+0000) 2024-05-15T14:14:29.107 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:28 smithi081 podman[74688]: 2024-05-15 14:14:28.938412278 +0000 UTC m=+5.909779956 container cleanup 321f32b34f2bf315a48f203a9860e66ab071ec6a86705aa72022af4bfd21eba4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-15T14:14:29.107 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:28 smithi081 bash[74688]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8 2024-05-15T14:14:29.191 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:28 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:28.729+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:29.191 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:28 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:28.729+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:29.191 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:28 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:28.729+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:37.642201+0000 front 2024-05-15T14:13:37.642108+0000 (oldest deadline 2024-05-15T14:14:02.342241+0000) 2024-05-15T14:14:29.191 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:28 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:28.729+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:46.443309+0000 front 2024-05-15T14:13:46.443491+0000 (oldest deadline 2024-05-15T14:14:11.743363+0000) 2024-05-15T14:14:29.192 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:28 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:28.729+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:56.345295+0000 front 2024-05-15T14:13:56.345229+0000 (oldest deadline 2024-05-15T14:14:22.245016+0000) 2024-05-15T14:14:29.192 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:28 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:28.729+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:02.245625+0000 front 2024-05-15T14:14:02.245696+0000 (oldest deadline 2024-05-15T14:14:25.745479+0000) 2024-05-15T14:14:29.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:28 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:28.887+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:29.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:28 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:28.887+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:29.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:28 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:28.887+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:29.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:28 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:28.887+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.671290+0000 front 2024-05-15T14:13:48.671390+0000 (oldest deadline 2024-05-15T14:14:13.371393+0000) 2024-05-15T14:14:29.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:28 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:28.887+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:53.371760+0000 front 2024-05-15T14:13:53.371804+0000 (oldest deadline 2024-05-15T14:14:19.172547+0000) 2024-05-15T14:14:29.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:28 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:28.887+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:02.573887+0000 front 2024-05-15T14:14:02.573912+0000 (oldest deadline 2024-05-15T14:14:26.073839+0000) 2024-05-15T14:14:29.562 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:29 smithi081 podman[74715]: 2024-05-15 14:14:29.198175935 +0000 UTC m=+0.627247672 container remove 321f32b34f2bf315a48f203a9860e66ab071ec6a86705aa72022af4bfd21eba4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-05-15T14:14:29.828 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:29 smithi081 podman[74804]: 2024-05-15 14:14:29.46183186 +0000 UTC m=+0.016909227 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-15T14:14:29.828 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:29 smithi081 podman[74804]: 2024-05-15 14:14:29.583860776 +0000 UTC m=+0.138938164 container create ff2de22021b0b5815c5f2910c9ec3fb7d954920acb6b208382bf428fe9573929 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-15T14:14:29.828 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:29 smithi081 podman[74804]: 2024-05-15 14:14:29.826960116 +0000 UTC m=+0.382037490 container init ff2de22021b0b5815c5f2910c9ec3fb7d954920acb6b208382bf428fe9573929 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-15T14:14:29.829 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:29 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:29.562+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:29.829 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:29 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:29.562+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:29.829 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:29 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:29.562+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:29.829 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:29 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:29.562+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:55.816245+0000 front 2024-05-15T14:13:55.816144+0000 (oldest deadline 2024-05-15T14:14:21.016641+0000) 2024-05-15T14:14:29.829 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:29 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:29.562+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:04.318554+0000 front 2024-05-15T14:14:04.318616+0000 (oldest deadline 2024-05-15T14:14:27.818829+0000) 2024-05-15T14:14:30.107 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:29 smithi081 podman[74804]: 2024-05-15 14:14:29.830198769 +0000 UTC m=+0.385276136 container start ff2de22021b0b5815c5f2910c9ec3fb7d954920acb6b208382bf428fe9573929 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-15T14:14:30.107 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:29 smithi081 podman[74804]: 2024-05-15 14:14:29.909646569 +0000 UTC m=+0.464723971 container attach ff2de22021b0b5815c5f2910c9ec3fb7d954920acb6b208382bf428fe9573929 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-15T14:14:30.191 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:29 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:29.777+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:30.191 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:29 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:29.777+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:30.191 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:29 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:29.777+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:37.642201+0000 front 2024-05-15T14:13:37.642108+0000 (oldest deadline 2024-05-15T14:14:02.342241+0000) 2024-05-15T14:14:30.192 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:29 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:29.777+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:46.443309+0000 front 2024-05-15T14:13:46.443491+0000 (oldest deadline 2024-05-15T14:14:11.743363+0000) 2024-05-15T14:14:30.192 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:29 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:29.777+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:56.345295+0000 front 2024-05-15T14:13:56.345229+0000 (oldest deadline 2024-05-15T14:14:22.245016+0000) 2024-05-15T14:14:30.192 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:29 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:29.777+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:02.245625+0000 front 2024-05-15T14:14:02.245696+0000 (oldest deadline 2024-05-15T14:14:25.745479+0000) 2024-05-15T14:14:30.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:29 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:29.935+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:30.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:29 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:29.935+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:30.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:29 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:29.935+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:30.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:29 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:29.935+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.671290+0000 front 2024-05-15T14:13:48.671390+0000 (oldest deadline 2024-05-15T14:14:13.371393+0000) 2024-05-15T14:14:30.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:29 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:29.935+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:53.371760+0000 front 2024-05-15T14:13:53.371804+0000 (oldest deadline 2024-05-15T14:14:19.172547+0000) 2024-05-15T14:14:30.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:29 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:29.935+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:02.573887+0000 front 2024-05-15T14:14:02.573912+0000 (oldest deadline 2024-05-15T14:14:26.073839+0000) 2024-05-15T14:14:30.465 INFO:journalctl@ceph.osd.8.smithi081.stdout:May 15 14:14:30 smithi081 podman[74804]: 2024-05-15 14:14:30.113994071 +0000 UTC m=+0.669071437 container died ff2de22021b0b5815c5f2910c9ec3fb7d954920acb6b208382bf428fe9573929 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T14:14:30.731 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:30 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:30.596+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:30.732 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:30 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:30.597+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:30.732 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:30 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:30.597+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:30.732 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:30 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:30.597+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:55.816245+0000 front 2024-05-15T14:13:55.816144+0000 (oldest deadline 2024-05-15T14:14:21.016641+0000) 2024-05-15T14:14:30.732 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:30 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:30.597+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:04.318554+0000 front 2024-05-15T14:14:04.318616+0000 (oldest deadline 2024-05-15T14:14:27.818829+0000) 2024-05-15T14:14:30.757 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.8.service' 2024-05-15T14:14:30.796 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T14:14:30.796 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-15T14:14:30.797 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-15T14:14:30.797 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.9 2024-05-15T14:14:31.115 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:30 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:30.806+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:31.116 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:30 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:30.806+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:31.116 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:30 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:30.806+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:37.642201+0000 front 2024-05-15T14:13:37.642108+0000 (oldest deadline 2024-05-15T14:14:02.342241+0000) 2024-05-15T14:14:31.116 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:30 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:30.806+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:46.443309+0000 front 2024-05-15T14:13:46.443491+0000 (oldest deadline 2024-05-15T14:14:11.743363+0000) 2024-05-15T14:14:31.116 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:30 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:30.806+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:56.345295+0000 front 2024-05-15T14:13:56.345229+0000 (oldest deadline 2024-05-15T14:14:22.245016+0000) 2024-05-15T14:14:31.116 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:30 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:30.806+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:02.245625+0000 front 2024-05-15T14:14:02.245696+0000 (oldest deadline 2024-05-15T14:14:25.745479+0000) 2024-05-15T14:14:31.116 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:30 smithi033 systemd[1]: Stopping Ceph osd.9 for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T14:14:31.117 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:31 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:31.112+0000 7f82481f1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T14:14:31.117 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:31 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:31.112+0000 7f82481f1700 -1 osd.9 101 *** Got signal Terminated *** 2024-05-15T14:14:31.117 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:31 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:31.112+0000 7f82481f1700 -1 osd.9 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-15T14:14:31.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:30 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:30.929+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:31.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:30 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:30.929+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:31.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:30 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:30.929+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:31.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:30 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:30.929+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.671290+0000 front 2024-05-15T14:13:48.671390+0000 (oldest deadline 2024-05-15T14:14:13.371393+0000) 2024-05-15T14:14:31.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:30 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:30.929+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:53.371760+0000 front 2024-05-15T14:13:53.371804+0000 (oldest deadline 2024-05-15T14:14:19.172547+0000) 2024-05-15T14:14:31.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:30 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:30.929+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:02.573887+0000 front 2024-05-15T14:14:02.573912+0000 (oldest deadline 2024-05-15T14:14:26.073839+0000) 2024-05-15T14:14:31.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:31 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:31.582+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:31.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:31 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:31.582+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:31.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:31 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:31.582+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:31.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:31 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:31.582+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:55.816245+0000 front 2024-05-15T14:13:55.816144+0000 (oldest deadline 2024-05-15T14:14:21.016641+0000) 2024-05-15T14:14:31.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:31 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:31.582+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:04.318554+0000 front 2024-05-15T14:14:04.318616+0000 (oldest deadline 2024-05-15T14:14:27.818829+0000) 2024-05-15T14:14:32.191 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:31 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:31.783+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:32.191 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:31 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:31.783+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:32.192 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:31 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:31.783+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:37.642201+0000 front 2024-05-15T14:13:37.642108+0000 (oldest deadline 2024-05-15T14:14:02.342241+0000) 2024-05-15T14:14:32.192 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:31 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:31.783+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:46.443309+0000 front 2024-05-15T14:13:46.443491+0000 (oldest deadline 2024-05-15T14:14:11.743363+0000) 2024-05-15T14:14:32.192 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:31 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:31.783+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:56.345295+0000 front 2024-05-15T14:13:56.345229+0000 (oldest deadline 2024-05-15T14:14:22.245016+0000) 2024-05-15T14:14:32.192 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:31 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:31.783+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:02.245625+0000 front 2024-05-15T14:14:02.245696+0000 (oldest deadline 2024-05-15T14:14:25.745479+0000) 2024-05-15T14:14:32.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:31 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:31.963+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:32.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:31 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:31.963+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:32.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:31 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:31.963+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:32.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:31 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:31.963+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.671290+0000 front 2024-05-15T14:13:48.671390+0000 (oldest deadline 2024-05-15T14:14:13.371393+0000) 2024-05-15T14:14:32.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:31 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:31.963+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:53.371760+0000 front 2024-05-15T14:13:53.371804+0000 (oldest deadline 2024-05-15T14:14:19.172547+0000) 2024-05-15T14:14:32.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:31 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:31.963+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:02.573887+0000 front 2024-05-15T14:14:02.573912+0000 (oldest deadline 2024-05-15T14:14:26.073839+0000) 2024-05-15T14:14:33.107 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:32 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:32.609+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:33.108 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:32 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:32.609+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:33.108 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:32 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:32.609+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:33.108 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:32 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:32.609+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:55.816245+0000 front 2024-05-15T14:13:55.816144+0000 (oldest deadline 2024-05-15T14:14:21.016641+0000) 2024-05-15T14:14:33.108 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:32 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:32.609+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:04.318554+0000 front 2024-05-15T14:14:04.318616+0000 (oldest deadline 2024-05-15T14:14:27.818829+0000) 2024-05-15T14:14:33.191 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:32 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:32.825+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:33.191 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:32 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:32.825+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:33.191 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:32 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:32.825+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:37.642201+0000 front 2024-05-15T14:13:37.642108+0000 (oldest deadline 2024-05-15T14:14:02.342241+0000) 2024-05-15T14:14:33.192 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:32 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:32.825+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:46.443309+0000 front 2024-05-15T14:13:46.443491+0000 (oldest deadline 2024-05-15T14:14:11.743363+0000) 2024-05-15T14:14:33.192 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:32 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:32.825+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:56.345295+0000 front 2024-05-15T14:13:56.345229+0000 (oldest deadline 2024-05-15T14:14:22.245016+0000) 2024-05-15T14:14:33.192 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:32 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:32.825+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:02.245625+0000 front 2024-05-15T14:14:02.245696+0000 (oldest deadline 2024-05-15T14:14:25.745479+0000) 2024-05-15T14:14:33.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:32 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:32.965+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:33.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:32 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:32.965+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:33.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:32 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:32.965+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:33.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:32 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:32.965+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.671290+0000 front 2024-05-15T14:13:48.671390+0000 (oldest deadline 2024-05-15T14:14:13.371393+0000) 2024-05-15T14:14:33.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:32 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:32.965+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:53.371760+0000 front 2024-05-15T14:13:53.371804+0000 (oldest deadline 2024-05-15T14:14:19.172547+0000) 2024-05-15T14:14:33.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:32 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:32.965+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:02.573887+0000 front 2024-05-15T14:14:02.573912+0000 (oldest deadline 2024-05-15T14:14:26.073839+0000) 2024-05-15T14:14:33.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:33 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:33.570+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:33.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:33 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:33.570+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:33.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:33 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:33.570+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:33.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:33 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:33.570+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:55.816245+0000 front 2024-05-15T14:13:55.816144+0000 (oldest deadline 2024-05-15T14:14:21.016641+0000) 2024-05-15T14:14:33.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:33 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:33.570+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:04.318554+0000 front 2024-05-15T14:14:04.318616+0000 (oldest deadline 2024-05-15T14:14:27.818829+0000) 2024-05-15T14:14:34.191 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:33 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:33.829+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:34.191 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:33 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:33.829+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:34.191 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:33 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:33.829+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:37.642201+0000 front 2024-05-15T14:13:37.642108+0000 (oldest deadline 2024-05-15T14:14:02.342241+0000) 2024-05-15T14:14:34.192 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:33 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:33.829+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:46.443309+0000 front 2024-05-15T14:13:46.443491+0000 (oldest deadline 2024-05-15T14:14:11.743363+0000) 2024-05-15T14:14:34.192 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:33 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:33.829+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:56.345295+0000 front 2024-05-15T14:13:56.345229+0000 (oldest deadline 2024-05-15T14:14:22.245016+0000) 2024-05-15T14:14:34.192 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:33 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:33.829+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:02.245625+0000 front 2024-05-15T14:14:02.245696+0000 (oldest deadline 2024-05-15T14:14:25.745479+0000) 2024-05-15T14:14:34.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:33 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:33.971+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:34.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:33 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:33.971+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:34.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:33 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:33.971+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:34.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:33 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:33.971+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.671290+0000 front 2024-05-15T14:13:48.671390+0000 (oldest deadline 2024-05-15T14:14:13.371393+0000) 2024-05-15T14:14:34.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:33 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:33.971+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:53.371760+0000 front 2024-05-15T14:13:53.371804+0000 (oldest deadline 2024-05-15T14:14:19.172547+0000) 2024-05-15T14:14:34.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:33 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:33.971+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:02.573887+0000 front 2024-05-15T14:14:02.573912+0000 (oldest deadline 2024-05-15T14:14:26.073839+0000) 2024-05-15T14:14:34.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:34 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:34.583+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:34.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:34 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:34.583+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:34.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:34 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:34.583+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:34.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:34 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:34.583+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:55.816245+0000 front 2024-05-15T14:13:55.816144+0000 (oldest deadline 2024-05-15T14:14:21.016641+0000) 2024-05-15T14:14:34.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:34 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:34.583+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:04.318554+0000 front 2024-05-15T14:14:04.318616+0000 (oldest deadline 2024-05-15T14:14:27.818829+0000) 2024-05-15T14:14:34.859 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:34 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:34.583+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-15T14:14:07.819560+0000 front 2024-05-15T14:14:07.819647+0000 (oldest deadline 2024-05-15T14:14:33.719492+0000) 2024-05-15T14:14:35.191 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:34 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:34.813+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:35.191 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:34 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:34.813+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:35.192 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:34 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:34.813+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:37.642201+0000 front 2024-05-15T14:13:37.642108+0000 (oldest deadline 2024-05-15T14:14:02.342241+0000) 2024-05-15T14:14:35.192 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:34 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:34.813+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:46.443309+0000 front 2024-05-15T14:13:46.443491+0000 (oldest deadline 2024-05-15T14:14:11.743363+0000) 2024-05-15T14:14:35.192 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:34 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:34.813+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:56.345295+0000 front 2024-05-15T14:13:56.345229+0000 (oldest deadline 2024-05-15T14:14:22.245016+0000) 2024-05-15T14:14:35.192 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:34 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:34.813+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:02.245625+0000 front 2024-05-15T14:14:02.245696+0000 (oldest deadline 2024-05-15T14:14:25.745479+0000) 2024-05-15T14:14:35.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:34 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:34.997+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:35.343 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:34 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:34.997+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:35.343 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:34 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:34.997+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:35.343 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:34 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:34.997+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.671290+0000 front 2024-05-15T14:13:48.671390+0000 (oldest deadline 2024-05-15T14:14:13.371393+0000) 2024-05-15T14:14:35.344 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:34 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:34.997+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:53.371760+0000 front 2024-05-15T14:13:53.371804+0000 (oldest deadline 2024-05-15T14:14:19.172547+0000) 2024-05-15T14:14:35.344 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:34 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:34.997+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:02.573887+0000 front 2024-05-15T14:14:02.573912+0000 (oldest deadline 2024-05-15T14:14:26.073839+0000) 2024-05-15T14:14:35.345 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:34 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:34.997+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-15T14:14:10.675335+0000 front 2024-05-15T14:14:10.675520+0000 (oldest deadline 2024-05-15T14:14:34.775227+0000) 2024-05-15T14:14:35.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:35 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:35.540+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:35.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:35 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:35.540+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:35.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:35 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:35.540+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:35.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:35 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:35.540+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:55.816245+0000 front 2024-05-15T14:13:55.816144+0000 (oldest deadline 2024-05-15T14:14:21.016641+0000) 2024-05-15T14:14:35.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:35 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:35.540+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:04.318554+0000 front 2024-05-15T14:14:04.318616+0000 (oldest deadline 2024-05-15T14:14:27.818829+0000) 2024-05-15T14:14:35.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:35 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:35.540+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-15T14:14:07.819560+0000 front 2024-05-15T14:14:07.819647+0000 (oldest deadline 2024-05-15T14:14:33.719492+0000) 2024-05-15T14:14:36.192 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:35 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:35.831+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:23.539978+0000 front 2024-05-15T14:13:23.539923+0000 (oldest deadline 2024-05-15T14:13:48.840089+0000) 2024-05-15T14:14:36.192 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:35 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:35.831+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:31.741582+0000 front 2024-05-15T14:13:31.741636+0000 (oldest deadline 2024-05-15T14:13:57.641531+0000) 2024-05-15T14:14:36.192 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:35 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:35.831+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:37.642201+0000 front 2024-05-15T14:13:37.642108+0000 (oldest deadline 2024-05-15T14:14:02.342241+0000) 2024-05-15T14:14:36.192 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:35 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:35.831+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:46.443309+0000 front 2024-05-15T14:13:46.443491+0000 (oldest deadline 2024-05-15T14:14:11.743363+0000) 2024-05-15T14:14:36.192 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:35 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:35.831+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:56.345295+0000 front 2024-05-15T14:13:56.345229+0000 (oldest deadline 2024-05-15T14:14:22.245016+0000) 2024-05-15T14:14:36.192 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:35 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:35.831+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:02.245625+0000 front 2024-05-15T14:14:02.245696+0000 (oldest deadline 2024-05-15T14:14:25.745479+0000) 2024-05-15T14:14:36.193 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:35 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9[64247]: 2024-05-15T14:14:35.831+0000 7f82407dd700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-15T14:14:11.546829+0000 front 2024-05-15T14:14:11.546806+0000 (oldest deadline 2024-05-15T14:14:35.046774+0000) 2024-05-15T14:14:36.298 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:35 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:35.997+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:36.298 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:35 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:35.997+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:36.298 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:35 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:35.997+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:36.298 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:35 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:35.997+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.671290+0000 front 2024-05-15T14:13:48.671390+0000 (oldest deadline 2024-05-15T14:14:13.371393+0000) 2024-05-15T14:14:36.298 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:35 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:35.997+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:53.371760+0000 front 2024-05-15T14:13:53.371804+0000 (oldest deadline 2024-05-15T14:14:19.172547+0000) 2024-05-15T14:14:36.298 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:35 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:35.997+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:02.573887+0000 front 2024-05-15T14:14:02.573912+0000 (oldest deadline 2024-05-15T14:14:26.073839+0000) 2024-05-15T14:14:36.299 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:35 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:35.997+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-15T14:14:10.675335+0000 front 2024-05-15T14:14:10.675520+0000 (oldest deadline 2024-05-15T14:14:34.775227+0000) 2024-05-15T14:14:36.816 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:36 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:36.550+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:36.817 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:36 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:36.550+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:36.817 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:36 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:36.550+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:36.817 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:36 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:36.550+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:55.816245+0000 front 2024-05-15T14:13:55.816144+0000 (oldest deadline 2024-05-15T14:14:21.016641+0000) 2024-05-15T14:14:36.817 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:36 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:36.550+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:04.318554+0000 front 2024-05-15T14:14:04.318616+0000 (oldest deadline 2024-05-15T14:14:27.818829+0000) 2024-05-15T14:14:36.817 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:36 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:36.550+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-15T14:14:07.819560+0000 front 2024-05-15T14:14:07.819647+0000 (oldest deadline 2024-05-15T14:14:33.719492+0000) 2024-05-15T14:14:36.913 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:36 smithi033 podman[104209]: 2024-05-15 14:14:36.644415668 +0000 UTC m=+5.643903428 container died 529998a6cd606aa1e480d368bbd8813d3e2f616119e900e7a4294c4ba92241e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.build-date=20240514) 2024-05-15T14:14:37.191 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:37 smithi033 podman[104209]: 2024-05-15 14:14:37.085897173 +0000 UTC m=+6.085384927 container cleanup 529998a6cd606aa1e480d368bbd8813d3e2f616119e900e7a4294c4ba92241e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-15T14:14:37.191 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:37 smithi033 bash[104209]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9 2024-05-15T14:14:37.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:36 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:36.976+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:37.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:36 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:36.976+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:37.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:36 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:36.976+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:37.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:36 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:36.976+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.671290+0000 front 2024-05-15T14:13:48.671390+0000 (oldest deadline 2024-05-15T14:14:13.371393+0000) 2024-05-15T14:14:37.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:36 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:36.976+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:53.371760+0000 front 2024-05-15T14:13:53.371804+0000 (oldest deadline 2024-05-15T14:14:19.172547+0000) 2024-05-15T14:14:37.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:36 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:36.976+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:02.573887+0000 front 2024-05-15T14:14:02.573912+0000 (oldest deadline 2024-05-15T14:14:26.073839+0000) 2024-05-15T14:14:37.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:36 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:36.976+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-15T14:14:10.675335+0000 front 2024-05-15T14:14:10.675520+0000 (oldest deadline 2024-05-15T14:14:34.775227+0000) 2024-05-15T14:14:37.691 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:37 smithi033 podman[104226]: 2024-05-15 14:14:37.295768869 +0000 UTC m=+0.649131774 container remove 529998a6cd606aa1e480d368bbd8813d3e2f616119e900e7a4294c4ba92241e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-15T14:14:37.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:37 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:37.512+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:37.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:37 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:37.512+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:37.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:37 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:37.512+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:37.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:37 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:37.512+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:55.816245+0000 front 2024-05-15T14:13:55.816144+0000 (oldest deadline 2024-05-15T14:14:21.016641+0000) 2024-05-15T14:14:37.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:37 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:37.512+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:04.318554+0000 front 2024-05-15T14:14:04.318616+0000 (oldest deadline 2024-05-15T14:14:27.818829+0000) 2024-05-15T14:14:37.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:37 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:37.512+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-15T14:14:07.819560+0000 front 2024-05-15T14:14:07.819647+0000 (oldest deadline 2024-05-15T14:14:33.719492+0000) 2024-05-15T14:14:38.091 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:37 smithi033 podman[104314]: 2024-05-15 14:14:37.712999773 +0000 UTC m=+0.109957058 container create c4a8fe31fa01ff0937b23061cd5b182b6e14d3d5c86238317a5f63b3f3926542 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9-deactivate, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-15T14:14:38.091 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:37 smithi033 podman[104314]: 2024-05-15 14:14:37.621054426 +0000 UTC m=+0.018011725 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-15T14:14:38.092 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:38 smithi033 podman[104314]: 2024-05-15 14:14:38.090256787 +0000 UTC m=+0.487214073 container init c4a8fe31fa01ff0937b23061cd5b182b6e14d3d5c86238317a5f63b3f3926542 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-15T14:14:38.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:37 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:37.936+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:38.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:37 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:37.936+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:38.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:37 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:37.936+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:38.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:37 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:37.936+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.671290+0000 front 2024-05-15T14:13:48.671390+0000 (oldest deadline 2024-05-15T14:14:13.371393+0000) 2024-05-15T14:14:38.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:37 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:37.936+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:53.371760+0000 front 2024-05-15T14:13:53.371804+0000 (oldest deadline 2024-05-15T14:14:19.172547+0000) 2024-05-15T14:14:38.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:37 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:37.936+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:02.573887+0000 front 2024-05-15T14:14:02.573912+0000 (oldest deadline 2024-05-15T14:14:26.073839+0000) 2024-05-15T14:14:38.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:37 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:37.936+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-15T14:14:10.675335+0000 front 2024-05-15T14:14:10.675520+0000 (oldest deadline 2024-05-15T14:14:34.775227+0000) 2024-05-15T14:14:38.414 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:38 smithi033 podman[104314]: 2024-05-15 14:14:38.09406077 +0000 UTC m=+0.491018070 container start c4a8fe31fa01ff0937b23061cd5b182b6e14d3d5c86238317a5f63b3f3926542 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-15T14:14:38.414 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:38 smithi033 podman[104314]: 2024-05-15 14:14:38.1734918 +0000 UTC m=+0.570449098 container attach c4a8fe31fa01ff0937b23061cd5b182b6e14d3d5c86238317a5f63b3f3926542 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514) 2024-05-15T14:14:38.691 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 15 14:14:38 smithi033 podman[104314]: 2024-05-15 14:14:38.412865435 +0000 UTC m=+0.809822744 container died c4a8fe31fa01ff0937b23061cd5b182b6e14d3d5c86238317a5f63b3f3926542 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, ceph=True) 2024-05-15T14:14:38.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:38 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:38.536+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:38.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:38 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:38.536+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:38.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:38 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:38.536+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:38.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:38 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:38.536+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:55.816245+0000 front 2024-05-15T14:13:55.816144+0000 (oldest deadline 2024-05-15T14:14:21.016641+0000) 2024-05-15T14:14:38.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:38 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:38.536+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:04.318554+0000 front 2024-05-15T14:14:04.318616+0000 (oldest deadline 2024-05-15T14:14:27.818829+0000) 2024-05-15T14:14:38.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:38 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:38.536+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-15T14:14:07.819560+0000 front 2024-05-15T14:14:07.819647+0000 (oldest deadline 2024-05-15T14:14:33.719492+0000) 2024-05-15T14:14:38.993 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.9.service' 2024-05-15T14:14:39.033 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T14:14:39.033 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-15T14:14:39.034 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-15T14:14:39.034 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.10 2024-05-15T14:14:39.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:38 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:38.942+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:39.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:38 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:38.942+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:39.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:38 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:38.942+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:39.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:38 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:38.942+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.671290+0000 front 2024-05-15T14:13:48.671390+0000 (oldest deadline 2024-05-15T14:14:13.371393+0000) 2024-05-15T14:14:39.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:38 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:38.942+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:53.371760+0000 front 2024-05-15T14:13:53.371804+0000 (oldest deadline 2024-05-15T14:14:19.172547+0000) 2024-05-15T14:14:39.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:38 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:38.942+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:02.573887+0000 front 2024-05-15T14:14:02.573912+0000 (oldest deadline 2024-05-15T14:14:26.073839+0000) 2024-05-15T14:14:39.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:38 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:38.942+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-15T14:14:10.675335+0000 front 2024-05-15T14:14:10.675520+0000 (oldest deadline 2024-05-15T14:14:34.775227+0000) 2024-05-15T14:14:39.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:39 smithi052 systemd[1]: Stopping Ceph osd.10 for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T14:14:39.841 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:39 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:39.341+0000 7fea2f166700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T14:14:39.841 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:39 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:39.341+0000 7fea2f166700 -1 osd.10 101 *** Got signal Terminated *** 2024-05-15T14:14:39.841 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:39 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:39.341+0000 7fea2f166700 -1 osd.10 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-15T14:14:39.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:39 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:39.555+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:39.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:39 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:39.555+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:39.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:39 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:39.555+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:39.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:39 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:39.555+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:55.816245+0000 front 2024-05-15T14:13:55.816144+0000 (oldest deadline 2024-05-15T14:14:21.016641+0000) 2024-05-15T14:14:39.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:39 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:39.555+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:04.318554+0000 front 2024-05-15T14:14:04.318616+0000 (oldest deadline 2024-05-15T14:14:27.818829+0000) 2024-05-15T14:14:39.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:39 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:39.555+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-15T14:14:07.819560+0000 front 2024-05-15T14:14:07.819647+0000 (oldest deadline 2024-05-15T14:14:33.719492+0000) 2024-05-15T14:14:40.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:39 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:39.895+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:40.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:39 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:39.896+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:40.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:39 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:39.896+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:40.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:39 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:39.896+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.671290+0000 front 2024-05-15T14:13:48.671390+0000 (oldest deadline 2024-05-15T14:14:13.371393+0000) 2024-05-15T14:14:40.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:39 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:39.896+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:53.371760+0000 front 2024-05-15T14:13:53.371804+0000 (oldest deadline 2024-05-15T14:14:19.172547+0000) 2024-05-15T14:14:40.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:39 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:39.896+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:02.573887+0000 front 2024-05-15T14:14:02.573912+0000 (oldest deadline 2024-05-15T14:14:26.073839+0000) 2024-05-15T14:14:40.342 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:39 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:39.896+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-15T14:14:10.675335+0000 front 2024-05-15T14:14:10.675520+0000 (oldest deadline 2024-05-15T14:14:34.775227+0000) 2024-05-15T14:14:40.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:40 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:40.586+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:40.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:40 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:40.586+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:40.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:40 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:40.586+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:40.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:40 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:40.586+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:55.816245+0000 front 2024-05-15T14:13:55.816144+0000 (oldest deadline 2024-05-15T14:14:21.016641+0000) 2024-05-15T14:14:40.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:40 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:40.586+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:04.318554+0000 front 2024-05-15T14:14:04.318616+0000 (oldest deadline 2024-05-15T14:14:27.818829+0000) 2024-05-15T14:14:40.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:40 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:40.586+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-15T14:14:07.819560+0000 front 2024-05-15T14:14:07.819647+0000 (oldest deadline 2024-05-15T14:14:33.719492+0000) 2024-05-15T14:14:41.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:40 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:40.858+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:41.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:40 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:40.858+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:41.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:40 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:40.858+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:41.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:40 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:40.858+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.671290+0000 front 2024-05-15T14:13:48.671390+0000 (oldest deadline 2024-05-15T14:14:13.371393+0000) 2024-05-15T14:14:41.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:40 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:40.858+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:53.371760+0000 front 2024-05-15T14:13:53.371804+0000 (oldest deadline 2024-05-15T14:14:19.172547+0000) 2024-05-15T14:14:41.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:40 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:40.858+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:02.573887+0000 front 2024-05-15T14:14:02.573912+0000 (oldest deadline 2024-05-15T14:14:26.073839+0000) 2024-05-15T14:14:41.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:40 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:40.858+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-15T14:14:10.675335+0000 front 2024-05-15T14:14:10.675520+0000 (oldest deadline 2024-05-15T14:14:34.775227+0000) 2024-05-15T14:14:41.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:41 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:41.544+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:41.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:41 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:41.544+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:41.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:41 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:41.544+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:41.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:41 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:41.544+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:55.816245+0000 front 2024-05-15T14:13:55.816144+0000 (oldest deadline 2024-05-15T14:14:21.016641+0000) 2024-05-15T14:14:41.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:41 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:41.544+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:04.318554+0000 front 2024-05-15T14:14:04.318616+0000 (oldest deadline 2024-05-15T14:14:27.818829+0000) 2024-05-15T14:14:41.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:41 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:41.544+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-15T14:14:07.819560+0000 front 2024-05-15T14:14:07.819647+0000 (oldest deadline 2024-05-15T14:14:33.719492+0000) 2024-05-15T14:14:42.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:41 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:41.820+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:42.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:41 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:41.820+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:42.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:41 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:41.820+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:42.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:41 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:41.820+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.671290+0000 front 2024-05-15T14:13:48.671390+0000 (oldest deadline 2024-05-15T14:14:13.371393+0000) 2024-05-15T14:14:42.092 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:41 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:41.820+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:53.371760+0000 front 2024-05-15T14:13:53.371804+0000 (oldest deadline 2024-05-15T14:14:19.172547+0000) 2024-05-15T14:14:42.092 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:41 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:41.820+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:02.573887+0000 front 2024-05-15T14:14:02.573912+0000 (oldest deadline 2024-05-15T14:14:26.073839+0000) 2024-05-15T14:14:42.092 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:41 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:41.820+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-15T14:14:10.675335+0000 front 2024-05-15T14:14:10.675520+0000 (oldest deadline 2024-05-15T14:14:34.775227+0000) 2024-05-15T14:14:42.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:42 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:42.589+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:42.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:42 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:42.589+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:42.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:42 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:42.589+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:42.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:42 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:42.589+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:55.816245+0000 front 2024-05-15T14:13:55.816144+0000 (oldest deadline 2024-05-15T14:14:21.016641+0000) 2024-05-15T14:14:42.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:42 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:42.589+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:04.318554+0000 front 2024-05-15T14:14:04.318616+0000 (oldest deadline 2024-05-15T14:14:27.818829+0000) 2024-05-15T14:14:42.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:42 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:42.589+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-15T14:14:07.819560+0000 front 2024-05-15T14:14:07.819647+0000 (oldest deadline 2024-05-15T14:14:33.719492+0000) 2024-05-15T14:14:43.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:42 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:42.811+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:43.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:42 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:42.811+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:43.092 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:42 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:42.811+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:43.092 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:42 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:42.811+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.671290+0000 front 2024-05-15T14:13:48.671390+0000 (oldest deadline 2024-05-15T14:14:13.371393+0000) 2024-05-15T14:14:43.092 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:42 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:42.811+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:53.371760+0000 front 2024-05-15T14:13:53.371804+0000 (oldest deadline 2024-05-15T14:14:19.172547+0000) 2024-05-15T14:14:43.092 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:42 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:42.811+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:02.573887+0000 front 2024-05-15T14:14:02.573912+0000 (oldest deadline 2024-05-15T14:14:26.073839+0000) 2024-05-15T14:14:43.092 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:42 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:42.811+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-15T14:14:10.675335+0000 front 2024-05-15T14:14:10.675520+0000 (oldest deadline 2024-05-15T14:14:34.775227+0000) 2024-05-15T14:14:43.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:43 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:43.551+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:43.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:43 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:43.551+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:43.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:43 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:43.551+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:43.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:43 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:43.551+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:55.816245+0000 front 2024-05-15T14:13:55.816144+0000 (oldest deadline 2024-05-15T14:14:21.016641+0000) 2024-05-15T14:14:43.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:43 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:43.551+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:04.318554+0000 front 2024-05-15T14:14:04.318616+0000 (oldest deadline 2024-05-15T14:14:27.818829+0000) 2024-05-15T14:14:43.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:43 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:43.551+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-15T14:14:07.819560+0000 front 2024-05-15T14:14:07.819647+0000 (oldest deadline 2024-05-15T14:14:33.719492+0000) 2024-05-15T14:14:44.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:43 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:43.765+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:24.365959+0000 front 2024-05-15T14:13:24.365998+0000 (oldest deadline 2024-05-15T14:13:50.265722+0000) 2024-05-15T14:14:44.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:43 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:43.765+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:32.567044+0000 front 2024-05-15T14:13:32.567112+0000 (oldest deadline 2024-05-15T14:13:57.267236+0000) 2024-05-15T14:14:44.091 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:43 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:43.765+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:40.068900+0000 front 2024-05-15T14:13:40.068993+0000 (oldest deadline 2024-05-15T14:14:05.368925+0000) 2024-05-15T14:14:44.092 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:43 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:43.765+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-15T14:13:48.671290+0000 front 2024-05-15T14:13:48.671390+0000 (oldest deadline 2024-05-15T14:14:13.371393+0000) 2024-05-15T14:14:44.092 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:43 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:43.765+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:53.371760+0000 front 2024-05-15T14:13:53.371804+0000 (oldest deadline 2024-05-15T14:14:19.172547+0000) 2024-05-15T14:14:44.092 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:43 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:43.765+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:02.573887+0000 front 2024-05-15T14:14:02.573912+0000 (oldest deadline 2024-05-15T14:14:26.073839+0000) 2024-05-15T14:14:44.092 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:43 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10[61252]: 2024-05-15T14:14:43.765+0000 7fea27752700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-15T14:14:10.675335+0000 front 2024-05-15T14:14:10.675520+0000 (oldest deadline 2024-05-15T14:14:34.775227+0000) 2024-05-15T14:14:44.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:44 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:44.510+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:44.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:44 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:44.510+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:44.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:44 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:44.510+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:44.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:44 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:44.510+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:55.816245+0000 front 2024-05-15T14:13:55.816144+0000 (oldest deadline 2024-05-15T14:14:21.016641+0000) 2024-05-15T14:14:44.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:44 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:44.510+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:04.318554+0000 front 2024-05-15T14:14:04.318616+0000 (oldest deadline 2024-05-15T14:14:27.818829+0000) 2024-05-15T14:14:44.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:44 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:44.510+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-15T14:14:07.819560+0000 front 2024-05-15T14:14:07.819647+0000 (oldest deadline 2024-05-15T14:14:33.719492+0000) 2024-05-15T14:14:44.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:44 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:44.510+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6820 osd.7 since back 2024-05-15T14:14:19.521247+0000 front 2024-05-15T14:14:19.521154+0000 (oldest deadline 2024-05-15T14:14:44.221009+0000) 2024-05-15T14:14:45.037 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:44 smithi052 podman[74307]: 2024-05-15 14:14:44.744333404 +0000 UTC m=+5.509903819 container died 1b4978797d636a3f66a8bfcfc3f73866a5f5576acc1bc37b4c7e38e563288e73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux ) 2024-05-15T14:14:45.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:45 smithi052 podman[74307]: 2024-05-15 14:14:45.108673896 +0000 UTC m=+5.874244302 container cleanup 1b4978797d636a3f66a8bfcfc3f73866a5f5576acc1bc37b4c7e38e563288e73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866) 2024-05-15T14:14:45.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:45 smithi052 bash[74307]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10 2024-05-15T14:14:45.707 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:45 smithi052 podman[74322]: 2024-05-15 14:14:45.368062022 +0000 UTC m=+0.614825087 container remove 1b4978797d636a3f66a8bfcfc3f73866a5f5576acc1bc37b4c7e38e563288e73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-15T14:14:45.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:45 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:45.461+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:45.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:45 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:45.461+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:45.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:45 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:45.461+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:45.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:45 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:45.461+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:55.816245+0000 front 2024-05-15T14:13:55.816144+0000 (oldest deadline 2024-05-15T14:14:21.016641+0000) 2024-05-15T14:14:45.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:45 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:45.461+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:04.318554+0000 front 2024-05-15T14:14:04.318616+0000 (oldest deadline 2024-05-15T14:14:27.818829+0000) 2024-05-15T14:14:45.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:45 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:45.461+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-15T14:14:07.819560+0000 front 2024-05-15T14:14:07.819647+0000 (oldest deadline 2024-05-15T14:14:33.719492+0000) 2024-05-15T14:14:45.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:45 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:45.461+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6820 osd.7 since back 2024-05-15T14:14:19.521247+0000 front 2024-05-15T14:14:19.521154+0000 (oldest deadline 2024-05-15T14:14:44.221009+0000) 2024-05-15T14:14:46.053 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:45 smithi052 podman[74413]: 2024-05-15 14:14:45.608823361 +0000 UTC m=+0.017652357 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-15T14:14:46.054 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:45 smithi052 podman[74413]: 2024-05-15 14:14:45.710462061 +0000 UTC m=+0.119291069 container create f922a36c376f80858e39a4f61e025b5805b0e29ab2f2a7434b7aebeba83a41c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD) 2024-05-15T14:14:46.054 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:45 smithi052 podman[74413]: 2024-05-15 14:14:45.920948597 +0000 UTC m=+0.329777587 container init f922a36c376f80858e39a4f61e025b5805b0e29ab2f2a7434b7aebeba83a41c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5) 2024-05-15T14:14:46.054 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:45 smithi052 podman[74413]: 2024-05-15 14:14:45.924032212 +0000 UTC m=+0.332861211 container start f922a36c376f80858e39a4f61e025b5805b0e29ab2f2a7434b7aebeba83a41c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T14:14:46.054 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:46 smithi052 podman[74413]: 2024-05-15 14:14:46.052158324 +0000 UTC m=+0.460987326 container attach f922a36c376f80858e39a4f61e025b5805b0e29ab2f2a7434b7aebeba83a41c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux ) 2024-05-15T14:14:46.341 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:46 smithi052 podman[74413]: 2024-05-15 14:14:46.22382002 +0000 UTC m=+0.632649010 container died f922a36c376f80858e39a4f61e025b5805b0e29ab2f2a7434b7aebeba83a41c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-15T14:14:46.759 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.10.service' 2024-05-15T14:14:46.782 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:46 smithi052 podman[74439]: 2024-05-15 14:14:46.733013746 +0000 UTC m=+0.499546764 container remove f922a36c376f80858e39a4f61e025b5805b0e29ab2f2a7434b7aebeba83a41c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-10-deactivate, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-15T14:14:46.782 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:46 smithi052 systemd[1]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.10.service: Deactivated successfully. 2024-05-15T14:14:46.782 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:46 smithi052 systemd[1]: Stopped Ceph osd.10 for bca78ca4-12c1-11ef-bc98-c7b262605968. 2024-05-15T14:14:46.782 INFO:journalctl@ceph.osd.10.smithi052.stdout:May 15 14:14:46 smithi052 systemd[1]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.10.service: Consumed 58.574s CPU time. 2024-05-15T14:14:46.798 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T14:14:46.798 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-15T14:14:46.798 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-15T14:14:46.799 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.11 2024-05-15T14:14:46.825 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:46 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:46.473+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:46.825 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:46 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:46.473+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:46.825 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:46 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:46.473+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:46.825 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:46 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:46.473+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:55.816245+0000 front 2024-05-15T14:13:55.816144+0000 (oldest deadline 2024-05-15T14:14:21.016641+0000) 2024-05-15T14:14:46.825 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:46 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:46.473+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:04.318554+0000 front 2024-05-15T14:14:04.318616+0000 (oldest deadline 2024-05-15T14:14:27.818829+0000) 2024-05-15T14:14:46.826 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:46 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:46.473+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-15T14:14:07.819560+0000 front 2024-05-15T14:14:07.819647+0000 (oldest deadline 2024-05-15T14:14:33.719492+0000) 2024-05-15T14:14:46.826 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:46 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:46.473+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6820 osd.7 since back 2024-05-15T14:14:19.521247+0000 front 2024-05-15T14:14:19.521154+0000 (oldest deadline 2024-05-15T14:14:44.221009+0000) 2024-05-15T14:14:47.101 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:46 smithi081 systemd[1]: Stopping Ceph osd.11 for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T14:14:47.102 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:47 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:47.098+0000 7ff46b026700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T14:14:47.357 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:47 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:47.098+0000 7ff46b026700 -1 osd.11 101 *** Got signal Terminated *** 2024-05-15T14:14:47.357 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:47 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:47.098+0000 7ff46b026700 -1 osd.11 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-15T14:14:47.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:47 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:47.511+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:47.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:47 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:47.511+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:47.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:47 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:47.511+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:47.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:47 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:47.511+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:55.816245+0000 front 2024-05-15T14:13:55.816144+0000 (oldest deadline 2024-05-15T14:14:21.016641+0000) 2024-05-15T14:14:47.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:47 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:47.511+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:04.318554+0000 front 2024-05-15T14:14:04.318616+0000 (oldest deadline 2024-05-15T14:14:27.818829+0000) 2024-05-15T14:14:47.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:47 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:47.511+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-15T14:14:07.819560+0000 front 2024-05-15T14:14:07.819647+0000 (oldest deadline 2024-05-15T14:14:33.719492+0000) 2024-05-15T14:14:47.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:47 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:47.511+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6820 osd.7 since back 2024-05-15T14:14:19.521247+0000 front 2024-05-15T14:14:19.521154+0000 (oldest deadline 2024-05-15T14:14:44.221009+0000) 2024-05-15T14:14:48.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:48 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:48.553+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:48.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:48 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:48.553+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:48.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:48 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:48.553+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:48.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:48 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:48.553+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:55.816245+0000 front 2024-05-15T14:13:55.816144+0000 (oldest deadline 2024-05-15T14:14:21.016641+0000) 2024-05-15T14:14:48.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:48 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:48.553+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:04.318554+0000 front 2024-05-15T14:14:04.318616+0000 (oldest deadline 2024-05-15T14:14:27.818829+0000) 2024-05-15T14:14:48.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:48 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:48.553+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-15T14:14:07.819560+0000 front 2024-05-15T14:14:07.819647+0000 (oldest deadline 2024-05-15T14:14:33.719492+0000) 2024-05-15T14:14:48.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:48 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:48.553+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6820 osd.7 since back 2024-05-15T14:14:19.521247+0000 front 2024-05-15T14:14:19.521154+0000 (oldest deadline 2024-05-15T14:14:44.221009+0000) 2024-05-15T14:14:49.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:49 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:49.542+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:49.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:49 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:49.542+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:49.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:49 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:49.542+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:49.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:49 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:49.542+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:55.816245+0000 front 2024-05-15T14:13:55.816144+0000 (oldest deadline 2024-05-15T14:14:21.016641+0000) 2024-05-15T14:14:49.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:49 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:49.542+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:04.318554+0000 front 2024-05-15T14:14:04.318616+0000 (oldest deadline 2024-05-15T14:14:27.818829+0000) 2024-05-15T14:14:49.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:49 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:49.542+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-15T14:14:07.819560+0000 front 2024-05-15T14:14:07.819647+0000 (oldest deadline 2024-05-15T14:14:33.719492+0000) 2024-05-15T14:14:49.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:49 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:49.542+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6820 osd.7 since back 2024-05-15T14:14:19.521247+0000 front 2024-05-15T14:14:19.521154+0000 (oldest deadline 2024-05-15T14:14:44.221009+0000) 2024-05-15T14:14:49.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:49 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:49.542+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6820 osd.8 since back 2024-05-15T14:14:24.221388+0000 front 2024-05-15T14:14:24.221409+0000 (oldest deadline 2024-05-15T14:14:49.521424+0000) 2024-05-15T14:14:50.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:50 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:50.527+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:50.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:50 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:50.527+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:50.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:50 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:50.527+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:50.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:50 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:50.527+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:55.816245+0000 front 2024-05-15T14:13:55.816144+0000 (oldest deadline 2024-05-15T14:14:21.016641+0000) 2024-05-15T14:14:50.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:50 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:50.527+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:04.318554+0000 front 2024-05-15T14:14:04.318616+0000 (oldest deadline 2024-05-15T14:14:27.818829+0000) 2024-05-15T14:14:50.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:50 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:50.527+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-15T14:14:07.819560+0000 front 2024-05-15T14:14:07.819647+0000 (oldest deadline 2024-05-15T14:14:33.719492+0000) 2024-05-15T14:14:50.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:50 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:50.527+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6820 osd.7 since back 2024-05-15T14:14:19.521247+0000 front 2024-05-15T14:14:19.521154+0000 (oldest deadline 2024-05-15T14:14:44.221009+0000) 2024-05-15T14:14:50.859 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:50 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:50.527+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6820 osd.8 since back 2024-05-15T14:14:24.221388+0000 front 2024-05-15T14:14:24.221409+0000 (oldest deadline 2024-05-15T14:14:49.521424+0000) 2024-05-15T14:14:51.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:51 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:51.566+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-15T14:13:20.810297+0000 front 2024-05-15T14:13:20.810319+0000 (oldest deadline 2024-05-15T14:13:45.410868+0000) 2024-05-15T14:14:51.857 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:51 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:51.566+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6804 osd.1 since back 2024-05-15T14:13:29.512339+0000 front 2024-05-15T14:13:29.512280+0000 (oldest deadline 2024-05-15T14:13:54.812122+0000) 2024-05-15T14:14:51.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:51 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:51.566+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6804 osd.2 since back 2024-05-15T14:13:39.512954+0000 front 2024-05-15T14:13:39.512984+0000 (oldest deadline 2024-05-15T14:14:01.213180+0000) 2024-05-15T14:14:51.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:51 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:51.566+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6812 osd.4 since back 2024-05-15T14:13:55.816245+0000 front 2024-05-15T14:13:55.816144+0000 (oldest deadline 2024-05-15T14:14:21.016641+0000) 2024-05-15T14:14:51.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:51 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:51.566+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6812 osd.5 since back 2024-05-15T14:14:04.318554+0000 front 2024-05-15T14:14:04.318616+0000 (oldest deadline 2024-05-15T14:14:27.818829+0000) 2024-05-15T14:14:51.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:51 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:51.566+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-15T14:14:07.819560+0000 front 2024-05-15T14:14:07.819647+0000 (oldest deadline 2024-05-15T14:14:33.719492+0000) 2024-05-15T14:14:51.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:51 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:51.566+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.52:6820 osd.7 since back 2024-05-15T14:14:19.521247+0000 front 2024-05-15T14:14:19.521154+0000 (oldest deadline 2024-05-15T14:14:44.221009+0000) 2024-05-15T14:14:51.858 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:51 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11[61609]: 2024-05-15T14:14:51.566+0000 7ff46761a700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.81:6820 osd.8 since back 2024-05-15T14:14:24.221388+0000 front 2024-05-15T14:14:24.221409+0000 (oldest deadline 2024-05-15T14:14:49.521424+0000) 2024-05-15T14:14:52.685 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:52 smithi081 podman[74910]: 2024-05-15 14:14:52.42366341 +0000 UTC m=+5.423120865 container died e9570a0c3fab52a7d459cc8ccb66ddb0ecf6c20725476aa24efbaf7fe7b5ce26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5) 2024-05-15T14:14:53.044 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:52 smithi081 podman[74910]: 2024-05-15 14:14:52.774237606 +0000 UTC m=+5.773695064 container cleanup e9570a0c3fab52a7d459cc8ccb66ddb0ecf6c20725476aa24efbaf7fe7b5ce26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-15T14:14:53.044 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:52 smithi081 bash[74910]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11 2024-05-15T14:14:53.357 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:53 smithi081 podman[74927]: 2024-05-15 14:14:53.042480017 +0000 UTC m=+0.621075578 container remove e9570a0c3fab52a7d459cc8ccb66ddb0ecf6c20725476aa24efbaf7fe7b5ce26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, ceph=True) 2024-05-15T14:14:53.688 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:53 smithi081 podman[75017]: 2024-05-15 14:14:53.38486516 +0000 UTC m=+0.111628810 container create ad7a0760f813b2c8006f807ef6960f6a36be9794e2259c5e6dc8ff9a3ab397ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T14:14:53.689 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:53 smithi081 podman[75017]: 2024-05-15 14:14:53.290154257 +0000 UTC m=+0.016917933 image pull c922e207469ef4b049fbf586bb5bae4126de6717fafa94a729705eed5414135a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d 2024-05-15T14:14:53.689 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:53 smithi081 podman[75017]: 2024-05-15 14:14:53.603786698 +0000 UTC m=+0.330550357 container init ad7a0760f813b2c8006f807ef6960f6a36be9794e2259c5e6dc8ff9a3ab397ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240514, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-15T14:14:53.689 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:53 smithi081 podman[75017]: 2024-05-15 14:14:53.606898391 +0000 UTC m=+0.333662047 container start ad7a0760f813b2c8006f807ef6960f6a36be9794e2259c5e6dc8ff9a3ab397ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11-deactivate, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-15T14:14:54.107 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:53 smithi081 podman[75017]: 2024-05-15 14:14:53.68711855 +0000 UTC m=+0.413882215 container attach ad7a0760f813b2c8006f807ef6960f6a36be9794e2259c5e6dc8ff9a3ab397ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-15T14:14:54.107 INFO:journalctl@ceph.osd.11.smithi081.stdout:May 15 14:14:53 smithi081 podman[75017]: 2024-05-15 14:14:53.899588686 +0000 UTC m=+0.626352347 container died ad7a0760f813b2c8006f807ef6960f6a36be9794e2259c5e6dc8ff9a3ab397ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T14:14:54.482 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@osd.11.service' 2024-05-15T14:14:54.559 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T14:14:54.560 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-15T14:14:54.560 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-15T14:14:54.560 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.a 2024-05-15T14:14:54.872 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 15 14:14:54 smithi033 systemd[1]: Stopping Ceph mds.a for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T14:14:55.191 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 15 14:14:54 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-a[71037]: 2024-05-15T14:14:54.870+0000 7ff7afa9d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T14:14:55.191 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 15 14:14:54 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-a[71037]: 2024-05-15T14:14:54.870+0000 7ff7afa9d700 -1 mds.a *** got signal Terminated *** 2024-05-15T14:14:57.248 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 15 14:14:56 smithi033 podman[104431]: 2024-05-15 14:14:56.994453844 +0000 UTC m=+2.224600168 container died 4e9df59fc51de448ad6e788147a7db999173849dcbacda3601e54e20a8352399 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-a, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-15T14:14:57.505 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 15 14:14:57 smithi033 podman[104431]: 2024-05-15 14:14:57.336568339 +0000 UTC m=+2.566714662 container cleanup 4e9df59fc51de448ad6e788147a7db999173849dcbacda3601e54e20a8352399 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-a, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-15T14:14:57.506 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 15 14:14:57 smithi033 bash[104431]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-a 2024-05-15T14:14:57.765 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 15 14:14:57 smithi033 podman[104444]: 2024-05-15 14:14:57.504325638 +0000 UTC m=+0.506123400 container remove 4e9df59fc51de448ad6e788147a7db999173849dcbacda3601e54e20a8352399 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5) 2024-05-15T14:14:57.770 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.a.service' 2024-05-15T14:14:57.806 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T14:14:57.806 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-15T14:14:57.807 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-15T14:14:57.807 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.d 2024-05-15T14:14:58.134 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 15 14:14:57 smithi033 systemd[1]: Stopping Ceph mds.d for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T14:14:58.441 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 15 14:14:58 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-d[69810]: 2024-05-15T14:14:58.132+0000 7fe844e07700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T14:14:58.441 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 15 14:14:58 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-d[69810]: 2024-05-15T14:14:58.132+0000 7fe844e07700 -1 mds.d *** got signal Terminated *** 2024-05-15T14:15:00.168 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 15 14:14:59 smithi033 podman[104554]: 2024-05-15 14:14:59.743864875 +0000 UTC m=+1.700919394 container died e29496fc34f2bec113367ed8eda431d81cd14da1ec7cd871dd3f492e592dab85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-d, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0) 2024-05-15T14:15:00.441 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 15 14:15:00 smithi033 podman[104554]: 2024-05-15 14:15:00.340706545 +0000 UTC m=+2.297761048 container cleanup e29496fc34f2bec113367ed8eda431d81cd14da1ec7cd871dd3f492e592dab85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, ceph=True) 2024-05-15T14:15:00.441 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 15 14:15:00 smithi033 bash[104554]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-d 2024-05-15T14:15:00.700 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.d.service' 2024-05-15T14:15:00.734 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T14:15:00.734 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-15T14:15:00.735 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-15T14:15:00.735 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.g 2024-05-15T14:15:01.191 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 15 14:15:00 smithi033 systemd[1]: Stopping Ceph mds.g for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T14:15:01.191 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 15 14:15:01 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-g[70615]: 2024-05-15T14:15:01.052+0000 7fe0ffced700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T14:15:01.191 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 15 14:15:01 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-g[70615]: 2024-05-15T14:15:01.052+0000 7fe0ffced700 -1 mds.g *** got signal Terminated *** 2024-05-15T14:15:05.330 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 15 14:15:05 smithi033 podman[104677]: 2024-05-15 14:15:05.066617125 +0000 UTC m=+4.107788992 container died a1133de4c9562db3746410cd464b2c93547d0afdc0d8419e2d7e3a0bb08dcac4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-g, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=) 2024-05-15T14:15:05.691 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 15 14:15:05 smithi033 podman[104677]: 2024-05-15 14:15:05.519123074 +0000 UTC m=+4.560294986 container cleanup a1133de4c9562db3746410cd464b2c93547d0afdc0d8419e2d7e3a0bb08dcac4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-g, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-15T14:15:05.691 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 15 14:15:05 smithi033 bash[104677]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-g 2024-05-15T14:15:05.966 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.g.service' 2024-05-15T14:15:06.001 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T14:15:06.001 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-15T14:15:06.001 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-15T14:15:06.001 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.j 2024-05-15T14:15:06.316 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 15 14:15:06 smithi033 systemd[1]: Stopping Ceph mds.j for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T14:15:06.691 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 15 14:15:06 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-j[69357]: 2024-05-15T14:15:06.314+0000 7f1cd2d29700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T14:15:06.691 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 15 14:15:06 smithi033 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-j[69357]: 2024-05-15T14:15:06.314+0000 7f1cd2d29700 -1 mds.j *** got signal Terminated *** 2024-05-15T14:15:07.917 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 15 14:15:07 smithi033 podman[104797]: 2024-05-15 14:15:07.648548596 +0000 UTC m=+1.428262822 container died 8e2339d6cdaa9d69d4d0071b9e6c11d2b7472b13d8e32c510a2b10c790d19230 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-j, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-15T14:15:08.191 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 15 14:15:08 smithi033 podman[104797]: 2024-05-15 14:15:08.039949215 +0000 UTC m=+1.819663439 container cleanup 8e2339d6cdaa9d69d4d0071b9e6c11d2b7472b13d8e32c510a2b10c790d19230 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-j, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-15T14:15:08.192 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 15 14:15:08 smithi033 bash[104797]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-j 2024-05-15T14:15:08.424 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.j.service' 2024-05-15T14:15:08.446 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 15 14:15:08 smithi033 podman[104810]: 2024-05-15 14:15:08.274381058 +0000 UTC m=+0.623226718 container remove 8e2339d6cdaa9d69d4d0071b9e6c11d2b7472b13d8e32c510a2b10c790d19230 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_CLEAN=True) 2024-05-15T14:15:08.446 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 15 14:15:08 smithi033 systemd[1]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.j.service: Deactivated successfully. 2024-05-15T14:15:08.446 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 15 14:15:08 smithi033 systemd[1]: Stopped Ceph mds.j for bca78ca4-12c1-11ef-bc98-c7b262605968. 2024-05-15T14:15:08.446 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 15 14:15:08 smithi033 systemd[1]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.j.service: Consumed 1.105s CPU time. 2024-05-15T14:15:08.457 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T14:15:08.457 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-15T14:15:08.457 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-15T14:15:08.457 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.b 2024-05-15T14:15:08.763 INFO:journalctl@ceph.mds.b.smithi052.stdout:May 15 14:15:08 smithi052 systemd[1]: Stopping Ceph mds.b for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T14:15:09.091 INFO:journalctl@ceph.mds.b.smithi052.stdout:May 15 14:15:08 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-b[66920]: 2024-05-15T14:15:08.758+0000 7fa3f0ff3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T14:15:09.091 INFO:journalctl@ceph.mds.b.smithi052.stdout:May 15 14:15:08 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-b[66920]: 2024-05-15T14:15:08.758+0000 7fa3f0ff3700 -1 mds.b *** got signal Terminated *** 2024-05-15T14:15:13.045 INFO:journalctl@ceph.mds.b.smithi052.stdout:May 15 14:15:12 smithi052 podman[74517]: 2024-05-15 14:15:12.763379547 +0000 UTC m=+4.098881057 container died da7651bf28afbd8f163b29ece52c05c4138c839ed06eff5155936c3aafd95232 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-b, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-15T14:15:13.341 INFO:journalctl@ceph.mds.b.smithi052.stdout:May 15 14:15:13 smithi052 podman[74517]: 2024-05-15 14:15:13.116796334 +0000 UTC m=+4.452297832 container cleanup da7651bf28afbd8f163b29ece52c05c4138c839ed06eff5155936c3aafd95232 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-15T14:15:13.341 INFO:journalctl@ceph.mds.b.smithi052.stdout:May 15 14:15:13 smithi052 bash[74517]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-b 2024-05-15T14:15:13.530 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.b.service' 2024-05-15T14:15:13.567 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T14:15:13.567 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-15T14:15:13.567 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-15T14:15:13.567 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.e 2024-05-15T14:15:13.898 INFO:journalctl@ceph.mds.e.smithi052.stdout:May 15 14:15:13 smithi052 systemd[1]: Stopping Ceph mds.e for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T14:15:14.341 INFO:journalctl@ceph.mds.e.smithi052.stdout:May 15 14:15:13 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-e[66295]: 2024-05-15T14:15:13.894+0000 7ff7fbe0b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T14:15:14.341 INFO:journalctl@ceph.mds.e.smithi052.stdout:May 15 14:15:13 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-e[66295]: 2024-05-15T14:15:13.894+0000 7ff7fbe0b700 -1 mds.e *** got signal Terminated *** 2024-05-15T14:15:15.215 INFO:journalctl@ceph.mds.e.smithi052.stdout:May 15 14:15:14 smithi052 podman[74651]: 2024-05-15 14:15:14.860674508 +0000 UTC m=+1.072817412 container died ce9ab971fa680cd130262bbcd5a70fc9311e3b941dd557cffddd05ae9ea2a3c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, ceph=True, org.label-schema.vendor=CentOS) 2024-05-15T14:15:15.473 INFO:journalctl@ceph.mds.e.smithi052.stdout:May 15 14:15:15 smithi052 podman[74651]: 2024-05-15 14:15:15.295981388 +0000 UTC m=+1.508124273 container cleanup ce9ab971fa680cd130262bbcd5a70fc9311e3b941dd557cffddd05ae9ea2a3c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, GIT_CLEAN=True, org.label-schema.build-date=20240514) 2024-05-15T14:15:15.474 INFO:journalctl@ceph.mds.e.smithi052.stdout:May 15 14:15:15 smithi052 bash[74651]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-e 2024-05-15T14:15:15.719 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.e.service' 2024-05-15T14:15:15.740 INFO:journalctl@ceph.mds.e.smithi052.stdout:May 15 14:15:15 smithi052 podman[74664]: 2024-05-15 14:15:15.472043021 +0000 UTC m=+0.611067843 container remove ce9ab971fa680cd130262bbcd5a70fc9311e3b941dd557cffddd05ae9ea2a3c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-e, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-15T14:15:15.740 INFO:journalctl@ceph.mds.e.smithi052.stdout:May 15 14:15:15 smithi052 systemd[1]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.e.service: Deactivated successfully. 2024-05-15T14:15:15.741 INFO:journalctl@ceph.mds.e.smithi052.stdout:May 15 14:15:15 smithi052 systemd[1]: Stopped Ceph mds.e for bca78ca4-12c1-11ef-bc98-c7b262605968. 2024-05-15T14:15:15.741 INFO:journalctl@ceph.mds.e.smithi052.stdout:May 15 14:15:15 smithi052 systemd[1]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.e.service: Consumed 1.131s CPU time. 2024-05-15T14:15:15.752 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T14:15:15.752 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-15T14:15:15.752 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-15T14:15:15.753 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.h 2024-05-15T14:15:16.076 INFO:journalctl@ceph.mds.h.smithi052.stdout:May 15 14:15:15 smithi052 systemd[1]: Stopping Ceph mds.h for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T14:15:16.341 INFO:journalctl@ceph.mds.h.smithi052.stdout:May 15 14:15:16 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-h[66613]: 2024-05-15T14:15:16.073+0000 7f2ed71ce700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T14:15:16.341 INFO:journalctl@ceph.mds.h.smithi052.stdout:May 15 14:15:16 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-h[66613]: 2024-05-15T14:15:16.073+0000 7f2ed71ce700 -1 mds.h *** got signal Terminated *** 2024-05-15T14:15:20.341 INFO:journalctl@ceph.mds.h.smithi052.stdout:May 15 14:15:20 smithi052 podman[74772]: 2024-05-15 14:15:20.085016057 +0000 UTC m=+4.112363425 container died ac645799e591769b04b962c73adcf5b72e1ae2b57f67f859888cd19a2106eec9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-h, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-15T14:15:20.618 INFO:journalctl@ceph.mds.h.smithi052.stdout:May 15 14:15:20 smithi052 podman[74772]: 2024-05-15 14:15:20.424224817 +0000 UTC m=+4.451572176 container cleanup ac645799e591769b04b962c73adcf5b72e1ae2b57f67f859888cd19a2106eec9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-15T14:15:20.619 INFO:journalctl@ceph.mds.h.smithi052.stdout:May 15 14:15:20 smithi052 bash[74772]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-h 2024-05-15T14:15:20.853 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.h.service' 2024-05-15T14:15:20.874 INFO:journalctl@ceph.mds.h.smithi052.stdout:May 15 14:15:20 smithi052 podman[74784]: 2024-05-15 14:15:20.617013543 +0000 UTC m=+0.528970440 container remove ac645799e591769b04b962c73adcf5b72e1ae2b57f67f859888cd19a2106eec9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-h, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0) 2024-05-15T14:15:20.874 INFO:journalctl@ceph.mds.h.smithi052.stdout:May 15 14:15:20 smithi052 systemd[1]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.h.service: Deactivated successfully. 2024-05-15T14:15:20.874 INFO:journalctl@ceph.mds.h.smithi052.stdout:May 15 14:15:20 smithi052 systemd[1]: Stopped Ceph mds.h for bca78ca4-12c1-11ef-bc98-c7b262605968. 2024-05-15T14:15:20.874 INFO:journalctl@ceph.mds.h.smithi052.stdout:May 15 14:15:20 smithi052 systemd[1]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.h.service: Consumed 6.379s CPU time. 2024-05-15T14:15:20.886 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T14:15:20.886 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-15T14:15:20.886 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-15T14:15:20.886 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.k 2024-05-15T14:15:21.204 INFO:journalctl@ceph.mds.k.smithi052.stdout:May 15 14:15:20 smithi052 systemd[1]: Stopping Ceph mds.k for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T14:15:21.591 INFO:journalctl@ceph.mds.k.smithi052.stdout:May 15 14:15:21 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-k[65980]: 2024-05-15T14:15:21.201+0000 7fd573a89700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T14:15:21.591 INFO:journalctl@ceph.mds.k.smithi052.stdout:May 15 14:15:21 smithi052 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-k[65980]: 2024-05-15T14:15:21.201+0000 7fd573a89700 -1 mds.k *** got signal Terminated *** 2024-05-15T14:15:24.239 INFO:journalctl@ceph.mds.k.smithi052.stdout:May 15 14:15:23 smithi052 podman[74890]: 2024-05-15 14:15:23.964743005 +0000 UTC m=+2.860342629 container died 3504755753aa99e33ad7df8432e4f481baf9298461041907b3b8443530c2b7a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240514, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T14:15:24.591 INFO:journalctl@ceph.mds.k.smithi052.stdout:May 15 14:15:24 smithi052 podman[74890]: 2024-05-15 14:15:24.344730179 +0000 UTC m=+3.240329802 container cleanup 3504755753aa99e33ad7df8432e4f481baf9298461041907b3b8443530c2b7a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-k, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240514, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-9b63866, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-15T14:15:24.591 INFO:journalctl@ceph.mds.k.smithi052.stdout:May 15 14:15:24 smithi052 bash[74890]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-k 2024-05-15T14:15:24.731 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.k.service' 2024-05-15T14:15:24.762 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T14:15:24.762 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-15T14:15:24.763 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-15T14:15:24.763 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.c 2024-05-15T14:15:25.051 INFO:journalctl@ceph.mds.c.smithi081.stdout:May 15 14:15:24 smithi081 systemd[1]: Stopping Ceph mds.c for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T14:15:25.322 INFO:journalctl@ceph.mds.c.smithi081.stdout:May 15 14:15:25 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-c[66566]: 2024-05-15T14:15:25.048+0000 7fe471de6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T14:15:25.322 INFO:journalctl@ceph.mds.c.smithi081.stdout:May 15 14:15:25 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-c[66566]: 2024-05-15T14:15:25.048+0000 7fe471de6700 -1 mds.c *** got signal Terminated *** 2024-05-15T14:15:29.052 INFO:journalctl@ceph.mds.c.smithi081.stdout:May 15 14:15:28 smithi081 podman[75122]: 2024-05-15 14:15:28.768378395 +0000 UTC m=+3.816517939 container died b722e0a0be650978392c956dc5573d0028a832f4ef87cdd769fe3f9d0445691f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.schema-version=1.0) 2024-05-15T14:15:29.357 INFO:journalctl@ceph.mds.c.smithi081.stdout:May 15 14:15:29 smithi081 podman[75122]: 2024-05-15 14:15:29.149193977 +0000 UTC m=+4.197333521 container cleanup b722e0a0be650978392c956dc5573d0028a832f4ef87cdd769fe3f9d0445691f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-15T14:15:29.358 INFO:journalctl@ceph.mds.c.smithi081.stdout:May 15 14:15:29 smithi081 bash[75122]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-c 2024-05-15T14:15:29.581 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.c.service' 2024-05-15T14:15:29.616 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T14:15:29.616 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-15T14:15:29.616 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-15T14:15:29.617 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.f 2024-05-15T14:15:29.947 INFO:journalctl@ceph.mds.f.smithi081.stdout:May 15 14:15:29 smithi081 systemd[1]: Stopping Ceph mds.f for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T14:15:30.357 INFO:journalctl@ceph.mds.f.smithi081.stdout:May 15 14:15:29 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-f[66264]: 2024-05-15T14:15:29.944+0000 7fa843f7b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T14:15:30.357 INFO:journalctl@ceph.mds.f.smithi081.stdout:May 15 14:15:29 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-f[66264]: 2024-05-15T14:15:29.944+0000 7fa843f7b700 -1 mds.f *** got signal Terminated *** 2024-05-15T14:15:33.055 INFO:journalctl@ceph.mds.f.smithi081.stdout:May 15 14:15:32 smithi081 podman[75254]: 2024-05-15 14:15:32.771259507 +0000 UTC m=+2.938218657 container died 031e85e5354540e45372fc76d2b9bf24c1416b65b3736c92c271e50a76077d09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-f, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-15T14:15:33.357 INFO:journalctl@ceph.mds.f.smithi081.stdout:May 15 14:15:33 smithi081 podman[75254]: 2024-05-15 14:15:33.153445942 +0000 UTC m=+3.320405084 container cleanup 031e85e5354540e45372fc76d2b9bf24c1416b65b3736c92c271e50a76077d09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240514) 2024-05-15T14:15:33.357 INFO:journalctl@ceph.mds.f.smithi081.stdout:May 15 14:15:33 smithi081 bash[75254]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-f 2024-05-15T14:15:33.538 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.f.service' 2024-05-15T14:15:33.570 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T14:15:33.570 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-15T14:15:33.570 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-15T14:15:33.570 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.i 2024-05-15T14:15:33.857 INFO:journalctl@ceph.mds.i.smithi081.stdout:May 15 14:15:33 smithi081 systemd[1]: Stopping Ceph mds.i for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T14:15:34.357 INFO:journalctl@ceph.mds.i.smithi081.stdout:May 15 14:15:33 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-i[66884]: 2024-05-15T14:15:33.881+0000 7ff3d9fc2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T14:15:34.358 INFO:journalctl@ceph.mds.i.smithi081.stdout:May 15 14:15:33 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-i[66884]: 2024-05-15T14:15:33.881+0000 7ff3d9fc2700 -1 mds.i *** got signal Terminated *** 2024-05-15T14:15:37.091 INFO:journalctl@ceph.mds.i.smithi081.stdout:May 15 14:15:36 smithi081 podman[75374]: 2024-05-15 14:15:36.762359668 +0000 UTC m=+2.982388342 container died 63576537a4ee5c900faad68ff9a5eeb06d039f0bb760e4f00e92b9f3938407fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-i, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, RELEASE=wip-lusov-quiesce-9b63866, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240514, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-15T14:15:37.091 INFO:journalctl@ceph.mds.i.smithi081.stdout:May 15 14:15:37 smithi081 podman[75374]: 2024-05-15 14:15:37.09014052 +0000 UTC m=+3.310169193 container cleanup 63576537a4ee5c900faad68ff9a5eeb06d039f0bb760e4f00e92b9f3938407fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-i, org.label-schema.build-date=20240514, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-9b63866, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.vendor=CentOS) 2024-05-15T14:15:37.357 INFO:journalctl@ceph.mds.i.smithi081.stdout:May 15 14:15:37 smithi081 bash[75374]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-i 2024-05-15T14:15:37.357 INFO:journalctl@ceph.mds.i.smithi081.stdout:May 15 14:15:37 smithi081 podman[75387]: 2024-05-15 14:15:37.324908506 +0000 UTC m=+0.560894390 container remove 63576537a4ee5c900faad68ff9a5eeb06d039f0bb760e4f00e92b9f3938407fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.build-date=20240514, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-15T14:15:37.469 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.i.service' 2024-05-15T14:15:37.500 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T14:15:37.501 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-15T14:15:37.501 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-15T14:15:37.501 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.l 2024-05-15T14:15:37.837 INFO:journalctl@ceph.mds.l.smithi081.stdout:May 15 14:15:37 smithi081 systemd[1]: Stopping Ceph mds.l for bca78ca4-12c1-11ef-bc98-c7b262605968... 2024-05-15T14:15:38.107 INFO:journalctl@ceph.mds.l.smithi081.stdout:May 15 14:15:37 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-l[65961]: 2024-05-15T14:15:37.834+0000 7f9af2f16700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-15T14:15:38.107 INFO:journalctl@ceph.mds.l.smithi081.stdout:May 15 14:15:37 smithi081 ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-l[65961]: 2024-05-15T14:15:37.835+0000 7f9af2f16700 -1 mds.l *** got signal Terminated *** 2024-05-15T14:15:41.046 INFO:journalctl@ceph.mds.l.smithi081.stdout:May 15 14:15:40 smithi081 podman[75495]: 2024-05-15 14:15:40.765573211 +0000 UTC m=+3.038613468 container died e49dda6b9f585795831a079acbc35831e8ea82d5f76ecced73941de7a46317d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-l, org.label-schema.build-date=20240514, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, org.label-schema.schema-version=1.0) 2024-05-15T14:15:41.357 INFO:journalctl@ceph.mds.l.smithi081.stdout:May 15 14:15:41 smithi081 podman[75495]: 2024-05-15 14:15:41.119457952 +0000 UTC m=+3.392498203 container cleanup e49dda6b9f585795831a079acbc35831e8ea82d5f76ecced73941de7a46317d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4cbcf646574c408ba120e1dd31f4c06da74af0d3a69bf38032025b1391d7897d, name=ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-l, GIT_BRANCH=HEAD, org.label-schema.build-date=20240514, GIT_COMMIT=d4a0f93b06679a5c9148c9e9b367478b2a0d04e0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-9b63866, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-15T14:15:41.358 INFO:journalctl@ceph.mds.l.smithi081.stdout:May 15 14:15:41 smithi081 bash[75495]: ceph-bca78ca4-12c1-11ef-bc98-c7b262605968-mds-l 2024-05-15T14:15:41.536 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bca78ca4-12c1-11ef-bc98-c7b262605968@mds.l.service' 2024-05-15T14:15:41.566 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-15T14:15:41.566 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-15T14:15:41.567 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-15T14:15:41.771 INFO:teuthology.orchestra.run.smithi033.stdout:Deleting cluster with fsid: bca78ca4-12c1-11ef-bc98-c7b262605968 2024-05-15T14:15:44.340 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-15T14:15:44.539 INFO:teuthology.orchestra.run.smithi052.stdout:Deleting cluster with fsid: bca78ca4-12c1-11ef-bc98-c7b262605968 2024-05-15T14:15:47.133 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-15T14:15:47.322 INFO:teuthology.orchestra.run.smithi081.stdout:Deleting cluster with fsid: bca78ca4-12c1-11ef-bc98-c7b262605968 2024-05-15T14:15:49.635 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-15T14:15:49.670 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-15T14:15:49.703 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-15T14:15:49.728 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-15T14:15:49.730 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-05-15_11:41:37-fs-wip-lusov-quiesce-distro-default-smithi/7707053/remote/smithi033/crash 2024-05-15T14:15:49.731 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/crash -- . 2024-05-15T14:15:49.774 INFO:teuthology.orchestra.run.smithi033.stderr:tar: /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-15T14:15:49.774 INFO:teuthology.orchestra.run.smithi033.stderr:tar: Error is not recoverable: exiting now 2024-05-15T14:15:49.776 DEBUG:teuthology.misc:Transferring archived files from smithi052:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-05-15_11:41:37-fs-wip-lusov-quiesce-distro-default-smithi/7707053/remote/smithi052/crash 2024-05-15T14:15:49.777 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar c -f - -C /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/crash -- . 2024-05-15T14:15:49.817 INFO:teuthology.orchestra.run.smithi052.stderr:tar: /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-15T14:15:49.817 INFO:teuthology.orchestra.run.smithi052.stderr:tar: Error is not recoverable: exiting now 2024-05-15T14:15:49.819 DEBUG:teuthology.misc:Transferring archived files from smithi081:/var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-05-15_11:41:37-fs-wip-lusov-quiesce-distro-default-smithi/7707053/remote/smithi081/crash 2024-05-15T14:15:49.820 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/crash -- . 2024-05-15T14:15:49.856 INFO:teuthology.orchestra.run.smithi081.stderr:tar: /var/lib/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-15T14:15:49.856 INFO:teuthology.orchestra.run.smithi081.stderr:tar: Error is not recoverable: exiting now 2024-05-15T14:15:49.857 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-15T14:15:49.857 DEBUG:teuthology.orchestra.run.smithi033:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-05-15T14:15:49.892 INFO:tasks.cephadm:Compressing logs... 2024-05-15T14:15:49.893 DEBUG:teuthology.orchestra.run.smithi033:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-15T14:15:49.935 DEBUG:teuthology.orchestra.run.smithi052:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-15T14:15:49.938 DEBUG:teuthology.orchestra.run.smithi081:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-15T14:15:49.961 INFO:teuthology.orchestra.run.smithi033.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph.log 2024-05-15T14:15:49.961 INFO:teuthology.orchestra.run.smithi033.stderr:: No such file or directory 2024-05-15T14:15:49.961 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-15T14:15:49.962 INFO:teuthology.orchestra.run.smithi081.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph.log 2024-05-15T14:15:49.963 INFO:teuthology.orchestra.run.smithi081.stderr:: No such file or directory 2024-05-15T14:15:49.963 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.2.log 2024-05-15T14:15:49.963 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.f.log 2024-05-15T14:15:49.963 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-15T14:15:49.963 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.5.log 2024-05-15T14:15:49.963 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.c.log 2024-05-15T14:15:49.975 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph.audit.log: /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mon.c.log 2024-05-15T14:15:49.975 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mgr.z.log 2024-05-15T14:15:49.976 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mon.c.log: /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mgr.z.log: 87.2% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-15T14:15:49.976 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.i.log 2024-05-15T14:15:49.976 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-15T14:15:49.976 INFO:teuthology.orchestra.run.smithi081.stderr: 90.5% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-15T14:15:49.976 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.11.log 2024-05-15T14:15:49.976 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph.cephadm.log: 83.7% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-15T14:15:49.976 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.8.log 2024-05-15T14:15:49.976 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.11.log: 92.1% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mgr.z.log.gz 2024-05-15T14:15:49.977 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.g.log 2024-05-15T14:15:49.977 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.j.log 2024-05-15T14:15:49.978 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.3.log 2024-05-15T14:15:49.978 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.0.log 2024-05-15T14:15:49.978 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.j.log: /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mgr.x.log 2024-05-15T14:15:49.978 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mon.a.log 2024-05-15T14:15:49.978 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.6.log 2024-05-15T14:15:49.978 INFO:teuthology.orchestra.run.smithi033.stderr: 87.2% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-15T14:15:49.978 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-15T14:15:49.979 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mon.a.log: /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.6.log: 90.3% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-15T14:15:49.979 INFO:teuthology.orchestra.run.smithi033.stderr: 94.6% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.j.log.gz 2024-05-15T14:15:49.979 INFO:teuthology.orchestra.run.smithi033.stderr: 94.6% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.g.log.gz 2024-05-15T14:15:49.981 INFO:teuthology.orchestra.run.smithi052.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph.log 2024-05-15T14:15:49.981 INFO:teuthology.orchestra.run.smithi052.stderr:: No such file or directory 2024-05-15T14:15:49.981 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-15T14:15:49.981 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.7.log 2024-05-15T14:15:49.982 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph.log: /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.b.log 2024-05-15T14:15:49.982 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.4.log 2024-05-15T14:15:49.982 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.1.log 2024-05-15T14:15:49.982 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.b.log: /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mgr.y.log 2024-05-15T14:15:49.982 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.h.log 2024-05-15T14:15:49.982 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.10.log 2024-05-15T14:15:49.982 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.h.log: 87.2% 90.5% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-15T14:15:49.983 INFO:teuthology.orchestra.run.smithi052.stderr: -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-15T14:15:49.983 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-15T14:15:49.983 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.e.log 2024-05-15T14:15:49.983 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph.cephadm.log: 83.8% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-15T14:15:49.983 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-15T14:15:49.983 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.e.log: 94.5% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.e.log.gz 2024-05-15T14:15:49.983 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mon.b.log 2024-05-15T14:15:49.983 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-volume.log: 92.0% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mgr.y.log.gz 2024-05-15T14:15:49.985 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-15T14:15:49.986 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.a.log 2024-05-15T14:15:49.986 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-15T14:15:49.987 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.a.log: 84.1% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-15T14:15:49.987 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.d.log 2024-05-15T14:15:49.987 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-volume.log: 94.7% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.a.log.gz 2024-05-15T14:15:49.989 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.k.log 2024-05-15T14:15:49.990 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mon.b.log: 91.1% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-15T14:15:49.991 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-15T14:15:49.992 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.9.log 2024-05-15T14:15:49.992 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-15T14:15:49.993 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.9.log: 91.1% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-15T14:15:49.995 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.l.log 2024-05-15T14:15:49.997 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.k.log: 94.5% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.k.log.gz 2024-05-15T14:15:49.998 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/cephadm.log: 94.6% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.d.log.gz 2024-05-15T14:15:49.999 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-15T14:15:50.002 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-15T14:15:50.004 INFO:teuthology.orchestra.run.smithi033.stderr: 94.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-15T14:15:50.014 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-15T14:15:50.019 INFO:teuthology.orchestra.run.smithi081.stderr: 91.0% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-15T14:15:50.371 INFO:teuthology.orchestra.run.smithi033.stderr: 90.0% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mgr.x.log.gz 2024-05-15T14:15:50.460 INFO:teuthology.orchestra.run.smithi081.stderr: 91.7% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mon.c.log.gz 2024-05-15T14:15:50.819 INFO:teuthology.orchestra.run.smithi052.stderr: 92.6% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mon.b.log.gz 2024-05-15T14:15:51.316 INFO:teuthology.orchestra.run.smithi033.stderr: 90.3% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mon.a.log.gz 2024-05-15T14:15:51.644 INFO:teuthology.orchestra.run.smithi052.stderr: 95.4% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.h.log.gz 2024-05-15T14:15:53.131 INFO:teuthology.orchestra.run.smithi081.stderr: 95.7% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.l.log.gz 2024-05-15T14:15:53.200 INFO:teuthology.orchestra.run.smithi033.stderr: 94.0% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.9.log.gz 2024-05-15T14:15:53.436 INFO:teuthology.orchestra.run.smithi052.stderr: 94.0% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.4.log.gz 2024-05-15T14:15:53.815 INFO:teuthology.orchestra.run.smithi033.stderr: 93.8% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.0.log.gz 2024-05-15T14:15:54.001 INFO:teuthology.orchestra.run.smithi081.stderr: 95.6% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.f.log.gz 2024-05-15T14:15:54.070 INFO:teuthology.orchestra.run.smithi052.stderr: 94.0% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.1.log.gz 2024-05-15T14:15:54.932 INFO:teuthology.orchestra.run.smithi033.stderr: 94.1% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.6.log.gz 2024-05-15T14:15:54.983 INFO:teuthology.orchestra.run.smithi033.stderr: 93.9% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.3.log.gz 2024-05-15T14:15:54.983 INFO:teuthology.orchestra.run.smithi033.stderr: 2024-05-15T14:15:54.983 INFO:teuthology.orchestra.run.smithi033.stderr:real 0m5.032s 2024-05-15T14:15:54.983 INFO:teuthology.orchestra.run.smithi033.stderr:user 0m18.211s 2024-05-15T14:15:54.983 INFO:teuthology.orchestra.run.smithi033.stderr:sys 0m0.641s 2024-05-15T14:15:55.082 INFO:teuthology.orchestra.run.smithi052.stderr: 94.2% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.10.log.gz 2024-05-15T14:15:55.452 INFO:teuthology.orchestra.run.smithi081.stderr: 94.2% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.11.log.gz 2024-05-15T14:15:56.191 INFO:teuthology.orchestra.run.smithi081.stderr: 94.2% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.2.log.gz 2024-05-15T14:15:56.654 INFO:teuthology.orchestra.run.smithi081.stderr: 94.3% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.8.log.gz 2024-05-15T14:15:57.164 INFO:teuthology.orchestra.run.smithi052.stderr: 94.2% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.7.log.gz 2024-05-15T14:15:57.652 INFO:teuthology.orchestra.run.smithi081.stderr: 94.3% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-osd.5.log.gz 2024-05-15T14:16:17.801 INFO:teuthology.orchestra.run.smithi081.stderr: 94.0% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.c.log.gz 2024-05-15T14:16:32.814 INFO:teuthology.orchestra.run.smithi052.stderr: 93.8% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.b.log.gz 2024-05-15T14:16:32.817 INFO:teuthology.orchestra.run.smithi052.stderr: 2024-05-15T14:16:32.817 INFO:teuthology.orchestra.run.smithi052.stderr:real 0m42.865s 2024-05-15T14:16:32.817 INFO:teuthology.orchestra.run.smithi052.stderr:user 1m3.240s 2024-05-15T14:16:32.817 INFO:teuthology.orchestra.run.smithi052.stderr:sys 0m1.960s 2024-05-15T14:17:01.936 INFO:teuthology.orchestra.run.smithi081.stderr: 93.7% -- replaced with /var/log/ceph/bca78ca4-12c1-11ef-bc98-c7b262605968/ceph-mds.i.log.gz 2024-05-15T14:17:01.938 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-15T14:17:01.938 INFO:teuthology.orchestra.run.smithi081.stderr:real 1m11.985s 2024-05-15T14:17:01.938 INFO:teuthology.orchestra.run.smithi081.stderr:user 2m8.825s 2024-05-15T14:17:01.938 INFO:teuthology.orchestra.run.smithi081.stderr:sys 0m4.049s 2024-05-15T14:17:01.939 INFO:tasks.cephadm:Archiving logs... 2024-05-15T14:17:01.939 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-05-15_11:41:37-fs-wip-lusov-quiesce-distro-default-smithi/7707053/remote/smithi033/log 2024-05-15T14:17:01.940 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-15T14:17:04.533 DEBUG:teuthology.misc:Transferring archived files from smithi052:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-05-15_11:41:37-fs-wip-lusov-quiesce-distro-default-smithi/7707053/remote/smithi052/log 2024-05-15T14:17:04.534 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-15T14:17:13.025 DEBUG:teuthology.misc:Transferring archived files from smithi081:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-05-15_11:41:37-fs-wip-lusov-quiesce-distro-default-smithi/7707053/remote/smithi081/log 2024-05-15T14:17:13.027 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-15T14:17:31.364 INFO:tasks.cephadm:Removing cluster... 2024-05-15T14:17:31.365 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 --force 2024-05-15T14:17:31.568 INFO:teuthology.orchestra.run.smithi033.stdout:Deleting cluster with fsid: bca78ca4-12c1-11ef-bc98-c7b262605968 2024-05-15T14:17:31.859 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 --force 2024-05-15T14:17:32.061 INFO:teuthology.orchestra.run.smithi052.stdout:Deleting cluster with fsid: bca78ca4-12c1-11ef-bc98-c7b262605968 2024-05-15T14:17:32.389 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bca78ca4-12c1-11ef-bc98-c7b262605968 --force 2024-05-15T14:17:32.582 INFO:teuthology.orchestra.run.smithi081.stdout:Deleting cluster with fsid: bca78ca4-12c1-11ef-bc98-c7b262605968 2024-05-15T14:17:33.034 INFO:tasks.cephadm:Removing cephadm ... 2024-05-15T14:17:33.034 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-15T14:17:33.055 DEBUG:teuthology.orchestra.run.smithi052:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-15T14:17:33.074 DEBUG:teuthology.orchestra.run.smithi081:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-15T14:17:33.091 INFO:tasks.cephadm:Teardown complete 2024-05-15T14:17:33.091 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-15T14:17:33.103 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-15T14:17:33.103 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-15T14:17:33.106 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-15T14:17:33.117 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-15T14:17:33.231 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-15T14:17:33.242 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-15T14:17:33.242 DEBUG:teuthology.orchestra.run.smithi033:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-15T14:17:33.245 DEBUG:teuthology.orchestra.run.smithi052:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-15T14:17:33.249 DEBUG:teuthology.orchestra.run.smithi081:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-15T14:17:33.260 INFO:teuthology.orchestra.run.smithi033.stderr:bash: line 1: ntpq: command not found 2024-05-15T14:17:33.262 INFO:teuthology.orchestra.run.smithi052.stderr:bash: line 1: ntpq: command not found 2024-05-15T14:17:33.263 INFO:teuthology.orchestra.run.smithi081.stderr:bash: line 1: ntpq: command not found 2024-05-15T14:17:33.266 INFO:teuthology.orchestra.run.smithi033.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-15T14:17:33.266 INFO:teuthology.orchestra.run.smithi033.stdout:=============================================================================== 2024-05-15T14:17:33.266 INFO:teuthology.orchestra.run.smithi033.stdout:^* hv01.front.sepia.ceph.com 3 6 377 5 +765us[ +770us] +/- 49ms 2024-05-15T14:17:33.266 INFO:teuthology.orchestra.run.smithi033.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 7 -855us[ -850us] +/- 53ms 2024-05-15T14:17:33.267 INFO:teuthology.orchestra.run.smithi033.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 6 -52us[ -47us] +/- 91ms 2024-05-15T14:17:33.267 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-15T14:17:33.269 INFO:teuthology.orchestra.run.smithi052.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-15T14:17:33.269 INFO:teuthology.orchestra.run.smithi052.stdout:=============================================================================== 2024-05-15T14:17:33.270 INFO:teuthology.orchestra.run.smithi052.stdout:^* hv01.front.sepia.ceph.com 3 7 377 70 +770us[ +778us] +/- 48ms 2024-05-15T14:17:33.270 INFO:teuthology.orchestra.run.smithi052.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 8 -867us[ -867us] +/- 53ms 2024-05-15T14:17:33.270 INFO:teuthology.orchestra.run.smithi052.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 -64us[ -64us] +/- 91ms 2024-05-15T14:17:33.270 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-15T14:17:33.270 INFO:teuthology.orchestra.run.smithi081.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-15T14:17:33.270 INFO:teuthology.orchestra.run.smithi081.stdout:=============================================================================== 2024-05-15T14:17:33.270 INFO:teuthology.orchestra.run.smithi081.stdout:^* hv01.front.sepia.ceph.com 3 6 377 4 +773us[ +754us] +/- 49ms 2024-05-15T14:17:33.271 INFO:teuthology.orchestra.run.smithi081.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 3 -840us[ -840us] +/- 53ms 2024-05-15T14:17:33.271 INFO:teuthology.orchestra.run.smithi081.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 4 -37us[ -56us] +/- 91ms 2024-05-15T14:17:33.271 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-15T14:17:33.271 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-15T14:17:33.282 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-15T14:17:33.283 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-15T14:17:33.294 DEBUG:teuthology.orchestra.run.smithi033:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-15T14:17:33.399 DEBUG:teuthology.orchestra.run.smithi052:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-15T14:17:33.473 DEBUG:teuthology.orchestra.run.smithi081:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-15T14:17:33.538 DEBUG:teuthology.orchestra.run.smithi033:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-15T14:17:33.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-15T14:17:33.569 DEBUG:teuthology.orchestra.run.smithi052:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-15T14:17:33.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-15T14:17:33.599 DEBUG:teuthology.orchestra.run.smithi081:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-15T14:17:33.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-15T14:17:33.627 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-15T14:17:33.638 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-15T14:17:33.649 INFO:teuthology.task.internal:Duration was 2172.244899 seconds 2024-05-15T14:17:33.650 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-15T14:17:33.664 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-15T14:17:33.664 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-15T14:17:33.666 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-15T14:17:33.669 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-15T14:17:33.706 INFO:teuthology.orchestra.run.smithi081.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-15T14:17:33.706 INFO:teuthology.orchestra.run.smithi033.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-15T14:17:33.707 INFO:teuthology.orchestra.run.smithi052.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-15T14:17:34.232 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-15T14:17:34.232 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi033.front.sepia.ceph.com 2024-05-15T14:17:34.233 DEBUG:teuthology.orchestra.run.smithi033:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-15T14:17:34.295 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi052.front.sepia.ceph.com 2024-05-15T14:17:34.295 DEBUG:teuthology.orchestra.run.smithi052:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-15T14:17:34.326 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi081.front.sepia.ceph.com 2024-05-15T14:17:34.326 DEBUG:teuthology.orchestra.run.smithi081:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-15T14:17:34.363 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-15T14:17:34.363 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-15T14:17:34.366 DEBUG:teuthology.orchestra.run.smithi052:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-15T14:17:34.369 DEBUG:teuthology.orchestra.run.smithi081:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-15T14:17:34.429 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-15T14:17:34.429 DEBUG:teuthology.orchestra.run.smithi033:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-15T14:17:34.434 DEBUG:teuthology.orchestra.run.smithi052:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-15T14:17:34.439 DEBUG:teuthology.orchestra.run.smithi081:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-15T14:17:35.156 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-15T14:17:35.168 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-15T14:17:35.169 DEBUG:teuthology.orchestra.run.smithi033:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-15T14:17:35.195 DEBUG:teuthology.orchestra.run.smithi052:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-15T14:17:35.225 DEBUG:teuthology.orchestra.run.smithi081:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-15T14:17:35.250 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-15T14:17:35.264 DEBUG:teuthology.orchestra.run.smithi033:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-15T14:17:35.267 DEBUG:teuthology.orchestra.run.smithi052:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-15T14:17:35.270 DEBUG:teuthology.orchestra.run.smithi081:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-15T14:17:35.289 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = core 2024-05-15T14:17:35.295 INFO:teuthology.orchestra.run.smithi052.stdout:kernel.core_pattern = core 2024-05-15T14:17:35.312 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern = core 2024-05-15T14:17:35.341 DEBUG:teuthology.orchestra.run.smithi033:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-15T14:17:35.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-15T14:17:35.358 DEBUG:teuthology.orchestra.run.smithi052:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-15T14:17:35.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-15T14:17:35.375 DEBUG:teuthology.orchestra.run.smithi081:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-15T14:17:35.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-15T14:17:35.396 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-15T14:17:35.408 INFO:teuthology.task.internal:Transferring archived files... 2024-05-15T14:17:35.409 DEBUG:teuthology.misc:Transferring archived files from smithi033:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-05-15_11:41:37-fs-wip-lusov-quiesce-distro-default-smithi/7707053/remote/smithi033 2024-05-15T14:17:35.410 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-15T14:17:35.461 DEBUG:teuthology.misc:Transferring archived files from smithi052:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-05-15_11:41:37-fs-wip-lusov-quiesce-distro-default-smithi/7707053/remote/smithi052 2024-05-15T14:17:35.462 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-15T14:17:35.500 DEBUG:teuthology.misc:Transferring archived files from smithi081:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-05-15_11:41:37-fs-wip-lusov-quiesce-distro-default-smithi/7707053/remote/smithi081 2024-05-15T14:17:35.500 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-15T14:17:35.537 INFO:teuthology.task.internal:Removing archive directory... 2024-05-15T14:17:35.538 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-15T14:17:35.540 DEBUG:teuthology.orchestra.run.smithi052:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-15T14:17:35.543 DEBUG:teuthology.orchestra.run.smithi081:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-15T14:17:35.585 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-15T14:17:35.599 INFO:teuthology.task.internal:Not uploading archives. 2024-05-15T14:17:35.599 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-15T14:17:35.612 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-15T14:17:35.613 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-15T14:17:35.616 DEBUG:teuthology.orchestra.run.smithi052:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-15T14:17:35.618 DEBUG:teuthology.orchestra.run.smithi081:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-15T14:17:35.629 INFO:teuthology.orchestra.run.smithi033.stdout: 265307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 15 14:17 /home/ubuntu/cephtest 2024-05-15T14:17:35.632 INFO:teuthology.orchestra.run.smithi052.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 15 14:17 /home/ubuntu/cephtest 2024-05-15T14:17:35.639 INFO:teuthology.orchestra.run.smithi081.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 15 14:17 /home/ubuntu/cephtest 2024-05-15T14:17:35.640 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-15T14:17:35.653 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-15T14:17:35.679 INFO:teuthology.nuke:Checking targets against current locks 2024-05-15T14:17:35.707 DEBUG:teuthology.nuke:shortname: smithi033 2024-05-15T14:17:35.707 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-15T14:17:35.729 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-15_11:41:37-fs-wip-lusov-quiesce-distro-default-smithi/7707053', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-15 13:33:06.182275', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDpGRjSdfgSUankN5XPJOkJiIA0VxcoD0zz+dYpyFodcyJ9HESornrG6dSqjjRMOFcL0ol70eJjW9yHQNOvhLew='} 2024-05-15T14:17:35.735 DEBUG:teuthology.nuke:shortname: smithi052 2024-05-15T14:17:35.735 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-15T14:17:35.758 DEBUG:teuthology.nuke:shortname: smithi081 2024-05-15T14:17:35.759 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-15T14:17:35.763 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi052.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-15_11:41:37-fs-wip-lusov-quiesce-distro-default-smithi/7707053', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-15 13:33:06.179570', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIC9ClE4WGx0zqnxVqBs7s8RIJeNuykeOAvQLxVRq8oBQLOslIX9ebXR0a58y5Xt8Y2THLDUEP3/bpeR3LwTAcU='} 2024-05-15T14:17:35.793 INFO:teuthology.orchestra.console.smithi033:Power off 2024-05-15T14:17:35.794 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-15T14:17:35.805 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi081.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-15_11:41:37-fs-wip-lusov-quiesce-distro-default-smithi/7707053', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-15 13:33:06.180855', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJpX0yPXiwAKdwnzHU3PD0i6JTvy/2hx17oCvLzOKWBgZgrGd3iPLtP/out24nBWNzZ+5/GtvB+6xlYXyWU0qKo='} 2024-05-15T14:17:35.817 DEBUG:teuthology.orchestra.console.smithi033:power off output: Chassis Power Control: Down/Off 2024-05-15T14:17:35.817 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-15T14:17:35.840 INFO:teuthology.orchestra.console.smithi052:Power off 2024-05-15T14:17:35.840 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-15T14:17:35.855 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on 2024-05-15T14:17:35.863 DEBUG:teuthology.orchestra.console.smithi052:power off output: Chassis Power Control: Down/Off 2024-05-15T14:17:35.864 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-15T14:17:35.883 INFO:teuthology.orchestra.console.smithi081:Power off 2024-05-15T14:17:35.884 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-15T14:17:35.893 DEBUG:teuthology.orchestra.console.smithi052:check power output: Chassis Power is on 2024-05-15T14:17:35.906 DEBUG:teuthology.orchestra.console.smithi081:power off output: Chassis Power Control: Down/Off 2024-05-15T14:17:35.907 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-15T14:17:35.934 DEBUG:teuthology.orchestra.console.smithi081:check power output: Chassis Power is on 2024-05-15T14:17:39.856 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-15T14:17:39.894 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-15T14:17:39.935 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-15T14:17:39.968 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on 2024-05-15T14:17:40.005 DEBUG:teuthology.orchestra.console.smithi052:check power output: Chassis Power is on 2024-05-15T14:17:40.045 DEBUG:teuthology.orchestra.console.smithi081:check power output: Chassis Power is on 2024-05-15T14:17:43.970 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-15T14:17:44.006 DEBUG:teuthology.orchestra.console.smithi052:pexpect command: ipmitool -H smithi052.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-15T14:17:44.046 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-15T14:17:44.080 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on 2024-05-15T14:17:44.116 DEBUG:teuthology.orchestra.console.smithi052:check power output: Chassis Power is off 2024-05-15T14:17:44.156 DEBUG:teuthology.orchestra.console.smithi081:check power output: Chassis Power is off 2024-05-15T14:17:44.218 INFO:teuthology.orchestra.console.smithi052:Power off completed 2024-05-15T14:17:44.257 INFO:teuthology.orchestra.console.smithi081:Power off completed 2024-05-15T14:17:48.082 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-15T14:17:48.194 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is off 2024-05-15T14:17:48.295 INFO:teuthology.orchestra.console.smithi033:Power off completed 2024-05-15T14:17:48.402 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} duration: 2172.2448992729187 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi033 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9b63866860f0b71216b2c0a73aedd2e0a8a5c98b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_leonidus@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=54515f164fe745a8a0a3d4b11c46fa42 status: fail success: false 2024-05-15T14:17:48.402 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-15T14:17:48.493 INFO:teuthology.run:FAIL